I connected Google Drive to as repo to my moodle installation. I think whenever I select a file from Google Drive it's being downloaded to my Moodle server and serve it from there. Can we serve files directly from Google Drive without downloading to the Moodle server, so that I can save space in my server.
Moodle 3.9? Info for that version below ...
Check default config of Google Drive repo and setting of the file resource in the courses.
To verify, you could query your DB for moodle - mdl_files table - and look for file by name. Those actually stored in Google Drive will show a URL to Google Drive.