I'm currently developing a plugin, where I need to process files uploaded to Moodle on my own servers (the users ofc know this happens). However, what would the best approach be for this? Is there a way to upload files from Moodle to AWS S3? Or should I send the content of the files over REST? The later approach could be rather inefficient though, as the files could get rather big.
Gustav Grimberg への返信
Re: Is there a way to transfer files from Moodle to e.g. AWS S3?
- Mark Johnson の投稿
Mark Johnson への返信
Re: Is there a way to transfer files from Moodle to e.g. AWS S3?
- Justin Hunt の投稿
ObjectFS is good for magically putting your data dir on S3. But if the real goal is to get them to the cloud, where you can process them (transcribe them, convert them, plagiarism check them etc) , then you should generate presigned upload URLs for S3, and give those to the moodle user. Then the file will not even hit Moodle. It will go straight to your S3 bucket. If you also need to store that file alongside the Moodle user, then you will need to store the URL to it somewhere in Moodle. Or use the Moodle files api to fetch it from S3 and save it in the Moodle data directory.
Justin Hunt への返信
Re: Is there a way to transfer files from Moodle to e.g. AWS S3?
- Gustav Grimberg の投稿
This is interesting! How exactly would I give the presigned URL to the user? My ideal workflow would be something like:
- User selects files and upload them
- Files are then transferred to S3, and URLs are returned
- I send these URLs to my other service
- My other service will process them
Do you have ideas/examples as to how the above should/could be implemented? Thanks!
Gustav Grimberg への返信
Re: Is there a way to transfer files from Moodle to e.g. AWS S3?
- Justin Hunt の投稿
I am not sure if you need to use Moodle in Step 1 or not. Assuming you don't need to, the workflow is quite simple.
You still have to get elbow deep in AWS though.
You would use the AWS SDK to generate presigned S3 upload URLs. You would not need these if you first collected the files in Moodle, because you could use the SDK APIs to just post the file to S3. The presigned URL allows you to send it to a users browser, giving them access to your S3 bucket, just for the upload. So you could init a JS drag/drop file box on screen with the presigned URL. When a file is dropped you upload it to the URL.
You can get the SDK from AWS, but Catalyst also packages one up as a Moodle plugin, which is handy.
https://moodle.org/plugins/local_aws
Once the file arrives in S3, AWS generates events/SNS messages that you can respond to do the rest . Commonly you would use an AWS lambda script to listen to events/SNS messages, and then do any routing of information to the next destination or service.
I suppose you could skip that and keep it simple, though its not what I would do. Since when you generate the presigned URL, you actually know the final destination (bucket and file path/name). You could send those details to the other service, which could start polling for the presence of the upload file.
Well thats a few ideas
You still have to get elbow deep in AWS though.
You would use the AWS SDK to generate presigned S3 upload URLs. You would not need these if you first collected the files in Moodle, because you could use the SDK APIs to just post the file to S3. The presigned URL allows you to send it to a users browser, giving them access to your S3 bucket, just for the upload. So you could init a JS drag/drop file box on screen with the presigned URL. When a file is dropped you upload it to the URL.
You can get the SDK from AWS, but Catalyst also packages one up as a Moodle plugin, which is handy.
https://moodle.org/plugins/local_aws
Once the file arrives in S3, AWS generates events/SNS messages that you can respond to do the rest . Commonly you would use an AWS lambda script to listen to events/SNS messages, and then do any routing of information to the next destination or service.
I suppose you could skip that and keep it simple, though its not what I would do. Since when you generate the presigned URL, you actually know the final destination (bucket and file path/name). You could send those details to the other service, which could start polling for the presence of the upload file.
Well thats a few ideas
Justin Hunt への返信
Re: Is there a way to transfer files from Moodle to e.g. AWS S3?
- Gustav Grimberg の投稿
Gustav Grimberg への返信
Re: Is there a way to transfer files from Moodle to e.g. AWS S3?
- Shree Ram Sharma の投稿
Please try this video. This one is a step by steps tutorials to handle this requirements.
https://www.youtube.com/embed/SlHN4Kk5wOg
https://www.youtube.com/embed/SlHN4Kk5wOg