finally, some progress on uploading multiple files

finally, some progress on uploading multiple files

by Brian Jones -
Number of replies: 13
Well, I don't claim that this is "done". Far from it. But the attached tarball represents a few weeks of sifting through the code, and trying my best to implement a new assignment type in moodle 1.5.2 "the right way".

The new assignment type is called "uploadmulti". There are a few downsides right now that relate to some specific needs I had. For example, when the assignment is first set up, the prof *must* define *all* required files, and I haven't yet written code to let the profs randomly add or delete files from the assignment. That's coming. I'll also soon add code to have a prof list a given file as "optional". My specific need was to enable a prof to name specific required files, and have the system check uploaded files against the list of required files. Students *can* upload multiple versions of the same file.

The only changes that I made to code outside of my code directory (/mod/assignment/type/uploadmulti/) were to /mod/assignment/view.php, and to /mod/assignment/db/mysql.sql.

The view.php code is an addition, not a change, and it's well documented. It's also probably not the best way to do what was needed, so I ask for help there.

The mysql.sql change was made to add a column to the assignment table, and to add the assignment_reqfiles table. I haven't edited mysql.php, so doing an upgrade from pre 1.5.2 won't make the changes.

It is urged that you only untar this file on a new, test installation of moodle. *DO NOT* install this in production. I do want people to check out the code and give me feedback, but the functionality that's here doesn't warrant risking your production site wink Also, note that this also means you *SHOULD NOT* point the test install at your production database! It should all be pretty safe and very non-intrusive, but better to be safe than sorry.

When you untar the file, it'll create a directory called assignment-uploadmulti. It's really meant to be untar'ed in the /mod/ directory. At that point, move the /mod/assignment directory to /mod/assignment-prod, then link assignment-uploadmulti to assignment, and you're on your way.

I hope this is helpful.
Average of ratings: -
In reply to Brian Jones

Re: finally, some progress on uploading multiple files

by Brian Jones -
Sorry - you'll also need these entries in /lang/en/assignment.php

$string['helpuploadmulti'] = '<p>This type of assignment allows each participant to upload multiple files of arbitrary type.</p>';
$string['numreqfiles'] = 'Number of required files';
$string['typeuploadmulti'] = 'Upload multiple files';

Average of ratings: -
In reply to Brian Jones

Re: finally, some progress on uploading multiple files

by Jan Dierckx -

Brian,

Nice you posted your solution. I tried to find a fix for the addition in view.php but couldn't. Sorry!

I do have another suggestion, just something I learned from experience when I tried adding an extra column to the glossary module:

in the long run adding an extra database table may not be the best choice. Have you considered storing the references to the uploaded files inside the data1 and data2 fields which are already part of the assignment_submissions table? There is plenty of room there and I can see some advantages:

  • other people don't have to add a column to the assignment table, and add a whole new assignment_reqfiles table

  • you can reuse the backup/restore code that comes with the assignment plugin. Currently, information stored in the assignment_reqfiles table is not being backed up.

  • it may be easier to upgrade the module in the future if it doesn't rely on having one column more than the standard assignment module.

Average of ratings: -
In reply to Jan Dierckx

Re: finally, some progress on uploading multiple files

by Brian Jones -
Hi,

Thanks for the feedback!

I actually did find a solution to get my code out of the main view.php, but I want to clean up one or two other things before I post a new tarball. As of this moment, I'm running the module on my development workstation in such a way that I can distribute the /mod/assignment/type/uploadmulti directory alone as the tarball. Wahoo!

As for the database stuff, this was something that I did as preparation for future features, really, because I'm not nearly done with the module. The assignment_reqfiles table is a lookup table to get details on the required files for a given assignment. For example, still unused (but not for long) is the "isoptional" field, so that a prof can mark a given required file as optional. Eventually, it may become necessary to have that table hold some date values as well, so that, say, an english teacher can require a first draft by one date, but the final by another, and still grade the whole thing as one assignment. Or a bio prof can require various data files at different dates, but grade everything together when the final analysis is handed in.

I also wanted the students to be able to see the comments that go with any of the required files. Right now, the comments get to the database, but they don't get presented (it's secondary, but it's definitely coming).

Once you start seeing the different data elements that can be associated with the required files themselves, they start to stand out more as objects unto themselves, just like submissions, and assignments, each of which also have their own tables smile  I hope that's clear. If I've missed something or you have a good suggestion to further shrink my impact on the existing system, do certainly let me know.

I'll think about it s'more myself before I start coding things for the backup system, etc.

Thanks again!
brian.




Average of ratings: -
In reply to Brian Jones

Re: finally, some progress on uploading multiple files

by Brian Jones -
OK, here's the tarball of just the /mod/assignment/type/uploadmulti directory. I've also attached the code I use ON A CLEAN INSTALL to update the database. If you apply this on a production system, YOU WILL blow away your data.

With these two files and the earlier "lang" lines I posted in my second post of this thread, you should have a working uploadmulti system. Some features are missing, but hopefully the ones that *are* implemented are not broken smile


Average of ratings: -
In reply to Brian Jones

Re: finally, some progress on uploading multiple files

by Brian Jones -
*sigh* the moodle forums have trouble with uploading multiple files as well, it seems ;-P

Here's the database blow-away-and-replace-with-uploadmultified-stuff file.
Average of ratings: -
In reply to Jan Dierckx

Re: finally, some progress on uploading multiple files

by Brian Jones -
By the way, a question about the assignment database table:

Do we know for sure that the var* columns are unused? Do we know that the data* columns in assignment_submissions are also unused? What's the point in putting them there if they're unused? Just to make upgrades easier or something? I don't really see the point, but I'd like to know if it's safe to rename and use these columns, because I want to add more features.


Average of ratings: -
In reply to Brian Jones

Re: finally, some progress on uploading multiple files

by Jan Dierckx -

There is indeed no point in putting data1 and data2 fields in there if every other assignment plugins adds it's own columns and tables. wink

They were added to allow people to write other types of assignments (hence I guess the generic names)

I looked up some stuff: var1, data1 and data2 fields are used by the online assignment type. They store the text and the textformat of the submissions written by a student. This doesn't prevent you from reusing them again in another way in your own assignment type. Every assignment type can redefine the way data1, data2, var1, var2, etc... are used. That's what makes the assignment module flexible and easy to expand: while the online assignment type uses data1 to store the text of the submission, a (yet uncomplete) team assignment type may use the data1 field to store a list of students working on the same task and your multi upload assignment type may use data1 to store a list of files that were uploaded. (Though I guess there may be circumstances in which an extra database table may be better)

You are free to use and reuse these fields in your own assignment type, but renaming them is unnecessary and seems like a bad idea. It will probably break the backup / restore code of the assignment module and will render the other assignment types useless.

Hope this helps...

Average of ratings: -
In reply to Jan Dierckx

Re: finally, some progress on uploading multiple files

by Brian Jones -
Thanks for this. After thinking about what you just said, I understand now how this all works, and can alter my code to use what's there, as you say. Thanks so much for the clarification. smile


Average of ratings: -
In reply to Jan Dierckx

Re: finally, some progress on uploading multiple files

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
And, another suggestion. I've created some complex assignment types by using serialize/unserialize with the data1 and data2 fields. That lets me store a lot of information in one field. Its not very good if you need to query on it, so I would recommend only using this method for data that isn't going to be queried on.

mike
Average of ratings: -
In reply to Mike Churchward

Re: finally, some progress on uploading multiple files

by Brian Jones -
Thanks for this suggestion. Luckily, I only have to store a couple of very small bits of data. As a former dba, I'd absolutely gag if I had to store more than one atomic unit of data in a particular tuple. I know people do it all the time, but I just can't do it, and I've been lucky enough to be able to find a better design solution whenever the temptation arose.

Thanks a bunch for this info. I'm working on a couple of small glitches in the code right now (like adding the ability to specify only a single required file), so if there's an interest in having this code, let me know, and when I get it to some form of "almost 1.0" quality, I'll post the tarball.

If you must have the code NOW, email me.
Average of ratings: -
In reply to Brian Jones

Re: finally, some progress on uploading multiple files

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Correct - I put the var and data fields in there exactly so that plugins that only needed a few data fields could use them.   The advantage is that Moodle backup/restore will work correctly even for third-party plugins.

Plugins that are really complex should create and maintain their own tables via mod/asssignment/type/assignmentplugin/db/mysql.* and not serialise data in the main tables (unless they really want to wink)

Looking forward to seeing your plugin completed!

Will it also just let students upload an arbitrary set of files if the teacher allows it?
Average of ratings: -
In reply to Martin Dougiamas

Re: finally, some progress on uploading multiple files

by Brian Jones -
Hi Martin,

Thanks for the input. My assignment type only creates one small table to keep track of the required files for the assignment. I haven't yet implemented the code that creates that table so that it "just happens" when the module is first invoked. That's last on the priority list for me right now, but I should be getting to it in the next week, because I'm almost ready to unleash a first cut of the module (maybe Monday).

The module will allow for the uploading of arbitrary files, and a professor *can* set the number of required files to "0" or even "1" for this assignment type, even though the initial purpose for this module was to handle multiple file uploads. Setting it to "1" gives an advantage over the "upload single" type in that the prof can specify that a particular file with a particular name is required. Unfortunately, I haven't implemented any code to have the required file name be dynamic. That might go to the heart of moodle to implement variables like that, so a prof can say that the required file should be called "file-%U%", where %U% evaluates to the username of the logged in user. That might be useful elsewhere as well.

The module also allows for a prof to use the "Files" interface to create a "/bin" directory under the assignment directory, where they can put a test script called "submit.scr", and if that exists *and* the logged in user has submitted all of the required files for the assignment, then a "test" button will appear, which will run the script against the user's submitted files and give back the output. This is useful in Comp Sci, where users will upload code and we want to let them test that it'll compile on the upload server without giving them a shell, and in the various sciences where tabular data can be analyzed at the press of the button (well, someone still has to write the submit.scr script, but you get the idea).

For that to be usable, though, I did have to override the method that creates the assignment directory so it would use the assignment name instead of the id, so the professor would recognize it.

There are lots of other features, and I integrated things into the "update this assignment" directory, and I didn't touch anything outside my assignment type's directory, so hopefully people can test and use this pretty widely, starting next week.

brian.
Average of ratings: -
In reply to Brian Jones

Re: finally, some progress on uploading multiple files

by Carlos Chiarella -

I am having a problem and I hope somebody can give me some help. I am new in the moodle environment. I downloaded the new version of the Assignment module. I have moodle version 1.5.2. If the assignment type is Online text everything works fine and the we can grade the assignment. But if the assignment type is Upload a single file when we try to download the assignment (word document) we are getting these message: The page cannot be found.

I think because the system is not searching in the correct place. I am in the submissions file. When I put the mouse over the icon of the document at the botton of the page I am getting this:

http://localhost/moodle/file.php/4/moddata/assignment/4/4/Moodle_Add-ins.doc

I think the system should be searching in:

moodledata/4/moddata/assignment/4/4/Moodle_Add-ins.doc

Thanks,

Carlos

Average of ratings: -