Building WebDAV support for DMS

Building WebDAV support for DMS

by Matt Schwartz -
Number of replies: 20

Hi all,

I am currently making the DMS file store accessible via WebDAV. I have run into a major stumbling block in that WebDAV deals with files and directories in the literal sense. It navigates a hierarchical file structure.

DMS in contrast navigates via a few URLs that are passed the ID for the file or directory.  This access scheme will not work with WebDAV. As far as I can tell, there is no way to make WebDAV request the document using the ID, without having that ID be visible to the user as part of the file name. So here is what I propose.

1) I will (already am) rework DMS to maintain the user directory structure in the filesystem, not in the database. That is, I propose creating a literal directory hierarchy to store all users' files. This can be flexible, the main requirement being that the hierarchy can easily and unamibiguously map from the requested file name to the actual one. Basically, the file lookup key will change from the id to the path and filename as per usual.

2) Document and folder metadata and permissions will still be stored in the database as before.

The above scheme is also much more scalable than the existing storage scheme where all files are stored in a single directory. Also, this will result in fewer database hits, depending on how I implement this.

Anyway, looking for feedback from knowledgeable WebDAVers and moodlers. Thanks,

 Matt

Average of ratings: -
In reply to Matt Schwartz

Re: Building WebDAV support for DMS

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

Hi Matt,

WebDav would be a really valuable feature to add - our lecturers would be very happy being able to use it!

For me, a big factor in using webdav is how access is controlled - eg how does teacher1 get webdav access to the "course1" area of the dms and not the "course2" area - or will the webdav access only be available to the admin of a site? - I don't see how Moodle can directly influence permissions required for WebDav access. Can the "system" you are developing do this? - or do WebDav permissions have to be set seperately from Moodle? - in this case, how would a site using the manual users/passwords stored in the moodle db access webdav to manage their files?

just a few thoughts......

smile

Dan

Average of ratings: -
In reply to Dan Marsden

Re: Building WebDAV support for DMS

by Matt Schwartz -

Hi Dan,

I am using the user authentication built into moodle. That is, I plan on having the webdav server live at a URL like myhost/moodle/dav/.

This will place the root directory starting at the /moodle/dav/ url. The server will be aliased to live at the above URL and parse the path info to serve files. So a request to myhost/moodle/dav/course1/myfile will serve up /course1/myfile in the dav filesystem. Before the file is served up, the dav server will use the permissions stored in the database by moodle to make sure the authenticated user is allowed to see the file. It will only list files that are allowed to be accessed in directory listings and will only serve those as well.

Just a start but that's the idea. Let me know how that works for you.

Matt

Average of ratings: -
In reply to Matt Schwartz

Re: Building WebDAV support for DMS

by Bob Boufford -

 Matt,

It's great to see you are implementing WebDAV for Moodle. It's one of the best features ever implemented in WebCT Campus Edition and WebCT Vista but sorely missing from Blackboard. (I teach in all three along with a Moodle course).

How you describe the the dav path is the same as WebCT CE with http://<webct_server>/dav/<course_id>/ . Once in "Network Neighborhood", the WebDAV folder is just like any other network folder. The only caveat we need to keep telling faculty that once a WebDAV folder is created, one does not have to be logged into the CMS to access the files in the WebDAV folder which means "Don't create WebDAV folders in a public lab and let Windows remember the password!".

If WebDAV for Moodle will also work with Dreamweaver MX, it will be a really great addition.  (Unfortunately, Dreamweave doesn't work with SSL-enabled WebDAV connections).

Cheers,

Bob

Average of ratings: -
In reply to Bob Boufford

Re: Building WebDAV support for DMS

by Matt Schwartz -

Hi Bob,

Yes, that's the idea. The users will have a dir structure available to them like that. I am implementing both course-centric folders and user-centric folders so users can have their own storage space, which moodle does not have at this time. I am open to any suggestions as I have little experience with WebDAV and little practical experience with CMSes in general. Take care!

  Matt

Average of ratings: -
In reply to Matt Schwartz

Re: Building WebDAV support for DMS

by Bob Boufford -
Matt,

I haven't done development in quite some time (anyone remember Pascal?), more in CMS administration, instructor training and my own teaching.

Are you familiar with WebDAV.org (http://www.webdav.org)? If not, there are like additional resources to help in implementation. It was there some years ago, a colleague posted instructions on how to implement WebDAV on a WebCT system before WebDAV was officially supported by WebCT.

Some things to consider in terms of control and authentication based on my current experience with WebCT's WebDAV implementation.

1. The system administrator or Moodle administrator should have control over enabling WebDAV for the entire Moodle system. That is a current setting in WebCT Administration. Some adminstrators are still very reluctant to allow WebDAV for security reasons even though I have not seen many security problems over the years if one teaches faculty that the same precautions should be taken for WebDAV as with any other secure access.

2. WebDAV access is controlled by role. In WebCT, only instructors/designers have the ability to use WebDAV, students do no have any WebDAV access even in Vista where everyone has a personal "MyFiles" file space. From an administrator standpoint, not allowing students to use WebDAV is one less support nightmare. However, there were times as an instructor that I would have liked giving students the ability to use WebDAV for various activities that involved a collaborative shared file space in WebCT CE.

3. Instructors can control WebDAV access to students within a course.  For reason given above related to student activities and collaborative projects.

Hope this helps...

Cheers,

Bob




Average of ratings: -
In reply to Bob Boufford

Re: Building WebDAV support for DMS

by Matt Schwartz -

Thanks Bob! I'll keep these requirements in mind. They are in line with what I was thinking. Which is nice. Take care!

Matt

Average of ratings: -
In reply to Bob Boufford

Re: Building WebDAV support for DMS

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Matt (and Bob)

After having used WebCT for the past 3 years our institution has decided to go for Moodle next year. Among the facilities I have come to find quite indispensable in the WebCT environment, WebDAV support ranks quite high. I can't be bothered to zip files to post them to my "course space" on a Moode server and unzip them there, as I used to do in pre-WebDAV days! Not only do I want to be able to access my Moodle workspace in a transparent way "à la Windows Explorer" but I also want to be able to manage my Moodle space directly from Dreamweaver, the way I am doing now in WebCT.

I have no idea what is required in order to make WebDAV support a standard fixture of Moodle but I will be grateful to those who will make it possible.

Bob, what do you mean when you say "Dreamweaver doesn't work with SSL-enabled WebDAV connections" ? I take it that if at the moment I can manage my distant WebCT files through Dreamweaver it means I am accessing our university WebCT/WebDAV server with a NON SSL'enabled connection?

All the best,

Joseph_R

Average of ratings: -
In reply to Joseph Rézeau

Re: Building WebDAV support for DMS

by Bob Boufford -

Hi Joseph,

We have found one cannot set up a WebDAV connection with Macromedia Dreamweaver to any SSL-enabled (httpssmile WebCT CE/Vista server. I understand this is supposed to be a Macromedia Dreamweaver issue. But I have never had a need or opportunity to test on any other SSL-enabled web site so I can't verify if this is strictly a Macromedia issue.

But WebDAV with Macromedia Dreamweaver to a standard (httpsmile WebCT installation is a feature that will be hard to give up.

Matt, I don't know the ins and outs of SSL and your DMS but if there is a way to selectively disable an SSL-enabled Moodle site for WebDAV purposes, that would be very useful.

Cheers,

Bob

Average of ratings: -
In reply to Bob Boufford

Re: Building WebDAV support for DMS

by Tony Hursh -
Macromedia really, really, REALLY needs to fix this. They've known about the problem for close to a year now.

Some of our students use Dreamweaver (that's what we recommend to them, since we use it ourselves and can therefore give them good support). Unfortunately UIUC's new student web server uses SSL-encrypted WebDAV, so they're forced to copy their files by hand. sad


Average of ratings: -
In reply to Tony Hursh

Re: Building WebDAV support for DMS

by Kingsley Kerce -
Hi Tony -- What do you mean by "copy their files by hand"?

Average of ratings: -
In reply to Kingsley Kerce

Re: Building WebDAV support for DMS

by Tony Hursh -
They have to publish to a local folder, then manually drag the files into the WebDAV folder. That requires that they either keep track of which files have changed or recopy the whole site onto the server any time something changes, neither of which is particularly satisfactory. smile

There's also a web interface to the student web space, but that's tedious to use if you have more than a few files (and, again, requires that you either keep track of which files have changed or upload the entire site again).

Average of ratings: -
In reply to Tony Hursh

Re: Building WebDAV support for DMS

by Kingsley Kerce -
O.k., just wanted to make sure they at least knew they could drag into a WebDAV folder.  A possible workaround might be to use one of those "drive letter mapper" utilities like Novell's iFolder (Google it; I believe it's free), which you would think would abstract away the issue with Dreamweaver.

Average of ratings: -
In reply to Matt Schwartz

Re: Building WebDAV support for DMS

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

Hi Matt,

how will your system perform the authentication? - will it provide a pop-up box requesting the information? - what happens if the authentication in moodle is set to something like ldap? - will it use Ldap to authenticate? 

Average of ratings: -
In reply to Dan Marsden

Re: Building WebDAV support for DMS

by Matt Schwartz -

Well, I am somewhat new to WebDAV, but on the front end it will use basic-auth or digest or whatever other web standard authentication WebDAV clients are allowed to use.

On the back end it is using the moodle APIs to authenticate the received credentials against moodle, so if moodle is set up to use LDAP I am assuming that the APIs are encapsulating that implementation detail.

Does that answer it for you? Take care,

  Matt

Average of ratings: -
In reply to Matt Schwartz

Re: Building WebDAV support for DMS

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

Hi Matt,

I wasn't aware that you could interrupt the basic-auth or digest credentials and pass them through your own api's - if you know of any docs/tutorials that show how this can be done, can you post them here?

thanks,

Dan

Average of ratings: -
In reply to Dan Marsden

Re: Building WebDAV support for DMS

by Matt Schwartz -

Hi Dan,

Well, as far as I know, these credentials are passed as http headers so you can intercept them. I am using the PEAR webdav library, which has an entry point where you can catch the credentials and check for authorization.

In the past, I have easily gotten the username and password from the basic-auth method in server side java code. I am assuming it is similar when using digest.

Hope this is what you were asking about. Take care,

  Matt

Average of ratings: -
In reply to Matt Schwartz

Re: Building WebDAV support for DMS

by Kingsley Kerce -
Hi Matt -- Thanks again for tackling a worthwhile and challenging project.  I just wanted to mention that the issues you describe in the start of this thread are addressed in the design and implementation of projects such as Jakarta Slide and Catacomb.  You probably think I'm repeating myself too much (http://moodle.org/mod/forum/discuss.php?d=8711#50205), but wheel reinvention has always bothered me.smile

In particular, note that the above projects use a relational database filestore, yet maintain the hierarchical URL requirements of WebDAV.

BTW, the author of Apache's mod_dav now works on the Subversion project, which is yet another rich source of design and implementation ideas for the integration of WebDAV into a software system.

I wish I had more time to devote to Moodle/WebDAV.  I'll do my best to answer any questions about what I've said above.

Kings

Average of ratings: -
In reply to Kingsley Kerce

Re: Building WebDAV support for DMS

by Matt Schwartz -

Hi Kingsley,

Thanks for your advice. I will always take help from those who know about this stuff. However, I did look at Slide and, unless I am misunderstanding, its implementation algorithm doesn't solve the problem that I addressed.

 The fundamental problem is that DMS does file and folder lookup by ID whereas WebDAV (and Slide's WebDAV adapter API) does that lookup by URI. This is a fundamental primary key mismatch. To solve this I need to rework the DMS file storage code to be able to look up the file by URI instead of ID. The underlying file storage doesn't need to map 1:1 to the URI structure but the database names of the files and resources will have to do this as far as I can see. There needs to be a way to look up a resource by unique URI. This is not currently possible so that is what I am implementing now. This is a fundamentally different file access algorithm than the DMS database schema implements now.

  Anyway, that is my understanding of the situation for now. Always open to helpful comments and assistance. Thanks Kingsley,

  Matt

Average of ratings: -
In reply to Matt Schwartz

Re: Building WebDAV support for DMS

by W Page -
Hi Matt!

What is the present status of your script?

WP1
Average of ratings: -