Forum Extras - adding 'read' tracking by user.

Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Number of replies: 73
Picture of Core developers Picture of Plugin developers Picture of Testers

The one thing I have heard consistently, from my users, and here, is the difficulty of seeing which posts have been read, and which haven't. So, I decided to do something about it.

I have attached this hack - although I'd like it to become accepted by Moodle so it isn't a hack - that does just that.

This adds a new table to Moodle, that tracks what posts each user has viewed (and therefore read), and adds a field to the 'forum' table. Now I know there are fears that adding more tables will slow down Moodle too much, but I believe this implementation is reasonably fast. The table is indexed by userid, postid, discussioid and forumid, so one database read per discussion display is needed - just get all entries for the given user in the given discussion. It also tracks the time of the first read and the latest read for each post. When the posts/discussion/forum are removed, so are the read records.

For now, this has been set up to work effectively only for threaded mode, although it can be expanded to the other modes. You can turn it on or off in the Forum settings (for each forum), be selecting or deselecting the 'track reads' option (also new). That way, if we do accept it into the full release, it can be turned off for those who fear the slow down, or those who simple don't want it.

If you do activate it now, you won't be able to select a forum viewing mode for that forum - it will force threaded. Any posts that you have not read will be highlighted in light green colour. I will change this to use a defined style.

To get the other modes to work, I will just need to add some kind of  'read' button, that allows the user to mark a post as read. Right now, if a post has been viewed, it has been read. Also, I will add a 'mark all as read' control, so that a user can flag all posts as read.

Martin, et al, I'd really like you to look at this, because I believe Moodle really needs this. I have enclosed all new and changed code in "Begin Forum Extras code" and "End Forum Extras code" comment blocks.

If you do try this, I suggest you try it on a non-production system, and copy your existing 'mod/forum' directory somewhere so you can restore the code easily.

(also, there is another setting you will find in there called, 'posttypes'. I'm playing around with another categorizing system for posts that's not ready yet - ignore this).

mike

Average of ratings: -
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Good on you man!  approve

I'm off to bed but a conclusion I reached previously about this was that it should only store flags for messages that are UNREAD within a certain timeframe. That should solve the scalability problem.
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

I'm not sure I follow you here... When you wake up, maybe you can elaborate?

I don't think we want to set up a table that lists all messages not yet read by a user. That would mean adding a new entry everytime a post was added, and everytime a user was added, and then trying to manage that. Its much easier to add an entry when a post is read.

The only thing that makes this way difficult is the nested modes - which I'm sure you like, and I like, but most of my users hate. They actually prefer threaded.

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Przemyslaw Stencel -

First of all, thanks for working on this, Mike! big grin I'd love to see this in the official Moodle distribution.

It seems that Martin hasn't woken up yet, but since I haven't gone to bed yet, I may elaborate on this - I remember a discussion I had about this with Martin waaay back.

If you store flags for READ messages, your table will keep growing endlessly. However, you can also choose to store flags for UNREAD messages. You're right Mike - it will mean adding a new entry everytime a post is added. However, Martin mentions messages being flagged unread within a certain timeframe. There could be a time setting (the best place I think would be the admin's forum module settings page), which, when set to e.g. 14 days, would automatically (using the cron script) unflag all messages which are 14 days old AND still unread. This way you can keep the table size low.

I'm not sure how new users should be dealt with. You suggest that a new user should initially have all past messages marked as unread. Just wondering - maybe it's not necessary? Maybe only messages posted after the user has enrolled in the course should be "new" for him/her? Or maybe this could be another admin setting? What do others think?

Przemek

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
If the table was for read posts only then imagine how big it would be for this site (to the order of 48000 posts x 2000 users ...) and most of it would be irrelevant information making the database work hard.

So I think it's best to add a whole bunch of entries every time a post is created, with a timestamp that matches the post. These are like little flags saying "read me!". This shifts the bulk of the processing to posting time, and makes the display of unread posts (which will happen more frequently) very easy and quick.

  • When a post is created, add flag records for all participants to a course.
  • When a post is updated, delete all existing flags for that post and add new flags for all participants in the course.
  • When a post is deleted, delete all existing flags for that post.
  • When a post is displayed on screen in full - delete the flag for that user+post from the table.
  • When an unread post is displayed within Moode at all, add an icon or colour to make it appear different (read posts don't need icons, etc)
  • When a post is sent out via email (singly or full digest) - delete the flag for that user from the table..
  • When flags get old ($CFG->forum_unreadpostsold) delete them.
  • When a new user is enrolled in a course, add a flag for every post newer than $CFG->forum_unreadpostsold ago.
  • When a site is upgraded to this system, do NOT add flags for everything.

I think that would cover it!
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Junaid Haroon -
Matrin, the table gets huge when records older than a configured time are kept and not regularly deleted whether the system stores READ or UNREAD flags.

I suggest to keep READ flags and remove READ flags after every $CFG->forum_readpostsold days. Older messages will always be considered READ as in your scheme. This way the posting time and viewing time both will be minimum.

If I rephrase the above points using this scheme, it would be...

When a post is created, add flag records for all participants to a course. When a post is updated, delete all existing flags for that post and add new flags for all participants in the course. When a post is deleted, delete all existing flags for that post. When a post is displayed on screen in full - delete  add the flag for that user+post from the table. When an unread post is displayed within Moode at all, add an icon or colour to make it appear different (read posts don't need icons, etc)When a post is sent out via email (singly or full digest) - delete  add the flag for that user from the table.. When flags get old ($CFG->forum_unreadpostsold) delete them. When a new user is enrolled in a course, add a flag for every post newer than $CFG->forum_unreadpostsold ago. When a site is upgraded to this system, do NOT add flags for everything.

I think this would make the system simpler with no increase in database size.

In reply to Junaid Haroon

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

Hi Martin / Junaid -

I think, a combination method like this one is a better solution. Martin, storing 'read' flags wouldn't be as big as you mentioned - 480000 posts x 2000 users - because every user doesn't read every post. In fact, I'll bet they read a small fraction of that. That's why I'd rather not use the nested views as indicators of having read the message. With threaded view, its much more simple.

However, if you go with your suggestion, we do need to have an entry for every user/post.

The read table will get cleaned up every time a forum is closed, or a post is deleted, and every time a user is unenrolled or removed.

However, if we add an 'older message' setting like the one suggested, we could regularly clean the table out even more (using a cron task). I think this a good solution. So, it would work like my original suggestion, and then any post older than the 'older message' setting would be removed from the 'read' table and always considered 'read'.

My vote!

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Przemyslaw Stencel -

Mike, you wrote: I'd rather not use the nested views as indicators of having read the message.

I'm not sure I understand you correctly here. Do you mean that if a user views messages in the nested mode, they will not be marked as read? What if someone uses mostly nested view to read discussions? When will that person's messages become read? mixed

Can you elaborate on this, please?

In reply to Przemyslaw Stencel

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

No - well maybe...

The problem with nested mode (and flat mode) is that all posts are displayed on the screen. If we use the logic that 'displaying a post on the screen = read', then the simple act of displaying a discussion marks all posts as read.

That's what I'd rather not do. Although technically, its the simplest.

Another way would be to have an actual 'read' button on each post in nested or flat mode, but I think that might be too onerous on the users.

Maybe its best to just mark them as read if they've been displayed?

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Przemyslaw Stencel -

If we use the logic that 'displaying a post on the screen = read'...

But the same logic is used in the threaded mode smile - if you display the message, it means you've read it. I use the nested mode almost exclusively and from my point of view all modes (threaded, nested and flat) should use the same logic (displayed = read).

... to have an actual 'read' button (...), but I think that might be too onerous...

You're right, it wouldn't make much sense.

To avoid confusion, maybe we should think of this in terms of seen/unseen rather than read/unread? This distinction should be explicit in the way this feature is implemented. E.g. all unseen messages would have an icon - when a user hovers the mouse over the icon, a popup (similar to the one used in the calendar) appears which says something like "you haven't seen this post yet".

Oh, and there is one more thing which we should discuss here - how do we treat messages which a user receives via forum subscriptions? Once they are sent off to the user's email, are they still unseen or should they from this moment be considered seen?

In reply to Przemyslaw Stencel

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

But the same logic is used in the threaded mode smile - if you display the message, it means you've read it.

Yes, but in threaded mode, I select the one message I'm interested in to mark it as read. I can display their subject lines with indicators of whether they are read or not first. So, clicking on a discussion in threaded mode doesn't display all of the messages on the screen at one time.

Maybe the 'seen/unseen' will work. Good idea.

I was avoiding this intentionally... wink   I think we ignore that for now, and treat it as unseen. Ideally, it would be nice to have a mail responder that would allow me to mark it as seen from my inbox - maybe even provide a reply - but that's future....

mike

In reply to Przemyslaw Stencel

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I mentioned the email case above ... I think getting it via email is the same as showing it on screen ... it will be particularly useful when discussion-level subscribing becomes possible.
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
> Maybe its best to just mark them as read if they've been displayed?

Absolutely.  Then, later, we have lasers shooting out of the screen reading their retinas.
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

big grin

But seriously, my concern is only for the nested and flat modes. If I join a discussion already in progress, or don't follow it as often as I should, or its simply just extremely active, then all of the posts are displayed the moment I access it. What if I don't get 'round to actually reading them all that time? That's where I find it a bad thing to mark 'displayed' as 'read'

With threaded, its much more obvious.

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
If it bothers you, then you just switch to threaded mode - problem solved!

Otherwise we have to force the user to manually mark posts as read in nested mode and I don't need laser vision to see that would be very unpopular. wink
In reply to Junaid Haroon

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
The database size would be kept down that way, yes, good idea! approve

But I remain to be convinced about the complexity of the SQL required (please convince me!).

I think the SQL for populating the course page and the forums page will be much simpler using my method (it's a simple matching count on two fields (user, forum) in one table). In fact, all forums can be retrieved in one simple get_records_select statement.

Otherwise we'd need to count all the read flags for each forum newer than a certain date, and subtract that from a total count for each forum from the forum_posts table.

I just like the elegance of this way around. It's the UNREAD entries that we are usually interested in (not the read ones). If an entry is in there, then it means the post is unread, otherwise not. Using the READ flasg method, we need to add exceptions all the time depending on the age of the post.

Since this gets repeated so often by every user I think it's significant.

I haven't looked at the SQL for the in-discussion displays but there's probably not much difference ...

Perhaps we should work out the SQL here first and talk about it.
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

The way I did it, its one simple SQL call:

function forum_get_discussion_read_records($userid, $discussionid) {
/// Returns all read records for the provided user and discussion, indexed by postid.
$select = 'userid = ''.$userid.'' AND discussionid = ''.$discussionid.'''
;
$fields = 'postid, firstread, lastread'
;
return get_records_select('forum_read', $select, '', $fields
);
}

This returns an array, indexed by 'postid' of all the read posts for the given discussion for the given user. Then, when I'm writing out the list of posts for the discussion, if the array has an entry for the given postid, I know its been read. I keep the array in a static variable so it doesn't keep rechecking. So, one call does it. Right now, I'm using it in the 'forum_print_posts_threaded' function.

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

So, as a follow on to this, all I need to do as I display the posts, is load an array with the read records from the previous function:

$user_read_array = forum_get_discussion_read_records($USER->id, $discussionid);

Then, as I write out the posts, a post is not read if its not in the array and its newer than the configured cut-off data (current time less old post duration):

if (!isset($user_read_array[$post->id]) and $post->modified > $olddate) {

The cron task will run periodically, and remove any read records for posts older than the cut-off date. That will clean out the table. This cron task will be pretty heavy, but thats a maintenance issue.

Am I missing something?

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Hi Mike! smile

This "in-discussions" part is not really the problem (as I mentioned here) the part I'm worried about is the display on the course and forum pages (and also MyMoodle pages later).

On the course index, for example, there will be a small text display next to each forum activity showing the number of unread messages for each forum.

This needs to be calculated for each user, on each page load (it can't be cached). The course page in particular is very heavily visited.

To create a nice array that the course page can use to put this data into the display is very simple when an unread flags method is used:

$unread_counts = get_records_select('forum_unread', "userid = '$user' GROUP BY forumid", '', "forumid, count(*)");

but it's significantly more complicated using the "read" flags method. (I'm rushing so the SQL is left as an exercise for you wink)
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

Okay.

Let's start at the list of discussions page. Here we currently list the discussions in the forum with number of replies. The only call we have to make at this time is to count the number of replies to the discussion post (already required):

SELECT p.discussion, (count(*)) as replies
    FROM forum_posts p, forum_discussions d
    WHERE p.parent > 0 AND p.discussion = d.id
    GROUP BY p.discussion

To add any other information (number read / unread), we will have to as a minimum, make one more SQL call.

My way, would require two more calls, one to count the posts before the cut-off date (and therefore read) :

SELECT COUNT(*)
    FROM forum_posts
    WHERE discussion = $discussionid
        AND modified < $cutoffdate

and one to count the read posts:

SELECT COUNT(*)
   
FROM forum_read
   
WHERE userid = $userid AND discussionid = $discussionid

The total unread would be the number of posts less the other two counts. We already do the total number of posts. The forum_read sql is relatively painless because we're searching on two indeces. The only expensive part is the count of posts by date - the one your way wouldn't need.

For the forums and course pages, currently we only count discussions, not posts. So, no matter what we would need at least one more SQL call. If all we want is to display unread posts, your way will be the cheapest by far. But, wouldn't we want more info - number of posts as well? I would think that if we're going to display unread posts per forum, we probably want to be able to display total posts per forum as well. In fact, it becomes pretty complex once you want to count posts for a forum.

$total_posts =
SELECT COUNT(*)
   
FROM forum_discussions d, forum_posts p
   
WHERE d.forum = $forumid AND p.discussion = d.id

$expired_posts =
SELECT COUNT(*)
    FROM forum_discussions d, forum_posts p
    WHERE d.forum = $forumid AND p.discussion = d.id
    AND p.modified < $cutoffdate

$user_read =
SELECT COUNT(*)
   
FROM forum_read
   
WHERE userid = $userid AND forumid = $forumid

Maybe we should also think of adding the forum id to each post? This would allow us to more easily count these records without having to merge the discussion table all the time. What do you think to that?

I'm not against storing unread posts, its just that I like the idea of tracking only one event by using read posts (view the post), vs. several using unread posts (add post, update post, add user, remove user).

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
PJ will turn in his grave if we de-normalize the forum tables (even further than they are) by putting forumid in the forum_posts table! big grin wink

I don't think total posts are needed on the course page ... I personally never look at these numbers on the phpBB-type sites though I suppose some people do. For Moodle I'm very wary of keeping the course page free of clutter. I can see it might be a good addition to the forums listing though.

I also do like the reduced number of changes to Moodle when using READ flags (although the difference is not quite one vs many as you say, see Junaid's post above) but I am painfully aware of efficiency since I run so many sites, some rather busy. A small slowdown for one crucial page can multiply into a dead site for someone on a low-power web host or an institution with 17000 students.

In fact, the SQL for the course page is even more complicated than you have described because you have not yet taken grouped forums into account yet...
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

You're right, I have to look at the SQL for groups...

The slowdown issue is really the only one we should worry about (well, other than functionality). The data normalization issue is an interesting one. I find that normalizing data, and creating fast data-based web applications, are sometimes at odds with each other - but that's really dependent on the back-end technology. I need to do some speed tests with MySQL to find out whether denormalizing the data by adding an extra redundant index is significantly faster than a two (or three) table merge. From what I read in the MySQL forums, MySQL is supposed to be blazing fast even with large numbers of table merges.

If it comes down to the forums listing page (and possibly the course page), this is an issue only if the extra data is displayed there. Even if we track reads, we could make the extra display an option that could be disabled.

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by W Page -
Hi Mike,

That sounds really neat. That option would allow Admin/Teacher to customize their courses in regards to how the Forum displays per Course News, Topic or Week.

Hopefully this code could eventually be placed into a future MyMoodle page.

WP1
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Hi Martin -

I got the SQL working so that I can pretty much do it with one extra statement at the forum index level. I measured it against some of my bigger courses and forums.

I added a timer to the pages (such as mod/forum/index.php) to time the script. What I saw was that if I wanted to show total posts and unread posts in each forum listed, it took about ten times as long (0.3 to 3.0). That could be significant!

However, if I set the 'forum' and 'groupid' fields in the 'forum_discussions' table, and the 'discussion' and 'userid' fields in the 'forum_posts' table to be MySQL indexes, then there is no significant difference in script time. In fact, the scripts ran with identical times whether or not I was tracking reads when I used these fields as indexes.

I think this might be the way to go. Other than a little disk space, I don't think there is any penalty for setting extra field indexes in MySQL. Do you know of a downside?

mike
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
With further testing, I can actually improve the speed quite a bit by splitting my queries into two separate ones, and merging the results with arrays in PHP. So much for 'lightening fast results, even while merging many tables' for MySQL.

Anyway, in either case, making those fields into indexes is crucial.

mike
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Enrique Castro -
Picture of Core developers Picture of Particularly helpful Moodlers
Hi,

I think that this functionality is really useful, and one of tah msot demanded. I would love to see tha forum discussions displayed as a usenet newsgroup (like in the news reader in Mozilla). This kind of things are standard in newsgroup readers.

However, the system could be even more useful if, in addition to an automatic flagging system by date there would be a manual system for flagging mesasges. We already have one: the rating system. I have been using forums with a custom scale "seen/revise later/important/erasable" (with all can rate messages, students can see only their own rates).
Using this scale goes beyond simply detecting the new activity, but serves this function too!

If a message do have a rate, you know you have read it.
If the message is unrated, it is probably new (yes, now only probably, but ...)
I think it wouldn't be very difficult to change this rate to "seen" after and only after the message has been displayed in screen (for instance adding the code to the last lines of forum_print_post, before return), provided that the message in unrated yet (do not overwrite manual rates). In this way, all displayed messages get the "seen" rate automatically for a given user (or only "displayed" if you want to be conservative about displaying/ real reading of a message).

To further, and easily, manage these "rates", the interface could include other buttons to apply a rate to all messages in a discussion:
  • mark all messasges as "something"
  • mark all unrated messages as "something"
  • mark messages in current thread as "something"
  • clear all rates
In addition, the message sorting options coud include "sorted by rate"

Rates are now a property or each message (in rated forums), would be very difficult to duplicate this property? Would it degrade database performance in a bad way?
How difficult would it be, to adapt the rating system to this function ? (trimming exporting rates to notebook etc).
I think that the ability to mark individual messages messages has a lot of posiible applications in addition to this one.

- Enrique Castro -

In reply to Mike Churchward

Thread Screen Shot

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

The thread screen shot, with unread highlighting.

Attachment threadscreen.jpg
In reply to Mike Churchward

Re: Thread Screen Shot

by W Page -
Hi Mike!

Happy to see that you decided to make this a project.

Just wanted to point you to some other thoughts on the matter which have been addressed in some other threads. Please check out,
Thomas Robb's suggestion in another forum ,

Moodle versus Blackboard --Forum messages read
http://moodle.org/mod/forum/discuss.php?d=9345&parent=45187
http://moodle.org/mod/forum/discuss.php?d=9345&parent=45193

Also it would be nice to have something like what is on the image below in a future MyMoodle page.

I see Martin has a center panel after login which is named "My Courses" so he appears to be quietly moving toward the
MyMoodle page.

WP1

Attachment forum_summary_example_01.gif
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

Nested screen. The open book means that the post has not been read while the closed book means it has been read. We could also try different colour backgrounds for read/unread posts. Note, that for nexted views, the posts will always be read after this screen is exited. Vieiwing them on your screen means they've been read.

mike

Attachment nestedscreen.jpg
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by W Page -
Hi Mike,

Could the code you are creating be applied to the features I referred to in the post I made above in the future?
.http://moodle.org/mod/forum/discuss.php?d=10072&parent=48649

That is,
  • Thomas Robb's suggestion
  • the creation of a "Summary" type of area for posts on a MyMoodle page.

WP1
In reply to W Page

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

I don't think the code I'm talking about is needed to implement that. From what I saw there, that code already exists, it just needs to be packaged differently.

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user. ! an alternative ... !

by Roland P. Becker -

Hi

What about some alternative approach. .. just the idea:

Beleaving that the user is who should decide, what he has read and may be what he wants to read further down the road, we may give the user the choice to mark a text for remembering or for a kind of "favorite" - this by the way may be useful not just for forum but for all areas ... maybe as a kind of "extra bookmark system". By the way thre is a nice Open Source script called "SideBar" at Sourceforge, which may have some potential for navigation and bookmarking

Then, once such a system is in place, some automatics could be established - like "watch this topic" --- or like "remind me to reply" ...

Well, its late by now and after a long day, I should come back to this topic later tomorrow. ...

Best regards from Costa Rica

Roland Becker

In reply to Roland P. Becker

Re: Forum Extras - adding 'read' tracking by user. ! an alternative ... !

by Trizit Benjaboonyazit -

Thanks for Moodle, I enjoy its functionabilities very much.

and I am interested in the read tracking code too.  My previous webboard is text file database with limited feature. But the good thing is that it can show how many people read your topic just like,


[ Home | Background | Main Board | Photo Gallery | PDBT | Pitlok | RaksaBangkaew | Tana | Nisit | KM Search | e-Learning ][ For Eng/Jap Page | Webcam eBook | TBK Link | GuestBook | GoInter | Donation | e-Commerce | New Topic | Help ]
Total 408 Topicssearch.gif Search Topics
NumberTopicsPosted by [Date]ReadReply [Date]
openfd.gif 0502กระทู้อำลา บ้านรักษ์บางแก้ว cam.gif เว็บมาสเตอร์ [11 ก.ค. 2547]180 18 [16 ก.ค. 2547]

I have downloaded and tried the read tracking codes from this topic.  but still it can not show, even though I have re-installed moodle with these code. My version is 1.4dev,  what is wrong with it ?

My moodle site is at http://elearning.bangkaew.com/ , it is about Thai Bangkaew Dog, please help.

In reply to Trizit Benjaboonyazit

Re: Forum Extras - adding 'read' tracking by user. ! an alternative ... !

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
It's very hard to tell what could be wrong, since you are using an unsupported patch to a mostly unsupported developer's version ... anything could be wrong.  smile 
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user. ! an alternative ... !

by Trizit Benjaboonyazit -

Thank you. I will wait until the next development.

But do you have any plan to put it into your next version.

I hope there will be some kind of switch to turn it off or on.

For light-load webboard, I think it is desirable to keep read tracking and show the number of read.

How about Mike ?  Do you have any suggestions ?

In reply to Trizit Benjaboonyazit

Re: Forum Extras - adding 'read' tracking by user. ! an alternative ... !

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Hi -

I agree with Martin - don't use the code I posted here. Its intended for developers and to be used as a discussion.

I'm doing some load and speed tests right now, mostly to determine what the impact of using 'read' records (as opposed to 'unread' records) will be. My intent is to allow this option to be toggled on or off. I'm hoping the speed and load issue will be minimal enough that the toggle won't be necessary, but at least its an option.

Martin has shown interest in this being part of Moodle and I'm sure he will include it in the development path if the code meets all requirements.

mike
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user. ! an alternative ... !

by Trizit Benjaboonyazit -

Thanks, Mike.  I wish you great success.

Sorry that I am not keen on coding.

I think these 2 webboard open source may give some hints to your work.

YaBB SE  and  PNphpBB2

In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Martin -

I have attached a zip file containing forum code changes and db updates to facilitate forum read tracking. This method uses 'read' tracking rather than 'unread' tracking. It also updates the databases to use extra index keys to speed up searching. The files need to go into your '/mod/forum' directory.

If you set the forums to use read tracking, than the code kicks in. If not, it runs as normal. The easiest way to set all forums to use read tracking is to set the flag for all entries in the forum table using phpMyAdmin.

I have also added a temporary debug feature to the forum view, index and discuss php files that allows you to see the speed of the page generation. If you turn debugging on, the time taken to generate the page (including all database access) will be displayed at the bottom of each of these pages. You can use this to see the difference in time between a forum tracking reads and one not tracking reads.

Can you test this on a system that has busy forums? Do you have a test system that you can do it on?

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
I forgot to mention, any post over 90 days old will be considered read.

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by W Page -
Hi Mike!

Looks nice.  I do not feel any increase in page loading, but, I only made one posting with some replies to check out the script.

WP1
Attachment forum_read_changes.gif
In reply to W Page

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
If you turn site debugging on, you can see the actual difference at the bottom of the pages. It will display the page load time with both read tracking on and off.

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I will try to have a look at it today.  I'm still worried about the extra load but I'll give it a go.
In reply to Martin Dougiamas

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Try it with debug on so you can see the load times. I tried in my busiest forums, and the time increased slightly, but I don't know if that will be an incremental increase depending on number of users and posts, or not.

Also, if this does work for you, I've seen places in the forums code where we could realize more speed by reworking some of the code. There are places where multiple, similar database calls are being made when they could be saved by parameter passing, object construction or (eep) globals.

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I actually already have this on at all times in my sites. This page took 0.582507 seconds.  smile

The page I'm most worried about has not yet been implemented - the course listing.  This is quite heavily hit.
In reply to Martin Dougiamas

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
I can implement that one, sure. So, you're talking about any course main page, displaying the counts of read/unread in each section, correct?

It should be comparable to the forums listing page for a course - it will use the same code.

mike
In reply to Martin Dougiamas

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Hi Martin -

The attached 'course/lib.php' file has an updated 'print_section' function that displays the total number of posts followed by unread posts '( nn / nn )' on the course view page. Again, you can use debug to see the difference in display time for tracking on or off.

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Hi Martin -

Have you had a chance to try this yet?

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Unfortunately not, Mike, I'm very sorry.  Just too many loose ends around already for 1.4.  I'll try to put it in soon after we branch.
In reply to Martin Dougiamas

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Okay. I was hoping to get it into 1.4, but I'll shoot for 1.5.

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by W Page -
Hi Mike,

Is there some kind of "icon" which could be used to demonstrate "read" and "unread" in the same location the words are located in now?  I think that would look much "cooler". than the text.

WP1
In reply to W Page

Re: Forum read tracking - Test Needed!

by Hans de Zwart -
I am not sure that I agree with that. When there is space for text, text often is less ambiguous and shows its meaning easier.

That made me think that "read" itself is ambiguous. Maybe people will click on it when they want to read the post instead of realising that it means they read it. Or people might want to click on it to mark it as read.

Oops, I just realised it probably wouldn't be a link. Maybe the text is fine then....
In reply to W Page

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
When I get Martin's blessing as to the tehnical direction to use, I'll create CSS elements that define how things look. That could include images.

I just don't want to spend the effort until approval.

mike
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by W Page -
Hi Mike!

Is there a way to hack the code so images can be used to depict "read" and "unread"?

WP1
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by W Page -
Mike,

Can you begin another thread about this topic?  This one is getting kinda long.

WP1
In reply to Mike Churchward

Re: Forum read tracking - Test Needed!

by C R -

what am i doing wrong?  i instaledl your new forum files and I get an error message that I am running an old version of moodle.

I downloaded and install the 8-11 version of moodle and i still get an error that i am running an old version.

I would appreciate your help.

In reply to C R

Re: Forum read tracking - Test Needed!

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Sorry, I was away on vacation last week...

The files I was using were based on the development version (1.4). I'll take a look at what they do to 1.3 - but they weren't really meant for a production system yet. These were meant to test with developers so that (hopefully) we could get it into the full version.

When do you get this message? When you first access the system, or when you access a forum?

mike

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Dennis Daniels -
I checked the previous posts for mention of a log. Logging the status of a post is important. I hope that the read status is written to the great logging tools!

A different RFE: I'm finding it very tedious to get to the next unread thread  ... I have to click up the thread and then back down again if there are multiple threads.  Maybe set the read status as a variable, if read then go to next unread.

thanks for a great app!
Dennis


In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by W Page -
Hi Mike!

What is the status of the "read" code?

When I installed it it worked well except for the following,
  • Missing language pack
  • No "icons" to determinine if a post was read or unread [just something I would like, the words were plainly evident].

QUESTIONS

  1. Have you done any code changes since the August 9th post? If so would it be "OK" to share them?
  2. I did not see it in v1.5 Dev. Will it eventually be placed into that version?

WP1
In reply to W Page

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
I've been in a holding pattern, waiting for Martin's time to review. I'd like to get the changes into the dev branch, rather than supporting them separately. I'll try and take another look at them next week and see if I can find some way of earning Martin's approval - he's concerned about the stress it might put on the course pages on very busy sites.

mike
In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by W Page -
Hi Mike!
Maybe we can ask Martin for a poll about this. smile  Similar to the one recently about Block Head Type.
Martin are you out there?  Can we have a poll about adding this feature?
WP1
In reply to W Page

Re: Forum Extras - adding 'read' tracking by user.

by Steve Ricketts -

What is the status of this extra?  Is it going to be added to the 1.5 version?  I've had some difficulty getting the code posted above to work with my Moodle Installation.

Thanks,
Steve Ricketts

In reply to Steve Ricketts

Re: Forum Extras - adding 'read' tracking by user.

by W Page -
Hi Steve!

I asked about this feature recently, but, I have not been able to get an answer. Mike C is working on this. I was hoping to see it in one of the 1.4.x versions but it has not been oficially put in. Would be very useful.

WP1
In reply to W Page

Re: Forum Extras - adding 'read' tracking by user.

by Steve Ricketts -

Well I'm hoping to hear something soon..Our District would like to implement Moodle but this is the one piece that is holding us back.  Have you had luck getting the code version above to work?  I copied the files into the 'Mod' directory and I have the buttons to turn on "read tracking" but I see no difference in the postings.

Thanks,

Steve Ricketts

In reply to Steve Ricketts

Re: Forum Extras - adding 'read' tracking by user.

by W Page -
Hi Steve!

The code has changed so much sine Mike posted the initial code that I am not surprised that it is not working with the present code.

I do not know why it has not been included in the official release as of yet.  It appears to be a feature many would like and maybe even need for whatever reason.  It might be a XML related reason.  I think v1.5 will have some XML code and MD may be holding off for that.

We just have to wait for his response to your query.

WP1

In reply to Steve Ricketts

Re: Forum Extras - adding 'read' tracking by user.

by Steve Hyndman -

Steve and everyone,

I agree, the ability to mark posts as read and to show only unread posts would be a very beneficial feature. I've facilitated online courses since 1995...starting with the University of Phoenix when they used "Conveine" and later when they switched to Outlook Express Newsgroups. Both of these systems were (are) very basic systems for delivering online content (neither come close to matching Moodle), but the ability to mark posts as read and hide/show read and unread messages were/are critical when conducting a class primarily via an asychornous online discussion.

I've also facilitated online and web assisted classes via WebCT and Blackboard...my university uses Blackboard now. The feature in Blackboard that allows you to mark individual messages as read and/or mark all messages in a thread/forum as read or unread is very handy. I've been teaching at my current university for 5 years and have used Blackboard during that time; however, starting with the next term (Jan 05) I will be switching my classes (three educational technology classes) to Moodle.

I have been experimenting with Moodle since July of this year and it does (as does all technology) have some drawbacks...the subject of this discussion thread being one of them. I'm going to teach my Spring courses using the moodle discussion forums and see how it goes...I think I can make it work, but won't know until I give it a try. I'll have about 30 students per class and in a typical two week (14 day) discussion, the class generates between 400 and 600 posts.

If I find that I (or my students) can't manage this through a moodle discussion forum, then I may end up using a PHPBB discussion board to take care of this element of my online class.

After reading this discussion I was thinking of a way to handle the need for better control over discussions and still use Moodle as my platform. So I installed a PHPBB discussion board on my site and then set a test install of Moodle to authenticate to the usernames and passwords on the PHPBB database. Everything seems to work great. If need be, it seems I could:

1. Have students create their accounts in PHPBB
2. Login to Moodle using that same username & password (or course they will need to create their moodle profile the first time they log in)
3. Whereever I want to use a discussion board in Moodle, I could just link to the PHPBB. (Of course, when students click on the link, they would have to log in, but that's minor, they would still be using the same username and password.)

I realize this is a "workaround", but it is one way to incorporate a more powerful discussion board into Moodle.

I think Moodle is a great piece of software and can't believe that something this powerful is out there "Open Source". I also can't believe why institutions (mine included) would pay for systems like "Blackboard" when this is available for free. Just think what would happen if all institutions (and I know several alrady do) would start using this and direct the energies of their technology people to improving this system.

Oh well....enough rambling for now. Hope this helps...I wouldn't let something like the ability to mark messages as being read stop you from incorporating this in your school....if that is critical, then there are workarounds until this becomes a part of Moodle.

Steve

In reply to Steve Hyndman

Re: Forum Extras - adding 'read' tracking by user.

by W Page -

Hi Steve!

If you have the time [I know it is holiday time right now].  Could you outline more specifically how you were able to get the phpBB to recognize the UserID and Password for each student in Moodle.  I am still somewhat of a novice and I have read many times on the forums of this being done with other programs but was too chicken to try it out.

Thanks in advance.

WP1

In reply to W Page

Re: Forum Extras - adding 'read' tracking by user.

by Steve Hyndman -

I actually did it the other way around...I set Moodle to recoginze the Username and Password for phpBB.

To do this:

1. log into Moodle as an Admin and click on "Users"
2. Click on "Authentication"
3. From the dropdown select "Use an external database"
4. Complete the database informaiton (blue area below) for your phpBB database.  You should know the database name, user and password...these are for your phpBB database. The table is named "xxx_users" (replace the x's with whatever you set the table prefix to be). The database user field is "username" and the password field is "user_password". (You may want to check these in your phpBB database just to make sure, but I assume these will be the same since they are created automoatically by phpBB).
5. Since the passwords in phpBB are encrypted in the database, I selected "MD5 encryption" for the last entry.
6. The remainder of the entries are optional and pretty self-explanitory.

Save these settings and you are done.

Now, you won't be able to create accounts in Moodle. You create your accounts in phpBB and then use the same username and password to log into Moodle. Once you create an account in phpBB, the first time you log into Moodle it will prompt you to complete your profile for Moodle.

Hope this helps.

Steve

In reply to Mike Churchward

Re: Forum Extras - adding 'read' tracking by user.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Forum read/unread tracking is now (finally!) in CVS (and 1.5). Thanks, Mike, for the bulk of the code and your persistence in sending it to me. smile

I still have some reservations about the scalability of the load on the database, (some caching may be necessary) but Mike's included a forum config variable to switch it off if big sites find it puts too much drag on their system.
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by W Page -
Hello All!

big grin big grin big grin big grin.  I think there are a lot of Moodlers who will be happy to hear this.

Couple of questions:
  • Is "read" & "unread" indicated by image or only text?
  • Would an image place more of a drag on the system
  • Is there an option to choose one or the other types of display? I know there was discussion by some Moodlers regarding preference for text or image. An option to have either could then,
    • Be determined by personal preference of the admin, instructor or even the student (I have been reading the posts relating to the new theme configuration in Moodle and possible individual choices of display by admin, instructors and maybe even students.)
    • Be determined by server. If it is a server drain (on some servers) to use images (I do not really know if it would be), then, only text could be used.
WP1
In reply to W Page

Re: Forum Extras - adding 'read' tracking by user.

by Ray Lawrence -
First of all can I just say this is really neat. I hope the server load issue doesn't limit the usefulness of this feature.

The highlighted "Unread" link in the discussions link navigates the user to the earliest unread post in the discussion (as far as I can see).

On the course homepage the highlighted text looks similar but is not clickable.

From the point of view of (gulp) usability it may be better if the latter has a different appearance e.g. remove the coloured highlight and make the text bold. Alternatively, if the appearance remains the same could the text be made into a link which would navigate the user to the earliest unread discussion? This would make navigation consistent at the respective "levels".

Also, I had a "doh!" moment when I realised that if Unread > Replies then the original post in the discussion had not been read.

It is the (correct) assumption that the posts in a discussion have not been read when it has not been visited. When the the discussion has been visited the assumption is made that all previously unread posts have been read - this may not be the case. There must be a word (or two) which conveys what is actually happening better than "Unread" but it eludes me. Do others feel this may cause some confusion or is it one to kick into the long grass?
In reply to Martin Dougiamas

Re: Forum Extras - adding 'read' tracking by user.

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Martin -

I still have some reservations about the scalability of the load on the database, (some caching may be necessary) but Mike's included a forum config variable to switch it off if big sites find it puts too much drag on their system.

I think there are a number of ways this can be improved in the code as well. Now that we have it as part of the base, I could see areas in the code where certain calls could be combined with information making some of the database calls less necessary. I will help in this area.

mike