What developer documentation do we need?

What developer documentation do we need?

by Pierre Pichet -
Number of replies: 22

Hi Tim ,

Thanks for your worry that I work unnecessarily on docs.

Given

  • that Gustav wrote me months ago that altough he and his group has worked a lot to implement class coding in Moodle code, the work was not finished everywhere and the question code was one of them,
  • that Howard wrote this week on import/export improvments
    http://moodle.org/mod/forum/discuss.php?d=66773
    which means among other things a better code harmonization between questiontype and format classes,
  • that the code (and the docs) to question-plug-in needs improvment,
  • that the new proposal of letting (hundreds) of students create, save, import, export questions will be useful if the question code is well known and rock solid,
  • that before getting back the code to where Gustav extract it, everybody needs a good (if not a perfect) knowlegde of the actual one,

my plan is to first try to set the actual code flow in the docs so that it can be studied, commented and improved.

There are parts of this code that remains mysterious to me, so others could contribute.

In the first file http://docs.moodle.org/en/Development:Questiontype_class

I plan to order (classify) the different functions and give there reference in the other files (there a three at the moments).

In complement with the other files already in the docs ( question engine, database etc.), I will try to show ( and comment if necessary) the actual code flow.
As usual, everybody is invited to complete although I understand very well that as an old professor, I have more time to do this than those like you (Tim, Howard etc.) who are always on the front solving bugs and trying to find time to develop new code.

Cheers

Pierre

P.S. My email is working but some time Uqam receive more than 100 000 emails in a day if not in an hour...

Otherwise put the message in Skype. I look at it at least twice a day (the Montréal one with a 5 hours delay from yours...)

Average of ratings: -
In reply to Pierre Pichet

What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்

You are absolutely right that the question bank code needs better documentation. And it is brilliant that you want to write it. Let me just explain the documentation I would like to see, and why I think that is the right approach, then I am happy for you to proceed in the way that seems best to you.

What makes code difficult to document is that it always changes. That is good; It is progress; but we need to think about how we are going to handle that change when writing the documentation, so it remains useful with as little maintenance work as possible.

Code needs to be documented at three levels of detail. Starting with the most detailed and working up they are:


1. The code itself

Particularly in Open Source, the code is not just a way for the programmer to tell the computer what to do. It is a way for programmers to collaborate with each other. Other programmes must be able to look at it and follow what is going on, and understand how they would need to modify it to fix bugs or add new features.

In a world where the code is constantly changing, it is particularely good if the code can document itself, becuase that documentation will never be out of date.

When I say 'the code', I include the brief inline comments that are used to clarify the code, for example:

// Print each answer in a separate row
The code itself should answer the questions: 'how does this part work?'.


2. A description of each class, function, method, etc.

Next, for each function, method, global variable, class, etc. in the code, there should be a description of what it is for. This should mainly be from the point of view of someone who wants to use this function, but who does not necessarily need to know how it works internally.

This is the level of documentation that is likely to go out of date quickest, because it needs to be quite detailed. Therefore, most modern programming languages have a way of including this documentation inline. For PHP the system is called phpDocumentor. To use it, you add specially formatted comments in the code, just before the function (or whatever) that you are documenting. For example:
 /**
 * The name this question should appear as in the create new question
 * dropdown.
 *
 * @return mixed the desired string, or false to hide this question type in the menu.
 */
 function menu_name() { 
The idea is that since this documentation is stored right next to the code, this maximises the chance that it is kept up to date when the code is changed.

Of course, it does mean that the comments are spread throughout the code, which makes them difficult to browse. And that is the clever half of phpDocumentor. You can use it to scan the whole code, and automatically build a documentation web site like this one: http://phpdocs.moodle.org/ (which currently seems to be broken. Drat!. MDLSITE-153).

(So Pierre, my concern was that what you had started doing was to emulate phpDocumentor manually in the wiki.)

This level of detail answers the questions: 'How do I use the function? What data do I need to pass it? What will I get back?'


3. An overview of the whole system

This is the part that belongs on docs.moodle.org, and which is currently missing or badly out of date.

This gives a brief summary of the whole system, so you have some idea of where to start looking at the details. It explains some of the higher level concepts that are used, like database structure or in-memory structures, and any conventions or organising principles. It should also include getting started guides, or howtos.

There is a bit of this so far, but a lot of it is out of date or incomplete, and there are some big gaps. However, even though it is out of date, it is still suprisingly useful because it describes the general ideas on which the code is build, and these change more slowly than the details of the code.

http://docs.moodle.org/en/Development:Question_engine

This is the most general overview. I have not even read it recently, so I don't know how good or up-to-date it is.

http://docs.moodle.org/en/Development:Quiz_database_structure

The diagram a the top is up to date and quite useful. The text underneath is horribly dated, but is still useful if you can relate the old table names to the new ones in the diagram.

http://docs.moodle.org/en/Development:Quiz_state_diagrams

Another diagram that I hope is useful, but it really needs to be explained.

http://docs.moodle.org/en/Development:Question type plugin how to

A good start, but only about half written.


I think that the main missing pieces are:

A. A page describing the main in-memory structures that are used throughout the question-bank code. That is, the two different ways that $question objects are stored in memory, and the $session, $state and $cmoptions objects.

B. A page describing what an activity module needs to do to use the question types. That is, what functions it needs to call when. I am not sure anyone really knows. All the knowledge is implicit in the current quiz code. When Jamie and Peter write the question creation as a studnet activity module, they will have to work it out, and if we are nice to them, they may be able to add it to the docs.

C. A summary list of all the question type methods and roughtly what they are for, with links to the documentation on phpdocs.moodle.org.


This is just what I think, but I hope you agree that there is some logic to it, and that is a good approach to making it as easy as possible for people to work with the code.

Another key requirement for documentation, I think, is for it to be as short as possible. I don't at all mean that we should leave pieces out. What I mean is that if the code is well structured, it should be possible for people to use and understand it without having to read pages of documentation first. It should be enough to read about a few key concepts that have been explained clearly and succinctly, and they be ready to go. Also, the less text it takes to explain everything, the less there is to keep up to date.



[P.S. I used my powers as forum moderator to remove a message from me to Pierre and his reply, which I only posted here because emails were not getting through. I hope that is OK. Then I renamed this thread to better represent the substance of the discussion.

I'm afraid Pierre that the alternative email address you gave me also bounced. Do you think UQAM has put the OU on some sort of black-list?]

In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -

Thanks to having put this a specific post.

Thanks for your comments and guidelines.

I known about the phpDocumentor but also of its state as the fact that it is not build often.

There is a need to a more centralize place where we can comment on the code and its improvment. The forum or the tracker do not allow structured comments.

We could put here the technical discussions for a question like:

Should we extract from save_question the lines doing validation of the question general parameters and put them in a separate validate_question_general parameters() usable by either the new HTML QuickForm and the format->import_process()? 

Doing this questiontype docs implied that the infos should be update regularly. As I am planing to work on Moodle for some years ahead, I could assure regular updates for this part of the docs.

Doing this work we will ask ourselves how to  improve class coding in Moodle question code.

Am I right if I say that in the old ( pre C++) coding the individual libraries where used to put code specific functions in a place that minimize names clashs and that in good class code the libraries should be minimal, all functions being accessed by a call to a class function?

If so, we could start someday a debate of how to improve class code of questiontypes.

Pierre

P.S.I will set another email (hotmail...) tonight.

In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -

Two other questions

1-Could I help somewhere so that the phpDocumentor works correctly and with a more useful update schedule ?

2-As PHP is an interpreted language unless you use PHP accelerators, how about the performance penalty when comments are added to the code?

Actual computers are very speedy, active memory are large but neverthless the files to read are larger and disk speed has not grown as fast as the other two parameters.

Pierre

In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
1. I filed an issue MDLSITE-153 about phpdocs.moodle.org being broken, and Martin has fixed it already.

2. Loading code with comments and stripping them out is a pretty negigible amount of processing. All modern operating systems cache recently accessed files in memory, and the comments will be stripped in the first parsing pass through each PHP file which is much less work than actually interpreting the code, or opening a connection to the database and getting data back. So, no excuse not to write well commented code கண்சிமிட்டு


I still owe you a full reply about issues of how to organise the validation code. Not tonight, I am tired. I genuinely don't know what the best answer is. I just know that:

A. It is not good object oriented design just to shove everything into a single class, but I don't know whether any of the standard design patters can be applied to this situation; and

B. This is not just an abstract problem in object oriented design, because we are also trying to fit in with the rest of Moodle, which consistently (now) puts validation code in the edititng form definition class.

C. Practically, it does not help if the questiontype.php files get too large. It makes them a pain to work with. If there is a logical way to break stuff up into more small files, that might be good. (But it is just a pain if stuff is subdivided arbitrarily and you don't know where to find things.)
In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -

Hi Tim,

I just look to the phpDocumentor and for obscures reasons although we do a lot of work on itsmile you have to be an expert to know where the questiontypes are.

Question gives you

question

Questions give you

questions

phpDocumentor v 1.3.1

finally I trace them  under quiz

quiz

I will continue to explore how we can get coherent Development/question docs.

Pierre

In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
The reason that the question types are put in the wrong place is because of 'packages'.

PHPdocumentor uses a lot of the same principles of JavaDoc. In Java, packages are an imortant concept and affect how the code works. They are also used by JavaDoc to organise the documentation.

PHP does not use packages at all, but PHPdocumentor thought they would still be useful for organising the docs. But since the package declaration has no effect on the code, it is easy no to notice when it is wrong.

Anyway, the problem is the comment

* @package quiz

at the top of question/type/questiontype.php.

I'll take a quick pass through the code some time today, and ensure that all the quiz and question code is at least in the right package, as a first step towards making the PHPdocs more useful.
In reply to Tim Hunt

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
OK, I have got through the files in the top level question folder. Now I just need to do all the question types and import/export formats, but I need to go now I'll do them later.

The packages I am using are described here: http://docs.moodle.org/en/Development:Question_engine#Code_documentation
In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -
In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
Great thanks. I see you have done the question types already.

At the moment I am really grateful for anything that makes my todo list shorter, it is rediculously long.
In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -

Job done , question types and formats.

However the actual Documentor version is 1.8 , do we have to merge all the modified Head files ?

Pierre

In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
I tried to do a merge, and it did not work automatically because of the stupid $Id$ tags. There were so many files involved that I decided it was not worth merging my hand. With developer documentation, I don't think we need to worry too much about old versions. After all, new development should happen on head.
In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -
However,how do we access to phpDocs for Moodle_Head.
I have set a phpDocs on my server to check if I include correctly the files.
In a first test I just look at the question directory.
I put the address in Skype although this is a first trial.
Pierre
P.S.I will use it to test my improvments on phpDocs before CVS.

In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
I agree we need phpdocs for head. I am discussing it in the general developers forum: http://moodle.org/mod/forum/discuss.php?d=67175
In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -

Temporarely I have put my first trial on the dept server.

http://www.chimie.uqam.ca/phpdocs/

In what package do you want to place the edit_qtype_forms, they are in package questions actually?

Pierre

P.S. I have to improve the filtering to exclude inappropriate files ( and surely change the colors!)

In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
edit_qtype_forms are part of the question types.

I think PHP documentor is themable. phpdocs.moodle.org looks much nicer. Yes, you can download various templates here: http://manual.phpdoc.org/
In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -
However the form are an extension of moodleform, the others being extension of question_type.
class question_edit_form extends moodleform {

Could it be better to set them
* @package questionbank
* @subpackage question_edit_forms

or something else
Pierre
P.S. I have changed the display (not yet transfered to ww.chimie.uqam.ca)
In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
No. There is no real corresponance between inheritance and packages.

If there was, wouldn't it be logical for all classes to be in the same package as stdClass கண்சிமிட்டு

The the editing form is logically part of the questiontype.
In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -
Job done for edit_qtype_forms.
A partial phpDocs with question, quiz, lesson and HTMLQuickform has been put on
http://www.chimie.uqam.ca/phpdocs/

This can serve as a start of how to use phpDocs to document the questions.
So what's the next step?
Pierre

In reply to Pierre Pichet

Re: What developer documentation do we need?

by Tim Hunt -
Core developers இன் படம் Documentation writers இன் படம் Particularly helpful Moodlers இன் படம் Peer reviewers இன் படம் Plugin developers இன் படம்
I think you should just keep working on improving the calculated question. Then whenever you:

1. Work out something about how the code works that is not properly explained in the comments, fix that.

2. When you get stuck becuase you don't understand it, and the docs you need are missing, post here asking to prompt someone else to write the appropriate comments.

3. Go through the calculated question type, which you probably know better than anyone else at the moment, and systematically comment everything that you understant.

However, be very careful not to comment something you don't fully understand. A comment that does not mach the code can be much more misleading that no comment at all.

Also, remember that while comments need to explain how the code works, they should do so as succinctly as possible, becuase people want to work on the code without having to read a novel first. For most of the code in Moodle, it should be clear how it works just from looking at it.

In reply to Tim Hunt

Re: What developer documentation do we need?

by Pierre Pichet -
Ok back to calculated question.
I will left the phpDocs done on http://www.chimie.uqam.ca/phpdocs/
until there is a moodle one.
Doing my work on calculated, I will add on the wiki a least flow charts showing
  • the three steps creation and edition process
  • the steps to build a question for a quiz and to get the correct answer.
Pierre