Learning standards: teacher/district accountability: indexing Moodle course items to standards

Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Stuart Douglas -
Number of replies: 30
If this has been discussed before, I couldn't find the topic so please point me to it. If not, has anyone tackled incorporating a method for linking Moodle course items (activities, quizzes, etc.) to any governmenatl education standards?

Here in the US, thanks to our fearful leader, public education is becoming more of an excercise in documentation than teaching. Everything has to be linkable to one or more standards established by each state's Dept. of Education. This was one of the first question my wife (Special Ed. teacher) raised when i started talking to her about Moodle.

Off-hand, I don't see anything natively in Moodle that would easily support this, but you all are the experts. I could see manually adding additional tables to the Moodle database along with the necessary functionality to view/select learning standards as a function of creating the various course items or as a bit of teacher admin after the fact.
Average of ratings: -
In reply to Stuart Douglas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
After visiting the US I've been thinking (again) about this recently. Other names are Curriculum Outcomes, or Learning Goals, or Competencies.

In the long run I would like to have:

  1. a database of these statements, categorised hierarchically and modifiable by the admin (import/export, too)
  2. each course can specify a number of these from the total list
  3. each activity can be linked to any number of these from the course list
  4. an overall report for teachers and students showing which ones are covered in the course and which ones are been completed and by who
In reply to Martin Dougiamas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Rob Johnson -
This would be a large task.  Each state has different standards, and they change over time.  I include a "For Teachers" section in the webquests I construct with Moodle, and one item I include is a page explaining "Standards Alignment."  Along with the California content standards (I teach in the LA area), I include the National Educational Technology Standards that my lessons align with.

Would it be nice to have links inside Moodle to the California Content Standards? Yes, but they would need to be maintained and current.
In reply to Rob Johnson

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Stuart Douglas -
Standards data freshness IS a big issue, but for Moodle in general it falls outside of the scope of the project other than to support the data elements and linkage/reporting functionality. it clearly would fall upon individual Moodle site operators/users to actually populate and maintain the data. This is where critical user mass has some value...if we as a group can convince our respective governing authorities to provide standards information in transportable database formats, then maintenance becomes a very simple task. Although this kind of establishment/user interaction isn't there currently, we should proceed with including this functionality in Moodle AND do the necessary correspondence to get the data structured and provided to suit our needs. These are governmental institutions after all...they work FOR us (supposedly).
Average of ratings: Useful (1)
In reply to Stuart Douglas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Ger Tielemans -
If Moodle could only deliver the schemes, so that we can fill in the blanks:
  • on top: a certificate (or global goals) catalogue, describing which combination of courses delivers which certificate. (All institute have to do this on their own)
  • then: a course catalogue, describing all the offered courses and which global competences are covered by which course (and on what level: insitutes in Europa need these formal descriptions to get acrreditated the next years on loacl and Europe level..
  • also a personal catalogue (part of the ePortfolio? where you can see which (certificate) goals the student did set and how far he is on this road *)
  • on course level: a table for each course with the included activities, how they are linked to the competences, and how they are graded.
    (All graded activities, yes quizzes too, get company of a grading-describing Rubrics scheme.THIS is vital for the system.. especially when you move to open assignments, ePortfolio..)
  • Additional you could create for each domain a kind of thematic catalogue, that shows how the others cover the basics stuff for that domain.
To make it less abstract for the student: I saw in NL a vocational training institute that uses a kind of visualisation.
  • For each student they have a standarized list - on paper - with short decriptions of the competences.
  • on the right side of each item you see a horizontal bar where the progress for that student is depicted.
  • then you can take the same scheme - not on paper but as a transparant, with the bars filled for a typical professional worker / profession
  • If you want to become a doctor, you take the transparant for a doctor and place that on top of your own paper sheet
  • In one glance you see which parts you have to develop before you can heaal your first patient smile

    (On paper, yes, possible, but I wonder how good Moodle could do this..) 
In reply to Martin Dougiamas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by A. T. Wyatt -
I am very interesting in this topic myself.  In the US, at least, federal and state education agencies are very interested in "standards based learning".  This doesn't stop with K-12 institutions (although they have mandated testing already).  It is also a very hot topic in higher education, particularly in the part of the US governed by the Southern Association of Colleges and Schools (SACS for short).  This organization accredits institutions (2 yr, 4 hr, and with graduate schools), and accreditation is of paramount importance.  They require all departments and programs to implement assessment plans in order to measure student outcomes.  I won't bore you with the details, but that is probably where much of the interest in electronic portfolios and such is coming from.  We don't have good tools or methods for assessment of fuzzy, complicated learning.

The best tool I have ever seen for this (and where Moodle might get some great ideas) is LiveText.  They are based in Chicago, and it is an electronic portfolio system with an added layer of database support for published national standards. Or, particularly powerful, they will allow an institution to add custom standards to this database.  After that, they work rather like the "scales" in Moodle.  When students upload the chosen artifact, the instructor grades it with a rubric that is already linked to the standards.  Power is added because multiple instructors can grade multiple artifacts all using this same rubric, and the results are aggregated and analyzed statistically to give you some real information about how your department or program (or even institution) is meeting key objectives.  And you measure these things by evaluating student outcomes (cognitive products), not the standard inputs such as hours sold, FTEs, etc.

Sorry if this is wordy.  I have spend a considerable amount of time on e-portfolio, assessment, standards, and how they can be used to improved student outcomes on my campus.  In the US, you can't get a Dept. of Ed grant if you do not have a very strong assessment plan, and they prefer experimental research design.  Even though that is quite difficult to do in an standard educational setting.  So I dream about this stuff!  I do this stuff!  And in the end, I have to say that I am a believer!

If you can, take a look at some e-portfolio systems.  They will likely have code that supports/approximates what you are trying to do.  And I cast my vote for such a wonderful ability for Moodle! 

atw
In reply to A. T. Wyatt

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
As you have experience, may I ask you to help educate us on what you think good e-portfolio systems should do?

For context, see also this discussion on Forums, Blogs and the nature of discussion.
In reply to Martin Dougiamas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Tom Murdock -
Imagine the glee of a principal who is able to pull all of the scores for graded activities attached to a given strand/goal.

This feature will also make Curriculum Mapping into a real-time event, rather than a double-entry exercise.
In reply to Martin Dougiamas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Art Lader -

This could be one of the most important, significant Moodle developments ever in my neck of the woods. Were we actually able to impliment this, I would have teachers lined up at my door every day to Moodlize their courses. I am sure that this is true.

Almost all of those teachers currently sitting on the fence would want to get in on this.

-- Art

In reply to Martin Dougiamas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by t h -
Being fairly new to Moodle (former Blackboard admin/user) I am impressed with its openness. It has re-invigorated some ideas I worked on a few years back towards developing a system for an electronic "Teacher Workflow" system (I spent a few years as private developer writing educational software for handhelds).

I spent some time anayzing the tasks a teacher does (the big picture) and sought to seek a way to integrate those into on electronic system. I wasn't (and still aren't) convinced that distance eLearning is all that relevant in K-12 environments, but I am certain that teachers need 21 century tools to manage their tasks. THe teacher workflow model was my (meager) attempt to shape this.

At the time I was trying to piece together disparate systems that my (large - 80000 students) school district was using and failed miserably (have retreated out of the Board office now and am happily a teaching vice principal at a large high school), but Moodle gives me an interesting new way to start to explore this again.

Anyway, back to this thread: Jurisdictional standards are a must, but don't need to be hammered into the system like some products do. Let the teacher opt-in to using them.

If anyone is interested in the workflow idea, or in pushing it forward a bit, I'd be happy to hear from you.

tim.hawes "at" ocdsb.ca

wfm

Forgive the unedited rambling of the attached document.
In reply to t h

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Colin McQueen -
An interesting diagram. Is it perhaps a bit USA biased? A major role ifor teachers in the uk is the assessment of coursework which is then moderated by exam boards. Also in the uk there is a pilot looking at on screen testing of student ICT capability via a simulated environment and staged tasks. If successful the goverment via QCA will be looking to nationally testing students for ICT capability in yr 9 (age 14?) in 2008 and reporting this as they already do for English, maths and science. This is quite a paradigm shift. Other subjects may well then follow. Recent uk government announcements have given a short term target of all learners in schools (teachers and students) having access to a learning platform (MLE/VLE) including a learning space/portfolio by 2008. This may have some bearing on your workflow diagram.
In reply to Colin McQueen

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by t h -
actually, its based on my experience in K-12 environments in Ottawa, Canada.

Learning standards (we tag them as "outcomes" or "Expectations") are set by the province, but are not tested on explicitly. They are there as guides to the curriculum. They are also useful when structuring resources (e.g. meta tagging learning objects for easily finding teaching/learning resources for a particular outcome)

./tim
In reply to Stuart Douglas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Bryan Williams -
Stuart,
Many content vendors handle this already by identfying, on a state-by-state basis, which standard of learning (SOL seems to be a popular term) that their courseware is designed to comply with. There are some patterns among states for K-12 standards that can be identified. Our approach in working with vendors to get their learning content within Moodle is to provide this information so that it may eventually become part of the "Settings" properties within a course. This would be very simple to do and perhaps we will see an additional field for this property appear in Moodle 1.6. This property could also appear within a new type of "Glossary" activity (resposity) which could be set up to share courses with teachers say within a state. The SOL value is where eventually a Moodle SIS will grab some of its information for accomplishing, among other things, some of what Tom mentions in his post. Your input would be very valuable at this time.
In reply to Bryan Williams

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Tom Murdock -
Yeah, at the Moodlemoot NE, Byfield, MA meeting we had a developer discussion about actually tagging individual activities. Mike Churchward has a hack for this already that grabs SOLS from a database.

I love this approach, but am, of course, a huge fan of simply using metadata for each of the activities. This way, an activity can be marked as a state standard, a national one, a subject-area one, or can simply be marked as "experiment" or "beandish" or some teacher-specific word that might be helpful in her later reporting or reflection on the year.

This metatagging approach (folksonomy) allows us to use full taxonomies and hiarchies as tags, but also allows us the freedom to change them as they change (which they do quite a bit).
In reply to Tom Murdock

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Bryan Williams -
Where do you see this metatagging being made? Is it possible that Settings would be an appropriate place to do this for a course, and that each individual activity have similar properties? I am probably mistaken, but thought this was a scheme considered some time back; that is, adding a separate (metatag-metadata) table that would collect this information that a future respository could call. 
In reply to Bryan Williams

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Tom Murdock -
Where do you see this metatagging being made?

Maybe on the page where the instance is created for any activity there would be a little metatag box.

Your memory is good. smile We'd talked about the settings screen as being a great place to mark the course for the repository.

This newish idea is more granular. It would allow teachers to tag the actual activities.

So, for example, maybe I had promised my department, principal, state, or country to do some oral presentations with my freshmen. Voila, when I place an off-line assignment in the course, I indicate that it fulfills the goal/strand by tagging it "oral presentation" or whatever other fancy name the powers-that-be call it. smile

I think teachers are almost always aware why they offer various activities. That kind of thinking usually happens during the creation of the event (thus during the creation of the activity, it would be a good time to tag). But many, many schools are doing things like curriculum mapping which asks us to reflect on the previous month and indicate what skills/resources/assessments/standards/and essential questions (or some combination of these elements). My first impulse is: been there, done that. Don't look back. But, of course, this isn't really a helpful way to report.

By tagging as we go, I see teachers saving on a reporting step. Also, it gives admins a powerful tool to make sure that all of the important strands were covered by various classes.

This also means that the course settings metatags might simply have a summary of the activity metatags... or we could use it as we had planned before...

Sorry for not being clear before.

In reply to Tom Murdock

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Ger Tielemans -

Maybe it would be wice to restrict the (forced) use of metadata...

Lets say for the situation when you cut a piece out of its context:

  • is it a course? tag the course?
  • is it a section? tag the section (not yet possible)
  • is it an activity? tag that..

Don't expect that normal teachers/professors will tag all these stupid singel parts: you will end up with lots of dots .. .

But please in all cases: do not invent your own tagging systems but follow the proopsals for standards from IMS and LOM.

In reply to Ger Tielemans

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Tom Murdock -
But please in all cases: do not invent your own tagging systems but follow the proopsals for standards from IMS and LOM.

This means that teachers would only tag activities in order to fulfill the expectations of some program czar...  If teachers can invent their own tagging systems, too/at-the-same-time, it would give them power and flexibility.

I don't know about you, but my desk, my desktop, and my file cabinets do not follow traditional taxonomies.  For a tool to be useful, people will need it to be flexible enough to handle their own quirks, I think. wink
In reply to Tom Murdock

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Ger Tielemans -
So you will organise meetings for your staff or coworkers to agree on the meaning of tags??? Knowing how time comsuming it is to simple translate the English words from the LOM into Dutch... well, I wish you luck..  
And yes, starting with the traditional taxonomies from the librarians liek Z3950 is a very clever time saving approach for domain classifications. (You only have to add and link your "new" competences to that taxonomy.)
My opinion: metadata is not for organising yourself but for communiaction with others in an attempt to talk about the same things you agree on metadata standards.
In reply to Tom Murdock

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Bryan Williams -
Yes, activity tagging has been the idea all along to make a respository something of real value (IMO). I'm inclined to think this level of more detailed tagging is going to be outside the province of many teachers/course creators, so the question comes up as to where and how this data is generated. The most obvious place would probably be in the activities properties when it is created. But we could work out an admin interface or a separate smart form that makes it easy for the course owner to generate the needed information. 
In reply to Bryan Williams

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Tom Murdock -
I've never known what to call that "properties page,' but I think we mean the same location when I said: where the instance is created for any activity there would be a little metatag box.

I think it could be placed rather inconspicuously so that "power-users" could access it.
In reply to Tom Murdock

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Ger Tielemans -
Tom, how many summaries filled your teachers on that activity creation page? Why then should they fill a complete set of metadata fields?
Only when they realise how useful summaries become for the student they start to fill in that field..
So the same for metadata: it only makes sense when you want to store a complete course or parts of it. A handy page where you can fill (or correct) the fields on that moment would be the nice.
Having both would be better: when you fill in the blanks during creation, you save time when you arrive on the export page: having the opportunity to refine the earlier filled fields would be elegant. 
In reply to Tom Murdock

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Hello -

I showed Tom a preliminary version of something I'm working on for a client in the UK. They have specific standard marking requirements that their assignments have to meet. In this case, there is a database of 'units'. Each unit has a number of possible learning objectives that have to be met. Each learning objective has three grading criteria (pass, merit, distinction).

For example:
Unit: The Music Industry
Learning Objective 1: Investigate the function of different areas of the music industry
Grading Criteria:
Pass: Identify the principal features of at least four areas of the music industry
Merit: Explain the functions of at least four areas of the music industry
Distinction: Comment critically on the operation of at least four areas of the music industry

There are 19 separate units, with a combined total of 80 Learning objectives.

For this development, I am creating a custom course format, several custom blocks, and a custom gradebook. I use a new course format so that I don't have to change the core code (as much as possible).

To do this, I have added an extra icon to the edit mode of valid activities in the course format. Clicking this icon allows you to assign learning objectives to the activity. Each association has a defined percentage for pass, merit and distinction. The numeric mark assigned to the activity determines the percentage and thus the associated grade given (fail, pass, merit, distinction). These association are kept in a separate data table.

The units, learning objectives and grade criteria can be configured to be part of the existing Moodle database, or pulled in from an external database. Using an external database means that those requirements can be managed separately, and automatically reflect in the Moodle app.

What would be better would be if we made this type of grading universally available to Moodle. Configuration would allow us to determine what grading system was being used. If it was part of the available core, it would allow us to put the grading association into each module screen as well as on the course screen.

mike
In reply to Mike Churchward

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Bernard Boucher -
Hi Mike,
great to hear that but sorry to missed that at the Moodlemoot NE!

Good idea to use a new course format with a new icon to assign learning objectives to a particular activity. That correspond to Tom metatagging before.

To approach the "indexing Moodle course items to standards" it should be interesting to be able at that phase to have also the choice to tag or to assign the activity to one of the standard or the competency of the particular cursus by selecting from a fixed list ( fixed by the district or the government ) . That way it will be possible start doing what Martin explain in his first answer of that thread.

At a lower level, with the future possibility of assigning many categories or tags ( including Bloom taxonomy ) to the same question ( quiz or lesson or questionaire or hotpot ) it will be possible to have great reporting feature like Ger dreamed before.

With all this tagging Moodle core will be ready for adaptive learning.

Happy to see that you are working on that,

Bernard

In reply to Bryan Williams

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Stuart Douglas -

I was wondering what had happened to this topic (sorry for starting it in the wrong forum, Martin).  smile

For Moodle, I think Martin has a good handle on the necessary data elements and functionality (at least from a US-centric perspective). I'd still like to see the standards bodies provide their data in database form, rather than strictly in document form. Similarly, if the content vendors could do the same with their offerings, Moodle could become a VERY powerful tool for school districts from an organizational/reporting perspective. Even if neither did this, Moodle having the proper data elements and linking functionality would at least empower individual sites to build and maintain that data themselves. It could even prompt someone to start a Moodle offering geared specifically as a central repository for such information (hmmm...).

Do you have any examples of the SOL linkages that content providers have? I'm very curious as to at what level the document this. Simply stating "our Science texts are geared towards the X state standards" doesn't do anyone much good, but having specific standards items indexed to individual text chapters, assignments, quizzes, problems, etc. would. This is what Moodle could be setup to provide as well...at least the buckets to put the data in and the means to connect them to the course items and subsequently report on them.

I could easily see Moodle evolving to include much education management/reporting functionality by going down this road, which would only enhance it's appeal within the industry. I'm glad to hear that much thought has already been applied in this direction.

In reply to Stuart Douglas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Bryan Williams -
I could easily see Moodle evolving to include much education management/reporting functionality by going down this road, which would only enhance it's appeal...

Stayed tuned to this channel!  smile
In reply to Bryan Williams

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Stuart Douglas -
Hi Bryan,

Just checking in to see if anything new or different is happening on this front.  If there's anything I can do to help...
In reply to Stuart Douglas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Art Lader -
http://education.smarttech.com/standards/

Very interesting site for USA standards.

-- Art
In reply to Art Lader

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Stuart Douglas -
Interesting indeed.  My wife is trying to get her district to fund some of that outfit's Smart Boards for she and the other Special Education teachers.  She didn't know about the standards search feature.

Tip-o-the-hat, Art!
In reply to Stuart Douglas

Re: Learning standards: teacher/district accountability: indexing Moodle course items to standards

by Art Lader -
> My wife is trying to get her district to fund some of that outfit's Smart Boards...

How did the funding effort go, Stuart? (I know a couple of teachers who are trying to do more or less the same thing.)

-- Art