Just a quick update, and then I'm also going to use this forum post as a TODO list for myself. We've been meeting with clients and end users about this over the past several weeks, and they've made some good adds from a functional perspective. They've also overwhelmingly confirmed the vision of removing the dependence on scales. Their suggestions will appear in my TODO list for updating the specification below. On our side, our technology team, which for this project will be headed by Kris Stokking and Mark Nielsen, has now reviewed the spec, and provided me with a lot of feedback (much of which will also appear in the TODO list below).
TODO List and Key Decisions:
- Course Mapping to Outcome Sets - We had a lot of discussion, both functionally, and technically about the importance of the mapping of a course to Outcome Set(s). From a functional perspective, this mapping has two purposes. First, it simplifies the mapping of activities, rubrics, and questions, by limiting down the number of Outcome Sets that I can select from. Second, the course level mapping defines which outcomes I want to report against. So, from a functional perspective, this is important. However, as I discussed with the tech guys, the mapping that is the most important is the mapping of the outcome against the content itself. Let me give an example to clarify: Let's say that there is a quiz question that is used in multiple courses through the question bank. That individual question might be mapped against two outcomes. However, for my course, the only outcome that I care about is the outcome that my course is mapped against. Why does this matter?
- Backup/Restore - If I restore a course that has items mapped against outcomes, the outcomes won't really appear in my reports, etc. unless we also map the course against that outcome.
- Shared Questions (see above)
- Accidental Deletion of Mapping - If I have mapped 1000 questions against my outcomes, then I accidently remove the Outcome Mapping at the course level, it should NOT delete all of my work in mapping the individual items.
- Report for "Unmapped Activites and Questions": Great customer suggestion here. If I am in a course that is using outcomes extensively, we should build a report that shows items that are not mapped against any of my associated outcomes. This would be like the coverage report but would be content centric, rather than outcome centric. The assumption is that if I am tying my course to outcomes, nearly every piece of content in the course should be mapped against an outcome, so if something is not, it probably should be.
- Detail on the Reporting - We need to go one level deeper on the reporting pages and define what happens when I click on the links in the summary reporting. One of the particular questions that came up was how easy it would be to gather the artifacts of student submissions.
- Use Case Add: Export Outcomes data through API to Portfolio System (such as efolio or Mahara)
- Outcomes Summary/Workflow Block for Teachers - An idea from a client is to create a block for the course home page that gives updates on outcomes workflow. More definition needed, but wanted to document the idea.
- Define XPath options on import as only non-complex elements.
- Clarify how "Average Grade" works for quizzes where questions can have different weight.
- Define Capabilities (Create Outcomes, Import Outcomes, Map Course, Unmap Course, Map Activities, Map Rubrics, Map Questions)
- Backup Restore Specification (include Common Cartridge)
- More work on Recommendation Engine - Users loved this concept BTW. One of the biggest problems was the sheer volume of work created by outcomes. Anything we can do to make that easier helps.
- Outcome In Use - Warn before editing
- Versioning of Outcomes - We know that this is a major use case, but we can't bite off everything this time. Let's make sure not to design ourselves into a corner on this from a technical perspective.
- Can instructors see how students did on the same outcome in a different course? Is this a setting?