What should we consider as the criteria for "success" of this working group? I've been thinking about this in the framework of learning analytics. As usual, I have broken this down into four models of learning: Academic, Pragmatic, Individualist, and Idealist. This involves an assumption that the purpose of this working group is some kind of learning, but I think this will make sense as I explore these four models further.
Academic - The goal of academic learning is to identify the best potential scholars and help them develop into professional-level scholarship. In the context of MLAWG, we are interested in identifying researchers (at any level) who work in the area of learning analytics, and helping them to conduct good research and publish the results. We would measure the success of this kind of learning by tracking how many journal articles, theses, etc. members of our community create and publish.
Pragmatic - The goal of pragmatic learning is to prepare as many participants as possible for participation in the "real world." In MLAWG, this could be participation as Moodle system administrators, teachers, students, developers, etc. Perhaps a measure of success might be how many participants implement learning analytics on their own sites, or develop new learning analytics models, but that would leave out teachers and students. We would hope that both teachers and students would become knowledgeable about the uses and limitations of learning analytics in general and within Moodle. We might develop a quiz for teachers or students that asks about how insights should be interpreted.
Individualistic - This model of learning is about helping participants identify and pursue their own goals. It is difficult to track "success" in this model, but one way is to ask participants when they join the group what their goals are and the timeline in which they hope to achieve those goals, and then ask again when that time has passed, to see if participants feel the working group has helped them on their journey. I have tried to implement these ideas in our online workshops-- enrolling in a workshop is considered interest in achieving a goal. However, this does not account for people who are interested in viewing the contents of the workshop, but don't actually intend to attempt the exercises. Importantly, this model of learning places the goals of the participants ahead of any goals Moodle as an organization might have. Moodle bears the responsibility to make potential participants aware of the working group, but we cannot judge the success of the working group on any particular standard of participation. (If no one chooses to participate in the working group at all, we might suppose that the working group is not meeting any needs, and is probably not worth the effort invested by Moodle HQ to operate.)
Idealistic - In this model of learning, there is a social good we hope the working group will empower participants to achieve. Moodle's mission statement is "Empowering educators to improve our world." In this case, participation is definitely a measure of success-- both within the working group and on production sites around the world. We believe that learning analytics can help improve the quality of learning (in all four of these modes!) and we want to help all Moodle users (students, teachers, site administrators, developers, etc.) to be able to make use of this tool. We should really measure success by attempting to measure increases in quality of education (as perceived by learners, teachers, and other members of their communities). I don't think we're quite ready to do that. But in the meantime, we should be asking participants whether they feel more empowered to improve educational quality by using learning analytics in their own sites. We should also critically examine the extent to which learning analytics can be used to help improve educational quality, e.g. as an aid to empowering teachers and learners, rather than merely as surveillance to enforce externally imposed "standards."
I think all four of these models of education are important and valid, and not necessarily in conflict. I think they are all relevant to this working group. However, we need to know how our participants feel.
What do you hope to gain from participating in this working group? What can we do to help you? And how do you think we should measure whether we have been successful?
We look forward to hearing from you!