Hello Moodle + Chrome Users,
I would like to invite you to have a look at a prototype English language learning activity that Frankie Kam and I have been developing this past year. It is a system for making speech recognition enabled chatbots. Unfortunately, you must use Chrome or Chrome Portable to make them work.
If you would like to experience making one yourself, go here:
Here is a webcam version: http://chatbotmaker.com/videofiles/04_vertical/mp4/
Again, to experience making one yourself, go here: http://chatbotmaker.com/video_recorder/record_your_answers08.html
What I am looking for are ESL-Moodle teachers who would like to either use one of these activities with their students, or collaborate in the development of a different speaking task with different questions. I am just now beginning to use these tasks with my college students, and I am interested to learn how another teacher might use them.
Unfortunately, I don't have an automatic account creation process in place yet. It is coming. Creating the individual chatbot pages for students therefore requires some hand coding, so I would like to collaborate with an adventurous soul (or two but not more) who is interested in trying this out with a group of students. What I hope to learn from you would be the the pedagogy/andragogy you bring to the task so that students come away having learned some English--nothing too elaborate.
This is not a Moodle plug-in, but it should work well with a Forum, Peer-Review or a Reflection activity.
I'm interested in learning more about what you're doing with Chatbotmaker.com . A platform for creation of video based chatbots is awesome, especially for ESL. FYI, I couldn't find anything about you or your projects on either Twitter or YouTube. I've also tried connecting with you on LinkedIn. You don't blog about your development?
Cheers, Marcus Endicott http://twitter.com/mendicot
Thanks for your interest. Yes, it is true. I have not been very actively promoting this project. Creating individual accounts "by hand" is time consuming. Once the automatic account creation script is in place, I will feel more confident about shouting about it from the rooftops. I have a someone on it, but alas I am low on his list of priorities.
Frankie Kam, a certifiable man of action and the technological brains behind the guts of the project, has bogged about it briefly here: http://moodurian.blogspot.ca/2012/06/create-your-own-chatbot-great-for.html
but, in truth, the pedagogy (the learning, teaching, and assessment parts) of the project needs to be more fully developed. In one sense, a chatbot is an ePortfolio of oral answers to questions that can be accessed in a non-linear conversational way. What conversations, therefore, do we ask our students to learn, and to what end? I have used a video chatbot to help train nurses to take a medical history from a virtual diabetic, but I am interested to know what other goal oriented conversations would be useful to teachers and students. With Chatbotmaker.com, the learner becomes the interviewee. The chatbot is a passive conversational partner, which makes sense where the power dynamic is unequal as with medical professional and patient. But what other contexts call for such a passive conversation partner?
Also, teachers and students still have to catch up with the technology. Not all colleges have Chrome and technical problems in college multimedia labs with Java abound. Perhaps a switch to Flash will necessary. (Matt Bury, please, weigh in.) Flash will no doubt offer other challenges.
If you have any questions about the project, let me know. I will happily answer your questions here or by email (email@example.com).
I noticed that we've both written unerringly similar blog posts on using Moodle's Glossary module for written corrective feedback:
- Yours (in 2010): http://www.profweb.qc.ca/en/publications/stories/moodle-correction-perfection/index.html
- Mine (in 2012): http://blog.matbury.com/2012/03/01/using-moodle-for-written-corrective-feedback/
I guess I'm behind the times!
I'm intrigued by your idea. I've tried the demos with Chrome but I'm not sure if they're working correctly or whether I've misunderstood what they're supposed to do. From a learner's point of view in a scenario, what is supposed to happen?
On the technical side:
The fact that it requires Google Chrome to work tells me that there's some kind of "Google Gears" (proprietary plugin) type API stuff going on. Google still uses Flash for many of its development projects (Flash cookies for user tracking, Street View, etc.) but it's dropping support for any public Flash APIs (I used to use Google Maps Flash API but not any more). If a Flash version of this comes out, I think it would have to be supported/developed by Google or a public API be made available.
Flash natively records mic input in WAV format (Speex CODEC) which is rather bulky. I've experimented with a mic recorder client that integrates with Moodle 1.9 without the need for a media server. The upload speeds for WAV data are too slow for anything more than a few seconds of recording. I've experimented with WAV to MP3 encoders in Flash but since Flash Player 11.3 and under don't support multithreading, it looks like the app have crashed while it's doing the encoding - not really practical. Flash Player 11.4 will support multithreading so that could be worth investigating further.
There's a big advantage in using the SWF Activity Module for this kind of thing. Apart from integrating seamlessly with Moodle's back end, it can also upload data as ByteArrays which get written as files on moodledata (Only in Moodle 1.9, not in the new 2.x file manager. The workaround is to use a directory outside of moodledata) which gets around any server limits on upload sizes and related memory limits.
On the language side:
I can see such an activity being used for preparation for the TOEFL speaking test. I remember that Auralog's Talk to Me and Tell Me More, and Google Venture's EnglishCentral.com have speech recognition software built in. They use it to grade learners' pronunciation and Auralog have some rudimentary dialogue activities. I've watched learners try dialogues with computers that used speech recognition (Tell Me More) over the course of a couple of academic years and I have to say I wasn't impressed. The learners weren't either.
From a cognitive-functional linguistics / usage-based linguistics perspective, while I think speech recognition has some potential for language assessment I don't think it would be terribly productive for aiding language acquisition by simulating conversation. Currently (if ever?), computers can't understand or interpret intent (agency) and therefore can't make sense of learners' utterances. I think it would be a kind of Turing test simulator: http://en.wikipedia.org/wiki/Turing_test. I remember a chat I had with with Matias Schulze (Artificial Intelligence) from the University of Waterloo about this kind of thing and he reckons we've got a long way to go before we can make meaningful use of AI for language acquisition.
A conversation with a computer would require both the human and the computer to memorise the same predetermined list of questions/prompts and acceptable answers/responses within predetermined parameters.
If we used speech to text and ran it through a grammar checker, we might get some interesting results. Although, it is possible to construct grammatically correct sentences that carry no coherent meaning.
Perhaps we could use it for some kind of adaptive testing?
I was hoping to catch your interest. I am a fan of your blog and have read your bit on the glossaries for corrective feedback. It was Joseph Rezeau who first described using the Glossary for corrective feedback here on Moodle.org in 2005. I developed quite an extensive glossary module for that purpose but have since given up and have developed a site dedicated to corrective feedback on L2 writing errors here: http://virtualwritingtutor.com . It is a work in progress, but it was your summary of Rod Ellis's work that inspired me to go beyond the glossary module and try something more adventurous. Frankie Kam has done an interesting hack with it here: http://moodurian.blogspot.ca/2012/08/extend-functionality-of-ouwiki-by.html Ah, Frankie! What a wonderful person to know!
Okay, as for artificial intelligence chatbots, I have tried them and I find them annoying. Shulze is right. As for intelligent people, I have tried them and I find them fascinating. So for me, a worthy development goal is to capture the charm and beauty of people by inviting them to create a chatbot of themselves.
So to explain, the system I am presenting here has two components: 1) a speech recognition enabled video or audio chatbot to practice a conversation with, and 2) a chatbot maker component for creating a chatbot for others to have a conversation with. For language learners, both could be interesting forms of speaking practice. Interviewing a chatbot for the person's phone number and other details is an interactive listening activity. You have to ask for the number and address to hear them. There should also be an learning advantage to having a tireless conversation partner to interview online. Medical interviews are very formulaic. Many other work-related and counter-service work conversations are very formulaic also.
Creating a chatbot is a different language learning experience entirely. It is more like creating an eportfolio of answers to standard questions. The fact that students can then interact with their peers' chatbots after they have been created promises to be an interesting opportunity for peer-talk and peer-evaluation.
You raise the question of open dialogue systems where a user must formulate a question that the chatbot knows. To create such a system, you must anticipate all the questions that a user might ask and have videos at the ready for each one. A closed dialogue system simply feeds the user the lines to say. Other than having access to the questions beforehand, the two system types are identical. My design provides the questions using a slider with clickable sentences linked to audio models. It works well-enough for langauge learning. Combining an open dialogue system with a grammar checker to improve the system's "understanding" is a very interesting idea. Something to think about.
With Google's implementation of x-webkit-speech in HTML5, enabling speech recognition on a webpage is as easy as putting in a textbox. If you know how to play a different video or audio file depending on what is submitted to the texbox, then you have a chatbot of a real person with real intelligence and charm. Only the conversation part is simulated.
The problem with recognizing speech using Google's speech recognition service is that it is designed to recognize Standard L1 American speech. Longer sentences cause the system to hang and accents can be a problem. Adding an additional training phase with a speech recognizer that can score pronunciation before the chatbot conversation would be great. I may have to switch from Google to a different system altogether. I am looking into finding a speech recognition service I can afford. For now, it is Google's free service.
Thanks for the information about Flash and its WAV (SPEEX) format. Right now, I use a Java video recorder and Java MP3 recorder. When students use them from home, they sometimes dismiss the request to run the applet instead of accepting, and then of course the Java aplets don't work. Students send me mystified emails. Students seem to be more familiar with Flash, so they are more likely to accept. Is there no third option?
The other reason why chatbotmaker is Chrome only is that my Java video recorder records MP4s, a problem for some browsers. So for now, its Chrome for the free version, and I'll have to come up with a pro version with cross-browser compatibilty.
Hi Nick Ever played Secret of Monkey Island or some other LucasArts early 90s game?
Well those text-graphic adventure games had multiple possible responses to the user's choice of acion. So just thinking that if there were a mechanism within chatbot to randomly choose a response to the question, say, "Are you married?", that would really make for N number of permutations of total conversations. So and answer to the "Are you married" question could be: "No, I'm not married. Who is asking?", "I'm sorry, I'm already taken" or "Yes, I'm married to a wonderful guy" or "I'm sorry that's too personal a question. Next question please!". That will gamify the whole experience and the user might want to go through multiple times of asking all 60 questions in order to get, what, a subset of the complete 200+ different conversation-sessions.
Of course the maker part of the system would have to allow multiple response recordings to one question and somehow be able to link the possible responses to a question asked. Not a trivial solution, programming wise. But possible.
Heh. The influence of Lucas Games no doubt. By the way I never did finish any of the Monkey Island games. I just Googled the walkthroughs and enjoyed the walkthrough solutions.
Branching scenarios! I remember the old paper-based adventure books which eventually got transfered onto early computers and started the RPG industry.
Hayo Reinders works with CALL and has worked with Rod Ellis among other notable researchers, who was originally his Ph.D. supervisor. He has some interesting observations: http://innovationinteaching.org/call/introduction/