NY Time article: To Really Learn, Quit Studying and Take a Test

NY Time article: To Really Learn, Quit Studying and Take a Test

by Tim Hunt -
Number of replies: 13
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

http://www.nytimes.com/2011/01/21/science/21memory.html?_r=2&ref=general&src=me&pagewanted=all

"The research, published online Thursday in the journal Science, found that students who read a passage, then took a test asking them to recall what they had read, retained about 50 percent more of the information a week later than students who used two other methods."

Sadly, I can't seem to get the full text of the original research article (even through the OU library) however, a quick google scholar search found two other pieces of research along the same lines:

http://duke.edu/~ab259/pubs/Butler(2010).pdf

http://uweb.cas.usf.edu/~drohrer/pdfs/Rohrer_et_al_2010JEPLMC.pdf

However, before we get too excited about this, I should note that if you look at the details, the kind of tests they are asking students to perform don't look much like Moodle quizzes.

Average of ratings: -
In reply to Tim Hunt

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Tim,

Thanks for pointing to those interesting pieces of research. I quote from Butler (2010):

Subjects studied passages about a variety of topics, and then they repeatedly restudied some passages and repeatedly took a test on other passages. The series of experiments was designed to explore progressively greater degrees of transfer. In Experiment 1a, the final test consisted of repeated questions (i.e., a verbatim representation of the questions that had been on the initial tests) to demonstrate that testing improves  retention of information relative to restudying the passages.

One problem is that the author does not seem to have defined the activity of "studying" in his article (my italics in the above quotation). To me, active studying is much more than just reading (text) or looking at (images, etc.). Active studying involves mental self-testing. Of course, actual testing (i.e. answering questions asked in a formal test) may contribute towards learning, but we know that there can be studying without learning (as well as learning without studying - incidental learning).

Joseph

PS.- ... and there can be teaching without learning.wink

In reply to Joseph Rézeau

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Well, studying is a skill, and therefore has to be learned. As an educator, you probably have very high study skills, and so you naturally engage deeply with anything you are given to read that you want to retain. Less experienced students don't know how to do that. So it seems reasonable to me that if the teacher prompts the student with some specific questions, then that is helpful.

Also, it seems consistent with constructivism that having the students actually write their answers down leads to a more powerful effect that just asking them to think about it. (It also takes more time.)

In an ideal world, the teacher will explain to the students what is going on, and encourage them to gradually become less dependent on the teacher's prompt question, and a more independent student.

I have to say that the Open University is very good at this. At least on the small sample of courses I have studied, the reading material contains good prompts to stop and reflect, to think about how the theory relates to your practice, without being too patronising. (And, indeed, to write down your answers to these questions, though I rarely bother. Oops) That is a result of having to produce materials that students will mostly study on their own, I suppose.

In reply to Tim Hunt

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Pierre Pichet -

I am pleased to see this kind of report which confirms my old practice ( back to 1970 ...) to give calculated questions on-line quiz ( using teletypes and FORTRAN...) .

Students have always better results in exams on questions they have already solved on-line.

 

Pierre

In reply to Pierre Pichet

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Colin Fraser -
Picture of Documentation writers Picture of Testers

But what is being tested? Learning or understanding or memory? Can students synthesize what they have learned and place it into another context? That is understanding. Do they simply regurgitate it all? That is memory. What is the more valuable? To a scholar the synthesis, to someone else, more likely, the ability to remember not the materials learned, but anything they need. I read one article in our local daily, (a Murdoch rag, but let's not go there), that seems to be commenting on the fact that 2 to 5 year olds were better at turning a computer on than they were tying their shoe laces. The world changes, and we have to change with it, our only problem is what is it the world is changing to?

In medieval times it was treason to kill the King's Messenger. The messenger was tranporting verbal messages to his lords and that included inflection and every nuance. There would be no misunderstanding the words, the mood, the nature of the message. Since the widespread introduction of literacy, there was no need to listen properly, to repeat exactly, it is now in writing. So Lord Lucan gives Nolan a verbal message, Nolan tells Cardigan who misinterprets it and leads the Light Brigade into the Charge. We no longer need to listen as carefully or replicate as closely, we have paper.

In our context, do we need to remember things? I suggest it is critical we do, for a whole range of reasons, too many to go into here. This generation of computer literates do not think so. They will always have their cell phones their iPads, they will always be able to connect to the Internet, they have no need to remember things. That is why these studies are a waste of time - they prove nothing. We know people can remember things, we also know we are not encouraging people to do something they really need to do. Quizzes are fine, but it is retention that is critical.

One similar article, years back now, found that doctors could pass their finals but if they retook them a couple of years later, they would fail. So how much does context of learning impinge itself? A quiz is an indicator, and that is all, but the real testing comes in the application of knowledge and I suggest that we cannot substitute any quiz for actually consistently demonstrating understanding. So use quizzes, but don't forget to constantly review them.

In reply to Colin Fraser

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Pierre Pichet -

Having learned lot of essential knowledges before the computers were no more accessible than punch cards, the actual access on-line to encycopledia modify what whe should learn i.e remember in our memory.

I don't know the answer but definitevely it is not the old way.

For example, in our history courses we had to know the dates of "important events" (i.e end of Rome Empire).

We are now more focused on why Roman Empire ended.

This is a progress.

 

Pierre

 

P.S. To answer calculated questions you need to know HOW to calculate the response,

this is more than memorize the date of the Cressy battle even if this last one is considered by many historians as the beginning of the end of classic chivalry.

In reply to Colin Fraser

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Itamar Tzadok -
A quiz is an indicator, and that is all, but the real testing comes in the application of knowledge and I suggest that we cannot substitute any quiz for actually consistently demonstrating understanding.

Or rather the real application comes in the testing of knowledge ...

Perhaps we cannot substitute any quis for actually consistently demonstrating understanding, but the real problem is that we cannot actually consistently test the demonstration of understanding. So we should use quizzes for what they test and leave the rest to the workings of the real world. smile

Average of ratings: Useful (1)
In reply to Itamar Tzadok

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Hi Pierre,
Just to throw a spanner in the works... I suspect we are of an age where the slide rule was very important, and as the SR-71 Blackbird was built using one, what a valuable tool it really was. But I am coming to the conclusion that much of what we needed, we thought of as important or what we think is important, is really not. I had one student tell me that "Nothing is really important, and if it was it probably would not matter anyway." Ha!

P.S. To answer calculated questions you need to know HOW to calculate the response,

Why do we need to know? We certainly need to know how to read and write and count, and I think we should be able to do mental arithmetic, but do we need to calculate manually? I am not so sure. If we understand how an equation works, and we have a calculator or a computer, or even a cell phone, do we need to do the actual calculations by hand? I am sot so sure.We have students who can do some whiz bang things in World of Warcraft, or whatever the latest game is, they manage entire armies, and build cities, they make incredible manouveres between levels in their games, but we ask them to sit down and eschew their computers, something they are good at, and use a pencil to calculate things? No wonder we cannot get students to take up maths.

We have to be able to work out, in our heads, that if I can buy a can of beans for $1, and yet can buy 3 cans for $2.50, there is a saving for me. That is not what we are teaching btw, I suggest we should, but somehow we consider it less important than it was. For some reason, it seems using our brains is given over to allowing the ubiquitous beige box to do it for us. It appears we do not need to remember things any more, we have the computer to do it for us.

Gutenberg developed his printing techniques and press in the 1430's, yet it was not until Erasmus, 70 years later, that the question of whether we were using it correctly was asked. It was another 150 odd years for Locke and, later,  Bentham to propose that universal literacy was not an unachievable goal with the printed word. Yet it was nearly another century, and an Industrial Revolution, before people worked out that it was, indeed possible. We do not have this kind of time line with the computer, we need to make it work now. When TV was introduced, it was touted as an educational tool, but look how quickly I love Lucy took over. (I even remember the ABC here putting AJP Taylor, I think it was, on talking about history, nothing else, just talking, delivering a lecture if you like. It was fascinating, even if I had no idea what he was talking about.) Today, we do not use a lot of TV in the classroom, I don't anyway, just for some strongly visual elements, but now I am more likely to use a data projector. 

I do not propose we drop computers from the education process, but I do suspect we need to target our use of them better, we certainly need better tools, and Moodle goes a long way towards that end. We should not dismiss the "old ways" because they are old, but because they are not valid. Rote learning is not practiced in our schools, but I wonder what the ratio of ADHD and Asperger suffers are in those countries that still use it compared to ours. We complain about kids not "knowing" their times tables - yet what do we do about that? Reintroduce mental arithmetic based on rote learning, no, not at all we seem to ignore it all and just continue on with practices that have demonstrable flaws.  

As an aside, people throw words like "constructivism" around and they have this clear picture of what it is. Almost inevitably, they miss one vital component of the concept, that is it directs children's attention along particular lines. This is OK for training purposes, but for an education? I am not so sure. If you direct attention into particular channels, which is what we do do, are other channels then invalidated? Do we in fact, as Ken Robinson suggests, kill initiative and imagination by adopting these philosophies or pursuing these lines of education? I do not know, but I do know we need to do it better than we are.

So Itamar, what is it then that we are actually testing? Understanding or memory? big grin

In reply to Colin Fraser

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Itamar Tzadok -

Neither, at least until we know exactly what actual processes the terms denote and how to measure these processes.

Some may say that we at least have good understanding of what memory is, but then again, if we don't have good understanding of what understanding is, how good can our understanding of memory really be?

To say that an application of knowledge is a demonstration of understanding is conveniently vague. If someone offers you an analogy he/she may be said to apply knowledge in the sense that a certain pattern is taken from one context and applied in another. But does that indicate to you that this person demonstrates understanding? In general you would probably require that the suggested analogy is a true analogy. But who's to say that the analogy is true or false? What if the evaluator just fails to see the connection because of, say, lack of understanding of the particular perspective from which the analogy makes sense?

I can tell you what I test, though. I test the ability to solve a given problem which has a definite solution, by a given set of definite techniques. I don't care if the person who takes the test solves the problem by means of memory or understanding or any combination of these faculties because I don't know how to assess that. smile

In reply to Itamar Tzadok

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Yes, Tim, and I interpreted Gardner's reponse in a way that is incredibly politically incorrect here in my system. I am suggesting that "progress" is not always beneficial, and "progressive educators" are not always right and I am sure that such heretical views will get me burnt at the stake. Underpinning that perspective is what I have put up here - which is more a missive to myself to get others commenting to help me gain a better understanding of the way in which the education processes can work. Look at Itamar's comments, what  great comments, they outline an essential dilemma of education in a structure that is easily understood, and something I can use in discussion elsewhere. They ask questions of me, and anyone else who cares to read them. Where else am I going to get such questions? They outline clearly stuff I have been trying to put into a context that is understandable.

This is how I learn, my learning style, and everything here, everything I put up in the Moodle Docs, everything in the forums is about me learning something. I say things, sometimes they are said badly, sometimes they are wrong, but I learn from those mistakes, and I do not have a single quiz in sight smile Check your comments on the Beginning Moodle 2.0 Administration page, (I moved them btw into the article's comments page). You were right, they were badly said,  and I understood what I meant, but they were not clear nor obvious, so they were changed.

In this context, I find that all testing would be doing is testing my memory, not my understanding. I always found tests at school easy, and would do no preperation work, no swatting, and the only time I failed a test was when I missed school because of childhood illness. So what was being tested? This does not dismiss Gardner's multiple intelligences, but it tries to place them into a better context, for me.

Getting long again.. sorry tongueout

In reply to Colin Fraser

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Oh, I was also kind of fooled by the tag line "Test-Taking Cements Knowledge better than studying..."  I doubted it... smile

In reply to Colin Fraser

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Colin, did you actually read the article, it seemed like a reasonably good summary of the research, and they seem to have considered those points.

In reply to Tim Hunt

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Yes, Tim, again, and what was new about it? I too have not seen the full Science article by Karpike and Blunt, so my comments are restricted to the NY Times piece and the abstract.

The basic questions remain unanswered in the story- mentioned but dismissed, ignored like they are uncomfortable warts. Acknowledging things does not make an appropriate analysis. What is being tested? The abstract is even worse, it says nothing much at all. "Our findings support the theory that retrieval practice enhances learning by retrieval-specific mechanisms rather than by elaborative study processes."  How much money was spent developing that piece of future rhetorical nonsense. It is why I asked the question above "Rote learning is not practiced in our schools, but I wonder what the ratio of ADHD and Asperger suffers are in those countries that still use it compared to ours."(?)  I would go further here and ask how many of our chidren are we condemning to Alzheimer's because we do not use rote learning methodologies? I suggest that rote learning enhances our ability to develop memory retrieval systems, they are the neural constructs that allow us to learn how to remember things. We can enhance that further by adding a jingle into that, a chant if you like. We can build a memory palace with practice and patience and a strong retrieval system. However, rote learning was replaced with directed activities and contructivist learning practices.  Why?

(As an aside, there was a story some years back about a convent full of nuns, in Canada, and to occupy their time, they played games, all sorts of word games, number games, board games, and other games. The convent had never had a reported case of dementia. One nun died unexpectedly and in the course of an autopsy they found the nun had the neural damage indicating that she had Alzhiemers. The nun had won a spelling bee just a week prior to her death. Like other such stories, there maty or may not be any truth in it at all, but the idea is that there is already established a link between maintaining good neural plasticity with constant memory usage and reducing the impact of Alzhiemers.)

I suggest it is here where some fundamental questions lie. One of those is why is it that we must have an all or nothing approach? For example, what Gardner has proposed and developed in his multiple intelligences, is sensible and is reflective of a lot of students, but his comments in the article suggest he has made that same error, the one-in-all-in, in his immediate response to the study. What about making an appraisal of what helps and what does not? Why do we need these studies when we already see the impact of them, if we care to look.

In reply to Colin Fraser

Re: NY Time article: To Really Learn, Quit Studying and Take a Test

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Oh, also, I am sure I have seen the Butler study late last year, but I cannot be certain, it seems familiar so I may have skimmed it. I had not run across the Rohrer et al document before, so thanks for that. It confirms something I had been thinking for a while, repeated exposure coupled with strong incentives to succeed in testing situations are very important, in combination, to longer term retention. Repeated exposure to any material is going to have a longer residual impact, that is obvious. Repeated exposure to things that are entirely contextual can actually be dismissed as unimportant therefore of no longer term importance and are easily forgotten. A strong reason to remember, then regurgitate, information is very useful, and we can remember things for a very long time. I had a former student I had not seen for 5 years complain to me that he can still remember the trigonometry formulae we learned in that class and he recited them to prove it. Unbelievable, given that he was not what you would call a co-operative student!!! Yes, we did tests then too - but that does not explain why he was able to recall such knowledge. There is no reason for him to recall or use that information in his employment, chef, so why did he recall it?

That is what makes people so interesting for me, these little quirks in our makeup.