Month: October 2014

Today, I stumbled on this 2011 New York Times article on testing as a learning tool. It seemed worth posting as a follow-up to my recent “Test to learn” post.

This post is one of an (intended) collection interacting with the book Make It Stick: The Science of Successful Learning. All in the collection are linked from the Introduction post.

[Edited to add, a few days after initial post: this New York Times article is an interesting read on the subject.]

Test…to learn?

First–please forget about standardized testing and “teaching to the test”. That’s a discussion mostly unrelated to “testing to learn”.

The authors of Make It Stick tell the story of a group of university employees challenged to give the locations of the fire extinguishers closest to their offices. Few could do so, but one professor–who’d worked there for 25 years–looked for the one closest to his office. He found it–right next to his office door.

Many are the times that I’ve listened to a good lecture or a good sermon, and an hour later remembered that it was especially insightful or interesting–but would have been hard pressed to tell you even the main points of the presentation. Ditto for any number of wonderful books I’ve read.

So…what’s up? Why can’t I remember what I’ve read or heard, and why can’t a university professor remember the fire extinguisher right next to his office door?

Perhaps it would help to listen to the lecture again, or read the book again–but it probably won’t help a lot. What will help, according to the evidence, is testing–or, if that term makes your skin crawl, “active recall”. When you’re forced to produce, or even to simply recall an answer, its “stickiness” in your long-term memory goes way up. The authors of Make It Stick cite one study of people asked to memorize random lists, to be tested a week later. Everyone immediately forgot about half of the list–not too bad, for a large, random list. But those who were tested immediately after seeing the lists did substantially better a week later than those who weren’t, a benefit that came through the simple act of actively recalling the information.

Even more interesting was what happened when people were tested several times immediately after initially learning the list. The result: after the initial drop-off, the repeatedly tested group forgot basically nothing more in the intervening week. Their never-tested counterparts, in the same time period, lost substantially more.

Perhaps more interesting than random lists were the stories of “testing to learn” in actual classroom settings. The upshot: it’s quite effective in that setting, supported both by controlled studies and anecdotal evidence.

The thing is, the act of remembering something is not just pulling a file out of your mental archives. Instead, when you recall knowledge or a skill, it’s as though you walked through your personal library, asked several people where the desired information might be, and finally found the file you were looking for. After finding it, reading it, and adding some new notes, you re-file the document in a slightly more prominent spot, making a mental note of where you can find the information the next time you want it. The simple act of recalling something tells your brain that it’s important, and should be retained and made as accessible as possible.

Besides strengthening what you already know, testing your knowledge is a great way to figure out what you don’t know, and to work on learning/re-learning them.

Competitors of “test to learn”

“Test to learn” does have a few competitors in the learning-techniques arena–competitors, at least, in their popularity. In effectiveness, there’s little contest. Perhaps you’ve tried some of them. One is frequent re-reading–perhaps with the aid of a highlighter, so that you can read only the important points. Another is “massed practice”, focused review on one narrow skill or set of facts. In brief: though a quick re-read after some time might be a helpful warm-up, but evidence shows that re-reading is unlikely to offer much benefit in long-term retention. Massed practice, which we’ll visit in more depth in another post, shows great short-term benefits, but those benefits quickly fade into oblivion.

I find analogies helpful here: “test to learn” is like an apprenticeship, offering lots of uncomfortable challenge but resulting in lifelong skill. Massed practice is like the “weekend warrior” approach to working out: it feels like an accomplishment, but doesn’t add a lot of value over the long term. And re-reading is like some of my magazine subscriptions used to be: magazines showed up at my door frequently, I sometimes actually read them, and those I did read seldom provided any lasting value. And so, despite my extensive exposure to magazines related to science and finance, I somehow managed to avoid developing exceptional genius in either field!

Making it practical

So…if you’re a learner, what does it mean to “test to learn”? If you’re helping others learn, how might you apply this? It’s worth pausing for a bit to think about this before proceeding. (Another lesson, mostly for another day, taken from Make It Stick.)

A few practical ideas include:

As a learner:

• As you read, or perhaps after finishing a chapter, write your own quiz. If you’re in a rush, even “writing” (and taking) the quiz in your own mind will be far more helpful than doing nothing. What are the “big ideas” of this text? Are there facts or lines of reasoning that you need to remember?
• As much as possible, test information in the way you’ll use it. The book offers the vivid example of a police officer who encountered an armed suspect. In a well-practiced maneuver, he broke the other’s hold on his weapon and seized it–then, in equally well-practiced fashion, he handed it back to the suspect, just as he’d done hundreds of times while practicing the maneuver with other officers. Fortunately, he recovered quickly enough to take possession of the weapon again and keep it.
• If you can and if the information is important to remember, come back in a few days and ask yourself the same questions–an example of spaced repetition, another powerful tool we’ll talk about later.
• Be a “mean” tester: make yourself work to answer your self-test! Some questions may have simple, factual answers–but as much as possible, lean toward questions that make you apply what you know and integrate it with the rest of your knowledge. The harder you have to work to answer these questions, the more “sticky” the learning will be.
• As a corollary, don’t be content with answers that you don’t understand. E.g., don’t just cram a math formula into your head by rote, but understand why it it works the way it does.
• If you’re reading a book with end-of-chapter questions, by all means use them, at least as a starting point!
• Bonus points if you enter your questions and answers in Anki, Memrise, or another spaced-repetition system and keep on using it, for an ultimate “mind hack” using the powers of both testing and spaced repetition.

As one who helps others learn:

• Use tests as a teaching tool. This doesn’t rule out tests as an assessment tool, but those are a different game. Take advantage of the power of testing in increasing recall, and save a bit of time for a low-stakes quiz at the end of class.
• Use spaced repetition along with testing: have people recall last week’s concepts, and those from a few weeks ago.
• Help learners develop the skills of “testing” themselves.
• If you’re a classroom teacher, use comprehensive tests (in this case, we’re talking about the tests normally used for “assessment”). Yes, your students might hate you for it. 🙂 But their true learning will go up substantially. Of course, some of the other points mentioned make this much more “doable” than it might otherwise be.

Your thoughts?

• How important is it to retain what you read or hear?
• How can you apply the power of testing to help you?
• If your job is helping others learn, how might you facilitate their learning through testing?
• What other applications of this idea might there be?
• …and where might it go wrong?