If you want to maximize your learning-per-reading-time, you should probably forget about my blog and read Farnam Street. I like learning about cognition, learning, and productivity; Shane Parrish at Farnam Street seems to live a life immersed in such learning, and does a great job of boiling it down for brief presentation. If you must stick around here, though, my WordPress stats will be thrilled–believe me, every page view shows up as a substantial percentage!
When I started this blog, it was a tentative thing–a platform whose significance I downplayed, but one through which I secretly hoped I’d accomplish something world-changing. I didn’t know what, but after all…who really needs just another “data dump” made public? I wanted to write valuable things, important things, things that earned respect.
Writing content that’s worth reading is a great idea. Unfortunately, this idea has translated into my writing very little. At the same time, I’ve gained appreciation for “simmers” of ideas: experiences like watching a dozen TED talks at a time, reading Alex Steffens’s Worldchanging book, or reading half a dozen books on related topics in a weekend. In this simmer, ideas collide, interact, and combine in wonderful ways. Ideas don’t need to be long; they don’t need to be fully developed to be useful. They just need to interact productively with other ideas. A tornado of index cards might have more value than a well-organized essay–even though the former looks like chaos, and is unlikely to win any literary prizes.
So…I’m going back to the origin of “blogging”. “Blog”, of course, is a contraction of “weblog”–in my interpretation, a stream of brief thoughts released to the Web. It may be that something grand emerges, but I’m not planning on it. What I am planning is that I’ll once again write to help order my understanding of the world. And sometimes, I’ll just share with little comment the small, delightful morsels that I’ve found on my experiential plate.
This post is one of an (intended) collection interacting with the book Make It Stick: The Science of Successful Learning. All in the collection are linked from the Introduction post.
[Edited to add, a few days after initial post: this New York Times article is an interesting read on the subject.]
First–please forget about standardized testing and “teaching to the test”. That’s a discussion mostly unrelated to “testing to learn”.
The authors of Make It Stick tell the story of a group of university employees challenged to give the locations of the fire extinguishers closest to their offices. Few could do so, but one professor–who’d worked there for 25 years–looked for the one closest to his office. He found it–right next to his office door.
Many are the times that I’ve listened to a good lecture or a good sermon, and an hour later remembered that it was especially insightful or interesting–but would have been hard pressed to tell you even the main points of the presentation. Ditto for any number of wonderful books I’ve read.
So…what’s up? Why can’t I remember what I’ve read or heard, and why can’t a university professor remember the fire extinguisher right next to his office door?
Perhaps it would help to listen to the lecture again, or read the book again–but it probably won’t help a lot. What will help, according to the evidence, is testing–or, if that term makes your skin crawl, “active recall”. When you’re forced to produce, or even to simply recall an answer, its “stickiness” in your long-term memory goes way up. The authors of Make It Stick cite one study of people asked to memorize random lists, to be tested a week later. Everyone immediately forgot about half of the list–not too bad, for a large, random list. But those who were tested immediately after seeing the lists did substantially better a week later than those who weren’t, a benefit that came through the simple act of actively recalling the information.
Even more interesting was what happened when people were tested several times immediately after initially learning the list. The result: after the initial drop-off, the repeatedly tested group forgot basically nothing more in the intervening week. Their never-tested counterparts, in the same time period, lost substantially more.
Perhaps more interesting than random lists were the stories of “testing to learn” in actual classroom settings. The upshot: it’s quite effective in that setting, supported both by controlled studies and anecdotal evidence.
The thing is, the act of remembering something is not just pulling a file out of your mental archives. Instead, when you recall knowledge or a skill, it’s as though you walked through your personal library, asked several people where the desired information might be, and finally found the file you were looking for. After finding it, reading it, and adding some new notes, you re-file the document in a slightly more prominent spot, making a mental note of where you can find the information the next time you want it. The simple act of recalling something tells your brain that it’s important, and should be retained and made as accessible as possible.
Besides strengthening what you already know, testing your knowledge is a great way to figure out what you don’t know, and to work on learning/re-learning them.
Competitors of “test to learn”
“Test to learn” does have a few competitors in the learning-techniques arena–competitors, at least, in their popularity. In effectiveness, there’s little contest. Perhaps you’ve tried some of them. One is frequent re-reading–perhaps with the aid of a highlighter, so that you can read only the important points. Another is “massed practice”, focused review on one narrow skill or set of facts. In brief: though a quick re-read after some time might be a helpful warm-up, but evidence shows that re-reading is unlikely to offer much benefit in long-term retention. Massed practice, which we’ll visit in more depth in another post, shows great short-term benefits, but those benefits quickly fade into oblivion.
I find analogies helpful here: “test to learn” is like an apprenticeship, offering lots of uncomfortable challenge but resulting in lifelong skill. Massed practice is like the “weekend warrior” approach to working out: it feels like an accomplishment, but doesn’t add a lot of value over the long term. And re-reading is like some of my magazine subscriptions used to be: magazines showed up at my door frequently, I sometimes actually read them, and those I did read seldom provided any lasting value. And so, despite my extensive exposure to magazines related to science and finance, I somehow managed to avoid developing exceptional genius in either field!
Making it practical
So…if you’re a learner, what does it mean to “test to learn”? If you’re helping others learn, how might you apply this? It’s worth pausing for a bit to think about this before proceeding. (Another lesson, mostly for another day, taken from Make It Stick.)
A few practical ideas include:
As a learner:
- As you read, or perhaps after finishing a chapter, write your own quiz. If you’re in a rush, even “writing” (and taking) the quiz in your own mind will be far more helpful than doing nothing. What are the “big ideas” of this text? Are there facts or lines of reasoning that you need to remember?
- As much as possible, test information in the way you’ll use it. The book offers the vivid example of a police officer who encountered an armed suspect. In a well-practiced maneuver, he broke the other’s hold on his weapon and seized it–then, in equally well-practiced fashion, he handed it back to the suspect, just as he’d done hundreds of times while practicing the maneuver with other officers. Fortunately, he recovered quickly enough to take possession of the weapon again and keep it.
- If you can and if the information is important to remember, come back in a few days and ask yourself the same questions–an example of spaced repetition, another powerful tool we’ll talk about later.
- Be a “mean” tester: make yourself work to answer your self-test! Some questions may have simple, factual answers–but as much as possible, lean toward questions that make you apply what you know and integrate it with the rest of your knowledge. The harder you have to work to answer these questions, the more “sticky” the learning will be.
- As a corollary, don’t be content with answers that you don’t understand. E.g., don’t just cram a math formula into your head by rote, but understand why it it works the way it does.
- If you’re reading a book with end-of-chapter questions, by all means use them, at least as a starting point!
- Bonus points if you enter your questions and answers in Anki, Memrise, or another spaced-repetition system and keep on using it, for an ultimate “mind hack” using the powers of both testing and spaced repetition.
As one who helps others learn:
- Use tests as a teaching tool. This doesn’t rule out tests as an assessment tool, but those are a different game. Take advantage of the power of testing in increasing recall, and save a bit of time for a low-stakes quiz at the end of class.
- Use spaced repetition along with testing: have people recall last week’s concepts, and those from a few weeks ago.
- Help learners develop the skills of “testing” themselves.
- If you’re a classroom teacher, use comprehensive tests (in this case, we’re talking about the tests normally used for “assessment”). Yes, your students might hate you for it. 🙂 But their true learning will go up substantially. Of course, some of the other points mentioned make this much more “doable” than it might otherwise be.
- How important is it to retain what you read or hear?
- How can you apply the power of testing to help you?
- If your job is helping others learn, how might you facilitate their learning through testing?
- What other applications of this idea might there be?
- …and where might it go wrong?
It goes by many names–the false dichotomy, the either/or when you should be thinking both/and, first things and second things. We have a mental weakness–we’re fine with exposure to various ideas, but we want them to combine into perfectly harmonized chords.
Perhaps we should be trying to synthesize ideas rather than to harmonize them. To synthesize literally means to put together; we’ve added a connotation of putting things together in a way that makes sense. To harmonize ideas is to make them fit together. The problem is, ideas aren’t always mutually consistent in themselves; they may each be true within specific contexts or in specific ways. Sometimes, we simply need to hold several mutually inconsistent ideas. Don’t force them to premature consistency; don’t stop asking why they seem both true and inconsistent, or on what underlying framework they both might fit.
When two ideas are in conflict, consider that the “opponents” may actually be on the same team, but playing by rules you don’t yet know. You have a deep framework, a network of ideas that makes up who you are. Some ideas are easily wired into the structure; others require a lot of reorganization. As your framework gets more complex, you gain in ability to process ideas. Apparent conflicts are a gift, forcing you to increase your mental abilities in order to handle them. Embrace this chance to grow.
Part of the reason that I am/will be blogging about books I read is to force myself to process them more deeply than I often do. Today, I thought of another tool that might be helpful: the “One Big Thing”.
“One Big Thing” is a phrase stolen from Immunity to Change. That book tells the story of a group of people who knew that they needed to change, but who didn’t each know exactly what needed to change. Rather than trying to change many things at once, they found it most helpful if each person decided, with others’ input, what was the One Big Thing he or she needed to focus on.
Pulling the “One Big Thing” into a different context, I hope to ask that question of each book I read: what is the “One Big Thing”, the unifying idea or the typical gem that defines this book? Why is the book worth reading (if indeed it is)?
I’m a convert. I’m an evangelist. And…I’m a skeptic. I’m rapidly going digital in practice and preference, and yet I’m wistful about the things my Kindling leaves behind.
Several years ago, I was working overseas for a number of months. It was then that I realized how much I missed my reading materials. Sure, I could still order books from Amazon, at $10 shipping per book. But, delivery time was measured in weeks rather than days–a trial for someone who likes to have his resources now. My magazines…well, they were still faithfully delivered to my address in the States. My 18 pounds of CFA curriculum? Ditto.
It was during this time that I learned of an amazing new product. Amazon had released the Kindle, a device with an e-ink screen and a battery that lasted for weeks–a device that would let you buy and read books anywhere in the world, almost instantly! Moreover, I wouldn’t have to use precious luggage space for the books I “might want”–I could have access to my entire digital library, from anywhere!
It was a while before I was completely hooked. After returning to the States, the Kindle’s $400 price tag was a luxury I wasn’t sure I could afford. And, as convenient as it is to “own” digital books in the cloud, I didn’t like the idea that Amazon controlled all of my access to books I’d bought and paid for–and that my library could, in theory, vanish into thin air if Amazon ceased to exist, if they erred or my account’s security were breached, or if I simply lost access to Amazon’s Kindle-reading devices and programs. I even thought about the evanescence of digital data: it’s not efficient to inscribe knowledge on clay tablets and fire them, but they’ll last for millennia! The same isn’t true of a Kindle, though the underlying data might last (see the Library of Congress for more information).
Despite a slow start, I’ve now sold out to Amazon–surprising myself at the speed of my dive into digital, for good or for ill. I still have reservations, but the benefits of digital reading hold sway:
- Portability. I’m contemplating quite a bit of overseas travel and living in the next while. The thought of being able to take almost all of my library with me in a few ounces, rather than trying to figure out which books should take their part of a 50-lb weight allowance, is a strong selling point.
- Accessibility. This is partly a consequence of portability. Beyond that, though, the ability to pull out my phone in a bit of “dead” time and read real books, thoughtful analyses, novels, or even “mind candy” is huge.
- Searchability. It’s easy to pull up that quote I know I saw somewhere in the book. Moreover, it’s a lot easier to search my archives for a book than to search my shelves for the “dead trees” version.
- Instant gratification. A number of times recently, I’ve needed information in the near term, and have either known what book contained it or found a promising book by searching. Moments later (and a few dollars later), I was reading the book. The Web is great, but book-length treatments are still often the most valuable.
- Integrated dictionary. My vocabulary is relatively large, but I still find it useful to read with an integrated dictionary. Recently, reading William Styron on Kindle, I found myself luxuriating in his use of language–but frequently looking up the words he used. Had I needed to a separate dictionary (whether digital or paper), I’d have been much less likely to look them up and expand my vocabulary.
- Sales and freebies. Amazon occasionally runs sales on their Kindle books. Many of them are books in which I have no interest; others are opportunities to acquire interesting books at a fraction of the normal price. There’s a lot of flexibility in pricing a product with a unit price that approaches zero, vs. the costs of physical production and distribution. Many classics are available for free in digital form, and promotional “freebies” of other books often are available (and, admittedly, worth that price a good portion of the time). Though it takes sorting through some dross, discounted or free Kindle books do add to the value proposition.
- Thanks to DRM (digital rights management), you’re locked into a single company or set of companies. You can’t read Nook books on your Kindle, or Kindle books on your Nook. You can, though, read Google Books on your Nook, and any of these formats on a smartphone, tablet, or computer. That “lock-in” also means that if the company with the key goes away, your books are vulnerable. Fortunately, many out-of-copyright and other books are not “protected” (as in, protected from the purchaser!) by DRM.
- For similar reasons, the selling company and the publishers decide whether and how you can “lend” e-books. For the Kindle, some books allow a single loan per book.
- I’ll sometimes leaf through a paper book to get an idea of its contents. Though the ability to preview Kindle books is nice, “leafing through” an e-book, even one I own, doesn’t seem to work very well. The same is true for a “skim reading” of a book.
- While in college, I realized that part of my mental “indexing system” is spatial: though I may not be able to pull up a fact or exact quote I vaguely recall, I may remember that it’s about 2/3 of the way through the book, at the top of a right-hand page. This is one small part of richly encoding information in my memory, but it’s unavailable with an e-book. I’d love if an e-book vendor could re-create such mental cues.
…fully titled Immunity to Change: How to Overcome It and Unlock the Potential in Yourself and Your Organization, by Robert Kegan and Lisa Laskow Lahey.
Immunity to Change is like many of the “personal growth” books I’ve read: the big ideas of the book make it worth sorting through the fluff. Like a number of books of this kind, it spends a lot of time telling you about why the ideas are worthwhile. But, the many stories and examples do help to bridge from “head” to “heart”.
The big ideas of Immunity to Change are:
- We have the ability to grow, increasing our ability to process complex situations, throughout our lives.
- Our environment brings an increasing need to deal with complexity. The book looks at three levels of complexity, and intends to move its users to higher levels on the spectrum. The first is the socialized mind, who relies on others for direction. Next is the self-authoring mind, an independent person who has a vision in mind, holds himself accountable to it, and recruits others to it. Finally comes the self-transforming mind, who uses the tools of the self-authoring mind but seeks to continually learn. He can hold competing ideas in his mind, and is aware that efficiently achieving the wrong vision is of no value. He learns from others, and is interdependent with them. He is able to look at some of the mindsets that he could previously only look through.
- Some problems can’t be solved by technical learning, but must be approached adaptively. Most of us know how to lose weight (the technical solution): eat fewer calories and burn more of them. Many of us, though, need to adapt–to change who we are and how we think–in order to actually lose weight. We need different tools for adaptive solutions than for technical ones.
- Most problems requiring adaptive solutions come from a mental “immune system”, which is protecting hidden goals. For example, you might sincerely want to lose weight, but your eating habits reflect that you just as sincerely want not to be perceived–by others or yourself–as “a health nut”. Those commitments in turn are driven by assumptions, which you can test–and in so doing, possibly change the way you look at the world.
- Part of the book’s value is in the processes it lays out–processes for engaging in adaptive change. It’s easy to say “yeah, I need to change the way I think about this”, but it’s hard to actually make such a change. Immunity to Change lays out a process for figuring out what your “competing commitments” and the assumptions beneath them are, and then for figuring out whether they’re valid.
- Both individuals and groups can have “immunities to change”. The processes for figuring out immunities are similar, but looking at group immunities and assumptions does need its own process–which the authors describe separately.
My recent reading has included a lot of books about growth, both of individuals and of organizations.
I’m interested in learning about myself and the way I (and humans in general) think, both because I enjoy learning and because that’s the first step in being able to change myself.
I’m interested in learning about organizations because, well…we live in organizations. Whether it’s a nation, a church, an association, or just a few close friends, each of us is part of a lot of groups–each with its own “syntality” (cool word meaning “group personality”, its own goals and assumptions, and its own efficiency in achieving those goals. I want to live in a world where most groups have good goals and are good at pursuing those goals. Since I can’t magically transform the world, I at least want to help my groups to pursue the right goals well.
A lot of the posts in the next while will result from re-visiting (usually “skimming”) books I’ve read recently, often related either to self-awareness/self-transformation or to awareness of group dynamics and how to re-engineer them.
If you have recommendations of books I should read, please let me know.
Seth Godin recently gave some sage advice for improving your writing: “Just write poorly. Continue to write poorly, in public, until you can write better….Do it every day.” I may not reach that standard, but I do want to write well. For a time, that may mean writing poorly. It will mean exposing my pet ideas and their presentation to your criticism, which I welcome. If you can help me improve–either in my writing style, or in writing about things you find (and think I’d find) interesting, let me know.
I expect to have a number of book reviews or brief summaries on the blog–books that I’m reading and find interesting. I’m not sure what else will make it onto the Web with my name attached.