You probably aren’t, now, an unfortunate consequence of migrating the blog from one host to another. If you want to follow, please re-subscribe at the left.
I stood on the sidelines recently of a Facebook debate over online shopping vs. buying locally. The thing is, I sympathize with both sides of the debate–but I think both are incomplete. Two of the biggest considerations are efficiency of a system and resiliency of a system. Other effects, harder to quantify or predict but real nonetheless, matter as well–but let’s stay primarily with efficiency and resilience.
First, let’s consider efficiency. A few questions worth asking include what efficiency means, whether (or in what cases) it’s good, and what tradeoffs it includes.
To be efficient is to accomplish a goal with a minimal investment of resources: time, money, energy, labor, or whatever you consider. Thus, to say a system is “efficient” is always to say that it is efficient in accomplishing [some effect or product], while minimizing the use of [some input], in [a specific environment]. Efficient use of one resource often involves the use of other resources or the introduction of other costs. For example, inter-state travel takes less time than it did in the era of the Oregon Trail, but at the cost of natural resources (petroleum fuel), environmental quality (pollution, paved roads), and even social capital within local communities. The system is designed to work “well” in an environment where energy is relatively inexpensive.
(A disclaimer here: a lot of this article blurs the lines between efficiency as described above and efficacy, the simple act of getting things done without reference to the particular costs. I think it’s a fair “blurring” for our purposes, but it is a blurring.)
Efficiency is in itself amoral, but its goals, costs, and benefits are matters of ethics. The Third Reich, of course, was quite efficient in achieving its deadly goals, which we see as evil. Adam Smith and Henry Ford, in valuing specialization, made manufacture of material goods more efficient (and thus more broadly available), with a culture of “artisanship” a casualty, and substantial changes to societal organization. Facebook and Google are increasingly efficient at targeting ads, again re-distributing influence and power. In farming, capital allows ultra-efficient production of crops with respect to human labor–which in turn empties rural communities into cities across the world. Efficiency may be evil, it may be good, but it’s often mixed.
An “efficient” system is generally efficient in a specific environment. Examples abound from various contexts:
- Darwin’s revelations about finches arose from their varied beaks, each adapted to the particular food sources available to it.
- Artificial-intelligence systems gain “intelligence” through training. If you train a system to identify butterfly species by showing it many thousands of images of butterflies, it will probably get better and better at making such identifications.
- Borrowing to invest in real estate, in an era of increasing real-estate prices, is efficient. It makes money, with minimal research required. The same is true for stocks.
- In the era leading up to the American Civil War, the North became efficient in its use of human labor, substituting capital. The South, with abundant slave labor, was “efficient” in producing a lot of cotton (and money) with little technological capital.
- For the average 20-year-old, major health insurance is an “inefficient” expenditure–they’ll probably not need it, and if they don’t need it it’s an unnecessary drag on their financial future.
Efficiency usually comes from specialization, a particular emphasis on some areas and de-emphasis of others. When contexts change, though, “efficient” systems often don’t handle the changes well.
- A finch with a fine beak that’s terrific at drilling into cactus fruits will suffer if the cactus population falls to disease.
- If your AI butterfly-ID system has been trained on entomological specimens, it might conclude that anything that doesn’t have pin in it with its wings wide open isn’t a butterfly.
- Leveraged investments in real estate worked well, until they didn’t. You know how 2008 worked out.
- In the American Civil War, it turned out that manufacturing capacity mattered a whole lot more in a conflict than did cotton production.
- For the 20-year-old without health insurance, a major health problem can quickly turn “efficiency” into financial disaster.
Higher levels of efficiency often go with higher levels of fragility. We can tune a system to run really, really well (however we define that) in a given environment–but our environment keeps on changing. It’s worth something to reduce the likelihood that a system will crash and burn–in other words, to invest in resilience.
Warren Buffett noted years ago that his first rule of investment is to “never lose money”. He noted as well that any number multiplied by zero is zero–and he has amply demonstrated a willingness to pass over “great investment opportunities” that offer too high a risk of going to zero.
Resilience has value. It even has monetary value, as demonstrated in the insurance, the options, and the futures markets. But the value of resilience goes beyond that–resilience in political systems and social capital has value of its own, even if it’s hard to quantify or monetize. And resilience, over the long term, is essential to maintaining whatever sorts of efficiency a society wants.
Efficiency has value as well, if its goals are good and its trade-offs are understood and reasonable. And a society that’s “efficient” in offering material, spiritual, social, and psychological “goods” will likely be good at achieving resilience as well.
What does this mean for the Amazon-versus-local-shopping debate? You decide. I don’t have comprehensive answers. I do have a few assorted thoughts, though:
- I love the empowerment and efficiency that Amazon gives me (along with other online stores), and the fact that everything is available.
- I love the accessibility of local stores, the chances for people to engage the public in their own right rather than as part of a large corporate system, and the chances for multifaceted personal connection as the same people interact in business, in community organizations, schools, and worship communities, as neighbors…and overall, as people rather than as economic units. I’m willing to pay a “tax” to keep this community thriving.
- Amazon isn’t evil. Amazon is far from an unmitigated good.
- Local shopping offers much good. It’s far from perfect.
- We need to keep on engaging this issue on all sides.
- Local stores can’t stagnate, or they’re complicit in their own deaths. (I went looking for ways to purchase online–or even browse–stuff from Hutchinson-area stores, and came up basically empty. Bluebird Books, with their online presence built on a national platform for independent bookstores, was the shining exception.)
- On the consumer side, a constant search for the bottom dollar will make it really difficult for local businesses to succeed, with real losses to a community.
- As utopian as the idea may seem, the online-shopping giants would do well to figure out how to partner with local communities and businesses, rather than just “disrupting” them.
Coding can be a playground that allows children to become producers, and not merely consumers, of technology
The playground approach to coding moves the conversation beyond the traditional view of coding as a technical skill. Coding is a literacy. As such, it invites new ways of thinking and carries the ability to produce an artifact detached from its creator, with its own meaning.
If you want to maximize your learning-per-reading-time, you should probably forget about my blog and read Farnam Street. I like learning about cognition, learning, and productivity; Shane Parrish at Farnam Street seems to live a life immersed in such learning, and does a great job of boiling it down for brief presentation. If you must stick around here, though, my WordPress stats will be thrilled–believe me, every page view shows up as a substantial percentage!
When I started this blog, it was a tentative thing–a platform whose significance I downplayed, but one through which I secretly hoped I’d accomplish something world-changing. I didn’t know what, but after all…who really needs just another “data dump” made public? I wanted to write valuable things, important things, things that earned respect.
Writing content that’s worth reading is a great idea. Unfortunately, this idea has translated into my writing very little. At the same time, I’ve gained appreciation for “simmers” of ideas: experiences like watching a dozen TED talks at a time, reading Alex Steffens’s Worldchanging book, or reading half a dozen books on related topics in a weekend. In this simmer, ideas collide, interact, and combine in wonderful ways. Ideas don’t need to be long; they don’t need to be fully developed to be useful. They just need to interact productively with other ideas. A tornado of index cards might have more value than a well-organized essay–even though the former looks like chaos, and is unlikely to win any literary prizes.
So…I’m going back to the origin of “blogging”. “Blog”, of course, is a contraction of “weblog”–in my interpretation, a stream of brief thoughts released to the Web. It may be that something grand emerges, but I’m not planning on it. What I am planning is that I’ll once again write to help order my understanding of the world. And sometimes, I’ll just share with little comment the small, delightful morsels that I’ve found on my experiential plate.
This post is one of an (intended) collection interacting with the book Make It Stick: The Science of Successful Learning. All in the collection are linked from the Introduction post.
[Edited to add, a few days after initial post: this New York Times article is an interesting read on the subject.]
First–please forget about standardized testing and “teaching to the test”. That’s a discussion mostly unrelated to “testing to learn”.
The authors of Make It Stick tell the story of a group of university employees challenged to give the locations of the fire extinguishers closest to their offices. Few could do so, but one professor–who’d worked there for 25 years–looked for the one closest to his office. He found it–right next to his office door.
Many are the times that I’ve listened to a good lecture or a good sermon, and an hour later remembered that it was especially insightful or interesting–but would have been hard pressed to tell you even the main points of the presentation. Ditto for any number of wonderful books I’ve read.
So…what’s up? Why can’t I remember what I’ve read or heard, and why can’t a university professor remember the fire extinguisher right next to his office door?
Perhaps it would help to listen to the lecture again, or read the book again–but it probably won’t help a lot. What will help, according to the evidence, is testing–or, if that term makes your skin crawl, “active recall”. When you’re forced to produce, or even to simply recall an answer, its “stickiness” in your long-term memory goes way up. The authors of Make It Stick cite one study of people asked to memorize random lists, to be tested a week later. Everyone immediately forgot about half of the list–not too bad, for a large, random list. But those who were tested immediately after seeing the lists did substantially better a week later than those who weren’t, a benefit that came through the simple act of actively recalling the information.
Even more interesting was what happened when people were tested several times immediately after initially learning the list. The result: after the initial drop-off, the repeatedly tested group forgot basically nothing more in the intervening week. Their never-tested counterparts, in the same time period, lost substantially more.
Perhaps more interesting than random lists were the stories of “testing to learn” in actual classroom settings. The upshot: it’s quite effective in that setting, supported both by controlled studies and anecdotal evidence.
The thing is, the act of remembering something is not just pulling a file out of your mental archives. Instead, when you recall knowledge or a skill, it’s as though you walked through your personal library, asked several people where the desired information might be, and finally found the file you were looking for. After finding it, reading it, and adding some new notes, you re-file the document in a slightly more prominent spot, making a mental note of where you can find the information the next time you want it. The simple act of recalling something tells your brain that it’s important, and should be retained and made as accessible as possible.
Besides strengthening what you already know, testing your knowledge is a great way to figure out what you don’t know, and to work on learning/re-learning them.
Competitors of “test to learn”
“Test to learn” does have a few competitors in the learning-techniques arena–competitors, at least, in their popularity. In effectiveness, there’s little contest. Perhaps you’ve tried some of them. One is frequent re-reading–perhaps with the aid of a highlighter, so that you can read only the important points. Another is “massed practice”, focused review on one narrow skill or set of facts. In brief: though a quick re-read after some time might be a helpful warm-up, but evidence shows that re-reading is unlikely to offer much benefit in long-term retention. Massed practice, which we’ll visit in more depth in another post, shows great short-term benefits, but those benefits quickly fade into oblivion.
I find analogies helpful here: “test to learn” is like an apprenticeship, offering lots of uncomfortable challenge but resulting in lifelong skill. Massed practice is like the “weekend warrior” approach to working out: it feels like an accomplishment, but doesn’t add a lot of value over the long term. And re-reading is like some of my magazine subscriptions used to be: magazines showed up at my door frequently, I sometimes actually read them, and those I did read seldom provided any lasting value. And so, despite my extensive exposure to magazines related to science and finance, I somehow managed to avoid developing exceptional genius in either field!
Making it practical
So…if you’re a learner, what does it mean to “test to learn”? If you’re helping others learn, how might you apply this? It’s worth pausing for a bit to think about this before proceeding. (Another lesson, mostly for another day, taken from Make It Stick.)
A few practical ideas include:
As a learner:
- As you read, or perhaps after finishing a chapter, write your own quiz. If you’re in a rush, even “writing” (and taking) the quiz in your own mind will be far more helpful than doing nothing. What are the “big ideas” of this text? Are there facts or lines of reasoning that you need to remember?
- As much as possible, test information in the way you’ll use it. The book offers the vivid example of a police officer who encountered an armed suspect. In a well-practiced maneuver, he broke the other’s hold on his weapon and seized it–then, in equally well-practiced fashion, he handed it back to the suspect, just as he’d done hundreds of times while practicing the maneuver with other officers. Fortunately, he recovered quickly enough to take possession of the weapon again and keep it.
- If you can and if the information is important to remember, come back in a few days and ask yourself the same questions–an example of spaced repetition, another powerful tool we’ll talk about later.
- Be a “mean” tester: make yourself work to answer your self-test! Some questions may have simple, factual answers–but as much as possible, lean toward questions that make you apply what you know and integrate it with the rest of your knowledge. The harder you have to work to answer these questions, the more “sticky” the learning will be.
- As a corollary, don’t be content with answers that you don’t understand. E.g., don’t just cram a math formula into your head by rote, but understand why it it works the way it does.
- If you’re reading a book with end-of-chapter questions, by all means use them, at least as a starting point!
- Bonus points if you enter your questions and answers in Anki, Memrise, or another spaced-repetition system and keep on using it, for an ultimate “mind hack” using the powers of both testing and spaced repetition.
As one who helps others learn:
- Use tests as a teaching tool. This doesn’t rule out tests as an assessment tool, but those are a different game. Take advantage of the power of testing in increasing recall, and save a bit of time for a low-stakes quiz at the end of class.
- Use spaced repetition along with testing: have people recall last week’s concepts, and those from a few weeks ago.
- Help learners develop the skills of “testing” themselves.
- If you’re a classroom teacher, use comprehensive tests (in this case, we’re talking about the tests normally used for “assessment”). Yes, your students might hate you for it. 🙂 But their true learning will go up substantially. Of course, some of the other points mentioned make this much more “doable” than it might otherwise be.
- How important is it to retain what you read or hear?
- How can you apply the power of testing to help you?
- If your job is helping others learn, how might you facilitate their learning through testing?
- What other applications of this idea might there be?
- …and where might it go wrong?