T-assessment: a bold suggestion modestly advanced

I sometimes use this column to float an idea I think deserves attention. Not on a whim, but after considerable thought and discussion with others expert in the relevant domain(s). This is one of those times. I already set the scene with last month’s post. Here is a nuanced, bullet-point summary of what I wrote then: 

1. The heart of learning mathematics is mastering a particular way of thinking – what I (and some others) call “mathematical thinking,” sometimes also described as “thinking like a mathematician.”
2. You can master mathematical thinking by focusing on any one branch of mathematics – arithmetic, geometry, algebra, trigonometry, calculus, etc. – and going fairly deep.
3. Once you have mastered mathematical thinking, you can fairly quickly acquire an equivalent mastery of any branch of mathematics with relatively shallow coverage. (There are limits to this. There is a complexity and abstraction hierarchy of branches of mathematics. But for K-12 mathematics, that is not an issue.)
4. Thus, to learn mathematics effectively, it suffices to (i) master mathematical thinking by the study of one branch of the subject, and (ii) acquire some breadth by branching out to a few other areas.
5. For the same reason, it meets society’s need for assessment of mathematics learning if we (i) assess mathematical thinking restricted to one branch, and (ii) measure the individual’s knowledge in a number of other branches.

If you buy this picture – and if you keep reading, I am going to try to sell it to you – then it means we can make use of modern technologies to be far more efficient, in terms of teacher and student time and of money spent, for both learning and assessment. I propose calling this approach T-learning and T-assessment, on account of the visualization of the model shown below.

The vertical of the T denotes the topic that is studied in depth to build mathematical thinking capacity. In last month’s post, I discussed a suggestion by mathematics learning expert Liping Ma that school arithmetic is the best subject for doing that, so I have put that down for the vertical. Read last month’s post to see my summary of her argument, and follow the link I gave there if you want to know more. But school arithmetic is not the only choice. Euclidean geometry could also work. In both cases (or with any other choice), it would be important to teach the T-vertical the right way, so as to bring out the general mathematical thinking patterns.

The horizontal bar of the T represents the collection of topics chosen to provide breadth. Each of the topics on the bar can be covered relatively quickly, once mathematical thinking has been mastered.

In my Introduction to Mathematical Thinking MOOC on Coursera, which has been running regularly since 2013, I used the structure of everyday language as the T-vertical, and some topics in elementary number theory for the bar of the T. I only needed one branch of mathematics on the bar, since the goal was to teach mathematical thinking itself, and for that, one application domain was enough. (I chose number theory since you need nothing more than arithmathetic to get into the early parts of the subject.) If the goal is to cover everything in the Common Core State Standards, you’d need a number of branches of mathematics (though only a handful).

So much for the diagram. Before I launch into my efficiencies sales pitch, let me make a few remarks about the itemized list above.

Item 1. I fear many readers will not really understand what I mean by this. Mathematicians surely will. But a sad consequence of the way mathematics has typically been taught, as a smorgasbord of definitions and facts to learn, tricks to remember, and procedures to practice, is that relatively few people survive their math education long enough to realize that the entire discipline revolves around a very small collection of thought patterns. I discussed this tragedy at some length in my last month’s post, citing some research results of my Stanford colleague Prof. Jo Boaler that show just how great a tragedy it can be.

Dr. Boaler is a former school teacher, education system administrator, and more recently a world-renowned mathematics education scholar of many years standing. She is one of a number of mathematics pedagogy experts I work and/or consult with. I mention that because my primary expertise is in mathematics, a discipline I have worked in for half a century. I do have a fair amount of knowledge of mathematics pedagogy, but purely as a result of studying the subject fairly extensively. I have not and do not engage in original research into mathematics pedagogy. I cannot, therefore, claim to be an expert in that domain. The suggestions I make here are, as always, in my capacity as a mathematician.

Item 2. As Boaler described, many students come away from the math class in despair, complaining that there are way too many things to remember. Yet, if you think back to your school days, there were probably one or two kids in the class for whom it seemed effortless. What was their secret? Were they simply math geniuses? Were they, as some parents like to say, “gifted”? I’ll answer those three questions in reverse order.

Very rarely, in various areas of human endeavors, exceptional individuals come along. That’s simply a feature of distributions with an element of randomization. But for the most part, children classified as “gifted” are simply the offspring of relatively well-off, educated parents who provide their children with excellent early role models and an educationally stimulating start in life. That is their “gift.” And indeed it is a gift; they were given it – by their parents. I point this out because, as research by Boaler and others has shown, labeling a child as “gifted” frequently turns out to have crippling consequences for that child. Having been told they were “gifted,” the child assumes that whatever level of effort leads to initial success in the math class will continue to do so – they rely on their “gift.” But in mathematics, as in many walks of life, the further you get into something, the harder it gets. While a child who accepts struggle and failure in math as part of the learning process will often keep trying, the “gifted” child may well (and often does) give up when they are no longer acing all the tests, perhaps claiming that they simply no longer found it interesting, in order to mask (from themselves as much as anyone else) the devastating consequence to their self-esteem that results from their having built up no tolerance of failure. (We learn when we get something wrong and figure out why. Getting something right simply gives us reassurance and maybe makes us feel good for a while.)

The “math geniuses” question. Unlike some of my colleagues, I don’t mind that term being used, as long as it is understood to refer to an individual who (a) was born with a brain particularly well suited to mathematical thinking, (b) found mathematics totally fascinating (for whatever reason, perhaps a desire to escape a miserable childhood environment by retreating into the mental world of mathematics), and (c) devoted thousands of childhood hours working on mathematics. For those are the three ingredients it takes to produce an individual who could merit being called a “genius.” There are very few such math geniuses in the world. In contrast, I suspect (on numerical grounds) there are a great many children born with a brain suited to mathematical thinking, who never pursue, or show prowess in, mathematics. The term “born genius,” which you sometimes come across, strikes me as idiotic.

The “What was their secret?” question. Those kids in your class who seemed to find math easy were the ones who, for whatever reason, managed to recognize that, for all that math was presented to them as a jumble of tricks and techniques, there was method to the seeming madness. Not just method, but a fairly simple method. Mathematics, they realized, was a theme-and-variations affair. There was no need to memorize anything beyond the basic multiplicative number bonds (the “times tables” as they were called when I was a lad growing up in the UK) – which even in today’s computation-rich environment are extremely useful to have at your mental fingertips.

ASIDE: Fortunately, the multiplicative number bonds can be committed to memory by using numbers often enough in meaningful contexts. But to my mind, since they can be mastered by rote (or even better, by playing one of several cheap, first-person-shooter, multiplication video games), you might as well get them out of the way as quickly as possible by a repetitive memorization process. Moreover, there is mathematical thinking mileage to be gained by this approach, when kids discover that there are various patterns that can be used to avoid actively memorizing most of the multiplication facts (x5, x10, and commutativity are three such time-saving patterns), leaving only a handful that have to be actually learned (6 x 7 and 7 x 8 are two such – though to this day I don’t have instant recall of 6 x 7, but rely on commutativity and instant recall of 7 x 6 – don’t ask!).

But I digress. The point is, that kid on the front row who annoyingly seemed to remember everything almost certainly remembered almost nothing; they worked out most of their answers on the fly. As I did with my answers for 6 x 7 and 8 x 9. (Okay, I guess I was one of those annoying kids.)

The point is, the crucial importance of approaching math learning as a process of acquiring a particular way of thinking does not just apply in the elementary grades – where many kids do manage to get by with pure memorization. The same is true all the way up into the more advanced parts of the subject, where memorization becomes impossible. Yet there is no need to memorize much of anything. Ever. Just a few key concepts, facts, and procedural details in each new branch of mathematics. In fact, if you continue in mathematics, after a while you realize that all the different branches of mathematics share what is essentially the same structure.

And the really nice thing is, mastering mathematical thinking to an adequate degree is like learning to ride a bike or to swim. Once you have it, you never lose it.

So much for the first two items on my initial list. But in elaborating on those, I’ve essentially covered Item 3, Item 4, and Item 5 as well. So we are done with that.

In fact, we’ve got so much useful stuff on the table now, it’s pretty straightforward to make that efficiencies sales pitch I promised you.

Current systemic assessments rely to a very high degree on digital technology, where students take a test presented and answered on a computer, which automatically grades their answers. To fit that format, questions are restricted to multiple-choice questions, questions that require a entry of single number as an answer, or some minor variant of one of these question types. (Earlier assessments used multiple-choice tests printed on paper that the student filled in with a pencil, with the completed test-paper then optically scanned into a computer.) This is fine for assessing what a student has learned on the horizontal (breadth) bar of the T. But on its own, the results of such a test are essentially useless. They measure either facts memorized or shallow (and often brittle) procedural manipulations based on memorized facts. They say nothing about an individual’s ability to think mathematically.

That is why the better systemic assessment systems on the market also present students with open-ended questions where the student has to solve a problem using paper and pencil, with the solutions for a whole class, school, or district being sent out for grading by trained human evaluators, who follow an evaluation rubric. Though this process does bring in an element of graders’ subjectivity – even with a well-thought-out and clearly expressed rubric, the graders are still faced with an often formidable interpretation task – it works remarkably well. But it is both time-consuming and expensive. It tends to be used only for major, summative assessments at the end of a unit or a school year. The time-delay alone makes it unsuitable for formative assessments intended to provide feedback to students about their progress and to alert teachers to the need for individual-student interventions or changes in the rest of the course.

With the T-model, only the core subject chosen to constitute the T-vertical needs to be assessed this way, of course. Even with existing assessment methods, making that restriction could lead to some cost reduction. But substantial savings, in personnel, time, and money, would be obtained if the subject in the T-vertical could be assessed automatically. With today’s technologies, it can.

To obtain good assessments of mathematical thinking, educators typically present students with what are known as “complex performance tasks” (or “rich performance tasks”), requiring multi-step reasoning. CPTs often (though not always) have more than one “correct” answer, with some answers being better than others. Even when there is a unique answer, there is frequently more than one solution (= sequence of reasoning steps) that gets to that answer. CPTs can range from very basic tasks, perhaps requiring only one or two individual steps (though with a period of reflective thought required in order to start) to the fiendishly difficult.

Some kinds of CPT (particularly in subjects such as arithmetic, geometry, and algebra) can be implemented as digital puzzles, where the student has to manipulate objects or symbols on a computer screen in order to find the solution. When deployed in this format, such CPTs can be used as systemic assessment tools. Not all mathematical subjects or topics lend themselves to this kind of presentation, so it is not a feasible approach for systemic assessment as currently conceived. But for T-assessment, it can work just fine. Simply specify the T-vertical to consist of mathematical topics that can be assessed using digitally-implemented CPTs. Because the assessment is conducted on the computer, the student’s entire solution to the CPT is captured and can be analyzed by an algorithm. In real-time. At no incremental cost. At whatever scale is required.

Of course, the key requirement here is to have a mathematical topic, or set of topics, and a set of CPTs in that area, that is collectively sufficient to demonstrate mathematical thinking ability. For that, remember, is what the T-vertical is all about.

Such digital assessment tools already exist. (Full disclosure: I am a member of one team developing and testing such tools.) So far, they have been subjected to limited testing on a small scale. The results have been encouraging. Conducting large-scale trials is clearly a necessary first step before they can be deployed in the manner I am suggesting. Moreover, to be useful, mathematics education has to be configured according to the T-model, where an in-depth study of one part of mathematics is used to develop the key capacity of mathematical thinking, coupled with much more shallow experiences in a number of other parts of math to achieve breadth.

That’s my suggestion. In putting it out, I might hear back that others have thought about, or advocated, something very much along the same lines. (In fact, Liping Ma essentially did just that in the article I discussed in my last post, albeit not in terms of the use of digital puzzles to provide automated assessments.)

I may also hear from psychometricians who will instantly recognize difficulties that would need to be overcome to put my proposal into practice. In fact, having talked with psychometricians, I am already aware of some issues that would need to be taken into account. Psychometrics is another of those disciplines of which I have some superficial knowledge but in which I have no expertise. But I have not yet encountered any reason why my suggestion cannot be made to work. (If I had, you would not be reading this article.)

To my mind, the really challenging obstacle is for the mathematics education establishment to accept, and then adopt, the T-model. Fifty years experience as a professional mathematician (the first fifteen or so in abstract pure mathematics, the remainder in various applied fields) has left me in no doubt that the T-model is not only perfectly viable, it is far superior to the “broad curriculum” approach we currently use, often referred to (derisively, but justifiably) as “a mile wide and an inch deep” education. But I am not in a position to mandate educational change. Nor, frankly, have I ever wanted to work my way into a position where I could have such influence. I like doing and teaching math too much! Instead, I am using what platform I have to put this suggestion out there in the hope that those who do have influence might take up the idea and run with it.

Of course, I can keep repeating my message. In fact, you can count on me doing that. :)

It's high time to re-focus systemic mathematics education - and change the way we assess it

“In math you have to remember, in other subjects you can think about it.” That statement by a female high-school student, was quoted by my Stanford colleague Prof Jo Boaler in her 2009 book What's Math Got To Do With It? I took it as the title of my June 2010 Devlin’s Angle post, which was in part a review of Boaler’s book. In a discussion peppered with quotations similar to that one, Boaler describes the conception of mathematics expressed by the students in the schools where she conducted her research. To those students, math was a seemingly endless succession of (mostly meaningless) rules to be learned and practiced. Among the remarks the students made, are (the highlighting is mine):

“We're usually set a task first and we're taught the skills needed to do the task, and then we get on with the task and we ask the teacher for help.”

“You're just set the task and then you go about it ... you explore the different things, and they help you in doing that ... so different skills are sort of tailored to different tasks.”

“In maths, there's a certain formula to get to, say from A to B, and there's no other way to get it. Or maybe there is, but you've got to remember the formula, you've got to remember it.”

Given that is their experience of mathematics, there is no surprise that many students that are taught that way give up and bail out at the first opportunity. In fact, a more natural question is, “Why do a few students enjoy math and do well in it, answering questions at the board with seeming ease.”

The answer is, the students who do well in math and enjoy it, are doing something very different from the activity described in the above quotes. Indeed, one of the things that attracts students to math is that it is the subject where you have to learn the least number of facts or methods, and can spend most of your time in creative thinking. Those “good” students have discovered that, in the math class, there is relatively little you have to remember; most of the time, you can wing it, and you’ll do just fine. It’s not about learning a wide range of formulas and special techniques, the trick is to learn to think a certain way.

For the few who know the “one big trick” professional mathematicians rely on, math class is an engaging and enjoyable creative experience. How those few get to that point seems to be exposure to an inspiring teacher at some point, hopefully before the rot sets in and the student has been completely turned off math, or perhaps some other fortuitous event. Absent such a stimulus, though, it’s no surprise that when fed a steady diet of math classes focused on mastering one concept, formula, or special technique after another, the majority sooner-or-later give up, and simply endure it (in bored frustration) until they are through with it.

Which brings me to the mathematics Common Core State Standards, rolled out in 2009 to guide developments in education required to meet the changing environment and needs of the 21st Century.

If you go onto the CCSS website, you will find a large database of specific standards items, one such (which I picked at random) being

CCSS.MATH.CONTENT.5.MD.C.5.   Relate volume to the operations of multiplication and addition and solve real world and mathematical problems involving volume.

Parsing out the reference code for this particular standard, it relates to Grade 5, Measurement & Data, Geometric measurement: understand concepts of volume, item 5.

It is important to realize that the Common Core is not a curriculum, nor does it stipulate how any topic should be taught. It is exactly what its name indicates: a set of standards that educators should aim to meet at each stage. But it is tempting to view it as a list of specific topics that a teacher should cover one after another. (Tempting, but not easy if you are working from the website since it isn’t presented as a list. I suspect that is deliberate.) If you do that, then there is an obvious danger that the result will be a continuation of the approach to math education that students experience as a process of learning one little trick after another, and you are back with the situation Boaler catalogued.

But those individual CC items are the terminal-nodes on a branching tree that has a regular structure, and it is in that structure that you see not only order but just a handful of basic principles. It is those basic principles that should guide math instruction. There are just eight of them. They are called the Common Core State Standards for Mathematical Practice. Here they are:

1. Make sense of problems and persevere in solving them.
2. Reason abstractly and quantitatively.
3. Construct viable arguments and critique the reasoning of others.
4. Model with mathematics.
5. Use appropriate tools strategically.
6. Attend to precision.
7. Look for and make use of structure.
8. Look for and express regularity in repeated reasoning.

Those eight principles (the website elaborates on each one) constitute the core of the mathematics Common Core. They encapsulate the key features of mathematics learning essential for anyone living or working in today’s world. Notice that there is nothing about having to learn specific facts, formulas, or techniques. The focus is entirely on thinking.

The same is true of the specific items you will find in the rest of the Common Core website. When you drill down, you will find targets to aim for in order to develop thinking, following those eight principles at each grade level.

When mathematics is taught as a way of thinking, along the lines specified in those eight Common Core principles, then along the way, a student will in fact pick up a whole range of facts, and meet and learn to use a variety of formulas and techniques. But the human brain does that naturally, as an automatic consequence of lived experience. We are hardwired that way!

In contrast, learning becomes hard when presented as a sequence of items to be learned and practiced one by one, each in isolation, based on the false premise that you must first learn the “basics” before you can “put them together” to form the whole. The moment you realize that mathematics is about process rather than content, about doing rather than knowing, the absurdity of the “must master the basics first” philosophy becomes apparent.

Notice I am not saying “the basics” are irrelevant. Rather, they are picked up far more easily, and in a robust fashion that will last a lifetime, by using them as part of living experience. For sure, a good teacher can speed the process up by helping a student recognize the used-all-the-time basics, and maybe provide instruction on how they can be used in other contexts. But the focus at all times should be on the thinking process. Because that’s what mathematics is!

If the above paragraph sounds a bit like learning to ride a bike, then all to the good. A child learning to ride a bike will acquire a good understanding of gravity, friction, mechanical advantage, and a host of other physics basics. An understanding that a physics teacher can use to motivate and exemplify lessons in those notions. But no one would say that you cannot learn to ride a bike until you have mastered those basics! Think of doing math as a mental equivalent to riding a bike. (I wrote about this parallel in Devlin’s Angle before, in my March 2014 post. My final point there was somewhat speculative, but as a mathematician who also rides bikes, I claim that the overall parallel between the two activities is very strong and illuminating.)

Now comes the point where I part company with the CCSS, and indeed much of the focus in present-day American mathematics education and standardized math assessment.

Let me ease myself in by way of my cycling. I learned to ride a bike as a child and used a bike to get around throughout my entire childhood up to graduation from high school. I then hardly ever got on a bike again until I was 55 years old and my knees gave out after a quarter century of serious running, and I bought my first (racing-style) road bike. For the first twenty minutes or so, I felt a bit unsteady on my new recreational toy, but I did not need to seek instruction or help in order to get on it that first time and ride. The basic bike-riding skill I had mastered as a small child was still there, available instinctively, albeit a bit rusty and in need of a bit of adjusting for the first few minutes.

Moreover, when I started riding with a local club, my fellow riders gave me lots of tips and advice that made me able to ride more safely and at higher speeds. Some of what I learned was not obvious, and I had to practice. It was not the same as riding a city bike at low speed.

Likewise, when I bought, first, a mountain bike, and then a gravel bike, I had to take my basic bike-riding ability and transfer it to a different device and different kinds of terrain, and, in each case, once again learn from experts how to make good, safe use of my new machine.

The point is, they were all bicycles and it was all cycling. So too with mathematics. Once someone has mastered — truly mastered — one part of mathematics, it is relatively easy to master another. Yes, you will need to learn some new things, including a new vocabulary, some new techniques, and likely a new ontology, and yes you will almost certainly benefit from (and possibly need) help, guidance, and advice from experts in that new area of math. But you already have the one key, crucial ability: you can think like a mathematician.

In terms of learning mathematics, what this means is that it is enough to devote considerable effort to genuinely mastering just one topic — say elementary arithmetic — and then spending some time going through the process of branching out from that one area to a number of others (perhaps algebra, geometry, trigonometry, and probability theory).

In terms of mathematics assessment, it means that it is enough to test students’ mathematical thinking ability focused on just one topic, and then test to see if they have sufficient knowledge of a number of other topics. The advantage of approaching assessment this way is that, at least with current assessment methods, testing thinking is time-consuming and expensive, since it requires a small army of trained human assessors to grade solutions to open-ended questions, often “complex performance tasks,” whereas assessing breadth of knowledge can be done with a variety of machine-grades tests. So there are significant savings in cost and time if assessing thinking ability is done separately on just one topic. Which is absolutely all that is required, since the ability to think mathematically is just like the ability to ride a bike — once someone can ride one kind of bike, they can, with perhaps some adjustment, ride any kind. There is absolutely no need to test for that. As a result of natural selection of many thousands of years, humans can all do it.

The only question that remains is what mathematical topic should we focus on to develop the ability to think mathematically — including, I should add, an understanding of the importance of the precise use of language, the ability to handle abstraction, the need for formal definitions, and the nature and significance of proof.

Well, why not once again take our cue from how most of us learn to ride a bike. What is the equivalent of our first child’s bicycle? Elementary arithmetic.

What’s that you say? “There isn’t enough meat in elementary arithmetic to learn all you need to know about thinking mathematically, with all those bells and whistles I just mentioned.” Think again. Alternatively, check out the article written by mathematics educator Liping Ma in the article she published in the November 2013 issue of the Notices of the American Mathematical Society, titled A Critique of the Structure of U.S. Elementary School Mathematics.

Based on her experience with mathematics education in China, Ma argues forcefully, and effectively, that there is more than enough depth and breadth in “school arithmetic” (as she calls it) to fully develop the ability to think mathematically. True, in the West we don’t teach elementary arithmetic that way; indeed, we present it as a series of basic number facts to be memorized and algorithms to be practiced, as in the Boaler critique. We do so, at some speed I should add, in large part because we are in so much of a hurry to move on to all the other mathematical topics that someone at some time in the past declared were “essential” to learn in school. But as many have pointed out over several decades, the result is that our mathematics curriculum is “a mile wide and an inch deep”, resulting in students leaving school believing that “In math you have to remember, in other subjects you can think about it.”

In the June 2010 Devlin’s Angle post I referred to earlier, where I talked about Boaler’s then-new book, I mentioned Ma, and said I agreed with her argument about using school arithmetic as the topic to develop the ability to think mathematically. I still do.

I also think school arithmetic provides the one topic you need to assess mathematical thinking ability — regardless of whether you are assessing student learning, teacher performance, or district system performance. Given that, assessment of whatever breadth is required can be done relatively easily and cheaply. Because the thinking part is essentially the same, the assessment of the breadth can focus on what is known (rather than what can be done with that knowledge).

And (of course), one really valuable benefit of focusing on school arithmetic is that it provides as level a playing field as you can hope for, with elementary arithmetic the one mathematical topic that everyone is exposed to at an early age.

In a future post, I’ll take this topic further, looking at the implications for teaching, the educational support infrastructure (including textbooks), the effective use of modern technologies, and the educational implications of those technologies.

Also, as the title makes clear, the focus of this article has been systemic mathematics education, the mathematics that states decide is essential for all future citizens to learn in order to survive and prosper and contribute to society. There is a whole other area of mathematics education, where the focus is on the subject as an important part of human culture. That’s actually the area where I have devoted most of my efforts over the years, writing books and articles, giving public talks, and participating in radio and television programs. So I’ll leave that for other times and other places.

What good is math and why do we teach it?

This month’s column comes in lecture format. It’s a narrated videostream of the presentation file that accompanied the featured address I made recently at the MidSchoolMath National Conference, held in Santa Fe, NM, on March 27-29. It lasts just under 30 minutes, including two embedded videos.

In the talk, I step back from the (now largely metaphorical) blackboard and take a broader look at why we and our students are there is the first place.

Download the video here.