Last week, I ended with a sample exam I might give a calculus course which included both Skills problems and Conceptual problems.  Before presenting the final installment of this series on assessment, I thought I’d take a few moments to discuss the genesis of this exam format.

Again, the assumption here is that we are working in a more traditional system, where students must be assigned grades, and these grades must in large part be based on performance on exams.

Given IMSA’s statements about the advanced nature of their curriculum, I had concerns about the fairly traditional exams we gave in mathematics.  In my mind, there was little to distinguish our exams from those given in any other rigorous calculus course.

The reason given to me by other mathematics faculty was that there just wasn’t time in a roughly hour-long exam to assess conceptual understanding.  I wasn’t convinced, and I started thinking of an alternative.  I did agree, however, that it wouldn’t work to have a conceptual question on an exam which might take half the exam period for a significant fraction of the students to complete.

What I finally settled upon was including a range of conceptual problems for which students only needed to provide a reasonable approach to solving.  If you chose a conceptual problem which happened to be centered on a student’s weakness, you wouldn’t be able to assess a broader conceptual understanding.  And if you insisted the problem be worked completely through, you encountered significant time constraints.

I’d like to share one last anecdote.  I recall a parent visitation day one Saturday, which happened to be the day after I gave a calculus exam.  Two of the parents approached me after the session and told me how much their son or daughter enjoyed my exam.  This indicated to me that for the student who can perform the routine procedures easily, they want to be challenged to think outside the box, and indeed they thrive on such challenge.  Shouldn’t we, as educators, find ways to stimulate all of our students, rather than be content with having students in the middle earn their B’s, making sure the struggling students earn their C’s, and relegating the very capable students to a sustained boredom?

And now for the last installment….

“Where does this bring us? Here are some key points as I see them.

1.  We should move away from assigning grades punitively.
2. We should reconsider the “point'”system of evaluating student performance.  Referring to the TIMSS (Third International Mathematics and Science Study):  “In our study, teachers were asked what ‘main thing’ they wanted students to learn from the lesson.  Sixty-one percent of U.S. teachers described skills they wanted their students to learn.  They wanted students to be able to perform a procedure, solve a particular kind of problem, and so on….On the same questionnaire, 73 percent of Japanese teachers said that the main thing they wanted their students to learn from the lesson was to think about things in a new way, such as to see new relationships between mathematical ideas.” (Stigler and Hiebert, The Teaching Gap, ISBN 0-684-85274-8, pp. 89-90.)  A point system reflects the assessment of procedural knowledge.
3. “We can think of all assessment uses as falling into one of two general categories — assessments FOR learning and assessments OF learning.”  (From an internal document distributed to mathematics teachers at IMSA.)  But why?  The distinction is artificial.  There are many other ways to compartmentalize assessments, such as timed/untimed, individual/group, skill/conceptual, procedural/relational, short-term/long-term, etc.  The main argument for focusing on the “for/or” distinction is its relationship to student motivation — but we are given no context for it.  I suggest that our typical IMSA student is highly motivated — certainly in relation to the average student in a typical high school classroom.
4. We should consider the assignment of letter grades in general.  Right now, it would be impractical to suggest that we have formal written evaluations of each student in each class.  But is it desirable?  And if so, what resources are necessary to support such a system?
5. We should discuss the assessment of problem-solving.

Will any of these suggestions help to illuminate the power of ideas?  I’m not sure.  With the current need to assign grades, and their current cultural meaning and importance — especially when it comes to applying to college — there will be the necessary compromises in the classroom.  I realize that many suggestions are of the “move away” rather than the “move toward” type.  But I suppose that if there is something I am moving toward, it’s giving students at all levels more of a BC Fast-Track experience regardless of the depth of content.

This means actively moving toward a classroom environment where earning good grades is subordinate to learning complex concepts.  Of course the two are not mutually exclusive — but I’d rather have students earn good grades because they learned, rather than learn in order to get good grades.

Of course many issues brought up in these remarks have been left hanging or only tentatively developed.  These brief comments are meant to suggest questions for discussion, not definitive answers.

I can’t resist ending with the following challenge from Maslow: “In order to be able to choose in accord with his own nature and develop it, the child must be permitted to retain the subjective experiences of delight and boredom, as the criteria of the correct choice for him. The alternative criterion is making the choice in terms of the wish of another person. The Self is lost when this happens.  (Maslow, source cited earlier, p. 58.)  Is it possible to create a mathematics curriculum which can survive this test of course selection?”

Thanks for staying with this series!  No, there is no simple resolution to any of the issues described in this essay.  But that doesn’t mean we shouldn’t be involved in a conversation about them….

Last week, I had ended with an interpretation for an A–D grading scale, shown below (here is a link to last week’s post for reference).

I remind you that this scale is not ideal; the purpose was to come up with some system of assigning grades which wasn’t punitive, but rather which motivated students to learn concepts rather than to avoid losing points on exams.

We continue with a discussion of how to use such a system in practice.

“Now let’s consider this in the context of an exam.  The first part of an exam is a skills portion, with, say, ten short problems of roughly equal length.  Expectations for this part of the exam [meaning a grade of B] are seven problems “essentially” correct, and four problems completely correct.  These expectations are written on the exam for students to see.

Are these expectations too low?  Perhaps.  But then an A [refer to the chart below] means eight problems “essentially” correct, with at least five completely correct. Of course we must ask what it means for a problem to be “essentially” correct — but when in doubt, err on behalf of the student.  (Students rarely suggest that their scores be lowered.)

Then grading is actually somewhat easier, and grades can be assigned as follows, with the abbreviations EC and CC meaning essentially correct and completely correct, respectively (for simplicity, a grade of C is assigned for all other cases not accounted for):

Now this eliminates the need for partial credit — but does require a judgment as to what “essentially correct” means.

This also makes grading much easier.  I would suggest that each problem be marked as “EC,” “CC,” or left blank.  Few comments, if any are necessary.  This is the approach I have taken in BC Fast-Track, and it encourages further learning as it leaves the student in the position of needing to work through their mistakes.

I would have students keep a section of their notebooks for exams and revisions, and there they can keep their reworked problems, should they choose to do so.  Then — as I did in BC Fast-Track — students could visit me periodically with their notebooks and I can take a look at their ongoing progress.  This “additional” work, if sufficiently well done, could boost their grade at the end of the semester.

I think this could have the same effect it did in BC Fast-Track — exams were easier and more enjoyable to grade.  But there were more discussions in my office about reworked exams and sources of error that were initiated by the students themselves, and these discussions were not about points, but about concepts.

Now what about the part of the exam which is intended to be more conceptual?  Let us suppose that there are three problems, roughly comparable in length, and of various difficulties.  Then grades might be assigned as follows:

More details about how this would fit in a classroom environment may be found in a later document [I cannot recall which document is being referred to here].  But this system allows for a more qualitative approach to grading.  Performance expectations are also clearer, but such expectations depend critically upon the nature of the problems given.  Moreover, grades are not assigned punitively, but the emphasis is on doing problems completely and correctly.

For an example, below could be a set of ten skills problems and three conceptual questions for a basic assessment on the rules of differentiation.  This would a 70-minute assessment.  Given expectations for completely correct problems, I think this is reasonable.

Skills questions:

1.  Evaluate

2.  If find

3.  Find

4.  Using the quotient rule, find

simplifying as much as possible.

5.  Find the derivative of

6.  Find the equation of the tangent line to at

7.  Using a definition of the derivative, find the derivative of

8.  Assume that and are differentiable functions. Find

9.  Find the derivative of

10.  Let be the greatest integer function. Using the definition of the derivative, determine whether or not the derivative exists at

Conceptual questions:

1.  Using the product rule, find

Explain your result.

2.  Suppose that the line is tangent to both and Find and

3.  Suppose that is a differentiable function. Discuss the following limit:

Stay tuned for next week’s final installment of this series on assessment….

Today, I’ll continue with the discussion of assessment I began a few weeks ago.  Last week, I ended with the observation that “The performance of our students determines our expectations, rather than the other way around.”  Before continuing, I just want to make clear the context for that remark.  When I wrote this essay, IMSA was fully funded by the state of Illinois.

What would it look like to legislators — who approved our funding — if you brought in the top students (relative to their local regions), and then they regularly received low grades, or worse yet, flunked out?  Not good.  And because the admissions process was fairly involved, would this indicate a major flaw in this process?

Yes, some students couldn’t handle the high-pressure academic environment.  But I know that in several of my classes, a student needed to work hard not to earn at least a C.  In other words, showing up, handing in homework on time, and doing a reasonable amount of studying for quizzes and exams would usually guarantee a grade of at least a C.

Now is not the time to dive more deeply into this artificial adjustment of expectations, but I did want to mention that this issue is a significant one, even at presumably elite schools for mathematics and science students.

So to continue with the essay….

“Now there is a natural give-and-take between evaluating student performance and setting expectations.  And, of course, the above remarks are nothing but generalizations.  But they illustrate some of the important issues at hand, and may bear fruitful discussion.

Moving to more concrete issues, I believe that the assignment of letter grades on exams in BC Fast-Track [recall, this was the colloquial name for the Honors Calculus sequence] was, on the whole, successful.  Without going into unnecessary detail, the classroom environment was such that the assigned grades were meaningful to the students.  To give a few examples, I assigned a grade of A+ for truly outstanding work, perhaps only a half-dozen times throughout the entire semester.  The students knew this, and so that accolade truly meant something.

Moreover, an A meant something as well.  It was truly rewarding to see the real pride of a student who, used to earning grades in the B range, began to earn the occasional, or perhaps more frequent, A.  Admittedly, students who made it to the second semester were essentially guaranteed a grade of no lower than a B-.  But this seemed to make an A that much more meaningful.

So student exams had two letter grades on them — one for the skills portion of the exam, and one for the conceptual portion of the exam.  No points were assigned, and few comments were made.  Students were expected to rework problems on which they made errors.

I bring up this point because I think this system of assigning grades really did motivate students to learn calculus rather than accumulate points.  This is the critical issue: I suggest that the way we assign grades does little to disabuse many students that taking a mathematics course is about accumulating sufficiently many — or losing sufficiently few — points.

Let’s take a particular example.  The past few semesters, I stopped assigning half-points on assessments [the usual practice at IMSA].  I might forgive a sign error now and then, but too many on a single assessment would warrant a point or two off.

In the past, I simply considered a sign error as a half-point off.  And so it was.  But consider that without being able to occasionally perform fairly involved calculations, it is not possible to become a successful mathematician.  Attention to detail is as important in mathematics as it is in any number of other disciplines, and we try to develop that skill punitively — you don’t attend to detail, and we will take off points.

Of course one might argue that points are given for work well done — but any of us could, I think, agree that when discussing the grading of an exam, it’s how many points off for a particular type of error that is discussed as much as, or even more than, how many points are given for work correctly done.

And so the idea of “partial credit” is born.  Perhaps now is not the place to begin this discussion, but consider that a student might meet expectations (that is, earn a B) without ever having done an entire complex problem on an assessment completely correctly.  (Some teachers have even gone so far as to give no partial credit.  See On Partial Credit, Letter to the Editor, MAA Focus, February 2002, p. 17.)

Why this system of points and partial credit?  One may speculate as to its origins, and there is controversy even now about its use on standardized exams.  But I cannot help feeling that one function of partial credit is that it allows a teacher to defend the assignment of a particular grade.  “Every sign error is a half-point off.  That’s why you got a B+ instead of an A-.   I have to use the exact same scale for everyone in order to be fair.”

But doesn’t this simply shift the responsibility for the grade onto a rubric?  I suggest that many of us would feel competent to take a set of calculus exams — with names removed — and within five or ten minutes, separate out all the A papers.  Of course this is subjective — but no less subjective than saying that this problem is worth six points while another is worth ten, or that sign errors are a half-point off, unless, of course, the derivative of the cosine is taken incorrectly, in which case it’s a whole point.

Thus the assigning of points is no more “objective” than giving a letter grade.  As I’m sure that anyone who has graded a complex word problem based on an assignment of points can attest to.  Consider the student who has the entire procedure correct, but because of a few algebra errors, has no intermediate calculation correct.  The problem is worth ten points.  How many points should the student receive?

Well, of course, you say you’d have to see the problem first.  But I say, no.  The student receives a C.  Having no intermediate calculations correct demonstrates — regardless of what else — that the student has not met expectations.

So is it possible to avoid points altogether?  Perhaps.  Consider the following grading system:

The essay continues in the next installment of On Assessment with a discussion of how to implement this system in practice.

Last week, I had mentioned finding an essay I wrote about assessment in an Honors Calculus sequence I had designed.  I took some time to set the stage for this essay — so now it’s time to dive right in!

A caveat:  This essay was written in 2011.  I will attempt to keep as true to the original essay as I can, though I’ll edit for clarity and updated information/links, and will also [add commentary in square brackets].  So here goes!

“To educate is to illuminate the power of ideas.

This, of course, is an ideal — certainly not the last to be articulated, nor perhaps the highest. But putting practicality aside for the moment, how might we take on such a view of education?

Consider the example of planetary orbits.  Tying together history, technology, politics, physics, and mathematics through a discussion of Kepler’s laws is — yes — illuminating. Of course there is no single idea at play here, but consider this: Kepler was perhaps the first astrologer to become an astronomer.  He, in contrast to so many of his contemporaries, asked not only where a particular planet should be on some future date, but why it should be there.  In his mind, predicting the positions of celestial bodies for the purpose of casting horoscopes for royal personages was not enough — he wanted to know why. That, however, would have to wait until Newton and the application of calculus.  And the power of that idea!

There is no need to present further historical examples.  What is important, however, is to move beyond historical ideas and address the issue of the power of a student’s own ideas.  Two semesters of teaching Advanced Problem Solving [a course I taught at IMSA which emphasized the writing of original mathematics problems] have shown me that having students write original problems motivates them to learn.  They are timid at first — for how could they come up with an original problem?  But after some success, students are excited to create — and as a result, learn about a particular topic in a more profoundly personal way than they might have otherwise.  (Interestingly, the writing of original problems as a teaching tool is common in eighth-grade Japanese mathematics classrooms.  See The Teaching Gap by Stigler and Hiebert, ISBN 0-684-85274-8, pp. 6–41.)

Other assessments are more routine: students are informed that they need to be able to solve linear and quadratic equations, as well as graph linear and quadratic functions.  This is perhaps more manageable — the task is well-defined within narrow limits.  But it is rather different in nature than creating an original piece of work.

Abraham Maslow articulates a similar difference in a discussion of growth theory (see Toward a Psychology of Being by Abraham Maslow, ISBN 0-442-03805-4, p. 47).  Psychologically, why do children grow?  “We grow forward when the delights of growth and anxieties of safety are greater than the anxieties of growth and the delights of safety.”   From an educator’s point of view, this implies making the classroom a safe environment for creativity and exploration.

“The opposite of the subjective experience of delight (trusting himself), so far as the child is concerned, is the opinion of other people (love, respect, approval, admiration, reward from others, trusting others other than himself).”  (Maslow, same source, p. 51.)  Who comes to mind is the student afraid of enrolling in BC Fast-Track [the colloquial name of the Honors Calculus sequence at IMSA] because he or she might not earn an A.  [I did have a very capable student who was so concerned about grades, he opted to take the more traditional calculus sequence.  I actually met with his father, who wanted a guarantee that his son would earn an A in the course.  Given the nature of the course, there was naturally no way I could give such a guarantee.]  Or perhaps the student who shuns a difficult course and takes an easier one instead.  Grades are just that — opinions of other people.

The assignment of letter grades is not necessary for learning, but is merely practical for other reasons.  The assignment of letter grades does nothing to illuminate the power of ideas.

A quick Internet search reveals that the assignment of A–F letter grades is a fairly recent phenomenon, not making its way into high schools until the mid-twentieth century.  In the early twentieth century, grades of E (Excellent), S (Satisfactory), N (Needs improvement), and U (Unsatisfactory) were also used.

The question of whether a letter-grade system of evaluation is the best option for a school like IMSA is perhaps a worthwhile one (some argue against grades at all [this was the link, but it is now a dead end:  http://www.alfiekohn.org/teaching/fdtd-g.htm ]).  That is a question which cannot be answered in the short-term, if the question is considered relevant. So the question is this: given that letter grades need to be assigned this semester, what approach should be taken?

What do letter grades mean?

The only consensus I have heard, so far, is that earning a grade of A, B, or C means that a student exceeds, meets, or does not meet expectations.  Of course, this is reminiscent of E, S, and N.  Moreover, a grade of C is passing, so that a student receives credit for a course even if they do not meet expectations.

But this is not what the grades currently mean.  Essentially, points are assigned to hundreds of problems given throughout semester, whether on assignments, papers, quizzes, or exams — and an arbitrary weighted sum of these point assignments is converted to a letter.  (Of course letters may need to be converted to numbers so that they may be used by an online grade calculator to compute a number which is then converted back to a letter.)

What does this mean about meeting expectations?

Well, nothing really.  I would venture to suggest that “meeting expectations” currently means “enough A’s, but not too many C’s.”  Perhaps this is politically necessary, but expedient.  The performance of our students determines our expectations, rather than the other way around.”

 

While downsizing for a recent move, I decided to sort through dozens of folders (thousands of pages!) of old problems, notes, exams, essays, and other documents generated in the course of twenty-five years of teaching mathematics.  I found an essay written in 2011 which I though might be of some interest.

This essay might be considered a more thorough follow-up to The Problem with Grades, a satire on assessment practices which I’ve shared with you already.  The essential point in that essay was that how we typically assign grades is punitive:  begin with a given number of points (say, 100), punish students for making mistakes (subtract a certain number of points from 100), and then — magically! — the resulting number reflects a student’s understanding.  (If you think that this is in fact a really great way to assess a student’s learning, you probably should stop reading right here….)

The essay I’d like to share here was written in the context of teaching an Honors Calculus sequence I designed while at the Illinois Mathematics and Science Academy (IMSA).  So before diving in, let me take a few moments to set the stage.

We had quite a broad range of students at IMSA — and so when it came to teaching calculus, some students were bored by the “average” pace of the class.  Of course this happens in a typical classroom; the tendency is to teach to the middle, with the necessary consequence that some students fall behind, and others could benefit from a faster pace or more advanced problems.

Now mathematics educators are looking at ways to design classroom environments where students can work more independently, at their own pace, incorporating methodologies such as differentiated learning and/or competency-based learning.  These ideas could be the subject for wide-ranging (future) discussions; but in my case, I was working in an environment where all students did essentially the same work in the classroom, and assessments were fairly traditional.

In addition to creating a faster-paced, more conceptually oriented course, I incorporated two features into this Honors Sequence.  First, I had students write and solve their own original mathematics problems, as well as reflect on the problem-posing process.  I won’t say more about that here, since I’ve written about this type of assessment in a previous blog post.

Second, I experimented with a different way of assessing student understanding on exams.  Each exam had two components:  a Skills portion and a Conceptual portion.  The Skills portion was what you might expect — a set of problems which simply tested whether students could apply routine procedures.  The Conceptual portion was somewhat more challenging, and included nonroutine problems which were different from problems students had seen before, but which did not require any additional/specialized knowledge to solve beyond what was needed to solve the Skills problems.  (As part of the essay I will share, examples of both types of problems will be included.  I also discuss these ideas in more detail in a previous post, On Grading.)

Moreover, I graded these problems in a nonstandard say:  each problem was either Completely Correct (CC), meaning that perhaps aside from a simple arithmetic error or two, the problem was correctly solved; Essentially Correct (EC), meaning that the student had a viable approach to solving the problem, but was not able to use it to make significant progress on the problem; and Not Correct (NC), meaning that the approach taken by the student would not result in significant progress toward solving the problem.

Letter grades (a necessity where I taught, as they are in most schools) were assigned on the basis of how many problems were CC and EC on the Skills/Conceptual portions.  While I won’t go into great detail here, the salient features of this system are, in my mind:

1.  Earning an A required a fair number of problems CC; in other words, a student couldn’t get an A just by amassing enough partial credit on problems — there needed to be some mastery;
2.  Earning an A required as least some progress on the Conceptual problems; typically, I would include three such problems, which would be assessed more leniently than the Skills problems.  In order to earn an A, a student would need to make some progress on one or two of these.  I felt that an A student should be able to demonstrate some conceptual understanding of main ideas;
3. A student could earn a B+ just by performing well on the Skills problems (or perhaps an A- for performing flawlessly), but an A was out of reach without some progress on the Conceptual problems;
4. The approach was not punitive:  students were assigned CC/EC based on their progress toward the solution to a problem, not how far they fell short of a solution.

Such a system is not new; I have talked with colleagues who used a 0/1/2 system of grading, for example.  Whatever the format, the approach is more holistic.  And given that problems in mathematics typically admit more than one solution, the idea of creating a point-based rubric for all possible solution paths does, in some real sense, border on the insane.

I should also add that our instructional approach at IMSA was essentially inquiry-based.  While such a classroom environment is conducive to having students write original problems and using alternative assessment strategies, it is not strictly necessary.  I have incorporated both the elements described above in a more traditional classroom setting, but with varying degrees of success.  That discussion is necessarily for another time.

So much for a brief introduction!  I would also like to comment that teaching this Honors Sequence was perhaps my most enjoyable and successful teaching experience in the past few decades.  Students who completed the two-semester sequence left thinking about mathematics in a fundamentally different way, and I stay connected with a few of these students many years later.

Next week, I’ll introduce the essay proper, adding commentary as necessary to flesh out details of the course not addressed here, as well as making remarks which reflect my teaching this Honors sequence in the years since the essay was written.  Until then, I’ll leave you with the opening words:

“To educate is to illuminate the power of ideas…..”