Archive for the ‘Assessment’ Category

Abandon the Red Pen!


I have a new Teaching with Tech column published in MAA FOCUS about digital grading. In particular…

  • Why would you want to grade papers digitally?
  • What kind of hardware/software would you need?
  • How do you manage the files and workflow?
  • How to use custom stamps to give more detailed feedback (more details on that one on an older blog post)

Abandon the Red Pen, MAA FOCUS, October/November 2011

Possibly Related Posts:


Share

Custom Stamps in Adobe Acrobat for Digital Grading


Many people have asked me to give a tutorial on creating custom stamps in Adobe Acrobat for paper grading.  There’s no reason why you couldn’t do something similar in other programs by pasting images into files, but there’s no doubt that the ease of one-click access to custom stamps is a nice feature of Adobe Acrobat.

Step One:  Create the content of the Custom Stamp

You can use any program on your computer to create the content: MathType, LaTeX, Wolfram Alpha, Mathematica, Maple, Sage, Word, Journal, etc.  Write the content and try to make it somewhat compact in width (aim for a square or squarish-rectangle rather than a long skinny rectangle).

Step Two: Capture an image of the Content

Use any screen capturing program to capture an image of your content.  You want to use one that has a “snipping” feature so that it’s not a screen capture of the entire screen.  Just capture the content you want in the stamp.  I usually use Jing or SnagIt to do this, although there are certainly many other options.

Step Three (optional): Make a Border

If I am making a longer comment, I like to put a border around my “stamp” content to make it clear that this was something that was added in the grading and not part of the original content of the exam or assignment.  Even free programs like Jing have the ability to add a rectangular “border” box on the image.  Save the file.

Step Four: Create the Custom Stamp

In Adobe Acrobat, open the stamp menu and choose “Create a Custom Stamp.”  Browse to find the image file you’ve created (Adobe defaults to finding PDF files, but you can use the drop-down menu to choose from other file formats).

 

 

You’ll find it helpful to have stamp categories (Limits, Derivatives, Integrals, Exam 2, etc.) to make stamps easy to find.

Step Five: Use the Custom Stamp (over and over and over and over)

At this point, you should be able to use the stamps by choosing them from Comments & Markup Tools –> Custom Stamps.


Once the custom stamp is inserted in a PDF document, it can be resized and moved all over the page.  You can use a custom stamp multiple times in the same document.

And now that notation error that requires you to explain in a lengthy comment is not such a burden to correct anymore.  I use custom stamps to explain the difference between d/dx and dy/dx, to insert missing limit notation, to explain the difference between a derivative and a differential, to explain how to rewrite an improper integral … once you can just stamp the comments, the explanations can be as clear as you want them to be.

Possibly Related Posts:


Share

Shifting Assessment in a World with WolframAlpha


I let my students use Wolfram Alpha when they are in class and when they are doing their homework (um, how would I stop them?).  Because of this, I’ve had to shift how I assess on more formal assignments.  For the record, it’s the same adjustment you might make if you were using ANY kind of Computer Algebra System (CAS).

The simplest shift is to stop asking for the answers to problems, and just give the students the answers.  After all, they live in a world where they CAN easily get the answers, so why pretend that it’s the answers that are important?  It’s the mathematical thinking that’s important, right?  Giving the students the answers turns problems into “proofs” where the evaluation (grade) is based on the thought-process to get from start to finish. It also eliminates the debate about whether to award points for a correct answer with no correct process.

Here are two examples of problems from a recent Calculus exam (old and new wording).

I wish I had thought to do this years ago, because students who insist on just doing the “shortcut” (and not learning what limits are all about) now have nothing to show for themselves (the answer, after all, is right THERE).

Again, a student that knows the derivative rules might get the right answer, but the right answer is now worth zero points.  The assessment is now clearly focused on the mathematical thinking using limits.

Another reason that I really like this is that it allows students to find mistakes that they are capable of finding “in the real world” where they can quickly use technology to get an answer.  They are now graded solely on their ability to explain, mathematically, the insides of a mathematical process.

Wolfram Alpha also allows me to pull real-world data into my tests much faster.  Here’s a question about curve shape (the graph is just a copy/paste from W|A):

If you haven’t begun to think about how assessment should change in a world with ubiquitous and free CAS, you should.  You don’t have to change all your problems, but I think some of them should change.  Otherwise, we’re just testing students on the same thing that a computer does, and that doesn’t sit well with me.  If you can be replaced by a computer, you’re likely to be replaced by a computer.  Let’s make sure we’re teaching students how to think mathematically, not how to compute mathematically.

Possibly Related Posts:


Share

Before You Give That Exam


Students lose SO many exam points because they just don’t read the directions and pay attention to details.  On the first exam, they usually discover this … but they don’t REMEMBER it for the other exams.

This is a very simple exercise that takes about 1 minute at the beginning of the test.

Just have the students repeat after you:

I promise … to read all the directions … for all the problems on the exam …

And if I finish early, … I promise … to RE-read all the directions … to make sure I haven’t missed some detail … or forgotten to come back to some question I skipped.

I understand that … it is not important to finish quickly … it IS important to demonstrate what I know … and once the points have been lost … the points cannot be regained.

Believe it or not, this results in a remarkable number of students that stay until the bitter end, making sure that they have been careful and answered every question completely.

Possibly Related Posts:


Share

Student Conceptions of Mathematics


sotl_blog_button1

Do you ever get the feeling that you’re not making any real progress with your students?  Sure, they pass tests and progress through the courses (well, most of them), but have you ever just had the uneasy feeling that they really don’t get what math is all about?  Suppose you were to ask your students the following question:

Think about the math that you’ve done so far.  What do you think mathematics is?

What do you think they would tell you?

Well, in 1994, a research group from Australia did ask 300 first-year university students this question (Crawford, Gordon, Nicholas, and Prosser).  They identified patterns in the responses, and classified all 300 responses into categories of conceptions in order to explore the relationships between (a) conceptions of mathematics, (b) approaches to learning mathematics, and (c) achievement.

If you want all the gory details about the study, you’ll have to track down the journal article, but let me attempt to summarize their findings.

Students’ conceptions of mathematics fell into one of five categories (Crawford et al., 1994, p. 335):

  1. Math is numbers, rules, and formulas.
  2. Math is numbers, rules, and formulas which can be applied to solve problems.
  3. Math is a complex logical system; a way of thinking.
  4. Math is a complex logical system which can be used to solve complex problems.
  5. Math is a complex logical system which can be used to solve complex problems and provides new insights used for understanding the world.

The first two categories represent a student view of mathematics that is termed “fragmented” while the last three categories present a more “cohesive” view of mathematics.  Note that the terms fragmented and cohesive are well-used throughout the international body research.  The categories above, as you may have noticed, form a hierarchical list, with each one building on the one above it.

Here’s where this study gets (even more) interesting.  The researchers also asked students about how they studied (changing the language slightly here to “americanize it” a bit):

Think about some math you understood really well. How did you go about studying that?  (It may help you to compare how you studied this with something you feel you didn’t fully understand.) How do you usually go about learning math?

Again, the researchers went through a meticulous process of categorization and came up with five categories (Crawford et al., 1994, p. 337):

  1. Learning by rote memorization, with an intention to reproduce knowledge and procedures.
  2. Learning by doing lots of examples, with an intention to reproduce knowledge and procedures.
  3. Learning by doing lots of examples with an intention of gaining a relational understanding of the theory and concepts.
  4. Learning by doing difficult problems, with an intention of gaining a relational understanding of the entire theory, and seeing its relationship with existing knowledge.
  5. Learning with the intention of gaining a relational understanding of the theory and looking for situations where the theory will apply.

Again, these five categories were grouped, this time according to intention, into two general categories: reproduction and understanding.  In the first two approaches to learning math, students simply try to reproduce the math using rote memorization and by doing lots of examples.  In the last three categories, students do try to understand the math, by doing examples, by doing difficult problems, and by applying theory.  Other researchers in this community have seen similar results on both general surveys of student learning and on subject-specific surveys and have termed this to be surface approach and deep approach to learning (see Marton, 1988).

Still reading?  Good.  Remember my first question? Do you ever get the feeling that you’re not making any real progress with your students? Let’s answer that now.

Here’s how the conceptions of math and the approaches to learning math correlated in this study (Crawford et al., p. 341):

conceptions_of_math

Did you catch that?  Look at how strongly conception and approach correlates.  It’s probably what you’ve always suspected, deep down inside, … but there’s the cold, hard, proof.

Of course, all this is not so meaningful unless there’s a correlation with achievement.  At the end of their first year, the students’ final exam scores were compared to the conceptions and approaches to mathematics (again, for technical details, get the article). The researchers made two statistically significant findings:

  1. Students with a cohesive conception of math tended to achieve at a higher level (p < .05).
  2. Students with a deep approach to learning math tended to achieve at a higher level (p < .01).

Okay, so where does this leave us?  Well, we don’t have causation, only correlation (at least, that’s all we have from the 1994 study).  However, Crawford, Gordon, Nicholas, and Prosser were nice enough to use their research to develop a survey inventory that we can use to measure students’ conceptions of mathematics (1998).  The 19-item inventory (5-point Likert scales) has been thoroughly tested for validity and reliability, and can be found in their 1998 paper, University mathematics students’ conceptions of Mathematics (p. 91).

Suppose you want to try something innovative in your math class, but you don’t know how to tell if it works.  You could, at the very least, try to measure a positive change on your students’ conceptions of math (there are other ways to measure the approach to learning, but this is already a long blog post and you’ll have to either wait for another week, or view my presentation How can we measure teaching and learning in math?).

measure_tlm

To give the Conceptions of Mathematics Questionnaire (CMQ) would take approximately 10 minutes of class time (you should ask for permission from Michael Prosser before you launch into any potentially publishable research).  This would give a baseline of whether students’ conceptions are fragmented or cohesive.   If you were to give the survey again, at the end of the semester, you would be able to see if there is any significant gain in cohesive conceptions (or loss of fragmented conceptions).

So, I have permission (I met Michael Prosser this summer when he was at a conference in the U.S.), and I’m going to use this in all my classes starting next week.  I figure that when I start to see major differences on those pre and post-semester CMQ inventories, that I’m doing something right.  If I’m not seeing any gain in cohesive understanding of mathematics, then I’m going to keep changing my instructional practices until I do.

Papers that you will want to find (and read!):

Crawford, K., Gordon, S., Nicholas, J., & Prosser, M. (1994). Conceptions of Mathematics and how it is learned: The perspectives of students entering University. Learning and Instruction, 4, 331-345.

Crawford, K., Gordon, S., Nicholas, J., & Prosser, M. (1998, March). University mathematics students’ conceptions of Mathematics. Studies in Higher Education, 23, 87-94.

Possibly Related Posts:


Share

How to Grade a Student Blog


tlda_blog_buttonLast semester I began using learning blogs as one of the assignments for Math for Elementary Teachers.  It was the first time I have ever used blogs as a graded student learning assessment, and I didn’t really know what to expect out of the students.  Would they all have created blogs before? [no]  Would they understand intuitively how to make hyperlinks, load in images, and embed videos? [no, no, and no] Would they write naturally in a conversational tone (in the style of most blogs)? [yes]  Would they make their blog posts two or three times a week (as directed) or would they cram them all in during the last couple days? [some of both]

Overall, I was thrilled with the results.  The students reflected on their learning, both in class and out of class.  They found and shared games, videos, articles, and vocabulary sites that they found on the web.  Some of them acted as a class reporter, summarizing what was covered in class each day (with their own personalities coming through).  Before you read the rest of this post, you might want to browse a few of their blogs to get an idea of the variety or writing and styles.

So let’s just say that this first time using blogs was a learning experience for both my students and for me.  I drafted a rubric for grading the blogs, and stuck to it all semester.  However, I realized that both the clarity of the assignment and the specificity of the rubric needed to be improved for “Round Two” (starting next week).

During the last round of blog grading, I revised my old rubric to try and tighten up the quality of the results.  Here are the specifics of the assignment now.

Set up a blog using Blogger or WordPress.  You should make at least six blog posts of at least two paragraphs each, using appropriate spelling and grammar.  The mathematics in your posts should be correct.  Blog posts should focus on what you have learned, what you’ve struggled with, or what you’ve found to help you learn.  Posts can discuss learning in class or out of class, but must relate to the current topics we are covering in the unit.  You should not refer to specific chapter or section numbers in your blog posts, and if you mention an activity from class, please use enough detail that a 3rd party reader would understand it.  Here are some specific details:

  • Blog posts should be spaced apart (not all at the last minute).
  • Your blog should include an appropriate  title (not just Maria’s Blog)
  • Your blog should include a profile (picture and brief bio). This can be fictional if need be.
  • Your blog should contain a “blogroll” with five of your favorite educational blogs.
  • Your blog should contain a list of tagged topics or categories.
  • Your blog should contain four images (or embedded videos) and should contain at least six links to web resources that you’ve found yourself.
  • Links to web resources should be properly “clickable” within the text of the post (not just a pasted URL).
  • Each post should be tagged with appropriate keywords.
  • You should make at least six comments on the blog posts of other students.

I think that the nature of the blog (what to write about) needs to stay as open as possible, but the fine detail of the assignments is difficult to assess if the quality of blogs varies wildly.   If you choose to try an assignment like this, I highly recommend a table-style rubric (like the one below) to keep track of where you are assigning points.

learn_via_blog_rubricI also found it helpful to use a screen-capture program (I used Jing and SnagIt) to make grading comments about specific blog posts (because, of course, you should not comment those in on a public blog site).

One last tip:  About halfway to the deadline, I give every student feedback on how they are doing so far.  I gently remind them about details that they might have forgotten so that they have time to correct or regroup.  I’ve found this results in immediate improvement in the blogs and is well worth the effort.  I use quick 1-3 minute Jing videos to give the feedback most of the time.

Note: You can see the rest of the learning projects and a “big picture” idea of how I fit all this in (timewise) by reading Transforming Math for Elementary Ed.

Possibly Related Posts:


Share

Can we Teach Students to Understand Math Tests?


A few weeks ago, I gave a test where the grades were less than stellar. Whenever this happens, I try to sit down and reflect on whether the poor test grades were a result of something I did differently in class, a poorly written test, or a result of poor studying habits. After careful reflection and analysis of my own, I was pretty sure that this was the result of lack of studying (a theory which was verified … later in this blog post).

I was dreading the task of passing back these tests and prepared myself for an onslaught of questions aimed at trying to discredit the test (or the teaching).  Then I got one of those great last-minute ideas that come to you right before you walk in to face the students.  Maybe I should let them “pick apart” the test BEFORE they see their own tests.  The class in question is Math for Elementary Teachers (MathET) and I figured that a detailed test analysis would not be an inappropriate topic for us to spend class time on.

First, I made some blank copies of the test (enough for each group to have one).  I also created a handout with every single learning objective and assignment that I had given the students for each of the sections on the test (these are all available in their Blackboard shell, but  I compiled these in a paper-based handout that was 3 pages single-spaced). Here is what one section looked like:

5.1 Integers
• Describe operations on signed numbers using number line models.
• Demonstrate operations on signed numbers using colored counter models.
• Explain why a negative times a negative is a positive.
• Add, subtract, multiply, and divide signed numbers (integers).
• Know the mathematical properties of integers (closure, identity, inverse, etc.)
• Complete #1, 3, 7, 9, 15, 17, 19, 21, 23, 25, 37
• Be able to model addition, subtraction, and multiplication of signed numbers using a number-line model
• Be able to model addition, multiplication, and division of signed numbers using a colored counter model (why not subtraction? because subtraction is really the addition of a negative – treat it so)
• Read the blog posts about why a negative times a negative is a positive and be able to paraphrase at least two arguments in your own words.
• Study for your Gateway on Signed Numbers (be able to add, subtract, multiply, or divide signed numbers)
• Play with Manipulative: NLVM Color Chips Addition
• Play with Manipulative: NLVM Circle Game

When we met in class, I counted the class into groups of 3 students each. Each group received a copy of the learning objectives & assignments (by section) and a blank copy of the test. All of these were un-stapled so that the group could share and divide up the pages as they wanted.

Task #1: Look at all the objectives and assigned tasks/problems. Determine where these objectives, tasks, and problems showed up on the exam. (20-30 min)

Objective: Make the connection between what I tell them they need to learn/do and what shows up on the test.

Task #2: What didn’t show up on the exam? We discussed why these objectives might have been left off (for example, maybe it was not something that I emphasized in class) (5 min)

Objective: Make the connection between what is likely to show up on an exam and what is not likely to show up (with the caveat that any of the learning objectives are really fair game).

Task #3: Make a chart that shows how the points for each test question were distributed between the sections that were covered on the exam. We then compared the results from each group and compared these results to my analysis of the point-distribution for the test. (5-10 min)

Objective: Clearly see that it is necessary to study ALL sections, not just a couple of them.

Task #4: I passed back the individual tests. Each student was given three questions to answer as they looked through their tests. 1. Where were the gaps in your knowledge? 2. What mistakes should you have caught before turning in your test? (read directions more carefully, do all the problems, etc.) 3. What can you do to better prepare for the NEXT test?  I collected these, made copies, and passed them back to the students the next class. (10 minutes)

Objective: Take responsibility for your own studying.

It was here that students surprised me by being honest on Question #3. Most of them confessed that they had not studied at all, but now realized that they needed to start studying. I cannot help but wonder whether I would have gotten the same result if I had simply passed back the tests with no analysis.

Task #5: At the same time the students were looking over their own exam, I passed around one more blank copy of the test and asked students to write their score  for each problem (no names) on that problem so that we could see what the score distributions looked like.  When this was done, I placed the pages on the document camera one-by-one so that they could see the scores problem by problem.

Objective: To show the students that for many questions, students either get the question almost completely right, or completely wrong (you know it or you don’t).

Task #6: We had already covered the first section of the next unit, so I had the students begin a set of “How to start the problem” flashcards. On the front of the flash card, they wrote a “test question” for the new unit. On the back of the card, they wrote some tips for starting the problem and details they might otherwise forget.

Objective: To begin to see tests from the perspective of a test-writer instead of a test-taker.

Take-home assignment: I told each student that they must come to the next test with at least 5 flash cards per section (35+ flash cards). I suggested that a great way to study would be to swap cards with each other and practice with someone else’s questions. I checked to see if they carried through (although I assigned no consequences if they didn’t). All but two carried through. Anyone want to guess how those two fared?

Results: I just finished grading the latest test (signed numbers, fractions, and decimals – not easy topics), and the test results were almost all A’s and B’s (instead of C’s and D’s on the previous test). Most students left the test with a smile on their face, and several finally got the “A” they had been trying for all semester.

Three questions for you, my busy and wise readers:
1. Will the students persist in better study habits now? For example, will they study like this for the next exam?
2. Would this detailed test analysis have had the same effect if the class hadn’t just crashed & burned on a test?  (i.e. is it necessary to “fail” first)
3. Was this a good use of class time? Would it be good use of class time in a different math course, like Calculus?

Possibly Related Posts:


Share

Assess Math Study Skills


Last week I was in Denver for a 1-day math conference and one of the speakers was Paul Nolting (who has written several books about math study skills).

One of the resources that he passed along to us was an online Math Study Skills Evaluation.  Paul suggested that rather than discussing a bad test during office hours, you have the students take the survey and bring the printout with them for discussion during office hours.

Although the survey printout refers to specific pages in Paul’s book, Winning at Math, it also tells the students a bit about why this particular behavior might be causing problems.  Here is an example of the results:

Winning At Math Survey Results

Especially for those of us that teach developmental math courses (although good for any student that is struggling), this survey would be a great activity to do right after the first exam.  Our students often focus on not being “smart enough” to do math, and this should bring the focus to the student not having the appropriate study skills.

Possibly Related Posts:


Share

Trick out your Copier!


Did you know that most “modern” copiers have the ability to send “copies” as a pdf attachment to your email account?
Once you get it set up, you can run a stack of papers through using the “scan” feature, and email it as one pdf file to yourself.
Why would you want to? Well, maybe you need to return graded tests to online students or you need to get some handwritten notes to a sick student – this will do the trick!
My online students must go to a testing center to take exams for online calculus. When I get the stack of exams, I can run them through the copier (ungraded) and email the pdf files to myself and then do the grading (using a tablet) in Adobe Acrobat using the red pencil. Then, I can email the graded exam to each student for immediate feedback (i.e. no more old-fashioned mailing back graded tests).
One technical note: Our copier will not do this nicely for 2-sided paper, so if you’re going to use this for tests for online classes, you might want to make the tests single sided for ease of feeding through the copier. Otherwise, you will have to copy every side of every page separately, and each page will come as a separate file (yuck – I learned this one the hard way!).
What if a student is taking tests at an off-campus site? Certainly you don’t want to run a faxed test through the copier (the quality of most faxes of penciled work is barely readable already). Convince your testing center or department to get a subscription to eFax (starting at $16.95 per month) or Comodo (starting at $29.95 per year). Then the off-campus testing center email the completed exam to the online service, where it can be picked up electronically.
BONUS: This faxing option will save a little bit of paper, so that will relieve some of the eco-guilt for the single-sided copying (or you can donate to the Arbor Day foundation).

Possibly Related Posts:


Share

The Multiple-Choice Math Test Problem … Solved!


We’ve been having a rather spirited discussion in my department about a common final exam for one of the math courses, and the need for an easy-to-score learning assessment (i.e. multiple choice).

The two biggest problems regarding math and multiple-choice tests are

  1. Students cannot show and get credit for work.
  2. Students can too easily “try out” answers to each problem (especially on factoring problems and equation solving problems).

Regarding #1, there is, I think, a point in the semester when students should be able to demonstrate that they can do problems, correctly, to completion. Especially in algebra-level courses, there is often not a lot of work that they could show that I might give them credit for.

If it’s a 50 question final exam, and each problem is worth 2 points for 100 points total, how much partial credit can there really be? Students who get every single problem 75% right do NOT deserve a passing grade of 75%. Every problem 75% right means 100% of the problems done with some kind of mistake. That is not a “passing” performance.

Now… on to issue #2. I think I have a solution to this problem… seriously. Why do we have to use the five choices on scantron tests as only 5 unique answers? Why not let these five choices (A,B,C,D,E) generate 25 unique answers instead? Take a look at my new take on “multiple-choice” and tell me what you think:

It’s about time we thought outside the box on these scantron forms!

Possibly Related Posts:


Share