My worst fear became a reality in my Teaching With Technology class the other day - I was asked to take a computer-based standardized test! In the beginning, I dutifully followed instructions and virtually maneuvered my way to the online testing website to begin my test. But my mind froze as I clicked START and the screen split in two. On one side, a random, out-of-context excerpt taken from the dullest publication in existence and on the other side, a long list of purposely misleading questions containing vocabulary even Einstein wouldn't recognize followed by answer choices seemingly written in the ancient Aztec Olmec language. My PTSD symptoms returned as memories of the MEAP, ACT, and GRE tests flooded my mind. Instead of attempting to answer the questions, as we were instructed to do, I blinked repeatedly at the screen hoping each time after I opened my eyes that the test would be over. When the time was up, I breathed a sigh of relief, quickly closed my testing window, and opened a window with recognizable words.
To be fair, this test was not graded nor was it used in any way as an assessment. It was used to supplement a class discussion about both standardized tests in general and also computer-based standardized tests specifically. As a student and recent participant in both paper and computer standardized tests, I can attest to the fact that neither are exactly pleasant. (Indeed, it is a form of torture designed to withdraw deeply-buried intel from the student brain!). Now, however, I have a new interest in testing/assessment as I am now viewing it from a different perspective - the teacher perspective. As a developing teacher, I now have lots of unanswered questions/thoughts about standardized testing and computer-based testing.
As far as standardized testing in general, I am not a huge fan, and this is not just because it is unpleasant (though this is undeniably true!). In most case, I believe standardized tests only test how well a student is at taking a standardized test. The GRE is a good example of this. The GRE tests ones ability to make sense out of obscure vocabulary and solve trick math questions very quickly. I question whether this is a reliable/valid measure of ability (at least any relevant ability) or simply a measure of who studied the most/took the most practiced tests. If you think about it, how much time and effort do schools put into preparing students for the ACT? Is the ACT, then, a good measure of anything other than which school was most successful at "teaching to the test"? Yet, my biggest problem with standardized test is that it emphasizes factual knowledge without considering other forms of intelligence (e.g. athletic ability, artistic ability, social ability, etc.). In my opinion, these other forms of intelligence are just as relevant, so why are we not considering them?
Finally, the issue of computer-based vs. paper-based testing. In my humble (and admittedly biased) opinion, I think computers complicate even more the issue of standardized testing and raises an entire list of new, unanswered questions. For instance, does a student takes a computer-based test have an advantage over a student who takes a paper an pencil test (or vice versa)? Would students more familiar with technology score higher on computer-based tests than those who are less familiar? Which computer skills (if any) should be taught in schools to help prepare students for these types of test? In addition to these questions, there are a variety of other concerns to consider such as physical consequences (visual fatigue, headaches, etc.), navigation issues, and reduced comprehension. The latter, in particular, concerns me as I am familiar with research that suggests digital screens interfere with text comprehension.
Perhaps I am too much a skeptic when it comes to standardized tests, especially computer-based standardized tests. In my defense, my skepticism stems from concern for my future students. At the very least, I think these issues need to be researched further and the consequences, both positive and negative, considered carefully.
I love your quote saying that standardized testing is only testing factual knowledge and not other forms of intelligence. I remember in high school when all I could think about was what I got on the ACT and what school would except that score. I was never really concerned about extra-curricular activities until a guidance counselor told me how important it was to do other things and keep my grades up. The ACT just shows schools how well you can take a test, and students are still taking that test to show how well they know how to do well on a test. I have always hated multiple choice because you are either right or wrong, no in-between. I am the Queen of partial credit city and although some multiple choice questions can be good, I think problems where students have to show work to get points is incredibly useful. But testing with open ended questions has so many things to consider like a key, time, and money. Do you think we will ever move away from the multiple choice era into a more show your work era of standardized tests?
ReplyDeleteI certainly share your sentiments! I also dislike multiple choice questions because they are either right or wrong in a world where rarely anything is black and white, but instead interpreted and perceived differently by different people. I also agree with you that problems where students show work are more useful than multiple choice questions because it helps both the teacher and the student visualize the thinking process that is taking place. It can be difficult to help a student who answers a multiple choice question wrong because the teacher does not now how the student arrived at the answer or where s/he made the error.
DeleteSort of shifting into your question of whether it is feasible to move away from multiple choice questions and standardized tests, I think it is definitely possible though, as you hint at, more time-consuming. Yet, if it proved to be a more valuable assessment of students than shouldn't we as teachers feel some obligation to begin moving in that direction? It wouldn't necessarily be easy, but I think the types of open-ended questions you suggest are more valuable assessment tools. One obstacle that I see as a future history teacher is that these "showing work" problems are more easily implemented in math and the sciences. . I am wondering if maybe the humanities can adapt these types of "showing work" problems into exercises where students must show evidence from the text to make their argument. I think something like this might be a more valuable assessment in history than memorization of historical facts.
As you obviously know, I am totally against the idea of standardized testing. However I heard the other day that computer-based standardized tests can be easily altered to be more aligned to each student's ability. As a student is taking a standardized test, depending on which questions they get right or wrong, the computer will generate harder or easier questions. Does this make the measure of intelligence more reasonable? Or does this make it less reasonable?
ReplyDeleteInteresting question! I, too, have heard about computer-based tests generating questions supposedly aligned with student ability, but I hadn't thought about it in this context. I certainly believe I need to familiarize myself with the more recent research on this topic to formulate a more well-thought out answer. My initial thought, though, is that this modification might create more problems than it solves. First, I ma skeptical as to how one would accurately determine which questions would be more challenging than others. For instance, if we took the same test and you chose the correct answer on the first problem and I chose the wrong answer, our second question should be different based on assessed ability. Your "more difficult" question might actually be more simple for me than you because we have different skill sets/backgrounds/experiences. Likewise, my second, "easier" question could actually be quite challenging to me. I think "hard" and "easy" questions are relative so I am unsure as to how this would be determined reliably across all students.
DeleteSecond, even if the questions are able to be aligned with student ability reliably, the test is still mostly measuring factual knowledge (memorization of facts in most cases). Personally, I do not believe this is an accurate measure of overall student intelligence or ability. Even if the questions fluctuated in difficulty based on student ability, student success on the test still depends largely reading pace and memorization.
I think these changes are at the very least evidence that some people are beginning to think about standardized testing and improvements, which is a start. However, I am still hopeful that a much better assessment can be developed.
I also find the idea of the GRE accurately predicting about how well someone is qualified for graduate work ridiculous. Especially because it has no way of measuring critical thinking skills, except maybe the writing section. Any graduate program is going to involve a lot of critical thinking as students start to do truly original research. So yeah, it's important that tests actually test for the needed skills, not something only tangentially related.
ReplyDelete