Hard or Easy?
How difficult should a memory test be? If it is too easy, most people will max out their score and the test won't be measuring anything. If it is too hard people will get discouraged. We don't want either of these situations to occur.People have different processing speeds and different memory capacities. One person may find no challenge in one test, but tremendous challenge in another.
Keep in mind that it is very easy to design a difficult test with a computer. Computers have perfect memories and they are very fast. It is simple to add more items, speed up the presentation rate and add complexity to the task. It is always a temptation to make things too difficult.
One approach to test difficulty is to make the test adaptive. The tests can get harder when a person is doing well, and back off in difficulty, if they are not. We have done this with Test 1 and Test 3 on the MemoryMigrations now. The problem is, of course, that this means each person is getting a different test. I think we have a scoring system that partially solves this problem.
Another obvious solution is to simply create a variety of tests some of which are hard and some of which are easy. Our next batch of tests for MemoryMigrations will follow this approach, some will be very hard and some will be quite easy.
To some degree, it does not matter, because we are tracing each person's memory development and tracking current results against past results. In other words, if you get a 60% score on a given test, that is your normal score. You can use this number to compare to future tests. Who cares if you are above or below a "norm" based on age and, etc.
I go along with this theory - up to point. We want people to be tested but we also want them to be entertained. This really drives cognitive scientists nuts. So the tests need to have a sense of challenge and exploration and a "proper" level of difficulty. More on this later.
posted by Henry Bowles @ 1:38 PM
