Draft 05

You are a poor judge of how much you vary.

If the conditions that shape a person’s work differ from person to person, then finding yours means studying yourself. The obvious way to do that is to ask yourself. That turns out to be the one method that does not work.

Evidence Ask someone how consistent they are. Then measure them five times a day for a week and calculate it. The two answers correlate at between .14 and .30. In a study of 399 people.

That is close to no relationship. What people believe about their own steadiness and what their reports actually do are nearly separate things.

This cuts against me in one direction and for me in the other, and I want both on the page. Against me, it means I cannot hand you a questionnaire asking how much you vary and treat the answer as data. For me, it is the reason to bother measuring at all. If introspection were accurate, none of this would be necessary.

Evidence There is a second trap underneath, and it is duller and more important. If you rate yourself on a 1 to 7 scale, how much you can possibly vary depends on where you sit. Someone answering near the middle has room to swing in both directions. Someone answering near an end does not. So a person who scores high looks steadier than a person who scores in the middle, before anything real has happened.

Anyone tracking themselves on a rating scale is walking into that. It manufactures patterns out of the shape of the scale.