A new study by Tel Aviv University and the University of Oxford, published in PNAS, says common mental health questionnaires can mislead psychiatric research if they capture response style as much as symptoms. The researchers found that some links reported in the field may reflect how participants answer surveys, not only their actual mental state.
The work was led by doctoral student Noam Serna of Tel Aviv University, under Prof. Reuven Dar of Tel Aviv University and Dr. Matan Mazor of Oxford. It combined a reanalysis of several large existing datasets with a pre-registered new experiment. The team examined relationships between mental health questionnaires and confidence ratings given during cognitive tasks.
Serna said psychiatry often measures traits that cannot be checked directly, unlike temperature or blood pressure, so it relies on self-report. The problem, he said, is that questionnaires also contain “noise,” including “acquiescence,” a general tendency to agree or disagree regardless of content, and “inattentive responding,” where people answer carelessly or randomly. That is especially relevant in computational psychiatry, which studies links between symptoms and measurable patterns of thinking, learning, and decision-making.
The researchers focused partly on compulsivity, a dimension linked to OCD, eating disorders and addictions. Earlier studies had suggested that people scoring higher on compulsivity also reported greater confidence in simple decisions, even though clinical OCD is usually associated with doubt and low confidence. In the new analysis, participants who tended to give high ratings on questionnaires also tended to report higher decision confidence, and low-frequency symptoms were likewise associated with higher confidence, which the authors suspect reflects inattentive responding rather than a clinical phenomenon.
In a new experiment with 195 participants, the team used attention-check items and neutral questions to detect response bias. Those identified as inattentive reported higher confidence on perceptual tasks and scored higher on an obsessive-compulsive questionnaire. After removing inattentive participants and controlling for response bias, the compulsivity-confidence link became much weaker. The researchers say the finding does not rule out real links between mental health and metacognition, but it shows that questionnaire bias must be accounted for in large self-report studies and that clinical theory should be combined with computational methods.
