Problems Measuring Utilitarian and Deontological Moral Thinking
A popular method for measuring individual differences in moral decision-making relies on mathematically flawed formulas and produces highly unreliable scores.
Source
Challenges in Process Dissociation Measures for Moral Cognition
What they did
The researchers evaluated the moral Process Dissociation Procedure (PDP) using both simulations and empirical data. They ran simulations with 10,000 simulated responders each to check if the mathematical formulas themselves create artificial patterns. Additionally, they collected online survey data from 1,043 participants, excluding 33 for failing attention checks, to analyze actual response consistency. Participants completed 9 moral PDP dilemma pairs (containing both congruent and incongruent moral dilemmas) and 12 high-conflict moral dilemmas from an alternative battery.
What they found
The simulations revealed that the mathematical formulas of the PDP generate strong, expected correlations with traditional moral scores even when using completely random data. In the empirical data, participants accepted harm 56.8% of the time in incongruent dilemmas and 23.9% of the time in congruent dilemmas. However, the reliability estimates for the calculated moral parameters were critically low, averaging just 0.30 for utilitarianism and 0.33 for deontology. Furthermore, the correlation between the PDP's traditional score and the alternative high-conflict moral dilemma battery was only 0.51, suggesting they do not reliably measure the same concept.
The limits
What it doesn't show
This study does not show that process dissociation is entirely useless for group-level experimental research, as aggregate data can cancel out individual errors. It also does not prove that the dual-process model of moral psychology is wrong, but only that this specific tool fails to measure its components at the individual level. Lastly, because the empirical data was collected through online self-reports, it does not demonstrate how participants would behave in real-world, high-stakes moral dilemmas.
Key terms
- Process Dissociation Procedure (PDP)
- A mathematical method used to estimate the relative contributions of two independent cognitive processes to a single observed behavior.
- Utilitarianism
- An ethical framework where the morality of an action is determined by its consequences, specifically aiming to maximize overall well-being.
- Deontology
- An ethical framework where actions are judged as inherently right or wrong based on rules, regardless of their consequences.
- Bipolar scale
- A measurement scale that represents two opposing concepts as opposite ends of a single continuous dimension.
- Reliability
- The degree to which a measurement tool produces consistent and stable results across different items or testing instances.
- Congruent dilemmas
- Moral scenarios in which both utilitarian outcomes and deontological rules suggest the same action is unacceptable.
Flashcards
Want these cards to stick?
Save the deck to NoteFren and study it with spaced repetition.
Quiz yourself
From which academic domain did the Process Dissociation Procedure (PDP) originate before being adapted for moral cognition?
Common questions
How does the moral Process Dissociation Procedure work?
It presents participants with moral scenarios that are either congruent (where both moral rules and consequences agree an action is wrong) or incongruent (where rules and consequences conflict). It then uses formulas to calculate separate scores for utilitarian and deontological tendencies.
Why do the authors call the PDP correlations 'mathematical artifacts'?
Because the formulas for the moral scores are calculated using the exact same variables. Therefore, a strong correlation between the parameters and traditional scores is mathematically guaranteed to appear, even if the answers are entirely random.
Does this study mean we should completely abandon the Process Dissociation Procedure?
Not necessarily. The authors note that the PDP can still be useful for group-level experiments where individual measurement errors average out, but they strongly advise against using it to measure stable individual differences between people.
Why is the low reliability of the PDP scores a major problem?
When a test has very low reliability (like the 0.30 and 0.33 found here), most of the variation in scores is just random noise. This makes it extremely difficult to find true relationships and highly likely to publish false positive findings.
More on Moral psychology