STUDY 1

Methods

Design and procedure

This study used a correlational design. Participants first completed a general knowledge test before rating their belief in a range of unverified and verified conspiracies. They then completed an assessment of cognitive reflection and of scientific reasoning, before providing demographic information. The primary outcome variable was the degree to which participants discriminate between verified and unverified conspiracies. This was a re‐analysis of data from a study that primarily aimed to examine the effects of a reasoning checklist on how people evaluate a medical diagnosis. Half of the conspiracy items were presented before this reasoning task and half were presented after.

Participants

Participants were recruited via Prolific using a targeted recruitment strategy. Months prior to taking part, participants fluent in English (no geographical restrictions) completed a brief pre‐screen assessing belief in four implausible claims (from Martire et al., 2020; e.g. Global warming is a hoax). Those who rated any claim above 60 (out of 100) were invited to take part in this study as tentative ‘Endorsers’, whereas those who rated all claims below 40 were invited as tentative ‘Non‐endorsers.’ Recruitment was balanced across these groups to ensure strong representation of higher‐endorsing participants, who might be underrepresented in general population samples (see Robson, Faasse, Gordon, et al., 2024; Van Prooijen et al., 2023). In all, the sample included 247 participants after two were excluded for not passing at least two of three attention checks. Post‐hoc sensitivity analyses indicate that a sample of this size is sufficient to detect small‐to‐medium effects with 80% power (r = .18; f 2 = .06).

Participants were primarily from the United States (26.7%) and the United Kingdom (23.1%), but also from other countries (10.1% from South Africa, 6.9% from Poland, 5.7% from Portugal; 4.5% from Australia and 23.1% from elsewhere). The mean age of the participants was 36.5 (SD = 13.2). Most (73.3%) identified as male whereas 25.1% identified as female (1.2% other; 0.4% did not say). A majority of the sample (70.9%) identified as White/Caucasian, followed by African American (9.3%), Hispanic (6.9%) and Asian (5.7%; 5.7% other; 1.6% did not say). Most of the participants (69.2%) had tertiary level education or higher (29.6% secondary education or less; 1.2% did not say). Most of the sample (64.0%) also reported English as their first language (36.0% said it was not).

Materials

Conspiracy belief

Participants rated their belief in 32 conspiracy claims—16 unverified and 16 verified. Unverified items were adapted from the Belief in Conspiracy Theory Inventory (BCTI; Swami et al., 2011) and verified items were drawn from Pennycook et al. (2025), with one item added. Items were divided into two sets. Each set included eight unverified conspiracies, eight verified conspiracies and one attention check. Participants completed both sets in a counterbalanced order; one set was presented before a reasoning task, and the other after, and items were in a different random order for each participant. The reasoning task itself is beyond the scope of the study, but materials are available on the OSF. Mean scores were calculated separately for the unverified and verified items. Belief was rated on a sliding scale ranging from 0 (Completely false) to 9 (Completely true).

General knowledge

Eight general knowledge questions were presented in a randomized order. All responses were provided using a scale from 0 (Not at all true) to 100 (Definitely true). A general knowledge score was computed by first reverse coding false items and then computing the mean, out of 100, across all eight questions. An example item was ‘spiders have six legs’. One attention check was also included (drag the slide to Definitely true).

Scientific reasoning scale (SRS)

The SRS (Drummond & Fischhoff, 2017) consists of 11 true/false questions designed to assess individual differences in scientific reasoning ability. This is the capacity to evaluate scientific findings with respect to the factors that determine their quality (e.g. control groups, confounds, causal inference). Scores were computed by summing the number of correct answers (range: 0–11).

Cognitive reflection test (CRT)

Participants answered seven CRT questions (Frederick, 2005; Thomson & Oppenheimer, 2016). These are designed to assess the capacity to suppress an incorrect intuitive (‘system 1’) answer to a question in favour of a correct reflective (‘system 2’) answer. For example, ‘a bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?’ The incorrect intuitive answer is 10c, whereas the correct reflective answer is 5c. Scores were calculated by summing the number of correct answers (range: 0–7). Participants also answered follow‐up questions about their confidence and the strategies they used.

Demographics

Participants were asked to provide their age, gender, ethnicity, level of education, and whether English was their first language. Only the woman/female (0) and man/male (1) gender categories were included in analyses, and education was converted to a five‐point ordinal scale.

Results and discussion

Conspiracy discrimination and response bias

The primary dependent variable was an index of discrimination between verified and unverified conspiracies based on Signal Detection Theory (SDT). SDT can be applied in this context because a participant says that a claim is either false (responses 0–4) or true (responses 5–9), and each conspiracy is either verified or unverified (a proxy for true and false in reality). This means there are four potential outcomes: a person says a verified conspiracy is true (Hit), a verified conspiracy is false (Miss), an unverified conspiracy is false (Correct rejection), or an unverified conspiracy is true (False alarm). Better discrimination means that a person tends to endorse verified conspiracies more than unverified ones.

Unlike many other signal detection metrics that dichotomize responses into correct/incorrect or yes/no (e.g. d'), we quantified discriminability via the empirical area under the curve (AUC). Unlike d', AUC does not rest on assumptions about underlying distributions. Instead, it leverages the full range of the response scale. This is particularly well suited to our data because belief is measured via graded levels of confidence rather than binary judgements. For the AUC method, a curve is constructed by plotting the hit rate against the false alarm rate across all decision thresholds (points along the response scale). The area underneath the curve can then be estimated numerically. We used the pROC package in R (Robin et al., 2011) to estimate AUC, which sums the areas of a series of adjacent trapezoids based on points at each decision threshold. An AUC of 0.5 reflects chance‐level performance, whereas values approaching 1 indicate stronger discrimination.

As a secondary measure, we computed each participant's response bias. This is the tendency to say true versus false (calculated as c; Macmillan & Creelman, 2005). Values greater than zero reflect a liberal response bias (tend to say ‘true’) whereas values less than zero reflect a conservative response bias (tend to say ‘false’).

Relationships between variables

We first examined zero‐order correlations across the entire sample (Table 1). Unsurprisingly, belief in unverified conspiracies, belief in verified conspiracies, and response bias were strongly positively intercorrelated. Consistent with Frenken et al. (2024), cognitive measures like scientific reasoning and cognitive reflection were significantly associated with reduced belief in unverified conspiracy theories. However, belief in verified conspiracies was associated with higher general knowledge scores. Additionally, better discrimination between verified and unverified conspiracies (AUC) was associated with better scientific reasoning, cognitive reflection and general knowledge, and identifying as male.

 | M | SD | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10
1. Unverified consp. belief | 3.54 | 2.18 | – |  |  |  |  |  |  |  |  | 
2. Verified consp. belief | 4.47 | 1.94 | 0.69** | – |  |  |  |  |  |  |  | 
3. Response bias (c) | −0.11 | 0.87 | 0.88** | 0.86** | – |  |  |  |  |  |  | 
4. Discriminability (AUC) | 0.58 | 0.16 | −0.55** | 0.18** | −0.24** | – |  |  |  |  |  | 
5. General knowledge | 83.64 | 13.62 | −0.09 | 0.14* | 0.01 | 0.25** | – |  |  |  |  | 
6. Scientific reasoning | 6.31 | 2.34 | −0.29** | −0.05 | −0.19** | 0.31** | 0.20** | – |  |  |  | 
7. Cognitive reflection | 4.09 | 2.05 | −0.22** | −0.03 | −0.16* | 0.25** | 0.23** | 0.38** | – |  |  | 
8. Age | 36.51 | 13.25 | 0.05 | 0.14* | 0.15* | 0.05 | 0.09 | 0.03 | 0.06 | – |  | 
9. Education | 2.84 | 0.73 | −0.07 | −0.06 | −0.05 | 0.01 | 0.00 | 0.03 | 0.03 | 0.10 | – | 
10. Gender | ‐ | ‐ | −0.10 | 0.05 | −0.06 | 0.19** | −0.01 | 0.14* | 0.23** | −0.14* | −0.03 | –
11. Native English | ‐ | ‐ | 0.17** | 0.15* | 0.19** | −0.07 | −0.00 | −0.02 | −0.13* | 0.42** | −0.04 | −0.10

Predicting discriminability

We formally examined the unique predictors of discriminability, controlling for other variables, using multiple linear regression analysis. The predictors together explained 18.0% of variance in discriminability. As illustrated in Table 2, scientific reasoning, general knowledge and identifying as male predicted better discrimination, consistent with the zero‐order correlations above. However, CRT performance was not a significant predictor; nor were age, education and English‐speaking status.

Predictor | β | b | SE | t‐value | p‐value
Intercept | – | 0.22 | 0.08 | 2.94 | .004
General knowledge | .19 | 0.00 | 0.00 | 3.15 | .002
Scientific reasoning | .24 | 0.02 | 0.00 | 3.66 | <.001
Cognitive reflection | .08 | 0.01 | 0.01 | 1.14 | .257
Age | .07 | 0.00 | 0.00 | 1.05 | .295
Education | −.02 | 0.00 | 0.01 | −0.29 | .772
Gender | .14 | 0.05 | 0.02 | 2.18 | .030
Native English | −.08 | −0.03 | 0.02 | −1.17 | .243

Predicting cognitive sophistication

We also flipped the analytic direction by conducting linear regression analyses to assess whether AUC more strongly predicts cognitive sophistication than response bias. Response bias alone did not significantly predict general knowledge (b = 0.21, p = .830), but it was a significant negative predictor for cognitive reflection (b = −0.38, p = .010) and scientific reasoning (b = −0.50, p = .003). When AUC was added as a second predictor, however, response bias no longer significantly predicted any cognitive measure (general knowledge: b = 1.23, p = .218; cognitive reflection: b = −0.25, p = .091; scientific reasoning: b = −0.32, p = .055). AUC, by contrast, was consistently a significant, positive predictor of cognitive sophistication (general knowledge: b = 22.64, p < .001; cognitive reflection: b = 2.87, p < .001; scientific reasoning: b = 4.09, p < .001).

Consistent with Frenken et al. (2024), cognitive variables like scientific reasoning and cognitive reflection ability predicted lower belief in unverified conspiracies. Unlike Frenken and colleagues, however, we integrated belief ratings into a measure of discriminability. General knowledge, scientific reasoning and cognitive reflection positively correlated with discriminability, as did gender (with men scoring higher than women). Flipping the analytic direction also revealed that response bias did not predict performance on cognitive measures above and beyond discriminability. Together, these results suggest that sophisticated reasoning is not simply about rejecting unverified conspiracies, but about differentiating claims with empirical support from those without. This has been obscured by the near exclusive focus on unverified or implausible conspiracy claims in past research.