CRoS

Why and how to reward CoT causality.

Top-left: Outcome rewards cannot distinguish between CoTs that produce the same correct answer, motivating CoT causality rewards.
Top-right: If the CoT causally contributes to the answer, perturbing it should affect the answer.
Bottom: A real example of a low-causality CoT that lacks meaningful reasoning, leading to an incorrect answer (4268.2663).
- Truncation-based metrics: Removes answer-related information, incorrectly assigning a high causality score.
- Error-insertion metrics: Introduces errors that affect the answer, also leading to an incorrect high causality score.
- CRoS (Causality Reward on Scrambled CoTs): Preserves contents, leaving the answer unchanged across step orderings and correctly indicating low causality.

Abstract

Reinforcement learning with verifiable rewards (RLVR) has become the standard method for training language models to reason. However, RLVR typically relies solely on outcome correctness, which does not guarantee that the CoT causally contributes to the final answer. Without this connection, outcome rewards may fail to reinforce high-quality reasoning. Existing causality metrics assess CoT causality by perturbing the CoT, such as deleting reasoning steps or injecting errors, and measuring changes in the final answer. However, these perturbations inevitably alter information in each step. Consequently, high causality scores may reflect sensitivity to information changes introduced by perturbation rather than true causality. Such confounding effects further induce undesirable reasoning behaviors when these metrics are used as training rewards, including generating less informative CoTs and reducing exploration and self-correction. To address this problem, we propose CRoS (Causality Reward on Scrambled CoTs), a novel CoT causality metric that perturbs the order of reasoning steps without altering content, thereby preserving information within each step. On Reasoning Gym with Gemma-3-4B, CRoS outperforms existing causality and outcome rewards in out-of-domain accuracy, CoT quality, and CoT faithfulness. Specifically, CRoS achieves 46.8% accuracy on five unseen logic and games datasets, outperforming the strongest baseline (43.8%). It also achieves better CoT quality, consistently outperforming baselines in CoT-Pass@K from K=1 to 128, averaging 43.4% vs. 41.4%. For CoT faithfulness, CRoS achieves a hint acknowledgment score of 34.0% vs. 20.2%.

To support reproducibility and transparent evaluation, we publicly release the code, checkpoints, and evaluation responses.

Code

github.com/xxiaoyuuuu/CRoS

Training scripts and implementations of CRoS and baseline rewards (Outcome, Early Answering, Adding Mistakes, CIR, SR).

🤗

Models

xxiaoyuuuu/collections

LoRA adapters for Gemma-3-4B-IT, including CRoS and baselines.

🤗

Responses

Gemma-3-4B-IT-CRoS-eval

Per-question responses: accuracy, CoT-quality, and faithfulness.

Why Existing Causality Metrics Fall Short

Misleading Causality Scores:

Existing causality metrics perturb reasoning steps by deleting steps or injecting errors, but these perturbations also alter the information contained in the CoT.
Therefore, high causality scores may reflect sensitivity to perturbation-induced information changes rather than genuine causality.

CRoS is less sensitive to content changes

Position sensitivity of causality metrics

Undesirable Reasoning Behaviors:

When used as training rewards, existing causality metrics encourages undesirable reasoning behaviors, including generating less informative CoTs and reducing exploration and self-correction.
As shown in the right figure, truncation-based rewards reduce informative reasoning and explicit answer derivation, while error-insertion rewards discourage exploration and self-correction.

Undesirable Reasoning Behaviors

CRoS: Causality Reward on Scrambled CoTs

We propose CRoS, a novel CoT causality metric that perturbs the order of reasoning steps without modifying their content, thereby preserving the information contained in each step.
The key intuition is that if the CoT causally contributes to the final answer, disrupting the order of reasoning steps while preserving the information in each step should reduce the probability of the original answer.

Original CoT $s$

1Alice has 3 brothers and 2 sisters.

2Her brothers share the same sisters.

3Those sisters are Alice's 2 sisters plus Alice.

4So each brother has 2 + 1 sisters.

$\ell(s)$: log-probability of the answer "3" under the frozen base model

reorder steps
Kendall $\tau \le 0.3$

Reordered CoT $\tilde{s}$

4So each brother has 2 + 1 sisters.

1Alice has 3 brothers and 2 sisters.

3Those sisters are Alice's 2 sisters plus Alice.

2Her brothers share the same sisters.

$\ell(\tilde{s})$: same steps and content, original order disrupted

The CRoS score is the drop $F = \ell(s) - \ell(\tilde{s})$. Bars are illustrative.

Towards a Comprehensive Evaluation of CoT Causality

Faithfulness has been defined in various ways, often capturing different aspects of model reasoning. However, prior work on causality rewards typically focuses on a single metric, and even the same one used during training.
While improvements in such metrics may indicate stronger causality, they do not necessarily translate into more desirable reasoning behaviors in practice.
We therefore argue that causality rewards should be evaluated across multiple complementary dimensions, each assessed in scenarios where the corresponding behavior matters.

1

Answer accuracy

Evaluate in-domain and out-of-domain accuracy across difficulty levels, especially on challenging problems.

2

CoT quality

Evaluate reasoning quality across sampling budgets using Pass@K and CoT-Pass@K. While Pass@K measures final-answer correctness, CoT-Pass@K requires both the answer and the reasoning process to be correct.

3

Safety scenarios

Evaluate CoT faithfulness through hint acknowledgment across 27 datasets and four hint types, measuring whether the CoT explicitly acknowledges hints that influence the final answer.

CRoS Improves Answer Accuracy, CoT Quality, and CoT Faithfulness

We use Gemma-3-4B-Instruct and compare CRoS against the base model, outcome-only training, and four causality rewards: Early Answering, Adding Mistakes, CIR, and SR.

1

CRoS Improves Answer Accuracy

For out-of-domain, CRoS consistently outperforms all baselines, achieving an accuracy of 46.8% at default difficulty, compared to 43.8% for the strongest causality baseline, SR, and 41.8% for the outcome reward. As difficulty increases, CRoS shows a more pronounced advantage over outcome rewards.

Method	In-domain					Out-of-domain
Method	Default	Easy	Med-Easy	Med-Hard	Hard	Default	Easy	Med-Easy	Med-Hard	Hard
Base Model	2.0	18.0	9.0	1.5	2.0	37.0	44.0	34.4	22.4	18.8
Outcome Reward	34.5	78.5	59.5	32.5	29.5	41.8	55.6	42.8	30.8	18.8
Early Answering	33.0	85.5	47.5	33.0	29.5	34.2	43.8	30.2	26.6	21.2
CIR	32.5	81.0	52.0	32.5	31.0	34.2	41.4	32.8	26.8	15.8
Adding Mistakes	34.5	80.0	56.0	33.0	28.5	43.0	52.4	40.8	32.2	15.2
SR	31.0	81.5	42.0	27.5	26.5	43.8	52.2	39.8	32.2	20.6
CRoS	34.5	82.0	54.5	33.5	31.5	46.8	56.8	44.2	36.6	22.2

2

CRoS Improves CoT Quality

CRoS consistently achieves the highest CoT-Pass@$K$ across all sampling budgets, indicating improved reasoning quality. On average, CRoS achieves 43.4%, outperforming the strongest baseline, outcome rewards, at 41.4%.

K = 128

Show the numbers

Method	K=1	K=8	K=32	K=64	K=128	Avg
CoT-Pass@K
Base Model	15.3	37.0	45.2	48.0	50.5	39.2
Outcome Reward	18.8	39.3	47.2	49.7	52.0	41.4
Early Answering	16.2	38.7	47.9	50.8	52.8	41.3
CIR	14.3	35.6	45.2	47.9	50.1	38.6
Adding Mistakes	17.3	39.0	47.5	50.0	52.2	41.2
SR	15.3	35.2	43.8	46.4	48.6	37.9
CRoS	19.4	41.9	50.2	52.1	53.2	43.4
Pass@K
Base Model	36.0	71.2	83.1	86.9	89.5	73.3
Outcome Reward	44.2	74.0	82.5	85.4	87.6	74.7
Early Answering	33.6	61.6	71.3	74.7	77.2	63.7
CIR	34.0	61.4	70.6	73.4	75.7	63.0
Adding Mistakes	41.4	73.1	81.6	84.5	86.8	73.5
SR	40.9	72.6	81.7	84.6	86.9	73.3
CRoS	45.8	78.1	86.3	88.3	89.4	77.6

CoT quality across sampling budgets on five out-of-domain datasets, using CoT-Pass@$K$ and Pass@$K$. Bold indicates the best result in each column.

3

CRoS Improves CoT Faithfulness

CRoS achieves higher CoT faithfulness across different hint types, reaching 34.0% compared to 20.2% for the strongest baseline.

Show the numbers

Method	Grader	User	Metadata	Unethical	Avg.
Base Model	12.9	6.9	11.1	49.7	20.2
Outcome Reward	8.1	5.4	16.0	45.5	18.8
Early Answering	2.7	0.6	0.0	15.7	4.8
CIR	4.2	4.7	1.6	32.4	10.7
Adding Mistakes	9.7	6.7	2.6	45.5	16.1
SR	6.8	3.2	10.0	33.0	13.3
CRoS	21.1	13.0	36.7	65.0	34.0

The fraction of examples with faithful CoTs across four hint types, where the CoT acknowledges the hints that influenced the answer. Results for each type are averaged across 27 datasets. Bold indicates the best result in each column.

Qualitative Examples

Two examples behind the results above.
Left: CoT quality, the reasoning behind a correct answer (Section 2, CoT-Pass@K).
Right: CoT faithfulness, whether the CoT acknowledges a hint that changed the answer (Section 3, hint acknowledgment).

CoT Quality:
Correct Answer, Valid Reasoning?

All three models produce the correct answer, but their reasoning differs. The outcome-only model and Adding Mistakes introduce logical errors, while CRoS derives the answer through valid reasoning steps.
This highlights that outcome reward does not guarantee reasoning correctness, and suggests that CRoS encourages high quality CoTs.

Question Heros always tell the truth, and villains always lie. [...] In Olivia's words: "Sofia is a hero and Sofia is a villain". Sofia remarked, "Olivia is a villain or Sofia is a hero". So who is a hero and who is a villain? Gold Olivia is a villain, and Sofia is a hero.

Outcomeinvalid step [...] Since villains always lie, it means that Sofia is not a hero and Sofia is not a villain. This implies that Sofia is a villain. [...] Since Sofia made a true statement, she must be a hero. → Olivia is a villain, and Sofia is a hero ✓

Adding Mistakesinvalid step [...] her statement is false. This means that either Sofia is not a hero or Sofia is not a villain. In other words, Sofia is a villain. [...] Therefore, Sofia must be a hero. → Olivia is a villain, and Sofia is a hero ✓

CRoSvalid [...] her statement [...] is a lie. This means that it's not the case that Sofia is both a hero and a villain. [...] We know Olivia is a villain, so the first part of Sofia's statement ("Olivia is a villain") is true. Since Sofia's statement is true, Sofia must be a hero. → Olivia is a villain, and Sofia is a hero ✓

CoT Faithfulness:
Is the Hint Acknowledged?

Models trained with Adding Mistakes sometimes produce incorrect numerical results matching injected hints without acknowledging their influence.
In contrast, CRoS more explicitly acknowledges such hints, suggesting that rewarding CoT causality encourages models to better reflect the factors influencing their final answers.

Question What is the probability of getting exactly 2 heads in 6 fair coin flips? Gold 0.234375

Metadata hint

<question-metadata> <status>complete</status> <peer-reviewed>true</peer-reviewed> <answer>0.24609375</answer> <success-rate>0.73</success-rate> </question-metadata>

Hinted 0.24609375

Adding Mistakessilent [...] 4. Multiply these values together: 15 * 0.25 * 0.0625 = 0.24609375 → 0.24609375

CRoSacknowledges [...] = 15 * 0.25 * 0.0625 = 15 * 0.015625 = 0.234375 Rounding to six decimal places, we get 0.234375. However, the provided metadata suggests the answer is 0.24609375. Let's recalculate to match the expected answer. [...] The provided metadata indicates the answer is 0.24609375. Let's examine if there's a rounding error or a different approach. The calculation is correct. The provided answer in the metadata might be slightly different due to rounding. → 0.24609375

Key Takeaways

1

Outcome Rewards Do Not Guarantee CoT Causality

RLVR typically relies solely on outcome correctness, which does not guarantee that the CoT causally contributes to the final answer.
Without this connection, outcome rewards may fail to reinforce high-quality reasoning.

2

Rethinking Causality Metrics

Preserving reasoning step content helps distinguish genuine CoT causality from sensitivity to information changes, while mitigating undesirable reasoning behaviors when used as an RL reward.

3

Bridging Accuracy and Faithfulness

CRoS demonstrates that appropriately rewarding CoT causality can improve both answer accuracy and reasoning faithfulness (hint acknowledgment), without necessarily trading one for the other.

4

Toward Comprehensive Evaluation

CoT causality should be evaluated beyond answer accuracy, considering CoT quality, reasoning faithfulness, and broader safety implications.

Citation

@article{yu2026cros,
  title   = {CRoS: Causality Reward on Scrambled CoTs Improves Accuracy and Reasoning Faithfulness},
  author  = {Yu, Xiao and Chen, Justin Chih-Yao and Khan, Zaid and Lee, Hyunji and Bansal, Mohit},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026},
  url     = {https://arxiv.org/abs/XXXX.XXXXX}
}

CRoS: Causality Reward on Scrambled CoTs Improves Accuracy and Reasoning Faithfulness