"We need 3rd party Training-Run Assessments" by Alex Meinke

Failed to add items

Sorry, we are unable to add the item because your shopping cart is already at capacity.

Add to basket failed.

Please try again later

Add to wishlist failed.

Please try again later

Remove from wishlist failed.

Please try again later

Adding to library failed

Please try again

Follow podcast failed

Unfollow podcast failed

"We need 3rd party Training-Run Assessments" by Alex Meinke

Listen for free

View show details

Training-run assessments conducted by a 3rd party should become a standard part of frontier AI safety.

By a Training-Run Assessment, or TRA, I mean an in-depth analysis of the post-training pipeline and dynamics leading up to a frontier model release. A TRA can look at intermediate checkpoints, training rollouts, RL environments, reward signals, SFT datasets, and the process by which the developer responded to warning signs.[1]

In this post I will argue that:

Final-checkpoint evaluations will be insufficient to assess scheming risks.
TRAs can be more effective at detecting scheming.
Frontier developers should involve third parties to do TRAs or verify safety claims by the developers.

The rest of the post lays out a taxonomy of TRAs and sketches a path toward a 3rd party ecosystem for them. We, at Apollo Research, are intending to conduct 3rd party Training-Run Assessments in the future.

Detecting Scheming may require Training-Run Assessments

By scheming I mean an AI covertly pursuing misaligned goals while deliberately concealing its intentions or capabilities from its developers. I restrict attention to “coherent” forms of scheming where the model pursues somewhat stable misaligned goals across context windows, rather than misalignment that surfaces only as isolated, context-dependent defections. [...]

---

Outline:

(01:23) Detecting Scheming may require Training-Run Assessments

(03:55) Why 3rd parties should perform Training-Run Assessments

(04:12) Developers may lack incentives to adequately assess scheming

(04:49) Developers' safety assessments lack credibility

(05:31) External evaluators can bundle expertise for assessing scheming

(06:17) 3rd party TRAs can be developed gradually

(08:50) Checkpoint evals

(08:54) What?

(10:30) How?

(11:15) Data inspections

(11:19) What?

(12:03) Why?

[... 16 more sections]

---

First published:
July 5th, 2026

Source:
https://www.lesswrong.com/posts/3HvvjffA65mHLwaWm/we-need-3rd-party-training-run-assessments

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

No reviews yet