Research

Measuring AI's ability to cause harm despite control measures informs risk assessments. In existing sabotage capability evaluations, when we evaluate an attacker against a monitor in one setting, we do not know how the result generalizes to other models or settings, or what scores warrant stronger control measures. To address this limitation, we posit that model sabotage capability has latent structure that enables prediction of unmeasured outcomes, and find evidence for this hypothesis. With just three parameters per attacker and two parameters per monitor, we can predict how well an attacker AI performs against a monitor without having measured the pair. We also can translate these parameters between control settings to predict sabotage success probability in different evaluations. The predictive power of our simple model contributes towards more integrated, sample-efficient, and decision-relevant sabotage capability evaluations. We furthermore use this model to find that sabotage capability trends increase with general capabilities, as measured by ECI.

At MATS, my collaborator and I developed a new finetuning-based propensity auditing method. Our work was selected as one of eight Spotlight talks at the end-of-fellowship MATS Symposium.

This project began as an attempt to apply Elizabeth Donoway's excess description length to propensity evaluations. We ended up somewhere quite different, but I still think EDL is important for understanding what the method measures. Some notes & links to her papers here.