Surrogate goals and safe Pareto improvements
This page lists research on surrogate goals and safe Pareto improvements (SPIs), some of it under our current SPI research agenda.
Caspar Oesterheld proposed surrogate goals in unpublished work while working at CLR in 2016. Tobias Baumann first published a blog post about them in a 2017 blog post, which also coined the term “surrogate goals”. Later, Caspar published a more rigorous, formal discussion of the idea under the term “safe Pareto improvements”, which is also intended to be more general. Eliezer Yudkowsky independently considered a similar idea in an article about “Separation from hyperexistential risk”.
Surrogate goals have also been discussed or at least mentioned in, among other places, Section 4.2 of CLR’s research agenda and the 80,000 hours podcast (guest: Paul Christiano).