Surrogate goals and safe Pareto improvements

Caspar Oesterheld proposed surrogate goals in unpublished work while working at CLR in 2016. Tobias Baumann first published a blog post about them in a 2017 blog post, which also coined the term “surrogate goals”. Later, Caspar published a more rigorous, formal discussion of the idea under the term “safe Pareto improvements”, which is also intended to be more general. Eliezer Yudkowsy independently proposed a similar idea in an article about “Separation from hyperexistential risk”. The following articles are fully dedicated to the idea.

Surrogate goals have also been discussed or at least mentioned in, among other places, Section 4.2 of CLR’s research agenda and the 80,000 hours podcast (guest: Paul Cristiano)."