Backup Utility Functions: A Fail-Safe AI Technique

Setting up the goal systems of advanced AIs in a way that results in benevolent behavior is expected to be difficult. We should account for the possibility that the goal systems of AIs fail to implement our values as originally intended. In this paper, we propose the idea of backup utility functions: Secondary utility functions that are used in case the primary ones “fail”.

Read more

Identifying Plausible Paths to Impact and their Strategic Implications

FRI’s research seeks to identify the best intervention(s) for suffering reducers to work on. Rather than continuing our research indefinitely, we will eventually have to focus our efforts on an intervention directly targeted at improving the world. This report outlines plausible candidates for FRI’s “path to impact” and distills some advice on how current movement building efforts can best prepare for them.

Read more

Our Mission

This is a snapshot of the Center on Long-Term Risk’s (formerly Foundational Research Institute) previous "Our Mission" page. The Foundational Research Institute (FRI) conducts research on how to best reduce the suffering of sentient beings in the long-term future. We publish essays and academic articles, make grants to support research on our priorities, and advise individuals and policymakers. Our focus is on exploring effective, robust and cooperative strategies to avoid risks of dystopian futures and working toward a future guided by careful ethical reflection. Our scope ranges from foundational questions about ethics, consciousness and game theory to policy implications for global cooperation or AI safety. Reflectiveness, values and technology The term “dystopian futures” elicits associations of cruel leadership and totalitarian […]

Read more