Our mission is to identify cooperative and effective strategies to reduce involuntary suffering. We believe that in a complex world where the long-run consequences of our actions are highly uncertain, such an undertaking requires foundational research. Currently, our research focuses on reducing risks of dystopian futures in the context of emerging technologies. Together with others in the effective altruism community, we want careful ethical reflection to guide the future of our civilization to the greatest extent possible.
Approval-directed agency and the decision theory of Newcomb-like problems
The quest for artificial intelligence poses questions relating to decision theory: How can we implement any given decision theory in an AI? Which decision theory (if any) describes the behavior of any existing AI design? This paper examines which decision theory (in particular, evidential or causal) is implemented by an approval-directed agent, i.e., an agent whose goal it is to maximize the score it receives from an overseer.
Download
Read online
Robust program equilibrium
One approach to achieving cooperation in the one-shot prisoner’s dilemma is Tennenholtz’s program equilibrium, in which the players of a game submit programs instead of strategies. These programs are then allowed to read each other’s source code to decide which action to take. Unfortunately, existing cooperative equilibria are either fragile or computationally challenging and therefore unlikely to be realized in practice. This paper proposes a new, simple, more efficient program to achieve more robust cooperative program equilibria.
Download
Read online
Superintelligence as a Cause or Cure for Risks of Astronomical Suffering
Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, usually understood as the risk of human extinction. We argue that suffering risks (s-risks) present comparable severity and probability. Just as with existential risks, s-risks can be caused as well as reduced by superintelligent AI.
Download
Read online
Cause prioritization for downside-focused value systems
This post discusses cause prioritization from the perspective of downside-focused value systems, i.e. views whose primary concern is the reduction of bads such as suffering. According to such value systems, interventions which reduce risks of astronomical suffering are likely more promising than interventions which primarily reduce extinction risks.
Read online
EAF/FRI are now the Center on Long-Term Risk (CLR)
We have renamed the Foundational Research Institute (FRI) to the Center on Long-Term Risk (CLR) and will stop using the Effective Altruism Foundation (EAF) brand (except as the name of our legal entities).
Read more
Our plans for 2020
We describe CLR's plans for 2020 and give an overview of our successes and mistakes in 2019.
Read more
Risk factors for s-risks
Traditional disaster risk prevention has a concept of risk factors. These factors are not risks in and of themselves, but they increase either the probability or the magnitude of a risk. For instance, inadequate governance structures do not cause a specific disaster, but if a disaster strikes it may impede an effective response, thus increasing the damage. Rather than considering individual scenarios of how s-risks could occur, which tends to be highly speculative, this post instead looks at risk factors – i.e. factors that would make s-risks more likely or more severe.
Read more
