Artificial intelligence (AI) will likely transform the world later this century. Whether uncontrolled or controlled AIs would create more suffering in expectation is a question to explore further. Regardless, the field of AI safety and policy seems to be a very important space where altruists can make a positive-sum impact along many dimensions.

This post discusses cause prioritization from the perspective of downside-focused value systems, i.e. views whose primary concern is the reduction of bads such as suffering. According to such value systems, interventions which reduce risks of astronomical suffering are likely more promising than interventions which primarily reduce extinction risks.

Two crucial questions in discussions about the risks of artificial superintelligence are: 1) How much more powerful could an AI become relative to humans, and 2) how easily could superhuman capability be acquired? To answer these questions, this article reviews the literature on human expertise and intelligence and discusses its relevance for AI.

Will we go extinct, or will we succeed in building a flourishing utopia? Discussions about the future trajectory of humanity often center around these two possibilities, which tends to ignore that survival does not always imply utopian outcomes, or that outcomes where humans go extinct could differ tremendously in how much suffering they contain.

Evaluating the effectiveness of our actions, or even just whether they're beneficial or harmful, is very difficult. One way to deal with uncertainty is to focus on actions that likely have positive effects across many scenarios. This approach often amounts to meta-level activities like promoting positive-sum institutions, reflectiveness, and effective altruism in general.

FRI’s research seeks to identify the best intervention(s) for suffering reducers to work on. Rather than continuing our research indefinitely, we will eventually have to focus our efforts on an intervention directly targeted at improving the world. This report outlines plausible candidates for FRI’s “path to impact” and distills some advice on how current movement building efforts can best prepare for them.

There's a decent chance that governments will be the first to build artificial general intelligence (AI). International hostility, especially an AI arms race, could exacerbate risk-taking, hostile motivations, and errors of judgment when creating AI. If so, then international cooperation could be an important factor to consider when evaluating the flow-through effects of charities.

Fast technological development carries a risk of creating extremely powerful tools, especially AI, before society has a chance to figure out how best to use those tools in positive ways for many value systems. Suffering reducers may want to help mitigate the arms race for AI so that AI developers take fewer risks and have more time to plan for how to avert suffering that may result from the AI's computations. The AI-focused work of the Machine Intelligence Research Institute (MIRI) seems to be one important way to tackle this issue. I suggest some other, broader approaches, like advancing philosophical sophistication, cosmopolitan perspective, and social institutions for cooperation. As a general heuristic, it seems like advancing technology may be net […]

Several arguments support the heuristic that we should help groups holding different value systems from our own when doing so is cheap, unless those groups prove uncooperative to our values. This is true even if we don't directly care at all about other groups' value systems. Exactly how nice to be depends on the particulars of the situation.

Global catastrophic risks – such as biotech disasters or nuclear war – would cause major damage in the short run, but their effects on the long-run trajectory that humanity takes are also significant. In particular, to the extent these disasters increase risks of war, they seem likely to precipitate AI arms races between nations and worsen prospects for compromise.

This article suggests a lower-bound Fermi calculation for the cost-effectiveness of promoting cooperation. The purpose of this exercise is to make our thinking more concrete about how cooperation might reduce suffering and to make its potential more tangible.

Learning is an extremely important activity for altruists. Learning can seem ineffective in the short run, but used properly, it can pay off more than most financial or single-domain-focused investments. It's important for young activists not to neglect learning in order to just "do more to help now."

“Suffering-focused ethics” is an umbrella term for moral views that place primary or particular importance on the prevention of suffering. Most views that fall into this category are pluralistic in that they hold that other things beside suffering reduction also matter morally. To illustrate the diversity within suffering-focused ethics as well as to present a convincing case for it, this article will introduce four separate motivating intuitions.

What makes an experience valuable or disvaluable? In contrast to hedonism, which holds that pleasure is what is good and pain is what is bad, tranquilism is an “absence of desire” theory that counts pleasure as instrumentally valuable only. According to tranquilism, what matters is whether an experience is free from bothersome components. States of contentment such as flow or meditative tranquility also qualify.

It's a classic debate among utilitarians: Should we care about an organism's happiness and suffering (hedonic wellbeing), or should we ultimately value fulfilling what it wants, whatever that may be (preferences)? This article discusses various intuitions on both sides and explores a hybrid view that gives greater weight to the hedonic subsystems of brains than to other overriding subsystems.

An example of value lexicality is that an outcome with both torture and happiness is bad, regardless of the amount of happiness. Value lexicality is important partly because it can lead to suffering-focused ethics. Key topics that this essay explains include strong versus weak lexicality, value aggregation, views on large numbers and sequence arguments.

Two variables seem particularly important when trying to make informed choices about how to best shape the long-term future: One’s normative goods-to-bads ratio and one’s expected bads-to-goods ratio. This essay discusses how one could measure these variables and investigates associated challenges.

Someone who wants to do good is faced with the question how to prioritize preventing badness vs. bringing about more individuals with good lives. A relevant idea is the ‘Asymmetry,’ which roughly says that it is bad to bring into existence individuals with bad lives but not good to add individuals with good lives. One objection to the Asymmetry is extinction thought experiments where the reader is asked to compare one outcome in which humanity survives to one in which it goes extinct. The objection says that it is better if humanity survives, which is taken to be a counterargument to the Asymmetry. But, as Professor Meacham has pointed out, the objection lacks force against the Asymmetry because it drags […]

This piece presents a hypothetical dialogue that explains why instrumental computational processes of a future superintelligence might evoke moral concern. Generally, agent-like components might emerge in many places, including the computing processes of a future civilization. Whether and how much these subroutines matter are questions for future generations to figure out, but it's good to keep an open mind to the possibility that our intuitions about what suffering is may change dramatically.

If we don't understand why we're conscious, how come we're so sure that extremely simple minds are not? I propose to think of consciousness as intrinsic to computation, although different types of computation may have very different types of consciousness – some so alien that we can't imagine them. Since all physical processes are computations, this view amounts to a kind of panpsychism. How we conceptualize consciousness is always a sort of spiritual poetry, but I think this perspective better accounts for why we ourselves are conscious despite not being different in a discontinuous way from the rest of the universe. Introduction "don't hold strong opinions about things you don't understand" --Derek Hess Susan Blackmore believes the way we typically […]