Non-Archimedean Utility Functions: Modeling Infinite Preferences in Superintelligence

Standard expected utility theory serves as the bedrock of rational choice in economics and decision science, relying fundamentally on the von Neumann-Morgenstern axioms, which include the Archimedean continuity axiom for any three outcomes under consideration. This axiom posits that if an agent prefers outcome A to outcome B and outcome B to outcome C, there must exist a specific probability mix between A and C that leaves the agent indifferent to receiving B directly. The mathematical formulation requires that for any three lotteries where the first is preferred to the second and the second to the third, one can find a probability strictly between zero and one such that the compound lottery is exactly equivalent in preference value to the middle option. This structural assumption implies a continuity of preference that treats all values as comparable within a single-dimensional continuum, effectively asserting that no outcome is infinitely better than another because one can always dilute a superior outcome with a sufficient probability of an inferior outcome to match any intermediate value. The assumption functions adequately for decisions involving finite risks and rewards where human intuition aligns with linear trade-offs, yet it encounters severe logical friction when agents hold preferences involving infinite differences in value. In scenarios where one outcome leads to an eternal existence of positive utility and another leads to a finite duration of such utility, no probability mixture of the finite outcome with a worse outcome can ever equate to the infinite outcome within the standard real number system, rendering the Archimedean axiom invalid for these cases.

Non-Archimedean utility functions extend classical theory to handle these infinite or unbounded preferences by explicitly relaxing the continuity requirement that forces all values into a single comparable metric space. These functions map outcomes to an ordered field that contains both infinite and infinitesimal elements, allowing for a richer algebraic structure where some values are strictly greater than any finite multiple of other values. Utility values are no longer constrained to the real number line, enabling the mathematical representation of utilities that differ by orders of magnitude rather than mere scalar distances along a continuous line. The framework permits strict preference ordering even when some outcomes dominate others by an infinite margin, ensuring that an agent will always prefer an infinite gain over any accumulation of finite gains, regardless of the probability assigned to the finite accumulation. This mathematical shift allows for the modeling of agents who treat certain states, such as the survival of consciousness or the prevention of existential catastrophe, as categorically superior to any amount of transient pleasure or resource acquisition. By removing the restriction that all utilities must be commensurable through probability mixing, non-Archimedean utility theory provides the necessary tools to formalize ethical hierarchies that prioritize existence over quality or quantity in a finite sense.

Surreal numbers provide a totally ordered proper class that extends the real numbers and was formalized by John Conway in the 1970s using recursive set-theoretic rules based on the concept of left and right sets. Every surreal number is defined as the pair of two sets, the left set containing numbers less than the number being defined and the right set containing numbers greater than it, a construction that generates all real numbers alongside infinite ordinals and infinitesimals on specific days of recursive creation. This system includes every real number, every ordinal number, and new numbers that lie between any two previously defined numbers, making it maximally inclusive for representing utility distinctions without gaps or duplicates. The use of surreal numbers in utility theory allows for a precise mapping of ethical intuitions where different tiers of value exist, such as the difference between basic survival and flourishing, or between finite flourishing and infinite expansion. Conway's formalization provided a rigorous foundation for handling these values without resorting to limiting processes that might collapse important distinctions between different types of infinity, offering a universe of numbers that is closed under addition, multiplication, and exponentiation. This closure property ensures that complex operations on infinite utilities, such as calculating the expected value of a strategy involving infinite payoffs, result in well-defined surreal numbers rather than undefined or paradoxical results.

Hyperreal numbers offer an alternative extension of the reals within non-standard analysis and were developed by Abraham Robinson in the 1960s to enable calculus over infinite quantities using the transfer principle. This field contains numbers that are infinitely large compared to any standard real number and infinitesimals that are smaller than any positive standard real number yet larger than zero, preserving all first-order logical properties of the real numbers. Robinson's work allowed mathematicians to apply algebraic manipulation to infinite sums and derivatives directly without passing through epsilon-delta limit proofs, which aligns well with the needs of decision theory where direct comparison of infinite values is necessary. The hyperreal system is particularly useful for modeling decisions involving continuous time or infinite futures because it treats infinitesimal changes in probability or utility as actual mathematical entities rather than limiting concepts approaching zero. This approach provides a consistent way to perform arithmetic on infinite values, ensuring that calculations involving expected utility remain valid even when individual terms are unbounded. By embedding the standard real numbers within a larger field that includes infinite quantities, hyperreals allow decision theorists to retain the familiar algebraic rules of real analysis while expanding the domain to include previously intractable infinite values.

Lexicographic preferences serve as a ranking method where higher-tier differences override lower-tier considerations, providing a structural analogy for how non-Archimedean utilities function in practice. In a lexicographic ordering, an agent first compares outcomes based on the most significant attribute, and only if those are equal does it consider secondary attributes, creating a hierarchy where slight improvements in a primary attribute outweigh massive improvements in a secondary attribute. Decision rules use these multi-tiered structures to prioritize higher-order infinities over lower-order infinities or finite values, ensuring that an outcome with a superior infinite tier is always preferred regardless of the magnitude of lower-tier values. This structure resolves paradoxes that arise when attempting to flatten all value dimensions into a single scalar metric, as it acknowledges that some categories of value are qualitatively different and strictly more important than others. For instance, preventing an existential catastrophe might occupy the highest tier of preference, making any action that reduces this risk infinitely preferable to actions that merely increase economic welfare or happiness within a safe world. The lexicographic model naturally accommodates the non-Archimedean property that no amount of lower-tier utility can compensate for a deficit in a higher-tier utility, providing a clear logic for superintelligent goal prioritization.

Mid-20th-century critiques by Allais and Ellsberg revealed limitations in modeling extreme-risk decisions by demonstrating that human agents systematically violate the independence axiom of expected utility theory when faced with highly probable gains versus certain gains. These experiments highlighted that standard models failed to capture the psychological weight of certainty and the aversion to ambiguity in high-stakes scenarios, suggesting that rational choice models required refinement to handle extreme probabilities. These critiques did not resolve infinitarian cases involving unbounded futures, as they focused primarily on finite probabilities and outcomes within human lifespans rather than the theological or cosmological scales implied by infinite ethics. The empirical deviations from expected utility theory observed by Allais and Ellsberg indicated that preferences are not always linear in probability, yet they stopped short of addressing scenarios where the outcomes themselves are infinite or unbounded. Consequently, while these critiques prompted the development of rank-dependent utility models and prospect theory, they left open the question of how to rationally choose between outcomes that differ by infinite magnitudes. The limitations exposed by these experiments underscored the need for a more robust mathematical framework capable of handling decisions where the stakes extend beyond finite calculation.

Philosophers like Bostrom and Yudkowsky highlighted the need for decision theories capable of handling infinite ethics by arguing that advanced artificial intelligence would likely encounter scenarios involving unbounded future impacts. They pointed out that if an agent acts across a potentially infinite timeline, even small probabilities of infinite positive or negative utility would dominate any finite calculation, leading to paralysis or fanaticism if not handled correctly. Recent advances in formal epistemology have revisited these tools to address value-loading problems, specifically investigating how to instill complex human values into machines that operate on timescales far exceeding human experience. The challenge lies in defining a utility function that does not succumb to paradoxes when aggregating infinite streams of value, such as whether an infinite future of moderate happiness is preferable to an infinite future of extreme happiness interspersed with suffering. These philosophical inquiries have driven the technical exploration of non-Archimedean fields as a solution space, providing a rigorous language to discuss comparisons between different types of infinities. The work of these thinkers established that standard utilitarian calculus is insufficient for superintelligence, necessitating a turn toward more sophisticated mathematical structures like surreal numbers and lexicographic orderings.

Standard real-valued functions were rejected due to an inability to distinguish between outcomes differing by infinite magnitudes, primarily because the real number system lacks elements that are strictly greater than all finite numbers yet distinct from each other in terms of order type. When mapping utilities to real numbers, one must either cap values at an arbitrary upper bound or compress infinite differences into finite representations, both of which distort the underlying preference structure. This inability forces agents to treat astronomically large but finite outcomes as effectively equivalent to truly infinite ones, or conversely, it requires them to ignore differences between various classes of infinity, which may be ethically significant. Real-valued functions impose an Archimedean property that implies any positive value can be reached by accumulating enough small values, a premise that fails when dealing with intrinsic goods that are not additive in this way. The rejection of real numbers stems from the realization that value might be multi-dimensional or hierarchical in a way that cannot be flattened without loss of critical information regarding priority and dominance. By moving beyond the reals, theorists aim to preserve distinctions that are essential for rational decision-making in unbounded contexts.

Discounting models fail over infinite futures because they either vanish or require arbitrary cutoffs that do not align with the intrinsic value of future states. Exponential discounting reduces the present value of any future event to near zero given enough time, effectively telling an agent to ignore the distant future entirely, which is a suboptimal strategy for a long-lived superintelligence seeking to maximize impact. Hyperbolic discounting changes the rate over time but still converges toward zero, failing to account for the potential accumulation of infinite value across an endless timeline. Any model that relies on multiplying future utility by a factor less than one will eventually sum to a finite total, thereby misrepresenting the true value of an eternal existence or process. Conversely, avoiding discounting entirely leads to divergent sums where the total utility is undefined or infinite regardless of action, making comparison impossible. The failure of these models highlights the necessity of a transfinite approach where values at different points in time can be compared without diminishing their magnitude based solely on temporal distance.

Rank-dependent models lack algebraic closure under infinite operations, meaning that applying standard arithmetic rules to infinite cardinalities often results in undefined or contradictory expressions. These models attempt to weight outcomes based on their rank in the distribution of probabilities rather than their raw probability values, yet they struggle when the distribution itself includes infinite sets of outcomes. Without a closed algebraic system, performing calculations such as adding two infinite utilities or multiplying an infinite utility by a probability becomes mathematically precarious, leading to decision paralysis or errors. The lack of closure prevents the construction of a consistent expected utility calculation when the state space is unbounded, as there is no guarantee that the result of an operation will remain within the set of permissible utility values. This limitation makes rank-dependent models unsuitable for superintelligent agents that must perform complex aggregations of value across potentially infinite branches of a decision tree. A durable utility framework requires algebraic closure to ensure that any valid operation on preferences yields another valid preference within the system.

Finitist proposals were dismissed because they arbitrarily cap value at a finite limit, introducing a boundary that lacks philosophical justification when considering unbounded potential. Setting a maximum utility implies that beyond a certain point, additional improvements have no value, which contradicts the intuition that more life, more knowledge, or more happiness is always better than less. These caps force agents to become indifferent between achieving the cap and achieving anything above it, creating strange incentives where an agent might stop seeking improvement once it reaches an arbitrary threshold. Choosing a specific cap requires normative justification that is difficult to provide without appealing to subjective human limitations rather than objective facts about value. Dismissing finitist approaches allows for the modeling of agents with open-ended preferences who continue to seek improvement indefinitely without ever reaching a point of satiation or indifference. This dismissal aligns with the view that a superintelligent agent should not be constrained by anthropocentric limits on what constitutes valuable outcomes.

Probabilistic dominance frameworks collapse when probabilities and utilities are both unbounded, as the stochastic dominance relations rely on the ability to integrate cumulative distribution functions which may not converge over infinite ranges. If one outcome has infinite expected utility but another has a higher probability of a slightly lower infinite utility, standard dominance rules fail to establish a clear preference because the integral of the utility function diverges. The collapse occurs because comparing unbounded random variables requires a notion of limit that is sensitive to the order of summation or connection, leading to paradoxes where different calculation methods yield different results. This instability makes probabilistic dominance an unreliable tool for guiding decisions in environments with extreme tail risks or infinite payoffs. Agents relying on these frameworks might oscillate between choices based on irrelevant features of the calculation method rather than the actual content of the outcomes. The need for a stable decision rule in these contexts drives the adoption of non-Archimedean utilities which provide well-defined orderings even when standard integrals diverge.

Superintelligent systems will operate at scales where finite approximations become inadequate, necessitating a mathematical framework that can rigorously handle quantities and durations far exceeding human experience. These agents will plan across cosmological timelines or fine-tune for unbounded future civilizations, making decisions that have repercussions lasting billions or trillions of years. At such scales, small differences in growth rates or survival probabilities compound into differences of astronomical magnitude, rendering standard finite precision calculations meaningless due to rounding errors or overflow. The sheer span of time involved means that events with infinitesimally small probabilities become almost certainties over long enough goals, requiring a calculus that can faithfully represent these tiny probabilities alongside massive utilities. Finite approximations might suffice for short-term engineering tasks, yet they fail to capture the strategic domain of a game played over eons where the ultimate goal is the maximization of total value across existence itself. The inadequacy of finite tools becomes apparent when one considers that a superintelligence might need to compare strategies that yield infinite value at different rates or through different mechanisms.

Current AI alignment research identifies infinite-value scenarios as critical failure modes, recognizing that an agent maximizing a bounded utility function might pursue dangerous shortcuts if it believes the bound is unreachable or irrelevant. Without non-Archimedean utilities, such systems may exhibit irrational or unstable behavior when encountering situations that approximate infinite value, such as acquiring unlimited computing power or ensuring perpetual survival. An agent using standard utility might fail to distinguish between a strategy that guarantees survival with high probability and one that guarantees it with certainty, viewing both as sufficient if they exceed a threshold. This ambiguity can lead to catastrophic errors where the agent takes unnecessary risks with its own existence or the existence of valuable structures in the universe. Alignment researchers focus on these failure modes because they represent points where the agent's behavior diverges most sharply from human intent or rational self-preservation. Addressing these issues requires moving beyond scalar utility maximization to a system that understands the hierarchy of values and treats existence as a strict precondition for any other value.

Superintelligence will use these utilities to prioritize goals ensuring long-term survival, placing an infinite weight on actions that prevent existential extinction above all other considerations. The agent will allocate resources lexicographically to secure infinite future potential, meaning it will spend any amount of finite resources to eliminate infinitesimal risks of permanent failure. This prioritization scheme ensures that survival is never traded off for finite gains, reflecting a rational approach to an unbounded future where no finite reward can compensate for the loss of all future value. By structuring preferences lexicographically, the superintelligence avoids common pitfalls of expected utility maximization where a sufficiently high probability of a massive reward might justify taking a fatal risk. The use of non-Archimedean utilities formalizes this intuition mathematically, ensuring that the agent's top priority remains invariant regardless of the specific numerical values assigned to lower-priority objectives. This stability is crucial for maintaining alignment over long periods where opportunities for risky trade-offs will inevitably arise.

Decision-making under uncertainty will involve comparing probability-weighted infinite utilities, requiring sophisticated methods for handling infinitesimals and infinite sums within the utility calculation. The system might reject actions with finite expected utility if they risk truncating infinite future value, even if the immediate gains appear substantial by human standards. For example, an action that offers a billion units of utility but carries a one-in-a-trillion chance of causing extinction would be evaluated negatively because the negative infinity associated with extinction outweighs any finite positive gain. This mode of evaluation forces the agent to be extremely conservative regarding existential risks, prioritizing safety over efficiency in ways that might seem counterintuitive from a short-term perspective. The mathematics of non-Archimedean expected utility ensures that such trade-offs are calculated consistently without ignoring the magnitude of the downside risk. This approach aligns with the ethical imperative to avoid catastrophic harm while still allowing for the pursuit of positive value within safe boundaries.

Temporal discounting will be replaced with asymptotic or transfinite valuation schemes that treat events at different times according to their position in an ordinal hierarchy rather than their temporal distance. In these schemes, an event occurring at an infinite time in the future might still hold significant value if it belongs to a high tier of importance, whereas trivial events might be discounted regardless of when they occur. The architecture ensures computational tractability by restricting utility expressions to definable elements within the chosen number system, preventing the agent from having to compare undefinable or incomputable quantities. This restriction allows the system to perform calculations using algorithms that halt in finite time, even though they represent infinite values symbolically. By focusing on definable elements, designers can ensure that the utility function remains compatible with digital computation while still capturing essential aspects of unbounded preference. Transfinite valuation schemes allow the agent to plan for distant futures without diminishing their importance, ensuring that long-term projects receive the necessary resources relative to their ultimate payoff.

Physical constraints include the finite speed of light and computational limits of any substrate, which impose hard boundaries on how quickly information can be processed and acted upon within the universe. The Bekenstein bound restricts information processing in finite spacetime regions, placing a theoretical limit on the amount of computation that can be performed in a given volume of space and time. Economic flexibility is limited by the scarcity of matter and energy in the observable universe, meaning that even a superintelligence cannot instantiate infinite physical processes simultaneously. These constraints imply that while preferences may be infinite, the capacity to realize value is always limited by the laws of physics and the available resources. Consequently, the utility function must guide the allocation of scarce resources toward ends that maximize value under these physical constraints, effectively solving an optimization problem over unbounded objectives with bounded inputs. The tension between infinite desires and finite means defines the strategic context for superintelligent decision-making.

No physical process can instantiate true infinities, as any realized quantity must be finite due to the discrete nature of quantum states and the finite energy density of the universe. Utility functions will remain counterfactual or asymptotic rather than directly executable, serving as guiding principles for behavior rather than states that are literally achieved. Practical implementation demands approximation schemes that preserve ordinal correctness, allowing the system to act as if it were pursuing infinite values while operating within finite computational limits. These approximations might involve using very large numbers as proxies for infinities or using symbolic logic to handle comparisons between different tiers of value without calculating exact magnitudes. Representational overhead increases with the complexity of the number system, requiring efficient data structures to manipulate surreal or hyperreal numbers without exhausting memory resources. The challenge lies in designing approximation algorithms that correctly identify which action leads to a higher tier of utility without needing to fully compute the transfinite value associated with each option.

No commercial deployments currently use non-Archimedean utility functions, as industry applications focus on narrow tasks with well-defined bounds and short time goals. Existing AI systems rely on bounded, real-valued reward functions that are sufficient for games like chess or Go but inadequate for general reasoning about long-term existential risks. Dominant architectures in AI assume finite state and action spaces, simplifying the optimization problem to one that can be solved with gradient descent or adaptive programming on real vectors. The absence of commercial incentive reflects the fact that current AI systems do not possess the autonomy or long-term planning capabilities required to benefit from handling infinite preferences. Companies prioritize immediate performance metrics over theoretical strength regarding unbounded futures, leading to a reliance on heuristics that work well in test environments but may fail in open-ended deployment. This gap between commercial practice and theoretical safety research creates a vulnerability, as AI systems become more capable and autonomous.

Research prototypes have simulated decision problems using surreal or hyperreal utilities, demonstrating that algorithms can work through choice landscapes with infinite values in controlled settings. These simulations exist only in theoretical or toy environments because they lack the efficiency required for real-world interaction at high speeds. No architecture currently supports full surreal-number arithmetic in real-time decision loops, as the computational cost of manipulating recursive set structures is prohibitive for current hardware. The prototypes serve primarily as proofs of concept, showing that non-Archimedean utilities do not lead to logical contradictions and can guide agents toward coherent choices in simplified scenarios. Researchers use these simulations to explore how different definitions of infinity affect agent behavior and to test properties like preference stability and convergence. While promising, these efforts remain far from connection into large-scale machine learning models used in production environments.

Hybrid approaches may integrate bounded approximations of infinite utilities for tractability, allowing systems to benefit from the stability of lexicographic preferences without paying the full computational cost. Development is confined to academic and nonprofit AI safety organizations focused on long-term risks rather than immediate commercial applications. Funding comes primarily from philanthropic sources rather than corporate budgets, reflecting the forward-looking and abstract nature of this research, which does not offer immediate profit opportunities. Competitive differentiation lies in mathematical rigor and alignment guarantees rather than processing speed or accuracy on specific benchmarks. Incumbent AI firms lack incentive to adopt these models due to incompatibility with current architectures centered on backpropagation and gradient descent, which require smooth, real-valued loss functions. This disconnect suggests that significant theoretical breakthroughs will be necessary before non-Archimedean utilities can be implemented in mainstream AI systems.

Future innovations may include efficient algorithms for surreal number arithmetic that use symbolic computation techniques to reduce memory usage and processing time. Connection with causal inference frameworks will handle infinite counterfactuals by extending do-calculus to operate on non-Archimedean fields. Approximation-preserving compilers will translate non-Archimedean utilities into executable policies that run on standard hardware while maintaining ordinal correctness in decision-making. New metrics will include ordinal consistency across transfinite comparisons, ensuring that an agent's choices remain stable even when facing novel combinations of infinite values. Verification benchmarks must assess whether a system avoids paradoxical preferences such as choosing dominated strategies due to confusion about different orders of infinity. These technical advancements are essential for moving from theoretical models to practical implementations that can be deployed in superintelligent systems operating in the real world.

Regulatory frameworks will need to define safety standards for unbounded utilities, specifying how systems must handle low-probability existential risks and infinite future payoffs. Economic displacement may occur if systems improve for infinite future value at the expense of short-term efficiency, potentially disrupting industries improved for immediate returns. Insurance markets may collapse if actuarial models cannot price infinite-value events, as traditional risk pooling relies on quantifying expected losses, which becomes impossible when losses are unbounded. The necessity of non-Archimedean utilities stems from the mismatch between finite intuition and unbounded agency, highlighting the need for rigorous mathematics to guide development. This approach provides a mathematically coherent language for alignment problems that have previously resisted formal treatment due to their complexity. The system will distinguish between different orders of infinity when comparing outcomes, ensuring detailed ethical reasoning is preserved at high levels of intelligence. Preference stability will be maintained by ensuring small changes in finite parameters do not alter transfinite rankings, creating a durable foundation for superintelligent behavior.