Publications and talks

1. 2026Book

   Radical AI Interpretability

   Daniel Herrmann and Benjamin A. Levinstein. Cambridge Elements in Philosophy of AI, Cambridge University Press (forthcoming).

   A framework for working out an AI system’s beliefs, desires, and meanings from facts about its computation, joining radical interpretation with mechanistic interpretability.

   arXiv:2606.26523Draft (PhilPapers)

   Cite

   @misc{herrmann-levinstein-radical,
     author        = {Herrmann, Daniel A. and Levinstein, Benjamin A.},
     title         = {Radical AI Interpretability},
     year          = {2026},
     eprint        = {2606.26523},
     archivePrefix = {arXiv}
   }

   Copy
2. 2026Paper

   A Decision-Theoretic Approach for Managing Misalignment

   Daniel Herrmann, Abinav Chari, Isabelle Qian, Sree Sharvesh, and Benjamin A. Levinstein. Proceedings of the International Association for Safe and Ethical AI.

   Delegating to an AI system in a given context can be rational despite significant misalignment, when its accuracy or reach outweighs it.

   ProceedingsarXiv:2512.15584

   Cite

   @inproceedings{herrmann-et-al-managing-misalignment,
     author    = {Herrmann, Daniel A. and Chari, Abinav and Qian, Isabelle and Sharvesh, Sree and Levinstein, Benjamin A.},
     title     = {A Decision-Theoretic Approach for Managing Misalignment},
     booktitle = {Proceedings of the International Association for Safe and Ethical AI},
     year      = {2026},
     eprint    = {2512.15584},
     archivePrefix = {arXiv}
   }

   Copy
3. 2026Paper

   Deference and Decision

   Daniel Herrmann and Gerard Rothfus. Theory and Decision.

   No decision theory that agrees with evidential and causal decision theory wherever they agree can both ignore events inconsistent with one’s evidence and defer to a better-informed version of oneself.

   JournalFree version

   Cite

   @article{herrmann-rothfus-deference,
     author  = {Herrmann, Daniel A. and Rothfus, Gerard J.},
     title   = {Deference and Decision},
     journal = {Theory and Decision},
     volume  = {101},
     number  = {2},
     pages   = {401--427},
     year    = {2026},
     doi     = {10.1007/s11238-025-10036-0}
   }

   Copy
4. 2025Paper

   Standards for Belief Representations in LLMs

   Daniel Herrmann and Benjamin A. Levinstein. Minds and Machines.

   Four conditions a representation in a language model must meet to count as belief-like: accuracy, coherence, uniformity, and use.

   JournalarXiv:2405.21030

   Cite

   @article{herrmann-levinstein-standards,
     author  = {Herrmann, Daniel A. and Levinstein, Benjamin A.},
     title   = {Standards for Belief Representations in {LLMs}},
     journal = {Minds and Machines},
     volume  = {35},
     number  = {1},
     pages   = {5},
     year    = {2025}
   }

   Copy
5. 2025Paper

   Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks

   Benjamin A. Levinstein and Daniel Herrmann. Philosophical Studies.

   Leading probes for belief in language models fail to generalize in basic ways, and face conceptual obstacles even if the models have beliefs.

   JournalarXiv:2307.00175

   Cite

   @article{levinstein-herrmann-lie-detector,
     author  = {Levinstein, Benjamin A. and Herrmann, Daniel A.},
     title   = {Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks},
     journal = {Philosophical Studies},
     volume  = {182},
     number  = {7},
     pages   = {1539--1565},
     year    = {2025}
   }

   Copy
6. 2023Dissertation

   Naturalizing Decision Theory

   Daniel Herrmann. PhD dissertation, University of California, Irvine.

   How an agent can see herself both as part of the world and as an agent. The propositions she takes herself to control can be read off her beliefs and desires.

   eScholarship

   Cite

   @phdthesis{herrmann-naturalizing,
     author = {Herrmann, Daniel A.},
     title  = {Naturalizing Decision Theory},
     school = {University of California, Irvine},
     year   = {2023}
   }

   Copy
7. 2022Paper

   The Tragedy of the AI Commons

   Travis LaCroix and Aydin Mohseni. Synthese.

   Compliance with AI ethics guidelines as a social dilemma, modeled with stochastic evolutionary game dynamics, and the conditions under which cooperation among developers can hold.

   arXiv:2006.05203

   Cite

   @article{lacroix-mohseni-ai-commons,
     author  = {LaCroix, Travis and Mohseni, Aydin},
     title   = {The Tragedy of the {AI} Commons},
     journal = {Synthese},
     volume  = {200},
     year    = {2022}
   }

   Copy

1. Working paper

   A Bayesian Reduction of Causation

   Daniel Herrmann, Aydin Mohseni, Benjamin A. Levinstein, and Bruce Rushing

   Causal reasoning in structural causal models reduces to subjective probability once interventions are treated as events in the world.

   Preprint

   Cite

   @unpublished{herrmann-et-al-bayesian-causation,
     author = {Herrmann, Daniel A. and Mohseni, Aydin and Levinstein, Benjamin A. and Rushing, Bruce},
     title  = {A Bayesian Reduction of Causation},
     note   = {Manuscript}
   }

   Copy
2. Working paper

   Margins of Misalignment

   Daniel Herrmann and Benjamin A. Levinstein

   When to delegate decisions to an AI system that may not share our values. Delegating every problem requires perfect alignment. Delegating only the problems we expect it to face tolerates a margin of misalignment.

   Cite

   @unpublished{herrmann-levinstein-margins,
     author = {Herrmann, Daniel A. and Levinstein, Benjamin A.},
     title  = {Margins of Misalignment},
     note   = {Manuscript}
   }

   Copy
3. Working paper

   AI Alignment as a Principal–Agent Problem

   Aydin Mohseni and L. G. Demo

   Formal models of delegation, incentives, and misalignment.

   Cite

   @unpublished{mohseni-demo-principal-agent,
     author = {Mohseni, Aydin and Demo, L. G.},
     title  = {AI Alignment as a Principal--Agent Problem},
     note   = {Manuscript}
   }

   Copy
4. Working paper

   Learning the Value of Value Learning

   Alex J. London and Aydin Mohseni

   Extends the Jeffrey–Bolker framework to the refinement of values, with a value-of-information theorem for learning what one values.

   arXiv:2511.17714

   Cite

   @unpublished{mohseni-london-value-learning,
     author = {London, Alex J. and Mohseni, Aydin},
     title  = {Learning the Value of Value Learning},
     note   = {Manuscript}
   }

   Copy
5. Working paper

   Measuring Agency: Belief and Preference Elicitation for Language Models

   Aydin Mohseni and Dorian Clay

   Cite

   @unpublished{mohseni-clay-measuring-agency,
     author = {Mohseni, Aydin and Clay, Dorian},
     title  = {Measuring Agency: Belief and Preference Elicitation for Language Models},
     note   = {Manuscript}
   }

   Copy
6. Working paper

   Bayes-Optimal Research Protocols for Automated Research

   Aydin Mohseni and Simon DeDeo

   How to allocate compute across research directions when AI systems do the research.

   Cite

   @unpublished{mohseni-dedeo-protocols,
     author = {Mohseni, Aydin and DeDeo, Simon},
     title  = {Bayes-Optimal Research Protocols for Automated Research},
     note   = {Manuscript}
   }

   Copy
7. Working paper

   Intrapersonal Comparisons under Preference Uncertainty

   Pietro Cibinel and Daniel Herrmann

   Acting for one person under uncertainty about her preferences raises the problem of interpersonal comparisons within a single person.

   Cite

   @unpublished{cibinel-herrmann-intrapersonal,
     author = {Cibinel, Pietro and Herrmann, Daniel A.},
     title  = {Intrapersonal Comparisons under Preference Uncertainty},
     note   = {Manuscript}
   }

   Copy
8. Working paper

   Subjective Naturalism & Act Probabilities

   Daniel Herrmann

   Can an agent assign probabilities to her own acts while she deliberates? Desirability tracking, the account of agency developed here, says she can. It also reads the decision problems she takes herself to face off her own attitudes.

   Cite

   @unpublished{herrmann-act-probabilities,
     author = {Herrmann, Daniel A.},
     title  = {Subjective Naturalism \& Act Probabilities},
     note   = {Manuscript}
   }

   Copy

Selected talks on AI and agency since 2024, most recent first. Later venues for the same talk are listed together.

1. 2026Talk

   Radical AI Interpretability

   Daniel Herrmann

   Minds, Meaning, and Machines, Hong Kong; AI Agency and Interpretability Workshop, Pittsburgh

2. 2026Talk

   Virtues and Vices of Analytic Philosophy with Respect to AI Safety

   Aydin Mohseni

   ILIAD, Berkeley

3. 2026Talk

   Margins of Misalignment: Delegation to AI Beyond the Bayesian Idealization

   Daniel Herrmann

   PPE Society Annual Meeting, New Orleans

4. 2026Talk

   Learning the Value of Value Learning

   Aydin Mohseni

   PPE Society, London; Beyond Basic Bayesianism Workshop, Helsinki; Formal Ethics Workshop, Buffalo; Agent Foundations Conference, Pittsburgh; earlier at the ICSD Workshop, Pittsburgh, 2024

5. 2026Talk

   A Decision-Theoretic Approach for Managing Misalignment

   Daniel Herrmann

   International Association for Safe and Ethical AI, Paris

6. 2026Talk

   A Bayesian Reduction of Causation

   Aydin Mohseni

   Philosophy of Science Association Biennial, San Diego; Society for the Philosophy of Causation, Paris; PhilML, Munich; earlier at the PPE Society, Agent Foundations Workshop, and Center for Philosophy of Science, Pittsburgh, 2025

7. 2026Talk

   Can We Attribute Beliefs and Desires to AIs? A Radical Approach

   Daniel Herrmann

   Bennington Center for AI Speaker Series, Bennington

8. 2026Talk

   A Mathematical Unification of Learning and Evolution

   Aydin Mohseni

   ILIAD, Berkeley

9. 2026Talk

   Intrapersonal Comparisons under Preference Uncertainty

   Daniel Herrmann

   PPE Society, London; Society for Social Choice and Welfare, Tokyo

10. 2026Talk

    The Metaphor Problem for Belief Representation in Large Language Models

    Daniel Herrmann

    Pacific APA

11. 2026Talk

    Machine Minds and the Propositional Projection Problem

    Daniel Herrmann

    Central APA, Chicago

12. 2026Talk

    Splitting the Atom: Atomless Semantics for Counterfactuals

    Daniel Herrmann

    Philosophy of Science Association Biennial, San Diego; PPE Society Annual Meeting, New Orleans, 2025

13. 2026Talk

    Standards for Belief Representation in LLMs

    Daniel Herrmann

    Bennington Science Workshop; earlier at MCMP, Munich; Philosophy of Science Association Biennial, New Orleans; InDeep Consortium, Groningen; Philosophy of Science Meets Machine Learning, Tübingen, 2024

14. 2025Talk

    Naturalizing Causation in Causal Models

    Daniel Herrmann

    Agent Foundations Workshop, Pittsburgh

15. 2025Talk

    Deference and Decision

    Daniel Herrmann

    Agent Foundations Workshop, Pittsburgh; Pittsburgh Formal Epistemology Workshop; UCI Game Theory Lab, Irvine

16. 2025Talk

    Manageable Misalignment

    Daniel Herrmann

    Agent Foundations Workshop, Pittsburgh; PPE Society Annual Meeting, New Orleans, 2024

17. 2024Talk

    AI Alignment as a Principal–Agent Problem

    Aydin Mohseni

    Foundations of Cooperative AI Lab, Pittsburgh; PPE Society, New Orleans; Philosophy of Science Meets Machine Learning, Tübingen; ILIAD, Berkeley

18. 2024Talk

    Bayesian Epistemology: Foundations, Limits, and Open Questions

    Aydin Mohseni

    ILIAD, Berkeley

19. 2024Talk

    Decision Theory Foundations

    Daniel Herrmann

    ILIAD, Berkeley

20. 2024Talk

    Measuring the Beliefs of LLMs

    Daniel Herrmann

    Purdue University Philosophy Colloquium, West Lafayette; AI and Logic Talk, Groningen

1. 2026Workshop

   Workshop on AI Agency

   Organized by Aydin Mohseni and Daniel Herrmann

   Carnegie Mellon University, Pittsburgh.