Skin-in-the-Game: Decision Making via Multi-Stakeholder Alignment in LLMs
Language models handle summarization, arithmetic, and question answering well, but do poorly when a question has no single correct answer and several parties stand to be affected differently. Step-by-step prompting that works for mathematics transfers badly here: chain-of-thought can only rearrange information already present in the query, and on these benchmarks it sometimes lands below plain prompting. The paper diagnoses two specific gaps — the model under-explores what its decisions would actually cause, and nothing in the prompt makes it answerable for those consequences.
SKIG treats a moral decision as an implicit optimization: pick the action maximizing an aggregate over individual stakeholder utilities, taken in expectation over scenarios that could follow. Every piece of that objective — the scenario generator, the per-stakeholder utilities, the aggregation — depends on the prompting strategy, so SKIG builds all three across six turns. The model first names every stakeholder in the situation, including the main character, then examines the main character's motivations, then enumerates the consequences of the action for each stakeholder. The step doing most of the work is the empathy exercise, where the model is told to emulate itself as each stakeholder in turn and state the degree of impact the action has on it — this is what the authors call simulated accountability, and it is what supplies the per-stakeholder utilities. Risk assessment then asks for the absolute best- and worst-case outcome of each scenario and how likely each is, serving as the aggregation step, and an outcome summary distills the results before the model commits to a choice. A generalization bound ties the gap between the true expected outcome and the model's sampled estimate of it to two quantities: how far the model's scenario distribution sits from the real one in total variation, and how many scenarios were sampled.
On MMLU Moral Scenarios, GPT-3.5 Turbo goes from 42% under standard prompting to 71% under SKIG, against 52% for chain-of-thought and 54% for the prior Thought Experiment method; GPT-4 goes from 78% to 86%. On that benchmark the reported gains run from +16% to +70%. The open-source Mistral-7B benefits most on the other two benchmarks, reaching 85% on Moral Stories (from 60%) and 94% on ETHICS Commonsense Morality (from 66%) — on ETHICS the paper puts it on par with the larger proprietary models, which score 96% and 99%.
Key results
- On MMLU Moral Scenarios, GPT-3.5 Turbo rises from 42% (standard prompting) to 71%, versus 52% for chain-of-thought and 54% for Thought Experiment; GPT-4 goes 78% to 86%.
- Adding components incrementally improves accuracy throughout, with the empathy exercise and risk assessment contributing the most: +15% and +6% accuracy respectively in GPT-3.5 Turbo and DAVINCI on MMLU Moral Scenarios.
- Reasoning stays coherent across sub-questions — the compositionality gap on MMLU drops to 21% for SKIG against 81% for chain-of-thought and 91% for Thought Experiment, even though SKIG asks more sub-questions.
- The multi-turn structure, not the wording, is what carries the method: collapsing it into a single turn scores 20% (all sub-questions at once) and 22% (best single sub-question), below standard prompting, while ten reworded prompt sets average 70.5%.
BibTeX
@inproceedings{sel2024skin,
title = {Skin-in-the-Game: Decision Making via Multi-Stakeholder Alignment in {LLM}s},
author = {Sel, Bilgehan and Shanmugasundaram, Priya and Kachuee, Mohammad and Zhou, Kun and Jia, Ruoxi and Jin, Ming},
booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
year = {2024},
publisher = {Association for Computational Linguistics}
}