The Steerability of Generative Models: Towards Bicycles for the Mind

Abstract

Generative models have rapidly advanced in their ability to produce diverse, high-quality outputs. Yet their practical utility often falls short: users frequently struggle to guide models toward desired outputs, even when the model is capable of producing those outputs. This thesis argues that unlocking the full potential of generative AI requires not only improving what models can produce (producibility), but also how effectively users can guide them toward producible outputs (steerability). In short, how can we make the entire producible sets of generative models easily accessible to humans? Our contributions are fourfold. First, we formally define steerability and introduce a framework for evaluating it independently of producibility. Second, we instantiate this framework through benchmarks on the steerability of text-to-image and language models. We find that not only is steerability poor, but steering doesn’t reliably improve with more attempts. Third, we propose a framework for designing and optimizing steering mechanisms – tools that help users articulate and achieve their goals with models – and introduce Reinforcement Learning for Human Steering (RLHS) to systematically optimize these mechanisms. Finally, we instantiate this framework in a new steering mechanism for image generation that enables users to steer via images rather than text prompts. This mechanism achieves over 2x improvement over traditional text-based prompting on our benchmark. Our mathematical framework provides a generalizable path forward for measuring and improving the steerability of generative models, while our implementations of that framework empirically demonstrate its utility and viability. Overall, we define a new axis – steerability – upon which we can vastly improve generative models not only as tools for automation, but as bicycles for the mind.