Design Theater: A Benchmark for Generative UI

Generative user interface tools promise to democratize UI design by turning natural language descriptions into complete interfaces. Tools such as Claude Artifacts, ChatGPT and Replit generate user-facing design rationales that explain their layout, accessibility and design choices. However, it remains unclear whether these stated rationales are actually reflected in the interfaces they produce. We call this disconnect “Design Theater”: plausible and confident design rationales that have little relationship to the actual implementation.

To study this phenomenon, we introduce a benchmark and three metrics for measuring Design Theater. The benchmark includes 24 UI generation tasks spanning structural, styling and functional design requirements. Using it, we evaluate 120 interfaces created by generative UI tools. On average, over 25% of user-facing design rationales are not implemented in the generated interface, and the implementation failure increases to 34% for functional requirements. Tools recognize roughly half of the UX principles embedded in prompts (mean = 0.54), with four of five tools implementing 6% or fewer functional principles.

We also measure interface similarity across tools and find signs of design homogenization, with relatively narrow pairwise differences in overall visual appearance and layout organization and greater spread in color choices. We discuss what these findings mean for the design and evaluation of generative UI tools.