Design Theater

Results

Cross-tool mean TFS was 0.75: roughly one stated rationale in four did not fully appear in the interface. Four tools clustered between 0.74 and 0.87; Firebase Studio sat well below at 0.53.

Thinking Fidelity Score by tool

Claude

0.87

Bolt

0.84

ChatGPT

0.80

Vercel v0

0.74

Firebase

0.53

Table 1 - Mean TFS by tool and tier
Tool	Overall	95% CI	Structural	Styling	Functional
Claude	0.87	0.82–0.91	0.83	0.93	0.85
Bolt	0.84	0.75–0.91	0.83	0.95	0.73
ChatGPT	0.80	0.75–0.84	0.85	0.78	0.75
Vercel v0	0.74	0.63–0.84	0.83	0.83	0.56
Firebase	0.53	0.45–0.61	0.62	0.55	0.43
Across tools	0.75		0.79	0.81	0.66

Functional tasks were weakest across the board (0.66), but the shape of failure differs by tool: Claude held steady while Vercel v0 dropped sharply to 0.56 with a wide interval (0.36-0.77). A tool with moderate but stable fidelity is a different risk than one that is unpredictable.

Principle adherence collapses on interaction

Mean PAS ran from 0.29 (Firebase Studio) to 0.73 (ChatGPT). Tools handled structural and styling principles reasonably well and then fell off a cliff: four of five implemented 6% or fewer of the functional principles.

Table 2 - Principle Adherence Score by tool and tier
Tool	Overall	95% CI	Structural	Styling	Functional
ChatGPT	0.73	0.59–0.83	0.94	0.88	0.38
Bolt	0.60	0.46–0.73	0.88	0.88	0.06
Vercel v0	0.56	0.42–0.69	0.81	0.81	0.06
Claude	0.54	0.40–0.67	0.81	0.75	0.06
Firebase	0.29	0.18–0.43	0.50	0.38	0.00
Across tools	0.54		0.79	0.74	0.11

Design Homogenization

Given the same prompt, tools produce narrow differences in visual appearance and layout, while their color choices vary most widely. Claude and Vercel v0 were consistently the closest pair on both visual and color distance.

Table 3 - DHI sub-measures. Lower distance = more similar.
Sub-measure	Most similar pair	Least similar pair	Range	Mean
Visual	Claude – v0	ChatGPT – Firebase	0.119–0.151	0.137
Color	Claude – v0	Bolt – ChatGPT	20.6–39.7	31.9
Layout	Bolt – Claude	Claude – ChatGPT	0.181–0.211	0.196

Figure 2 (web). Spread of pairwise distances per sub-measure. Vermilion ticks mark closest / most-distant pairs; dark tick is the mean.

Thinking Fidelity Score by tool

Claude

0.87

Bolt

0.84

ChatGPT

0.80

Vercel v0

0.74

Firebase

0.53

Table 1 - Mean TFS by tool and tier
Tool	Overall	95% CI	Structural	Styling	Functional
Claude	0.87	0.82–0.91	0.83	0.93	0.85
Bolt	0.84	0.75–0.91	0.83	0.95	0.73
ChatGPT	0.80	0.75–0.84	0.85	0.78	0.75
Vercel v0	0.74	0.63–0.84	0.83	0.83	0.56
Firebase	0.53	0.45–0.61	0.62	0.55	0.43
Across tools	0.75		0.79	0.81	0.66

Principle adherence collapses on interaction

Table 2 - Principle Adherence Score by tool and tier
Tool	Overall	95% CI	Structural	Styling	Functional
ChatGPT	0.73	0.59–0.83	0.94	0.88	0.38
Bolt	0.60	0.46–0.73	0.88	0.88	0.06
Vercel v0	0.56	0.42–0.69	0.81	0.81	0.06
Claude	0.54	0.40–0.67	0.81	0.75	0.06
Firebase	0.29	0.18–0.43	0.50	0.38	0.00
Across tools	0.54		0.79	0.74	0.11

Table 3 - DHI sub-measures. Lower distance = more similar.

Sub-measure

Most similar pair

Least similar pair

Range

Mean

Visual

Claude – v0

ChatGPT – Firebase

0.119–0.151

0.137

Color

Claude – v0

Bolt – ChatGPT

20.6–39.7

31.9

Layout

Bolt – Claude

Claude – ChatGPT

0.181–0.211

0.196