Prompt

Prompt sensitivity is why methodology matters. "Best roofer in Phoenix" and "trustworthy roofer for an insurance claim in Phoenix" retrieve different evidence and surface different names. A measurement that hides its prompts cannot be compared, repeated, or trusted.

The comparison to survey design is exact, and the same failure modes apply. A leading prompt produces the answer it implies. A prompt naming one company invites the assistant to discuss that company. A prompt phrased the way an agency talks rather than the way a homeowner talks retrieves agency content. In each case the result is real — the assistant genuinely said it — and meaningless as a description of what buyers encounter.

Two design choices carry most of the weight. Prompts should be written the way customers actually speak, which for local services means describing a situation rather than issuing a query. And a set should mix phrasings deliberately: straightforward commercial questions, urgency-driven ones, and qualifying ones about price or credentials. A market can look settled under one phrasing and wide open under another, and only a spread reveals that.

Context beyond the question also shapes the answer. Whether web search is enabled, whether a location is supplied, and whether earlier turns in a conversation have already named businesses all change what comes back. Measurements that vary these silently are not comparable across runs, which is why ours holds them fixed and states them.

Ours are public: six buyer questions per trade, mixing straightforward commercial phrasing with situational ones, listed verbatim on every research page. Reproducibility is what separates research from content — if you cannot re-run someone's prompts and get a similar picture, you have been shown a conclusion rather than a finding.