We get called in two ways: before a founder signs with a development agency, or eighteen months after. The second call is more common and enormously more expensive. This piece is an attempt to convert type-two calls into type-one.
Agencies — including the good ones — run polished sales processes tuned to founder pattern-matching: impressive portfolio, confident timeline, a price that feels like a stretch but survivable. The pitch tells you nothing about the next eighteen months. These questions do.
The questions
1. "Who exactly will write my code?" The pitch team is not the delivery team. Meet the actual engineers or assume a bait-and-switch.
2. "Show me something you shipped two years ago that's still running." Demos age like fruit. Production systems age like evidence.
3. "What happens when you're wrong about an estimate?" Listen for a process — early warnings, scope trades. "We're rarely wrong" means you found the door.
4. "Walk me through a handoff to an in-house team." Shops that build for their own replacement describe this fluently. Shops whose business model is lock-in change the subject.
5. "What would you push back on?" Ask it about your own idea. If they love everything you say, you're hiring a feature factory with a account manager.
6. "How do you handle security, specifically?" Secrets management, dependency updates, access reviews. Vibes are not a control.
7. "What's in the repo besides code?" Setup docs, deploy docs, a README a stranger could follow. Otherwise you're buying a puzzle with a subscription.
8. "Whose name is on the cloud account?" Yours. From day one. Also the domain, the app store listing, and the repo. Non-negotiable, in writing.
9. "What does 'done' mean on this project?" Tested how, deployed where, documented for whom. Ambiguity here is where the budget dies.
10. "Introduce me to a client whose project went badly." Everyone has one. The honest shops make the intro. That reference call outweighs the entire portfolio page.
Scoring it
There's no points system. There's a pattern: good shops expect to be replaced someday and act like it; bad shops build moats. Every question above is really that one question wearing different clothes. Two or three moat-shaped answers and you should keep looking — the market is deep, and the switching cost after signing is brutal.
If you've already signed
Run questions 7 and 8 today, as an audit. Access to your own repo, cloud account, and credentials is the difference between a rough breakup and a hostage negotiation. If the answers are wrong, fixing that — politely, contractually, this month — is more urgent than any feature on the roadmap.
And if you want a neutral read on what an agency has actually built for you so far, that's precisely what a technical health check is for. It's free, and "the agency knows someone is looking" has a remarkable effect on delivery quality all by itself.