How to Technically Interview a Development Agency (Non-Technical Founders)

Non-technical founders lose more money to agencies that pass the sales call than to the ones that fail it. The obvious failures screen themselves out: a portfolio of template sites, a $4,000 quote for a two-sided marketplace, a proposal riddled with copy-paste errors. The expensive mistakes arrive polished. Clean deck, three case studies, a confident answer to every question you already knew how to ask.

The problem is not that founders ask too few questions. Most have already worked through a standard list of questions to ask a software development company, and every one of those questions has a checkable answer. “Do you do automated testing?” has one correct response and every agency on earth knows it. A useful technical interview software agency process is built the opposite way. You ask questions where no single answer is correct, and you grade the shape of the response instead: whether it contains specifics, trade-offs, and named failure modes, or whether it slides into reassurance.

That distinction matters more than any credential on the proposal. An agency that says “we’d probably denormalize that table and eat the write cost, because your read pattern is 40 to 1” has thought about your project. An agency that says “we follow industry best practices and build for scale” has thought about your signature.

This guide gives you eight questions you can ask without writing a line of code, what a competent answer sounds like, and exactly what hand-waving sounds like when it arrives. It also covers when to stop doing this yourself and pay someone $1,500 to do it for you.

McKinsey’s study of 5,400 large IT projects, conducted with the University of Oxford, found that on average these projects run 45% over budget and 7% over time while delivering 56% less value than predicted. 

What Is a Technical Interview for a Software Agency?

A technical interview software agency evaluation is a structured conversation in which a buyer assesses an agency’s engineering competence through reasoning, trade-off explanation, and failure-handling rather than code review. It replaces credential checking with answer-quality analysis, making it usable by non-technical founders, procurement leads, and operations managers.

The format is deliberately non-adversarial. You are not trying to catch anyone out. You are trying to find out whether the people quoting you have built something similar and remember what hurt.

The Core Problem: Non Technical Founder Vetting Devs Without a Baseline

Every hiring decision needs a reference point. When you hire a head of sales, you have sold something. When you hire a designer, you have opinions about what looks wrong. Software removes that anchor entirely, and it removes it at the exact moment the stakes peak.

Agency engagements for a first production build typically land between $35,000 and $120,000 and run three to seven months, which is roughly in line with what custom software development actually costs at that scope. By the time quality becomes visible, you are usually 60% through the budget. Refactoring a poorly architected build costs 1.5x to 2x the original quote, because someone has to understand the existing code before replacing it.

The information gap is structural. Agencies have run hundreds of these sales conversations and know precisely which reassurances land. You are running your first or second. That asymmetry is why non technical founder vetting devs usually defaults to proxy signals: portfolio logos, Clutch reviews, headcount, office photos.

Those proxies fail in specific ways. Portfolio logos frequently represent white-label subcontract work the agency never architected, which is why it pays to check a software company portfolio properly before the call. Review counts reflect account-management quality, not engineering quality, and the two diverge constantly. Headcount tells you nothing about who gets staffed on your project, which is usually the difference between a good outcome and a technical debt cleanup in month eight.

A properly run technical interview software agency conversation attacks the asymmetry directly. You cannot evaluate the answer’s technical correctness. You can absolutely evaluate whether it is specific, whether it contains a trade-off, and whether the person answering has clearly lived through the scenario before.

How to Assess Dev Skill Without Coding: The Shape-of-the-Answer Method

Three markers separate substantive answers from sales answers, and all three are visible to a non-technical listener.

Specificity under pressure. Competent engineers reach for numbers, names, and constraints unprompted. They say “Postgres, because your reporting needs joins,” not “a robust database solution.”

Volunteered trade-offs. Real engineering decisions cost something. An answer with no downside in it is a marketing answer. Listen for the word “but.”

Named failure modes. People who have shipped software remember what broke. They describe specific incidents, not categories of incident.

Score every answer across those three markers. Below are the eight questions to run in your technical interview software agency session, each with the competent pattern and the evasion pattern.

1. “A requirement changes in week six. Walk me through exactly what happens.”

These tests change control, which is where most fixed-scope engagements quietly die. Strong answers describe a mechanism that mirrors a real software development change order process: an impact assessment, a written change order with revised hours, a decision on what gets pushed out of the current sprint to make room. They will tell you who signs it and how long it takes.

Hand-waving sounds like: “We’re agile, so we’re flexible.” Flexibility is not a process. Push once, and if the second answer is “we’d just absorb it,” you have found an agency that will either eat the cost until quality drops or invoice you in month five with an explanation you cannot audit. Scope creep handling is the single strongest predictor of budget outcome in this entire list.

2. “Who specifically writes our code, and will they be here in month five?”

Bait-and-switch staffing is the most common agency failure and the easiest to detect. Ask for names, seniority, and current allocation percentage. Good answers are uncomfortable and honest: “Two people, one at 100% and one at 50%, and the senior rolls off a project in three weeks.”

Weak answers describe the company rather than the team. “We have 80 engineers” is a deflection, and it is the point where many buyers quietly discover they are dealing with staff augmentation partners rather than a delivery team. Ask whether the people in the sales meeting will be on the project. If nobody in the room writes code, escalate this question during your technical interview software agency call until you get names you can look up on LinkedIn.

3. “What breaks first if we get 10x the traffic we planned for?”

This is the highest-yield of the architecture questions to ask agency teams, because it cannot be answered generically. A real answer names one component: the database connection pool, the image processing queue, an unindexed query on the search page. It also usually says the fix is cheap and can wait.

Hand-waving sounds like: “We build everything to scale” or “we’d move to microservices.” An agency that promises unlimited scale for a pre-revenue product is either overbuilding your budget or has not thought about it. The correct answer to a scaling question is often “nothing, until 50,000 users, and here’s what we’d fix then.”

4. “Describe your testing approach for the checkout flow specifically.”

Naming a real feature prevents the rehearsed answer. Competent responses separate layers: unit tests on payment calculation logic, integration tests against a payment sandbox, manual quality assurance on the full purchase path before release, which is the same scope you would define if you were briefing a team to post a testing and QA project. They will tell you what they do not test and why, because 100% coverage is a red flag, not a credential.

Hand-waving sounds like: “We have comprehensive test coverage” or “our QA team tests everything.” Follow up with “what did you not automate on your last project?” Silence there tells you testing is aspirational, and it is a signal to get acceptance criteria written into the contract before any money moves.

5. “Show me your estimate, and tell me which line is weakest.”

Every estimate contains one item the agency is guessing at. Asking them to name it tests honesty and self-awareness simultaneously, and it is far more revealing than trying to compare software development quotes on totals alone. Strong answers point at the integration nobody controls: the third-party API integration, the legacy system migration, the client-side approval loop.

Hand-waving sounds like: “We’ve built this before, so the estimate is solid.” Confidence across every line item means nobody stress-tested the numbers. Agencies that name their riskiest estimate in a technical interview software agency conversation are the ones who will flag overruns in week four instead of week fourteen.

6. “Walk me from a developer’s commitment to it being live for users.”

This maps the DevOps pipeline and the code review process in one question. Good answers are a sequence: pull request, review by a second engineer, automated checks, staging deployment, release. They will tell you how often they deploy and how they roll back.

Google’s DORA research consistently finds deployment frequency and change failure rate to be reliable indicators of delivery performance. Hand-waving sounds like: “Our lead handles deployments.” One person with production access is a bus-factor problem and usually means no review culture.

7. “Who owns the code, the repositories, the cloud account, and the third-party keys?”

Source code ownership disputes are the ugliest way an engagement ends. The right answer is immediate and boring: you own everything, repositories sit in your organization from day one, cloud accounts are in your name with the agency granted access.

Hand-waving sounds like: “We’ll transfer everything at handover.” Delayed transfer is leverage, and leverage gets used during disputes, which is exactly the gap a software escrow agreement is designed to close. This question costs nothing to ask and has saved buyers six-figure exit costs.

8. “Tell me about a project that went badly and what changed afterward.”

This tests whether the organization learns. Substantive answers are specific and slightly unflattering: a missed integration deadline, an underestimated discovery phase, a client who could not provide content. Crucially, they describe a process change that followed.

Hand-waving sounds like: “We haven’t really had one” or a story where the client is the only villain. Every agency with real volume has shipped something late. The ones that cannot say so out loud will not tell you when yours is slipping either, and that is how buyers end up having to exit a software development contract mid-build.

A 7-Step Technical Screening Agency Process

Run these in order. The full technical screening agency sequence takes eight to twelve working days across a shortlist of three, which is realistic if you already know how to shortlist an IT agency in 72 hours.

1. Shortlist three to five verified agencies with relevant domain experience, not just relevant tech stacks.
2. Send a one-page scope with two deliberate ambiguities and see which agencies ask about them.
3. Run a 60-minute call per agency using the eight questions above, with the same interviewer for consistency.
4. Score each answer 0 to 2 on specificity, trade-offs, and named failure modes.
5. Request two reference calls with clients whose projects have ended, not current clients.
6. Verify one portfolio claim independently by asking who architected the system and who maintains it now.
7. Commission a paid discovery phase or independent review before signing the full build.

When to Bring Your Own CTO Review

A bring your own CTO review means paying an independent senior engineer, typically $150 to $250 per hour for six to ten hours, to sit in on the final call and read the proposal. It is the correct move when the build exceeds roughly $60,000, when the product has regulatory exposure, or when two finalists score within two points of each other.

The reviewer is not there to redesign your product. They are there to ask the second and third follow-up questions that you cannot generate in real time, and to read the agreement against a software development contract checklist. Used well, this converts your technical interview software agency process from a screening exercise into genuine vendor due diligence for under $2,000.

Real-World Application

A logistics SaaS founder, non-technical, two finalists. Both quoted within 8% of each other. On question three, one agency named the specific table that would lock up under concurrent driver updates and proposed a $0 fix involving a different write pattern. The other promised a scalable microservices architecture. The founder chose the first agency, shipped in 19 weeks against a 22-week estimate, and spent nothing on re-architecture in the following year.

A D2C brand replatforming a five-year-old storefront. The agency’s answer to the ownership question revealed that the previous vendor still held the cloud account and domain registrar, a reminder that you have to own your code completely when outsourcing rather than assume it. Surfing during the technical interview software agency stage, rather than at migration, added three weeks to the timeline but avoided an estimated $18,000 in emergency recovery work and a projected nine days of downtime.

Decision Framework: Scoring the Answers You Get

Use this as a live scorecard during each call. Four signal areas, scored 0 to 2, gives you a 0 to 8 total per agency and a defensible comparison you can show a board or a co-founder.

A total of 6 or above across all four is a genuine shortlist candidate. Anything at 3 or below should be eliminated regardless of price, because the discount will not cover the rework. Where two agencies tie, weight team continuity highest; sprint velocity and code quality both track the people, not the logo.

What Most Teams Get Wrong

The dominant mistake is treating the technical interview software agency process as a gate rather than a sample. Buyers run one call, feel reassured, and sign, skipping the rest of what it takes to hire a software development company properly. The call is a sample of how this agency communicates under mild pressure, and communication quality is what you are actually purchasing for the next five months. Grade the conversation the way you would grade a work product.

The second mistake is over-indexing on the tech stack. Founders spend an hour interrogating React versus Vue and zero minutes on who reviews the code. Stack choice is close to irrelevant at this budget level. Review discipline, deployment frequency, and staffing stability explain almost all of the variance in outcomes.

Third, and least discussed: buyers penalize agencies for saying “I don’t know.” That incentive is backwards. The agency that says “I’d need to see your data volume before answering that” is showing you the exact behavior you want in month three. The one that answers everything instantly is showing you a sales reflex. Reward the pause.

Finally, most buyers never run reference calls with former clients. Current clients are mid-engagement and diplomatically compromised. Ask for two projects that ended in the last 18 months. Agencies that cannot produce them are telling you something without saying it.

Start With a Shortlist Worth Interviewing

A technical interview software agency process only works if the shortlist is real. Most founders waste the first two weeks filtering out agencies that should never have reached a call, which is a sourcing problem rather than an interviewing one.

GetProjects removes that first filter. Posting a project takes under two minutes and is free, agencies are verified across website, domain, reviews, and team details before any profile goes live, and AI matching compares 50+ data points to surface the agencies that have actually built something like yours. There is no bidding, no commission, and no platform cut on what you pay the agency you hire.

Post your scope, get a shortlist of verified agencies, and run the eight questions above on people worth interviewing: getprojects.ai

FAQ

How do you evaluate a software development agency without technical knowledge?

Evaluate the structure of answers rather than their technical accuracy. Ask open questions about change handling, testing, scaling, and staffing, then score each response on three visible markers: specificity, volunteered trade-offs, and named failure modes. A non-technical founder can grade all three reliably without ever reading code.

What questions should I ask a development agency before signing a contract?

Cover eight areas: requirement changes, named team members, scaling bottlenecks, testing approach for a specific feature, the weakest line in the estimate, the commit-to-production pipeline, ownership of code and infrastructure, and a project that went badly. The ownership question and the change-control question carry the most financial weight.

Should I hire a fractional CTO to review an agency?

For builds above roughly $60,000, yes. Six to ten hours of independent senior review typically costs $1,000 to $2,500 and pays for itself if it prevents a single month of misdirected work. Below that threshold, a structured self-run process is usually sufficient.

What are red flags in a development agency proposal?

Watch for estimates with no risk items flagged, no named team members, delayed code and repository transfer, fixed prices on undefined scope, and portfolio claims that cannot be traced to a specific architect. A proposal that contains no uncertainty was not written by the people who will do the work.

How long should agency vetting take?

Eight to twelve working days for a shortlist of three, including calls, reference checks, and one independent review. Compressing this below a week is where most bad selections originate, and a paid discovery phase before the full build adds another one to two weeks for a fraction of total project cost.

Can I run this process while comparing multiple agencies at once?

Yes, and you should. Running the same eight questions across three agencies in the same week makes differences in answer quality obvious in a way sequential calls never do. Keep one interviewer, one scorecard, and identical scope documents so the comparison holds up.