LLMs Are Catnip for Founders. Stay in Your Lane.

LLMs are like catnip for founders: addictive, intoxicating, and a one-way ticket to Delusionland. I will never stop encouraging new technologies, nor will I discourage anybody from experimenting. But if things are getting serious, maybe instead of that $200 ChatGPT Pro subscription, hire a consultant. MIT NANDA studied 300 enterprise AI deployments in 2025 and found 95% returned no measurable impact. The model was rarely the problem. The human judgment around it was.

3 min read4 sectionsMay 1, 2026

Why founders fall in love with LLMs

You ask the LLM a question. It gives an articulate answer. You feel smarter. You ask another question. It gives another articulate answer. You feel even smarter.

Forty minutes later, you have a strategic plan, a tech architecture, a product roadmap, and a marketing positioning all written by an entity that has never met your customers, never seen your books, and has no stake in your outcomes.

The LLM is not lying to you. It is doing what it was trained to do: produce text that pattern-matches what you asked for. Modern models are trained on human feedback (RLHF), and that training optimizes for answers people rate highly, not answers that are true. Researchers call the result sycophancy, and a 2025 study in Science found it across all 17 models they tested, where the AI agreed with users even when the users were plainly wrong. The problem is that you asked for confidence and got confidence. You did not ask for hard truths.

Where this becomes dangerous

When founders use LLMs to make decisions instead of generate options.

Generating options is the right use. Brainstorm 20 names. Outline a structure. Draft an email. Speed up the boring parts. The LLM is great at this.

Making decisions is the wrong use. Should we pivot. Should we hire. Should we raise. Should we fire. Should we bet on this market. The LLM has no stake in the outcome and no context about your specific situation. Letting it decide is letting an autocomplete drive your business.

The failure mode is subtle. You start using the LLM for small decisions. Each one feels harmless. After three months, your roadmap is shaped by what the LLM said about market trends in conversations that no one else was part of. This is exactly what MIT NANDA documented in its 2025 State of AI in Business report: across 52 executive interviews and 300 deployments, 95% of generative AI pilots delivered no measurable return, and the cause was almost never the model. It was that nobody had wired the output into a real workflow or held it to a real outcome.

When to hire a human instead

Three signals tell you it is time to hire a consultant or specialist instead of asking the LLM.

First: the decision affects revenue or runway. Anything that materially changes the financial trajectory of the business deserves a human who has skin in the game and accountability for the recommendation.

Second: the decision is irreversible. Hiring, firing, signing long-term contracts, choosing a tech stack you will live with for 5 years. The LLM produces a confident answer that can still be wrong, and the rate climbs the harder the question gets. When Vectara moved its hallucination benchmark to a tougher set of articles in November 2025, the top models, including GPT-5, Claude Sonnet 4.5, and Gemini 3 Pro, all jumped past 10% on grounded summarization, a task they used to ace. Reversibility matters, because the answer reads exactly the same whether it is right or wrong.

Third: the decision requires context the LLM does not have. Your customer relationships. Your team dynamics. Your competitive landscape. Your personal capacity. The LLM has none of this and will produce a generic answer that fits anyone, which means it fits no one.

The Pineido perspective

We use LLMs daily at Pineido. They are productivity multipliers. They write tests, draft documentation, prototype code, generate analysis. We would not function without them.

We also do not let them make decisions. Every recommendation we give a client is reviewed by a human who has read the situation, talked to the team, and has a stake in the outcome being right. The LLM is a research assistant, not a partner. This is the same line the MIT NANDA report drew between the 5% of AI projects that worked and the 95% that did not: the winners picked one real pain point and wired it into an actual workflow, instead of letting a chatbot freelance the strategy.

If you are a service business owner getting strategic advice from ChatGPT, stop. Buy a session with a consultant or specialist instead. ChatGPT Pro now runs $200 a month at the top tier. Twenty months of that adds up to one $4K consultant. The consultant says the hard truths the LLM is too polite to say, because the LLM is literally trained not to say them. Remember April 2025, when OpenAI had to roll back GPT-4o for praising bad ideas a little too enthusiastically.

Do not become a passenger princess in your own car. Stay in your lane. Stay safe. Keep your eyes on the road and your hands up on the wheel.

Get operations insights weekly

Join service-business operators who get one actionable insight on ops, automation, and growth every week.

Checking that you're human. You can send in a moment.

No spam. Unsubscribe anytime.

You’re subscribed! Look for our next issue in your inbox.

Ready to ship?

Tell us where you are stuck. We will review it and send a focused recommendation within one business day.

Frequently asked questions

Are you saying LLMs are bad?

No. LLMs are excellent at generating options, writing first drafts, summarizing documents, and accelerating boring work. They are bad at deciding what matters in a specific business context they cannot see, and they are trained to agree with you rather than correct you, which is why a 2025 Science study found sycophancy across every model it tested.

How do I know if I am using LLMs wrong?

Three signs. First, you use them for strategic decisions instead of just brainstorming. Second, you implement what they suggest without external review. Third, your roadmap has shifted in ways your team cannot trace to a real conversation. MIT NANDA found 95% of enterprise AI pilots fail for exactly this reason: output that nobody grounded in a real workflow.

When should I hire a consultant instead?

When the decision affects revenue or runway, when it is irreversible, or when it requires context only humans can have. The model reads as confident either way, and on harder questions even GPT-5 and Claude Sonnet 4.5 hallucinate over 10% of the time on grounded tasks. For everything else, LLMs are fine as research assistants.

How does Pineido use LLMs?

Every day, for productivity. Never to make recommendations to clients. Every recommendation is reviewed by a human with stake in the outcome, the same discipline that separated the 5% of AI projects that worked from the 95% that did not.

Written by

Kostadin Stamboliev

Founder and Imaginator

Founder and Imaginator at Pineido. Practitioner across IT consulting, app and web development, technical SEO, and AI / automation. Ships production code in 30-day sprints with fixed scope and clear ownership.

LinkedIn