PONDER US

JB: Hi. I read an interesting article in the International Business Times by Vinay Patel titled, “Builder.ai Collapses: $1.5bn ‘AI’ Startup Exposed as ‘Actually Indians’ Pretending to Be Bots.” Apparently, the AI goldrush involves quite a bit of “Fool’s Gold.” The article made me buy, “The AI Con” a book by Emily M. Bender & Alex Hanna. While I’m just a few chapters in, and appreciate their skepticism, especially in light of the Builder.AI story, I still feel like we’re experiencing a big tech transformation because I use AI daily. How would you recommend we separate the hype from the reality as we move forward, especially when investing in AI companies?

Gemini: That’s an excellent question, and the Builder.ai story, as well as the book “The AI Con,” highlight a crucial challenge in the current AI landscape: distinguishing genuine innovation from marketing “AI washing.” It’s clear that while AI is indeed transformative, it’s also ripe for exaggeration and even outright deception.

Here’s how to separate the hype from the reality when moving forward, especially when investing in AI companies:

1. Go Beyond the Buzzwords: Understand the “How” and “What”

• Look for concrete applications, not just “AI-powered”: Many companies will simply slap “AI” onto their product description. Instead, ask how AI is being used. Is it genuinely solving a complex problem that couldn’t be solved efficiently by traditional software or human labor?
• Differentiate true AI from statistical models or automation: As the article you read suggests, some “AI” might just be sophisticated statistical analysis or basic automation. True AI, particularly machine learning, should involve:
  • Learning from data: The system should improve its performance over time as it’s exposed to more data.
  • Adaptability: It should be able to adjust to new or changing conditions without constant manual reprogramming.
  • Multiple inputs: Real AI models often leverage various data types and external variables (e.g., market trends, economic indicators) to make predictions or decisions, unlike simpler statistical models that might rely on a single data series.
• Demand transparency and explainability: Can the company explain how their AI models arrive at their conclusions? While some advanced AI models are inherently “black boxes,” a reputable company should be able to provide insights into the drivers influencing their predictions and the methodologies used.

2. Deep Dive into the Technology and Data

• Examine the data strategy: AI is only as good as the data it’s trained on.
  • Data quantity and quality: Does the company have access to large, high-quality, and relevant datasets? How do they ensure data accuracy and cleanliness?
  • Data collection and ethical practices: How is the data collected? Are there any privacy or ethical concerns?
  • Proprietary vs. public data: Are they using unique, specialized datasets that provide a competitive advantage, or are they simply leveraging publicly available information?
• Assess the algorithms and models:
  • Complexity and novelty: Is there genuine algorithmic innovation, or are they just using off-the-shelf models wrapped in a new interface?
  • Scalability: Can their AI solutions scale to handle larger datasets and more complex problems as the business grows?
  • Reproducibility: Can their AI models be reliably reproduced and maintained?
• Beware of “thin wrappers”: Some “AI startups” are essentially just using existing large language models (LLMs) or other AI services from major providers (like OpenAI or Google) and putting a thin user interface on top. While this can still be a viable business, it’s crucial to understand if they have a unique value proposition beyond simply integrating existing AI.

3. Scrutinize the Team and Business Model

• Expertise in AI/ML: Does the team have actual AI/machine learning scientists and engineers with relevant credentials and experience? Be wary if the leadership team has no deep technical expertise in AI.
• Realistic value proposition: Does their AI genuinely solve a problem that customers are willing to pay for? Is there a clear path to monetization?
• Sustainable competitive advantage: What gives them a long-term edge beyond just “having AI”? Is it proprietary data, unique algorithms, deep domain expertise, or something else that’s difficult to replicate?
• Customer traction and use cases: Are there real-world customers using their product and seeing tangible benefits? Look for strong case studies and testimonials that demonstrate genuine impact.
• Scalability of operations, not just technology: Beyond the tech, can the company scale its operations, sales, and support to meet demand? Lack of robust support can be a major red flag.

4. Be Wary of Red Flags

• Exaggerated claims and promises: Be highly skeptical of claims of guaranteed, high returns with little or no risk, or promises that AI will solve all problems instantly.
• Lack of regulatory compliance awareness: AI development and deployment are increasingly subject to regulations regarding data privacy, bias, and ethics. Ensure the company is addressing these concerns.
• High-pressure sales tactics or vague answers: If a company is pushy or unwilling to provide clear, detailed answers about their technology and how it works, proceed with extreme caution.
• “AI-washing” without details: If a vendor claims “AI-powered” without offering any specifics about what their product can really do, that’s a significant red flag.
• Sole reliance on AI-generated information for investment decisions: Be cautious about using AI-generated information (e.g., from a chatbot) to make investment decisions, as it can be inaccurate, incomplete, or misleading. Always verify information from multiple, reliable sources.

The “AI gold rush” certainly has its share of “fool’s gold.” By adopting a rigorous, skeptical, and informed approach to due diligence, focusing on concrete applications, technical depth, and a strong, ethical team, you can better navigate the hype and identify truly transformative AI investment opportunities. The Bender & Hanna book will undoubtedly provide valuable frameworks for this critical discernment.

JB: Good advice. What percentage of AI companies would you guess are “mostly BS,” versus the “real deal?”

Gemini: That’s the million-dollar (or perhaps billion-dollar, given AI valuations) question! It’s incredibly difficult to put a precise number on it, as the line between “mostly BS” and “real deal” can be blurry, and companies evolve. However, based on the current landscape, industry reports, and expert opinions, I can give you a qualitative assessment and some common estimates.

It’s likely that a significant majority of “AI companies” today fall into the “mostly BS” or “overhyped” category.

Here’s why and what kind of percentages you might hear:

• The “90% failure rate” for startups applies heavily to AI:¹ Various reports on startup failure rates, including those specifically for AI startups, often cite figures around 85% to 90%. While not all failures are due to “BS,” a large portion are due to a lack of genuine market need, inability to monetize, or overpromising on capabilities that don’t materialize. This often ties into the “mostly BS” aspect – they either didn’t have a truly innovative AI solution or couldn’t execute on it.
  • Some reports even suggest that around 90% of AI startups fail, with many not surviving past their first few years.² This is a stark number.
• “AI Washing” is rampant: Many companies that claim to use AI are simply integrating existing, off-the-shelf AI tools (like a large language model API) into their product, or even just using advanced statistical analysis and calling it AI.³ They might not have proprietary AI technology or a deep understanding of its nuances. This isn’t necessarily “BS” in the sense of malicious deception, but it’s certainly “hype” that overstates their unique AI capabilities.
• Defining “Real Deal” is tricky:
  • Is a company truly innovative if it’s building on top of foundational models from OpenAI or Google, but creating a genuinely valuable application? Many would say yes, if the application itself is novel and solves a real problem.
  • Is a company “real” if its AI is still in early development, but showing immense promise? Yes, but it carries higher investment risk.

My Educated Guess (Qualitative):

If we define “real deal” as companies that:

1. Have genuinely proprietary AI technology or significant advancements in AI application.
2. Have a clear, demonstrable problem that AI is uniquely solving.
3. Have a strong team with deep AI/ML expertise.
4. Show clear customer adoption and a viable path to sustainable revenue.

Then, I’d say the percentage of “real deal” companies is probably much smaller than many assume.

• “Mostly BS” or “Overhyped” (at least 70-80%): This category would include companies that:
  • Are simply reselling or repackaging existing AI services with minimal unique value.
  • Have an “AI” component that is actually just advanced automation or statistics.
  • Make grandiose claims that their technology cannot yet support.
  • Lack the deep technical expertise or data strategy to truly deliver on AI’s promise.
  • Are chasing trends without a clear problem-solution fit.
• “Genuine Potential” or “Real Deal” (20-30%): These are the companies doing truly innovative work, even if they are early stage. They might be:
  • Developing novel AI algorithms or models.
  • Applying AI to highly specific, underserved problems with significant impact.
  • Building unique datasets or data pipelines that are crucial for AI performance.
  • Demonstrating measurable, tangible benefits from their AI solutions.

Important Nuances:

• Industry and application matters: Some industries are more mature in AI adoption (e.g., certain aspects of finance, e-commerce, cybersecurity), while others are just beginning. The “hype” factor might be higher in newer application areas.
• Stage of company: Early-stage startups often rely more on vision and promise, while established companies have more data to back up their claims.
• The “AI Bubble” concern: Many experts, like the authors of “The AI Con,” are concerned that we are in an AI bubble, similar to the dot-com bubble.⁴ In such a scenario, many companies get inflated valuations based on hype, leading to a higher percentage of “mostly BS” or eventually failed ventures.

Ultimately, your skepticism is well-founded. The Builder.ai story is just one prominent example, and there are countless smaller cases of companies overstating their AI capabilities.⁵ Diligent research and focusing on the substance, as we discussed, are paramount.

JB: Thanks. I appreciate your sobering assessment.

Gemini: You’re most welcome. I think a “sobering assessment” is precisely what’s needed in the current AI climate. It’s easy to get swept up in the excitement and promises, but the reality, as you’ve astutely noted, can be quite different.

My goal is to provide a realistic perspective, helping you to cut through the noise and identify where genuine value and innovation lie. The principles of critical thinking and due diligence that apply to any investment or technological assessment are even more crucial when it comes to a field as prone to hype as AI.

Keep asking those insightful questions, and good luck with your continued exploration of “The AI Con” and your future investments.