Why You Should Build Systems That Scale with Compute

If you're exploring AI for your business or just getting started in this space, there's one powerful idea that can save you time and help you build future-proof solutions:

Let’s unpack what this means—and how you can use it.

The Core Insight: Build for Flexibility, Not Just Function

In the early days of language models, developers had to write tons of custom code around weak AI systems just to get them to function properly. As models improved (from GPT-2 to today’s Gemini, GPT-4, Claude, etc.), much of that hand-written scaffolding became unnecessary.

That shift is key. When your system is tightly coupled to brittle rules, it breaks when the world changes. But when you build around AI systems that scale with compute, your software improves naturally as the underlying model gets smarter.

This is what happened at an insurance company.

Case Study: How an Insurance company automates CSV data ingestion

When new users join, they often import transaction data from other platforms. But each provider’s CSV files have different column names, formats, and schema.

Initial approach: Manually write code for the top 50 providers.

• Works, but time-consuming.

• Breaks if providers update formats.

Improved approach: Use AI to classify columns (merchant, amount, date, etc.)

• Less manual work.

• Some flexibility, but still relies heavily on classical scripting.

Best approach: Feed the CSV to an AI agent with code interpreter access.

• The agent writes its own code (e.g. Python/pandas) to transform data.

• Includes built-in verification (unit tests) to check accuracy.

• Run it 50 times in parallel for robustness.

The result? It works across unpredictable formats, adapts as models improve, and saves engineer time.

Visualizing 3 AI System Architectures

Think of your AI integrations like pipelines:

1. Traditional Rigid System: All logic is hardcoded. No AI involved.

2. Hybrid System: Classic code calls an LLM for tasks like classification or summarization.

3. AI-First System: The LLM handles the full task—generating and running its own logic, deciding when to use external tools.

Most of today’s apps sit somewhere between (1) and (2). But the future is trending toward (3), where LLMs act as autonomous agents coordinating tools and systems.

Big Picture: LLMs as Your Backend

Here’s a wild but real example: an experimental Gmail client where the entire backend is an LLM.

• It receives the user's Gmail token.

• It renders the UI via markdown.

• It fetches emails and handles logic like marking read/unread.

The frontend is mostly a thin shell. The LLM makes all the decisions and drives the experience.

Is it fast? Not yet. Does it work reliably? Kind of. Is it the future? Possibly.

Actionable Advice for Beginners

You don’t need to be a deep AI engineer to use this thinking:

✅ Choose tools that adapt — Use LLMs that evolve over time (like Gemini or GPT).

✅ Automate edge cases — Let AI handle unstructured input (like emails, PDFs, or messy data).

✅ Think modular — Design systems that can swap in smarter agents over time without rewriting everything.

✅ Use AI for glue code — Let AI write scripts, extract data, format results, or validate tasks.

✅ Experiment and iterate — Even if it’s not perfect today, design for tomorrow’s AI.

Final Thought: Build Systems That Get Better Without You

Exponential growth in AI model capabilities means that even without touching your code, systems built around LLMs will improve in accuracy, speed, and reasoning.

In a world where compute is cheap and model quality improves monthly, your biggest asset is a system that rides that curve—getting smarter every day, even while you sleep.