In partnership with
Everywhere you look, the job title “AI Engineer” is popping up.
But if you ask 10 people what it actually means, you’ll get 10 different answers. Some say it’s just a rebranded ML engineer. Others think it’s data scientists dabbling in prompts. Increasingly though, it’s something else entirely:
Software engineers building real-world products powered by large language models (LLMs).
I’ve been tracking how companies are adapting to this shift, and what AI engineering really looks like in practice. Across industries and company sizes, some clear patterns (and pitfalls) are emerging.
In this edition, you’ll see:
🚀 7 real companies and the AI features they shipped (from incident bots to legal copilots)
🛠️ The tech stacks they used and why Postgres + Bedrock is becoming the quiet default
⚡ How software engineers transition into AI engineers (and what leaders can learn)
By the end, you’ll know what AI engineering looks like on the ground — and how to apply those lessons.
🚀 What Companies Are Actually Building
The most successful AI features today aren’t sci-fi “agents” replacing employees. They’re sidekicks that embed into workflows and make teams faster, smarter, or more consistent.
Incident.io (Incident Management)
Built two powerful features:
→ An AI note-taker that transcribes calls, summarizes key points, and captures decisions in real time.
→ An “incident investigator” that digs through logs, old incidents, and monitoring data to suggest root causes.
Sentry (Application Monitoring)
→ Autofix: traces bugs to their root cause, drafts a fix, and creates a GitHub PR with unit tests.
→ Smarter issue grouping that uses embeddings + approximate neighbor search to cut down noisy alerts.
Autofix identifies the root cause of an issue with a useless Stack Trace
Wordsmith (Legal AI)
→ AI contract review that highlights risks and generates marked-up Word docs.
→ A workspace for analyzing communications and drafting legal responses.
AI contract review: AI tool detects, highlights and summarizes potential contractual issues
Augment Code (Dev Tools)
→ AI coding assistant fine-tuned for large, complex codebases.
→ IDE + Slack extensions, powered by fine-tuned LLMs specialized for coding tasks.Elsevier (Publishing)
→ Built an enterprise-wide RAG platform so multiple teams could reuse the same embeddings + LLM API.
→ Products on top include: a tutoring system for medical students and clinical decision support for doctors.Simply Business (Insurance)
→ AI chatbot for customers — with strict guardrails. The bot only answers from an “approved knowledge base.” If it doesn’t know, it escalates to a human.DSI (HR SaaS)
→ Summarization feature for employee engagement surveys, turning thousands of free-text comments into clear themes and categories managers can act on.
Summary of teamwork-related comments: The feature builds a word cloud of common topics from thousands of comments, and provides an overview of any term
💡 Pattern to notice: Almost every first AI feature starts with summarization, grouping, or automating repetitive triage work. These are high-impact, low-risk — and great training grounds for teams.
⚡ How Engineers Transition to AI Engineering
The most fascinating insight: these systems weren’t built by ML experts. They were built by software engineers who were willing to learn fast, experiment, and adapt to a new style of problem-solving.
Here’s what that transition actually looks like in practice:
1. From Outsourcing to Owning
At DSI (HR SaaS), leadership initially hired an external AI vendor to build a survey comment summarizer. The proposal was heavyweight: SageMaker pipelines, multiple databases, and 6–9 months of dev time — at a cost higher than DSI’s entire infra budget.
Instead of accepting it, one veteran engineer (with zero prior AI experience) decided to self-teach:
Month 1 → Reading docs, prototyping with AWS Bedrock and Cohere embeddings
Month 2 → Integrated Postgres for vector queries + React for UI
Result → A production-ready feature in 2 months at a fraction of the cost
💡 Pattern: Agencies often over-engineer. Empowering engineers to learn by doing not only saves money but builds long-term capability.
2. Learning to Live with Non-Determinism
At Wordsmith (Legal AI), new hires struggled at first. Traditional engineering is deterministic: same input → same output. With LLMs, the same prompt can yield slightly different answers.
To succeed, engineers had to:
Shift mindset from “exact outputs” → to “acceptable ranges of quality”
Build evaluation pipelines (human-in-the-loop reviews, accuracy scoring, customer feedback loops)
Develop domain expertise — because in law, “good enough” isn’t good enough
💡 Pattern: AI engineers must be comfortable with ambiguity — and design systems that can measure fuzzy quality.
A word from our sponsor
The Gold standard for AI news
AI keeps coming up at work, but you still don't get it?
That's exactly why 1M+ professionals working at Google, Meta, and OpenAI read Superhuman AI daily.
Here's what you get:
Daily AI news that matters for your career - Filtered from 1000s of sources so you know what affects your industry.
Step-by-step tutorials you can use immediately - Real prompts and workflows that solve actual business problems.
New AI tools tested and reviewed - We try everything to deliver tools that drive real results.
All in just 3 minutes a day
3. Managers Going Hands-On Again
At Elsevier, an engineering manager decided to step back into an IC role to work directly on the company’s RAG platform. Why? Because building with LLMs rekindled the joy of coding.
His reasoning:
Senior engineers have the mental models to break problems into solvable chunks
LLMs act as force multipliers for those with strong fundamentals
But for junior devs, heavy reliance on AI tools can short-circuit skill development
💡 Pattern: Experienced engineers thrive in AI engineering because they know when to trust the tools, and when to fall back on fundamentals.
4. The “Side Project to Expert” Path
At multiple companies, engineers started with small, low-stakes prototypes:
A summarizer for internal Slack threads
A chatbot for FAQs that could fall back to humans
A personal script for code refactoring
These side projects created internal momentum. Suddenly, the engineer who hacked together a working prototype became the “AI expert” inside the org.
💡 Pattern: In AI engineering, expertise is self-made. The fastest learners become the go-to people, regardless of their prior background.
5. The New Skill Stack for AI Engineers
From these transitions, we can see the emerging skill set for AI engineers:
Prompt literacy → understanding how to structure inputs for consistent outputs
Evaluation design → creating metrics + workflows to judge AI quality
Stack pragmatism → knowing when Postgres is enough vs. when to add Pinecone or custom infra
Domain fluency → legal, healthcare, finance… understanding the rules matters more than just calling an API
User empathy → educating users on what AI can/can’t do is part of the job
💡 Lesson: The AI engineer isn’t a different species. They’re software engineers who adapt to non-determinism, fast prototyping, and constant iteration.
🛠️ The Tech Stacks Emerging as Standards
One of the biggest myths about AI products is that there’s a “standard stack.” In reality, every company makes trade-offs based on cost, scale, compliance, and talent.
Looking across 7 companies, here’s what’s emerging:
1. Foundation: Where Models Run
AWS Bedrock → Most popular for startups and mid-size companies. Gives access to Anthropic Claude, Cohere, and other models with enterprise-grade privacy (no training on your data).
Azure OpenAI → Preferred in regulated industries (e.g. legal, healthcare) because you can pin workloads to specific regions, meeting compliance requirements.
Google Vertex/Gemini → Less used, but valuable where companies are already deep in GCP.
Custom GPU Clusters (NVIDIA A100/H100) → At scale, dev tools like Augment Code fine-tune and serve models on their own infra. This cuts costs and boosts speed but requires deep ML expertise.
💡 Pattern: Startups lean on managed APIs (Bedrock/Azure). Once usage grows, the cost/latency trade-offs force a move closer to the hardware.
2. Data & Embeddings: Organizing Knowledge
Postgres + pgvector → The quiet winner. Most companies (incident.io, Sentry, DSI) extend Postgres with vector search for embeddings. It’s simple, reliable, and fits into existing stacks.
Pinecone → Used by Wordsmith for legal AI. Great for dedicated vector DB performance, but adds vendor lock-in and costs.
OpenSearch / ElasticSearch → Elsevier leaned on OpenSearch for large-scale vector search tied into their content pipelines.
💡 Pattern: If you’re prototyping → Postgres is plenty. If you’re scaling across terabytes of docs → Pinecone or OpenSearch shines.
3. Integration & Orchestration
LangChain → Common for prototypes, but often replaced when complexity grows. Sentry built its own framework because LangChain abstractions didn’t fit their architecture.
LlamaIndex → Used at Wordsmith to connect contracts, docs, and comms streams into LLMs.
Custom In-House Tooling → incident.io and Sentry both chose to roll their own “agent orchestration” to better align with their existing systems and developer workflows.
💡 Pattern: Off-the-shelf frameworks are great to start. Mature teams build their own orchestration for flexibility + maintainability.
4. Evaluation, Observability & Guardrails
This is where most prototypes fail — and where production AI gets serious.
Langfuse → An open-source platform for debugging and monitoring LLM outputs. Proposed by vendors at DSI.
Custom Evals → Teams like Wordsmith run human-in-the-loop evals for contract reviews to prevent legal mistakes.
Guardrails (AWS Bedrock / Anthropic Sonnet) → Used by Simply Business to ensure their insurance chatbot only returns “approved” responses.
💡 Pattern: Eval + monitoring + guardrails are as critical as the model itself. Without them, trust breaks down fast.
5. Languages, Frameworks & Infrastructure
Go + React → incident.io built their AI features inside their monolith using Go on backend and React/TypeScript on frontend.
Python + PyTorch → Sentry and Augment Code rely on Python for training/inference workflows. PyTorch remains the standard for custom ML work.
FastAPI → Elsevier exposed their RAG platform via FastAPI, making it easy for multiple teams to consume.
Ruby on Rails → Simply Business stuck with Rails for their chatbot integration — showing you don’t need exotic frameworks to ship AI features.
💡 Pattern: AI engineering is less about “new languages” and more about plugging LLMs into your existing stack.
🧠 Key Takeaways
If you’re exploring AI in your business, here’s the distilled playbook:
Start with augmentation. Sidekick features (summaries, grouping, triage) deliver value fast and build muscle.
Prototype, then harden. A demo is easy. Production means evals, guardrails, and cost management.
Invest in internal skills. Agencies can help, but long-term leverage comes from your team learning by doing.
Right-size your stack. The simplest setup that works is usually the best.
Educate your users. Adoption fails if people don’t trust the system or know its limits.
Expect iteration. Nobody nails it first try. Success comes from fast cycles and feedback.
🌎 The Bigger Picture
AI engineering isn’t just a trendy title. It’s the beginning of a new discipline in software development — one that blends classic engineering principles with the realities of working with non-deterministic systems.
The companies winning today aren’t the ones chasing hype. They’re the ones:
Embedding AI into real workflows (not shiny demos)
Training their engineers to adapt and iterate fast
Building lean, right-sized stacks instead of overengineering
And keeping humans in the loop where trust and accuracy matter most
The lesson for builders and leaders is simple:
Start small, stay practical, and learn by doing.
Because the future of software isn’t “AI vs. engineers.”
It’s engineers who know how to wield AI — and they’re already shaping the next generation of products.