Experiment-Driven AI Development: Building the Plane While Flying 0 like May 1, 2025 Share this post Author(s): Kris Naleszkiewicz Originally published on Towards AI. Remember when everyone was scrambling to build their first RAG-AI assistant? How many of you heard things like, “Just connect GPT-4 to our knowledge base, add some vector embeddings, and boom — contextual AI!” If you did, then you probably remember a couple of weeks and several hundred prompt engineering iterations later, everyone asking the same questions: Why is our assistant hallucinating pricing for 2018? Where are these citations from — do they even exist? And why are we nowhere near where we thought we’d be? If you’ve built AI products in the past year, you’re nodding right now. Maybe even with a slight eye twitch. This isn’t just about RAG systems either. As we’ve expanded into multi-agent architectures and modular AI systems, the complexity has only multiplied. Each component can introduce emergent behaviors that interact in ways no feature specification anticipates. Meanwhile, stakeholders and clients grow increasingly confused: “What do you mean you are behind schedule? Do you need some more people? Do you need more engineers? More GPUs? A different model?” Building the plane while flying. Image by author with Dall-E. I’ve been in those meetings. You probably have too. And we’ve all tried to explain that while resources help, the fundamental challenge isn’t capacity — it’s methodology. Traditional software development assumes certainty, where AI development offers only probability. It expects predictability from systems designed to surprise. It demands fixed timelines in a space where discovery is the actual work. What we need isn’t just a different process — it’s a different mindset that is built around systematic experimentation rather than feature execution. After building AI products with dozens of organizations — from small organizations to Fortune 100 companies — I’ve seen that successful teams aren’t the ones with the most detailed plans or the most impressive tech stacks, but the ones who have mastered the art and science of experiment-driven development (EDD). Why Feature-Driven Models Struggle with GenAI Traditional software development thrives on certainty, characterized by defined features, reliable estimates, and predictable milestones. But generative AI introduces fundamental misalignments that challenge this approach. Discovery vs. Construction Feature-driven development assumes you’re building something with known parameters. But with AI, you’re simultaneously discovering what’s possible while building it. When you’re integrating a large language model, you can’t accurately predict its capabilities, limitations, or edge cases until you’ve tested them. The process is inherently exploratory. False Progress Metrics Tracking feature completion creates an illusion of progress that can mask critical issues. A “completed” RAG implementation might check all the boxes on your feature list while still hallucinating in production or failing on real user queries. Feature completion doesn’t necessarily correlate with actual system effectiveness. Counterproductive Incentives When teams are evaluated on feature delivery timelines, two problematic behaviors emerge: First, they avoid risky but potentially transformative experiments in favor of predictable increments. Second, they rush implementations before validating core assumptions, creating technical debt that compounds as the system grows. Traditional development methodologies aren’t wrong — they’re optimized for different conditions. They excel when the problem space is well-understood, requirements are stable, and technical risks are predictable. GenAI simply doesn’t offer those luxuries. The best builders recognize this reality. They don’t abandon planning or accountability, but they reframe their approach from “executing known solutions” to “discovering effective solutions through structured learning.” This is where Experiment-Driven Development becomes essential. Experiment-Driven Development (EDD) As AI systems become more generative and less predictable, we need a fundamental shift in how we approach development. Experiment-Driven Development (EDD) isn’t just a different process — it’s a different mindset that embraces uncertainty as a starting point rather than an obstacle. Feature v Experiment-driven Development. Image by Author. EDD transforms traditional development in four key dimensions. From Features to Hypotheses Instead of starting with requirements to build, we begin with assumptions to test. Each development cycle kicks off not with “let’s build this capability,” but with “let’s validate whether this approach solves our problem.” This reframes development work as scientific inquiry rather than production. From Roadmaps to Learning Paths Traditional roadmaps assume a linear journey toward a known destination or output. EDD replaces this with learning paths — structured sequences of experiments designed to reduce uncertainty progressively. You still have direction, but you recognize that the terrain may change as you explore it. From Delivery Milestones to Knowledge Milestones Success is measured not by the number of features shipped but by the uncertainties resolved. A successful sprint might deliver zero user-facing features but eliminate three existential risks that would have derailed the product later. This is genuine progress, even without visible output. From Implementation to Investigation Developers shift from being implementers of predetermined solutions to investigators of complex problems. Your value isn’t in how quickly you can code a spec — it’s in how effectively you can design experiments that reveal the path forward. EDD in Practice What makes EDD more than just “trying stuff until something works” is its systematic approach to experimentation. Targeted Uncertainty Reduction Every experiment aims to resolve a specific, high-impact uncertainty. You’re not exploring randomly; you’re methodically eliminating the riskiest unknowns first. Explicit Hypothesis Formation Experiments start with clear, testable hypotheses: “We believe fine-tuning on domain data will improve answer relevance by at least 15% compared to prompt engineering alone.” This forces precision in both thinking and testing. Minimal Viable Experiments The best experiments are the smallest ones that can validate or invalidate a hypothesis. This isn’t about cutting corners — it’s about optimizing the learning-to-effort ratio. Cumulative Learning Each experiment builds on the last, creating a compound interest effect in knowledge. Failed experiments aren’t wasted effort; they’re valuable data points that shape the next cycle. Evidence-Based Pivots When experiments invalidate assumptions, teams don’t push forward anyway — they pivot based on evidence. This isn’t flakiness; it’s responsible adaptation to reality. In EDD, learning is the product — until the actual product is ready. This approach doesn’t abandon planning or accountability; it simply recognizes that in AI, the most valuable plan is one that systematically reduces uncertainty rather than pretending it doesn’t exist. BCG’s survey of AI Experts regarding advanced AI capabilities, including reasoning, planning, memory management, and social understanding, shows the variations in maturity and limitations. Development teams need to recognize that the expanding frontier of AI capabilities necessitates consideration of how these capabilities will interact in complex AI systems, particularly as models evolve from specialized functions to interconnected agents with emergent behaviors. BCG AI Platforms Group Analysis. (BCG, 2025) Now that we’ve defined what Experiment-Driven Development is, the next question is obvious. How do you actually try EDD in your AI Products? Implementing EDD: From Theory to Practice Adopting Experiment-Driven Development changes to how teams operate, measure progress, and communicate value. The core of EDD is a structured experimental cycle that turns uncertainty into knowledge. Target Critical Uncertainties Begin by mapping your riskiest unknowns — the “dragons” that could sink your project: Technical viability (e.g., Can our RAG system handle multi-hop reasoning reliably?) User adoption (e.g., Will users trust model-generated outputs for high-stakes decisions?) Business impact (e.g., Will faster responses improve engagement?) Ethical concerns (e.g., Do our guardrails prevent harmful outputs across diverse user groups?) The best teams prioritize uncertainties by potential impact, not ease of investigation. Craft Precise Hypotheses Transform each uncertainty into a testable hypothesis with clear success criteria: We believe chunking documents by semantic sections will improve retrieval accuracy by 30% compared to fixed-size chunks. We predict that displaying confidence scores with generative responses will increase user trust by 25%. If you can’t express your hypothesis in measurable terms, you’re not ready to experiment. Design Minimal Experiments Create the smallest experiment that will validate or invalidate your hypothesis. Think days, not weeks: A/B test two chunking strategies against your benchmark dataset Run small-scale user testing with different UI treatments before committing to a full redesign Fine-tune a model on 10% of your domain data to assess the improvement trajectory Your goal is maximum learning per unit of effort, not polish. Execute with Discipline Run experiments with the same rigor you’d apply to production systems: Version everything — data, code, parameters, environment Document decisions and context alongside results Maintain consistent evaluation metrics across experiments Without this discipline, you’ll struggle to build on what you learn. Synthesize and Pivot After each experiment: Evaluate results against your hypothesis Capture unexpected learnings Update your model of how the system behaves Determine your next most valuable experiment An invalidated hypothesis isn’t a failure — it’s valuable intelligence that prevents wasted investment. The Foundation: MLOps as an Enabler EDD’s experimental approach requires technical infrastructure that makes iteration sustainable. Without the right tooling, even the best experimental mindset crumbles under practical constraints: Experiment Tracking Tools like MLflow, Weights & Biases, or custom implementations must capture: Model versions and parameters Dataset lineage and transformations Evaluation metrics and artifacts Experimental context and decisions Reproducibility Pipeline Automate the repetitive steps in your experimental workflow: Training and evaluation runs Data preprocessing and feature extraction Metrics calculation and reporting Fast Feedback Mechanisms Design systems that accelerate learning velocity: Automated evaluation against benchmark datasets Lightweight but consistent user testing protocols Real-time dashboards for experiment monitoring This MLOps foundation isn’t separate from EDD — it’s what makes systematic experimentation possible at scale, preventing the discipline from degrading into chaos as complexity grows. Reframing Progress: The Learning-Centered Update Perhaps the most challenging aspect of EDD is changing how teams communicate progress, especially to stakeholders accustomed to feature-based reporting: From: ✅ Completed: Query rewriting module✅ In progress: Citation generator (70% complete)✅ Blocked: Context compression feature To: 🔬 Validated: Hypothesis that vector database sharding improves query latency by 65% with minimal relevance impact🔬 Invalidated: Assumption that users prefer concise answers over comprehensive ones (user testing showed opposite)📉 Risks addressed: Hallucination on financial advice (reduced by 43% through retrieval enhancements)🧠 Key insight: User perception of model "expertise" tied more to explanation quality than answer brevity🎯 Next focus: Testing whether explicit uncertainty acknowledgment increases or decreases trust This learning-centered update focuses on knowledge gained, not just features shipped, by linking activities to risk reduction and insights that might otherwise remain hidden, thereby creating a clear link between past and future work. In organizations where stakeholders resist this shift, consider a transitional approach: map learning milestones to future capabilities they enable, helping traditional stakeholders see the connection between experimentation and eventual delivery. The path to effective AI isn’t through perfectly executed plans but through structured learning that builds on itself. By making experimentation disciplined, infrastructure-supported, and communicated, teams can transform AI development from a chaotic scramble into a systematic journey from uncertainty to confidence. Industry Leaders Validate the Experiment-Driven Approach Experiment-Driven Development is the status quo among organizations at the cutting edge of AI. The world’s most successful AI pioneers didn’t arrive at this approach by accident — they converged on it through hard-won experience. Why are Google DeepMind, Stanford HAI, and industry consultants all advocating for experimental approaches? Because they’ve learned there’s simply no other way to build effective AI systems at scale. Google DeepMind: Experimentation as Core Philosophy DeepMind’s landmark achievements — AlphaGo, AlphaFold, Gemini — weren’t built by executing predetermined feature lists. Their development process reveals three principles we should adopt: Continuous Learning Environments: DeepMind builds systems that improve through ongoing interaction rather than static datasets — acknowledging that model capabilities evolve through experience, not just initial training. Emergent Behavior Focus: Rather than attempting to predict every capability upfront, DeepMind designs experiments specifically to reveal unexpected model behaviors — then systematically studies these emergent properties. Dynamic Benchmarking: DeepMind creates new evaluation frameworks as capabilities evolve, recognizing that static metrics fail to capture the nuanced abilities of advanced systems. If the world’s leading AI research lab doesn’t pretend to fully predict how their models will behave, enterprise teams certainly shouldn’t either. Reid Hoffman’s Iterative Deployment In “Superagency,” Hoffman argues compellingly that AI development requires real-world learning, not theoretical perfection. He asserts that the best way to ensure AI benefits humanity is not by delaying its progress through overregulation or fear-driven pauses but by continuously testing, refining, and improving AI systems in real-world conditions. This “iterative deployment” philosophy means: Launching capabilities early with appropriate safeguards Gathering actual user feedback rather than speculating about reactions Course-correcting based on observed behavior, not just intentions Hoffman’s perspective validates that even at scale, the experimental mindset remains essential — progress happens through structured cycles of real-world learning. Stanford HAI: The Human-Centered Dimension Stanford’s Human-Centered AI Institute adds a crucial dimension to experimentation: ethical considerations must be integrated into the experimental process itself. Their research shows effective AI development requires: Testing for human impacts as rigorously as technical performance Validating assumptions about fairness and safety throughout development Using qualitative human oversight alongside quantitative metrics This reminds us that EDD must validate not just what technically works, but what creates beneficial human outcomes — expanding our experimental scope beyond pure functionality. The Convergence: Why All Roads Lead to EDD Despite their different objectives — scientific advancement, commercial success, ethical deployment — these diverse organizations have all landed on remarkably similar methodologies. This convergence isn’t coincidental. It reflects a fundamental truth: when dealing with systems as complex and unpredictable as generative AI, experimentation isn’t just a better approach — it’s the only viable approach. The question isn’t whether to adopt an experimental approach to AI development — it’s how quickly your organization can make the transition before competitors who already have leave you behind. EDD is a Mindset, Not Just a Method The beauty of Experiment-Driven Development isn’t in complex processes or sophisticated tools, but its acknowledgement of the fundamental truth that we don’t know what we don’t know. EDD isn’t about abandoning discipline — it’s about redirecting that discipline toward learning rather than executing. It replaces the false certainty of feature roadmaps with the honest confidence of validated knowledge. The choice isn’t whether to experiment — all AI development involves experimentation. The choice is whether to make that experimentation disciplined, systematic, and central to how you work. As AI capabilities continue to expand and AI systems become increasingly autonomous, the experimental mindset will only become more essential. The teams that thrive won’t be those with the most detailed roadmaps, but those with the most effective learning engines. Citations & Further Reading The Ladder: A Reliable Leaderboard for Machine Learning Competitions The organizer of a machine learning competition faces the problem of maintaining an accurate leaderboard that… arxiv.org AI has grown beyond human knowledge, says Google's DeepMind unit A new agentic approach called 'streams' will let AI models learn from the experience of the environment without human… www.zdnet.com Superagency in the workplace: Empowering people to unlock AI's full potential Almost all companies invest in AI, but just 1% believe they are at maturity. Our new report looks at how AI is being… www.mckinsey.com Laying the Tech Foundation for GenAI Success To harness the power of generative AI and keep up with the wave of constant innovation, CTOs need to rethink their… www.bcg.com How an AI-enabled software product development life cycle will fuel innovation AI has the potential to fundamentally transform the development of software products, increasing the pace of the… www.mckinsey.com Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor. Published via Towards AI Towards AI - Medium Share this post