Experiment
2025
The un-conference for real-world lessons in building and deploying production-grade AI.
Experiment 2025 is a half-day, “science-fair” in San Francisco where ML engineers and AI builders trade unfiltered stories, from failed runs to production breakthroughs.
Join us to learn from the collective trials, errors, and insights behind deploying real-world AI applications.
What You'll Discover
Experiment, Explore, Learn, Evolve.
Skip the keynotes.
At Experiment 2025, you’ll trade real lessons with fellow AI practitioners and learn what broke, what worked, and what you’d do differently. Co-create the agenda, join breakout sessions, and walk away with insights you won’t find in docs, blogs, or polished talks.
Agenda
Welcome
Registration & Breakfast
9:00 - 9:30Opening Circle & Agenda Creation
With Salma Mayorquin and Rachel Chalmers
9:30 - 10:00Breakouts
30 minute breakout sessions decided and presented by attendees
10:00 - 12:00Lunch
Generously sponsored by Generationship
12:00 - 12:30Closing circle
Attendees report out from sessions
12:30 - 1:00
From Idea --> Deployment
Join engineers, researchers, and builders exchanging practical insights across six tracks that follow the scientific method.
Knowledge Acquisition / Discovery
Systematically exploring and validating data sources to surface patterns, gaps, and relationships that will ground later hypotheses.
Hypothesis Generation
Turning exploratory insights and domain knowledge into clear, falsifiable statements about how a model or variable is expected to behave.
Experiment Design & Justification
Crafting controlled tests that isolate variables, define success metrics, and allocate compute so each run provides statistically sound evidence.
Feedback Gathering & Integration
Collecting human or automated signals from real-world use, monitoring drift, and feeding the data back into continuous training loops.
Post-mortems & Learnings
Conducting blameless reviews of outcomes or incidents to trace root causes, capture lessons, and update processes and documentation.
Scaling
Extending the entire scientific loop—data, experiments, feedback, governance—to larger volumes, model fleets, and teams through orchestration and cost-aware automation.