From Pilot to Production: A Year of Lessons from an AI Documentation Agent
- Presentation
- AI & Automation in Technical Communication
- 24. September
- 02:05 PM (CEST) - 02:50 PM (CEST)
- Scandium
- finished
-
Anna Goncharova
- inDrive
Contents
A year ago, an AI agent that auto-generates technical documentation from code changes sounded like a promising experiment. Today, it runs across more than a dozen repositories, and the gap between "it works in the pilot" and "it works in production" turned out to be larger than expected.
This talk is an honest look back at what happened in between. Drawing on lessons from rolling out an AI documentation agent at scale, it covers the problems that only show up once real teams depend on the output: hallucinations that survived the pilot, accidental regeneration of documents that were deleted on purpose, noisy commit histories, prompt drift, and the quiet erosion of trust when AI-generated docs get something wrong.
The talk shares the patterns that helped — anti-hallucination guardrails, deletion protection, per-commit analysis, semantic filenames, and model fallback strategies — and the ones that did not. Attendees will leave with a realistic picture of what production-grade AI documentation actually requires.
Takeaways
Attendees will leave with a realistic view of what AI-generated documentation requires in production, plus concrete patterns for anti-hallucination, deletion protection, and sustaining trust over time.