Ex-NASA dev reveals his Agentic Engineering Workflow

The lecture begins by highlighting the limitations of current AI agents, stating they can solve problems but struggle with creating maintainable code without human oversight. The speaker introduces "program design" as a crucial, often skipped, step before letting agents run. This involves defining clear, measurable outputs and understanding the decisions the agent might make.

The video then delves into the concept of a "software factory" and how it has evolved, particularly with the advent of AI. It contrasts the traditional human-led software factory with an AI-driven one, emphasizing the need for clear feedback loops and automated processes. The speaker uses diagrams to illustrate these workflows and the challenges of integrating AI agents into existing development cycles.

A significant portion of the lecture is dedicated to explaining "context engineering," which involves structuring input for LLMs to improve their performance and reduce costs. The speaker shows how different formats, like JSON and structured prompts, can be used to feed relevant information to the AI. He also touches upon observability and the importance of tracking AI usage, errors, and latency.

The latter half of the lecture addresses the challenges of benchmarking AI agents, arguing that traditional benchmarks are insufficient. The speaker proposes a more nuanced approach to evaluation, focusing on real-world performance and testability. He also discusses the importance of program design in AI development and how to build AI systems that are both powerful and maintainable, drawing parallels to the development of software factories in the past. The lecture concludes with a call to action for developers to focus on building robust AI systems by understanding the underlying principles and leveraging human intuition alongside AI capabilities.