Universal 3.5 Pro Demo: Smarter Speech-to-Text with Contextual Awareness

The video introduces AssemblyAI's Universal 3.5 Pro model, emphasizing its enhanced contextual awareness and prompting capabilities for improved speech-to-text accuracy. The presenter walks through two main features: contextual prompting and conversation context. Contextual prompting works by providing the model with information about the audio's domain, scenario, or detailed descriptions, which helps it handle uncommon names and terms. Key term prompting is also discussed, where specifying important terms boosts their recognition accuracy. A demonstration in the playground shows how adding a key term like 'Klebanoff' ensures accurate transcription. The video then explains the 'conversation context' feature, which passes previous transcriptions and agent replies as context to the model. This allows the model to understand the ongoing conversation and anticipate user intent, providing more relevant responses and improving accuracy, particularly in voice agent use cases. Statistical improvements on voice agent datasets are briefly shown. Finally, a detailed demo in the 'Context Carryover Playground' illustrates how providing agent context dynamically, even with poor audio, leads to better transcriptions and more intelligent agent interactions.