Building a Model Evaluation Practice for Production AI

AWS New York’s official community presents a practical framework for evaluating language models against real production workloads. The session covers versioned golden datasets, hallucination and precision metrics, cost and latency, evaluation tools such as Ragas, PromptFoo, LangSmith, and Amazon Bedrock Model Evaluation, plus a side-by-side model demonstration. Security check-in begins at 4:30 PM, programming starts at 4:45 PM, and Q&A and networking run through 6:30 PM.