How to build function calling and JSON mode for LLMs
We recently introduced built-in functionality for function calling and structured output for LLM deployments with our TensorRT-LLM Engine Builder. In this webinar, we’ll dive into how we built it!
What you'll learn:
Understanding structured output and function calling: Learn how these features ensure schema-compliant model outputs and enable LLMs to select and execute specific tools.
Building JSON mode and tool use: Dive into the implementation details, including defining schemas and tools, building a state machine, and using logit biasing to force valid output.
Hands-on demo: See these features in action with live demos and a comparison with alternative methods.
Why attend?
Direct insights from the engineers who built it.
Learn how to deploy more reliable and efficient LLM-based systems.
Get your questions answered live.
Don’t miss this opportunity to pick the brains of the engineers behind these features and enhance your AI products—register now and secure your spot!
Host
Speakers
Create custom environments for deployments on Baseten
Test and deploy ML models reliably with production-ready custom environments, persistent endpoints, and seamless CI/CD.
Introducing canary deployments on Baseten
Our canary deployments feature lets you roll out new model deployments with minimal risk to your end-user experience.
Evaluating NVIDIA H200 Tensor Core GPUs for LLM inference
Are NVIDIA H200 GPUs cost-effective for model inference? We tested an 8xH200 cluster provided by Lambda to discover suitable inference workload profiles.
AI Tinkerers NYC October Meetup
Join us for an exciting evening of hands-on demos, talks, and networking at the AI Tinkerers October meetup!
NYC Tech Breakfast Club: ML Engineering Leaders
If you're an AI leader in NYC, join us for a hearty breakfast and good conversation!
Few-shot Happy Hour (SF Tech Week)
Come join us for a happy hour with the most cracked AI engineers, founders, and investors in the Bay.
Building Compound AI with Baseten Chains (SF Tech Week)
Join us for a presentation about building compound AI for next-generation AI products, followed by a happy hour!
