Basten @ NVIDIA GTC
Join us at NVIDIA GTC March 17-21 in San Jose, CA and experience Baseten’s AI inference platform, offering industry-leading performance, security, and scale for organizations building AI products. With features like on-chip model optimization, autoscaling, and a developer-friendly interface, Baseten enables rapid, secure, and cost-effective deployment, ensuring you can maintain competitive user experiences while adhering to strict compliance requirements.
To learn more:
Visit us at booth #1233
Schedule a Demo for your "Artificially Intelligent” Sweatshirt
Grab Coffee with one of our Executives and learn more about what Baseten can do for you
Watch for more updates on how to engage with us!
We look forward to seeing you in March!
Host
Baseten
Speakers
NYC Tech Breakfast Club: ML Engineering Leaders
Join us for the NYC Tech Breakfast Club for ML Engineers
Driving model performance optimization: 2024 highlights
Baseten's model performance team works to optimize customer models for latency, throughput, quality, cost, features, and developer efficiency.
New observability features: activity logging, LLM metrics, and metrics dashboard customization
We added three new observability features for improved monitoring and debugging: an activity log, LLM metrics, and customizable metrics dashboards.
How we built production-ready speculative decoding with TensorRT-LLM
Our TensorRT-LLM Engine Builder now supports speculative decoding, which can improve LLM inference speeds.
A quick introduction to speculative decoding
Speculative decoding improves LLM inference latency by using a smaller model to generate draft tokens that the larger target model can accept during inference.
