Baseten @ KubeCon Europe 2025
We look forward to seeing you at KubeCon Europe 2025 in London!
Experience Baseten’s AI inference platform, offering industry-leading performance, security, and scale for organizations building AI products. With features like on-chip model optimization, autoscaling, and a developer-friendly interface, Baseten enables rapid, secure, and cost-effective deployment, ensuring you can maintain competitive user experiences while adhering to strict compliance requirements.
Host
Baseten
NYC Tech Breakfast Club: ML Engineering Leaders
Join us for the NYC Tech Breakfast Club for ML Engineers
Driving model performance optimization: 2024 highlights
Baseten's model performance team works to optimize customer models for latency, throughput, quality, cost, features, and developer efficiency.
New observability features: activity logging, LLM metrics, and metrics dashboard customization
We added three new observability features for improved monitoring and debugging: an activity log, LLM metrics, and customizable metrics dashboards.
How we built production-ready speculative decoding with TensorRT-LLM
Our TensorRT-LLM Engine Builder now supports speculative decoding, which can improve LLM inference speeds.
A quick introduction to speculative decoding
Speculative decoding improves LLM inference latency by using a smaller model to generate draft tokens that the larger target model can accept during inference.
