Together AI Tech Stack
AI infrastructure platform for inference, model training, and GPU compute
Together AI operates a full-stack AI platform spanning inference engines, on-demand GPU clusters, and pre-training infrastructure. The tech stack reveals a systems-first engineering culture: heavy emphasis on performance-critical layers (CUDA, Triton, FlashAttention, vLLM, SGLang) alongside orchestration and data pipelines (Kubernetes, Airflow, dbt, ClickHouse). The company is scaling aggressively—28 engineering roles posted in the last 30 days—and pain points cluster around GPU utilization, latency optimization, and cost control, suggesting they're hitting limits on both their internal infrastructure and customer workload density.
Tech Stack 158 technologies
What Together AI Is Building
◆Challenges
- Scaling data center infrastructure
- Scaling hiring for core engineering functions
- Handling millions events daily
- Optimizing latency
- Cost optimization for storage
- Lower the cost of modern ai systems
- Reducing inference latency and cost
- Optimizing gpu utilization
- Scaling to massive concurrent users
- Scaling compute infrastructure
▲Active Projects
- Integration of partner models into serving stack
- Runtime inference services
- Design and operate medallion data warehouse stack
- Build airflow orchestrated pipelines and dbt transformation projects
- Cuda graph optimization for inference
- Build scalable infrastructure with ansible, terraform, kubernetes
- Sla monitoring system
- Distributed gpu scheduling system
- Customer-facing cloud platform services
- Global management plane
Hiring Activity
Department
Seniority
Notable leadership hires: Tax Director