Member of Technical Staff - Distributed Systems

Role Summary

Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will become an expert; nothing less will do in our immensely competitive market.

Build the systems that make AI inference fast, reliable, and cost-efficient at global scale. You’ll design the control plane that schedules a huge queue of tokens over a diverse fleet of machines, spread all over the world.

Responsibilities

• Design and implement high-performance schedulers (admission control, queuing, priority, fairness, preemption, bin packing).
• Build global routing and traffic management (latency-aware dispatch, predictive autoscaling, failover strategies).
• LLM-specific routing optimizations, e.g. KV caching that lets us trade memory for compute, across the pyramid of GPU RAM, CPU RAM, and NVMe flash.
• Build deep observability: we want to trace every millisecond of our systems, and catch failures early enough that we can make things right before the customer even notices.

Requirements

• Strong distributed systems fundamentals (concurrency, networking, databases, performance engineering).
• Eagerness to work with agents. Distributed systems are not easy to one-shot; you’ll always have to think carefully about testing correctness and edge cases. Writing extremely clear plans and tests is a must.
• Bonus: experience with ML inference stacks (vLLM/SGLang), GPUs/accelerators.

About the Company

LIFE AT SAIL

We work out of a beautiful, sunny office in downtown San Francisco. All meals are on us (and actually great; SF is a food paradise and it would be a shame to eat only bowl slop). Everyone gets a Studio Display at their desk. We are serious about investing in anything that saves us time or energy. There are six different ways to make coffee or tea in the office. A friendly (hypoallergenic) black cat named Coco visits occasionally.

Additional Information

1. Meet the CEO. This is the first step because we respect your time. Ask any question and get a definitive answer immediately.
2. Meet the CTO, who will ask about your experience, and share as much technical detail about Sail as you want to hear.
3. Come in to Sail's SF office for an interview day. Meet the whole team, then you'll have 3-4 hours to work on a problem that closely simulates the work we do daily. It's an objectively scored task, so you'll have immediate feedback on how well your code is working - just like we do in production! AI assistance is highly encouraged, and we'll provide a laptop with all the best tools set up. Finish with a short presentation describing your process, learnings, and results.
4. Offer. Once the team decides we want to work with you, we make a strong offer quickly and will be quite persistent over email/text/calls :)