Black Forest Labs seeks a Senior Member of Technical Staff to design and maintain large-scale data infrastructure for training generative AI models across thousands of GPUs.
This role focuses on engineering the data systems that power the company's largest training runs. The team, known for Latent Diffusion, Stable Diffusion, and FLUX, is looking for an infrastructure engineer to handle peta-to-exabyte scale data challenges while supporting millions of users of their generative models.
Responsibilities
- Develop scalable data loaders to support training runs spanning thousands of GPUs.
- Design and implement efficient storage and retrieval mechanisms for petabyte-scale datasets.
- Create a multi-cloud object storage abstraction layer.
- Manage large-scale data migrations between different storage systems and providers.
- Identify and resolve performance bottlenecks in distributed data loading environments.
Requirements
- Proven experience building and operating data pipelines at a petabyte scale.
- Demonstrated ability to optimize data loading processes.
- Hands-on work with petabyte-scale video and image datasets.
- Experience writing processing jobs that handle millions of files.
- Track record of debugging distributed system bottlenecks across large machine fleets.
Nice to have
- Familiarity with streaming dataset formats like WebDataset.
- Understanding of video codec internals and frame-accurate seeking.
- Background in distributed systems.
- Experience using Slurm and Kubernetes for job orchestration.
- Knowledge of object storage performance tuning across different providers.
What the company offers
- Flexible hybrid work model: join in Freiburg or SF at least 2 days a week, or work remotely with a monthly in-person week.
- Cover reasonable travel costs for in-person meetings.
- Base Annual Salary: EU €100,000 - €230,000 + Equity
- Base Annual Salary: US $150,000 - $300,000 + Equity
About the company
Black Forest Labs is a frontier research lab headquartered in Freiburg, Germany, with a growing presence in San Francisco. The company is known for its work on Latent Diffusion, Stable Diffusion, and FLUX, creating generative models for images and video used by millions. The team operates as a distributed group with real offices that people actually use, emphasizing in-person collaboration while maintaining a flexible work model. The company values research excellence, open science, and building technology that expands human creativity.
- The team operates with four core values: Obsessed, Low Ego, Bold, and Kind.
- The best idea wins, no matter who said it. Credit is shared. Nobody is above any task.
- We take the ambitious bet. We ship, we do not wait for conditions to be perfect.
- People over politics. We treat each other with genuine warmth. Agency without empathy creates chaos.
- We think in-person time matters, and we’ve structured things to make it accessible to all.
Quelle: öffentlich zugängliche Karriereseite des Arbeitgebers. Batchly ist nicht der Arbeitgeber und steht nicht notwendigerweise in einem Vertragsverhältnis mit dem Unternehmen.