Job
- Level
- Experienced
- Job Field
- IT, Software, DevOps
- Employment Type
- Full Time
- Contract Type
- Permanent employment
- Location
- Berlin
- Working Model
- Hybrid, Onsite
Job Summary
In this role, you continuously develop our LLM proxy, facilitate access to LLM providers, ensure data protection, and support product teams with integration and model selection.
Job Technologies
Your role in the team
- You continuously operate and develop our LiteLLM-based LLM proxy, providing product teams with reliable, consistent access to LLM providers (Azure OpenAI, AWS Bedrock, among others).
- You operate the gateway as a production-critical service, define SLOs, provider failover, and rate limiting, and take the lead in incident response.
- You ensure data protection at the gateway through GDPR-compliant routing, EU data residency, clear boundaries of data processing by providers, and handling of personal data — while keeping an eye on the relevant provisions of the EU AI Act for AI in education.
- You advise product teams early on regarding model selection, integration patterns, and prompt strategies, and get involved before issues reach production.
- You create clear, practical developer documentation and best practice examples, and support teams with onboarding and daily platform usage.
- You are the primary contact for product teams working with LLMs.
- You establish centralized observability for LLM and agent applications, including tracing, logging, and monitoring via Langfuse.
This text has been machine translated. Show original
Our expectations of you
Qualifications
- You possess solid Kubernetes knowledge and are familiar with at least one cloud platform (Azure, AWS, or GCP).
- You are familiar with Azure OpenAI, AWS Bedrock, or comparable model providers.
- You have a genuine consulting mindset – you enjoy helping other engineers succeed and can persuade without formal authority.
- You have the ability to clearly communicate technical trade-offs to product teams and non-technical stakeholders.
- You have strong skills in documentation and knowledge transfer.
- Very good German and English skills complete your profile.
Experience
- You have practical experience operating an LLM proxy / gateway in production, with LiteLLM strongly preferred, or a comparable solution.
- You have experience with observability tooling, ideally Langfuse or a similar tracing and monitoring stack.
This text has been machine translated. Show original
What we offer
- Shape the future of education in a meaningful environment.
- Experience optimal work-life balance with 30 days of vacation and flexible working hours.
- Work flexibly remotely up to 3 days a week and enjoy up to 20 days of workation in Europe (EWR) per year.
- Work agile in dynamic and cross-functional teams.
- Enhance your health and well-being with the offerings from evermood and Wellhub.
- Benefit from attractive perks such as childcare and after-school care allowances, discounts on books, company pension schemes, and asset-building benefits.
- Rely on sustainable mobility with company bike leasing and a reduced Germany ticket – good for you and our environment.
- Enjoy locally-sourced organic food in our in-house canteen during shared lunch breaks with your colleagues.
- Unleash your potential through personal and professional development opportunities (further training, industry conferences, e-learning & networking).
- Experience strong team spirit at sports events, company outings, and staff festivals.
This text has been machine translated. Show original
Benefits
Work-Life-Integration
Topics that you deal with on the job
Job Locations
This is your employer
Cornelsen Gruppe
Mit Unternehmen und Marken wie dem Cornelsen Verlag, DUDEN, VERITAS oder Verlag an der Ruhr zählt die Cornelsen Gruppe zum führenden Bildungsmedienanbieter im deutschsprachigen Raum. Seit über sieben Jahrzehnten lernen und unterrichten Menschen mit Bildungsmedien von Cornelsen. Seitdem haben Millionen von Lehrenden und Lernenden die starken Marken der Cornelsen Gruppe genutzt, um sich weiter zu entwickeln.
Description
- Founding year
- 1946
- Company Type
- Established Company
- Working Model
- Hybrid, Onsite
- Industry
- Media, Publishing, Internet, IT, Telecommunication
Platform Engineer LLMOps/AI Infrastructure
- Location
- Berlin
- Working Model
- Hybrid, Onsite
- Diversity
- Open for all genders
