Test Engineer - AI Voice Systems

Your role in the team

Our expectations of you

Experience

• 5+ years in test automation / quality engineering with strong Python skills, including ownership of a test framework rather than only writing cases in someone else's. Hands-on experience evaluating LLM or other non-deterministic systems: building evaluation pipelines, LLM-as-judge patterns, embedding/cosine-similarity scoring, and reasoning about scoring reliability and inter-rater agreement. Practical command of LLM quality metrics - faithfulness, hallucination rate, relevance, plus at least one of BLEU / WER / CER - and the judgment to know what each does and does not prove. Experience with LangChain / LangGraph or equivalent, and with an LLM observability tool (LangFuse, Phoenix, Langsmith or comparable) for tracing, prompt versioning and evaluation runs. Prompt regression testing in practice: detecting quality drift across prompt edits and model version changes. Solid conventional test engineering: pytest, code coverage, test case design across positive, negative, boundary and regression cases; testcontainers or equivalent integration test tooling. Experience integrating ASR and TTS services (Azure Speech, Whisper, Cerence, Google TTS or equivalent), including streaming/real-time behavior. CI/CD integration of automated test suites (Azure DevOps Pipelines, GitHub Actions) and familiarity with Docker; working knowledge of REST APIs and SQL/PostgreSQL for test data and results. English C1, and willingness to work onsite in the Ingolstadt/Munich area, including hands-on test sessions in vehicles and on the test rack.