Job
- Level
- Senior
- Job Field
- Data, Back End
- Employment Type
- Full Time
- Contract Type
- Permanent employment
- Location
- Bad Friedrichshall
- Working Model
- Onsite
Job Summary
In this role, you design, develop, and maintain high-performance data flows in Google Cloud, optimize architecture, and implement automated testing for real-time and batch data processing.
Job Technologies
Your role in the team
- Data Product Engineering: You design, develop, and operate high-performance batch and streaming data pipelines in our central Google Cloud Data Lakehouse for analytics and machine learning use cases.
- Architecture & Scaling: You optimize the cloud architecture (GCP), ensure the scalability of our solutions, and further develop our standards for real-time and batch data processing.
- Software Excellence: You apply modern software engineering methods and set new standards for CI/CD, automated testing, and deployment processes in data organization.
- Framework Co-Design: You support the further development of our central orchestration tools (based on Airflow and dbt) to enhance the efficiency and quality of data delivery across the company.
- Data Quality & Governance: You implement automated validation processes (e.g., for streaming events) and ensure compliance with security and data protection standards.
- Agile Collaboration: You work closely with interdisciplinary teams, including Product Owners, Data Scientists, and business units, to translate complex business requirements into technical solutions.
This text has been machine translated. Show original
Our expectations of you
Qualifications
- Tech Stack: Very good knowledge of Python and Spark. You are confident in containerization (Kubernetes/Docker).
- Software best practices: You are confident in handling the software development cycle, version control (Git), as well as DevOps principles and automated testing.
- Conceptual strength: You are enthusiastic about modern architecture concepts such as Data Lakehouses and Data Mesh, and you enjoy actively shaping technological standards.
- Personality: Analytical thinking, an agile mindset, as well as a responsible and solution-oriented work approach.
- Languages: Very good English skills; German skills are preferred.
Experience
- Experience: Several years of solid experience in Data Engineering, Big Data environments, or Software Engineering, ideally in a senior role.
- Cloud & Streaming Expertise: You possess solid knowledge of cloud infrastructures (Google Cloud Platform) as well as extensive experience in real-time data processing with Apache Kafka.
- Orchestration & Transformation: Experience with tools like Apache Airflow and transformations via dbt is highly advantageous.
This text has been machine translated. Show original
Benefits
Work-Life-Integration
Topics that you deal with on the job
Job Locations
This is your employer
Schwarz Unternehmenskommunikation GmbH & Co. KG
The Schwarz Group, based in Neckarsulm, is a significant German conglomerate and one of the largest retail groups in Europe. It operates over 13,900 stores under the brands Lidl and Kaufland and employs around 575,000 people.
Description
- Company Type
- Established Company
- Working Model
- Full Remote, Hybrid, Onsite
- Industry
- Trade
Senior Data Engineer - Google Cloud Platform
- Location
- Bad Friedrichshall
- Working Model
- Onsite
- Diversity
- Open for all genders
