Job
- Level
- Senior
- Job Field
- Data
- Employment Type
- Full Time
- Contract Type
- Permanent employment
- Location
- Bad Friedrichshall
- Working Model
- Onsite
Job Summary
In this role, you will develop productive ETL pipelines for analysis and machine learning, optimize their performance, and ensure data quality through validation and cleansing within agile teams.
Job Technologies
Your role in the team
- Develop and operate productive ETL pipelines for reports, analysis, and machine learning, and ensure their quality and sustainability.
- Use your software development skills and set new standards for development and deployment.
- Optimize the performance of data engineering solutions and guarantee their scalability.
- Ensure data quality and consistency through data validation and cleansing.
- Monitor and fix errors in the data infrastructure to minimize downtime.
- Implement data security and privacy measures.
- Work in interdisciplinary agile teams with data scientists, data engineers, product owners, and departments.
This text has been machine translated. Show original
Our expectations of you
Qualifications
- Expertise in developing with Python. Scala knowledge is a plus.
- Good command of Apache Airflow for workflow orchestration.
- Knowledge of the software development cycle and its best practices (e.g. CI/CD, DevOps, testing).
- Analytical skills, an eye for data quality, agile mindset, independence and hands-on mentality.
Experience
- +4 years of industry experience as a Data Engineer.
- Experience developing large scale batch & streaming ETLs in Spark, preferably in Databricks.
- Experience working in cloud environments, preferably GCP.
- Strong hands-on experience with modern, open-source table formats such as Apache Iceberg or Delta Lake to manage large-scale data lakehouse architectures.
- Proven track record managing data access controls, CLS, RLS and cataloging workflows. Experience with Unity Catalog is a plus.
- Hands-on experience working with containerized applications, preferably orchestrated in Kubernetes.
- Experience working with Google Pub/Sub is preferred.
This text has been machine translated. Show original
What we offer
- Join us and contribute to digital sovereignty in Europe.
- With us, you will work at the intersection of agility and security: You will benefit from fast decision-making processes, enjoy genuine creative freedom in your projects, and be able to build upon the stable foundation of the Schwarz Group.
This text has been machine translated. Show original
Benefits
Work-Life-Integration
Topics that you deal with on the job
Job Locations
This is your employer
Schwarz Unternehmenskommunikation GmbH & Co. KG
The Schwarz Group, based in Neckarsulm, is a significant German conglomerate and one of the largest retail groups in Europe. It operates over 13,900 stores under the brands Lidl and Kaufland and employs around 575,000 people.
Description
- Company Type
- Established Company
- Working Model
- Full Remote, Hybrid, Onsite
- Industry
- Trade
Data Engineer
- Location
- Bad Friedrichshall
- Working Model
- Onsite
- Diversity
- Open for all genders
- English Only
- English only required
