Job
- Level
- Experienced
- Job Field
- Data
- Employment Type
- Full Time
- Contract Type
- Permanent employment
- Location
- Bad Friedrichshall
- Working Model
- Onsite
Job Summary
In this role, you will develop powerful ETL pipelines for reports, analyses, and machine learning. You will ensure data quality, optimize the performance of solutions, and collaborate closely with interdisciplinary teams.
Job Technologies
Your role in the team
- Develop and operate productive ETL pipelines for reports, analysis, and machine learning, and ensure their quality and sustainability.
- Use your software development skills and set new standards for development and deployment.
- Optimize the performance of data engineering solutions and guarantee their scalability.
- Ensure data quality and consistency through data validation and cleansing.
- Monitor and fix errors in the data infrastructure to minimize downtime.
- Implement data security and privacy measures.
- Work in interdisciplinary agile teams with data scientists, data engineers, product owners, and departments.
This text has been machine translated. Show original
Our expectations of you
Qualifications
- Expertise in developing with Python. Scala knowledge is a plus.
- Gute Kenntnisse in Apache Airflow für die Workflow-Orchestrierung.
- Knowledge of the software development cycle and its best practices (e.g. CI/CD, DevOps, testing).
- Analytical skills, an eye for data quality, agile mindset, independence and hands-on mentality.
Experience
- +4 years of industry experience as a Data Engineer.
- Experience developing large scale batch & streaming ETLs in Spark, preferably in Databricks.
- Experience working in cloud environments, preferably GCP.
- Strong hands-on experience with modern, open-source table formats such as Apache Iceberg or Delta Lake to manage large-scale data lakehouse architectures.
- Proven track record managing data access controls, CLS, RLS, and cataloging workflows. Experience with Unity Catalog is a plus.
- Hands-on experience working with containerized applications, preferably orchestrated in Kubernetes.
- Experience working with Google Pub/Sub is preferred.
This text has been machine translated. Show original
Benefits
Work-Life-Integration
Topics that you deal with on the job
Job Locations
This is your employer
Schwarz Unternehmenskommunikation GmbH & Co. KG
The Schwarz Group, based in Neckarsulm, is a significant German conglomerate and one of the largest retail groups in Europe. It operates over 13,900 stores under the brands Lidl and Kaufland and employs around 575,000 people.
Description
- Company Type
- Established Company
- Working Model
- Full Remote, Hybrid, Onsite
- Industry
- Trade
Data Engineer
- Location
- Bad Friedrichshall
- Working Model
- Onsite
- Diversity
- Open for all genders
- English Only
- English only required
