Job
In this role, you will develop automated data pipelines, optimize data processing workflows, and implement modern data architectures for batch and streaming data on hybrid platforms.
This listing has been machine translated. Show original
Job technologies
Your role in the team
- Pipeline Development: You develop and maintain automated data ingestion pipelines and ensure the reliable processing of batch and streaming data.
- Participation in the platform: You assist in the development of modern, hybrid data platforms (e.g., based on Spark/Databricks or comparable technologies) and implement features for modern data architectures (Lakehouse & DWH).
- Data Quality: You implement schema validations and data quality checks during the ingestion process and assist with metadata extraction.
- Governance Support: You work on implementing access controls and data versioning to ensure the reproducibility of datasets.
- Performance Support: You assist in optimizing data processing workflows and help minimize unnecessary data movements within the system.
- Growth: In collaboration with our seniors, you will deepen your knowledge in distributed systems, MLOps tooling, and modern Lakehouse designs.
Our expectations of you
Education
- Excellent German and English skills as well as a completed degree in Computer Science, Data Engineering, or a related field.
Qualifications
- Programming: Good knowledge of Python and confident handling of relational databases (SQL).
- Cloud basis: Basic understanding of cloud technologies (Azure or AWS) and initial contact with Docker.
- Version control: Proficient in Git and understanding of Data Lineage concepts.
- Very good knowledge: Basic understanding of automated workflows and deployments.
- Data modeling: Knowledge of partitioning and fundamental architecture principles of Data Lakes.
- Data modeling: Basic understanding of dimensional data modeling (Star/Snowflake Schema) and architecture principles of Data Lakes.
- Willingness to learn: You are highly motivated to familiarize yourself with the rapidly evolving data and AI landscape.
- Team spirit: You enjoy working collaboratively with engineering and analytics teams.
- Structured working style: You approach tasks methodically and have an eye for data integrity.
Experience
- Solid knowledge: First practical experience with Distributed Computing (e.g., Spark/Databricks or comparable technologies) and building data pipelines.
- Quality requirement: Experience with testing procedures for data processes and Clean Code standards.
Benefits
Work-Life-Integration
Food & drink
Health, fitness & fun
Topics you will work on
Job locations
About your employer
BCM Solutions GmbH stands for high-quality software solutions and services in the field of software development.
- Digital Agency
- Onsite, Hybrid, Full Remote
- Internet, IT, Telecommunication
Employer reviews
- Engineering
- 3.3
- Working conditions
- 4.1
- Team culture
- 4.1
- Career growth
- 4.3
More jobs
These jobs might also interest you
Hello, let's say WE! We - the HUK-COBURG Insurance Group - are among the top 10 in Germany. Founded over 90 years ago, we are now the leading insurer for private households with over 13 million customers and the largest German auto insurer. Our...
The role As a Forward Deployed AI Engineer, you go directly into our specialist departments, understand the processes, and develop production-ready AI and automation solutions that will fundamentally change the way the entire GGW Group operates. You do...
Consultant & Project Manager – You are the primary contact for our clients and take on the leadership of new projects. Optimize processes & create added value – With your expertise, you will design more efficient workflows and support our clients with...
