Job
In this role, you will develop scalable web crawlers that transform dynamic websites into structured data while ensuring the stability of these systems as you counter anti-bot measures.
This listing has been machine translated. Show original
Job technologies
Your role in the team
- We build and operate large-scale web crawling and extraction systems that turn unstructured, dynamic websites into clean, structured, and correct data.
- You will own crawlers end-to-end: discovery, resilient fetching against real anti-bot defenses, and turning HTML/PDF into trustworthy structured output.
- Build and scale crawlers that handle dynamic, large-scale sites - and keep them running as those sites change and fight back.
- Bleiben Sie bei Anti-Bot-Maßnahmen einen Schritt voraus: TLS/Browser-Fingerprinting, Proxys, Sitzungs- und Ratenbegrenzungsstrategie, Änderungsdetektion.
- Turn raw HTML and PDF into clean, correct structured data - including LLM-assisted extraction - where silently-wrong output is worse than a crash.
- Design and operate pipelines for ingestion, deduplication, versioning, and enrichment.
- Ship and run containerized apps in the cloud, with CI/CD.
- Shape our technical architecture and engineering culture.
Our expectations of you
Qualifications
- Starkes Python - echter Produktionscode, async.
- A data-correctness mindset - you care that extracted data is right, not just that the job finished.
- Docker + mindestens eine große Cloud-Plattform (GCP, Azure oder AWS), Git, vertraut mit der CLI.
- Clear communicator who works well in a small, fast-moving team.
- Nice to have: LLM-assisted extraction / working with LLMs, ML pipelines, or data labeling workflows.
- Nice to have: Go or Java/Spring Boot.
- Nice to have: Frontend (React) - or eagerness to pick it up.
- Nice to have: Terraform, Kubernetes, modern DevOps tooling.
Experience
- 3+ Jahre professionelle Softwareentwicklung (5+ für die Senior-Position).
- Hands-on web scraping at scale, including firsthand experience with anti-bot defenses and the failure modes of large-scale collection.
What we offer
- A vibrant in-office culture in Munich with the option for up to 60% remote work.
- Flexible working hours.
- 30 vacation days per year (including 4 "company rest days" over Christmas).
- 30 days of "workation" per year, within the EU and selected countries.
- High autonomy and flat hierarchies.
Benefits
Work-Life-Integration
Topics you will work on
Job locations
About your employer
Certivity is a RegTech startup based in Munich that develops an AI-powered platform to support companies in regulated industries. This platform transforms regulatory documents into structured information and assists teams in monitoring regulatory changes. Since its founding in 2021, the company has gained over 15,000 users in 10 countries.
- Startup
- Onsite, Hybrid
- Internet, IT, Telecommunication
More jobs
These jobs might also interest you
We - the HUK-COBURG Insurance Group - are among the top 10 largest in Germany. Founded over 90 years ago, we are now the leading insurer for private households with over 13 million customers and the largest German auto insurer. Our tradition of often...
You actively shape the technical development of our software products and influence their long-term architecture. You develop our core applications primarily using C# as the central programming language and technological foundation. You design, develop,...
As a Senior Software Engineer (m/f/d), you will take on a central technical role in challenging client projects and develop sustainable, scalable software solutions in complex enterprise environments. In your role, you will primarily be responsible for:...