Python Software Engineer with Data engineering (IRC297367)

Job Description

We are seeking a software engineer with an excellent background in Python to join the core team responsible for the client's company data refinery flow. The team aims to make data accessible for deep learning and feature teams working on an Autonomous Driving solution.

Required competences and knowledge:

• Advanced Python: OOP, software design patterns, TDD, concurrency, algorithms, and libraries (pandas, numpy, plotly, pytest).
• Databases & SQL: Advanced PostgreSQL (query analysis, functions, caching, partitioning), alembic, and SQLAlchemy.
• Data Engineering: Building ETL pipelines and handling data container structures (.mf4, .hdf5, .pcap, etc.).
• Core DevOps & Tools: Linux, Shell scripting, Git, Docker, and AI-driven development tools (GitHub Copilot).
• System Design & Agile: Pipeline architecture (UML diagrams) and Agile/Scrum workflows (Jira, Confluence).
• Minimum B2 Level English skills

It would be a plus if the candidate has the following skills and experience:

• Orchestration & CI/CD: OpenShift, Apache Airflow (YAML configuration), Singularity containers, and Zuul/Gerrit/GoCD pipelines.
• API and Message Broker: REST APIs (FastAPI, Flask) and Kafka Message Broker.
• DevOps Utilities: Modern package management (uv), Artifactory policy management, and Grafana monitoring.
• Basic C++

Job Responsibilities

Contribute to various activities related to vision data refinery, some examples:

• Implementing and extending tools for automatic checking of raw data and annotation quality
• Automation of the data exchange with annotation companies
• Data curation (gathering data into collections according to the requirements of deep learning and feature teams)
• Development and monitoring function/feature KPIs (how changes in data influence function/feature performance and quality)
• Improving the speed and efficiency of data and training pipelines
• Implementing a dashboard for monitoring data collection, storage, and annotation
• Development of micro services/API towards external users

Department/Project Description

Our client is a Tier-1 Automotive company from Europe, developing full-stack software solutions that include algorithms for functions ranging from sensing to vehicle control, and targets a variety of applications from ADAS to advanced AD cars.