Senior Data Engineer (Autonomous Vehicles Data)

Senior Data Engineer (Autonomous Vehicles Data)
At RSB Automotive Consulting, we work with engineers and technology teams developing advanced mobility solutions. We are currently looking for a Senior Data Engineer to join a long-term project focused on data processing and analytics for Autonomous Vehicle (AV) development. In this role, you will work with large-scale sensor datasets collected from test vehicles and support teams building simulation environments and machine learning pipelines used to validate safety-critical AV systems.

Project context: data pipelines and analytics supporting Autonomous Vehicle (AV) development
Tech stack: Cloud, Python, SQL, Spark / PySpark, Databricks
Data scale: up to ~1 TB of sensor data per hour (cameras, LiDAR, radar)
Focus: time-series data, advanced analytics, simulation-ready datasets, and ML data preparation
Work mode: remote (any location), with daily overlap with the US team
Start: ASAP
Your responsibilities
Analyze large volumes of sensor and time-series data from autonomous vehicle test fleets

Develop advanced SQL, Python, and PySpark queries to filter, transform, validate, and aggregate datasets

Design and maintain ETL and data processing pipelines handling large-scale structured and semi-structured data

Support and troubleshoot distributed data workflows and pipeline operations

Monitor and optimize orchestration pipelines (e.g. Airflow, Argo Workflows, or similar technologies)

Identify and extract data suitable for AV simulation scenarios and ML training pipelines

Support the discovery of rare or complex driving situations (e.g. unusual traffic scenarios, hard braking events, edge cases)

Investigate data inconsistencies, pipeline failures, and performance bottlenecks

Develop scripts and internal tools supporting data mining, debugging, and operational automation

Collaborate with engineers working across backend, infrastructure, and autonomous driving technology teams

What we’re looking for
Strong software engineering background

Advanced SQL skills with experience writing complex queries

Advanced Python programming

Understanding of distributed data processing and large-scale data workflows

Experience working with cloud-based data platforms

Experience with workflow orchestration tools such as Airflow, Argo Workflows, or similar

Understanding of infrastructure concepts including storage systems, microservices, and pipeline architecture

Experience working with notebooks and analytical workflows

Familiarity with troubleshooting and operational support of production data pipelines

Understanding of data preparation for machine learning workflows

Nice to have
Hands-on experience with Spark / PySpark

Experience working with Databricks

Experience in advanced data analytics and time-series analysis

Experience supporting analytics, simulation, or ML-related data pipelines

Understanding of Autonomous Vehicle development context and real-world edge cases

Degree in Computer Science or related field

Project context
You will work with sensor data generated by autonomous vehicle test fleets equipped with multiple cameras, LiDAR, and radar systems. These platforms generate extremely large datasets used to build and validate simulation environments supporting autonomous driving algorithms.
The role focuses on transforming raw sensor data into structured, simulation-ready datasets used by engineering and research teams working on safety-critical AV features, including obstacle detection, path planning, and complex traffic scenarios.
If you are interested in working with large-scale data systems and real-world autonomous driving datasets, we would be happy to connect.