Data Engineer
For our customer, an international building group, we are looking for a Data Engineer to support the project.
Tasks:
• Identify meaningful, reliable and accurate data across a diverse data landscape;
• Design and develop new data engineering pipelines with cloud tools and frameworks;
• Prepare and accompany the roll-out of our data pipelines and applications by ensuring data availability and consistency in our Data Lake;
• Requirement engineering of reporting needs and communication with different stakeholders;
• Contribute as a valued team member to testing, debugging, QA, and documentation of data pipelines and systems;
Qualifications:
• Strong analytical and problem-solving skills;
• Data Engineering working experience, ideally with Python;
• Experience with big data technology stack, for example, (Py-)Spark, Delta Tables, etc.
• Several years of experience with general-purpose languages such as C#, Python, Java, Scala etc.
• Fluency in SQL and relational DB systems (PostgreSQL, SQL Server);
Nice to have:
• Much Experience with data warehousing, infrastructure, ETL/ ELT and analytic tools such as Azure Synapse or Databricks
• Experience with multiple Big Data file formats (Parquet, Avro, Delta Lake)
• Experience with time-series databases and streaming data processing
• Proficiency with PySpark
• Proficiency with .Net
• Experience setting up MLOps processes and a machine learning model registry
• Experience with CI/CD pipelines on Gitlab
• Some experience with Terraform and Kubernetes