Senior Data Engineer

Project description

We are hiring a Senior Full-Stack Software Developer. Our client team consists of frontend and backend developers, data engineers, data scientists, QA engineers, cloud engineers, and project managers.

Responsibilities

Participate in requirements clarification and sprint planning sessions.

Design technical solutions and implement them, inc ETL Pipelines

Build robust data pipelines in PySpark to extract, transform, using PySpark

Optimize ETL Processes

Enhance and tune existing ETL processes for better performance, scalability, and reliability

Writing unit and integration tests.

Support QA teammates in the acceptance process.

Resolving PROD incidents as a 3rd line engineer.

Skills

Must have

Min 7 Years of experience in IT/Data

Bachelor in IT or related field.

Exceptional logical reasoning and problem-solving skills

Programming: Proficiency in PySpark for distributed computing and Python for ETL development.

SQL: Strong expertise in writing and optimizing complex SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake.

Data Warehousing: Experience working with data warehousing concepts and platforms, ideally DataBricks

ETL Tools: Familiarity with ETL tools & processes

Data Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design.

Version Control: Proficiency with version control tools like Git to manage codebases and collaborate on development.

Data Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance.

Data Quality Tools: Experience implementing data validation, cleansing, and quality framework

Nice to have

Understanding of Investment Data domain.

 

Languages

English: B2 Upper Intermediate

Published 4 June
17 views
ยท
0 applications
To apply for this and other jobs on Djinni login or signup.
Loading...