Data Engineer Offline

Project Description:

We are hiring a Senior Full-Stack Software Developer. Our client team consists of frontend and backend developers, data engineers, data scientists, QA engineers, cloud engineers, and project managers.

Responsibilities:

• Participate in requirements clarification and sprint planning sessions.
• Design technical solutions and implement them, inc ETL Pipelines - Build robust data pipelines in PySpark to extract, transform, using PySpark
• Optimize ETL Processes - Enhance and tune existing ETL processes for better performance, scalability, and reliability
• Writing unit and integration tests.
• Support QA teammates in the acceptance process.
• Resolving PROD incidents as a 3rd line engineer.

Mandatory Skills Description:

* Min 5 Years of experience in IT/Data
* Bachelor in IT or related field.
* Exceptional logical reasoning and problem-solving skills
* Programming: Proficiency in PySpark for distributed computing and Python for ETL development.
* SQL: Strong expertise in writing and optimizing complex SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake.
* Data Warehousing: Experience working with data warehousing concepts and platforms, ideally DataBricks
* ETL Tools: Familiarity with ETL tools & processes
* Data Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design.
* Version Control: Proficiency with version control tools like Git to manage codebases and collaborate on development.
* Data Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance.
* Data Quality Tools: Experience implementing data validation, cleansing, and quality framewo

Nice-to-Have Skills Description:

Understanding of Investment Data domain.

Languages:

English: B2 Upper Intermediate