Our client is the leader in the Early Childhood Education (ECE) sector, providing the most widely used developmental assessment and curriculum system in the US. They are looking for a highly skilled Data Engineer to join their team and contribute to the development and maintenance of their Data Pipelines. Working under the guidance of their Senior Data Engineer, you will be responsible for executing tasks independently, leveraging your expertise in Python, Spark, Airflow, S3, and Redshift. Your primary focus will be on enhancing and optimizing data pipelines, as well as crafting and executing SQL transformations to update and store data in delta lake tables.
As a Data Engineer, you will:
- Develop and optimize data transformations using Python, Spark, Airflow, S3, and Redshift;
- Craft SQL queries to efficiently manipulate data for storage in delta lake tables;
- Ensure the reliability, scalability, and performance of our data processing solutions;
- Collaborate with our Senior Data Engineer to implement and enhance data pipelines;
- Independently execute tasks related to building, maintaining, and improving data pipelines.
- 3+ years of hands-on experience in a data engineering or data system development role;
- Proven expertise in building and optimizing 'big data' pipelines and architectures;
- Proficiency in Python for data processing;
- Strong experience with Spark, with a focus on PySpark;
- Familiarity with AWS services such as EMR, S3, and MWAA (Airflow);
- Solid understanding of data modeling, data warehousing, and ETL/ELT techniques;
- Proficiency in working with data in various formats, including parquet, structured, and unstructured data, stored in S3, Redshift, or other columnar databases;
- Exceptional collaboration skills with excellent verbal and written communication abilities;
- Keen attention to detail and a track record of solving complex problems.
- Competitive market salary;
- Fully remote work;
- Convenient and somewhat flexible work hours (you will have to work until (19:00 - 20:00 Kyiv time (12 pm - 1 pm ET);
- 15 days of paid vacation per year;
- At least 7 additional days off per year (on national holidays);
- 5 days of paid sick leave per year;
- Working on a very socially valuable product, alongside a highly professional, US-based, international team;
- Interesting challenges and the ability to grow and learn.
About GM CONSULTING GROUP
GM CONSULTING GROUP is a professional IT consulting services company, delivering specialized technology-enabled business solutions.
Our platform is evolving, so you need to be someone who can build new functionality and learn new technologies quickly, as well as be able to maintain and extend existing code to support organizational needs.
Our specialization is applying knowledge, derived from industry expertise of technology and capabilities, to our clients’ business challenges.
DOU company page:
Job posted on
20 November 2023
To apply for this and other jobs on Djinni