Data Engineer (offline)
Our company is looking for a Data Engineer to join a global industrial project, with the passion to drive the design, execution, and ongoing support of data services enabling large-scale data collection, near real-time and offline analytics, distributed search, using AI and ML for security, engineering, and business intelligence purposes.
Stack of Technologies:
GCP;
BigData;
Spark or mapReduce;
SQL/BigQuery/PubSub
BigQuery, Cloud Composer, Data Fusion, GCS and GKE;
Python or JavaScript;
Terraform scripting.
About the role:
As a Data Engineer you will work with one of the largest B2B data sets in the world using the latest cutting-edge technologies. You will have an opportunity to build data pipelines and analyze data structures. And connect sources to data lake and work with Geodata.
Responsibilities:
β’ Maintain and support existing ingestion pipelines;
β’ Optimize queries based on performance testing;
β’ Design and implement new ingestion pipelines that bring data from external data sources (HTTPS, SFTP) or internal data sources (JDBC, HTTP, MQQT);
β’ Work with Application engineers and product managers on refining data requirements;
β’ Implement and test fine grained access control setup (per dataset, per column).
Required skills:
β’ Strong experience in building cloud-native data engineering solutions using GCP or AWS platforms;
β’ 2+ years of development experience with BigData;
β’ Prior experience of building data ingestion pipelines of telemetry data in GCP/AWS including app monitoring, performance monitoring and network monitoring logs;
β’ Background in building data integration applications using Spark or mapReduce frameworks;
β’ Track record of producing software artifacts of exceptional quality by adhering to coding standards, design patterns and best practices;
β’ Strong background in SQL / BigQuery / PubSub;
β’ Experience with GCP products such as BigQuery, Cloud Composer, Data Fusion, GCS and GKE or corresponding technologies on AWS platform;
β’ High proficiency in working with Git, automated build and CI/CD pipelines;
β’ ETL scripting in Python or Javascript;
β’ Knowledge in some terraform scripting (adding new datasets, buckets, IAM);
β’ Intermediate level of English or higher.
Our benefits:
β’ Strong opportunities for professional and career growth - Meetups, TechClubs, Professional Library and more;
β’ Challenging tasks with a friendly experienced team;
β’ Flat hierarchy without micromanagement β our doors are open, and all teammates are approachable;
β’ Direct communication with stakeholders and the ability to influence product development;
β’ Up to 50% compensation of educational courses and conferences price for professional growth;
β’ Free English classes and English Business Course;
β’ 23 business days leave and medical Covid support;
β’ Regular team events and activities;
β’ Gifts for significant life events.
Stack of Technologies:
GCP;
BigData;
Spark or mapReduce;
SQL/BigQuery/PubSub
BigQuery, Cloud Composer, Data Fusion, GCS and GKE;
Python or JavaScript;
Terraform scripting.
About the role:
As a Data Engineer you will work with one of the largest B2B data sets in the world using the latest cutting-edge technologies. You will have an opportunity to build data pipelines and analyze data structures. And connect sources to data lake and work with Geodata.
Responsibilities:
β’ Maintain and support existing ingestion pipelines;
β’ Optimize queries based on performance testing;
β’ Design and implement new ingestion pipelines that bring data from external data sources (HTTPS, SFTP) or internal data sources (JDBC, HTTP, MQQT);
β’ Work with Application engineers and product managers on refining data requirements;
β’ Implement and test fine grained access control setup (per dataset, per column).
Required skills:
β’ Strong experience in building cloud-native data engineering solutions using GCP or AWS platforms;
β’ 2+ years of development experience with BigData;
β’ Prior experience of building data ingestion pipelines of telemetry data in GCP/AWS including app monitoring, performance monitoring and network monitoring logs;
β’ Background in building data integration applications using Spark or mapReduce frameworks;
β’ Track record of producing software artifacts of exceptional quality by adhering to coding standards, design patterns and best practices;
β’ Strong background in SQL / BigQuery / PubSub;
β’ Experience with GCP products such as BigQuery, Cloud Composer, Data Fusion, GCS and GKE or corresponding technologies on AWS platform;
β’ High proficiency in working with Git, automated build and CI/CD pipelines;
β’ ETL scripting in Python or Javascript;
β’ Knowledge in some terraform scripting (adding new datasets, buckets, IAM);
β’ Intermediate level of English or higher.
Our benefits:
β’ Strong opportunities for professional and career growth - Meetups, TechClubs, Professional Library and more;
β’ Challenging tasks with a friendly experienced team;
β’ Flat hierarchy without micromanagement β our doors are open, and all teammates are approachable;
β’ Direct communication with stakeholders and the ability to influence product development;
β’ Up to 50% compensation of educational courses and conferences price for professional growth;
β’ Free English classes and English Business Course;
β’ 23 business days leave and medical Covid support;
β’ Regular team events and activities;
β’ Gifts for significant life events.
About tech-stack.io
TechStack - the team of ambitious and energetic IT professionals. Our philosophy is a total IT service.Company website:
https://tech-stack.ip
The job ad is no longer active
Job unpublished on
8 July 2021
Look at the current jobs Data Science Kharkiv→
Average salary range of similar jobs in
analytics β
Similar jobs
ML Engineer at Outstaff your team
Ukraine
Data Scientist at Athena Legal
Ukraine
All jobs Data Science Kharkiv All jobs Techstack