Senior Data Engineer (offline)
As a part of data migration into Azure Cloud - extract data from Oracle Hive cluster, convert into Parquet format and load to the Azure Blob Storage.
Utilize Spark for further data manipulations and analysis in a distributed manner.
Responsibilities:
β’ Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure 'big data' technologies;
β’ Implement data flows connecting operational systems, BI systems, and the big data platform;
β’ Build real-time, reliable, scalable, high-performing, distributed, fault tolerant systems;
β’ Clean and transform data into a usable state for analytics. Build data dictionary;
β’ Create data tools for analytics and data scientist team members that assist them in their ML endeavors;
β’ Design and develop code, scripts and data pipelines that leverage structured and unstructured data;
β’ Implement measures to address data privacy, security, compliance.
Mandatory Skills:
HiveQL, Scala, Java, Apache HBase, Python, Kafka Streams, Big Data, Apache Kafka, Hadoop
β’ Experience with designing data and analytics architectures in Microsoft Azure cloud;
β’ Experience with Big Data technologies like Spark, Hadoop, Hive, HBase, Kafka etc.;
β’ Fluency in several programming languages such as Python, Scala, Java, with the ability to pick up new languages and technologies quickly;
β’ Experience with data warehousing, data ingestion, and data profiling;
β’ Demonstrated teamwork, strong communication skills, and collaborative in complex engineering projects.
Nice-to-Have Skills:BS in computer science or related STEM field;
About Luxoft
Luxoft is a high-end application outsourcing provider of choice and a trusted technology advisor to Global 2000 and medium-sized growth companies that apply compelling technologies to obtain leadership positions in their respective markets.Luxoft today finds the Best talents, proposes career growth & employment benefits. Our teams are involved in high complicity & innovative projects for the Top leaders companies around the Globe.
Company website:
https://career.luxoft.com/locations/ukraine/
DOU company page:
https://jobs.dou.ua/companies/luxoft/
The job ad is no longer active
Job unpublished on
3 December 2020
Look at the current jobs Data Science Kyiv→
Similar jobs
Portfolio Manager at Avans.credit
Lithuania, Ukraine
Machine Learning Engineer, Search at robota.ua
Ukraine
Senior MLOps Engineer at ΠΠ£ΠΠ
Ukraine
All jobs Data Science Kyiv All jobs Luxoft