We are looking for Senior/Regular Data Engineer (Python, Spark, Hadoop)

As a part of data migration into Azure Cloud - extract data from Oracle Hive cluster, convert into Parquet format and load to the Azure Blob Storage.
Utilize Spark for further data manipulations and analysis in a distributed manner.
Responsibilities:
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure 'big data' technologies;
• Implement data flows connecting operational systems, BI systems, and the big data platform;
• Build real-time, reliable, scalable, high-performing, distributed, fault tolerant systems;
• Clean and transform data into a usable state for analytics. Build data dictionary;
• Create data tools for analytics and data scientist team members that assist them in their ML endeavors;
• Design and develop code, scripts and data pipelines that leverage structured and unstructured data;
• Implement measures to address data privacy, security, compliance.
Mandatory Skills:
HiveQL, Scala, Java, Apache HBase, Python, Kafka Streams, Big Data, Apache Kafka, Hadoop

• Experience with designing data and analytics architectures in Microsoft Azure cloud;
• Experience with Big Data technologies like Spark, Hadoop, Hive, HBase, Kafka etc.;
• Fluency in several programming languages such as Python, Scala, Java, with the ability to pick up new languages and technologies quickly;
• Experience with data warehousing, data ingestion, and data profiling;
• Demonstrated teamwork, strong communication skills, and collaborative in complex engineering projects.
Nice-to-Have Skills:BS in computer science or related STEM field;
Languages:
English: B2 Upper Intermediate

About Luxoft

Luxoft is a high-end application outsourcing provider of choice and a trusted technology advisor to Global 2000 and medium-sized growth companies that apply compelling technologies to obtain leadership positions in their respective markets.
Luxoft today finds the Best talents, proposes career growth & employment benefits. Our teams are involved in high complicity & innovative projects for the Top leaders companies around the Globe.

Company website:
https://www.luxoft.ru

DOU company page:
https://jobs.dou.ua/companies/luxoft/

Job posted on 16 January 2021
2 views


Для отклика на эту и другие вакансии на Джинне войдите или зарегистрируйтесь.
Similar jobs

DevOps Engineer at Ciklum

Kyiv, remote

Python Developer at Infopulse

Kyiv, Kharkiv, Lviv, Odesa, Vinnitsia, Zhytomyr, remote


All jobs Python Kyiv    All jobs Luxoft
  Receive new jobs in Telegram