Big Data Engineer, Multinational Financial Services Offline

About the vacancy

DataArt's specialists participate in data governance and data improvement processes for one of the top three UK insurance companies. Our team helps the client in finding and clearing issues with the data.

The Big Data engineer will be responsible for creating applications and solutions that will manage data flow. The specialist will also participate in the creation of new data pipelines, identify existing data gaps, and offer automated solutions for providing advanced analytical capabilities and enriching data with application APIs.

Must have

- Strong knowledge of Big Data general concepts

- Hands-on experience with the Hadoop ecosystem (Spark, MapReduce, Hive, etc.)

- Knowledge of standard data structures and algorithms

- Experience handling large data using streaming

- Experience with tools like Hive, Spark, HBase, Sqoop, Impala, Kafka, Flume, Oozie, MapReduce, Indexers, etc.

- Hands-on programming experience in Java, Scala or Python

- Experience with end-to-end design and build processes for Near-Real-Time and Batch Data Pipelines

- Strong experience with SQL and Data modeling

- Analytical thinking

- Spoken English

Would be a plus

- Experience working in an agile environment and understanding of various phases of the software development life cycle

- Skills working with the Cloudera platform

- Skills using source code and version control systems like SVN, Git, etc.

- Knowledge of Hadoop architecture components

- Ability to work with minimal supervision

- Understand the current state and identify and recommend possible solutions

- Natural curiosity and a desire to explore solutions in a highly focused team

Learn more about our policy of equal opportunities in employment