Middle/Senior Big Data Engineer (Visual Platform) (offline)

We are searching for a Big Data professional who knows how to organize raw data masterly and to build an accessible and reliable platform for this purpose.
You are likely to love this role if you are:
- full of ideas, inspired to create robust and beautiful solutions
- excited about dynamic projects with challenging tasks
- initiative at choosing technologies and approaches in making a solution from the start

Project
Our client is a UK-based web development company that has an ambitious goal to make Internet more human-friendly by improving visual representation of data within the listed structures in the Web. By building a data visualization platform that will process big volumes of user interaction data, the company improves user experience and usability of existing client sites.

Our development team works on an efficient visual platform that dramatically improves browsing the web on any device without any plugins or responsive design. The project is in the active development phase, and the main technology stack is based on the WebGL/threejs technology on the frontend and Ruby on the backend.
As a Big Data Engineer, you will be a part of the data engineering team that will design and implement the Data Platform from scratch. Data Platform’s goal is to provide comprehensive analytics based on user interaction data from a client’s web sites like user heatmap click events, user navigation events, conversion events and other system events to improve visualization experience of the web sites.
The customer is extremely open to the professional opinion of the team, so you will have a chance to initiate changes, choose technologies and approaches.

Responsibilities
Design and build Data Lake repository to store raw data ingested from different sources
Design Data Warehouse structures and relations using Snowflake
Implement ingestion pipelines to collect data from different sources using cloud-based solutions
Create data processing flows using Spark and cloud services to transform and store raw information from data sources
Apply Data Governance practices to meet regulatory requirements for data security and data quality

Requirements:
Solid development experience with Java, Scala, or Python
Solid experience with Spark
Experience with designing or developing Data Lakes and Data Warehouses
Experience with NoSQL and RDBMS technologies
Experience with at least one of cloud providers like AWS, Google Cloud, or Azure
Would be a plus:
Experience with designing or developing Data Lakes and Data Warehouses on AWS
Designing and implementing CI/CD practices
Experience with key-value databases
Experience with Grafana
Experience with AWS Big Data services like EMR, Glue
Experience with serverless solutions like AWS Lambda

About Sigma Software

Sigma Software is a place where Nordic Traditions meet Ukrainian Spirit to create Superior Software. We combine the best practices and approaches from Swedish and Ukrainian cultures. Taking high demand for quality, minimal hierarchy, freedom of decision-making, and attention to every opinion from Swedes, as a Ukrainian company we demonstrate flexibility and dedication to every project and every customer.

We are IT consulting and software product company with development offices in Ukraine, Sweden, the USA, Canada, Poland, and Australia. We deliver smartest solutions to our customers in the areas of government, telecommunications, advertising, automotive, gaming, and others. Being a part of Sigma Group, one of the largest Nordic IT corporations, we are a global player with more than 5000 employees in 12 countries and over 1500 in Ukraine.

We work with startups, software houses, and enterprises providing the products and services that suit our clients the most. Company`s R&D centers are mastering the trending technologies and directions: Infotainment, AR/VR, Blockchain, Machine Learning, Data Science, Artificial Intelligence, and others.

Company website:
https://career.sigma.software/

DOU company page:
https://jobs.dou.ua/companies/sigma-software/