Python Software Engineer (Big Data project) (offline)

About Product:
Data Platform that delivers the performance, simplicity, concurrency, and affordability for data collecting, rapid analytics, and extracting data-driven insights for business users.
Lambda architecture, designed to handle massive quantities of data by taking advantage of both batch and stream-processing methods. This approach to architecture attempts to balance latency, throughput, and fault-tolerance by using batch processing to provide comprehensive and accurate views of batch data, while simultaneously using real-time stream processing to provide views of online data.
In many organizations, data consumption processes for timely reporting or crucial analytical requirements are hindered by delays in query execution and information presentation. The main challenge is to minimize decision delay with the help of flexible solutions with open and simplified architecture and high performance based on future oriented technologies using AI models and forecasting analytics.

Stack:
Kafka (KSQL, Streams), Kafka Connect, Google Cloud as Cloud Hadoop, Google Dataflow (Apache Beam), BigQuery, SQL, Kubernetes, Java/Python

In numbers:
Up to 1 PB of data from 500+ partners in long-term storage
30+ M new records daily
Latency for insight based on data delivery time 5 sec

Skills & Experience:
— 2+ years of experience in Python;
— Experience with designing, implementing RESTful API’s (Aiohttp, FastAPI);
— Experience working on relational databases (PostgreSQL, Microsoft SQL Server);
— Experience working on NoSQL databases (Elasticsearch, Redis);
— Knowledge of performance tuning of ETL jobs, SQL’s, Partitioning, Indexing;
— Experience with Cloud Services (preference to Google Cloud Platform);
— Experience working on Linux, Kubernetes, Docker;
— Experience in incorporating and utilizing BDD / TDD / Unit testing;
— Exceptional problem solving, technical and data analysis skills;
— Extensive knowledge of best practices in software design and design patterns;
— Strong Computer Science fundamentals.

Will be a plus:
— Knowledge in Data Warehouse concepts;
— Experience with Data visualization/Business Intelligence tools;
— Some experience in any Big Data technologies (MapReduce, Spark, Presto, Drill...);
— Some experience in any Graph databases (Neo4j, AgensGraph);
— Some experience with IoC Tools (Terraform, Ansible);
— Some experience working with Workflow tools (such as Apache Airflow);
— Some experience with near real-time data processing.

Challenges:
— Development and support of Python applications;
— Designing and implementing REST-based APIs;
— Participate in complex project design teams;
— Work closely with data analysts, data scientists, IT, and business partners;
— Define, design, and implement different data ingestion patterns based on business needs and requirements

What can we offer you in addition to the invaluable experience and the innovative project?
A team of experts in the field of Data Science & Machine Learning, who are open to sharing knowledge and experience and often act as speakers at technical conferences in Ukraine and abroad.
Support in training and development, starting from free English courses in the office and developing IDP to paying for Ukrainian and foreign conferences and other events.
Well, of course:
— Health insurance
— Paid vacation and sick leave
— We keep work-life balance
— Flexible working hours and the possibility to work remotely
— Relocation bonus
— Silent office near Lukyanivska metro station

About SoftConstruct

SoftConstruct conducts basic and applied research in four key areas: data science, computer vision, big data, real-time processing. Our experience is extremely wide: from working with complex computer and engineering systems, programming for data science — to developing and putting into practice innovative solutions in the field of sports, eSports, and security. We use various technologies, knowledge of experts from multiply areas and unique tools to find new approaches and original solutions to problems.

Company website:
http://www.softconstruct.io/

DOU company page:
https://jobs.dou.ua/companies/softconstruct-ukraine/