Senior Data Engineer (offline)

About Project:
Developing the only payroll, benefits, and talent Management Platform. It's more than Management System software: it's the compliance and benefits consulting you need. The Product mission is to empower everyone to make data-driven decisions about people. From HR to the C-Suite to employees, we believe that accessible data has the power to make all of us more strategic and impactful.

MS system consists of the following core modules:
Payroll
Benefits administration
Performance management
Human capital management

Overall Project Technologies:
1st stack: Ruby with Redux, React on Front End; microservices in Go, Python, .NET Core
2nd stack: ASP.NET, SQLServer, PostgreSQL
AWS as a default hosting platform
Web, desktop, mobile

Skills:
3-5 years of experience building data frameworks for unit testing, data lineage tracking, and automation;
Fluency in Scala and at least one other server-side programming languages (e.g. Python, Java, Go);
Proficient on Apache Spark;
Familiarity with streaming technologies (e.g., Kafka, Kinesis, Flink).

Responsibilities:
-The candidate should be technically strong and ready to provide guidance to other team members, not only the coding;
-Set up testing, CI/CD, code conventions;
-No emphasis on DevOps side — leaning on hosted solutions;
-Streaming + batching (98% batching). In long term perspective — more streaming;
-Sometimes team has performance issues. Possible reason — no much experience with Spark;
-Datasets are very diverse; processing is usually complex — so it’s not a routine etl day to day, but more like solving new challenge each time;
-Considering Apache Flink for stream processing (not implemented right now);
-Joining ’report & analytics development’ team — ETL, data pipelines (though will not work with reports themselves).

About Data Science UA

We understand that you need more than just a search engine to find IT and technical professionals or to progress your own career, that is why we established Data Science UA.
One place connecting business and developers.

Company website:
https://data-science-ua.com/

DOU company page:
https://jobs.dou.ua/companies/data-science-ua/