Data Engineering Lead

About the project:
Our client is an Israeli fintech company, a pioneering firm in AI-powered financial analysis. They specialize in transforming complex financial data into actionable insights, aiding investors globally. Our technology covers an extensive range of publicly traded stocks, empowering various financial sectors.

Stage: new development; the team is growing.
Client team: tech team is around 15 people, Israel and Ukraine.
English: upper-intermediate.

Requirements:
— 5+ years of experience in software development and data engineering;
— 2+ years of experience in leadership position;
— Strong knowledge of Spark, practical experience of Spark jobs performance optimization;
— Demonstrated expertise in applying SOLID principles and Object-Oriented Design (OOD);
— Proficiency in PySpark, with in-depth knowledge in at least several of the following areas: Window functions, Broadcast join, Sort&merge join, Watermark, UDF, Lazy computation, Partition Skew;
— Experience with other big data technologies and familiarity with cloud environments like AWS, Azure, or Google Cloud are highly desirable;
— Excellent problem-solving skills and a proactive approach to overcoming challenges;
— Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field;
— Strong communication skills, capable of effectively articulating technical concepts and collaborating with cross-functional teams.

Must have:
— Practical experience with performance optimization of the Spark job;
— Knowing what is SOLID and knowledge of OOD;
— Familiar, ideally, with all PySpark topics (minimum 4):
* Window
* Broadcast join
* Sort&merge join
* Watermark
* UDF
* Lazy computation
* Partition Skew.

Responsibilities:
— Lead the optimization and scaling of data processing architectures using Spark to enhance system performance and capability.
— Design and develop robust data solutions that are integral to financial analytics and intelligence platforms.
— Mentor and lead a team of data engineers, promoting best practices in data processing and usage of PySpark.
— Collaborate with product managers and other tech leads to innovate and integrate advanced data technologies into our services.
— Ensure the integrity, accuracy, and timeliness of processed data, maintaining the highest standards of quality and data governance.
— Keep abreast of the latest industry trends in big data technologies, implementing advancements that can benefit the company.

Why choose us?
— “Family and Friends”. We are no longer a start-up, but we still, have a family atmosphere in our supportive and spirited team, who are all working together on the same goal.
— “Just break down all barriers and find a better way”. Every day you’ll meet with interesting and challenging (international) projects that cover industries from commercial aviation to fintech (different technologies, different products).
— “Hungry for learning”. You will get a lot of chances for career advancement and the development of new skills, mentorship opportunities, or learning from more experienced colleagues.

Benefits from 8allocate:
— You’ll work with a supportive and spirited team of professionals.
— Corporate events, holidays, and team buildings for your joy.
— Training and development: we have a huge library (about 500 books!) and a budget for your professional development.
— Sport or massage compensation according to the general policy.
— Compensation for psychological support or medical insurance compensation according to the general policy.
— People-oriented management without bureaucracy.
— Paid vacation and sick leaves.

About 8allocate

8allocate is a provider of software development solutions to companies all over the globe, from North America to the EU. We deliver market-ready solutions for FinTech/InsurTech, eCommerce, AI, Web, Mobile, and other domains.

Company website:
https://8allocate.com/

DOU company page:
https://jobs.dou.ua/companies/8allocate/