Senior\Lead BigData Engineer (offline)

About the vacancy:

Our client is a global music and entertainment conglomerate headquartered in the US with offices in over 40 countries. The company has some of the world’s biggest music stars on its roster and provides innovative analytical tools both to their signed artists and to independent musicians.

The client is actively integrating automation, data science, and other modern technologies into its operations and plans to expand the team in various projects, such as processing tools for new artists and tracks, license and rights management, marketing, and A/R activity.

Many projects and applications are built around the concept of a unified data storage that is capable of hosting all sales and consumer data in one place and providing a wide variety of customizable reports to stakeholders at all levels of the company. That helps develop products that best fit the client’s business goals.

Technology stack: Spark 3.0+ + Scala; OOP, DI (Guice); AWS (Lambda, S3, Secrets Manager); GCP (BigTable); Databricks; DataDog.

Responsibilities:
Elicit business needs and requirements, determine the project's scope, team, and timeline
Lead project implementation from ideation to completion
Serve as a top-level escalation point for technical and process concerns
Select and integrate any Big Data tools and frameworks required to provide the requested capabilities
Implement ETL process
Monitor performance and advice on any necessary infrastructure changes
Establish Agile delivery processes in a team

Must have:
Experience architecting complex BigData solutions
Expertise with Spark (+ Scala)
Hands-on experience with processing frameworks, or other Big Data technology areas: data ingestion, consolidation, streaming, or batching
Expertise in the RDBMS, SQL and NoSQL engines
Hands-on experience waking with Spark
Cloud practice with AWS (Lambda, S3)
Spoken English

Would be a plus:
Experience leading a team of 2+ Data Engineers
Practice implementing Data Lakes, Data Warehousing or analytics systems
GCP (BigTable)

*What we offer:*

— Paid sick leaves and mental health day;
— Medical insurance;
— Life insurance;
— Competitive salary and regular perfomance appraisal;
— Many other nice things :)