Data Engineer/Senior Spark Developer Offline

Requirements

- 5+ years of hands on experience with Apache

Spark(Pyspark/SparkR/sparklyr)

- Experience using Spark in the Databricks Ecosystem

- Deep knowledge in the design and implementation of Distributed File

Systems, Data Warehousing and Streaming technologies, including data

architecture patterns, using Azure

- Experience in deploying and managing Spark ML models at scale

- Excellent communication and presentation skills

- Passionate about learning and experimenting with new technologies

- English - Upper Intermediate (daily conversation with customer side)

Responsibilities

- Provide expertise and guidance in the architectural design and implementation of Spark projects, from Data Engineering using Cloud Storage to Machine Learning/Deep Learning pipelines in Data Science and AI

- Implementation and delivery of globally scalable and supportable Spark workloads to a production grade level of quality in an Agile Devops working environment

- Engage and consult with end users to understand real world problems worth solving and relating how distributed cloud computing can solve them

- Provide technical leadership and impart Spark knowledge to project team members and others in the Data Science Community