Middle Data Engineer Offline

Middle Data Engineer

The Data Engineer will be responsible for developing, designing, and maintaining end-to-end optimized, scalable Big Data pipelines for products and applications. In this role, you will collaborate closely with team leads across various departments and receive support from peers and experts across multiple fields.

Responsibilities:

 • Follow and promote best practices and design principles for Big Data ETL jobs

 • Help in technological decision-making for the business’s future data management and analysis needs by conducting POCs

 • Monitor and troubleshoot performance issues on data warehouse/lakehouse systems

 • Provide day-to-day support of data warehouse management

 • Assist in improving data organization and accuracy

 • Collaborate with data analysts, scientists, and engineers to ensure best practices in terms of technology, coding, data processing, and storage technologies

 • Ensure that all deliverables adhere to our world-class standards

Skills:

 • 3+ years of overall experience in Data Warehouse development and database design

 • Deep understanding of distributed computing principles

 • Experience with AWS cloud platform, and big data platforms like EMR, Databricks, EC2, S3, Redshift

 • Experience with Spark, PySpark, Hive, Yarn, etc.

 • Experience in SQL and NoSQL databases, as well as experience with data modeling and schema design

 • Proficiency in programming languages such as Scala or Python for implementing data processing algorithms and workflows

 • Experience with Presto and Kafka is a plus

 • Experience with DevOps practices and tools for automating deployment, monitoring, and management of big data applications is a plus

 • Excellent communication, analytical, and problem-solving skills

 • Knowledge of scalable service architecture

 • Experience in scalable data processing jobs on high-volume data

 • Self-starter, proactive, and able to work to deadlines