Data Engineer

Role Overview:

 

As a Data Engineer at QuintaGroup, you will design and optimize data pipelines within the AWS ecosystem for a US-based B2B marketplace platform. The platform simplifies and accelerates business operations by providing seamless data solutions and advanced analytics. You’ll collaborate with data scientists, analysts, and cross-functional teams to deliver innovative results. 

 

Key Responsibilities:

 

• Develop, implement, and optimize data pipelines using PySpark in AWS environments.

• Utilize AWS services such as S3, Glue, Lambda, EMR to create scalable and efficient data solutions.

• Enhance PySpark workflows for performance, reliability, and cost-effectiveness.

• Maintain data quality through rigorous testing and monitoring processes.

• Apply data governance, security, and compliance best practices.

• Document workflows, processes, and designs to support team collaboration and maintenance.

 

Requirements:

 

• 3+ years of experience in data engineering, with a focus on PySpark.

• Strong experience with AWS services.

• Proficiency in Python and related frameworks or libraries.

• Solid understanding of distributed computing and Apache Spark.

• Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation) is a plus.

• Strong analytical and problem-solving skills with attention to detail.

• Excellent communication skills and ability to work in dynamic, team-oriented environments.

• Upper-Intermediate level of English.

 

Tech Stack:

 

Programming Languages: Python (with Pandas and PySpark).

AWS Services: S3, Glue (Glue Data Catalog, Glue Crawler, Glue Jobs with PySpark), Lambda, ECS, Athena, Aurora (RDS), AppConfig, API Gateway, Step Functions, Quicksight, EventBridge.

Frameworks and Tools: Django, AppSync, DynamoDB, Auth0.

Infrastructure: Terraform for Infrastructure-as-Code.

 

We Offer:

 

• Flexible working format: remote, office-based, or hybrid.

• Competitive salary and compensation package.

• Personalized career growth and mentorship programs.

• Professional development tools, including tech talks and training sessions.

• Access to active tech communities and regular knowledge sharing.

• Education reimbursement opportunities.

• Memorable anniversary gifts.

• Corporate events and team-building activities.

• Location-specific benefits.

 

Ready to take on this exciting challenge? Join QuintaGroup and become part of a team shaping the future of data engineering!