PySpark to $4000 (offline)

PySpark Developer (Architecture and Refactoring). Our goal is to create scalable, maintainable, and production-ready software solutions that can be integrated seamlessly into various organizational SDKs. We are seeking Software Engineer to advise on architecture decisions and refactor our existing codebase. Extensive use of PySpark. 

Key Responsibilities: Refactor and optimize the existing codebase, primarily written in PySpark. • ⁠Collaborate with data scientist to understand functional requirements and translate them into technical specifications. • ⁠Develop and maintain software documentation and technical specifications. • ⁠Implement best practices in software development, including code reviews, unit testing, and continuous integration. • ⁠Troubleshoot, debug, and upgrade existing software as necessary.  

Required Skills and Qualifications: Proven experience with a strong focus on Python and PySpark. 

Extensive experience in refactoring codebases for better performance, maintainability, and scalability. Strong understanding of software architecture and design principles. Solid understanding of software development best practices, including version control, testing, and documentation. Experience with building and integrating software packages and libraries. Excellent problem-solving skills and the ability to work independently as well as part of a team.

Preferred Qualifications: Experience with Apache Spark. Experience with container orchestration systems like Kubernetes. 

Familiarity with time series data analysis and machine learning. Knowledge of cloud platforms and services (e.g., AWS, Azure). Experience with continuous integration/continuous deployment (CI/CD) pipelines.

What We Offer: Flexible work hours and remote work options.

English: lower intermediate