Python Web Scraping Engineer

We are looking for an experienced Python Web Scraping Engineer to build and maintain scalable data extraction systems. You will develop robust web scrapers, automate data collection workflows, and help create market intelligence solutions by aggregating and analyzing large volumes of web data.

This is a part-time position. The initial phase of the project focuses on building a scalable web scraping platform and is expected to take up to 3 weeks. Once the platform is completed, the role will transition into ongoing maintenance, optimization, and feature improvements, requiring an average commitment of approximately 2 days per week (or equivalent part-time hours spread across the week).

Responsibilities

Design, develop, and maintain web scraping solutions using Python.
Build a scalable web scraping platform from the ground up.
Extract, process, and validate data from multiple web sources.
Handle website protection mechanisms and ensure scraper reliability.
Integrate and manage residential proxy solutions.
Build automated data pipelines and load data into BigQuery.
Deploy and monitor scraping services using Google Cloud Run.
Optimize scraper performance, scalability, and data quality.
Maintain and enhance the platform after the initial development phase.
Collaborate with stakeholders to define data collection requirements.
 

Requirements

4+ years of commercial experience with Python.
Proven experience building and maintaining web scrapers.
Strong understanding of HTTP, HTML, CSS Selectors, XPath, and modern web technologies.
Experience with scraping frameworks and tools such as Scrapy, Playwright, Selenium, BeautifulSoup, or similar.
Experience working with proxy networks and IP rotation.
Solid SQL skills and experience with BigQuery or other data warehouses.
Experience with cloud platforms, preferably Google Cloud Platform (GCP).
Ability to troubleshoot, optimize, and maintain large-scale scraping operations.
Intermediate or higher English level.
 

Nice to Have
 

Experience with Scrapling.
Experience deploying applications with Google Cloud Run.
Docker and containerization experience.
Experience building ETL/data pipeline solutions.
Familiarity with data analytics and market intelligence projects.
 

What We Offer
 

Competitive compensation.
Part-time remote contract.
Flexible working hours.
An initial project lasting up to 3 weeks to build the platform, followed by long-term maintenance and optimization work with an average workload of approximately 2 days per week (or equivalent part-time hours).
Opportunity to work on large-scale data collection and analytics projects.
Direct impact on technical decisions and system architecture.