C# .NET Developer (Web Crawling Data Scraping) Offline

We are looking for a C# .NET Developer on behalf of our client, a global leader in AI-driven data analysis and web intelligence solutions. The company specializes in gathering and processing vast amounts of structured and unstructured data to provide actionable insights for Fortune 500 brands and enterprises across various industries, including eCommerce, finance, and cybersecurity.

As a C# .NET Developer, you will be responsible for developing and maintaining web crawlers and data extraction tools, ensuring high efficiency, reliability, and scalability. The ideal candidate has expertise in C# .NET, strong experience with XPath, HTML selectors, and JSON, and is proficient in HTTP requests and API testing using Postman. Knowledge of TypeScript is a strong advantage.

Key Responsibilities:

• Develop, maintain, and optimize web crawlers and data extraction tools using C# .NET.

• Use XPath and HTML selectors to efficiently parse and extract structured data from web pages.

• Work with JSON-based APIs and HTTP requests to collect and process web data.

• Handle challenges related to proxies, CAPTCHAs, and anti-bot mechanisms.

• Use Postman to test and debug API interactions.

• Optimize performance and scalability of web crawlers to handle large-scale data extraction.

• Collaborate with cross-functional teams to integrate extracted data into business workflows.

• Maintain code quality and version control using Git.

• Work with TypeScript for scripting, automation, or integrations when needed.

Qualifications:

• 3+ years of experience in C# .NET development.

• Strong experience in web crawling and data scraping using XPath and HTML selectors.

• Solid understanding of JSON processing and working with RESTful APIs.

• Proficiency in HTTP requests, including headers, authentication, and response handling.

• Experience using Postman for API testing and debugging.

• Familiarity with Git for version control and collaborative development.

• Knowledge of TypeScript for automation, scripting, or integration tasks.

• Strong problem-solving skills and ability to optimize crawlers for efficiency and accuracy.

Preferred Qualifications:

• Experience with proxy management and CAPTCHA bypass techniques.

• Understanding of cloud-based architectures for large-scale web scraping.

• Familiarity with headless browsers and automation tools.

If you are passionate about web crawling, data extraction, and working with large-scale datasets, we’d love to hear from you!