This role requires expertise in Python, Selenium, Scrapy, and handling complex scraping challenges such as login sessions, JavaScript-heavy pages, CAPTCHA, and proxy management. You will work closely with our backend team to ensure scraped data feeds directly into our real-time marketplace and automation tools.

Key Responsibilities:

• Automate data extraction from structured and unstructured web content

• Handle authentication, session management, and headless browser environments

• Detect and resolve scraping issues, implement retry logic, error handling, and alerting systems

• Work with rotating proxies, static ISP proxies, and CAPTCHA-solving tools

• Collaborate with backend developers to integrate scraped data into APIs and databases

• Maintain scraping infrastructure, manage cron jobs, logs, and performance optimization

Required Skills & Qualifications:

• 5+ years of experience in Python

• 3+ years of experience with Selenium and Scrapy

• Proven experience scraping dynamic and JavaScript-rendered websites

• Knowledge of BeautifulSoup, Playwright (bonus), Puppeteer, or similar tools

• Familiarity with proxy management (residential, ISP, rotating)

• Strong debugging skills and experience with logging, error capture, and alerting

• Experience working with APIs and backend systems (RESTful APIs)

• Git/GitHub experience and ability to collaborate with a team

Nice to Have:

• Experience with Dockerized scrapers and distributed scraping infrastructure

• Familiarity with tools like 2Captcha or Anti-Captcha

• Understanding of terminal operations, shipping, or trucking logistics

What We Offer:

• Competitive salary (DOE)

• Flexible remote work

• Long-term position with growth opportunities

• Work on a product making real impact in the logistics industry