Scraping Developer

As a Scraping Developer, your role will focus on implementing methods to collect the data required (client-side Javascript, API, and website scraping).

Responsibilities:

• Reverse-engineering APIs to consume with client-side Javascript
• Design the data flow process so that the data collected is valid
• Maintain data collection API
• Maintain and improve the current system
• Maintain the performance of existing and new implementations
• Maintain the quality of the data gathered
• Implement both technical and business metrics
• Perform code reviews
• Perform research on collecting data from new sources

Required:

• Experience with data scraping and building APIs
• Experience in Javascript / NodeJS / Typescript / Webpack
• Database knowledge to store the results (SQL/NoSQL)
• Good troubleshooting and debugging skills
• Knowledge of browser security policies (CSP, CORS, etc.)

Other required qualifications:

• 3–4 years of experience in a similar role
• Ability to write modular and testable code
• Experience with Unit tests
• Experience with Linux
• Communicative spoken and written English

We’d also love to see, but don’t require experience with:

• Docker
• Redis
• NestJS
• Cloud AWS/Azure
• Kubernetes

The hiring process is executed in collaboration with the Red Sky team.