Web Scraping Expert (Scrapy & Python) WFH

by Aimleap | Jun 6, 2024

Web Scraping Expert (Scrapy & Python) WFH

by Aimleap | Jun 6, 2024

Aimleap

India (Remote)

Posted on Jun 06, 2024

AIMLEAP is Hiring:

Web Scraping Expert (Scrapy & Python) WFH

Experience: 3 to 5 years

Location: Work from Home/Bangalore/India

Salary: 3 Lacs to 7 Lacs INR PA

No of positions: 4

Employment time: Full time

Educational qualification: CS/IT/Engineering

Industry: IT-ITES

Notice Period: Immediate

Job Description:

Experienced in developing and maintaining robust web scraping pipelines using Scrapy, a high-performance Python framework

Proficient in Implementing efficient crawling strategies to handle pagination, dynamic content, and complex website structures

Exposure in data cleaning and manipulation libraries (Pandas, NumPy) and data extraction methods (CSS selectors, XPath)

RESPONSIBILITIES:

Design, develop, and maintain web scraping projects using Scrapy, including:

Crawling websites to extract specific data points

Implementing efficient crawling strategies to handle pagination, dynamic content, and complex website structures

Employing data extraction techniques with CSS selectors and XPath

Processing and cleaning scraped data using Python libraries (e.g., Pandas, NumPy)

Storing extracted data in appropriate formats (e.g., CSV, JSON, databases)

Collaborate with data engineers and analysts to identify data needs and define scraping requirements

Write well-documented, maintainable, and efficient Scrapy code

Integrate scraping pipelines with other Python frameworks and tools as needed

Stay up-to-date with the latest web scraping trends and best practices

Troubleshoot and address challenges related to rate limiting, authentication, and website changes

Implement caching strategies for efficient data retrieval and reduced website load

Consider potential ethical and legal implications of web scraping

QUALIFICATIONS:

Proven experience with web scraping techniques and tools (Scrapy preferred)

Strong proficiency in Python programming

Knowledge of HTML and CSS for navigating website structures

Experience with data extraction methods (CSS selectors, XPath)

Familiarity with data cleaning and manipulation libraries (Pandas, NumPy)

Understanding of distributed crawling concepts for large-scale data extraction

Excellent problem-solving and analytical skills

Ability to work independently and as part of a team

Strong communication and documentation skills

About Us:

AIMLEAP is an ISO 9001:2015 and ISO/IEC 27001:2013 certified global technology consulting and service provider offering Digital IT, AI-augmented Data Solutions, Automation, and Research & Analytics Services.

AIMLEAP has been recognized as The Great Place to Work. With focus on AI and automation-first approach, our services include end-to-end IT application management, Mobile App Development, Data Management, Data Mining Services, Web Data Scraping, Self-serving BI reporting solutions, Digital Marketing, and Analytics solutions.

We started in 2012 and successfully delivered projects in IT & digital transformation, automation driven data solutions, and digital marketing for more than 750 fast-growing companies in the USA, Europe, New Zealand, Australia, Canada; and more.

An ISO 9001:2015 and ISO/IEC 27001:2013 certified

Served 750+ customers

12+ Years of industry experience

98% Client Retention

Great Place to Work Certified

Global Delivery Centers in the USA, Canada, India & Australia. .