Web Scraping Mastery: 100 Projects with Scrapy,BS4 and more
100 Real-World Projects with Scrapy, Selenium,bs4,requests,xpath,apis and more

What you will learn

100 web scraping projects (from beginner to advanced projects)

How to download python libraries/python

How to use scrapy,beautifulsoup and other data scraping techniques

How to setup a virtual enviroment

How to scrape sites like tiktok,instagram ,amazon,jumia

Handle pagination with any scraping tool

Scrape infinite scroll websites


Welcome to the Web Scraping Mastery: 100 Projects with Scrapy, Selenium, and More course!

In this project-intensive course, you will embark on a 100 Day journey to master the art of web scraping using Python and its powerful libraries. Each day, you will dive into a new topic and work on exciting projects that will help you develop a solid foundation in web scraping techniques.

Here’s a sneak peek at what you’ll learn during this course:

Day 1: Introduction to Web Scraping and the Requests Library

  • Make GET and POST requests using the requests library.
  • Inspect the response object and access its content and headers.

Day 2: Introduction to BeautifulSoup and the Parse Tree

  • Create a BeautifulSoup object from HTML or XML content.
  • Navigate the parse tree using tags, attributes, and methods.
  • Access the text and values of the elements.

Day 3: Web Scraping Basics

  • Scrape data from a simple static website using requests and BeautifulSoup.
  • Extract the title, headings, paragraphs, links, images, and tables from the website.

Day 4: Web Scraping with CSS Selectors

  • Use CSS selectors to find elements in the parse tree.
  • Apply CSS selectors using the select() and select_one() methods of BeautifulSoup.

Day 5: Web Scraping with Regular Expressions

Get Instant Notification of New Courses on our Telegram channel.

  • Search for patterns in text or attributes using regular expressions.
  • Apply regular expressions using the re module and the find_all() method of BeautifulSoup.

Day 6: Web Scraping with Filters

  • Find elements based on their properties or functions using filters.
  • Apply filters using the find_all() and find() methods of BeautifulSoup.

Day 7: Web Scraping with Lambda Functions

  • Create custom filters for finding elements using lambda functions.
  • Apply lambda functions using the find_all() and find() methods of BeautifulSoup.

Day 8: Web Scraping with Pagination

  • Scrape data from multiple pages of a website using pagination.
  • Follow links or parameters indicating the next page using requests and BeautifulSoup.

Day 9: Web Scraping with JavaScript

  • Scrape data from websites that use JavaScript.To

Day 100:

Throughout this course, you’ll work on various projects that involve scraping data from well-known websites such as Amazon Audible, Twitter, Google, Football Stats, Worldometers, and Movie pages. These projects will provide you with hands-on experience and help you build a portfolio of web scraping projects that you can showcase to potential employers or clients.

Let’s get started.



Introduction to Web Scraping

introduction to webscraping|REQUESTS
How to Download Python
what is web scraping?

web scraping with beautifulsoup

Introduction to BeautifulSoup
webscraping with beauitifulsoup

web scraping with css selectors

webscraping with css selectors
web scraping with css selectors