Web Scraping — An Easier Way

Learn how to scrape the web with ScrapingAnt and Python to simplify an often tedious task

Scraping Something — Photo by Mark Stebnicki from Pexels

It wasn’t too long into dealing with data before I realized I needed some information that wasn’t specifically offered via API or easily obtainable using simple functions. That led me down the tried and true path of Selenium and Chrome browser piggy-backing and headaches.

Oh the headaches!

As my specific needs increased, so to did those headaches. I had more sites to scrape and not everything played well. I needed a different solution to try and maintain my sanity.

Like most people in my situation, I turned to the results of various searches to see what was out there. I needed some pretty standard features to incorporate into my code:

1. It needed to be self-contained. There are a lot of half-baked systems out there with a whole slew of dependencies and requirements. I wanted something that was complete for my needs out of the box without adding a whole new technical challenge.
2. It needed to be quick to understand and integrate. I’m lazy and did not want to spend an inordinate amount of time trying to learn the intricacies of what was essentially a one-time utility.
3. It needed to be flexible enough to work with what I was doing. Since I was scraping several…