Web Scraping

Getting the Data I Need

Overview

With graphics cards being a very important part of a gaming PC, it is important to make sure that you buy a powerful one for a good price. To help people discover which graphics card is right for them, I added a command to my Discord bot that shows people how powerful the Radeon and GeForce cards are compared to one another to help them pick a card.

Data Source

When I was deciding what I would be buying for gaming, I used a great chart by Logical Increments. I thought the cards were well-organized, so I thought that it would be the perfect site to scrape. However, it wasn't super simple to scrape. The actual data was fetched with JavaScript, so just sending a GET request with Python requests wasn't going to have the data I needed. It wasn't a huge barrier, so I opened the console and inspected the network requests being sent when the page loaded. From here, it wasn't too hard to find the URL that I needed to GET.

Command output
Sample Output from the Finished Command

Connecting Data to Discord

Once I was able to get data into a Python script, connecting the data to Discord wasn't too hard. I always use Discord.py when making bots, so I just added a command to my main bot that sent a Discord embed to the channel to where the command was sent.


Source Code

Below are two GitHub gists containing the scraping file itself and a sample output file.