How to Extract eBay Product Reviews Data using Python?

Despite Amazon's presence in e-commerce marketplaces, eBay still maintains a significant presence in the online retail business. To obtain a competitive edge, brands selling online should keep an eye on prices on eBay.

Scraping review data from eBay on a large scale regularly is a difficult task for data analysts to solve. Here's an example of how to use Python to scrape eBay review data for mobile phone costs.

Consider a scenario in which you need to keep track of the price of a product, such as an eBay phone. You'll also want to see the variety of price options accessible on the cell phone you're tracking. Furthermore, you may want to examine the pricing of different mobile phones that you are considering.

In this blog, we'll scrape eBay for phone costs and compare reviews available on the eBay website.

Procedure to Scrape eBay Product Review Data

Here, you will learn the process of scraping eBay for items and reviews:

1. Selecting the Required Information

Identifying the target website is the first step in web scraping. It is the web page from which you must extract all of the necessary data.

We will scrape eBay for product review data, so we can just open the targeted eBay website page. All you have to do now is copy the URL from the browser after the website is finished with all of the product listings for that product. Your desired URL will be this one. The URL in our instance will be:

The "nkw" (new keyword) and "pgn" (page number) arguments in this URL should be noted. The search query is defined by these arguments in the URL. If we alter the "pgn" parameter to 2, it will load the second page of product listings for the Samsung Galaxy Note 8, and if we change "nkw" to iPhone X, eBay will look for iPhone X and display you the results.

2. Finalizing the Extraction Tags

You will need to comprehend the HTML layout of the target website page once scraping product data is completed. This is the most fundamental and crucial phase in web scraping, and it requires a basic understanding of HTML.

Do "inspect element" and activate the developer tools windows on the target web page, or just press CTRL+SHIFT+I. The script of the target website will appear in a new window. Because all of the products are stated as list components in our situation, we must collect all of these lists.

3. Placing the Extracted Data in Structured Format

We will only have to extract particular bits of the HTML text after we've created our extractors/identifiers. After that, we must arrange the data in a logical, organized manner. After this, you will need to make a table with the brand name in one column and pricing in another column.

4. Searching the Result

You will also illustrate the findings as you will undergo a comparison on two similar mobile phones. This is not an important step but it is part of the process of turning required data into necessary information. Boxplots will be used to examine the pattern of pricing for the Galaxy Note 8 and the iPhone 8.

Planning to extract eBay product reviews data?

Get a Quote!

Necessary Libraries and Installations

You'll need python, pip (python package installer), and the BeautifulSoup module in python to implement web scraping for this use case. To organize the acquired data into a structured way, you'll need to have the pandas and numpy libraries.

1. Installation of Python and PIP

You can set up Python and Pip in your system by following this blog link, depending on your operating system.

2. Installation of BeautifulSoup Library

apt-get install python-bs4 pip install beautifulsoup4

3. Installing Pandas and numpy

pip install pandas pip install numpy

How Python is Used to Scrape eBay Data?

Here, we will perform scraping for two products: iPhone 8 and another for Samsung Galaxy Note 8. For easy understanding, the implementation was performed for the two mobile phones. Two independent scraping tasks could be consolidated into one in a more streamlined version, but this isn't necessary right now.

Scraping Galaxy Note 8 from eBay Products

item_name = []
prices = []

for i in range(1,10):

    ebayUrl = "https://www.ebay.com/sch/i.html?_from=R40&_nkw=note+8&_sacat=0&_pgn="+str(i)
    r= requests.get(ebayUrl)
    data=r.text
    soup=BeautifulSoup(data)

    listings = soup.find_all('li', attrs={'class': 's-item'})

    for listing in listings:
        prod_name=" "
        prod_price = " "
        for name in listing.find_all('h3', attrs={'class':"s-item__title"}):
            if(str(name.find(text=True, recursive=False))!="None"):
                prod_name=str(name.find(text=True, recursive=False))
                item_name.append(prod_name)

        if(prod_name!=" "):
            price = listing.find('span', attrs={'class':"s-item__price"})
            prod_price = str(price.find(text=True, recursive=False))
            prod_price = int(sub(",","",prod_price.split("INR")[1].split(".")[0]))
            prices.append(prod_price)

from scipy import stats
import numpy as np

data_note_8 = pd.DataFrame({"Name":item_name, "Prices": prices})
data_note_8 = data_note_8.iloc[np.abs(stats.zscore(data_note_8["Prices"]))< 3,]

Extracted Data for Galaxy Note 8

Scraping eBay Product Data for iPhone 8

item_name = [] 
prices = [] 
for i in range(1,10): 
    ebayUrl = "https://www.ebay.com/sch/i.html?_from=R40&_nkw=iphone+8_sacat=0_pgn="+str(i) 
    r= requests.get(ebayUrl) 
    data=r.text 
    soup=BeautifulSoup(data) 
    listings = soup.find_all('li', attrs={'class': 's-item'}) 
    for listing in listings: 
        prod_name=" " 
        prod_price = " " 
        for name in listing.find_all('h3', attrs={'class':"s-item__title"}): 
            if(str(name.find(text=True, recursive=False))!="None"): 
                prod_name=str(name.find(text=True, recursive=False)) 
                item_name.append(prod_name) 
                if(prod_name!=" "): 
                    price = listing.find('span', attrs={'class':"s-item__price"}) 
                    prod_price = str(price.find(text=True, recursive=False)) 
                    prod_price = int(sub(",","",prod_price.split("INR")[1].split(".")[0])) 
                    prices.append(prod_price) from scipy 
                    import stats 
                    import numpy as np data_note_8 = pd.DataFrame({"Name":item_name, "Prices": prices}) 
                    data_note_8 = data_note_8.iloc[np.abs(stats.zscore(data_note_8["Prices"])) < 3,]

Extracted Data for iPhone 8

Checking the Product Pricing

Now you can put the scraped data into context. The boxplots will be used to search for the pricing distribution of mobile phones. The boxplot aids in seeing a numerical trend.

The median of the pricing data is shown by the green line. The median of the pricing data is shown by the green line. The box encompasses the data from the first through third quartiles, with a line at the median (Q2). The whiskers stretch beyond the box's boundaries to represent the data's range.

ReviewGators- Best Scraping Partners

There are various web scraping tools to assist you in scraping data. ReviewGators, on the other hand, can aid you if you require expert support with little technical knowledge. We have a very well and transparent approach for scraping data from the internet in real-time and delivering it in the format you require. ReviewGators has developed comprehensive solutions for the majority of these use-cases, ranging from support to the recruiting sector python/ to retail solutions.

For more Ecommerce Product Review Scraping, you can contact ReviewGators today!

Request for a quote!