{"id":10825,"date":"2024-08-28T09:02:46","date_gmt":"2024-08-28T09:02:46","guid":{"rendered":"https:\/\/scrapingdog.com\/?p=10825"},"modified":"2025-08-25T10:14:40","modified_gmt":"2025-08-25T10:14:40","slug":"scrape-ebay","status":"publish","type":"post","link":"https:\/\/www.scrapingdog.com\/blog\/scrape-ebay\/","title":{"rendered":"Scrape eBay Using Python to Extract Product Details"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"10825\" class=\"elementor elementor-10825\" data-elementor-post-type=\"post\">\n\t\t\t\t<div class=\"elementor-element elementor-element-d08f479 e-con-full e-flex e-con e-parent\" data-id=\"d08f479\" data-element_type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6ce0936 elementor-widget elementor-widget-html\" data-id=\"6ce0936\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<!-- Gutenberg \u201cCustom HTML\u201d block -->\r\n<div style=\"\r\n  background:#d9f4e5;\r\n  border-left:4px solid #1d9b6c;\r\n  padding:18px 24px;\r\n  margin:24px 0;\r\n  border-radius:6px;\r\n  font-family:'Montserrat',sans-serif;\r\n  font-size:18px;\r\n  line-height:1.65;\r\n  color:#1a1a1a;\">\r\n  <p style=\"margin:0 0 8px 0; font-weight:600;\">TL;DR<\/p>\r\n\r\n  <ul style=\"margin:0; padding-left:20px;\">\r\n    <li>Python tutorial to scrape an eBay product with <code>requests<\/code> + <code>BeautifulSoup<\/code>.<\/li>\r\n    <li>Extracts title, rating, price, list price \/ discount, shipping, and image URLs; full code included.<\/li>\r\n    <li>Fine for demos; at scale eBay blocks , use <strong>Scrapingdog<\/strong> to fetch reliably; 1,000 free credits; &gt;99% success.<\/li>\r\n    <li>Useful for price tracking and market analysis.<\/li>\r\n  <\/ul>\r\n<\/div>\r\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d146e4f font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"d146e4f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>eBay scraper can collect a large amount of data about sellers, their offer price, their ratings, etc. This data can help you analyze the dynamic market or if you are a seller itself on eBay then you can\u00a0monitor your own competitor by\u00a0<a href=\"https:\/\/www.scrapingdog.com\/blog\/scrape-prices\/\" data-type=\"URL\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/scrape-prices\/\">scraping prices<\/a>. Over time once you have a large data set you can identify trends in product popularity, seller behavior, and buyer preferences, which can help you stay ahead of the curve in your industry.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-730f74a font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"730f74a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>In this article, we are going to extract product information from eBay by creating our own eBay scraper. Through this, we will learn how we can create a seamless data pipeline from eBay by extracting crucial information for target products.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8c02d09 elementor-widget elementor-widget-heading\" data-id=\"8c02d09\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Setting up the prerequisites for eBay scraping<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-929d80e font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"929d80e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>I hope you have already installed Python 3.x on your machine. If not then you can download it from\u00a0<a href=\"https:\/\/www.python.org\/downloads\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">here<\/a>. Apart from this, we will require two III-party libraries of Python.<\/p><ul class=\"wp-block-list\"><li><strong>Requests<\/strong>\u2013 Using this library we will make an HTTP connection with the Amazon page. This library will help us to\u00a0<a href=\"https:\/\/www.scrapingdog.com\/blog\/how-to-extract-data-from-website\/\" data-type=\"URL\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/how-to-extract-data-from-website\/\">extract the raw HTML from the target page<\/a>.<\/li><li><strong>BeautifulSoup<\/strong>\u2013 This is a powerful data parsing library. Using this we will extract necessary data out of the raw HTML we get using the requests library.<\/li><\/ul><p>Before we install these libraries we will have to create a dedicated folder for our project.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-622a7bb elementor-widget elementor-widget-code-highlight\" data-id=\"622a7bb\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp>mkdir ebayscraper<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c3e1300 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"c3e1300\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Now, we will have to install the above two libraries in this folder. Here is how you can do it.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d621f99 elementor-widget elementor-widget-code-highlight\" data-id=\"d621f99\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp>pip install beautifulsoup4\r\npip install requests<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0f02fdd font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"0f02fdd\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Now, you can create a Python file by any name you wish. This will be the main file where we will keep our code. I am naming it\u00a0<code>ebay.py<\/code>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-aeaa335 elementor-widget elementor-widget-heading\" data-id=\"aeaa335\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Downloading raw data from ebay.com<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fc353d2 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"fc353d2\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Before we dive in let\u2019s test type a small Python script to proof test it. This is just to determine that our code will be able to scrape eBay without getting blocked.<\/p><p>Our <a href=\"https:\/\/www.ebay.com\/itm\/182905307044\" target=\"_blank\" rel=\"noopener\">target page is\u00a0this<\/a>. It is a wristwatch.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-89fc440 elementor-widget elementor-widget-code-highlight\" data-id=\"89fc440\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp>import requests\r\nfrom bs4 import BeautifulSoup\r\n\r\ntarget_url=\"https:\/\/www.ebay.com\/itm\/182905307044\"\r\n\r\nresp = requests.get(target_url)\r\n\r\nprint(resp.status_code)<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0e5d1b3 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"0e5d1b3\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"f441\">Well, the code is very clear and concise. But let me explain to you line by line.<\/p><ol class=\"wp-block-list\"><li>The\u00a0<code>requests<\/code>\u00a0library is imported in the first line.<\/li><li>The\u00a0<code>BeautifulSoup<\/code>\u00a0class from the\u00a0<code>bs4<\/code>\u00a0(Beautiful Soup) library is also imported.<\/li><li>The\u00a0<code>target_url<\/code>\u00a0variable is assigned the value of the desired URL to scrape, in this case, \u201c<a href=\"https:\/\/www.ebay.com\/itm\/182905307044\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/www.ebay.com\/itm\/182905307044<\/a>\u201c. You can replace this URL with the desired target URL.<\/li><li>The\u00a0<code>requests.get()<\/code>\u00a0function is used to send an HTTP GET request to the\u00a0<code>target_url<\/code>. The response from the server is stored in the\u00a0<code>resp<\/code>\u00a0variable.<\/li><li>The\u00a0<code>resp.status_code<\/code>\u00a0is printed, which will display the HTTP status code returned by the server in response to the GET request.<br \/><br \/><\/li><\/ol><p id=\"eb90\">By executing this code, you will be able to retrieve the HTTP status code of the response received from the target URL. The status code can provide information about the success or failure of the request, such as\u00a0<code>200<\/code>\u00a0for a successful request or\u00a0<a href=\"https:\/\/wpclerks.com\/fix-wordpress-404-not-found-error\/\" target=\"_blank\" rel=\"noopener\"><code>404<\/code>\u00a0for a page not found error<\/a>.<\/p><p id=\"7566\">Once you run this code you should see\u00a0<strong>200<\/strong>\u00a0on the console.<\/p><p id=\"4b5b\">So, our setup script successfully scraped eBay and downloaded the HTML we wanted. Now, let\u2019s decide what exact information we want to extract from the page.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-68c0609 elementor-widget elementor-widget-heading\" data-id=\"68c0609\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">What are we going to scrape from eBay?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-64395ab font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"64395ab\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Deciding this in advance helps us identify the location of all the data elements inside the DOM at once.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0a08267 elementor-widget elementor-widget-image\" data-id=\"0a08267\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"800\" height=\"415\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-18-5.jpg\" class=\"attachment-large size-large wp-image-10850\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-18-5.jpg 828w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-18-5-300x156.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-18-5-768x399.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-db4084f font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"db4084f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"510e\">We are going to scrape seven data elements from the page.<\/p><ul class=\"wp-block-list\"><li>Images of the product<\/li><li>Title of the product<\/li><li>Rating<\/li><li>Price<\/li><li>Listed Price<\/li><li>Discount\/Savings<\/li><li>Shipping charges<\/li><\/ul><p id=\"c881\">To start, we will use the\u00a0<code>requests<\/code>\u00a0library to send a GET request to the target page. After that, we will utilize the BeautifulSoup library to extract the desired data from the response. While there are alternative libraries like\u00a0<code>lxml<\/code>\u00a0available for parsing, BeautifulSoup stands out with its robust and user-friendly API.<\/p><p id=\"a626\">Prior to sending the request, we will analyze the page and determine the position of each element within the Document Object Model (DOM). It is advisable to perform this exercise in order to accurately locate each element on the page.<\/p><p id=\"36fa\">We will accomplish this by utilizing the developer tool. To access it, simply right-click on the desired element and select \u201c<strong>Inspect<\/strong>\u201d. This is a commonly used method, and you may already be familiar with it.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d373055 elementor-widget elementor-widget-heading\" data-id=\"d373055\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Identifying the location of each element<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-26ae3ce elementor-widget elementor-widget-heading\" data-id=\"26ae3ce\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Scraping Title of the product<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-deb6684 elementor-widget elementor-widget-image\" data-id=\"deb6684\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"180\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-19-1024x231-1.jpg\" class=\"attachment-large size-large wp-image-10857\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-19-1024x231-1.jpg 1024w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-19-1024x231-1-300x68.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-19-1024x231-1-768x173.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ffc07a1 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"ffc07a1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"8e88\">As you can see the title of the product is stored inside the\u00a0<code>h1 tag<\/code>\u00a0with class\u00a0<code>x-item-title__mainTitle<\/code>. So, scraping this would be super simple.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-11e0edf elementor-widget elementor-widget-code-highlight\" data-id=\"11e0edf\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp>l=[]\r\no={}\r\nimage_list=[]\r\n\r\nsoup=BeautifulSoup(resp.text,'html.parser')\r\n\r\no[\"title\"]=soup.find(\"h1\",{\"class\":\"x-item-title__mainTitle\"}).text<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9e8d7e0 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"9e8d7e0\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"7987\">First, we have declared some empty lists and an object to store all the scraped data.<\/p><p id=\"5cbd\">The fourth line creates a BeautifulSoup object named\u00a0<code>soup<\/code>\u00a0by parsing the HTML content of the\u00a0<code>resp<\/code>\u00a0response object. The HTML parser used is specified as\u00a0<code>'html.parser'<\/code>.<\/p><p id=\"a637\">The last line finds an\u00a0<code>h1<\/code>\u00a0element with a class attribute of\u00a0<code>\"x-item-title__mainTitle\"<\/code>\u00a0within the parsed HTML content. It then accesses the\u00a0<code>text<\/code>\u00a0attribute to retrieve the text content of the\u00a0<code>h1<\/code>\u00a0element. Finally, it assigns this text to the\u00a0<code>o<\/code>\u00a0dictionary with the key\u00a0<code>\"title\"<\/code>.<\/p><p id=\"6855\">To fully grasp the meaning of the code snippet provided above, it is essential to have prior knowledge of the preceding section where we discussed the process of retrieving HTML data from the target page. Therefore, I strongly recommend reviewing the earlier section before proceeding further with the coding.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-87488d8 elementor-widget elementor-widget-heading\" data-id=\"87488d8\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Scraping Rating of the product<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-db88823 elementor-widget elementor-widget-image\" data-id=\"db88823\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"237\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-20-1024x303-1.jpg\" class=\"attachment-large size-large wp-image-10858\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-20-1024x303-1.jpg 1024w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-20-1024x303-1-300x89.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-20-1024x303-1-768x227.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2fe3a45 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"2fe3a45\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>As you can see in the above image, the rating is hidden inside the\u00a0<code>span<\/code>\u00a0tag with class\u00a0<code>ebay-review-start-rating<\/code>. Scraping rating is super simple. Here is how you can do it.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bf2ea73 elementor-widget elementor-widget-code-highlight\" data-id=\"bf2ea73\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp>o[\"rating\"]=soup.find(\"span\",{\"class\":\"ebay-review-start-rating\"}).text.strip()<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e029fcf font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"e029fcf\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The\u00a0<code>strip()<\/code>\u00a0method is then applied to remove any leading or trailing whitespace characters from the extracted text.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1432502 elementor-widget elementor-widget-heading\" data-id=\"1432502\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Scraping the Actual Price<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8ad852a elementor-widget elementor-widget-image\" data-id=\"8ad852a\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"241\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-21-1-2.jpg\" class=\"attachment-large size-large wp-image-10864\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-21-1-2.jpg 828w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-21-1-2-300x90.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-21-1-2-768x231.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1d2daf2 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"1d2daf2\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The price is hidden inside\u00a0<code>span<\/code>\u00a0tag with attribute\u00a0<code>itemprop<\/code>\u00a0and value as\u00a0<code>price<\/code>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ddb7ccc elementor-widget elementor-widget-heading\" data-id=\"ddb7ccc\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">\nScraping the list price and the discount<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-678c794 elementor-widget elementor-widget-image\" data-id=\"678c794\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"189\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-22-1-2.jpg\" class=\"attachment-large size-large wp-image-10881\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-22-1-2.jpg 828w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-22-1-2-300x71.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-22-1-2-768x182.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-356b00c font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"356b00c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>When you inspect you will notice that both the listing price and the discount are listed inside the\u00a0<code>div<\/code>\u00a0tag with class\u00a0<code>x-additional-info<\/code>. So, let\u2019s first find this class using the\u00a0<code>.find()<\/code>\u00a0method of BS4.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e39fed2 elementor-widget elementor-widget-code-highlight\" data-id=\"e39fed2\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>box=soup.find(\"div\",{\"class\":\"x-additional-info\"})<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f0f5ba8 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"f0f5ba8\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Now, let\u2019s find where the price and discount are stored.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1214d84 elementor-widget elementor-widget-image\" data-id=\"1214d84\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"248\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-23-1-2.jpg\" class=\"attachment-large size-large wp-image-10894\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-23-1-2.jpg 828w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-23-1-2-300x93.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-23-1-2-768x238.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-30a7db7 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"30a7db7\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>So, the listed price is stored inside\u00a0<code>span<\/code>\u00a0tag with class\u00a0<code>ux-textspans \u2014 STRIKETHROUGH<\/code>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-630d1f2 elementor-widget elementor-widget-code-highlight\" data-id=\"630d1f2\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>o[\"list_price\"]=box.find(\"span\",{\"class\":\"ux-textspans--STRIKETHROUGH\"}).text<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8007741 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"8007741\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Similarly, the discount is stored inside a\u00a0<code>span<\/code>\u00a0tag with class\u00a0<code>ux-textspans \u2014 EMPHASIS<\/code>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-de07834 elementor-widget elementor-widget-code-highlight\" data-id=\"de07834\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>o[\"discount\"]=box.find(\"span\",{\"class\":\"ux-textspans--EMPHASIS\"}).text<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2441b3a elementor-widget elementor-widget-heading\" data-id=\"2441b3a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Scraping the Shipping Charges<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-df1bd2a elementor-widget elementor-widget-image\" data-id=\"df1bd2a\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"328\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-24-1-1.jpg\" class=\"attachment-large size-large wp-image-10913\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-24-1-1.jpg 828w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-24-1-1-300x123.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-24-1-1-768x314.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9671285 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"9671285\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Shipping charges are stored inside the\u00a0<code>span<\/code>\u00a0tag with class\u00a0<code>ux-textspans \u2014 BOLD<\/code>\u00a0and this\u00a0<code>span<\/code>\u00a0tag is inside the\u00a0<code>div<\/code>\u00a0tag with id\u00a0<code>SRPSection<\/code>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3a03a16 elementor-widget elementor-widget-code-highlight\" data-id=\"3a03a16\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>o[\"shipping_price\"]=soup.find(\"div\",{\"id\":\"SRPSection\"}).find(\"span\",{\"class\":\"ux-textspans--BOLD\"}).text<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-792bb96 elementor-widget elementor-widget-heading\" data-id=\"792bb96\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Scraping the product Images<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-80c57f9 elementor-widget elementor-widget-image\" data-id=\"80c57f9\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"322\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-25-2.jpg\" class=\"attachment-large size-large wp-image-10952\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-25-2.jpg 828w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-25-2-300x121.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-25-2-768x309.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-df7d334 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"df7d334\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"c1ef\">All the product images are stored inside the\u00a0<code>div<\/code>\u00a0tag with the\u00a0<code>class ux-image-carousel-item<\/code>. Inside each of these\u00a0<code>div<\/code>\u00a0tags, there is an\u00a0<code>img<\/code>\u00a0tag where the URL of the image is stored.<\/p><p id=\"10ad\">So, let\u2019s first crawl all of these\u00a0<code>div<\/code>\u00a0tags and then with the help of\u00a0<code>for<\/code>\u00a0loop we will extract values of the\u00a0<code>src<\/code>\u00a0attributes of the\u00a0<code>img<\/code>\u00a0tag.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e0cca7b elementor-widget elementor-widget-code-highlight\" data-id=\"e0cca7b\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>images=soup.find_all(\"div\",{\"class\":\"ux-image-carousel-item\"})<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cec0893 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"cec0893\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"112f\">Here we are looking for\u00a0<code>&lt;div&gt;<\/code>\u00a0elements with the class attribute \u201cux-image-carousel-item\u201d. The\u00a0<code>find_all()<\/code>\u00a0method returns a list of all matching elements found in the HTML document. The extracted elements are assigned to the\u00a0<code>images<\/code>\u00a0variable.<\/p><p id=\"77d7\">Now we will use<strong>\u00a0for<\/strong>\u00a0loop to iterate over all the\u00a0<code>div<\/code>\u00a0tags and extract the image URL.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-049cf09 elementor-widget elementor-widget-code-highlight\" data-id=\"049cf09\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>for image in images:\r\n    image_url=image.find(\"img\").get('data-src')\r\n    image_list.append(image_url)\r\n\r\n\r\nl.append(o)\r\nl.append(image_list)\r\nprint(l)<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b44af2e font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"b44af2e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"60ec\">All the images are inside the\u00a0<code>src<\/code>\u00a0attribute. But once you scrape the page it will appear as\u00a0<code>data-src<\/code>\u00a0instead of simply src. That\u2019s why we have used\u00a0<code>data-src<\/code>\u00a0above.<\/p><p id=\"0442\">With this, we have scraped all the data we decided on earlier. You can also use\u00a0<code>try\/except<\/code>\u00a0statement to avoid any kind of error.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f640f9c elementor-widget elementor-widget-heading\" data-id=\"f640f9c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Complete Code<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1f73e37 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"1f73e37\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"c2e1\">On our target page, there is a tremendous amount of data that can be used for data analysis. You can of course make changes to the code and extract more data points from the page.<\/p><p id=\"33f8\">You can even use a\u00a0<a href=\"https:\/\/pypi.org\/project\/python-crontab\/\" rel=\"nofollow noopener\" target=\"_blank\">cron job<\/a>\u00a0to run a crawler at a particular time and once the product is available at your target price it will send you an email.<\/p><p id=\"97d2\">But for now, the code will look like this.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-484513e elementor-widget elementor-widget-code-highlight\" data-id=\"484513e\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>import requests\r\nfrom bs4 import BeautifulSoup\r\n\r\nl=[]\r\no={}\r\nimage_list=[]\r\ntarget_url=\"https:\/\/www.ebay.com\/itm\/182905307044\"\r\n\r\nresp = requests.get(target_url)\r\n\r\nsoup=BeautifulSoup(resp.text,'html.parser')\r\n\r\n\r\no[\"title\"]=soup.find(\"h1\",{\"class\":\"x-item-title__mainTitle\"}).text\r\no[\"rating\"]=soup.find(\"span\",{\"class\":\"ebay-review-start-rating\"}).text.strip()\r\no[\"actual_price\"]=soup.find(\"span\",{\"itemprop\":\"price\"}).text\r\nbox=soup.find(\"div\",{\"class\":\"x-additional-info\"})\r\nimages=soup.find_all(\"div\",{\"class\":\"ux-image-carousel-item\"})\r\nfor image in images:\r\n    image_url=image.find(\"img\").get('data-src')\r\n    image_list.append(image_url)\r\no[\"list_price\"]=box.find(\"span\",{\"class\":\"ux-textspans--STRIKETHROUGH\"}).text\r\no[\"discount\"]=box.find(\"span\",{\"class\":\"ux-textspans--EMPHASIS\"}).text\r\no[\"shipping_price\"]=soup.find(\"div\",{\"id\":\"SRPSection\"}).find(\"span\",{\"class\":\"ux-textspans--BOLD\"}).text\r\nl.append(o)\r\nl.append(image_list)\r\nprint(l)<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-29a3fda font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"29a3fda\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Once you run this code it will return this.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1d8f100 elementor-widget elementor-widget-image\" data-id=\"1d8f100\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"63\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-26-1.jpg\" class=\"attachment-large size-large wp-image-10954\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-26-1.jpg 828w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-26-1-300x24.jpg 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/image-26-1-768x60.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6e5ad90 elementor-widget elementor-widget-heading\" data-id=\"6e5ad90\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Using Scrapingdog for eBay scraping<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0b624f9 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"0b624f9\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"1cb5\">The technique we have used will not be enough for crawling eBay at scale. Because eBay detects that the traffic is coming from a bot\/script it will block our IP and your data pipeline will stop pulling data from the website.<\/p><p id=\"0e79\">Here you can use Web Scraping APIs like <a href=\"https:\/\/www.scrapingdog.com\/\" target=\"_blank\" rel=\"noopener\">Scrapingdog<\/a> which can help you pull millions of pages from eBay without getting blocked. Scrapingdog uses new IP on every request which makes scraping completely anonymous.<\/p><p id=\"4b15\">You can start with 1000 free credits that you get once you <a href=\"https:\/\/api.scrapingdog.com\/register\" rel=\"nofollow noopener\" target=\"_blank\">sign up<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1271b1d elementor-widget elementor-widget-image\" data-id=\"1271b1d\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"388\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/sign-up-scrapingdog-1024x496.png\" class=\"attachment-large size-large wp-image-17484\" alt=\"scrapingdog homepage\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/sign-up-scrapingdog-1024x496.png 1024w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/sign-up-scrapingdog-300x145.png 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/sign-up-scrapingdog-768x372.png 768w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/sign-up-scrapingdog-1536x744.png 1536w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/08\/sign-up-scrapingdog.png 1901w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3655623 font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"3655623\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Once you sign up, you will be redirected to your dashboard. The dashboard will look somewhat like this.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0c7dcc1 elementor-widget elementor-widget-image\" data-id=\"0c7dcc1\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"373\" src=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/11\/1_nLbseqzbR5yev8zPputyWQ-1024x477.webp\" class=\"attachment-large size-large wp-image-19885\" alt=\"\" srcset=\"https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/11\/1_nLbseqzbR5yev8zPputyWQ-1024x477.webp 1024w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/11\/1_nLbseqzbR5yev8zPputyWQ-300x140.webp 300w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/11\/1_nLbseqzbR5yev8zPputyWQ-768x358.webp 768w, https:\/\/www.scrapingdog.com\/wp-content\/uploads\/2024\/11\/1_nLbseqzbR5yev8zPputyWQ.webp 1400w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-40fba7c font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"40fba7c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"f3d0\">You have to use your API key.<\/p><p id=\"ad08\">Now, you can paste your target indeed target page link to the left and then select\u00a0<strong>JS Rendering<\/strong>\u00a0as\u00a0<strong><em>No<\/em><\/strong>. After this click on\u00a0<strong><em>Copy Code<\/em><\/strong>\u00a0from the right. Now use this API in your script to scrape eBay.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c51fe28 elementor-widget elementor-widget-code-highlight\" data-id=\"c51fe28\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>import requests\r\nfrom bs4 import BeautifulSoup\r\n\r\nl=[]\r\no={}\r\nimage_list=[]\r\ntarget_url=\"https:\/\/api.scrapingdog.com\/scrape?api_key=xxxxxxxxxxxxxx&url=https:\/\/www.ebay.com\/itm\/182905307044&dynamic=false\"\r\n\r\nresp = requests.get(target_url)\r\n\r\nsoup=BeautifulSoup(resp.text,'html.parser')\r\n\r\n\r\no[\"title\"]=soup.find(\"h1\",{\"class\":\"x-item-title__mainTitle\"}).text\r\no[\"rating\"]=soup.find(\"span\",{\"class\":\"ebay-review-start-rating\"}).text.strip()\r\no[\"actual_price\"]=soup.find(\"span\",{\"itemprop\":\"price\"}).text\r\nbox=soup.find(\"div\",{\"class\":\"x-additional-info\"})\r\nimages=soup.find_all(\"div\",{\"class\":\"ux-image-carousel-item\"})\r\nfor image in images:\r\n    image_url=image.find(\"img\").get('data-src')\r\n    image_list.append(image_url)\r\no[\"list_price\"]=box.find(\"span\",{\"class\":\"ux-textspans--STRIKETHROUGH\"}).text\r\no[\"discount\"]=box.find(\"span\",{\"class\":\"ux-textspans--EMPHASIS\"}).text\r\no[\"shipping_price\"]=soup.find(\"div\",{\"id\":\"SRPSection\"}).find(\"span\",{\"class\":\"ux-textspans--BOLD\"}).text\r\nl.append(o)\r\nl.append(image_list)\r\nprint(l)<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b573bcd font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"b573bcd\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"0a9d\">You will notice the code will remain somewhat the same as above. We just have to change one thing and that is our target URL.<\/p><p id=\"210c\">You don\u2019t have to bother about retrying on a failed request or even passing any header. All of this will be handled by Scrapingdog itself.<\/p><p id=\"7993\">So, give Scrapingdog\u2019s\u00a0<a href=\"https:\/\/www.scrapingdog.com\/\" rel=\"nofollow\" data-type=\"link\" data-id=\"https:\/\/www.scrapingdog.com\/\">Scraping API<\/a>\u00a0a try. You can sign up from\u00a0<a href=\"https:\/\/api.scrapingdog.com\/register\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">here<\/a>\u00a0(NO CREDIT CARD REQUIRED)<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c43fe9c elementor-widget elementor-widget-heading\" data-id=\"c43fe9c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Conclusion<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ec5eb8a font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"ec5eb8a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"0cb5\">In this tutorial, we scraped various data elements from eBay. First, we used the requests library to download the raw HTML, and then using BS4 we parsed the data we wanted. Finding the locations of each element was super simple in this tutorial. I must say scraping eBay is way easier than\u00a0<a href=\"https:\/\/www.scrapingdog.com\/blog\/scrape-amazon\/\" data-type=\"link\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/scrape-amazon\/\">Amazon\u00a0price scraping<\/a>\u00a0or\u00a0<a href=\"https:\/\/www.scrapingdog.com\/blog\/scrape-walmart\/\" data-type=\"URL\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/scrape-walmart\/\">scraping Walmart<\/a>\u00a0for a beginner. I do have a recent guide on\u00a0<a href=\"https:\/\/www.scrapingdog.com\/blog\/scrape-amazon-reviews\/\" target=\"_blank\" rel=\"noreferrer noopener\" data-type=\"link\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/scrape-amazon-reviews\/\">scraping Amazon reviews using Python<\/a>, do check it out too!<\/p><p id=\"4321\">Combination of\u00a0<code>requests<\/code>\u00a0and\u00a0<code>Scrapingdog<\/code>\u00a0can help you scale your eBay scraping. You will get more than a 99% success rate while scraping eBay with Scrapingdog.<\/p><p id=\"eb92\">I hope you like this little tutorial in which we learned how to build an eBay scraper. And if you liked it then please do not forget to share it with your friends and on your social media.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ed4933d elementor-widget elementor-widget-heading\" data-id=\"ed4933d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Additional Resources<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3c83bdc font-color-green elementor-widget elementor-widget-text-editor\" data-id=\"3c83bdc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Here are a few additional resources that you may find helpful during your web scraping journey:<\/p><div class=\"wp-block-spacer\" aria-hidden=\"true\">\u00a0<\/div><ul><li><a href=\"https:\/\/www.scrapingdog.com\/blog\/build-amazon-price-tracker\/\" target=\"_blank\" rel=\"noreferrer noopener\" data-type=\"link\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/build-amazon-price-tracker\/\">Build a Price Tracker using Python for Amazon Product<\/a><\/li><li><a href=\"https:\/\/www.scrapingdog.com\/blog\/amazon-price-tracker-using-scrapingdog-amazon-scraper-api-and-make\/\" target=\"_blank\" rel=\"noopener\">Automating &amp; Building a Price Tracker for Scraping Amazon Product Prices using Scrapingdog&#8217;s API &amp; Make.com<\/a><\/li><li><a href=\"https:\/\/www.scrapingdog.com\/blog\/web-scraping-myntra\/\" data-type=\"link\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/web-scraping-myntra\/\">Web Scraping with Myntra<\/a><\/li><li><a href=\"https:\/\/www.scrapingdog.com\/blog\/scrape-google-shopping\/\">Web scraping Google Shopping using Python<\/a><\/li><li><a href=\"https:\/\/www.scrapingdog.com\/blog\/scrape-flipkart\/\" target=\"_blank\" rel=\"noopener\">Web Scraping Flipkart<\/a><\/li><li><a href=\"https:\/\/www.scrapingdog.com\/blog\/web-scraping-with-python\/\" target=\"_blank\" rel=\"noreferrer noopener\" data-type=\"URL\" data-id=\"https:\/\/www.scrapingdog.com\/blog\/web-scraping-with-python\/\">Web scraping with Python<\/a><\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-d0acbc4 e-con-full web-scraping-right-con elementor-hidden-desktop elementor-hidden-tablet e-flex e-con e-child\" data-id=\"d0acbc4\" data-element_type=\"container\" data-settings=\"{&quot;background_background&quot;:&quot;classic&quot;,&quot;sticky&quot;:&quot;top&quot;,&quot;sticky_on&quot;:[&quot;desktop&quot;,&quot;tablet&quot;],&quot;sticky_parent&quot;:&quot;yes&quot;,&quot;sticky_offset&quot;:0,&quot;sticky_effects_offset&quot;:0}\">\n\t\t<div class=\"elementor-element elementor-element-0d50e64 e-con-full e-flex e-con e-child\" data-id=\"0d50e64\" data-element_type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-877324b elementor-widget elementor-widget-heading\" data-id=\"877324b\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Web Scraping with Scrapingdog<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1a01492 elementor-widget elementor-widget-text-editor\" data-id=\"1a01492\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tScrape the web without the hassle of getting blocked\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c503484 e-con-full e-flex e-con e-child\" data-id=\"c503484\" data-element_type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-e09b8f0 elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"e09b8f0\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/api.scrapingdog.com\/register\" target=\"_blank\" rel=\"noopener\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Try for Free<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3b819e3 elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"3b819e3\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/share.hsforms.com\/1ex4xYy1pTt6rrqFlRAquwQ4h1b2\" target=\"_blank\" rel=\"noopener\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Contact sales<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-cb6c196 e-con-full e-flex e-con e-child\" data-id=\"cb6c196\" data-element_type=\"container\">\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>We will be scraping ebay in this tutorial using Python.<\/p>\n","protected":false},"author":5,"featured_media":18686,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[25],"tags":[],"class_list":["post-10825","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/posts\/10825","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/comments?post=10825"}],"version-history":[{"count":0,"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/posts\/10825\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/media\/18686"}],"wp:attachment":[{"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/media?parent=10825"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/categories?post=10825"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.scrapingdog.com\/wp-json\/wp\/v2\/tags?post=10825"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}