# Scrape Product Prices with Python from JSON-LD

> Scrape product prices with Python from schema.org JSON-LD: Offer, offer lists, AggregateOffer, @graph and variants, tested on live stores, and where it breaks.

- URL: https://datatooly.xyz/guides/scrape-product-prices-json-ld-python/
- Author: Omar Eldeeb (https://datatooly.xyz/about/)
- Published: 2026-10-20
- Updated: 2026-10-20
- Free tool: [E-commerce price tracker — scrape product prices, stock and variants](https://datatooly.xyz/ecommerce-price-tracker/)
- Tags: python, webscraping, ecommerce, json

To scrape product prices with Python, fetch the page with `requests`, parse every `<script type="application/ld+json">` tag with `json.loads`, find the schema.org `Product` nodes, and read `offers`: `price`, `priceCurrency` and `availability`. Handle the real-world shapes: `@graph` arrays, lists of offers, `AggregateOffer` (`lowPrice`/`highPrice`), prices inside `priceSpecification`, and `ProductGroup` variants.

Below is a script that handles all of these, its output on live stores on 2026-10-20, the stores where the same approach returned nothing that day, and how [E-commerce Product Scraper Pro](https://apify.com/constructive_calm/ecommerce-product-scraper-pro?fpr=v77kxu) handles the cases plain JSON-LD can't.

## Why JSON-LD

Many stores embed schema.org product data in their HTML for search engines, usually as JSON-LD. It's machine-readable data the store publishes on purpose, so it changes less often than the page's CSS classes. The fields you need are defined by schema.org:

- [`Offer`](https://schema.org/Offer): `price`, `priceCurrency`, `availability` (a URL such as `https://schema.org/InStock`), often `sku` and `url`.
- [`AggregateOffer`](https://schema.org/AggregateOffer): used "when a single product is associated with multiple offers", with `lowPrice`, `highPrice` and `offerCount`.
- [`ProductGroup`](https://schema.org/ProductGroup): "a group of Products that vary only in certain well-described ways, such as by size, color", with the variants in `hasVariant`.

## The shapes you'll actually meet

Every row below is a live page I parsed on 2026-10-20:

| Store (platform) | Where the price was |
|---|---|
| IKEA US | A plain `Offer` with `"price": "79"` as a string, `priceCurrency` alongside |
| Marks & Spencer | No top-level `price`: it's in `offers.priceSpecification` (a `UnitPriceSpecification` with `price`, `minPrice`, `maxPrice`, `priceCurrency`) |
| Nalgene (WooCommerce) | Inside `@graph`; a `ProductGroup` whose `hasVariant` items each have an offer; `priceSpecification` serialised as an object with a `"0"` key holding the currency |
| Death Wish Coffee (Shopify) | One `Product` with a list of `Offer`s, one per variant, each with its own `sku` and price |
| Skullcandy (Shopify) | A `ProductGroup` with `hasVariant`, plus a separate `Product` for the default variant |

A parser that only reads `data["offers"]["price"]` gets two of these five. The script walks the whole document instead.

## The script

```python
"""Extract product prices from schema.org JSON-LD. pip install requests beautifulsoup4"""
import json
import sys

import requests
from bs4 import BeautifulSoup

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/141.0 Safari/537.36",
    "Accept-Language": "en-US,en;q=0.9",
}


def walk(node):
    """Yield every dict in a JSON-LD document: top-level lists, @graph, nested values."""
    if isinstance(node, list):
        for item in node:
            yield from walk(item)
    elif isinstance(node, dict):
        yield node
        for value in node.values():
            if isinstance(value, (dict, list)):
                yield from walk(value)


def has_type(node, name):
    t = node.get("@type")
    return any(str(x).rsplit("/", 1)[-1] == name for x in (t if isinstance(t, list) else [t]) if x)


def number(value):
    try:
        return float(str(value).replace(",", "")) if value not in (None, "") else None
    except ValueError:
        return None


def price_spec(spec):
    """priceSpecification can be a dict, a list, or (WooCommerce) a dict with a "0" key."""
    specs = spec if isinstance(spec, list) else [spec]
    for s in specs:
        if not isinstance(s, dict):
            continue
        nested = s.get("0")
        price = s.get("price", nested.get("price") if isinstance(nested, dict) else None)
        currency = s.get("priceCurrency") or (nested.get("priceCurrency") if isinstance(nested, dict) else None)
        if price is not None:
            return number(price), currency
    return None, None


def offers(product):
    """One dict per offer: handles Offer, a list of Offers and AggregateOffer."""
    raw = product.get("offers") or []
    out = []
    for o in raw if isinstance(raw, list) else [raw]:
        if not isinstance(o, dict):
            continue
        if has_type(o, "AggregateOffer") and o.get("offers"):
            out += offers({"offers": o["offers"]})          # real offers nested inside
            continue
        price, currency = number(o.get("price")), o.get("priceCurrency")
        if has_type(o, "AggregateOffer"):
            price = number(o.get("lowPrice"))
        if price is None and "priceSpecification" in o:
            price, spec_currency = price_spec(o["priceSpecification"])
            currency = currency or spec_currency
        out.append({
            "price": price,
            "high_price": number(o.get("highPrice")),        # AggregateOffer only
            "currency": currency,
            "availability": str(o.get("availability") or "").rsplit("/", 1)[-1] or None,
            "sku": o.get("sku") or product.get("sku"),
            "url": o.get("url"),
        })
    return out


def extract(html):
    """All (product name, offer) pairs in the page's JSON-LD, de-duplicated."""
    soup = BeautifulSoup(html, "html.parser")
    rows, seen = [], set()
    for tag in soup.find_all("script", type="application/ld+json"):
        try:
            doc = json.loads(tag.string or "")
        except json.JSONDecodeError:
            continue                                        # malformed JSON-LD is common; skip it
        for node in walk(doc):                              # also reaches ProductGroup.hasVariant
            if not has_type(node, "Product"):
                continue
            for o in offers(node):
                key = (o["sku"], o["price"], o["url"])
                if o["price"] is not None and key not in seen:
                    seen.add(key)
                    rows.append({"name": node.get("name"), **o})
    return rows


def fetch(url):
    try:
        r = requests.get(url, headers=HEADERS, timeout=20)
    except requests.RequestException as e:
        return None, f"request failed: {type(e).__name__}"
    return r, f"HTTP {r.status_code}"


if __name__ == "__main__":
    for url in sys.argv[1:]:
        r, status = fetch(url)
        rows = extract(r.text) if r is not None and r.ok else []
        print(f"\n{url}\n  {status}, {len(rows)} offer(s) in JSON-LD")
        for row in rows[:3]:
            print(f"  {row['name'][:45]!r} | {row['sku']} | {row['price']} {row['currency']} | {row['availability']}")
```

Design notes:

- **`walk()` visits every dict**, so it finds `Product` nodes at the top level, in a list, inside `@graph`, and inside `ProductGroup.hasVariant` without special cases. The `seen` set removes the duplicates that this produces when a page describes the same variant twice (Skullcandy does).
- **`has_type()` accepts lists and full URLs**, because `@type` can be `["Product", "Thing"]` or `"http://schema.org/Product"`.
- **`number()` handles strings**, because `"79"` and `79` are both common.
- **Bad JSON is skipped, not fatal.** Some stores publish invalid JSON-LD.
- **A browser-like `User-Agent`** is sent because several stores answer the default `python-requests` agent differently. It doesn't get you past real bot protection (see below).

Run it with Python 3.12 (`pip install requests beautifulsoup4`):

```bash
python product_prices.py https://www.ikea.com/us/en/p/billy-bookcase-white-20522046/ https://www.marksandspencer.com/fair-isle-crew-neck-relaxed-jumper-with-wool/p/clp61248839 https://nalgene.com/product/32oz-wide-mouth-surrealist-print-bottles/ https://www.deathwishcoffee.com/products/power-surge-coffee https://www.skullcandy.com/products/crusher-540-active
```

Output on 2026-10-20:

```text
https://www.ikea.com/us/en/p/billy-bookcase-white-20522046/
  HTTP 200, 1 offer(s) in JSON-LD
  'BILLY Bookcase - white 31 1/2x11x79 1/2 "' | 205.220.46 | 79.0 USD | InStock

https://www.marksandspencer.com/fair-isle-crew-neck-relaxed-jumper-with-wool/p/clp61248839
  HTTP 200, 1 offer(s) in JSON-LD
  'Fair Isle Crew Neck Relaxed Jumper' | P61248839 | 34.0 GBP | InStock

https://nalgene.com/product/32oz-wide-mouth-surrealist-print-bottles/
  HTTP 200, 3 offer(s) in JSON-LD
  '32oz Wide Mouth Surrealist Bottles - landscap' | 682024-0052 | 18.99 USD | None
  '32oz Wide Mouth Surrealist Bottles - hummingb' | 682024-0051 | 18.99 USD | None
  '32oz Wide Mouth Surrealist Bottles - astro-mo' | 682024-0050 | 18.99 USD | None

https://www.deathwishcoffee.com/products/power-surge-coffee
  HTTP 200, 8 offer(s) in JSON-LD
  'Power Surge Coffee' | LGPWSR16 | 19.99 USD | InStock
  'Power Surge Coffee' | LGPWSR32K | 39.98 USD | InStock
  'Power Surge Coffee' | LGPWSR48K | 59.97 USD | InStock

https://www.skullcandy.com/products/crusher-540-active
  HTTP 200, 6 offer(s) in JSON-LD
  'Crusher® 540 Active - Skullcandy X Baker Skat' | S6EVW-U054 | 229.99 USD | InStock
  'Crusher® 540 Active - Smoke' | S6EVW-T013 | 209.99 USD | InStock
  'Crusher® 540 Active - Forest' | S6EVW-T016 | 209.99 USD | InStock
```

Five stores, five shapes, one parser. Nalgene's variant offers carry no `availability`, so it prints `None`. That is "unknown", not "out of stock".

### Testing the AggregateOffer branch

None of the pages I tried on 2026-10-20 used `AggregateOffer`, so the branch is tested on a small inline document shaped like schema.org's definition, together with a broken tag:

```python
from product_prices import extract

html = """
<script type="application/ld+json">
{"@context": "https://schema.org", "@type": "Product", "name": "Demo kettle", "sku": "K-1",
 "offers": {"@type": "AggregateOffer", "lowPrice": "39.99", "highPrice": "54.00",
            "offerCount": 4, "priceCurrency": "USD"}}
</script>
<script type="application/ld+json">{"@type": "Product", "name": "Broken", </script>
"""
for row in extract(html):
    print(row)
```

```text
{'name': 'Demo kettle', 'price': 39.99, 'high_price': 54.0, 'currency': 'USD', 'availability': None, 'sku': 'K-1', 'url': None}
```

An `AggregateOffer` is a price range, not a price. Store `lowPrice` and `highPrice` separately and don't compare a range with a single price in a price-change alert.

## Where it breaks

The same script, same day, same connection (one home connection outside the US; results depend on where and when you run it):

```text
https://www.target.com/p/lego-marvel-spider-man-vs-anti-venom-heist-30725/-/A-94828796
  HTTP 200, 0 offer(s) in JSON-LD

https://www.amazon.com/dp/B088NRLMPV
  HTTP 200, 0 offer(s) in JSON-LD

https://www.sephora.com/product/peptide-glazing-fluid-P517531
  HTTP 403, 0 offer(s) in JSON-LD

https://www.zalando.de/puma-club-ii-era-unisex-sneaker-low-whiteblackvapor-gray-pu115o0sb-a11.html
  HTTP 403, 0 offer(s) in JSON-LD
```

Each failure is a different problem:

1. **The data is in the app's state, not JSON-LD.** Target's and Walmart's product pages returned HTTP 200 with the product title, but no Product JSON-LD in what `requests` received. Both pages contained a `__NEXT_DATA__` script (Target also `__TGT_DATA__`): the page is built from a JavaScript state object. You'd parse that object per site, or render the page.
2. **The store doesn't use JSON-LD for the price.** Amazon's page returned 200 with the price in its HTML (`a-price` elements), not in JSON-LD. That needs site-specific parsing.
3. **Bot protection.** Sephora and Zalando answered 403; John Lewis and Waitrose didn't answer at all within 20 seconds (`ReadTimeout`) in earlier tries. A different User-Agent doesn't fix this; it needs other network routes or a real browser, and respect for the site's terms.
4. **Variants.** Some pages describe only the selected variant. Nalgene and Death Wish publish every variant; other stores don't, and then sizes and colours need the store's own product data.
5. **Pagination and discovery.** JSON-LD describes one product page. To cover a category you need its product links first, through page parameters, "load more" APIs or the store's search, and a rule for when you've seen everything.

## Scaling up: the same pages through the actor

The actor starts with the same structured data and adds what the script lacks: per-site adapters (Shopify and WooCommerce product APIs, Walmart's page data, Amazon's HTML), a route ladder that moves from datacenter to a crawler User-Agent, residential proxies or a browser only when a page is blocked, variants as `variants[]`, pagination up to 200 pages per listing where the site supports it, and one status row per input explaining any failure. The README's [supported-sites matrix](https://apify.com/constructive_calm/ecommerce-product-scraper-pro?fpr=v77kxu) says what works where; Sephora product pages are best-effort through a browser, and eBay, Best Buy, Home Depot, Costco and Etsy are unsupported.

```python
"""Same pages through the Apify actor. pip install apify-client"""
import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("constructive_calm/ecommerce-product-scraper-pro").call(
    run_input={
        "detailsUrls": [
            {"url": "https://www.marksandspencer.com/fair-isle-crew-neck-relaxed-jumper-with-wool/p/clp61248839"},
            {"url": "https://nalgene.com/product/32oz-wide-mouth-surrealist-print-bottles/"},
            {"url": "https://www.target.com/p/lego-marvel-spider-man-vs-anti-venom-heist-30725/-/A-94828796"},
            {"url": "https://www.amazon.com/dp/B088NRLMPV"},
        ],
        "maxProductResults": 4,
    },
    max_total_charge_usd=0.10,          # hard cap on what this run may cost
)
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{row['site']:<16} {row['price']:>7} {row['currency']}  inStock={row['inStock']}"
          f"  variants={row['variantCount']:<2} tier={row['tierUsed']:<11} from={'+'.join(row['dataSources'])}")
```

Output on 2026-10-20 (the client also streams the run's log while it waits):

```text
marksandspencer       34 GBP  inStock=True  variants=0  tier=datacenter  from=jsonld+meta
target              4.99 USD  inStock=True  variants=0  tier=crawler-ua  from=jsonld+meta
amazon-us           9.99 USD  inStock=True  variants=12 tier=datacenter  from=amazon-html
nalgene.com        18.99 USD  inStock=True  variants=3  tier=datacenter  from=jsonld+woo-api
```

`dataSources` shows where each row came from. Target delivered through the crawler User-Agent route, where its page includes JSON-LD. Amazon came from its HTML. Nalgene combined JSON-LD with the WooCommerce Store API, and its row has a stock flag (`inStock=True`) where the page's JSON-LD had none.

Cost: the actor is pay per event, and these four products are one start plus four product rows on datacenter routes (no traffic fee): 4 × $0.0048 + $0.005 = **$0.0242** on Apify's FREE tier, 4 × $0.0008 + $0.005 = **$0.0082** on GOLD, from the README's pricing table. Residential and browser sites add per-MB and per-render fees. Apify's free plan gives $5 of usage a month with no credit card required; there's no in-actor free trial.

## During Black Friday and the holiday season

The script is enough to start a baseline for a handful of stores that serve JSON-LD: run it once a day from cron or Task Scheduler and append the rows to a CSV, beginning now, about five weeks before Black Friday (Friday 27 November 2026). In the sale window, prices change several times a day, so increase the frequency for the few products you care about and keep the daily readings for medians. Compare the sale price with your own 30-day median, not with the store's "was" price: [how to spot fake Black Friday discounts](https://datatooly.xyz/guides/spot-fake-black-friday-discounts/) shows the method, and the [holiday sales calendar](https://datatooly.xyz/guides/holiday-sales-calendar-price-tracking/) lists when each event's baseline should start.

Related: [scraping product variants](https://datatooly.xyz/guides/scrape-product-variants-sizes-colors/), [Shopify products.json pagination](https://datatooly.xyz/guides/shopify-products-json-pagination/), [competitor price monitoring](https://datatooly.xyz/guides/competitor-price-monitoring-scraper/), and the [e-commerce price tracker query builder](https://datatooly.xyz/ecommerce-price-tracker/). Respect each site's terms and robots rules, and keep request rates low.

*Disclosure: I built the actor and this site. The JSON-LD script above works on its own, without either.*

Sources: [schema.org Offer](https://schema.org/Offer), [schema.org AggregateOffer](https://schema.org/AggregateOffer), [schema.org ProductGroup](https://schema.org/ProductGroup), [E-commerce Product Scraper Pro README](https://apify.com/constructive_calm/ecommerce-product-scraper-pro?fpr=v77kxu) (supported sites, pricing), and live requests to the listed product pages on 2026-10-20.

## Quick answers

### What is the easiest way to scrape a product price in Python?

Fetch the product page with requests and read the schema.org Product data in its `<script type="application/ld+json">` tags: the price is in `offers.price` with `offers.priceCurrency`. It's structured data the store publishes on purpose, so it's more stable than CSS selectors. It only works on pages that include it in the HTML they send.

### Why does my Python scraper return no price?

The usual causes: the page builds its content with JavaScript, so the price isn't in the HTML requests receives; the site blocked the request (HTTP 403, a captcha page or a hang); or the price sits in a shape your code doesn't handle, such as `priceSpecification`, an `AggregateOffer` or a `@graph` array. Print the status code and the JSON-LD types first.

### Should I use BeautifulSoup or a regex for JSON-LD?

Use an HTML parser such as BeautifulSoup to find the script tags, then `json.loads` on each one's text. A regex over raw HTML breaks on attribute order, whitespace and multiple tags. Wrap `json.loads` in try/except, because some stores publish invalid JSON-LD.

### How do I get the price of each size or colour?

Look for one Offer per variant in `offers` (common on Shopify) or for a `ProductGroup` whose `hasVariant` lists one Product per variant, each with its own offer and SKU. If the JSON-LD only describes the selected variant, you need the store's own product data (for example Shopify's product JSON) or a rendered page.

## Related guides

- [Retail Sales Calendar 2026-2027: When to Track Prices](https://datatooly.xyz/guides/holiday-sales-calendar-price-tracking/)
- [Amazon Price History Without Keepa: Build Your Own](https://datatooly.xyz/guides/amazon-price-history-without-keepa/)
- [Amazon Price Scraper: Prices by ASIN on Any Marketplace](https://datatooly.xyz/guides/amazon-price-scraper-asin/)
- [Back in Stock Alert Scraper: Products, Sizes, Colours](https://datatooly.xyz/guides/back-in-stock-alert-scraper/)
- [Back to School Price Tracker: Tax Holidays & Laptops](https://datatooly.xyz/guides/back-to-school-price-tracker/)
- [Compare Black Friday Prices Across Stores (Google Shopping)](https://datatooly.xyz/guides/black-friday-google-shopping-price-comparison/)
- [Black Friday Price Tracker: Watch 2026 Prices Daily](https://datatooly.xyz/guides/black-friday-price-tracker/)
- [Boxing Day Sales Tracker UK 2026: Prices in GBP](https://datatooly.xyz/guides/boxing-day-sales-price-tracker-uk/)
- [Christmas Price Tracker: When to Buy Gifts Cheapest](https://datatooly.xyz/guides/christmas-price-tracker/)
- [Competitor Price Monitoring: Scraper Setup and Cost](https://datatooly.xyz/guides/competitor-price-monitoring-scraper/)
- [Cyber Monday Price Tracker: Hourly Checks for Nov 30](https://datatooly.xyz/guides/cyber-monday-deal-tracker/)
- [Grocery Price Tracker: Build a Basket Index in Python](https://datatooly.xyz/guides/grocery-price-tracker-scraper/)
- [MAP Monitoring: Find Minimum Advertised Price Violations](https://datatooly.xyz/guides/map-price-monitoring/)
- [Price Drop Alerts to Slack or Email with a Scraper](https://datatooly.xyz/guides/price-drop-alerts-email-slack/)
- [Prime Day Price Tracker: ASINs, Big Deal Days, 2027](https://datatooly.xyz/guides/prime-day-price-tracker/)
- [Ramadan Sale Price Tracker 2027: Noon & Amazon UAE](https://datatooly.xyz/guides/ramadan-eid-sales-price-tracking/)
- [Scrape Product Reviews from Walmart, Target and Amazon](https://datatooly.xyz/guides/scrape-product-reviews-walmart-target-amazon/)
- [Scrape Product Variants: Sizes, Colors and Price per SKU](https://datatooly.xyz/guides/scrape-product-variants-sizes-colors/)
- [Fake Black Friday Deals: Check the Real Discount](https://datatooly.xyz/guides/spot-fake-black-friday-discounts/)
- [Apify E-commerce Scraping Tool Alternative: Same Input](https://datatooly.xyz/guides/switch-from-apify-ecommerce-scraping-tool/)
- [When Do Black Friday Prices Drop? The 2026 Timeline](https://datatooly.xyz/guides/thanksgiving-black-friday-price-drop-timeline/)
- [Walmart and Target Price Scraper: Search and Product Pages](https://datatooly.xyz/guides/walmart-target-price-scraper/)
- [White Friday & 11.11 Tracker: Noon and Amazon UAE Prices](https://datatooly.xyz/guides/white-friday-11-11-noon-amazon-uae-prices/)
- [Shopify products.json Pagination: 250 Limit and the 25,000 Cap](https://datatooly.xyz/guides/shopify-products-json-pagination/)
- [AI Share of Voice: Measure It in ChatGPT and Perplexity](https://datatooly.xyz/guides/ai-share-of-voice-chatgpt-perplexity/)