IMDb Scraper: Extract Movie Ratings, Cast, and Release Dates with Python

Jul 11, 2026 · tutorial · #python, #imdb scraper, #imdb, #json, #beautifulsoup, #web-scraping, #proxiesapi

If you search for an IMDb scraper, you usually want more than a page title. The useful fields are structured metadata you can analyze later: rating, cast, release date, canonical URL, and title type.

The practical way to get there is a two-step pipeline:

use IMDb's public suggestion JSON endpoint to seed matching titles
fetch selected title pages and parse application/ld+json

That approach works better than betting everything on brittle CSS selectors.

What we are scraping

For a query like matrix, IMDb exposes a suggestion endpoint:

https://v3.sg.media-imdb.com/suggestion/m/matrix.json

The suggestion payload already gives you:

title id
title label
year
title type
cast snippet

Then the title page can add richer fields:

aggregate rating
published/release date
cast list
canonical title URL

Here is the strategy in one view:

Source	Best for	Why it matters
suggestion JSON	search results and ids	fast, structured, cheap
title page `ld+json`	rating, release date, actors	semantically stable
CSS classes	last-resort fallback	visible but easier to break

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install requests beautifulsoup4 lxml

Optional:

export PROXIESAPI_KEY="YOUR_KEY"

Step 1: Fetch the suggestion endpoint

from __future__ import annotations

import csv
import json
import os
from dataclasses import dataclass, asdict
from typing import Any
from urllib.parse import quote

import requests
from bs4 import BeautifulSoup

TIMEOUT = (10, 30)
IMDB_BASE = "https://www.imdb.com"
PROXIESAPI_KEY = os.getenv("PROXIESAPI_KEY", "").strip()

session = requests.Session()
session.headers.update(
    {
        "User-Agent": (
            "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
            "AppleWebKit/537.36 (KHTML, like Gecko) "
            "Chrome/137.0.0.0 Safari/537.36"
        ),
        "Accept-Language": "en-US,en;q=0.9",
    }
)


def suggestion_url(query: str) -> str:
    q = query.strip().lower().replace(" ", "_")
    if not q:
        raise ValueError("query cannot be empty")
    return f"https://v3.sg.media-imdb.com/suggestion/{q[0]}/{quote(q)}.json"


def fetch_json(url: str) -> dict[str, Any]:
    response = session.get(url, timeout=TIMEOUT)
    response.raise_for_status()
    return response.json()


@dataclass
class SuggestionCard:
    imdb_id: str
    title: str
    title_type: str | None
    year: int | None
    cast_snippet: str | None
    rank: int | None
    title_url: str


def parse_cards(payload: dict[str, Any]) -> list[SuggestionCard]:
    cards: list[SuggestionCard] = []

    for item in payload.get("d", []):
        imdb_id = item.get("id")
        title = item.get("l")
        if not imdb_id or not title:
            continue

        cards.append(
            SuggestionCard(
                imdb_id=imdb_id,
                title=title,
                title_type=item.get("q"),
                year=item.get("y") if isinstance(item.get("y"), int) else None,
                cast_snippet=item.get("s"),
                rank=item.get("rank") if isinstance(item.get("rank"), int) else None,
                title_url=f"{IMDB_BASE}/title/{imdb_id}/",
            )
        )

    return cards

Quick test:

payload = fetch_json(suggestion_url("matrix"))
cards = parse_cards(payload)
print(asdict(cards[0]))

Typical output:

{
  'imdb_id': 'tt0133093',
  'title': 'The Matrix',
  'title_type': 'feature',
  'year': 1999,
  'cast_snippet': 'Keanu Reeves, Laurence Fishburne',
  'rank': 296,
  'title_url': 'https://www.imdb.com/title/tt0133093/'
}

Step 2: Route title pages through ProxiesAPI when needed

Direct requests to IMDb title pages are often where people get stuck. The parser can stay simple if the fetch layer is honest about that risk.

def maybe_proxied_url(target_url: str) -> str:
    if not PROXIESAPI_KEY:
        return target_url
    return (
        "https://api.proxiesapi.com/?auth_key="
        + quote(PROXIESAPI_KEY, safe="")
        + "&url="
        + quote(target_url, safe="")
    )


def fetch_html(url: str) -> str:
    response = session.get(maybe_proxied_url(url), timeout=TIMEOUT)
    response.raise_for_status()
    html = response.text or ""
    if len(html) < 500:
        raise RuntimeError(f"unexpectedly short response for {url}")
    return html

Step 3: Parse structured metadata from `ld+json`

IMDb's rendered classes can move around. Structured data is a better first target.

def extract_ld_json(soup: BeautifulSoup) -> list[dict[str, Any]]:
    out: list[dict[str, Any]] = []
    for node in soup.select('script[type="application/ld+json"]'):
        text = node.string or node.get_text(strip=True)
        if not text:
            continue
        try:
            data = json.loads(text)
        except json.JSONDecodeError:
            continue

        if isinstance(data, dict):
            out.append(data)
        elif isinstance(data, list):
            out.extend(item for item in data if isinstance(item, dict))
    return out


def parse_title_metadata(html: str, imdb_id: str) -> dict[str, Any]:
    soup = BeautifulSoup(html, "lxml")

    for block in extract_ld_json(soup):
        if block.get("@type") not in {"Movie", "TVSeries", "TVEpisode"}:
            continue

        actors = []
        for actor in block.get("actor", []) or []:
            if isinstance(actor, dict) and actor.get("name"):
                actors.append(actor["name"])

        rating_value = None
        aggregate = block.get("aggregateRating")
        if isinstance(aggregate, dict):
            rating_value = aggregate.get("ratingValue")

        return {
            "imdb_id": imdb_id,
            "title": block.get("name"),
            "canonical_url": block.get("url") or f"{IMDB_BASE}/title/{imdb_id}/",
            "title_type": block.get("@type"),
            "release_date": block.get("datePublished"),
            "rating": rating_value,
            "cast": ", ".join(actors[:8]) if actors else None,
        }

    raise RuntimeError(f"no structured metadata found for {imdb_id}")

Step 4: Combine the two stages

def scrape_imdb(query: str, limit: int = 10) -> list[dict[str, Any]]:
    payload = fetch_json(suggestion_url(query))
    cards = parse_cards(payload)[:limit]

    rows: list[dict[str, Any]] = []
    for card in cards:
        row = asdict(card)
        try:
            html = fetch_html(card.title_url)
            row.update(parse_title_metadata(html, card.imdb_id))
        except Exception as exc:
            row["enrichment_error"] = str(exc)
        rows.append(row)

    return rows


def write_csv(rows: list[dict[str, Any]], path: str = "imdb_titles.csv") -> None:
    fieldnames = sorted({key for row in rows for key in row.keys()})
    with open(path, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(rows)


if __name__ == "__main__":
    rows = scrape_imdb("matrix", limit=5)
    write_csv(rows)
    print(rows[0])

Why this IMDb scraper holds up better

The key decision is not the parser library. It is the extraction order.

Bad approach:

hit a search results page
depend on visible class names for everything
break when the markup changes

Better approach:

use the suggestion JSON for discovery
use ld+json for semantic title metadata
keep the fetch layer proxy-aware

That gives you a scraper that is easier to debug and easier to extend.

Practical extensions

add genre and duration from the same ld+json block
persist daily snapshots so you can track rating movement over time
scrape multiple queries and dedupe by imdb_id
enrich only the top N results to control cost

If your actual goal is a movie-intelligence dataset rather than a brittle demo, this two-stage pipeline is the right place to start.

Use ProxiesAPI when IMDb title pages stop behaving like static HTML

IMDb often serves inconsistent responses to plain requests. ProxiesAPI gives you a clean way to stabilize title-page fetches while keeping the parsing code focused on structured data.

Get 1,000 free API calls View pricing

Use ProxiesAPI to collect IMDb search result cards, follow title pages, and export ratings, years, genres, and URLs.

tutorial#python#imdb#web-scraping

Scrape IMDb Top 250 Movies into a Dataset (Python + ProxiesAPI)

Extract IMDb Top 250 movies (rank, title, year, rating, vote count) into clean CSV/JSON — with robust parsing, retries, and polite crawling.

tutorial#python#imdb#web-scraping

Scrape IMDb Top 250 Movies into a Dataset

Pull rank, title, year, rating, and votes into clean CSV/JSON for analysis with working Python code.

tutorial#python#imdb#web-scraping

Amazon Best Sellers Scraper: Track Category Rankings and Price Moves

Scrape Amazon Best Sellers pages into repeatable snapshots, extract ranks and prices, and compute movement over time with a parser that is honest about block risk.

tutorial#amazon best sellers scraper#amazon#python

IMDb Scraper: Extract Movie Ratings, Cast, and Release Dates with Python

Related guides