Scrape Rightmove Sold Prices

Jun 11, 2026 · tutorial · #python, #rightmove, #real-estate, #web-scraping, #beautifulsoup, #csv, #proxiesapi

If you want to analyze UK property markets, Rightmove's sold-prices pages are one of the fastest ways to build a useful dataset.

The valuable part is that each card already exposes:

address
property type
bedroom count
tenure when available
sale date
sale price
detail page URL

That means you can build a clean sold-prices pipeline without rendering the page in a browser for every request.

In this walkthrough we will:

fetch a Rightmove sold-prices area page
parse listing cards using current selectors
follow pagination
export CSV/JSON
optionally fan out into detail pages later with ProxiesAPI

Rightmove sold prices results page with property cards and historical transactions

Keep sold-price crawls stable with ProxiesAPI

Rightmove dataset jobs combine pagination with many detail-page requests. ProxiesAPI helps keep those long runs from turning into a retry storm.

Get 1,000 free API calls View pricing

What the page looks like today

As of June 2026, a sold-prices page like:

https://www.rightmove.co.uk/house-prices/london.html

contains property cards with data-testid="propertyCard".

Inside each card, the important elements are:

Purpose	Selector / pattern
property card link	`a[data-testid="propertyCard"]`
address	`h2` inside the card
property badges	`div[aria-label^="Property Type:"]`, `div[aria-label^="Bedrooms:"]`, `div[aria-label^="Tenure:"]`
transaction rows	`table tbody tr` inside the card
current sale row	first transaction row whose text looks like a date + price
pagination nav	`nav[aria-label="Pagination Navigation"]`

The current markup also includes a transaction table inside each card. A real row looks like:

<tr>
  <td>31 Mar 2026</td>
  <td><div aria-label="£820,000.">£820,000</div></td>
</tr>

That is enough for a very solid first-pass dataset.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install requests beautifulsoup4 lxml tenacity

Step 1: Create a fetch helper with optional ProxiesAPI

from __future__ import annotations

import csv
import json
import os
import re
from dataclasses import dataclass, asdict
from typing import Iterable
from urllib.parse import quote, urljoin

import requests
from bs4 import BeautifulSoup
from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_exponential_jitter

TIMEOUT = (10, 30)
HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/136.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-GB,en;q=0.9",
}

session = requests.Session()
session.headers.update(HEADERS)


def build_fetch_url(target_url: str) -> str:
    api_key = os.getenv("PROXIESAPI_KEY", "").strip()
    if not api_key:
        return target_url
    return (
        "https://api.proxiesapi.com/?auth_key="
        + quote(api_key, safe="")
        + "&url="
        + quote(target_url, safe="")
    )


@retry(
    reraise=True,
    stop=stop_after_attempt(4),
    wait=wait_exponential_jitter(initial=1, max=12),
    retry=retry_if_exception_type(requests.RequestException),
)
def fetch_html(url: str) -> str:
    response = session.get(build_fetch_url(url), timeout=TIMEOUT)
    response.raise_for_status()
    return response.text

This is the same pattern as the Vinted scraper: direct by default, ProxiesAPI when PROXIESAPI_KEY is present.

Step 2: Parse the card-level fields

The current card already contains enough data to avoid opening every detail page on day one.

DATE_RE = re.compile(r"\d{1,2}\s+[A-Z][a-z]{2}\s+\d{4}")


@dataclass
class SoldPriceRow:
    address: str
    detail_url: str
    property_type: str | None
    bedrooms: str | None
    tenure: str | None
    sold_date: str | None
    sold_price: str | None


def get_badge_text(card, prefix: str) -> str | None:
    node = card.select_one(f'div[aria-label^="{prefix}"]')
    if not node:
        return None
    text = node.get_text(" ", strip=True)
    return text or None


def extract_transaction(card) -> tuple[str | None, str | None]:
    for row in card.select("table tbody tr"):
        cells = row.select("td")
        if len(cells) < 2:
            continue

        left = cells[0].get_text(" ", strip=True)
        right = cells[1].get_text(" ", strip=True)

        if DATE_RE.search(left) and "£" in right:
            return left, right

    return None, None


def parse_card(card) -> SoldPriceRow:
    href = card.get("href", "")
    detail_url = href if href.startswith("http") else urljoin("https://www.rightmove.co.uk", href)

    title = card.select_one("h2")
    sold_date, sold_price = extract_transaction(card)

    return SoldPriceRow(
        address=title.get_text(" ", strip=True) if title else "",
        detail_url=detail_url,
        property_type=get_badge_text(card, "Property Type:"),
        bedrooms=get_badge_text(card, "Bedrooms:"),
        tenure=get_badge_text(card, "Tenure:"),
        sold_date=sold_date,
        sold_price=sold_price,
    )

extract_transaction() deliberately skips the "Today / See what it's worth now" row and looks for the first real date + price pair.

Step 3: Parse one results page and pagination

def parse_results_page(html: str) -> tuple[list[SoldPriceRow], str | None]:
    soup = BeautifulSoup(html, "lxml")

    cards = soup.select('a[data-testid="propertyCard"]')
    rows = [parse_card(card) for card in cards]

    next_link = soup.select_one('nav[aria-label="Pagination Navigation"] a[aria-label="Next"]')
    next_href = next_link.get("href") if next_link else None
    if next_href and next_href.startswith("/"):
        next_href = urljoin("https://www.rightmove.co.uk", next_href)

    return rows, next_href

If you inspect the current page HTML, the paginator is a normal navigation element, so you do not need Playwright just to reach page 2 or 3.

Step 4: Crawl multiple pages

def crawl_area(area_url: str, max_pages: int = 5) -> list[SoldPriceRow]:
    rows: list[SoldPriceRow] = []
    seen: set[str] = set()
    next_url = area_url

    for page_number in range(1, max_pages + 1):
        html = fetch_html(next_url)
        batch, next_candidate = parse_results_page(html)

        for row in batch:
            if not row.detail_url or row.detail_url in seen:
                continue
            seen.add(row.detail_url)
            rows.append(row)

        print(f"page={page_number} batch={len(batch)} total={len(rows)}")

        if not next_candidate:
            break
        next_url = next_candidate

    return rows

Step 5: Export clean data

def save_csv(rows: Iterable[SoldPriceRow], path: str) -> None:
    rows = list(rows)
    if not rows:
        return
    with open(path, "w", newline="", encoding="utf-8") as fh:
        writer = csv.DictWriter(fh, fieldnames=list(asdict(rows[0]).keys()))
        writer.writeheader()
        for row in rows:
            writer.writerow(asdict(row))


def save_json(rows: Iterable[SoldPriceRow], path: str) -> None:
    with open(path, "w", encoding="utf-8") as fh:
        json.dump([asdict(row) for row in rows], fh, ensure_ascii=False, indent=2)

Full runnable script

from __future__ import annotations

import csv
import json
import os
import re
from dataclasses import dataclass, asdict
from typing import Iterable
from urllib.parse import quote, urljoin

import requests
from bs4 import BeautifulSoup
from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_exponential_jitter

BASE = "https://www.rightmove.co.uk"
TIMEOUT = (10, 30)
HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/136.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-GB,en;q=0.9",
}
DATE_RE = re.compile(r"\d{1,2}\s+[A-Z][a-z]{2}\s+\d{4}")

session = requests.Session()
session.headers.update(HEADERS)


@dataclass
class SoldPriceRow:
    address: str
    detail_url: str
    property_type: str | None
    bedrooms: str | None
    tenure: str | None
    sold_date: str | None
    sold_price: str | None


def build_fetch_url(target_url: str) -> str:
    api_key = os.getenv("PROXIESAPI_KEY", "").strip()
    if not api_key:
        return target_url
    return (
        "https://api.proxiesapi.com/?auth_key="
        + quote(api_key, safe="")
        + "&url="
        + quote(target_url, safe="")
    )


@retry(
    reraise=True,
    stop=stop_after_attempt(4),
    wait=wait_exponential_jitter(initial=1, max=12),
    retry=retry_if_exception_type(requests.RequestException),
)
def fetch_html(url: str) -> str:
    response = session.get(build_fetch_url(url), timeout=TIMEOUT)
    response.raise_for_status()
    return response.text


def get_badge_text(card, prefix: str) -> str | None:
    node = card.select_one(f'div[aria-label^="{prefix}"]')
    return node.get_text(" ", strip=True) if node else None


def extract_transaction(card) -> tuple[str | None, str | None]:
    for row in card.select("table tbody tr"):
        cells = row.select("td")
        if len(cells) < 2:
            continue

        left = cells[0].get_text(" ", strip=True)
        right = cells[1].get_text(" ", strip=True)

        if DATE_RE.search(left) and "£" in right:
            return left, right
    return None, None


def parse_card(card) -> SoldPriceRow:
    href = card.get("href", "")
    detail_url = href if href.startswith("http") else urljoin(BASE, href)

    title = card.select_one("h2")
    sold_date, sold_price = extract_transaction(card)

    return SoldPriceRow(
        address=title.get_text(" ", strip=True) if title else "",
        detail_url=detail_url,
        property_type=get_badge_text(card, "Property Type:"),
        bedrooms=get_badge_text(card, "Bedrooms:"),
        tenure=get_badge_text(card, "Tenure:"),
        sold_date=sold_date,
        sold_price=sold_price,
    )


def parse_results_page(html: str) -> tuple[list[SoldPriceRow], str | None]:
    soup = BeautifulSoup(html, "lxml")
    cards = soup.select('a[data-testid="propertyCard"]')
    rows = [parse_card(card) for card in cards]

    next_link = soup.select_one('nav[aria-label="Pagination Navigation"] a[aria-label="Next"]')
    next_href = next_link.get("href") if next_link else None
    if next_href and next_href.startswith("/"):
        next_href = urljoin(BASE, next_href)

    return rows, next_href


def crawl_area(area_url: str, max_pages: int = 3) -> list[SoldPriceRow]:
    next_url = area_url
    seen: set[str] = set()
    all_rows: list[SoldPriceRow] = []

    for page_number in range(1, max_pages + 1):
        html = fetch_html(next_url)
        batch, next_candidate = parse_results_page(html)

        for row in batch:
            if row.detail_url in seen:
                continue
            seen.add(row.detail_url)
            all_rows.append(row)

        print(f"page={page_number} batch={len(batch)} total={len(all_rows)}")

        if not next_candidate:
            break
        next_url = next_candidate

    return all_rows


def save_csv(rows: Iterable[SoldPriceRow], path: str) -> None:
    rows = list(rows)
    if not rows:
        return
    with open(path, "w", newline="", encoding="utf-8") as fh:
        writer = csv.DictWriter(fh, fieldnames=list(asdict(rows[0]).keys()))
        writer.writeheader()
        for row in rows:
            writer.writerow(asdict(row))


def save_json(rows: Iterable[SoldPriceRow], path: str) -> None:
    with open(path, "w", encoding="utf-8") as fh:
        json.dump([asdict(row) for row in rows], fh, ensure_ascii=False, indent=2)


if __name__ == "__main__":
    london_url = "https://www.rightmove.co.uk/house-prices/london.html"
    rows = crawl_area(london_url, max_pages=3)
    save_csv(rows, "rightmove_london_sold_prices.csv")
    save_json(rows, "rightmove_london_sold_prices.json")
    print(f"saved {len(rows)} rows")

Typical output:

page=1 batch=25 total=25
page=2 batch=25 total=50
page=3 batch=25 total=75
saved 75 rows

How to extend this into a richer dataset

The card-level scrape is enough for:

neighborhood pricing snapshots
recent comparable sales
property-type averages
bedroom-level filters

If you need more fields, the next step is to request each detail_url and extract:

broader sale history
estimated valuation text
image counts
nearby sold listings

That is exactly where ProxiesAPI becomes more valuable, because one area page turns into dozens or hundreds of detail requests very quickly.

Practical advice before you scale it

1. Keep the first pipeline shallow

Do not scrape every detail page on day one. First make sure:

the card parser is stable
your dedupe key works
pagination finishes cleanly
your export is correct

2. Treat selectors as contracts

The best Rightmove hooks today are:

data-testid="propertyCard"
the transaction table rows
ARIA labels on the property badges

Those are more reliable than brittle generated CSS class names.

3. Expect partial cards

Not every card exposes every badge. Tenure is a good example. Your parser should return None, not crash.

4. Add backoff before you add concurrency

If you later fan out to detail pages, the usual mistake is raising concurrency too early. A smaller worker pool with retries and jitter is much healthier than a huge burst that triggers blocks.

Rightmove sold-prices pages are friendly enough to support a clean HTML-first scraper. The current card structure already gives you address, property metadata, and at least one real transaction row, which is more than enough to build a useful property dataset.

Validate the result pages first. Once you are ready to crawl deeper and more often, route those jobs through ProxiesAPI so the dataset builder stays reliable instead of brittle.

Keep sold-price crawls stable with ProxiesAPI

Rightmove dataset jobs combine pagination with many detail-page requests. ProxiesAPI helps keep those long runs from turning into a retry storm.

Get 1,000 free API calls View pricing

Show how to collect Rightmove listing prices, addresses, agent names, and URLs into a reusable UK property dataset with Python and ProxiesAPI.

tutorial#python#rightmove#real-estate

Scrape Rightmove Sold Prices

Build a sold-price dataset with Rightmove property cards, detail pages, sale dates, and historical prices using real selectors and a screenshot.

tutorial#python#rightmove#real-estate

Scrape UK Property Prices from Rightmove (Dataset Builder)

Build a sold-price dataset from Rightmove: crawl results, follow listing links, extract key fields, handle retries, and export to CSV using ProxiesAPI.

tutorial#python#rightmove#real-estate

Scrape UK Property Prices from Rightmove (Dataset Builder + Screenshots)