Back to Library

How We Indexed 21.1M UK Property Records for Sub-15ms Search on a £5/mo VPS

• 9 min read
How We Indexed 21.1M UK Property Records for Sub-15ms Search on a £5/mo VPS

How We Indexed 21.1M UK Property Records for Sub-15ms Search on a £5/mo VPS

By AHM Labs Engineering Team • Published August 17, 2026

Address autocompletion is one of the most deceptively expensive bottlenecks in modern e-commerce and SaaS. Every checkout form, user onboarding flow, and shipping calculator depends on it.

Yet for years, UK developers have faced an absurd pricing disconnect: incumbent address providers charge £1.50 to £5.00 per 1,000 lookups while introducing 150ms–300ms round-trip latency to query data that is largely funded by UK taxpayers.

At AHM Labs, we decided to solve this from first principles. We engineered FastAddress UK—a single-purpose, deterministic search engine in Go that indexes over 21.1 million UK property records with sub-15ms global P50 latencies, running comfortably on a standard £5/month bare-metal VPS.

Here is the complete engineering teardown of the data pipeline, database architecture, edge caching, and zero-friction React SDK.

1. The Incumbent Tax: Why Are Legacy Address APIs So Slow and Expensive?

Most legacy UK address lookup tools rely on commercial licensing of the Royal Mail Postcode Address File (PAF). While PAF is an essential dataset for bulk commercial sorting discounts, the vast majority of web applications only need:

  1. Fast, typo-tolerant address suggestions as the user types.
  2. Standardized street, town, and postal code formatting.
  3. Official Ordnance Survey Unique Property Reference Numbers (UPRN) and GPS coordinates.

Incumbents wrap these queries in heavy Java or .NET enterprise middleware, query bloated SQL clusters, and bill developers per keystroke lookup. When a user types a 7-character postcode, that’s often 5 to 7 billable API hits at 200ms latency each.

Legacy APIs:   [User Keystroke] ──► [Heavy API Gateway] ──► [Remote SQL Cluster] (180ms - 320ms)
FastAddress:   [User Keystroke] ──► [Edge LRU Cache]   ──► [Go SQLite FTS5]      (< 15ms)

2. Ingesting & Normalizing 21.1M Public Records

Rather than paying recurring licensing ransoms, we collated and cleansed the United Kingdom’s primary open government datasets under the Open Government Licence v3.0 (OGL) and Crown Copyright:

  • Domestic Energy Performance Certificate (EPC) Register: Over 21+ million physical property certificates detailing address lines, building numbers, and postcodes.
  • Ordnance Survey OpenData & Open Identifiers: Definitive 12-digit Unique Property Reference Numbers (UPRN) and WGS84 GPS coordinate centroids.
  • OpenStreetMap (OSM) GB Footprints: Physical building centroid vectors for missing residential units.
  • HM Land Registry Price Paid Open Data: Definitive property title designations across England and Wales.

The Normalization Bottleneck

Raw address records in the UK are notoriously inconsistent. Assessor records contain dozens of permutations for the same flat or unit:

  • "Flat 1, 12 High Street" vs. "1 (Flat), 12 High Street" vs. "First Floor Flat, 12 High Street"
  • Inconsistent comma delimiters and spacing variations (SW1A1AA vs SW1A 1AA).

We wrote a custom Go streaming ingest CLI (cmd/ingest-epc/main.go) that processes gigabyte-scale gzip archives in chunks:

// Ingest pipeline: Normalizing flat numbering patterns
func cleanAndNormalizeAddress(raw string) string {
    raw = strings.TrimSpace(raw)
    if strings.Contains(strings.ToLower(raw), " (flat)") {
        raw = strings.Replace(raw, " (Flat)", "", -1)
        raw = strings.Replace(raw, " (flat)", "", -1)
        raw = "Flat " + raw
    }
    raw = strings.Replace(raw, ", ,", ",", -1)
    return strings.TrimSuffix(strings.TrimSpace(raw), ",")
}

An in-memory MD5 hash ring ensures duplicate entries between OSM, Land Registry, and EPC datasets are eliminated before writing to disk.

3. Database Architecture: Why SQLite FTS5 Crushes Traditional Clusters

Instead of running an expensive Elasticsearch, Meilisearch, or PostgreSQL cluster, we compiled the entire 21.1M dataset into a single SQLite database with the FTS5 (Full-Text Search 5) extension.

The FTS5 Virtual Table Schema

CREATE VIRTUAL TABLE v_addresses USING fts5(
    id UNINDEXED,
    address_string,
    postcode,
    street_line,
    latitude UNINDEXED,
    longitude UNINDEXED,
    uprn UNINDEXED,
    tokenize="porter unicode61 remove_diacritics 2"
);

Why This Outperforms Dedicated Search Clusters on Cheap VPS Hardware:

  1. Zero Network Serialization Overhead: The Go binary talks directly to SQLite via memory pointers using the pure-Go Turso driver. There is zero TCP loopback or HTTP serialization latency between the search engine and the database.
  2. OS Page Cache & Memory Mapping (mmap): By enabling SQLite PRAGMA mmap_size = 268435456 (256MB) and WAL mode (PRAGMA journal_mode = WAL), the operating system maps hot index pages directly into RAM. Reads never touch physical SSD storage once cached.
  3. Prefix Tokenizer Optimization: The porter unicode61 tokenizer splits alphanumeric tokens so queries like SW1A 2 or 10 Downing match prefixes instantly using B-Tree index lookups in under 1.2ms.

4. Concurrency & Benchmark Results

We built an automated spike load tester (cmd/benchmark/main.go) that hammered the Go server with 3,000 concurrent simulated retail checkouts:

Benchmark MetricMeasured ResultIndustry Incumbent Median
Simulated Concurrency3,000 req/sec~250 req/sec (rate limited)
P50 Query Latency1.2 ms120 ms – 180 ms
P95 Query Latency2.8 ms220 ms – 290 ms
P99 Query Latency4.8 ms450 ms+
Server Idle RAM< 65 MB> 2,048 MB (JVM / Elastic)
Server Peak RAM (3k load)< 180 MB> 4,096 MB

You can inspect live, real-world comparative telemetry anytime on the FastAddress Live Benchmarks Page.

5. Shipping the Developer Experience: The Zero-Dependency React SDK

Having a fast API is useless if integrating it into a checkout form is painful. Developers hate writing custom debounce timers, race-condition cancellations, and keyboard-accessible combobox menus.

We published the official zero-dependency package @ahm-labs/fast-address-uk (under 2.5KB gzipped) featuring:

1. Drop-in Accessible Combobox (AddressAutocomplete)

Fully compliant with WAI-ARIA 1.2 specifications, complete with keyboard navigation (ArrowUp, ArrowDown, Enter, Escape):

import { AddressAutocomplete } from '@ahm-labs/fast-address-uk/react';

export function CheckoutPage() {
  return (
    <AddressAutocomplete
      apiKey="demo"
      placeholder="Start typing postcode (e.g. SW1A 1AA)..."
      onSelect={(address) => {
        console.log('Selected UPRN:', address.uprn);
        console.log('Coordinates:', address.latitude, address.longitude);
      }}
    />
  );
}

2. Client-Side In-Memory LRU Caching

The SDK includes a built-in 50-item LRU cache. If a user types "SW1A", deletes a character to "SW1", and types "SW1A" again, 0 network requests are sent. It resolves in 0ms client-side, saving API quota and eliminating latency.

6. Real-World Integration & Next Steps

By combining open government data, Go concurrency, SQLite FTS5, and modern edge proxies, we proved that high-performance infrastructure doesn’t need expensive cloud bills or monopoly pricing.

Back to Library