HomeTechProjectsAboutSubscribe
Home/Projects/Post

How I Built a Real-Time Price Aggregator for Collectibles

grailhaus home paage

Pricing a sports card is genuinely annoying. You search eBay, find five sold listings, three of which are clearly wrong — different grade, different year, someone titled their auction "Mike Trout Lot" and it technically contains the card you want. You open COMC in another tab. Then Goldin. Then you're fifteen minutes in and you still don't have a number you trust.

The problem isn't access to data — there's plenty of it. The problem is that it's scattered across six different platforms that each have their own search UX, their own concept of what "sold" means, and their own noise-to-signal ratio. Goldin runs high-end auctions with strong realized prices. COMC is fixed-price inventory, useful for a different reason. 130point shows you Best Offer sales that eBay's own API doesn't even surface. If you want an accurate comp, you need all of them.

That's what Grailhaus is: a single search that queries seven marketplaces simultaneously, scores every result against what you actually searched for, weights recent sales more heavily than old ones, and gives you a number you can trust. This post gets into how it works.

What Grailhaus Actually Does

Before the architecture, it helps to understand what the product is from the outside.

You type a query — something like "2020 Prizm Justin Herbert PSA 10 RC" — and the results start streaming in before the page finishes loading. eBay comes back first, usually within a second. PriceCharting right after. The Puppeteer scrapers — COMC, Goldin, PWCC, MySlabs, 130point — take longer, anywhere from three to twelve seconds depending on how cooperative the sites feel that day. As each source resolves, its results appear on screen with a match score and a sale date.

At the top you get an estimated value: a single number derived from all the sales weighted by how closely they match and how recently they happened. Below that, a table of individual comps sorted by relevance. On the right, a price history chart showing how the card has moved over time.

If you're logged in, you can save the card to a collection. Saved cards get their prices refreshed daily by a background cron job, and you can see the full price history going back as far as Grailhaus has data for them.

The whole thing is built on Next.js API routes with no separate backend server.

Seven Sources, Two Tiers

The data layer divides cleanly into two groups.

eBay and PriceCharting are proper APIs. eBay's Browse API gives you active listings; the Marketplace Insights API gives you completed sales. Both get called on every search. eBay imposes a 5,000 request-per-day global limit — that number gets tracked in Redis, and searches that would push past it fall back to cached results rather than failing visibly. PriceCharting provides grade-level aggregated values scraped from eBay's completed listings — lower fidelity than hitting eBay directly, but the data is clean and the response is fast.

The other five — COMC, Goldin, PWCC, MySlabs, and 130point — require scraping. None of them have public APIs. Each one spins up a headless Chromium instance via Puppeteer, loads the sold listings page for the query, and parses the DOM. This is the slow, fragile, operationally annoying part of the project, and it's also why Grailhaus surfaces information that eBay search doesn't.

130point is the most valuable of the scraper sources. eBay's completed listings API hides Best Offer sale prices — you know a sale happened, but not what it closed at. 130point aggregates those hidden prices. For actively-traded cards, it's often the most accurate signal in the set.

COMC and Goldin serve different purposes. COMC is fixed-price inventory — its numbers tell you what sellers think the card is worth right now. Goldin is auction-realized prices from serious buyers at the high end of the market. Together they bracket the range.

All seven run in parallel. There's no waterfall, no waiting for eBay to finish before starting the scrapers. A Promise.allSettled catches failures individually so one hung scraper doesn't block the others.

The Card Parser

Before any API calls happen, the query string needs to become structured data. "2020 Prizm Justin Herbert PSA 10 RC" isn't useful to a search algorithm. But this is:

{
  originalQuery: "2020 Prizm Justin Herbert PSA 10 RC",
  normalizedQuery: "justin herbert|year:2020|set:prizm|grade:psa-10|rc:true",
  playerName: "Justin Herbert",
  year: 2020,
  setName: "Prizm",
  grader: "PSA",
  grade: 10,
  isRookie: true,
  category: "football"
}

Card collectors don't type structured queries. They type things like:

2020 Prizm Justin Herbert PSA 10 RC
Pikachu Illustrator PSA 10
1986 Fleer Jordan /50 BGS 9.5

There's no standard format. The year might come first or last. The grade might be written as "PSA 10" or "psa10" or "10 PSA". Some queries include a set name, some don't. Pokemon cards have entirely different vocabulary than baseball cards.

The parser runs eight extraction steps in sequence, each one narrowing down the working string:

Grade first. PSA 10, BGS 9.5, SGC 9 — these are the most specific attributes and the easiest to extract with regex. Pull them out and strip them from the string so they don't confuse later steps.

Year next. Four-digit years between 1950 and 2029, or two-digit shorthand like '23. Years stay in the string because they're useful context for downstream searches.

Numbering. The /99, /10, /1 pattern indicates print run. A /1 one-of-one is worth roughly 20x the base card. This matters enough that it needs its own field.

Parallel type. Here's where it gets tricky. "Gold Refractor" is a specific parallel. "Gold" alone is different. "Refractor" alone is different again. The parser checks compound patterns before simple ones — if you check for "Refractor" first, you'd incorrectly match "Gold Refractor" as just "Refractor". The same logic applies to Pokemon: VMAX, VSTAR, Alt Art, and Full Art all need to match as units.

Everything else — rookie, auto, numbered, category (sport/Pokemon/etc.), set name normalization — follows the same principle: specific before general, pull each thing out cleanly so what's left is recognizably a player name.

The normalizedQuery field is the cache key. "Justin Herbert 2020 Prizm PSA 10 RC" and "2020 RC PSA 10 Prizm Justin Herbert" produce the same normalized key, so the second search is a free cache hit. Without normalization you'd be making fresh API calls for every variation of the same query.

Scoring: Which Sales Actually Match

Seven sources returning raw results still gives you noise. eBay especially — seller titles are written for SEO, so a search for "Justin Herbert Prizm PSA 10" might return a "2019 Panini Prizm Justin Herbert PSA 9" because all those words are in the title. That's not a comparable sale.

Every returned sale gets a match score between 0 and 1. The score is a weighted sum:

Attribute           | Weight
--------------------|--------
Player name         | 30 pts
Year                | 20 pts
Set name            | 15 pts
Card number/variant | 15 pts
Grade               | 15 pts
Parallel            |  5 pts

Each attribute scores 0, 0.5, or 1.0. Player name either matches (1.0), partially matches on token overlap (0.5), or doesn't (0). Grade requires exact grader plus numeric grade for a full point, grader-only for half. Set name uses fuzzy matching to handle "Prizm" vs "Panini Prizm" vs "Prizm Football".

The parallel weight isn't fixed at 5. For a /1 one-of-one, it bumps to 25 and every other attribute scales down proportionally so the total stays at 100. The price difference between a /1 Gold Refractor and the base card can be 20x — if the parallel doesn't match, that sale is almost worthless as a comp, and the scoring reflects that.

Any sale scoring below 0.6 gets dropped. What remains are results that are genuinely comparable to the card you searched for.

The Recency Problem

A close match from two years ago and a weak match from last week aren't equally useful, but neither is a blanket "newer is better" rule. The right decay rate depends on the market.

Football card prices swing hard on game performance, injury news, and playoff runs. A Patrick Mahomes comp from eight months ago tells you almost nothing about today's price. Pokemon is different — a Charizard from 2022 is still a reasonable reference point. Vintage baseball barely moves at all.

The solution is exponential decay with a category-specific constant:

weight = e^(-λ × daysSinceSale)

Where λ varies:

  • Football, basketball, UFC: 0.04 — a 30-day-old sale weighs about 0.30
  • Baseball, hockey: 0.03 — same sale weighs about 0.41
  • Pokemon, non-sports: 0.02 — same sale weighs about 0.55
  • Pre-2000 vintage: 0.01 — same sale weighs about 0.74

The final estimated value uses both the match score and recency weight together:

estimatedValue = Σ(matchScore × recencyWeight × price) / Σ(matchScore × recencyWeight)

A sale that's a close match and happened recently dominates the estimate. An old sale with a weak match barely registers. This is why the comps actually feel trustworthy — you're not averaging everything, you're averaging the things that matter.

The Scraper Layer

The five Puppeteer scrapers are where most of the operational complexity lives.

Each scraper uses puppeteer-core with the puppeteer-extra-plugin-stealth plugin, which patches enough browser fingerprinting signals to avoid most bot detection: navigator.webdriver is hidden, Chrome's runtime is spoofed, WebGL and Canvas fingerprints are randomized. It's not foolproof, but it gets past the lightweight checks most collector platforms run.

Every scraper has an independent timeout. If COMC doesn't respond within 12 seconds, it fails gracefully — its slot in the results shows a timeout error and the other six sources continue normally. The Promise.allSettled pattern means a hung browser instance can't hold up the whole response.

Goldin and PWCC are auction platforms, which introduces a wrinkle: auction results pages are dynamic, meaning the data you want is rendered by JavaScript after the initial page load. Puppeteer handles this with waitForSelector calls targeting the specific DOM elements that indicate the results table has loaded. The selector strategy is brittle by nature — a front-end redesign can break a scraper — so the scrapers are written to fail loudly rather than silently return wrong data.

130point is the most technically interesting scraper. eBay completed listings show a sale occurred but mask the final price for Best Offer transactions, replacing it with a dash. 130point aggregates these hidden prices by pulling from a different data feed. The scraper parses its results table and maps each row back to the normalized card attributes, which means running the parser again in reverse on the sale title.

Rate limiting varies by source. COMC is tolerant. Goldin is not — more than a few requests per minute triggers a 429. The scrapers don't currently implement per-source throttling beyond request timeouts, which means high-traffic periods can occasionally result in temporary blocks. It's on the backlog.

Making It Feel Fast

A cold search — no cache, all seven sources — takes anywhere from 5 to 15 seconds. That's unavoidable when you're waiting on Puppeteer to load Goldin's auction results.

Two things make it feel faster than it is.

Streaming. The search API runs in NDJSON streaming mode. Each source emits its results as a newline-delimited JSON object the moment it finishes, rather than buffering until all seven are done:

{"type":"source","source":"ebay","sales":[...],"stats":{...}}
{"type":"source","source":"pricecharting","sales":[...],"stats":{...}}
{"type":"source","source":"comc","success":false,"error":"Timeout"}
...
{"type":"done","aggregatedStats":{...},"totalSales":89}

eBay typically resolves in under a second. The UI starts rendering real comps immediately while the scrapers are still running. The done event carries the final aggregated stats, which is when the estimated value at the top of the page locks in. The progress indicator on each source tile reflects its actual state — loading, complete, or failed — in real time.

Caching. Once a search has been run, the result is stored in Redis for 24 hours keyed to the normalized query string. "Justin Herbert 2020 Prizm PSA 10" and "PSA 10 RC Justin Herbert Prizm 2020" hit the same cache key. Cache hits skip all seven sources entirely, return in under 100ms, and don't count against the eBay daily request limit.

The cache stores a trimmed version of the result. Seven sources returning 100 results each is around 300KB of JSON. The cache keeps the top 20 sales by match score plus the aggregated stats — about 5-10KB per entry. A lightweight reconstruction step re-derives any computed fields the client needs before returning.

There's also a circuit breaker on the Redis connection. Two consecutive failures trigger a five-minute cooldown where the app bypasses the cache entirely rather than hanging on every request. Searches still work during the window — just slower. A caching layer going down shouldn't take the whole product with it.

Tracking Collections

Beyond one-off price checks, Grailhaus supports persistent card collections. Logged-in users can save any card they search for — either as something they own or something they're watching — and the app maintains a price history for each one.

The data model is straightforward: a cards table stores the parsed card attributes, a user_cards table links users to cards with an ownership flag and their purchase price if applicable, and a price_history table records the daily estimated value for each card.

price_history (
  card_id     uuid references cards(id),
  estimated   numeric(10,2),
  sample_size integer,
  recorded_at date,
  primary key (card_id, recorded_at)
)

A Vercel cron job runs at 2 AM daily and refreshes prices for every card that has at least one active follower. It runs the full search pipeline — same parser, same sources, same scoring — and writes a new row to price_history. The cron runs under a service account with elevated eBay API quota, so it doesn't eat into the per-user daily limit.

The collection page shows each card with a 30-day sparkline built with Recharts. A card that's moved significantly in either direction gets flagged. The intent is eventually to support price alerts, but for now it's read-only.

Deploying It

The entire backend runs on Vercel. No separate server, no container orchestration. The scrapers are serverless functions — each search request spins up whatever Puppeteer instances it needs and they're gone when it's done.

One real gotcha with serverless Puppeteer: Vercel doesn't let you install system dependencies, so the full puppeteer package (which bundles a Chromium download) fails at build time. The solution is puppeteer-core paired with @sparticuz/chromium, a package that provides a compressed Chromium binary small enough to fit in a serverless function. A small browser utility switches between the local Chromium path (for development) and the Lambda path (for production) based on the environment.

The infrastructure footprint is minimal:

  • Neon (serverless Postgres) for the database
  • Upstash (serverless Redis) for caching and rate limiting
  • eBay developer account for API credentials
  • An AUTH_SECRET env var so NextAuth's middleware initializes correctly

Everything else — Stripe, transactional email, price alerts — can be left blank for a basic search-only deployment. The app degrades cleanly when optional integrations are absent.

The part of this that surprised me most was how much the scoring system mattered. The naive version of this problem is "query all the sources and average the prices." That produces garbage results — you're averaging unrelated sales and calling it a comp. The match scoring and recency weighting are what turn a price aggregator into something you'd actually trust. Without them, Grailhaus would just be a more complicated way to get the wrong number.

Newsletter

Enjoyed This? Get the Next One.

One email when something worth reading drops. No spam, ever.

Transmission

Read It Before
Everyone Else.

Tech posts and project updates, straight to your inbox. No algorithm deciding what you see. Just the good stuff, direct.

No spam Weekly Unsubscribe anytime

Powered by Beehiiv · Your email stays yours.