How we score a tender

Introduction#

A firm that sells to the public sector does not have a discovery problem. Portals publish more notices every morning than any bid team can read, and a keyword alert on "rail" or "water" returns hundreds of them. The problem is judgement: which of these is worth the two weeks it costs to bid, and can that judgement be defended to a partner who asks why the team passed on something that was later won by a competitor.

We score every tender against every firm and attach the reasoning to the score. This post describes how that number is produced. It is not a claim that the number is correct - we say what it measures, and at the end, what it does not.

The model#

A single similarity score cannot express procurement fit, because the dimensions of fit contradict each other. A tender can be an excellent semantic match for what a firm does and still be a bad bid: wrong country, above any budget band the firm has delivered in, or in a market its compliance position rules out. Ranking by one number puts the unwinnable tender above the winnable one whenever the unwinnable one is better written.

We therefore compute ten signals independently. Each answers one question about the firm and tender pair, each returns a value or an explicit failure, and each carries a weight. The semantic signal is cosine similarity between stored embeddings, retrieved through pgvector^[1]; the sector signal scores the notice against the 45 divisions of the EU Common Procurement Vocabulary^[2]; the embeddings themselves are 1024-dimension vectors from a provider chosen per environment, including the open bge-m3 model^[3] in development.

The weights sum to more than one. They are relative importances, renormalised over whichever signals actually returned a value, which is the first of the two decisions worth explaining.

Redistribution, not zero#

A signal that cannot be computed returns a missing marker, and we redistribute its weight across the signals that did fire. We do not score it zero. Scoring it zero would conflate two statements that have nothing to do with each other - "we checked, and this is a poor fit" and "we could not check" - and only the first is evidence.

The consequence of getting this wrong is not subtle. A firm that has not yet uploaded reference documents would be penalised on the reference signal for every tender in the corpus, and the ranking it saw would be a ranking of its own profile completeness rather than of the market. The country-risk service is the clearest case in the other direction: we call it with a 2s timeout, and when it does not answer, its 0.08 is spread across the rest. What degrades is our confidence in the score, not the tender position.

Calibration before weighting#

Raw signals do not share a scale, so weighting them directly lets a signal variance rather than its assigned weight drive the ranking. Cosine similarity between two pieces of procurement text does not use the full range - it clusters in a narrow band well above zero. A geographic overlap score has a different floor again. Summing those raw values against their weights would let the widest-spread signal dominate whatever the weights said.

We therefore normalise each signal against its own empirical range before weighting: the semantic signal against bounds of 0.15/0.95, the geographic signal against a floor of 0.33. Composite scores below 0.5 are filtered rather than shown as weak matches.

What requirement coverage checks#

Requirement coverage is the signal that does the most work in a bid conversation, so it is worth stating precisely what it does. We extract the individual requirements a tender states - skills, certifications, geographic experience, minimum track record - and check the firm profile and documents for evidence of each one.

This is the difference between a hunch and a qualification check. Semantic similarity says a tender looks relevant. Requirement coverage says the tender demands nine things, the firm has evidence for seven, and names the two it does not. Extraction runs at temperature zero into a structured schema, and every extracted claim must be backed by a verified span of at least 10 characters from the source text. A fabricated requirement is worse than a missed one, because a missed requirement costs a bid and a fabricated one costs the reader trust in every other requirement on the list.

Where the cost falls#

The ensemble is expensive and browsing must be free, so the two are separated. A cron job runs the full ensemble for every firm at 06:00 UTC and writes results to a per-firm cache, which makes the morning shortlist a database read. Saved filter views re-slice that cache in SQL, so switching one costs nothing and returns immediately. The cross-encoder that reranks a shortlist is skipped when pre-rerank similarity is below 0.20, because reranking near-zero candidates is expensive and changes no ordering. The one on-demand exception is promoting a single notice, which runs the whole orchestrator on it including live requirement extraction.

Discussion#

Limitations#

The weights are chosen, not learned. They are a considered prior about what matters in procurement fit, and they are identical for every firm on the platform - a firm whose real constraint is team capacity rather than sector fit is scored by the same 0.05 on capacity as everyone else. We log every scoring run and map user actions to labels against it, which is the data a learned weighting would need, but the weights in Table 1 are not currently fitted to it.

The calibration bounds are empirical constants rather than per-corpus statistics. They were set against the distribution the corpus had when they were chosen, and a corpus that shifts substantially - a large new source in a different language, for instance - would drift away from them before anything alerted us.

The score measures fit, and fit is not the same as winnability. We do not model the competitive field: two firms with identical profiles receive identical scores on a tender only one of them can win. Nor do we model incumbency, which in public procurement is often the single largest determinant of who wins a re-tendered contract. A high score means the tender matches what the firm can prove it has done. It does not mean the firm will win it, and we do not present it as a probability.

Requirement coverage is bounded by what the firm has uploaded. A capability held by a team but not evidenced in any document is invisible to it, and the score will understate fit. That failure is quiet, which makes it the one we would most like to surface and currently do not.

The most important limitation is that we have not published an evaluation, because we have not done one. There is no held-out set, no reported precision at any cut-off, and no measured agreement between the ranking and a bid manager who ranked the same tenders by hand. Everything above describes what the ensemble computes and why each part is shaped the way it is. None of it is evidence that the ordering is good, and a reader should not take it as such.

The labels that an evaluation would need are being collected. Every scoring run is logged with its signal values, and user actions on a tender map to labels against that run. They are a weaker signal than they look: a dismissal can mean the match was wrong, or that the firm had no capacity that quarter, or that somebody was clearing a backlog, and nothing distinguishes those. An evaluation built on them would measure agreement with past behaviour rather than correctness, and past behaviour is exactly what a firm is paying us to improve on.

Why the breakdown is the product#

Given those limitations, the number itself is the less useful half of the output. What persists with it is the per-signal breakdown: which signals fired, what each returned, what weight each carried after redistribution. A bid manager defending a go decision six months later has the same evidence the engine had, and a bid manager overruling the engine can point at the signal they think is wrong. We would rather ship a score a bid manager can interrogate than one that looks more authoritative and cannot be questioned.

What would change the weights#

Fitting weights to the logged labels is the obvious next step and not the first one we would take, because of the confound above. The tractable experiment is narrower: ask bid managers to rank a set of tenders their firm has already seen, and measure rank correlation against the ensemble. That isolates the ordering question from the capacity and backlog noise in the action labels, and it needs tens of judgements rather than thousands of outcomes.

If that showed systematic disagreement on one signal, the fix would probably be per-sector weights rather than global learned ones. A firm bidding on framework agreements and a firm bidding on single large contracts are not weighting budget fit the same way, and one global vector cannot represent both.

Conclusion#

Ten signals, calibrated then weighted, with missing signals redistributing rather than penalising, and the whole breakdown persisted alongside the score. The design choices that matter are not the model - they are the decisions about what to do when a signal is absent, and about what the resulting number is allowed to claim. On the second question our current answer is deliberately modest: the score orders a feed and shows its working, and we have not yet measured whether the ordering is right.