Choosing which past projects to cite
Every bid names a handful of past projects as evidence that a firm can do the work. Choosing them well matters and is hard to evaluate, because the counterfactual is unobservable: nobody knows which bid the other four references would have won. We rank candidates by semantic similarity discounted for age, rerank on four explicit factors, and are candid that the objective we optimise is a proxy for the one that matters.
Published
The problem
An expression of interest carries a short list of past projects, and the list is doing real work: it is the firm answer to "why you". A bid manager assembling it by hand is running a ranking over everything the firm has ever delivered, against a tender they read this morning, using judgement they cannot fully articulate. It takes an afternoon and the result is uneven, because the projects that come to mind are the recent ones and the ones the bid manager personally worked on.
The problem is well posed as ranking and badly posed as learning, because there is no label. A won bid does not tell you the references were good, and a lost bid does not tell you they were bad. Nobody observes the counterfactual.
Method
Candidates are retrieved from the firm own document corpus by semantic similarity against the tender, using hybrid dense and sparse retrieval with a Postgres fallback through pgvector[1], filtered by sector and geography where the tender states them.
Similarity alone ranks a well-written project from fifteen years ago above a closely comparable one from last year, so retrieval applies a temporal discount: a 10-year half-life on the age of the delivered work. A project whose year we cannot determine is not excluded and is not treated as current. It receives a fixed factor of 0.7, which places it below a recent project and above an old one.
The discounted shortlist is then reranked on four factors we can name, rather than on one score we cannot decompose.
| Factor | Asks |
|---|---|
| Geography | Was this delivered somewhere relevant to where the work is |
| Sector | Is it the same kind of work, not merely similar language |
| Recency | How much has the discount already reduced it |
| Scale | Is it comparable in size to what is being procured |
A model rerank over the top 12 candidates is available on top of that. We cap it deliberately: a cross-encoder over the whole corpus costs far more and reorders almost nothing below the shortlist, because the deterministic factors have already separated the plausible from the implausible.
Discussion
Limitations
The objective is a proxy. We rank for similarity to the tender, discounted for age and adjusted for four factors, and what a bid needs is persuasiveness to an evaluator against a scoring rubric. Those correlate and are not the same. A project that is somewhat less similar but was delivered for the same buying authority is often the stronger reference, and nothing in the ranking knows that.
The decay curve is a heuristic with no empirical basis we can point at. A 10-year half-life is a plausible rate at which delivered work stops being persuasive, chosen because it produces sensible orderings, not because it was fitted to anything. Procurement evaluators frequently state their own recency windows in the tender documents, and those windows vary by sector; a single global curve cannot express that and we do not currently read the stated window.
The unknown-year default of 0.7 is the weakest number in the system. It is applied to projects whose date we failed to extract, which is not a random sample of projects. Older documents are likelier to be scanned, and scanned documents are likelier to lose their dates, so the default probably favours exactly the projects the decay curve is meant to discount.
The four factors are computed independently and combined without interaction terms. Scale and sector plainly interact - a small project in a specialist sector can be strong evidence where a small project in a commodity sector is not - and the ranking cannot express that.
Finally, we evaluate this by inspection. There is no held-out set, no measured agreement with bid managers, and no win-rate comparison, because assembling any of those requires outcome data at a volume we do not have. Anyone reading this note should treat the design as reasoned rather than validated.
What would make this measurable
The tractable experiment is not win rate, which is confounded by everything else in a bid. It is agreement: put the ranked shortlist in front of bid managers who then assemble the list by hand, and measure overlap and rank correlation. That produces a signal without needing outcomes, and it would tell us whether the four factors are the right four before anyone tries to fit weights to them. Reading the recency window a tender actually states, rather than assuming one, is the change we would make first regardless.
Conclusion
Retrieval, an age discount, four named factors, and a capped model rerank. Every component is separately inspectable, which is deliberate: while the objective is a proxy and the evaluation is qualitative, a bid manager who disagrees with the shortlist should be able to see which factor produced it rather than being asked to trust a number.
Stated plainly, because the distinction matters more here than in most of what we write: this is a reasoned ordering, not a validated one. Nothing above establishes that the shortlist is better than the one a bid manager would assemble unaided, and the agreement study in the previous section is the thing that would settle it. It has not been run.
References
- [1]pgvector: open-source vector similarity search for Postgres. pgvector contributors. The extension the corpus is retrieved from by cosine similarity.