Turning a firm document into a capability with evidence attached

The problem#

A matching engine that scores a firm against a tender needs a machine-readable answer to "what can this firm prove it has done". Firms have that answer, but it is distributed across project references, detail sheets, consultant CVs and past submissions, written in prose, as PDFs. Asking a firm to type its capabilities into a form produces a list of aspirations; deriving them from the documents produces a list of claims that may or may not be supported.

We want the third thing: a capability that carries the evidence for itself, so a downstream consumer can weight it, and a human can check it.

Method#

A firm document is parsed to markdown, chunked at roughly 3000 characters with about 500 characters of overlap, and embedded as 1024-dimension vectors stored in Postgres through pgvector^[1] alongside a sparse index for hybrid retrieval. Development runs the open bge-m3 model^[2] for the same purpose.

Extraction then runs in two passes rather than one, because the two questions are different. The first is a single document-level pass producing a structured manifest of the document as a whole: skills, methodologies, sectors, countries, client types, roles, values, years. The second runs once per chunk, in parallel, and tags that chunk against a controlled vocabulary. Both run at temperature 0.0.

The per-chunk pass is constrained to a seeded capability vocabulary rather than being free to invent labels. This is the decision the whole design turns on, and we discuss its cost below. A tag outside the vocabulary is not silently dropped: the model emits a sentinel marking a genuinely new capability, which is how we learn the vocabulary is short.

The result is a tree from firm to category to capability to the documents evidencing it. Downstream, retrieval over the same corpus returns chunk-level citations with page numbers above a cosine threshold of 0.7, so a generated bid draft can point at the passage it drew from rather than asserting.

Why explicit and implied are separated#

A firm profile that says "we specialise in pavement management" and a project reference that describes managing a pavement programme are different kinds of evidence. The first is a statement of intent, which a firm controls. The second is a description of delivered work, which is harder to write without having done it. We keep them as distinct labels rather than folding both into one score, because a consumer weighting a capability for a bid should be able to prefer demonstrated work over stated positioning.

Discussion#

Limitations#

The vocabulary is seeded, and the seed is not neutral. It is heavily weighted toward infrastructure and road-sector terms, which reflects where the corpus came from rather than where procurement is. A firm working in a sector the vocabulary covers thinly gets fewer, coarser capabilities from the same quality of documents, and the failure is invisible from inside the product: the graph looks sparse rather than looking wrong. The novel-capability sentinel tells us when this happens, but out-of-vocabulary terms are not automatically promoted into the vocabulary, so noticing is a manual step.

The confidence label and the evidence score are model outputs, not measurements. The 0.5-0.95 range is what the model returns, and we use it directly as an edge weight; we have not calibrated it against human judgement, so a capability with a higher evidence score is not demonstrably more likely to be real than one with a lower score. It orders capabilities within a firm reasonably. We would not defend it as comparable across firms.

The category layer is presentation, not data. Capabilities are grouped into visual categories by a mapping in the frontend, and the database holds no category taxonomy at all. Anything that appears to be a claim about how capabilities relate to each other is a rendering choice, and should not be read as structure the extraction discovered.

Finally, this extracts what a document says, which is not what a firm can do. Work delivered before the firm started keeping references is absent, capabilities held by people who have not uploaded a CV are absent, and a firm that writes sparse project descriptions is penalised relative to one that writes expansive ones. The graph is a model of a firm paperwork.

What we would do next#

The vocabulary problem is the one worth solving first, because it bounds everything downstream. Promoting frequently-seen novel capabilities into the seeded vocabulary, per sector rather than globally, would let the graph adapt to a firm domain instead of asking the domain to fit the graph. Calibrating the evidence score against human review would come second, and would need review data we do not currently collect.

Conclusion#

Two passes, a controlled vocabulary, and an evidence payload on every edge. What that buys is checkability: a capability points at the passage that produced it, so a reader can disagree with one label rather than having to distrust the graph as a whole, and the explicit and implied distinction keeps a stated position from being counted as delivered work.

What it does not buy is any assurance that the graph is complete, or that an evidence score means the same thing for two different firms. Neither has been measured, and the vocabulary bound is why we would not expect the first to hold: a firm outside the seeded sectors gets a thinner graph from the same documents, and nothing in the product tells it so. Read this as a description of a method and an argument for its shape, not as a result.