A purchasing manager we work with tells a story about a small plumbing repair.
It was a property management job — low value, low risk, nothing that warranted a formal process. His team got a few quick quotes, picked the cheapest, and the vendor looked legitimate enough. No qualification form, no references, no history. There was no obvious reason to make a production of it.
The work was done. Then it leaked.
The repair itself cost very little to put right. That was not the problem. The problem was that the buildings belonged to a different owner, and his firm managed them on that owner’s behalf. Someone now had to explain why an unqualified supplier had been chosen, on price alone, for work in a building they did not own.
That conversation is where the real cost landed. Not in the plumbing.
His summary of it is the reason this article exists: when things go right, nobody asks how you chose. When they go wrong, the only defense you have is being able to show your work.
A vendor scorecard is how you show your work.
The gap nobody closes
Here is the odd thing about vendor performance. Almost every business already has the information. Somebody on your team knows exactly which contractor turns up when they say they will, which one disappears for a week in July, and which one is worth the extra fifteen percent.
That knowledge just never gets written down in a form that survives. It lives in individual memory — and memory is not a system. It fades, it argues, and eventually it resigns and goes to work somewhere else.
The scorecard is unglamorous on purpose. It is the mechanism that converts scattered experience into something you can act on. And unlike most procurement discipline, it pays back on the very next job rather than in some abstract future.
Score at the middle, not only at the end
Most people assume vendor review is something you do when the work is finished. For a two-day job, fine. For anything of length it is wrong, because performance changes shape over time and a single end-of-job score captures only the last impression.
It moves in both directions.
A vendor can start badly simply because they are new to you — unfamiliar with your site, your people, your way of specifying work — then finish strongly once they have found their rhythm and put the right crew on the ground. Score them only at the start and you would have replaced someone who turned out to be excellent.
It also runs the other way, and this is the more common failure. A vendor starts well, loses key personnel mid-engagement, and the service quietly degrades. Score only at completion and you record a mediocre performance without knowing that the first half was genuinely good, or that the cause was a specific and possibly temporary change on their side.
Score once at the midpoint and once at completion and you capture the shape of the relationship, rather than a snapshot of its final week.
The vendors who never finish
There is a category the standard advice ignores entirely: the vendors whose work has no end date.
Your general building supplier. The logistics partner who has handled your deliveries for four years. The commercial sales rep you call for everything. There is no project completion to trigger a review, so no review ever happens — and these are frequently the relationships with the most money running through them.
The fix is to add a third cadence. In practice you need all three:
| Cadence | Applies to | When to score |
|---|---|---|
| Job-based | Short, discrete scopes | At completion |
| Project-cycle | Long engagements | Midpoint and end of each scope |
| Calendar-based | Perpetual vendors | Mid-year and year-end |
For a routine ongoing vendor, quarterly or twice a year is the right frequency. That is the band where you collect enough to form a sound judgment without exhausting the goodwill of the people you are asking. Ask monthly and participation collapses. Ask once every two years and the data is worthless.
Naming a number matters here. Vague advice to “review vendors regularly” is why this never gets implemented.
Who does the scoring
Not you, alone, from your desk. This is what separates a real scorecard from a procurement opinion.
The people who should score are the people who actually did business with the vendor: the manager on site, the superintendents, the field staff who dealt with them daily, the administrator who chased their invoices. On a short property-management scope — window cleaning across a building, say — that might be the building manager plus a handful of residents who watched it happen.
The fairness argument is the one that wins people over. A good vendor might complete a hundred tasks for you in a year. On one of them something goes wrong — a van breaks down, a crew is short, a delivery is late. One person’s account of that one bad day is an anecdote, not a record. If it is the only feedback ever captured, you will make a bad decision on the strength of it.
Collect across several people and across many occasions, run a running average, and you get something that survives challenge — from the vendor, from your own team, and from whoever asks you six months later why you made the call you made.
The six things to score
Keep the criteria short. A scorecard with fifteen dimensions gets abandoned by week three.
Price. Not just whether they were cheapest, but whether the final number resembled the quoted one.
Quality. Did the work meet the specification, and did it hold up?
Schedule. Did they start when they said they would, and finish when they said they would?
Communication. Did you hear about problems from them, or from somebody else?
Adaptability. When circumstances changed — and they always do — did they absorb it or fight it?
Ease of doing business. The one everybody lists and nobody defines. It gets its own section.
What “ease of doing business” actually means
This appears on every scorecard template on the internet as a vague catch-all. Here is what it actually measures.
Picture the same window cleaning vendor working two of your buildings. One is new, tall, and straightforward. The other is smaller and older, with a lot of elderly residents.
Can that vendor serve both well, and tailor to each? At the smaller building, that means quieter hours and choosing days that cause the least disruption to people who are home all day. At the taller one, it means being willing to work at height, and completing a full vertical run in a single day rather than splitting it across two and leaving equipment on site overnight.
Same vendor, same scope of work, two genuinely different jobs. Ease of doing business is whether they see that difference without being told.
Then there is the administrative half, which people forget to score at all:
- Payment terms you can actually work with
- Calling ahead rather than arriving unannounced
- Confirming details in advance instead of scrambling on the day
- Paperwork that arrives correct the first time
The plain-language test: do they create headache for your field team? That is professionalism and flexibility combined, and over a long relationship it is worth as much as price.
Stars or written comments? Both — and here is the split
The temptation is to ask for a proper written review. Resist it.
Stars carry participation, and participation is everything. Ask a busy site manager for a paragraph and you will get silence. Ask for a one-to-five rating and you will get an answer in under a minute. A scorecard nobody fills in is worth precisely nothing, so the format that gets used beats the format that is theoretically better.
Stars also average cleanly, which is what makes a running score possible across many jobs and many reviewers.
Comments carry context, and they matter most at the extremes. A single one-star against a history of fours and fives is exactly the entry you need to read. The number tells you something happened. Only the comment tells you whether it was the vendor’s fault, a one-off, or the beginning of a pattern.
So ask for both, but weight the effort correctly: a star rating, plus one or two sentences. That combination produces the most useful feedback per unit of effort you are asking someone to spend, and it guards against the known weakness of star ratings, which is that they skew without context.
You can score a vendor before there is a job
Worth knowing, because it costs almost nothing.
At the prequalification stage, before any work has been awarded, you can already build a partial record by asking your own team a single question: has anyone here crossed paths with this company before, and what was it like?
In a team with any tenure, this is the cheapest risk reduction available to you. It also means a vendor’s file is not blank on the day you first consider them — which makes the eventual decision both easier and better documented. See our guide to building an approved vendor list for where this fits in the wider process.
Building the sheet
A few practical choices make the difference between a scorecard that runs for years and one that dies quietly.
Use a one-to-five scale, not one-to-ten. Finer scales invent precision that does not exist. Nobody can meaningfully distinguish a 6 from a 7, and asking them to try slows submission down.
Make the comment field optional but prominent. Required free text kills completion rates. An inviting, optional field gets filled in when it matters.
Decide who compiles. Someone has to own the average and act on it, or the scores accumulate and nothing changes.
Score against the vendor, not the invoice. The record needs to follow the company through every job they do for you.
What the record is worth on the next job
This is where it pays back.
The next time you need that trade, you are not choosing from memory or from a list of names. You are choosing from evidence — filtered by category, by status, and by how vendors have actually performed on work you have given them before. The decision takes less time, produces a better outcome, and when somebody asks, you can explain exactly how you reached it.
That is the loop. You rate the vendor when the job closes, the record builds itself one project at a time, and every subsequent decision is easier and more defensible than the last.
PowerRFP tracks vendor ratings and performance history against every vendor in your directory, so the record builds as you work rather than as a separate administrative exercise. Try Pro free for 30 days.