Quality is weighted 40 per cent, but what does that actually mean?

Many suppliers read the weighting in a competition as a work plan: quality 40 per cent, so 40 per cent of the effort. It is a reasonable thought, but it does not capture the whole picture. The weighting says how much each criterion should count. But how much the points actually move the result appears to be shaped just as much by the evaluation model, and you often do not get to see it. Here we explain the difference as we understand it, and what you can do about it.

What the weighting actually says

Award criteria are what the contracting authority ranks the tenders by. The main rule in public procurement is that the contract goes to the tender with the best price-quality ratio. Awarding on lowest price or cost alone is still permitted under the EU directive, but Article 67(2) allows member states to restrict or prohibit it, and several have done so.

For contracts above the EU threshold value, Article 67(5) of Directive 2014/24/EU requires the contracting authority to specify in the procurement documents the relative weighting it gives to each criterion. The weighting may be expressed as a range with an appropriate maximum spread, and only where weighting is not possible for objective reasons may the criteria instead be listed in descending order of importance.

So far so good: you can see that price counts 60 per cent and quality 40, and you know where to put the weight. The problem starts when the percentages have to become points.

Weighting and the evaluation model are two different things

An evaluation model is the arithmetic that turns price and quality into comparable quantities. The most common type is a points model: each criterion is scored, the scores are multiplied by the weight of the criterion, and the highest total wins. The alternative is to price quality, where the quality differences are given a value in money that adjusts the tender price, and the lowest evaluated price wins. That is the approach the Norwegian procurement agency DFØ now recommends first.

Here is the important part: the procurement rules contain no provisions on evaluation models. The starting point is that the contracting authority is free to choose the model, and review bodies have held repeatedly that the choice sits within the authority's professional procurement discretion and can only be reviewed to a limited extent.

The Court of Justice of the EU also held in Case C-6/15 (Dimarso) that a contracting authority has no obligation to disclose the evaluation model in the procurement documents. At the same time the Court set some limits worth knowing: the model cannot change the award criteria or their weighting, it must as a rule be decided before the tenders are opened, and once the authority has stated which model will be used, it is bound by it.

In other words: you are entitled to know the weighting, but not necessarily the arithmetic.

The arithmetic that shows why this matters

Take a hypothetical example with round numbers: a competition with price at 60 per cent and quality at 40 per cent, a scale of 0 to 10, and two tenders where the more expensive one sits 6 per cent above the lowest.

Model A is the basic linear model: 10 points to the lowest price, 0 points to a tender that is twice as expensive. The more expensive tender then gets 9.4 points on price. The difference of 0.6 points, weighted at 60 per cent, gives 0.36 points in the total. To make that up, the more expensive tender needs less than one point more on quality.

Model B is a steeper linear variant where, for the sake of the example, we let a tender 20 per cent above the lowest price score 0. Models with a steeper scale come in many variants. Now the more expensive tender gets 7 points on price. The difference is 3 points, weighted 1.8 points, and the quality lead has to be 4.5 points on a scale from 0 to 10 to make up for it.

Same weighting. Same prices. A completely different outcome, at least in this hypothetical example. DFØ makes the same point in its own guidance, running the same example prices through four different models and getting four different results.

The issue turns up in complaints cases

This is not just a theoretical nicety. The question comes up regularly in complaints cases, and two decisions from the Norwegian complaints board KOFA illustrate it from opposite sides.

• In KOFA 2024/214 (Vennesla municipality) the board found that the contracting authority had breached the rules by using an unlawful evaluation method, but that the breach had not affected the outcome of the competition.
• In the grand board case KOFA 2021/1000 (Ørsta municipality) the complainant argued that the evaluation method shifted the stated weighting of the award criteria. None of the arguments succeeded, and the board held that a contracting authority is not obliged to normalise the scores on qualitative award criteria, meaning there is no duty to scale them up so that the best tender gets full marks. Our reading of the consequence: without normalisation the score differences on quality can end up small, and then the quality criterion pulls less in the total than the weighting alone suggests.

Norwegian decisions do not bind authorities in other countries, but they turn on rules that come from the same directive, so the reasoning travels. As we read the two cases, they show two sides of the same coin: the model can be unlawful without changing the result, and it can be lawful even though the points in practice move less than the percentages suggest.

Our point is not that you should complain more. Case law gives contracting authorities a lot of room here, and the choice of model can only be reviewed to a limited extent. The point is that you should plan on the basis that the percentages and the real payoff can be two different things.

What you should do

This is advice and judgement from us, not requirements in the rules:

• Look for the model before you read the requirements. If it is described in the tender documents, you have most of the answer key, and the contracting authority is bound by it.
• Ask if it is not there. Put the question well before the deadline for questions: which model is used, which scale, and where is the zero point on price set? The answers are shared with all tenderers, and a question about the model gives away nothing about the content of your bid.
• Ask what separates 8 from 10. Procurement guidance is clear that the score differences should reflect the real willingness to pay. In our view it is then legitimate to ask what actually earns full marks.
• Look for the maximum value when quality is priced. If the authority uses a model that prices quality, and the model with maximum values is stated, you can see in black and white how much money the quality is worth. That is probably the most precise information you can get about willingness to pay.
• Consider the price spread in the market. If the field is priced tightly, a high price weighting can mean less in practice than the percentage suggests. If the spread is wide, it can mean more.
• Do not opt out because price is weighted heavily. 70 per cent price with a tight price picture can in practice be decided on quality.

Does this hold across Europe?

Most of it does. The rule on the most economically advantageous tender and the duty to state the relative weighting follow from the EU procurement directive (2014/24/EU), and the Dimarso judgment applies across the EU and the EEA. The way of thinking, read the weighting, find the model, work out what it means, should therefore work just as well in a Danish or Dutch competition as in a Norwegian one, even though national practice varies.

What does vary is the national additions. Norway, for example, requires climate and environmental considerations to be weighted at a minimum of 30 per cent in competitions subject to publication where the criteria are weighted. In other countries the environmental weighting can look completely different, so check what your own national rules ask for.

In closing

The weighting is one of the most important pieces of information in a notice, but it is not the whole answer. The evaluation model is a large part of the rest, and you often have to go and get it yourself. It is a cheap exercise: one question before the question deadline can change the whole prioritisation of the work.

As always, this is our judgement and understanding of the rules, not legal advice. Rules change, practice develops, and every competition has to be assessed on its own facts, so treat what you find here as a starting point rather than an answer key.

This is a natural next step after the seven things you should check in a notice, and it ties closely to how you build the structure of your response, which we have written about in winning tenders: what the best bid managers do.

We at Cobrief are happy to talk about how you can read weightings and evaluation models more accurately in the tenders you are considering.