Some countries have been praised for their swift responses to the coronavirus, while others have been criticised for doing too little, too late.

But an increasing focus on data is leading to questions over whether we are comparing like with like. After all, the way nations record tests and deaths varies, and there may be societal differences that need to be considered.

Here are three reasons why direct comparisons between countries are challenging.

1. Death rates are complicated

There are a number of reasons to be sceptical about figures telling us one country has a worse death rate than another. The first is the issue of “excess deaths” – a temporary increase in the death rate, compared with historic trends. The large number of excess deaths currently seen in many countries is much higher than the official numbers of deaths attributed to COVID-19.

A number of studies have looked at this, from the London Business School to The New York Times. The Financial Times says globally excess deaths were around 122,000, over the weeks of different outbreaks in March and April – much higher than the 77,000 official COVID-19 deaths reported for the same places and time periods.

There is some debate over the reasons for this discrepancy, ranging from underreporting of coronavirus cases, to some patients avoiding hospital and dying from preventable conditions.

David Spiegelhalter, statistician and professor at the UK’s University of Cambridge, told The Financial Times: “There are so many questions about the rise we’ve seen in deaths that have not got [COVID-19] on the death certificate, yet you feel are inevitably linked in some way to this epidemic.”

Another thing to consider is the size of a country’s population. When viewed through this lens, Belgium has the highest rate. Although the US has seen by far the most deaths – currently at more than 58,000 – Belgium has been hit hardest when deaths are counted per 100,000 of the population.

2. Countries collect data differently

There are widespread variations in data collection, which has an impact on perceptions of a country’s response to the crisis.

The daily figure for COVID-19 deaths in a country might, for example, only include those dying in hospitals (this was the case in England, until late April). Other nations, such as France, included deaths in care homes in their figures.

Numbers of COVID-19 cases are also recorded differently in many places, with some including pending tests and others leaving out the results from some labs.

There are big differences in approaches to testing too. The UK has prioritised testing frontline workers and people who are so ill they need to be hospitalised. Other countries, like South Korea, have adopted blanket testing models. This means one country’s data may be less complete than others, making its death rate per confirmed case look relatively high.

3. Countries are very different

Countries and continents are very different. Africa, for example, has a much younger population than Europe – a factor some experts say may help explain why it has, to date, reported a relatively small number of confirmed deaths.

There are striking differences in the spread of COVID-19 between rural and urban areas within a country. Epidemiologist Professor Christl Donnelly of Imperial College London told BBC Radio 4’s statistics programme More or Less: “The more dense the population, the more you would expect transmission to happen.”

That said, some rural communities have become virus hot spots, and well-equipped cities such as Singapore had success in containing the virus – so generalisations are difficult.

All of which suggest that, while we may be able to draw some conclusions about the success of nations’ COVID-19 responses, we should exercise caution when making direct comparisons.

This article is curated from the World Economic Forum.