All estimates based on a sample are subject to what is known as ‘sample variation’. However tightly controlled, the answers from one sample of people will differ somewhat from another sample of people drawn in exactly the same way.
There are various statistical tests which can be used to assess whether changes in NRS data from period to period (or between publications) are actually real changes rather than the result of sample variation.
For any estimate taken from the NRS it is possible to work out a ‘confidence limit’. This is a way of expressing how much variation we might expect. For instance:
- Magazine X has estimated readership of 1,818,000 adults (3.7% of adults)
- The 95% confidence limits for this estimate are +/- 125,000
- This means we can be 95% certain that the true estimate for Magazine X is between 1,693,000 and 1,943,000 (3.4% to 4.0% of adults)
NRS has created two easy-to-use spreadsheets to:
- Find out the confidence limits for a particular estimate, be it readership or any other estimate on the survey.
- Find out whether the difference between two estimates is statistically significant
- All users need to do is to enter the estimates they wish to test into the boxes shown on the spreadsheet, along with the population and sample size on which the estimates are based.
NRS also has available a booklet called ‘Statistical tests for NRS data’ which goes into more detail about the tests, and the formulae on which they are based.
