GC approves new harms survey method despite oversampling gamblers

A pilot scheme run in spring 2022 produced results suggesting that 1.3% of the British population would be classed as problem gamblers, far higher than the 0.4% estimate from Health Survey for England 2018 (HSE 2018). However, the regulator noted that this could be for a number of reasons, unrelated to genuine increases in harm.

As a result, the Commission warned that the new figure “should not be used as an estimate of problem gambling at this stage”.

The percentage of respondents that were classed as being at moderate risk and low risk of gambling harm were also higher in the pilot scheme survey, at 2.3% and 7.8%.

In particular, it highlighted the fact that a larger portion of the pilot scheme sample were gamblers when compared to a random sampling of the population. For the pilot, 63% of respondents gambled, compared to 54% in the HSE 2018, with the starkest difference in online gambling rates. 

In its assessment of the pilot, NatCen social research – which won the tender to conduct the scheme – noted that the difference cannot be explained by a genuine spike in gambling activity. This, it said, would have been picked up by the Commission’s quarterly surveys on participation.

Instead, the regulator said some of the difference may have been caused by the “push-to-web approach” of the survey, which “uses offline contact methods to encourage people to go online and complete a questionnaire”. This was complemented by a shorter postal questionnaire in order to reduce selection bias toward frequent internet users.

As a result, the higher sampling of gamblers may be due to the fact that gamblers are naturally more inclined to fill out a survey about gambling. NatCen expressed a similar sentiment but also raised the possibility that respondents to telephone surveys could be “reluctant” to report gambling.

“It is very difficult to distinguish between these competing hypotheses, and all may be true to an extent,” it explained.

In addition, it noted differences in the way some respondents filled out surveys. NatCen noted that levels of low and moderate levels of risk were higher in the pilot survey even after controlling for their level of gambling activity.

For example, even among HSE 2018 participants who had taken part in only one gambling activity in the past year, 2% had a Problem Gambling Severity Index (PGSI) score of one or more. In the pilot survey, the total was 8%.

Overall, after controlling for gambling activity, the estimated odds of having a PGSI score of 1+ was found to be 2.9 times higher for the pilot survey than for HSE 2018.

“So, something in the nature of the pilot survey means that pilot survey participants with the same gambling behaviour as those in HSE 2018 responded to the PGSI items in a way that resulted in their gambling scoring as low or moderate risk or problematic,” NatCen said.

NatCen found that the discrepancy in both harm rates and overall gambling levels was most apparent with younger people, and particularly women, while the rates recorded among older people were similar in the two surveys.

NatCen also added that the new survey producing higher figures was “not unexpected”, as a review of a prior GambleAware/YouGov survey had suggested that similar surveys may over-estimate prevalence.

Despite the rates of harm being higher than phone surveys, NatCen also noted that they were “lower than those typically generated by online panel surveys”.

In terms of participation rates, NatCen said the pilot was successful, exceeding response rate expectations.

The Commission noted that “offering a postal survey completion option is a must” for future surveys given that 40% of respondents submitted their answers by post.

As a result the regulator concluded that the pilot survey had been a success, opting to use a similar methodology of survey in the future, after putting these recommendations in place.

“The pilot survey has demonstrated that we have a sound base for a methodology and can now move forward into the experimental stage of our project,” it commented.

NatCen offered a number of recommendations for future editions of the survey. It suggested that further work be taken to eliminate a bias towards gamblers in the sample, such as changing invitation letters to focus more on mental health rather than gambling specifically.

In addition, it said the sample size should be increased to up to 20,000, in order to allow for analysis of sub-samples, as well as shortening the survey so it can be completed in 20 minutes.