Participant recruitment

The pilot comprised three groups, capped at 25 participants in each group for manageability by the facilitator. The three groups commenced two weeks apart from September 2018. Volunteers were recruited in July and August 2018 via promotion of the pilot on the HSE-TFIP’s QUIT website and social media. The following inclusion criteria applied: current smoker interested in quitting; aged ≥18 years; Facebook account holder; and living in the Republic of Ireland. Pregnant smokers were excluded from this study. Group assignment was in order of recruitment and participants provided informed consent (including potential withdrawal) for participation in the pilot intervention for the HSE-TFI Programme and to take part in the research study. Participants who completed the evaluation were entered into a draw for a €100 shopping voucher.

Evaluation approach

The Centre for Disease Control and Prevention Framework for Program Evaluation in Public Health, a user-focused approach in line with WHO recommendations on health promotion evaluation, guided the approach²⁴. Evidence was gathered using mixed-methods, so conclusions could draw on the combined strengths of qualitative and quantitative data²⁵. An explanatory sequential design was chosen: quantitative methods were used first, followed by qualitative methods to explain quantitative results in more depth.

Data collection and analysis

All participants were invited to complete an online questionnaire before and one week after the end of the program; it collected information on demographics, smoking behavior, Fagerström Nicotine Dependence Score²⁶, Smoking Abstinence Self-Efficacy Score²⁷, Contemplation Ladder²⁸, and Motivation to Stop Smoking Scale²⁹. The post-program questionnaire also included questions on participant experience and engagement, net-promoter score (how likely a participant is to recommend the program to someone else)³⁰ and self-reported seven-day point prevalence abstinence, a common measure for short-term follow-up of stop smoking interventions³¹. In line with the Russell Standards, smoking abstinence was measured on an intention-to-treat basis for all those who accepted the invitation to join a Facebook group³². Measures of each participant’s daily activity within the Facebook group (comments on and ‘likes’ of posts in the group; and number of views of the videos posted by the facilitator) were also collected and were used to assign individual and group activity categories. Given small numbers, binomial exact 95% confidence intervals were derived for the proportion achieving smoking abstinence and Fisher’s exact tests were performed to detect significant differences between groups. Data were analyzed in SPSS.

Qualitative data were collected and analyzed in line with good practices³³. One-to-one phone interviews were conducted one to two weeks after a participant completed the program and the interviewer (LH) used a topic guide flexibly in discussions. Purposive sampling of the study participants as described above was used to maximize interviewee variation in terms of sex, quit status at the end of the intervention and level of activity in the Facebook group. Interviews were performed until saturation was reached and no new themes arose in the interviews. All interviews were transcribed verbatim. Thematic analysis was chosen to explore participant program experience. This is a method for identifying, analyzing, organizing, describing, and reporting themes found within a data set that is argued to be well suited for communication between quantitative and qualitative analysis, as intended in this mixed-methods study. An inductive approach was taken to identify themes and the traditional manual ‘cut and paste method’ was used³⁴. After data familiarization, an initial set of data codes was generated by the primary researcher (LH). A second experienced qualitative researcher (EM) checked a sample of three transcripts and independently coded these for reliability. Codes were reviewed and discussed. Both researchers generated similar codes from the data and agreed on the over-arching themes arising from the sample. The primary researcher continued to systematically code all transcripts. A dictionary of codes for each theme was compiled, along with the frequency with which they appeared in the data. Charting was performed to demonstrate the relationships between the data. Consolidated criteria for reporting qualitative research (COREQ) were used by the authors to design, execute and ensure trustworthiness of the qualitative component of this mixed-methods study³⁵.

Participant overview

There were 83 people who responded to the recruitment campaign and all were invited to join a Facebook group, with 52 participants ultimately joining a group (Figure 1). Attrition was high with 23 people (44.2%) completing the post-program survey. A further 13 were contacted by telephone so self-reported seven-day point prevalence smoking absence was available for a total of 36 participants. Sixteen participants could not be traced and were counted as smokers.

Figure 1

Overview of study participants

As shown in Table 1, most participants were female (45; 86.5%), aged 35–54 years (34; 65.4%), and many had completed third level education (21; 40.4%); and nearly half had dependent children at home (24; 46.2%). Almost all had previous quit attempts (50; 96.2%) but only 16 (30.8%) had previously used a cessation aid. At baseline, most participants were moderately nicotine dependent (25; 48.1%) and in the preparation phase of the ‘Stages of Change’ (37; 71.2%). The mean Smoking Abstinence Self-Efficacy score was 14.4, suggesting the group was not very confident in their ability to quit smoking. Despite low self-efficacy, the group reported high levels of determination, and 28 (53.8%) were extremely determined to quit based on their answers to the Smoking Abstinence Self-Efficacy questionnaire.

Due to the small numbers in Group One (n=5) and for comparative analysis purposes, Groups One and Two were combined and categorized as ‘less active’ groups (total likes/comments/posts in the groups, 294); Group Three was categorized as a ‘more active group’ (total likes/comments/posts in the group, 1013).

Participant engagement and activity

Figure 2 shows participant activity trends over the 12-week program. Overall, participant engagement and activity declined over the 12-week program, with 50 (96.1%, 95% CI: 86.8–99.5%) versus 10 (19.2%, 95% CI 9.6–32.5) members making at least one ‘like’ or comment at week one versus 12, respectively. As outlined above, Groups One and Two combined were less active (total likes/comments/posts, 294) than Group Three (total likes/comments/posts, 1013).

Figure 2

Total weekly number of likes, comments and posts over the course of the program, three groups combined.

Program impact and outcome

At program end, 13 participants (25.0%, 95% CI: 14.0– 39.0) reported 7-day point prevalence abstinence. Use of cessation aids, participation in a more active group and above average individual activity were not associated with participant outcome (Fisher’s exact tests, p>0.05). For those who completed the program and reported continued smoking, while Fagerström Nicotine Dependence Scores decreased, this was not significant. Participants who completed the program reported Smoking Abstinence Self Efficacy scores similar to baseline (mean scores 14.4 versus 14.7, respectively), albeit participants who quit reported a higher mean score compared to participants who continued to smoke (21.2 vs 8.8).

Participant experience

The program net promoter score was 4.4 (range: -100 to +100, with any score above 0 indicating satisfaction). Ten participants (43.5%) reported they would recommend the program to a friend. Compared to participants in less active groups (Group One and Two), a higher proportion of participants in the more active group (Group Three) were identified as promoters (81.8% vs 8.3%: Fisher exact test statistic 0.0006; result significant at p<0.05). As shown in Table 2, while many participants reported a positive program experience, compared with the less active groups, a higher proportion in the more active group reported a positive experience.

Thirteen participants were interviewed: 11 females; 8 aged ≤45 years; 6 in a more active group; and 6 reported end of the program smoking abstinence. Three over-arching themes were identified; the importance of social interactions, the perception of health information, and the appeal of online support.

The importance of social interactions was the main discussion topic raised, and peer support was the main reason for joining the group. Having someone to turn to, who was going through the same situation, was perceived as important:

‘I suppose the option to contact somebody, you know when you need, maybe that's for me a very important thing.’ (Female, 35–44 years, smoking, inactive group)

Other group participants were seen as a valuable source of encouragement, advice and inspiration when a quit attempt was going well, but also during a relapse:

‘You can cheer people on and see, wow, someone's doing really well, that's brilliant.’ (Female, 45–54 years, smoking, active group)

Support was seen as particularly necessary for managing cravings or relapses:

‘... if you have a slip, someone is there to say keep going and try again.’ (Female, 35–44 years, quit smoking, active group)

Turning to other group members for help was perceived as easier than talking to family and friends. Their support was seen as non-judgmental, understanding, and more objective:

‘… if I'm having a bad day I can post and it's not my family, it's strangers, so you feel better doing it because you don't know them personally. But it's not someone giving out to you or saying don't or someone putting you on the guilt trip, you know family or friends that don't like you doing it.’ (Female, 45–54 years, smoking, inactive group)

Advice from others in the group was valued, as it was based on lived experience. There was evidence that advice shared between group members was followed:

‘I asked, I'm overweight and obviously that's a fear and I put up a post asking, has anyone been putting loads of weight on? Someone actually came on and said, actually I've lost weight! I couldn't believe it! Because she took up walking, running and drunk water for the cravings, and I found that the water was a great help for me for the cravings and someone told me to take up knitting to occupy your mind from food, so I'm on my third scarf.’ (Female, 35–44 years, smoking, active)

Three participants mentioned ‘the quit journey’ and how they found it easier to relate to other smokers and ex-smokers rather than a counsellor:

‘I didn't like the pep talk from somebody I didn't know, who's not a smoker themselves.’ (Female, 45–54 years, smoking, active group)

Three participants mentioned feeling inspired by others’ success, which gave them hope for personal success:

‘I think it was harder because I gave them up for so long and went back, I think the group actually helped me get through the hard times because I kept beating myself up about going back and it was nice just to hear people have been off them ten years and someone went back on them.’ (Female, 35–44 years, smoking, active)

While those in the active group praised the positive effects of group interactions, those in the inactive groups expressed almost universal disappointment at the lack of group interaction:

‘I left the group because I just felt there was no point in me being in it … I did post and there’d be nothing back ... there was no real support there, apart from the videos, they were good.’ (Female, 45–54 years, smoking, inactive group)

‘ ... That Facebook group didn't happen as far as I'm concerned, you know, no-one got engaged with it …’ (Female, 35–44 years, quit smoking, inactive group)

Participants suggested various solutions, including larger groups with more members. Two participants mentioned someone, a ‘mole’ or a ‘stooge’, who is a part of the group and would facilitate group discussion:

‘I don't know if this is ethical, but I'd nearly put a few stooges in there just to keep the conversation going or generate conversation ...’ (Female, 45–54 years, quit smoking, inactive group)

‘… You'd nearly like have to have a mole in the group … to like jizz people along, how's everyone today? But not make them really obvious.’ (Female, 35–44 years, quit smoking, inactive group)

There was diverse perception among participants of the health-related information provided during the program. Some participants complained about it and perceived it as ‘old knowledge’:

‘And then there were these random generic videos … that like anyone who's been smoking for 20 odd years knows everything that's going to be in them videos. I thought as a group we'd support each other, not just generic videos getting sent out, you know.’ (Female, 35–44 years, quit smoking, inactive group)

These participants all stated that they joined the group for support, rather than information. In contrast, some participants found the health information very helpful:

‘Because in the past, I tried to just kind of wean myself off them, cut down, or even just quit cold turkey and it never worked. So I guess really the main thing that I got from the group was learning about the replacement therapy definitely.’ (Female, 18–24 years, quit smoking, active group)

The majority of participants felt the use of Facebook for smoking cessation was appealing. Ease of use, at a time convenient to you, was repeatedly cited as reasons for choosing online group support over a face-to-face group:

‘... whereas if you have it on your phone and you can ask the question at any time, it's quite useful.’ (Female, 18–24 years, quit smoking, active group)

The online groups suited those with busy lifestyles:

‘I suppose being a full-time mum, full time worker, it was just easy to get into a group that I didn't have to go to, I could commit to it.’ (Female, 35–44, quit smoking, active group)

Accessibility was also mentioned as a factor in deciding to use online support:

‘To be honest about the clinics, there's not many around for myself because they're very far and I don't drive.’ (Female, 35–44 years, smoking, active)

Privacy was also mentioned by four participants as an important factor:

‘It's private, no one else can see what you're posting.’ (Female, 35–44 years, smoking, active)

One participant even acknowledged how ‘strange’ this sounded, in light of recent data concerns involving Facebook. However, the private group meant that family and friends of the participants were not necessarily aware of the quit attempt, which for some participants was a relief from the potential pressure and judgement from family and friends. The distance from the other participants and online anonymity made it less intimidating for some participants to share their struggles:

‘If you are having an off day, it seems to just write something rather than having to face someone, it's easier.’ (Female, 45–54 years, smoking, inactive group)

Strengths and limitations

This utilization-focused evaluation of the piloting of a new stop smoking service development, provided useful local information to determine future planning, as well as offering more generalizable evidence. The mixed methods approach provided richer and more rounded evidence for decision-making, since qualitative data in this study triangulated with quantitative data. The study highlights the value of embedding evaluation into stop smoking service development, especially services using new and innovative methods, like digital health interventions, to ensure that benefits established in trials are delivered in day-to-day practice. Using evaluation to guide digital health implementation is an approach recommended by the WHO⁶, and also highlighted recently by the National Institute of Health and Care Excellent (NICE), in its guidelines on using digital and mobile interventions to support behavior change⁴².

The evaluation was limited by small numbers, albeit comparable to the scale of many previously reported studies in this area^8,9, and also by attrition, which is common in studies of stop smoking interventions⁴³. To account for attrition, in line with good practice, outcomes were reported using the Russell Standards³². The decision to stratify some analyses by ‘more active’ versus ‘less active’ groups was made post hoc, and while this is explained, appropriate statistical tests are applied, and it is well-aligned with the mixed-methods paradigm of the study, these analyses should be interpreted with caution. While only short-term outcomes were measured and were not biochemically verified, the feasibility and value of these additional measures in population-based studies of stop smoking studies is questionable⁴⁴. Furthermore, participant’s usual social media use, was not measured. Future studies should include a longer period of follow-up and measurement of participant’s social media usage.