Design

The observational, longitudinal (prospective cohort), multicenter study was conducted in smoking clinics in daily clinical practice in Spain and the Argentine Republic in five tertiary hospitals, three secondary hospitals and a community specialized smoking unit. Patients were consecutively enrolled as they attended consultations from 1 October 2014 to 31 October 2015, and all patients were followed for one year. This study adhered to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist for observational research (Supplementary file SM2).

Collected variables

The quantitative variables consisted of age, age at initiation of tobacco use, cigarette consumption per day (as a continuous variable; and categorical variable: 0–10, 11–20, 21–30 and >30 cigarettes/day), number of years smoking, cumulative consumption (pack-years), number of previous attempts to quit, number of previous attempts to quit in the past year, weight (kg), height (cm), body mass index (kg/m2), carbon monoxide in exhaled air (ppm), follow-up time in months, the RT, the HMPMT, the KWT, the Fagerström test for nicotine dependence (FTND)¹⁷, the Heaviness of Smoking Index (HSI)¹⁸ score, and the VAS (scale of discrete values, range: 0–10), all of which were used as either continuous and/or categorical variables.

The qualitative variables consisted of patient referral (primary care, other specialists, or own free will), level of studies (basic, secondary, university), sex, reason for quitting smoking (yes/no), father and mother smoke/smoked (no/ yes/don’t know), older brother smokes/smoked (no/yes/don’t know/not applicable), rest of the brothers, the friends, co-workers and siblings smoke (don’t smoke/smoke mostly/same number of smokers as non-smokers/not applicable), comorbidities (yes/no), pharmacological and psychological treatment (yes/no; for different combinations of treatments), and outcome/result (failure/success).

Smoking cessation interventions: procedures

We defined abstinence as ‘continuous abstinence’¹⁹. We consider continuous abstinence when the subject refrained from smoking, from the moment they stopped smoking until the end of the follow-up or when they affirmed, by means of a telephone interview when not in person, that they had been abstinent in the previous three months. The treatment and follow-up intervention to stop smoking followed the regulations in force with a known protocol²⁰. This protocol included at least 9 follow-up visits throughout the year (face-to-face and some by telephone). The patient attended the consultation in person after the initial visit at 15 and 45 days, at three, sixth, nineth months, and at one year, in addition, between the previous ones, 2 or 3 telephone visits were added (duration of the first visit was 40 minutes and in the follow-up visits 15 minutes). Smokers were treated in each clinic by its staff (doctor, nurse, and psychologist was added in some clinics) and were assigned to the treatment that has demonstrated greater effectiveness in quitting smoking following current treatment protocols (multicomponent treatment: combination of psychological-behavioral and pharmacological treatment).

We measure motivation to quit smoking using quantitative questionnaires, including the Richmond Test (RT)⁹, the Henri Mondor Paris Motivation Test (HMPMT)¹⁰ and the Khimji-Watts Test (KWT)¹¹, and a semi-quantitative scale, the Visual Analogue Scale (VAS).

To corroborate self-reported abstinence by subjects, an expired air carbon monoxide (CO) meter was used at each clinic and at all visits during the 12-month follow-up (we used a cut-off point of CO ≤6 ppm to be considered a non-smoker)²¹. When consultation was by telephone, we recorded the verbal self-report of not smoking.

Statistical analysis

Qualitative variables were described by absolute value and percentage, and quantitative variables were described as mean with standard deviation, and range of values. The association between qualitative variables was evaluated using a chi-squared test. When the expected value criterion was not fulfilled in table cells ≥5, Fisher’s exact test was used in the case of 2 × 2 tables. In some cases of ordinal qualitative variable, the linear association test was also used. In the case of finding a significant association between the qualitative variables, the difference in percentages between the categories that introduce significance is expressed with the standard error. The relationship between qualitative and quantitative variables was evaluated using Student’s t-test. Normal distribution was verified using the Kolmogorov-Smirnov test, and the homogeneity of the variances was evaluated using the Levene test. In the case of non-compliance with any of these 2 assumptions, the non-parametric U-Mann-Whitney test was used for analysis. In the case of statistically significant difference in means, this is expressed by the standard error and 95% confidence interval.

Association between the outcome variable of the attempt to quit smoking and the motivation scales (RT, HMPMT, KWT and VAS) was tested using logistic regression. Given the high number of variables, the procedure was performed as follows. The statistical significance of the variable Motivation scale to quit smoking was assessed, controlling for the variables sex, age and Type of treatment, and, one by one, each of the remaining variables. The variable Type of treatment chosen for these models was Varenicline alone or Combined vs Other treatments vs No treatment, because varenicline is the more effective and powerful treatment to quit smoking²². When models were evaluated for other treatments, such as bupropion, nicotine replacement treatment (NRT), or combinations of NRT, the mentioned treatment variable was replaced with the corresponding treatment variable. All models included the variables in question and all possible first-order interactions between them. The backward step regression or backward elimination method was used as the automatic variable selection method to evaluate the interactions. Furthermore, when deemed necessary, the selection process for the best logistic regression equation for predictive purposes was performed in an automated way using a script or extension command for SPSS designed by the Laboratori d’Estadistica Aplicada of the Universidad Autónoma from Barcelona (Spain)²³. In this case, the following selection criteria were used for the best logistic regression model: 1) The Akaike information criterion; 2) area under the receiver operating characteristic (ROC) curve; 3) sensitivity and specificity of the models for a cut-off point of 0.5; 4) value of -2 times the logarithm of the verisimilitude (-2LL); and 5) degree of significance of the Hosmer-Lemeshow adjustment index and degree of significance of the adjustment index of Le Cessie-van Houwelingen. The proportion of variance of the dependent variable explained by the predictor (independent) variables was evaluated using the Cox and Snell R² and Nagelkerke’s R.

The following aspects were analyzed once the final regression model was obtained: 1) independence from residuals; 2) linearity; 3) absence of collinearity; and 4) absence of distant values of the response variable or of the predictor variables, and of influential values that affect the estimates. After evaluating the diagnosis of the model, its validity was studied by constructing the ROC curve and estimating the area under it. The accuracy of the area of the ROC curve was assessed following the Swets classification²⁴. Subsequently, the cut-off point of prevalence or probability of success was selected that yielded the highest efficiency values (highest percentage of correct classifications), together with the highest sensitivity value plus specificity. Subjects with missing values were excluded from the statistical analyses.

Analyses were performed using the statistical programs SPSS v.20 for Windows (Armonk, NY: IBM Corp.), MedCalc Statistical Software v.16.8. (MedCalc Software bvba, Ostend, Belgium), and STATA/SE 15.1 (Stata Corp. 2017. Stata Statistical Software: Release 15. College Station, TX: Stata Corp LLC). A p-value <0.05 was considered statistically significant.

Descriptive analysis

A total of 404 subjects were included, of whom 108 only attended the first visit. Of the remaining 296, a total of 23 subjects had a lack of data on any variable; therefore, 273 subjects (147 women, 53.8%; mean age 51±11 years) were ultimately included in the analysis (Figure 1). Material given in Supplementary file SM3 shows the characteristics (frequencies) of all qualitative variables for the total sample and by sex, and comparison between sexes. Supplementary file SM4 shows the descriptive values of the quantitative variables, for the total sample and by sex, with comparison between sexes.

Figure 1

Flowchart detailing subjects’ selection

Descriptive and bivariate analysis of the result variables

At one year, 53.5% (36.13% by intention to treat) of the subjects (146) were successful in quitting smoking, i.e. 66 (45.2%) males and 80 (54.8%) females, with no significant differences between the sexes. Supplementary file SM5 and SM6 show the percentages of each qualitative and quantitative variable, respectively, in relation to the Result variable. We identified an association between the Result of quitting smoking and the educational level (p=0.011). No association was observed between the mean scores of any of the motivation tests included to quit smoking (RT, HMPMT, KWT or VAS) or the success or failure, either for the overall series or by sex (Supplementary file SM5 and SM6).

Multivariate analysis of motivation tests to quit smoking

Table 1 shows the degree of significance of the different variables in the different tests of motivation to quit smoking, controlling for the variables age, sex and type of treatment at the first analysis.

In examining the effect of the RT score on the variable Result in the abstinence maintenance, none of the models showed any association between either variable.

The results of the study of the effect of the HMPMT score on the variable Result are shown in Table 2, and the regression model is shown in Figure 2. The score of the HMPMT was associated with the variable Result in quitting smoking in a complex model in which the variables Number of previous attempts, Level of studies, and Reason for quitting smoking (Health/Reduction of symptoms) intervene. The higher the HMPMT score, the greater the probability of quitting, although it depends on the variable Number of previous attempts. The statement of the previous point is valid for subjects with few previous attempts to quit smoking (≤2). Conversely, subjects with a high number of previous attempts to quit smoking are more likely to quit smoking with low scores on the HMPMT (as the score of this test increases, the probability of quitting smoking decreases, especially after 10 points). Therefore, the decision to start or not start treatment in a subject based on the score of the HMPMT should also take into account other variables, such as the Number of previous attempts, the Level of studies, and whether their motivation to quit smoking is due to health/decrease in symptoms. In this case, we could not show that abstinence was determined by the previously chosen smoking cessation treatment. The area under the ROC curve of this model was 0.678±0.032 (95% CI: 0.618–0.734, p<0.001), showing low accuracy (Figure 3a).

Figure 2

Graphic representation of the distribution of predicted probabilities according to the model in Table 2 for the different combinations of values of dummy subjects of the variables Henri Mondor Paris Motivation Test, Number of previous quit attempts, Level of studies, and Reason to quit smoking (Health/decrease in symptoms)

Figure 3

ROC curves corresponding to the logistic regression models of the Visual Analogue Scale and the Henri Mondor Paris Motivation Test

The outcomes of the effect of the VAS score on the variable Result are shown in Table 2. The only variable with which the VAS variable showed statistical significance with the variable Result was Age of onset of tobacco use. The regression model presented a good fit of the model to the data (Hosmer-Lemeshow adjustment index was not significant p=0.594); however, the overall significance of the model was almost significant (p=0.079). The study of the internal validity of the model showed that the ROC curve was almost superimposable to the diagonal line of the graph, indicating poor predictive capacity of the model. The area under the ROC curve was 0.573±0.035 (95% CI: 0.512–0.633), discretely above the area of the diagonal that was 0.5 (Figure 3b). The best cut-off point of probability of success, which produces the best percentage of correct classifications was ≥0.411, but this percentage of correct classifications was only 58.82%, with a sensitivity of 98.63%, and a specificity of 12.70%. Given these results of internal validity, the model lacks relevant predictive capacity regarding association with the Result variable. Therefore, although there might be a certain association between the score of the VAS and the Result variable, this association would be modulated by the variable Age of onset of tobacco use to smoking; thus, we did not identify an association between the variables.

The results of this study on the effect of KWT score on the variable Result are also shown in Table 2. The score of the KWT to quit smoking was associated with the variable Result in quitting smoking in a complex model in which the variables: sex, father smokes/smoked, height, and number of treatments with NRT. However, the height variable was a variable with low probability or biological plausibility of association with smoking abstinence; so, when it was excluded, the KWT did not reach statistical significance.

Limitations

The present work has several limitations. First, the findings were obtained using smokers who voluntarily attended smoking cessation clinics, and the surveys were performed in different scenarios and geographical locations, which might not reflect the general population. Second, both the dependency and motivation tests were developed to be answered face-to-face. However, in some cases, the questionnaires were delivered to and completed at the participant’s home and brought back at a later time. Third, the use of questionnaires in patients is not always accurate. Fourth, the fact that some of the motivational measures were associated with smoking outcomes could be due to the fact that these are not validated measures, so they could be measuring something other than motivation to quit smoking. Fifth, the sampling strategy and the dimensions of the sample may not have sufficient statistical strength to identify differences. Sixth, although there were few subjects in our work with missing data, they were excluded from the work, we are therefore aware that they could have an effect on the conclusions, reducing the representativeness of the sample obtained and therefore distorting the inferences about the population. This variability could lead to other results.

In view of the results, we suggest that it is not worth measuring the motivation to quit smoking with the instruments analyzed, since they do not predict smoking abstinence and can be misleading in the approach of each specific case, in the sense that a low score in such tests may suggest that the subject is not motivated and consequently not offered treatment that could lead to smoking abstinence, a decision based on clearly unreliable instruments.