Implementation of the “ Saúde na Boa ” intervention : process evaluation and characteristics of participants and dropouts Implementação da intervenção “ Saúde na Boa ”

School-based interventions have been evaluated to examine its implementation quality and to identify the characteristics of successful participants. The objective was to analyze the process evaluation of the “Saúde na Boa” intervention and to test differences between successful participants and dropouts. Twenty schools were randomly selected to receive the intervention (n= 10) or the control (n= 10), in Recife-PE and Florianopolis-SC. The process evaluation was performed in the intervention schools using a checklist of items from five domains (knowledge, visibility, environment, Physical Education classes and extracurricular activities). The operational intervention quality was classified using the median score from each domain (median from 0.0 to 1.0= low, from 1.1 to 2.0= moderate, and from 2.1 to 3.0= high). Student characteristics were assessed at baseline (March 2006) and were compared between the participants who successfully completed the intervention (n= 989) and the dropouts (n= 1,166). We verified moderate to high operational quality of the intervention for knowledge (median= 2.0), fruit distribution and bike rack installation (median= 1.5), acquisition of materials (median= 3.0) and distribution of journals (median= 2.0). Students who dropped out were significantly more frequent in male (47.8% vs. 40.3%), in workers students (49.2% vs. 44.2%), those who smoked (21.8% vs. 13.6%) and consumed alcoholic beverages (57.4% vs. 49.5%), and slept ≥8 h/day (19.5% vs. 27.3%), than participants who did not drop out. Therefore, the “Saúde na Boa” intervention was of moderate operational quality. Participants and dropout students differed according to their gender, work status and use of licit drugs.


INTRODUCTION
Building an educational system that encourages lifetime values and health is a challenge for all professionals involved in the school environment.All people involved in the learning process are therefore important for its success, including the parents, directors, community and students themselves.Scientific research has indicated that school-based interventions can be an effective strategy to promote healthy lifestyles among young people 1,2 .Additionally, strategies that combine environmental and educational components, complex 3 initiatives for lifestyle promotion, and health promotion in the school system 4 have all been found to be effective 1,2 .
Investigating the effectiveness and the operational process of any interventions is a crucial step in school-based health promotion 5 .Although process evaluation is usually neglected, it is necessary to guide how interventions in the school setting can be more effective in contributing to public policies that are focused on healthy lifestyles in youth 6 .
In 2006, the "Saúde na Boa" intervention program was performed with the aim of promoting physical activity and healthy eating among high school students in Florianopolis and Recife, Brazil.The methodological aspects of this study, including the study design, instrument validation, description of the theoretical model and intervention strategies, have been described elsewhere 7,8 .However, the evaluation of the intervention implementation and the characteristics of the students who remained in the intervention or dropped out have not yet been presented.
The present study was therefore performed to fill these gaps in our understanding of this promising intervention.The information we report here may help to predict the potential complications and effectiveness of future interventions in schools.Additionally, the interpretation of the effectiveness of the "Saúde na Boa" intervention on lifestyle factors was facilitated with information about the implementation process and the characteristics of the participants.There were two objectives of this study: 1) to describe and analyze the results of the process evaluation of the "Saúde na Boa" intervention and 2) to identify differences between the participants and dropout students in terms of their socio-demographic conditions, lifestyle, health and psychosocial status.

METHODS
The methodological synthesis of the "Saúde na Boa" intervention The "Saúde na Boa" (a Portuguese expression that translates to "Health can be fun") intervention was developed in 2006, with the slogan "Being healthy is all good".Its intervention strategy was based on the WHO Health Promoting School's Philosophy 9 and the CDC's Guidelines for School and Community Programs 10 .Three major areas were addressed: 1) education about physical activity and healthy eating, 2) environmental and organizational changes, and 3) staff training and engagement.
The "Saúde na Boa" intervention was a randomized-controlled, schoolbased study that was conducted in two Brazilian cities (Florianopolis, SC and Recife, PE).Ten public schools were selected in each city, and five schools were randomly allocated to the intervention group and five to the control group.The sample consisted of night shift students between the ages of 15 and 24 enrolled in their first or second year of high school.Detailed methodological information is available elsewhere 8 .
The main objective of this study was to examine the effects of an intervention that took place over a single academic year (from March to December) on the physical activity and healthy eating habits of high school students.Secondary aims included testing the effectiveness of the intervention in influencing other behaviors relevant to health.We also highlight that the questionnaire used here was developed and validated for this population, as detailed in a previous publication 7 .

Process evaluation of the "Saúde na Boa" intervention
Process evaluations were carried out in schools in Florianopolis (between October and December of 2006) and Recife (between December of 2006 and February of 2007) to analyze the operational execution during and after the intervention, respectively.The evaluations were performed on specific dates due to strikes within the school systems of both cities.Importantly, the process evaluations were performed during the same stage of the intervention, allowing for a meaningful comparison of the obtained information.
All schools in the intervention group (five from each city) were included in the process evaluation.The process evaluation was based on the five domains described in Box 1.These domains and their respective items were selected according to recommendations from the Centers for Disease Control and Prevention 6 .We assessed whether planned strategies were implemented in schools (e.g., the distribution of flyers) or whether these strategies reached their primary purposes (e.g., knowledge about the intervention within the school community), as based on the goals and targets defined for the "Saúde na Boa" intervention.
The process evaluation was conducted by two independent Physical Education professionals, one in each city, who were specifically trained to perform the evaluation.Importantly, the professionals responsible for the process evaluation were not integrated within the research team that was involved in the planning and implementation of the intervention.
The data collection consisted of an interview with a structured questionnaire that was administered to students, teachers and staff from the intervention schools.A general evaluation of the intervention was completed by the evaluator.During the interview, the evaluator assigned a score to the answer for each item on a Likert scale from 0 to 3 (where 0 means "nothing was done with no observable change", and 3 means "the goal was fully achieved with an easily observable change").For analysis proposes, a positive change was documented for an evaluation item when the evaluator assigned a score of one or greater.Students, teachers and staff responded to the items from Domain 1 during class days.The items from Domains 2 and 3 were assessed based on the observations of the evaluators within each school.If the evaluators did not visualize the promotional posters, t-shirts, Frisbees and/or bottles in the school environment, an interview was performed with students (n= 6-8), teachers (n= 4-5) and staff (n= 2-4) about these materials.Domain 4 included items about Physical Education information (distribution of newsletters, modified activity practices and new materials in these classes) and was evaluated in the format of conversations with groups of students.The items from Domain 5 included information about events, weekend activities and school partnerships.This information was obtained by interviewing the students and school coordinators.Finally, an item about overall implementation of the intervention in the school was answered by the evaluator.

Baseline characteristics of the participants and the dropout students
One questionnaire was administered at baseline, before the intervention in March 2006.The characteristics of the students were subsequently compared between the participants who remained involved until the end of the intervention and the dropout students.This comparison was performed to identify subgroups likely to abandon the intervention.Thus, we compared the proportion of participants and dropout students according to the intervention condition (control or intervention) used and the student's city, gender, marital status, work status, living situation and skin color.
Lifestyle and health factors were also compared.Physical inactivity (no physical activity within a typical week and the previous week), inactive commuting to school (car/bus), physical inactivity during leisuretime and watching TV or using computer and video games during two or more hours daily were all documented for comparison.Additionally, the proportion of students who did not consume fruits and vegetables in a typical week, who consumed sweets, snacks and soft drinks daily, who used cigarettes and alcohol in the month preceding the survey and who were overweight 11 were compared.Finally, several psychosocial determinants, including the student's attitude towards physical activity, preference for eating fruits and vegetables, and self-rated health, quality and duration sleep, and body weight satisfaction were documented and compared between the groups.

Statistical procedures
We used absolute (n) and relative (%) frequencies to describe the data, along with the mean and standard deviation (SD) for each age.To analyze the operational process of the intervention, we considered the median scores of the items from each Domain.We used the following arbitrary cut-off points to break the median score into categories: from 0.0 to 1.0 was defined as low quality, from 1.1 to 2.0 was defined as moderate quality and from 2.1 to 3.0 was described as high quality.The two proportions comparison and Mann-Whitney tests were used to compare the characteristics of the participants and dropouts students.A p-value ≤ 0.05 was considered significant.Statistical analyses were performed using Stata version 11.0.

Ethical aspects
All procedures were approved by the Ethics Committee of the Federal University of Santa Catarina (031/2005) and the Instituto Materno Infantil de Pernambuco (587/2005).Negative consent ("passive parental consent form") was obtained from students older than 18 and from the parents or guardians of students younger than 18.

Evaluation of the implementation process of the intervention
Table 1 describes the number of schools that were classified as having positive changes during and after the intervention period, with a median domain score according to the city.Domain 1 included items about the knowledge of the intervention, and all items were found to have a high operational quality score (with a median of 3.0) in Florianopolis.In Recife, there was a low quality score (with a median of 1.0) in the items from Domain 1.In general, we obtained moderate operational quality scores (with a median of 1.5 to 2.0) both during and after the intervention.Domain 2 included items about the visibility of the intervention.For this domain, high implementation quality scores were only obtained in Florianopolis, especially for the items concerning the posters and the distribution of newsletters (median of 3.0).The combined score of both cities suggested that the intervention was of low visibility (with a median between 0.0 and 1.0), independent of the evaluation period.Domain 3 included items about environmental changes.In this domain, only the fruit distribution in Florianopolis and the installation of bike racks in Recife were found to have high implementation values (with a median of 3.0).The overall score indicated a moderate implementation quality (with a median of 1.5).Domain 4 addressed the Physical Education classes.The distribution of newsletters was found to have a high implementation (with a median of 3.0) and the acquisition of new materials was found to have a moderate implementation (with a median of 2.0) in Florianopolis.In Recife, a high implementation (with a median of 3.0) was noted for these items.Overall, we found moderate or high implementation scores (with a median of 2.0 or greater) for these items.
No extracurricular activity (Domain 5) was observed in Recife.In Florianopolis, four out the five intervention schools had a score of 2.0 for items about weekend and athletic activities.Overall, this domain had low implementation score (with a median between 0.0 and 0.5).
Thus, our overall evaluation suggested a moderate operating implementation quality for this intervention across schools in two cities (with a combined median of 1.5, Table 1).

Baseline characteristics of the participants and the dropout students
The average age of the students who remained in the intervention was 18.2 years (with a standard deviation of 2.5 years), while the average age of the dropout students was 18.5 years (with a standard deviation of 2.2 years).There was no difference between the cities (p = 0.96) or study condition (intervention versus control; p = 0.29) in terms of the proportion of participants and dropout students.The dropout students were more likely to be male (p < 0.01) and working (p = 0.02) than the participating students (47.8% vs. 40.3%and 49.2% vs. 44.2%,respectively).We identified no other sociodemographic differences between the participants and dropout students.
Considering their lifestyle and health characteristics, the group of dropout students had a higher proportion of smokers (21.8% vs. 13.6%;p < 0.01) and alcohol consumers (57.4% vs. 49.5%;p < 0.01), and we presented lower proportion of sufficient sleep duration (19.5% vs. 27.3%;p < 0.01), than the group of participating students.There was no significant difference for other lifestyle and health characteristics (p > 0.05).
The dropout students group also had a higher proportion subjects who reported that they did not like eating fruits and vegetables than the participant group (21.2% vs. 17.6%;p = 0.04).No difference was observed between the groups in terms of other psychosocial characteristics (Table 2).Similarly, there were no statistically significant differences between the participating and dropout students according to their study condition (control and intervention groups, data not shown).Median score into categories: from 0.0 to 1.0 was defined as low quality, from 1.1 to 2.0 was defined as moderate quality and from 2.1 to 3.0 was described as high quality.

Evaluation of the implementation process of the intervention
In general, we report a moderate or high implementation of the intervention, especially in terms of the knowledge of the intervention, the distribution of newsletters in Physical Education classes, the distribution of fruit and the installation of bike racks at the schools.In Florianopolis, the implementation items that had elevated median scores were the knowledge and visibility of the intervention, fruit distribution, newsletter distribution and acquisition of new materials for Physical Education classes.In Recife, the items with the highest implementation were the installation of bike racks and the acquisition of new materials for the Physical Education classes.
In the "Saúde na Boa" intervention, strategies to disseminate information about the intervention were primarily dependent on the school staff (e.g., teachers, coordinators and other associated personnel).For example, the posters that provided information on the intervention objectives and encouraged participation were first distributed to the staff, who were then responsible for further communication within each school.They had the discretion to place or not place the material in strategic locations.However, no observation of educative materials on the school walls suggests that either the posters were not displayed or were displayed only briefly.We attribute this to a communication failure within the schools, as well as the general disorganization resulting from the period of strike.Additionally, the initial evaluations in Recife were performed at the beginning of a new year (2007) during a period in which the posters might have been removed for cleaning and painting of the school facilities.Other informational resources might be an easier way to disseminate information, such as making promotional notebooks or other materials that are frequently used by the students.
The responsibility of paying for, purchasing, and determining guidelines for sanitizing fruit distribution to affect environmental change was the responsibility of the "Saúde na Boa" research team.However, fruit distribution within the schools was the responsibility of the school directors and coordinators.This strategy was better implemented in Florianopolis.Another environmental change was the installation of bike racks and, inversely, this was better implemented in Recife.Other items such as changes in schools canteens were found to have a lower implementation score.A literature review shown moderate effectiveness of educational interventions on eating habits; however, few multicomponent interventions were tested and inconclusive evidence was obtained regarding environmental interventions among adolescents 12 .
Strategies related to Physical Education classes, such as delivering newsletters, had a high level of implementation.Additionally, the availability of new materials (e.g., bike racks, balls and other materials) was also well implemented, especially in Recife.Physical Education class can be an excellent moment for dialogue about the importance of adopting healthy habits.A recent review has shown that Physical Education strategies, such as the teacher training, curriculum changes, and an increased availability of equipment and materials, can help increase the physical activity level of students 2 .

Characteristics at baseline of the participants and dropouts of the intervention
The number of dropout students in the "Saúde na Boa" intervention (n = 1,166, 54.1% of students at baseline) was higher than that normally observed and expected in physical activity and healthy eating habits interventions 6 .Several reasons might account for this phenomenon, as reported by Nahas et al. 8 .
For example, a third of the dropout students actually dropped out of school altogether to work or left during the school year due to the strike of 2006 8 .
However, the socio-demographic, lifestyle, health and psychosocial characteristics were similar between the students who remained in and who dropped out the intervention, especially as measured by the primary variables of the study (i.e., physical activity and healthy eating habits) and whether students were from the control or the intervention schools.Additionally, the proportion of participants and dropout students were similar between the two cities.These findings should hopefully reduce misgivings about the high dropout rate observed here and support the internal and external validity of the "Saúde na Boa" intervention.
It is important to note that the dropout students did have some specific characteristics.The dropout group consisted of a high proportion of working male students.Indeed, a survey conducted before the "Saúde na Boa" intervention found that 68% of the night shift students worked and 6 out of 10 working students were male 13 .Additionally, working was the most frequently cited reason for dropping out of school 8 .Another possible explanation is that the strike occurred during the intervention and likely contributed to higher dropout rates among the working students.
The proportion of students who smoked and used alcohol was higher among the dropout students than the participating students.Elevated tobacco use and alcohol consumption has been documented as more typical behavior for the night shift students 14 , and this group seems to be more susceptible to school evasion and more resistant to health messages 15 .These findings demonstrate the difficulty of implementing interventions focused on shifts where there is greater physical and mental tiredness and that include students with greater exposure to unhealthy behaviors.
One important strategy of the "Saúde na Boa" intervention was the weekly distribution of fruit to the schools.This was included to encourage the students to consume healthy foods frequently 8 .Other educational activities, such as the distribution of flyers and training about healthy eating, were also executed 8 .However, those who remained in the intervention were likely to have already reported that they enjoyed consuming fruits and vegetables.Similar findings have been shown elsewhere 16 and suggest that the pleasure and satisfaction obtained from the consumption of healthy food is an important variable in the continuity of students in dietary interventions.

Limitations and strengths of the findings
Despite the standardization of the various tested domains and the data collection, there was no fixed number of interviews to be conducted for some of the evaluation items, such that there may have been more interviews in for one group than for another.For example, more staff may have been interviewed than teachers.Interview subjects were selected according to convenience, and thus only those who agreed to participate in an interview answered the questions.
The strike may also have considerably influenced the "Saúde na Boa" intervention.Even for those students who would have otherwise rigorously followed the implementation schedule, the strike may have considerably influenced the dropout rates, which reflect both complete school evasion and intervention dropout.In addition to the reduction of the sample size, the disruption of the intervention may have affected the results, especially in the subset of male students who used tobacco and alcoholic beverages, and slept an insufficient duration.
This was an original and useful intervention because it specifically addressed the lifestyle and health of night shift high school students, which represents a distinct group the more commonly studied day shift students.This intervention was performed in two cities with different socio-cultural and structural characteristics.The use of two such different cities enriches the analysis of the quality of the investigated strategies and their possible implementation in a variety of different contexts.

CONCLUSIONS
Our process evaluation demonstrates that some strategies have a higher operational quality than others, especially regarding the knowledge of the intervention, the environmental alterations and the Physical Education class changes.Indeed, the implementation quality differed between the cities of Florianopolis and Recife in terms of some evaluation items.An important fact was that the dropout rates were similar between the cities; however, the participants and dropout students differed according to their gender, occupation and substance use.Interventions with a similar structure could be conducted on students in both the night and daytime shifts to further evaluate possible implementation strategies and their effectiveness to promote healthier lifestyles in young people.

Box 1 . 3 Distribution
Items evaluated in the process of implementation of the intervention "Saúde na Boa" and their measurement scales.Education (PE) classes Information about the intervention in PE class 0-

Table 1 .
Number of schools with positive changes observed in process evaluation and median item scores according to city and in the total sample.

Table 2 .
Comparison of baseline characteristics between participants and dropout students in the "Saúde na Boa" intervention.