Agreement and reliabilty of an eletronic questionnarie for children ( WEBDAFA )

The objective was to analyse the agreement between the printed version (PV) and electronic version (EV) of an instrument to assess PA in children and to verify the reliability between two applications of the EV. The sample included 230 seven to tenyear-old children, from two schools (public and private). The sample was divided into two groups: a) analysis of agreement (n = 130), two applications of the questionnaire (EV vs. PV); b) reproducibility analysis (n = 127), two applications of the EV (test-retest). There was no significant difference between the PV and PE in the PA domain (48,6 vs 46,8). The intraclass correlation coefficient (analysis of concordance) was significant for all types of PA (p <0.05). In the reliability analysis, the difference between the scores of PA test-retest was of 2.70, the concordance was over 70% and the intraclass correlation was 0.84 (95% CI = 0.78 to 0.89). The electronic questionnaire (WEBDAFA) presented appropriate psychometric indices of agreement and reliability when applied to children aged seven to ten years.


INTRODUCTION
Physical activity (PA) is associated with health benefits for children and adolescents 1,2 .Children who are physically active have better bone and muscle growth 3 , besides presenting lower chance of developing non-contagious chronic diseases such as obesity, diabetes, high blood pressure, and metabolic syndrome [1][2][3] .Evidence also suggests that behaviors acquired and consolidated during childhood and adolescence are more likely to remain in adulthood [4][5][6] .Thus, the quantification of PA in children becomes important in order to identify its prevalence and also to diagnose AP patterns of population subgroups.
Objective measurements of PA such as direct observation, accelerometry, pedometers, and energy expenditure are accurate and valid in children, however, they are costly and require specific equipment, which limits their application in epidemiological studies [7][8][9] .Subjective measures such as questionnaires, interviews, and diaries based on the self-report of a child or parent are widely used to measure PA 8,9 .However, the use of questionnaires to assess the habitual practice of PA in children has limitations because children are involved in spontaneous and intermittent activities, which can result in errors of recall bias in the self-report of PA 10,11 , not to mention that there are few valid tools for this population.
In an attempt to minimize these errors and possible misunderstandings in the interpretation of complex questions, questionnaires to assess PA in children have used specific tools with appropriate language for this public 12 .In Brazil, the questionnaire typical day of PA and food intake (DAFA, in Portuguese) is the only tool developed and validated specifically for Brazilian children of ages between seven and ten years old.This tool identifies the PA done during a typical day based on illustrative drawings, making the evaluation more attractive and less complex for children 12 .The PA section of this tool was validated based on the reports from parents and teachers 12 .
Recently studies have used electronic questionnaires via internet to measure different outcomes related to health [13][14][15][16] .It is believed that this type of application can be more attractive among children and result in more accurate values to measure PA 17 .Further advantages relate to financial savings because it eliminates the printing of the questionnaires, it saves time, and it is also possible to apply the tool at different locations simultaneously.This technology gives direct access and greater accuracy in the collection and storage of the data 13,18 .Despite all these benefits listed above, studies conducted in Brazil were not found in the literature that have evaluated the agreement and reliability of electronic tools for measuring PA in children.
Faced with the need to provide an tool via the internet that is a consistent and appropriate method to measure the practice of habitual PA in children, minimize costs, and also improve the quality of information obtained, the present study presents the following objectives: analyze the agreement of the measurements of PA in children obtained by applying the printed version (PV) of the DAFA questionnaire versus the electronic version of the WEBDAFA (EV) questionnaire, as well as check the reliability (test-retest) of the PA measurements obtained through the WEBDAFA electronic questionnaire in children.

Study Profile
We selected 230 children between the age of seven and ten who were enrolled in two schools (one public and one private).In each school two groups were selected: a) concordance analysis (n=130) and b) the reproducibility analysis (n=127).The selection of the schools was done intentionally.Eight groups were randomly selected in each school (1st through 4th grade), two from each grade, and all students from each group selected were invited to participate in the study (n=400).After sending consent forms, 250 (return rate of 57.5%) of these returned and 20 of these students were excluded because they were over 10 years old.
In the analysis of the agreement, the students first answered the PV and then latter the EV.The reliability was checked by applying the electronic tool on two occasions.The interval between applications was of one day for the procedures of agreement and test-retest, respectively.The data collection procedures were conducted between the months of October and November 2010 in a computer lab room under the guidance of the researcher and supervision of the teacher responsible for the class.
In the application of both tools, the researcher presented the questionnaire to the students with a concise explanation of all the sections, highlighting the concept of a typical day, meaning the PA done on most days of the week 12 using a vocabulary appropriate for the age group of children and gestures to emphasize the actions and with the help of a projector of images.

Application of printed tool
The printed tool used as the reference method was the DAFA (Typical Day of Physical Activities and Food Intake) questionnaire, which is a structured questionnaire developed with the purpose of obtaining information about the PA habits of a typical day and the Food Intake of children between the ages of seven and ten years old.It was tested with Brazilian schoolchildren and presented adequate psychometric measurements of validity and reproducibility for measuring PA 12 .This study's project was approved by the Ethics Committee on Human Research at the State University of Western Paraná (decision 494/2010).
This tool consists of three sections: a) identification of personal data (gender, grade, shift, body mass, and height); b) types of PA done in a usual day (typical day); c) food consumed in five meals during a typical day.This study will present the results regarding the PA section.The representation of each type of PA is done by means of 11 drawings that indicate each category (dancing, walking, playing with a pet, doing household chores, riding a bike, jumping rope, climbing stairs, playing ball, swimming, skateboarding, and doing gymnastics or stretching) and intensity (slow, fast, and very fast) of the PA.In the analysis of the physical activity of each category, a value was assigned for each intensity, slow (=3), fast (=5), and very fast (=9), which can generate scores by category as follows: (0) zero for children who said they had not done any physical activity and 3.5 or 9 for children who marked only one of the three intensities.In the other cases the possibilities of the PA scores would be as follows: 8 (slow + fast), 12 (slow + very fast), 14 (fast + very fast), and 17 (slow + fast + very fast).The overall score of PA would be the result of the sum of all categories of PA, which may result in a PA overall score of 187 presented in the tool.To apply the printed tool we followed the guidelines proposed in the original study 12 .

Application of Electronic tool (internet)
The EV of the Typical Day of Physical Activities and Food Intake) questionnaire, named by the authors as Webdafa, was developed through an internet (web) application that used the programming language Hypertext Preprocessor 5 (PHP 5) 19 , Java Script 20 .It has a graphical interface where users interact using Adobe Flash technology 21 .The project is hosted on the site http://www.criancaativaesaudavel.com.br and the form can be accessed through web browsers (Internet Explorer, Mozilla, Google Chrome).
The Webdafa consists of the same structure of the printed tool, differing in only the interface mode for filling out the answers.A management system was added of a database adopted for storing information via the MySQL 5 method 19,22 , which makes it possible to generate reports that can be exported to spreadsheets in Excel for Windows (xls).In order to submit answers in the electronic version, the children were instructed to move the mouse's cursor over the picture and double click on it (Figure 1).

Analysis of data
The information on the printed questionnaire (DAFA) was entered into the program Epidata 3.1 and inconsistencies checked by the double entry method.The data from the Webdafa questionnaire were obtained from a spreadsheet from the Excel for Windows program generated by the electronic MySQL5 system 19 .
The data was analyzed using the SPSS version 17.0 program for Windows.A descriptive analysis (mean, standard deviation, and frequency percentage) was done to describe the sample.The normality of the data was analyzed using the Kolmogorov-Smirnov test.All the variables of the habitual practice of PA differed from the normality curve and because of this non-parametric tests were used to compare the groups.There was no significant difference (p<0.05) between the scores of PA according to gender and type of school so the analyses were performed grouped.The comparison between the percentages was obtained by the chi-square test.
The agreement was analyzed by comparing the mean of the scores (mean scores of posts) obtained by the printed and electronic versions.Individual analyses were done per question and by the overall PA score through the Mann-Whitney U test.Furthermore, the intraclass correlation coefficients and the percentage of agreement between the answers of the first and second application were used.The overall score of PA was obtained by adding the scores of each type of PA.The dispersion was tested between the methods by the Bland and Altmann test 23 .The Kappa test was used to analyze the agreement between the classifications obtained from both methods and because of this the overall score was dichotomized in relation to the median.
For the reliability analysis, the mean difference was checked between the scores of the electronic questionnaire Webdafa obtained on two occasions (test-retest) through the Mann-Whitney U test.The percentage of agreement and intraclass correlation coefficient were also employed in this analysis.

Ethical aspects
All participants in the study were volunteers and the parents or guardians signed the informed consent form authorizing participation in the

RESULTS
The study sample consisted of two groups from public (42.2%) and private schools with the following characteristics: age of 8.4 years (sd=1.1 years), height of 1.30 m (sd=0.7),body mass of 32.1 kg (sd=7.1),and body mass index of 17.4 kg/m 2 (sd=2.6).The sample of agreements was composed of 103 children (50.5% girls) with a predominant age of eight years old.The reproducibility sample was composed of 127 children (45.7% girls).In this sample a higher proportion of girls from public schools and overweight was observed in relation to the number of boys (p<0.05).Other characteristics of the sample are presented in table 1.There was no significant difference between the PA scores obtained from the PV versus the EV (Table 2).The intraclass correlation coefficient was significant for all types of PA (p<0.05),ranging from 0.79 (household chores) to 0.93 (swimming).Among the 11 types of PA analyzed, only the items playing ball (25.2%), jumping rope (36.8%), and swimming (41%) showed relative agreement of less than 70%.However there was high agreement (88.3%) of the overall score of PA and a kappa index of 0.76 (p<0.001) when comparing the questionnaires printed versus electronic (data not shown).The dispersion analysis (figure 2) indicated a mean error of 1.7 (CI95% = -25.6;29.1).PA scores, low and medium, showed good agreement, but the higher scores presented higher dispersion.
Reliability was checked by test-retest (table 3).All items presented agreement (%C) above 70% and an intraclass correlation coefficient between 0.84 (riding bicycle) and 0.95 (playing with a skateboard).There was a significant difference (p<0.05) in the PA scores between the two applications of the EV of the WEBDAFA questionnaire only for the types of PA "household chores and riding bicycle" (table 3).* There was no difference between the application methods (Mann-Whitney U). p<0.05.ICC: Intraclass Correlation Coefficient.CI95%: confidence interval.Abbreviations: PV -Printed Version; EV -Electronic Version.

DISCUSSION
The analysis of agreement between the printed and electronic methods showed significant intraclass correlation values for most items analyzed (ICC> 0.79).The overall PA score obtained by the two methods did not differ statistically, indicating agreement between the methods.The reliability analysis indicated an agreement percentage higher than 70% and a high intraclass correlation coefficient (ICC> 0.84) for all items of the electronic tool.Among the 11 categories of PA, only three had mean scores that differed (p<0.05).This indicates appropriate values of reliability for the tool applied in the electronic version on children.
The Webdafa tool was designed based on the use of new electronic technologies to collect health-related information.The mode of application aroused the interest of the child to answer the tool and to fill it out was fun and fast.The electronic application of the tool has advantages such as how to eliminate the cases missing from the analysis.There is no need to enter the results from the questionnaires into the database, thus eliminating the chance of typing errors, saving time in applying the questionnaire, and the data can be available in real time via internet for different locations.These advantages listed have encouraged some researches and surveys via internet with various health-related outcomes 24,25 .
In general, tools to assess PA in an electronic format arouse the interest in children the most [25][26][27][28][29] .In general, when analyzed individually, the measures of the habitual PA obtained through electronic questionnaires are undervalued and have low or moderate correlation with measurements obtained from accelerometers [27][28][29] .However, according to some authors, the electronic questionnaires are more reliable estimates for the PA of groups than the individual estimates 28,29 .
Although this study did not conduct objective measures of PA, no differences were observed between the sum of the PA scores obtained by the printed questionnaire versus the electronic questionnaire.Even when the analyses were made between gender, these results corroborate with studies done previously with this type of tool 25,27,29 .
In the reliability analysis, the Webdafa electronic questionnaire showed intraclass correlation coefficients between 0.73 to 0.93.This data is superior to those found in the original version, which ranged from 0.50 to 0.83 30 .In general, electronic questionnaires have shown high intraclass correlation coefficients 17,28,29 .This demonstrates that electronic questionnaires have a good internal consistency and stability, which suggests that they are promising to use in epidemiologic surveys because it is an inexpensive method, viable, and useful for monitoring levels (behavior) of habitual PA of children by group level.
Currently a significant proportion of schools in southern and southeastern Brazil have computers connected to the worldwide network, which can provide a large number of children with immediate access to the Webdafa electronic questionnaire with relative ease.In theory, researchers from various regions of Brazil could investigate the PA in children simultaneously.Thus, we suggest that further studies be conducted using the Webdafa questionnaire with different population groups.
The intentional selection of schools and classes, the short period between the first application (test) and second application (retest), as well as the lack of a reference measurement for validating the PA scores obtained by the electronic questionnaire may be considered the main limitations of this study.Therefore, we recommend that further studies of agreement and validation of the WEBDAFA questionnaire be done in other regions of Brazil with children from different social classes, especially where they do not have much familiarity with computers.Because of this, the results of this study should be treated with caution and cannot be extrapolated to other regions of Brazil.

CONCLUSION
Finally, it is concluded that the Webdafa questionnaire applied in its electronic form presented values of agreement and reliability suitable for measuring PA in children between seven and ten years old.

Figure 1 .
Figure 1.Illustration of the WEBDAFA electronic tool for measuring physical activity.

Figure 2 .
Figure 2. Analysis of the dispersion of the scores obtained through the application of printed and electronic tool on children.

Table 1 .
Descriptive characteristics of the sample in the study of agreement and reliability of the electronic questionnaire WEBDAFA applied to children ages seven to ten years old.

Table 2 .
Average scores of the physical activities, agreement, and correlation between the questionnaire's printed and electronic versions to measure physical activity in children ages seven to ten years old (n=103).

Table 3 .
Analysis of reliability (test-retest) of applying the WEBDAFA electronic questionnaire to measure physical activity in children ages 7-10 years old (n=127).