ORIGINAL ARTICLE

Comparative validity of the ASSO–Food Frequency Questionnaire for the web-based assessment of food and nutrients intake in adolescents

Garden Tabacchi^1*, Anna Rita Filippi¹, João Breda², Laura Censi³, Emanuele Amodio¹, Giuseppe Napoli¹, Antonino Bianco⁴, Monèm Jemni⁵, Alberto Firenze¹ and Caterina Mammina¹

¹Department of Sciences for Health Promotion and Mother Child Care ‘G. D'Alessandro’, University of Palermo, Palermo, Italy; ²Division of Non-communicable Diseases and Life-Course, World Health Organization Regional Office for Europe, Copenhagen, Denmark; ³Agricultural Research Council, Food and Nutrition Research Centre (CRA-NUT), Rome, Italy; ⁴Sport and Exercise Sciences Unit, University of Palermo, Palermo, Italy; ⁵Centre for Sport and Human Performance, University of Greenwich, London, UK

Abstract

Background: A new web-based food frequency questionnaire (the ASSO–FFQ) was developed within the ASSO Project funded by the Italian Ministry of Health.

Objective: The aim of the present study is to assess the validity of the ASSO–FFQ at food groups, energy, and nutrients level.

Design and subjects: The validation study compared the ASSO–FFQ against a weighted food record (WFR) measuring foods, beverages and supplements intake, compiled during the week following the ASSO–FFQ administration. Ninety-two subjects aged 14–17, recruited from secondary schools in Palermo (Italy), completed the ASSO–FFQ and WFR. The intake of 24 food groups, energy, and 52 nutrients were taken as main outcomes. Tests for paired observations, Spearman and Pearson's correlation coefficients (cc), kappa statistics and classification in quintiles, Bland–Altman plots and multiple regressions, on untransformed and transformed data were used for the statistical analysis.

Results: High cc (≥0.40) were found for soft drinks, milk, tea/coffee, vegetables, and lactose; fair energy-adjusted cc (0.25–0.40) for water, alcoholic drinks, breakfast cereals, fishery products, savory food, fruit juice, eggs, and 19 nutrients. The subjects classified in the same or adjacent quintile for food groups ranged from 40% (alcoholic drinks) to 100% (dried fruit); for energy and nutrients from 43% (phosphorus, thiamin, niacin) to 77% (lactose). Mean differences were not significant for water, soft drinks, meat, sweets, animal fats, milk and white bread, and vitamin B₁₂ and folate. Limits of Agreement were broad for all food groups and nutrients. School, gender, alcohol consumption and between meals mainly affected most food groups’ intake differences. Gender stratification showed females had increased Pearson's cc for energy and 28 nutrients, such as almost all fats, carbohydrates, vitamins and minerals.

Conclusions: The ASSO–FFQ could be applied in epidemiological studies for the assessment of dietary consumption in adolescents to adequately rank food, energy and nutrient intakes at a group level.

Keywords: food frequency questionnaire; intake; nutrient; adolescent; validation; validity

Received: 6 October 2014; Revised: 6 March 2015; Accepted: 10 March 2015; Published: 15 April 2015

Food & Nutrition Research 2015. © 2015 Garden Tabacchi et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.

Citation: Food & Nutrition Research 2015, 59: 26216 - http://dx.doi.org/10.3402/fnr.v59.26216

Different tools for the assessment of dietary intake at population level have been developed worldwide, all having strength and limitations (1–3). Since the energy and nutrients intake estimated through these tools are subjected to over- or underestimation, the instrument must be previously validated to recognize the degree of error included in the tool estimation and to adjust epidemiologic measurements. Food frequency questionnaires (FFQ) have been demonstrated to be valid in ranking subjects on a range of food and nutrient intakes (4–8) even though there is an on-going need for the refinement of these existing approaches (9).

The ASSO (Adolescents and Surveillance System for the Obesity prevention) Project funded by the Italian Ministry of Health and involving different national and international partners, aims at developing an innovative web-based system for a standardized collection of data on food consumption, behaviors, and lifestyles in adolescents (10). To this purpose, an ASSO-toolkit was developed within the project (11), including different instruments for the measurement of body composition and fitness, and questionnaires for the collection of lifestyle-related behaviors and food habits. The ASSO–FFQ is one of these tools, designed for the assessment of adolescents’ food consumption. This tool was developed following a systematic literature review (9) and a meta-analysis (Tabacchi G, Filippi AR, Amodio E, Jemni M, Bianco A, Firenze A, Mammina C 2014, unpublished observations) specially performed to investigate the validation of the FFQ in adolescents. Both articles offered indications on the need to develop a new semi-quantitative FFQ that could be valid, reproducible, user-friendly, fast, and provided suggestions on its design and validation. The reproducibility of the ASSO–FFQ was investigated, showing the reliability of this instrument to correctly estimate food groups, energy and nutrients intake in adolescents (Filippi AR, Amodio E, Napoli G, Breda J, Bianco A, Jemni M, Censi A, Mammina C, Tabacchi G. The web-based ASSO-food frequency questionnaire for adolescents: relative and absolute reproducibility assessment. Nutrition Journal, 2014, 13:119. DOI: 10.1186/1475-2891-13-119).

The present study describes the comparative validity of the developed ASSO–FFQ against a 7-day weighted food record (WFR), designed to estimate food groups, energy and nutrients’ intake.

Methods

Study design and subjects

A representative sample of 92 boys and girls aged 14–17 was recruited between 2012 and 2013 from three high schools in Palermo (Italy), including one lyceum, one technical, and one professional institute. Students were provided with an informed consent to be signed by their parents and an information sheet explaining the purposes and the methodologies of the ASSO Project. Participants were asked to compile the web-based ASSO–FFQ, and on the same day a hardcopy of a 7-day WFR was distributed to them to be filled in the week following the ASSO–FFQ compilation.

The ASSO Project also involved teachers of the selected schools, who were appropriately trained through a PowerPoint presentation and a specially designed web-based tutorial, realized to well manage materials and tools of the data collection and to carefully assist students during their web compilation.

The ethical committee of the Azienda Ospedaliera Universitaria Policlinico ‘Paolo Giaccone’ in Palermo approved the study protocol (approval code n.9/2011) and all participants provided written informed consent.

ASSO–FFQ design

The ASSO–FFQ is a web-based semi-quantitative questionnaire included in the ASSO–NutFit software properly developed by the ASSO team.

The NutFit software accessed by students surveyed was in Italian language; also an English version was implemented to make the software accessible for international populations.

It requires on average 20 min to be compiled and has been developed with close ended answers, in order to allow a more direct and simple elaboration of data. It is structured in three sections (foods, beverages and dietary supplements) including 20 major groups: 12 food groups (fruit/vegetables/legumes, cereals/bread/substitutes, pasta/rice/couscous, potatoes, sweets, cheeses/yogurt, fishery products, meat, eggs, fats/oils, savory foods, regional dishes); seven beverages groups (water, soft drinks, juice/milkshakes, milk, tea, coffee, alcoholic drinks), and one dietary supplements group. Each main group is distinguished into different subgroups, for a total of 106 items. The adopted classification considered both raw and cooked vegetables and different cooking techniques for some foods, such as for potatoes.

The analyses were based on data collected between November 2013 and February 2014. The questionnaire asked to indicate the portion size and the frequency of consumption in the previous month. Portion size was assessed through the use of three pictures showing three sizes of the food/beverage (small, medium, large), and when necessary household units were used. Eight possible classes of frequency were proposed: never, 1–2 times/month, once/week, 2–4 times/week, 5–6 times/week, once/day, twice/day, 3–5 times/day.

WFR design

Reference method for the comparative analysis was a 7-day WFR, properly developed by the ASSO team. It asked to report every food/beverage/supplement consumed during a week, with details on the type of main meal (from breakfast to dinner) and between meals, the time and place, the food/beverage/supplement brand and weighted quantity, and condiments. It was asked to weigh raw foods through the use of a weight scale, and to weigh separately each component of a composed food (e.g. pasta with tomato and parmesan, sandwich with ham and cheese, etc.). Notes were also included to indicate measurement units (mainly g, ml and household units) and other suggestions for a correct compilation of the food record. After data collection, a reviewing process was performed by the ASSO team, in order to identify commonly missing items (oils, butter, etc.) or clearing up illegible terms.

Data treatment

The Italian tables of nutrient composition of INRAN (12) were used to create a database including the mean values of all nutrients that composed the different foods belonging to main groups. When content values were missing, the food composition databases of USDA (United States Department of Agriculture) (13) were consulted; for brand-named products, the related food labels were retrieved. Standard recipes and serving sizes were used to estimate the nutritional composition of preparations that were not included in the software database. Data from FFQ were then automatically transformed into daily estimated energy and nutrient intakes by the ASSO–NutFit software.

Data from the paper-based WFR were collected and transferred into the software, and, such as those from the FFQ, were automatically transformed into food/beverage/supplement daily frequencies, in order to perform the validation study.

To gather all the foods in groups having a similar nutrient composition, a final number of 24 food/beverage items were considered for the purpose of the validation study: vegetables, fresh fruit, dried fruit, nuts, legumes, breakfast cereals, white bread, bread substitutes, pasta/rice/couscous, potatoes, sweets, cheeses/yogurt, fishery products, meat, eggs, animal fats, oils, savory food, water, soft drinks, fruit juice, milk, tea/coffee, alcoholic drinks.

With regards to the nutrient analysis, a total of 52 nutrient values were considered as outcomes: total fat, SFA, myristic acid, palmitic acid, stearic acid, MUFA, oleic acid, PUFA, linoleic acid, linolenic acid, arachidonic acid, EPA, DHA, TFA, cholesterol, proteins, arginine, cystine, phenylalanine, isoleucine, histidine, leucine, lysine, methionine, tyrosine, threonine, tryptophan, valine, carbohydrates, sugar, fructose, lactose, sucrose, starch, fiber, water, calcium, phosphorus, iron, magnesium, vit A (RAE), thiamin, riboflavin, niacin, vit B₆, folate, vit B₁₂, vit C, vit D, vit E, ethanol, caffeine. Furthermore, the value of energy intake was considered. A total of 15 FFQs and one WFR were excluded from this step of analysis due to implausible energy intakes (>7,000 kcal/day).

Statistical analyses

Statistical analyses were performed in two phases.

The first phase included food groups’ intake validity analysis. Mean, median, and interquartile ranges of food group intake (measured in g and ml) were estimated both for WFR and for ASSO–FFQ; the intake estimates were not normally distributed, as evinced from the Shapiro–Wilk test. Estimates of untransformed intakes in g and ml were considered in non-parametric methodologies to compare the ranking of individuals by food group: the Wilcoxon signed rank test for difference between paired observations and Spearman rank correlation coefficient (cc) were performed to assess, respectively, the turn-out and the degree of agreement between the rankings of these two methods.

Kappa (k) was used as a measure of agreement (14) between ASSO–FFQ and WFR, and was weighted to take into account the degree of disagreement between the two instruments. The following thresholds established by Landis and Koch (15): ≤0=less than chance agreement; 0.01–0.20=slight agreement; 0.21–0.40=fair agreement; 0.41–0.60=moderate agreement; 0.61–0.80=substantial agreement; 0.81–0.99=almost perfect agreement. The food, energy, and nutrient intakes estimated by the FFQ and the WFR were categorized into quintiles, and the proportion of subjects categorized in the same quintile by both methods, in the same or adjacent quintile and in all the other quintiles (misclassification) was estimated.

Agreement in estimating absolute intakes was assessed by using parametric procedures for which log-(natural) transformed estimates were considered in order to achieve a normal distribution. A small quantity of 1 was added to estimates before making them linear with the log-transformation if participants did not consume some food group. A conservative formulation in case of small sample (n<100) was suggested to calculate the Limits of Agreement (LOA), described by Bland and Altman (16), to allow that one method, that is, ASSO–FFQ, can substitute the WFR (17):

where is the mean difference between methods, sd is the standard deviation of the difference between methods, t_n−1,0.05 is the value of t corresponding to two-sided p=0.05 for n−1 degrees of freedom and is an adjustment for small sample size.

LOA by food groups were obtained by overlaying the plot of difference versus mean between the two methods. The exponentiated mean difference provided the ratio of intake estimated by ASSO–FFQ relative to the WFR, while the exponentiated LOA ranging between 50 and 200% indicated an acceptable agreement (18).

Student's t-tests for paired samples by food groups were carried out on transformed data to indicate whether, on average, the ASSO–FFQ consistently overestimated or underestimated the WFR. Food groups showing significant difference between ASSO–FFQ and WFR were deeply investigated to identify factors associated with the validity of ASSO–FFQ intake estimates.

Multiple regression analysis was performed with the (transformed) difference of food intakes between the two methods as dependent variable, and personal characteristics, lifestyles and food habits of participants as explanatory variables. When any factor was significantly associated with the differences, it would have indicated that this factor was associated with the validity of the measures.

Personal characteristic, lifestyles, and food habits assessed included: type of school attended (lyceum, professional institute, technical institute); age; gender (male, female); weight status (underweight, normal weight, overweight, obese) estimated according to the body mass index cut-off based on international data (19) (weight and height were directly measured by the properly trained teachers); diagnosed diseases (yes, no); use of dietary supplements (yes, no); slimming regime (yes, no); being sedentary (yes, no) estimated according to adolescents watching TV or using pc/videogames for more than 3 h/day; smoking (yes, no); alcohol consumption (yes, occasionally, no); adequacy of meals (yes, no) estimated on the absence of junk food; between meals (more than one, one, none); organic food consumption (yes, no).

For food groups showing significant dependences of the differences in intake on the mean intake, the multiple regression model included also the mean intake as a predictor of the difference in intake estimates. R² was calculated to quantify the proportion of total variation explained by the explanatory variables all together.

In the second phase, energy and nutrients intakes were considered. The statistical analysis was performed following the same methodology as for food groups, with additional cc on the normalized data that were energy-adjusted weighting for the daily energy intake; a conclusive comparison between raw unadjusted, raw transformed and energy-adjusted (E-adj) transformed Pearson's cc was showed for all subjects and by gender categories. No regression analysis was performed on energy and nutrient intakes.

All statistical tests were two-sided and a significance level was considered at p<0.05. All data were analyzed by using the statistical software STATA/MP 12.1 (StataCorpLP, College Station, TX, USA).

Results

Food groups

All 92 subjects aged 14–17, mean age 15.8±1.34 years, compiled the FFQ and WFR were included in the validity analysis for food groups.

Estimated median intakes from ASSO–FFQ were at least twice those estimated from WFR for 15 of the 24 food groups considered (Table 1). Wilcoxon signed rank test assessed significant difference between the medians for 19 food groups: only white bread, sweets, eggs, animal fats, and soft drinks estimated intakes from ASSO–FFQ were not significantly different with respect to the WFR.

Spearman's cc ranged from 0.00 (pasta/rice/couscous) to 0.61 (milk) (Table 1). Threshold given by Cohen (20) for a high cc (r≥0.4) was overtaken by vegetables, soft drinks, milk and tea/coffee, although breakfast cereals, sweets, fishery products, eggs, savory food, water, fruit juice, and alcoholic drinks had a cc≥0.25, a medium value that can be considered acceptable. Lower values were found for the other groups.

The percentage of individuals correctly classified into the same quintile ranged from 14% (cheeses/yogurt) to 53% (milk), while the percentage of correctly or adjacent classified ranged from 40% (alcoholic drinks) to 100% (dried fruit) (Table 1). The weighted kappa values showed moderate/substantial agreement (k=0.41–0.80) for soft drinks, milk and tea/coffee; for vegetables, breakfast cereals, savory foods, sweets, fishery products, water and fruit juice a fair agreement (k=0.21–0.40) was found; all the other nutrients had slight agreement (k=0.01–0.20) except pasta/rice/couscous and oils, which showed lack of agreement (Table 1).

Mean differences estimated through the Bland–Altman method on transformed data showed that there is an overall overestimation of intake from ASSO–FFQ compared to the WFR, and the Student's t-test revealed that this difference is significant for 17 food groups (Table 1). When the exponentiated values of mean differences (mean ratio) were considered, they were higher than 2 for 13 food groups, namely vegetables, fresh fruit, nuts, legumes, breakfast cereals, bread substitutes, potatoes, cheeses/yogurt, fishery products, oils, fruit juice, tea/coffee, alcoholic drinks (data not shown). The LOA were too wide in all food groups, according to the thresholds suggested above; however, all food groups included at least 90% of units within the LOA (Table 1). This is shown for vegetables, pasta/rice/couscous, meat and oils intake in Fig. 1, where the differences obtained by the two methods in each subject are quite well distributed around their mean and they are within the LOA in almost all subjects.

Fig. 1.   Bland–Altman plots for the comparative validity analysis of vegetables, pasta/rice/couscous, meat, and oils. The solid horizontal lines indicate the mean difference between the two methods and the broken horizontal lines indicate the lower and upper Limits of Agreement (±t_91;0.025SDs).

Successively, for the 17 food groups whose mean difference resulted significant a relative validity analysis was performed, for which each food group was included as response variable in a regression model. The multiple regression models revealed two subgroups of the included variables, according to the higher or smaller intake's differences with the baseline categories (Table 2). The differences were smaller in those adolescents who attended a technical institute, compared to the other types of school, for fresh fruit, bread substitutes, savory food, fruit juice, and alcoholic drinks. Smaller differences were found for female compared to male adolescents in estimated intake of dried fruit, savory food, and alcoholic drinks. Obese adolescents had a mean difference higher than the others for fresh fruit. Subjects who have diseases diagnosed showed a smaller difference for vegetables. Bread substitutes were significantly associated with the adoption of a slimming regime. Smaller differences in legumes’ intake were found for sedentary subjects, whereas smokers showed smaller differences for oils. Alcohol consumers (both regular and occasional) showed higher differences in food intake compared to non-consumers for eggs and tea/coffee. Between meals involved a higher mean difference for fresh fruit, bread substitutes and savory food. Consumption of organic food showed higher differences in fresh fruit estimated intake. Age, use of dietary supplements, and adequacy of meals were found not to be associated with differences for any food group estimated intake.

No observed variable was found to be associated with the mean difference in food intake for nuts, potatoes, cheeses/yogurt, and fishery products.

The multiple regression models showed that for seven food groups out of the 17 groups, differences were significantly dependent on the mean intake: in particular, for six food groups (vegetables, dried fruit, nuts, breakfast cereals, cheeses/yogurt, and oils), the association was positive, with the differences significantly increasing with the magnitude of intake; for bread substitutes, differences were significantly larger at lower levels of intake (Table 2). R² ranged from as low as 15% (potatoes) up to 53% (breakfast cereals), explaining at least 25% of the difference variability for 11 of the 17 food groups considered.

Energy and nutrients

Out of the 92 participants, a final number of 76 subjects (65 males and 11 females) were included in the validity study for energy and nutrients, after eliminating all the implausible values related to the high energy intake estimated from the reported food consumptions in the ASSO–FFQ. The mean age was 15.8±1.26 years.

As shown in Table 3, estimated intakes from ASSO–FFQ were at least twice those estimated from WFR for 12 of the 52 nutrients considered: SFA, DHA, TFA, methionine, fructose, lactose, sucrose, starch, niacin, vit C, vit D, and caffeine. Wilcoxon signed rank test revealed significant differences between the ASSO–FFQ and WFR for all nutrients, except for vit B₁₂. The percentage of individuals correctly classified into the same quintile ranged from 9% (fructose) to 41% (lactose). The percentage of correctly or adjacently classified ranged from 43% (phosphorus, thiamin, and niacin) to 77% (lactose). Misclassification was higher for phosphorus, thiamin and niacin.

For 21 nutrients, the two methods did not show agreement, with weighted kappa values being null or less than 0; for 29 nutrients a slight agreement was found, with kappa values very close to fair levels for arachidonic acid, EPA, DHA, calcium, riboflavin, vit B12 and vit C; fair values were found for vit A and ethanol, while a moderate kappa value only for lactose.

After log-transformation, mean difference and LOA between ASSO–FFQ and WFR were estimated (Table 3). Student's t-test revealed significant difference between the ASSO–FFQ and the WFR for all nutrients, except for vit B₁₂ and folate. Exponentiated mean differences were found to be higher than 2 only for SFA, fructose, sucrose, starch, niacin, ethanol, and caffeine (data not shown). For energy intake, a mean ratio of 1.54 was found out, showing that energy intake estimate provided by the ASSO–FFQ was on average 54% higher than the real value.

Bland–Altman plots for energy and all nutrients showed that differences in intake are distributed quite well around their mean, and they were within the LOA for almost all subjects (only a few units are placed outside), even though the LOA of almost all nutrients were wider than the threshold suggested above. In general, 35 out of 52 nutrients showed at least 90% of units within the LOA and this is reported in Fig. 2 for energy, carbohydrate, calcium, and vit C.

Fig. 2.   Bland–Altman for the comparative validity analysis of energy, carbohydrate, calcium, and vitamin C. The solid horizontal lines indicate the mean difference between the two methods and the broken horizontal lines indicate the lower and upper Limits of Agreement (±t_91;0.025SDs).

Spearman's cc and crude and E-adj Pearson's cc are shown in Table 4. The threshold of 0.4 was overtaken only by lactose, while for all the others nutrients unadjusted Spearman's and Pearson's (on transformed data) cc ranged from 0 (several nutrients) to 0.38 (DHA and ethanol); however, most of those cc were not significant. Applying the adjustment for energy intake, cc values became significantly fair (cc≥0.25 and <0.4) for 19 nutrients (Table 4).

The stratification by gender showed a clear behavior: 21 out of 48 Spearman's coefficients that first were poor, increased considerably for females (namely energy, total fat, palmitic acid, MUFA, PUFA, linolenic acid, TFA, carbohydrate, sugar, fructose, starch, fiber, water, phosphorus, iron, thiamin, riboflavin, vit B6, folate, vit B12, vit E, and caffeine), and arachidonic acid and vit A cc increased for males (Table 4). Raw Pearson's cc became fair/high (≥0.25) for energy and 28 nutrients (almost all fats, carbohydrates, vitamins, and minerals); after adjustment for energy intake, 13 Pearson's cc for females were found to be higher.

Discussion

In the present study, the comparative validity of a new web-based FFQ in measuring food consumption and energy and nutrients’ intake was evaluated.

Results showed that the tested FFQ is an appropriate tool for ranking adolescents in classes of food groups, even though it is not suitable for measuring the absolute intakes of all food groups. These results are in line with other studies (4–8, 21–24).

Different analytical procedures were used to estimate the validity of the ASSO–FFQ. Based on the comparison between food group medians, the measured intakes of white bread, sweets, eggs, animal fats, and soft drinks were not significantly different between the two methods, thus indicating a high consistency in estimating these foods’ intake by the ASSO–FFQ. The other foods were overestimated, as reported in other studies (25, 26).

Around 50% of the food groups, namely vegetables, breakfast cereals, fishery products, eggs, savory food, water, fruit juice, soft drinks, milk, tea/coffee, and alcoholic drinks, showed fair/high Spearman's cc values. In this study, vegetable correlation was quite high, and this is not in line with most studies reporting generally low cc values for vegetables (7, 27). Fruit and sweets cc were low, as reported in the study by Vereecken et al. (28).

When the estimation of agreement was performed through the classification into quintiles, results similar to those reported by Deschamps et al. (5) were obtained; on average 63% of individuals correctly classified into the same or adjacent quintile, thus demonstrating that the ASSO–FFQ is able to correctly rank food groups. The foods that best ranked individuals were dried fruits, eggs, fishery products, animal fats, soft drinks, milk, tea/coffee. Misclassification was higher for breakfast cereals and alcoholic drinks, whose assessment through the ASSO–FFQ should be taken into consideration. Breakfast cereals are quite difficult to quantify, as much as the alcoholic drinks, which are often consumed only during the weekend and according to a ‘binge drinking’ approach.

The intake mean differences obtained through the Bland–Altman method for food groups suggested that for 40% of food groups intake estimates provided by ASSO–FFQ were less than twice those estimated from the WFR, a value that we consider acceptable. However, as revealed by the wide LOA obtained, the ASSO–FFQ was not able to correctly estimate absolute food intakes, and this is in line with different validation studies of other FFQs (7, 8). Nonetheless, a good distribution of values around the mean difference was found for most of the food items.

Type of school, gender, alcohol consumption, and between meals were significant explanatory variables of the intake differences between FFQ and WFR, even though weight, diseases, slimming regime, sedentary behavior, smoking, and organic food consumption can influence the validity for some food group. These findings are in line with those reported by others (7, 25, 29) and confirm different levels of validity depending on personal characteristics, lifestyles and food habits of participants. Moreover, differences between the WFR and FFQ were not constant across levels of intake, but were dependent on the magnitude of intake for seven food groups. The models for vegetables, fresh fruits, dried fruits, nuts, breakfast cereals, bread substitutes, eggs, oils, savory food, fruit juice and alcoholic drinks explained at least 25% of the variation in difference between ASSO–FFQ and WFR. The multivariable modeling suggests that ASSO–FFQ is not able to correctly quantify in particular the absolute intake of nuts, cheeses/yogurt, fishery products, and potatoes, and this may be explained by different reasons. As nuts are not a frequently consumed food, adolescents probably did not eat nuts in the week they compiled the food record. For cheeses/yogurt, the need to improve the photographs for the portion size estimation could be discussed. Consumption of fishery products could be affected by the presence of seasonality: lower intake levels reported in the WFR could be explained by a lack of fish availability on the market, due to adverse meteorological conditions. Finally, potato intake estimates were quantified in detail within the ASSO–FFQ, since they were separated according to the different types of cooking (fried, fried in the package, boiled or baked, mashed and potato dumplings) and it is a frequently consumed food; so the disagreement might depend on the photographs used to quantify the portion, which in this case should be modified, or on other factors not accounted in this validation study.

With regards to the nutrients, the authors conclude that the ASSO–FFQ overestimates nutrients intake and is not appropriate to assess absolute intake of most nutrients; however, the FFQ ranks individuals with reasonable accuracy. These findings are in line with different previous studies (4, 5, 8, 24, 30).

The agreement in quintile categorization was similar to that observed by Deschamps et al. (5) and Hong et al. (30). On average, 54% of individuals were correctly classified into the same or adjacent quintile, thus demonstrating that ASSO–FFQ is able to ranking subjects on a range of nutrient intakes. Presenting data categorized in quintiles provides compact information concerning the capacity of both methods to allocate individuals according to dietary intake distribution, and it is considered more adequate than the correlation coefficient, which merely produces information concerning the possible relations between the variables estimated by both methods (6).

The mean agreement and the LOA between the tested FFQ and the WFR obtained according to the Bland–Altman procedures were not acceptable for SFA, fructose, sucrose, starch, niacin, ethanol, and caffeine, since their overestimation was more than double; this is in line with the study from Lietz et al. (26). Mean agreement was good for vit B₁₂ that showed no significant difference between the two methods, and was acceptable for all the other nutrients. Even though the Bland–Altman LOA resulted quite wide, the differences in the intake were well distributed around their mean and more than half nutrients had at least 90% of units within the LOA, according to some authors (25, 31).

When Spearman's cc were considered, values were found to be low/very low for energy and most nutrients, even though they were not significant; with concern to energy, the very low correlation found was in contrast with most authors that reported values >0.20 (26, 32, 33). Only EPA, DHA, lactose, vit C, and ethanol had significant cc>0.25. Although comparison with other studies is difficult due to methodological differences, our findings are in line with other studies reporting low cc for different nutrients (26, 34–37), but disagree with the other validation studies reporting in general fair/high cc (6, 25, 38). After energy adjustment, correlations increased and could be retained fair/high for about half nutrients, such as different subgroups of fats, almost all amino acids, lactose, phosphorus, iron, vit C and vit D. The energy adjustment's procedure allowed us to obtain more precise and higher estimates of the correlation coefficient, as reported also in other validation studies (26, 30, 33). These higher correlations may be explained by a decrease in correlated measurement error for total energy and nutrients that exceeded the reduction in between-person variations for nutrient intake as a consequence of controlling for total energy intake (33). The decreased correlation coefficients reported after energy adjustment by other authors (6, 39) could be explained if considering that subjects may not have reported foods rich in nutrients such as fat in the same way during assessment for both dietary intake methods (30). Instead, for total fat, SFA (excepted palmitic acid), MUFA, PUFA (with the exception of linolenic acid, EPA and DHA), TFA, cholesterol, proteins, sugar, fructose, sucrose, starch, fiber, water, calcium, magnesium, and most vitamins, the ASSO–FFQ validity seems to be low even after adjustment for energy, thus it is not accurate in determining the magnitude of intake for these nutrients. These results are discouraging if gender was ignored; in fact, when the stratification by gender was assessed, the correlation increased significantly in females for most nutrients, since females should report intakes more correctly, as already suggested by different authors (29–31, 40). Our results may then be explained by the low number of females recruited in the study compared to males, which could have affected the validity of the FFQ, also considering that the ASSO–FFQ did not perform as well for the females as it did for the males in adequately classifying individuals according to their nutrient intakes. This could indeed be considered one limitation of our study.

When comparison of food and nutrients intake validity through cc was performed, intakes of both milk and lactose (which is contained almost exclusively in milk) showed good correlation; the same happened with EPA and DHA, principally contained in fishery products, which is hypothesized to be affected by seasonal conditions; and with SFA comprised in animal fats, milk, eggs.

In summary, significant correlations were observed for food groups, and significant cc after energy adjustment and after stratification by gender were observed for energy and nutrients, but when assessing the between-method differences using the Bland–Altman analysis, the LOA were quite broad, which is unacceptable for dietary assessment purposes. Thus, ASSO–FFQ is not able to assess absolute intake of foods, energy and nutrients, and the agreement with the WFR estimated by the kappa statistics was very poor for nutrient intakes; however, as its ability to classify individuals into quintiles was good, it could be considered a valid tool for correctly ranking subjects in classes of food, energy and nutrient intakes.

A WFR was adopted as reference method because it does not depend on the recall and it has the potential for allowing direct measurement of food quantities (41); nonetheless, WFR may have the disadvantage of recalling inadequately measure of foods episodically consumed (42), that could have affected the intake report, such as the case for nuts.

As underreporting is the main bias that interferes with the food intake data obtained from food records (43–45), and overreporting is usual in FFQ (4, 24), possible underreporting in the food records and overreporting in the FFQ might have had a synergic effect in influencing the results observed in this study, for example, by increasing the intake mean differences between the two methods. As stated by Vereecken et al. (8), over- and underestimation are important issues and the current findings suggest to increase the cut-off values of the recommended intakes or to apply a correction factor to decrease the reported intake of the nutrients too overestimated (such as fiber, vitamin C, calcium, and iron), and to increase the percentage of energy from fat before being evaluated for feedback.

Twenty-four food groups, energy, and 52 nutrients were analyzed in multiple ways; this could have introduced multiple testing bias and should be taken into account in further studies.

Another aspect is that adolescents have difficulties in recalling what food they have eaten (36), for example, food eaten away from home (46); they can have limited knowledge of food and food preparation or problems in the perception and quantification of the portion size (46). According to Goran (47), adolescents tend to have better recall of the preferred foods, and to forget or underestimate those items that they do not like. Furthermore, the composite dishes (e.g. pasta/rice/couscous with meat) included in the FFQ, by which dishes were classified as the whole mixture or on the basis of its primary ingredient, could play a role (48). According to these considerations, the presence of systematic errors cannot be excluded.

Conclusions

Findings suggest that the ASSO–FFQ could be applied in epidemiological studies for the assessment of dietary consumption in adolescents to adequately rank food, energy, and nutrient intakes at a group level. The presence of a section where regional/local foods may be selected makes it suitable for use in other regions for the comparison of food and nutrient consumptions. Moreover, it is also developed in the English language and could be easily translated into other languages, with the potential to be applied in other countries after adaptation to the local culture and food habits.

Authors’ contributions

GT performed the conception and design of the study, carried it out, analyzed and interpreted data, and wrote the article. AR performed the statistical analysis and interpretation of data and contributed in drafting the article. EA and AB contributed in drafting the article and revising it critically. JB, LC, MJ, GN, AF, and CM critically revised the article.

Acknowledgements

The work has been performed within the Adolescents and Surveillance System for the Obesity prevention (ASSO) Project (code GR-2008-1140742, CUP I85J10000500001), a young researcher's project funded by the Italian Ministry of Health. The ethical approval was given by the ethical committee of the ‘Azienda Ospedaliera Universitaria Policlinico Paolo Giaccone’ (approval code n.9/2011).

Conflict of interest and funding

The authors declare that they have no conflict of interest and funding. João Breda is staff member of the WHO Regional Office for Europe. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions or the stated policy of the World Health Organization.

References

1. Thompson FE, Subar AF. Dietary assessment methodology. In: Coulston AM, Allen L, Boushey CJ, eds. Nutrition in the prevention and treatment of disease. 2nd ed. San Diego, CA: Academic Press; 2005, pp. 5–45.
2. Ngo J, Engelen A, Molag M, Roesle J, García-Segovia P, Serra-Majem L. A review of the use of information and communication technologies for dietary assessment. Br J Nutr 2009; 101(Suppl 2): 102–12.
3. Poslusna K, Ruprich J, de Vries JH, Jakubikova M, van't Veer P. Misreporting of energy and micronutrient intake estimated by food records and 24 hour recalls, control and adjustment methods in practice. Br J Nutr 2009; 101(Suppl 2): 73–85.
4. Araujo MC, Yokoo EM, Pereira RA. Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc 2010; 110: 1170–7. PubMed Abstract
5. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009; 63: 282–91. PubMed Abstract | PubMed Central Full Text
6. Slater B, Philippi ST, Fisberg RM, Latorre MR. Validation of a semi-quantitative adolescent food frequency questionnaire applied at a public school in São Paulo, Brazil. Eur J Clin Nutr 2003; 57: 629–35. PubMed Abstract
7. Matthys C, Pynaert I, De Keyzer W, De Henauw S. Validity and reproducibility of an adolescent web-based food frequency questionnaire. J Am Diet Assoc 2007; 107: 605–10. PubMed Abstract
8. Vereecken CA, De Bourdeaudhuij I, Maes L. The HELENA online food frequency questionnaire: reproducibility and comparison with four 24-h recalls in Belgian-Flemish adolescents. Eur J Clin Nutr 2010; 64: 541–8. PubMed Abstract
9. Tabacchi G, Amodio E, Di Pasquale M, Bianco A, Jemni M, Mammina C. Validation and reproducibility of dietary assessment methods in adolescents: a systematic literature review. Public Health Nutr 2014; 17(12): 2700–14. doi: 10.1017/S1368980013003157. PubMed Abstract | Publisher Full Text
10. Tabacchi G. Asso project: a challenge in the obesity prevention context. J Sport Sci Law 2011; 1–3(III) 2: 267–9.
11. Tabacchi G, Bianco A. Methodological aspects in the development of the lifestyle surveillance toolkit in the ASSO project. J Sport Sci Law 2011; IV(4), 2: 163–6.
12. The Italian tables of nutrient composition of INRAN, 2000. Available from: http://nut.entecra.it/646/tabelle_di_composizione_degli_alimenti.html
13. U.S. Department of Agriculture, Agricultural Research Service. USDA national nutrient database for standard reference, release 26. Nutrient Data Laboratory; 2013. Available from: http://www.ars.usda.gov/ba/bhnrc/ndl
14. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70: 213–20. PubMed Abstract
15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–74. PubMed Abstract
16. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–10. PubMed Abstract
17. Ludbrook J. Statistical techniques for comparing measurers and methods of measurement: a critical review. Clin Exp Pharmacol Physiol 2002; 29: 527–36. PubMed Abstract
18. Ambrosini GL, van Roosbroeck SA, Mackerras D, Fritschi L, de Klerk NH, Musk AW. The reliability of ten-year dietary recall: implications for cancer research. J Nutr 2003; 133: 2663–8. PubMed Abstract
19. Cole TJ, Bellizzi MC, Flegal KM, Dietz WH. Establishing a standard definition for child overweight and obesity worldwide: international survey. BMJ 2000; 320: 1240–3. PubMed Abstract | PubMed Central Full Text
20. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
21. Lassale C, Guilbert C, Keogh J, Syrette J, Lange K, Cox DN. Estimating food intakes in Australia: validation of the Commonwealth Scientific and Industrial Research Organisation (CSIRO) food frequency questionnaire against weighed dietary intakes. J Hum Nutr Diet 2009; 6: 559–66.
22. Hjartåker A, Andersen LF, Lund E. Comparison of diet measures from a food-frequency questionnaire with measures from repeated 24-hour dietary recalls. The Norwegian Women and Cancer Study. Public Health Nutr 2007; 10: 1094–103.
23. Sasaki S, Matsumura Y, Ishihara J, Tsugane S, JPHC. Validity of a self-administered food frequency questionnaire used in the 5-year follow-up survey of the JPHC Study Cohort I to assess dietary fiber intake: comparison with dietary records. J Epidemiol 2003; 13(Suppl 1): 106–14.
24. Ambrosini GL, de Klerk NH, O'Sullivan TA, Beilin LJ, Oddy WH. The reliability of a food frequency questionnaire for use among adolescents. Eur J Clin Nutr 2009; 63: 1251–9. PubMed Abstract
25. Bertoli S, Petroni ML, Pagliato E, Mora S, Weber G, Chiumello G, et al. Validation of food frequency questionnaire for assessing dietary macronutrients and calcium intake in Italian children and adolescents. J Pediatr Gastroenterol Nutr 2005; 40: 555–60. PubMed Abstract
26. Lietz G, Barton KL, Longbottom PJ, Anderson A. Can the EPIC food-frequency questionnaire be used in adolescent populations? Public Health Nutr 2002; 5: 783–9. PubMed Abstract
27. Van Assema P, Brug J, Ronda G, Steenhuis I, Oenema A. A short Dutch questionnaire to measure fruit and vegetable intake: relative validity among adults and adolescents. Nutr Health 2002; 16: 85–106. PubMed Abstract
28. Vereecken CA, Maes L. A Belgian study on the reliability and relative validity of the health behaviour in school-aged children food-frequency questionnaire. Public Health Nutr 2003; 6: 581–8. PubMed Abstract
29. Marks GC, Hughes MC, van der Pols JC. Relative validity of food intake estimates using a food frequency questionnaire is associated with sex, age, and other personal characteristics. J Nutr 2006; 136: 459–65. PubMed Abstract
30. Hong TK, Dibley MJ, Sibbritt D. Validity and reliability of an FFQ for use with adolescents in Ho Chi Minh city, Vietnam. Public Health Nutr 2010; 13: 368–75. PubMed Abstract
31. Shatenstein B, Amre D, Jabbour M, Feguery H. Examining the relative validity of an adult food frequency questionnaire in children and adolescents. J Pediatr Gastroenterol Nutr 2010; 51: 645–52. PubMed Abstract
32. Watanabe M, Yamaoka K, Yokotsuka M, Adachi M, Tango T. Validity and reproducibility of the FFQ (FFQW82) for dietary assessment in female adolescents. Public Health Nutr 2011; 14: 297–305. PubMed Abstract
33. Rockett HR, Berkey CS, Colditz GA. Comparison of a short food frequency questionnaire with the youth/adolescent questionnaire in the growing up today study. Int J Pediatr Obes 2007; 2: 31–9. PubMed Abstract
34. Cullen KW, Zakeri I. The youth/adolescent questionnaire has low validity and modest reliability among low-income African-American and Hispanic seventh-and eighth-grade youth. J Am Diet Assoc 2004; 104: 1415–19. PubMed Abstract
35. Field AE, Peterson KE, Gortmaker SL, Cheung L, Rockett H, Fox MK, et al. Reproducibility and validity of a food frequency questionnaire among fourth to seventh grade inner-city school children: implications of age and day-to-day variation in dietary intake. Public Health Nutr 1999; 2: 293–300. PubMed Abstract
36. Rockett RH, Colditz GA. Assessing diets of children and adolescents. Am J Clin Nutr 1997; 65(Suppl 4): 1116–22.
37. Harnack LJ, Lytle LA, Story M, Galuska DA, Schmitz K, Jacobs DR, et al. Reliability and validity of a brief questionnaire to assess calcium intake of middle-school-aged children. J Am Diet Assoc 2006; 106: 1790–5. PubMed Abstract
38. Papadopoulou SK, Barboukis V, Dalkiranis A, Hassapidou M, Petridou A, Mougios V. Validation of a questionnaire assessing food frequency and nutritional intake in Greek adolescents. Int J Food Sci Nutr 2008; 59: 148–54. PubMed Abstract
39. Cullen KW, Watson K, Zakeri I. Relative reliability and validity of the block kids questionnaire among youth aged 10 to 17 years. J Am Diet Assoc 2008; 108: 862–6. PubMed Abstract
40. Sjoberg A, Hulthén L. Assessment of habitual meal pattern and intake of foods, energy and nutrients in Swedish adolescent girls: comparison of diet history with 7-day record. Eur J Clin Nutr 2004; 58: 1181–9. PubMed Abstract
41. Gibson RS. Principles of nutritional assessment. New York: Oxford University Press; 2005.
42. Hoffmann K, Kroke A, Klipstein-Grobusch K, Boeing H. Standardization of dietary intake measurements by nonlinear calibration using short-term reference data. Am J Epidemiol 2002; 156: 862–70. PubMed Abstract
43. Livingstone MB, Robson PJ, Wallace JM. Issues in dietary intake assessment of children and adolescents. Br J Nutr 2004; 92(Suppl 2): 213–22.
44. Chinnock A. Validation of an estimated food record. Public Health Nutr 2006; 9: 934–41. PubMed Abstract
45. Andersen LF, Pollestad ML, Jacobs DR Jr, Løvø A, Hustvedt BE. Validation of a pre-coded food diary used among 13-year-olds: comparison of energy intake with energy expenditure. Public Health Nutr 2005; 8: 1315–21. PubMed Abstract
46. Rockett HR, Wolf AM, Colditz GA. Development and reproducibility of a food frequency questionnaire to assess diets of older children and adolescents. J Am Diet Assoc 1995; 95: 336–40. PubMed Abstract
47. Goran MJ. Measurement issues related to studies of childhood obesity: assessment of body composition, body fat distribution, physical activity, and food intake. Pediatrics 1998; 101: 505–18. PubMed Abstract
48. Krebs-Smith SM, Cronin FJ, Haytowitz DB, Cook DA. Contributions of food groups to intakes of energy, nutrients, cholesterol, and fiber in women's diets: effect of method of classifying food mixtures. J Am Diet Assoc 1990; 90: 1541–6. PubMed Abstract

^*Garden Tabacchi
Department of Sciences for Health Promotion and Mother Child Care ‘G.
D'Alessandro’
University of Palermo
Via Del Vespro 133, IT-90127 Palermo
Italy
Email: tabacchi.garden@libero.it