Data were collected over a period from October to November 2011. Four 9th-grade classes were informed about the study, and an information/invitation letter and consent form for both parents and students were handed out. After consenting to participate, the students received the first FFQ by e-mail. The e-mail contained short information about the web-based FFQ and a link to the FFQ. The participants had 1 week to finish the FFQ. A reminder message was sent after 4 days to those who had not filled in the FFQ. The first 24-HDR took place 1 week after the participant had finished the FFQ, and the second, 4 weeks after the first 24-HDR. All participants were called on a random day. Those who had not filled in the FFQ were not called.

Two weeks after the students had filled in the FFQ, blood samples were taken at school.

For the test–retest reliability study, a new link to the FFQ was sent to those who had filled in the FFQ for the first time. This was done 4 weeks after the first time FFQ. Figure 1 gives an overview of the design and recruitment.

A total of 285 students aged 13–14 from four different schools in Vest-Agder County were invited to participate in the validity and reliability study. Of which 120 students agreed to participate, and 100 students filled in the first FFQ. The same 100 students were called on a random day to take part in the first 24-HDR. Two students were missed due to a mistake, resulting in 98 students participating in the first 24-HDR. Ninety-three students participated in the second and last 24-HDR. In the results from the relative validity, only those who completed the first FFQ and both 24-HDR were included (n=93). In the test–retest study, 98 students filled in the first FFQ. Of these students 58 filled in the questionnaire the second time, which was the number included in the test–retest analyses. The absolute validation study included 92 participants who filled in the FFQ the first time and also had a blood sample taken.

The FFQ is self-administered and is based on a Danish FFQ for adolescents (19).

The FFQ consists of 156 questions. The first part includes questions about gender, age, weight and height (self-reported), and the level of activity (i.e. how many hours physical activity at school and how many hours spent in front of a computer to chat, play, surf, and do homework on a daily basis). The activity questions are validated questions from the HEVAS study (20). Furthermore, the FFQ contains questions on the average consumption of 131 food and beverage items during the past 4 weeks. These food items are divided into different sections in the FFQ (i.e. beverage, dairy products, bread and cereals, spread/cold cuts, fish and meat, fruit and vegetables, cakes/desserts/snacks). Of relevance to the validity study, the questionnaire posed questions on the intake of fatty fish, egg, milk, butter, margarine, cod liver oil, and vitamin D supplements. The frequency intervals vary according to food and beverage items. For beverages, the intervals range were never, 1–3 glasses per month, 1 glass per week, 2–6 glasses per week, 1 glass per day, 2–3 glasses per day, more than 3 glasses per day. For frequently consumed food items, the intervals range were ‘never’, 1–3 times per month, once a week, 2–3 times per week and more, 4–6 times per week to ‘once or more times a day’.

The respondents indicate their answers in ‘units per day’, ‘units per week’, and ‘units per month’. For most items, a unit is a standard portion size (i.e. a glass of juice, a bowl of cereals, an apple). For some items, extra information is provided (i.e. for soda: 1 bottle = 0.5 l).

All questions are close ended, except four which are related to dietary supplements, food-allergies, if there was anything they avoided eating, and if the last 4 weeks had been average in terms of eating habits. These questions were placed at the end of the questionnaire.

When calculating the nutrient intake, the standard portion included in the FFQ (e.g. glass of juice), which was coded according to the Norwegian standards (21) in grams, was multiplied by the frequency ticked off in the FFQ. Furthermore, each food item had a food-code linkage corresponding to the numbers in the Norwegian food composition table. For dishes, the codes adjusted for nutrient loss during cooking were chosen. Using FoodCalc and the Norwegian food composition table (22–24), the nutrient intake was calculated.

The 24-HDRs were conducted on two non-consecutive days and were based on interviews done by three master's degree students and one bachelor degree student, using the ‘triple-pass’ method from King's College London (22). The four interviewers were all given the same information on how to perform the interviews and they followed a detailed description according to the triple-pass method. They practiced in advance and all the four interviewed each other afterwards to check if they were harmonized according to the triple-pass method.

All week days were randomly assigned to the participants, aiming to cover all days of the week equally. However, due to difficulties in making contact with the participants on given days, the distribution of days for the two recalls was skewed. For recall 1 and 2, respectively, the distribution was as follows: Monday (14%, 31%), Tuesday (23%, 16%), Wednesday (25%, 14%), Thursday (12%, 10%), Friday (8%, 5%), Saturday (4%, 5%), Sunday (15%, 18%).

To help quantify the food intake for the participants, a validated food picture booklet (23) with pictures of different dishes and portion sizes was handed out at school before the first interview. The booklet consisted of 16 pictures of different household measurements and portion sizes. The 16 food pictures were categorized into four different portion sizes named A–D, where A was the smallest portion and D was the largest.

The interviewer called the participant while they were at home on their home phone/cell phone. The participant was free to get help from his or her parents while taking part in the interview and could also check food packages and brand name at home. At the beginning of the interview, the interviewer introduced the procedure, explained the ways the picture booklet was to be used, and emphasized that the participant could at any time ask questions to the interviewer if anything was unclear.

The interview was standardized according to the triple-pass method (22), which consists of three phases to assess food intake. First, the participants were asked to give a brief report of what they have eaten and drunk the previous day, without the interviewer interrupting them. Second, the interviewer collects detailed information concerning the items in ‘the quick list’ (a list of additional questions about what the participants answered in the first part). During this phase, details about composite dishes and all food items eaten were recorded. Recipes were asked for, and if they knew or the parent knew, the recipes were noted. In the third, and last, part the respondents are given the opportunity to provide additional information about the food they have eaten and liquids they have drunk over the last 4 weeks. In this part, the interviewer also had the opportunity to ask for information about foods and drinks not mentioned. Use of dietary supplements/vitamins and questions about allergies were asked for at the end of the interview.

All the information gathered by the interviewer was written down on record sheets while the interview took place. At the end of the data collection all notes were handled systematically and transferred to Excel. A food-code linkage was provided, using FoodCalc and the Norwegian food composition table (24–26) to calculate the nutrient intake. The four interviewers standardized their coding by having meetings at several stages during the coding phase. Composite dishes and recipes were coded by ingredients if given during the interview or by using a similar dish and recipe if no details were available. Food items that were not in the food composition table were discussed and similar foods were selected and the same code was used by all interviewers. Fats used in cooking were coded when informed of. For standardized portion sizes that were not in the photographic booklet, the Norwegian standards were used (21). Nutrients presented in the relative validation and reliability study are calculated without dietary supplements. However, for the absolute validation dietary supplements were included.

For the absolute validity study, we selected fatty acids and vitamin D concentration in whole blood to correlate with the calculated nutrient intake from the FFQ. Several health outcomes have been related to these nutrients and are therefore interesting (27, 28). As for vitamin D, it is found naturally in a few food items like fatty fish and fish liver (28), and is a critical and essential nutrient for human health; however, the relevance for cancer and cardiovascular diseases are currently discussed (28). The fatty acids and vitamin D concentrations were measured in capillary non-fasting whole blood using dried blood spot (DBS). The fatty acids analyzed were alpha-linolenic acid (ALA), eicosapentaenoic acid (EPA), docosapentaenoic acid (DPA), docosahexaenoic acid (DHA), and linoleic acid (LA); and for vitamin D, 25 hydroxy vitamin D₃ (25(OH)D₃) was analyzed. According to the laboratory at VitAs, all assays on DBS are compared with both internal and external reference methods on plasma or serum. The chosen laboratory function is a reference lab for fat-soluble vitamins for the Nordic External Quality assurance Program, and qualify and participates and is compliant in the largest vitamin D External Quality Assessment, called DEQAS. As found by others, both 25(OH)D₃ and fatty acid analysis perform equally well on DBS as in ordinary serum or whole blood samples (29). Drops of blood from each participant were transferred on to a special filter paper after a pinprick on fingertip. Three nurses were responsible for this procedure, and the students were taken out of their class into another classroom for the blood test. Samples were air dried for at least 2 h as recommended, packed separately in the supplied airtight Bioindex DBS transport bags containing desiccant, deep frozen at −20°C, and sent to the laboratory (Vitas AS) (30) for analyses of levels of the biomarkers 25(OH)D₃ and full fatty acid profile. The technique used to analyze fatty acids was gas chromatography with flame ionization detection (30), and the analysis technique used on 25(OH)D₃ was liquid chromatography with mass spectrometry detection. Fatty acids in blood were expressed as g/100 g FAME and 25(OH)D₃ as nmol/L.

Our study has been performed in accordance with the guidelines laid down in the Declaration of Helsinki, and all procedures involving human subjects were approved by the Regional Ethics Committee of southern Norway. Written informed consent was obtained from all participants and their parents.

Descriptive analyses were used to evaluate the characteristics of the participants (mean age, gender, height, weight, BMI). The total intake of energy and selected nutrients from FFQs and 24-HDR were not normally distributed and are therefore presented as median with 25th and 75th percentile.

For the relative validity study, the mean intake from the two 24-HDRs was calculated and used in the correlation analysis. Median daily intakes of the selected nutrients and E% were calculated separately based on the FFQs and the 24-HDRs. Median food intakes reported from FFQs and 24-HDRs are presented in these food groups: dairy products; eggs; meat and poultry; fish; bread and cereals; potatoes, fruit, and vegetables; sugar, sweets, and snacks; butter, margarine, and oils; beverages (water, juice, sodas, tea, and coffee).

Correlations between the food, nutrient, and energy intakes estimated by the two FFQs (reproducibility study) and between FFQ and the mean of two 24-HDR were done with the use of Spearman's correlation coefficient (r). To adjust for the within-person variation in the measurements, corrections of correlation coefficients were performed according to Willett (16). The corrected correlation, R ₁, was calculated as follows: R 1 = R 0 1 s w 2 s b 2 / n , where s w 2 / s b 2 is the ratio of the within- and between-person variances and n is the number of replicates per person for the given variable. The within- and between-person variances were obtained from an ANOVA model. These correlations are presented in Table 4.

To test the FFQ's ability to rank individuals according to their nutrient and energy intake, cross classifications between the FFQ and 24HDRs, and FFQ and fatty acids in blood and FFQ and second FFQ were carried out. The nutrient and energy variables were divided into quartiles, and results are presented as participants classified into the same quartile [correctly classified (CC)] and those who were grossly misclassified (GM) to a non-adjacent quartile.

Self-reported weight and height were used to calculate body mass index (BMI=kg/m²). The significance level was set to 5%, and all statistical analysis were carried out using the computer program SPSS version 18 (SPSS Inc. Chicago, IL, USA).

The characteristics of the 93 students who completed the FFQ and the two 24-HDRs are presented in Table 1. There was an equal distribution of girls and boys in the study. Mean age of participants was 13.8 years while mean BMI was 19.7 kg/m².

There were 58 students who participated in the reliability study, 22 boys and 36 girls. The median Spearman correlation coefficient for all nutrients in the test–retest study was 0.57 (Table 2). The correlations ranged from r=0.280 for trans-fat to r=0.680 for cholesterol. All correlations were significant. For most nutrients, more than 50% of the participants were correctly classified into the same quartile of intake. For trans-fat, only 29% participants were correctly classified. For all but three nutrients (trans-fat, monounsaturated fat, and saturated fat), less than 20% of the participants were GM into a non-adjacent quartile. The median Spearman coefficient for all food groups was 0.52 (Table 3), with a range from r=0.35 for bread and cereals to r=0.67 for beverages. For all but the food group ‘bread and cereals’, less than 20% of the participants were GM, that is, classified into a non-adjacent quartile.

The median Spearman correlation coefficient for all nutrients in the relative validity study was r=0.26, ranging from r=−0.07 for energy percent from polyunsaturated fat to r=0.55 for calcium (Table 4). Significant correlations were found for all nutrients, except intake of saturated fat, monounsaturated fat, cholesterol, energy percent from fat, and polyunsaturated fat. To correct for within-person variation between the two 24-HDRs, corrected correlation coefficients are presented in Table 4. Most correlations were improved, especially correlations for polyunsaturated fat, cholesterol, vitamin D, vitamin E, and energy percent from sugar.

The correct classifications of participants into same quartiles were also low for these nutrients, ranging from 22% for energy percent from monounsaturated fat to 30% for E% from saturated fat. Best classifications were found for energy intake (40%) and vitamin C intake (42%).

The median Spearman correlation coefficient for all food groups was r=0.19 (Table 5), ranging from r=0.01 for sugar, sweets, and cakes to r=0.627 for dairy products. There was a low percentage of the participants being correctly classified into quartiles of food intakes (22–42%) (Table 5).

Significant correlations were found for the n-3 fatty acids, EPA, DPA, and DHA; however, not for the fatty acids ALA and LA, total n-3 intake and vitamin D from FFQ compared to whole blood analyses (Table 6). The correlation coefficients for n-3 fatty acids were low ranging from r=0.05 to 0.25, and absent for vitamin D (r=0.000). The correct classifications into quartiles were low for all fatty acids and vitamin D.

The general test–retest reliability of the questionnaire was good with median correlations of 0.53 for nutrients and 0.52 for food groups, which fell within the range considered good for an FFQ (12, 16). Low correlations were only found for trans-fat (r=0.28). This might be because intake of trans-fat generally is low in the Norwegian public (31) and further that foods, such as cookies and biscuits, containing trans-fat are eaten seldom, making that intake more difficult to measure. In our sample, median intake of sweet biscuits was 2 g/day (Q1: 2 g/day, Q4: 2 g/day), where of course few of these include trans-fat (data not shown).

Our results are comparable to other studies involving adolescents (32, 33). Kobayashi et al. tested the reproducibility of two newly developed questionnaires for adolescents and found correlations between 0.67 and 0.84 for nutrients in the first questionnaire and 0.39 to 0.83 for the other. In general, these correlations are somewhat higher than those found in the present study. However, the present correlations are higher than what was reported by Preston et al. (mean r=0.43) among adolescents. In their general review, Cade et al. found that correlations between 0.5 and 0.7 were most common in reproducibility studies (12). The strength of our study is the time lapse between the two FFQ administrations, which was 4 weeks, and which according to Cade et al. is in line with approximately 34% of all the test–retest studies (12). Cade et al. state that it is not wise to administer a questionnaire at a very short interval as respondents may remember their answers; however, true changes in diet may occur if the interval is too long (12). A limitation with this study is the low sample size (n=58), as it is lower than the ideal sample size of 100–200 (16). The participants in this study were involved in many assessments, and the second FFQ was the last thing they did, and we assume that that is an explanation for the low participation.

Comparisons of nutrient and energy intakes estimated from the FFQ with the reference method, 2×24-HDRs, showed limited agreement. The correlation coefficients between the FFQ and the 24-HDRs varied from −0.07 to 0.55. In most validation studies, correlation coefficients are considered to be poor if r<0.49 and good if r>0.5 (12, 34). Overall, poor to good associations were observed for 10 of the nutrients and E% from trans-fat. In this study, three of the nutrients (riboflavine, vitamin C, and calcium) had a correlation coefficient like Willett recommend (r>0.5–0.7), and in total eight of the nutrients were over the threshold that Cade suggests (r>0.4) (12, 16). The correlation was considered good for dairy products but poor for most other food groups. Although comparison to other studies is difficult due to differences in methodology for reference methods, comparing our results with six relevant studies representing FFQs for adolescents (10, 34–38) show that our results are comparable for several nutrients like fiber and calcium. A similar study also found lower correlations for food intake compared to nutrient intake (37). Few participants were classified into the right quartile based on the two assessment methods. Overall, based on the percentage of subjects classified into the same quartile, the ranking ability of the questionnaire was evaluated to be limited. The lack of agreement between the two methods may be due to limitations of the FFQ, the reference measure (the 24-HDR), or how the study was conducted. One challenge in the validation of an FFQ is the selection of a proper reference method. When validating an FFQ, dietary records are likely to have the least correlated errors according to Willett (16). Multiple 24-HDRs are the primary alternative to dietary records (16). However, it is a suboptimal alternative since both rely on memory and perception of serving sizes. Willett argue that in situations, as when subjects are less than highly motivated, 24-HDRs are good alternatives to dietary records (16). It is important to be aware of the fact that FFQ is designed to measure usual intake while the 24-HDR is designed to measure short-term intake. However, two 24-HDRs are sufficient to capture usual food intake if first, random within-person error is removed by statistical modelling, as done in Table 4 in our study. Several correlations were strengthened after this correction, probably because by using the arithmetic mean of both interviews in the first correlation, the proportions in the upper and lower tails of the intake distribution were overestimated. Then for individual-level intake assessment, the correlations between the two methods were probably somewhat underestimated due to regression dilution bias. Second, two 24-HDRs are sufficient to capture usual food intake if random between-person error is eliminated by including sufficient observations. The numbers included were not optimal; however, compared to others, we believe that the 24-HDRs to some extent serve as a superior reference method for the relative validity assessment.

A limitation with the 24-HDR is that subjects knowing that their diet is under investigation, may change their dietary intake and this creates a distortion of their actual intake (12, 39). The students knew what week they would be called, but not what day of the week. Being aware of the week they would be called may have resulted in more healthy eating habits in that particular week. This is often subconscious and has been shown for adolescents, especially girls (40, 41). One should further notice the particular days the recall took place. A majority of the days were weekdays, and it is well known that diet changes during the weekends, especially Friday and Saturday (42). This is a limitation with the study and could have led to underreporting of certain nutrients like different fats and sugar and foods like sweets and cakes. There is of course a general limitation with underreporting to be aware of, which is a particular problem among girls (43). Another consideration is the number of days for recall; the low number of days (n=2) may not be enough to assess the usual intake for adolescents (37). Several other validation studies have more than two, often 3–4 days of recall (35–37). Willett also suggests that 3–10 days of recall would be appropriate if the population investigated is small (16).

There are several additional limitations to be considered in our validation study. One limitation is the low response rate and the large dropout rate from those who handed in the written consent to those who finished the first FFQ, even though a reminder message was sent after the first e-mail invitation to the FFQ to increase the response rate (44). Low response rate is also seen in other validation studies (45). Another limitation refers to the problem of estimating portion size and how detailed the information had to be. Estimating food with photographs, as done in the 24-HDRs, is a complex task (23). Since only a short booklet was used in this study, there were no pictures of all the different kinds of food items or portion sizes. Therefore, the interviewers could end up in a situation where they had to help the participants to remember and suggest estimates of the food. In the FFQ, household measurements and specific portion sizes were used. A large number of the questions were also grouped (more than one food item in one question), and the students did not have the opportunity to answer as precisely as in the 24-HDRs (i.e. if the pasta was whole grain). This can explain some of the differences observed.

The evaluation of this FFQ's absolute validity showed poor correlations in EPA, DPA, and DHA compared to the same nutrients in blood. The correlation coefficients between FFQ and blood in this study ranged from r=0.000 to 0.291. These are low values especially when correlation is considered to be poor if r < 0.49 (3). None of the nutrients in this study reached a correlation level above this. In addition, the classification of participants into the right quartile calculated by these two assessment methods was low, on average 26.1%, meaning that the results allow for misclassification (23).

There are few studies comparing self-reported dietary intake with biomarkers of fat from whole blood as reviewed by many (46, 47), particularly data in children are scarce. Two studies in children were identified: Burrows et al. (48) and Okuda et al. (49). Burrows et al. (48) found about the same correlation in EPA (r=0.24), lower correlation in LA (r=0.01), ALA (r=0.03), DPA (r=0.03), and DHA (r=0.11) compared to the present study. Okuda et al. (49) found roughly similar results as our study for EPA (r=0.281) and DHA (r=0.233), lower correlation for DPA (r=0.093). All values were unadjusted (49). The correlation coefficients between vitamin D and 25(OH)D₃ were lower than what is observed by others (50, 51). One could speculate if the FFQ did not include enough detailed questions regarding sources of fatty acid and vitamin D. We could have included more questions on oils with regard to fatty acids. As for vitamin D, most food sources were included in the FFQ. One could argue that since some vitamin D stems from sunlight a good correlation would be difficult to show, especially since the samples were taken in October where adolescents have spent less time outdoors. However, according to Pedersen (52) sunlight supplies ignorable amounts of vitamin D at our latitude in October–April. Brantsæter et al. found among pregnant women that the correlation between dietary intake measured by FFQ and plasma concentrations of vitamin D increased during winter compared to the whole year (53). This supports ignorable amounts from sunlight during winter (51). Although it is a reference method for vitamin D measurements, the method of DBS sampling is still a critical method and our analyses were done on non-standardized samples, that is, the blood were not drawn while the subjects had an overnight fast.