Validity and reliability of a simplified food frequency questionnaire: a cross sectional study among physical health examination adults in southwest region of China

Abstract

Background

In China, many people are regarded suitable for participating in regular physical examination for diagnosis and prevention of diseases. Some simplified food frequency questionnaires have been designed and used; however, the accuracy of the questionnaire is absent. This study aimed to examine the reliability and validity of simplified food frequency questionnaire (SFFQ) used among adults undergoing physical examination in southwest region of China.

Methods

This was a cross-sectional study conducted among physical health examination adults in the Southwest region of China. A total of 239 participants aged 20–65 were included from February 2019 to June 2019. The performance of SFFQ was evaluated by means of a three-day 24-h dietary record (3R24). The relative validity and agreement was assessed by Pearson’s correlation and intra-class correlation coefficients (ICC), respectively.

Results

The median energy-adjusted ICC of food groups between SFFQ2 and SFFQ1 was 0.59 (range: 0.49–0.73) and the ICC of nutrients was 0.47(range: 0.39–0.76). The Pearson correlation showed a valid comparisons between SFFQ1 and 3R24, ranging from -0.086 to 0.93 for food and 0.21 to 0.71 for nutrition, respectively. The energy-adjustment slightly increased the correlation coefficients.

Conclusions

The reliability and validity of SFFQ was acceptable, and it could be an appropriate dietary assessment tool for the future epidemiological studies conducted among physical health examination adults of southwest China.

Trial registration

CHiCTR, ChiCTR1900020934, Registered 22 January 2019, https://www.chictr.org.cn/edit.aspx?pid=35414&htm=4.

Introduction

Although quantification and assessment of diet are difficult tasks, they are necessary to check the relationship between nutrients and non-transmissible chronic diseases (NTCDs) [1]. Such studies belonged to the field of nutritional epidemiology.

Several types of instruments are used to assess both present as well as previous diet: the 24 h diet record (R24), food-intake record (FR), and the food frequency questionnaire (FFQ). But all these instruments had their own advantages or limitations. FFQ is a cost-effective method used for dietary assessment in large population groups [2]. Currently, it is widely used in epidemiological studies with the intention to assess the relation between diet and disease. It could provide information on the frequency and proportion of a defined food items list. FFQ is a feasible questionnaire, and involves low cost and low recall bias [3]. It is recommended as one of the first-line methods for dietary measurements, especially when conducting a population survey or community trials. For assessing the factors that have effects on the incidence or prevalence of chronic diseases, the dietary pattern or various food intakes included in FFQ should be valid [4]. Recently, many reports have been put forwarded on the use of FFQ in evaluating single nutrients [5]. Some scholars have reported the use of food frequency questionnaires to assess disease risk [6].

But food intake largely varies based on ethnicity, social, and cultural background of the study population. Validation study should be conducted to improve the application of FFQ within specific samples [7] and accuracy, and reduce bias levels related to disease occurrence [8].

In China, many people are considered suitable for participating in regular physical examination for diagnosis and prevention. It is vital to evaluate the relationship of diet with their physical examination data, which is typically large in size and require continuous collection. This data is an important source for forecasting the risk factors associated with disease. However, in China, the traditional food frequency questionnaires include more than one hundred kinds of foods, which would nearly take 30 min to finish. It is generally assumed that the questionnaire length has a significant effect on the survey response rate as respondents get tired, bored and/or distracted by external factors [5]. Some simplified food frequency questionnaires (SFFQs) have been designed and used in special populations [9, 10]. However, the validity of SFFQ in Chinese population undergoing physical examination is not discussed. It is also unclear as to whether SFFQ could evaluate the intake of samples that are similar to large-scale crowd survey.

Hence, in this study, by combining the traditional FFQ questionnaires of China and the dietary habits of residents of southwest China, a new SFFQ was designed. To collect the dietary intake in the preceding 3 months, its reliability and validity was assessed among adults undergoing physical examination in the Southwest region of China.

Materials and Methods

Participants

This was a cross-sectional study conducted among adults undergoing physical health examination in southwest region of China. Participants included those undergoing physical examination in the Medical Examination Center of Sichuan Provincial People`s Hospital from February to June in 2019. Subjects aged 20–65, and those who could fill in the questionnaire independently and accepted a phone interview were included. The dietary information was collected using SFFQ and 3R24s. All information was collected by 3 trained interviewers. Participants were included in this study for a period of 2 weeks (Fig. 1). The first SFFQ (SFFQ1) was collected at the medical examination site, and the second SFFQ (SFFQ2) was collected after 2 weeks through phone. The contents of both SFFQ2 and SFFQ1 remained the same, but the order of questions generated by the computer was random. The 3R24s were designed on Saturday, Monday, and Tuesday, in which the two weekdays and one weekend day accounted for intraindividual variation between day types were covered and completed by interviewers through phone. A total of 239 completed 3R24s and SFFQ1 + 2 were collected. All participants were informed about the study and obtained formal consents before being interviewed. This study was approved by the Ethics Committee of the Sichuan Provincial People's Hospital (Approval Number:2017–153), China.

Fig. 1

Flow chart of the study for the SFFQ and 3R24

Dietary Assessment

Simplified Food Frequency Questionnaire (SFFQ)

Based on an FFQ used in Chinese population [10], involving the eating habits of southwest China, such as preserved meat, chilli and soy products, a new SFFQ was designed. The SFFQ consisted of 50 items of food. The following three sections were included in the questionnaire: the food list, the frequency response, and the average intake. The food items were divided into 14 categories, Table 1. When filling the SFFQ, all participants were required to indicate how many times they consumed each food category per month/week/day in the past 3 months. The frequency of consumption was measured by selecting one of the following options: (1) never or less than once per month, (2) 1–3 times per month, (3) 1–3 times per week, (4) 4–6 times per week, and (5) how many times per day. Because there was no unified portion size in China, pictures of utensils and food portions were taken into account to assist with the description along with trained investigators. Food intake (g/day) was calculated by multiplying the frequency of consumption by portion size with the mean frequencies of consumption per day for each group.

Table 1 Food groups and their components in SFFQ

3-day 24-h Dietary Record (3R24)

The study population completed 3R24s in a week, which included two working days and one rest day. These questionnaires were completed by interviewers through phone over a course of 3-day period. The resulting dietary data included 4,024 “food mentions”, i.e., combination of entire food items of all participants. These data were coded into 14 broad food groups and 12 types of nutrients according to the SFFQ of Chinese Residents Dietary Guidelines’ selection. The mean weights were calculated respectively.

Assessment

The daily nutrient consumption of participants was calculated by matching the food code from the China food composition database. If a food was not listed in the database, then the similar food item was used instead of calculating the nutritional values.

Statistical Analyses

All data were entered twice into the Epidata. Pearson correlation coefficients were used to evaluate the validity of food frequency acquired by SFFQ1 with an average of 3R24 data (food weight). According to a recent systematic review, the validity of FFQs should be re-evaluated because energy, carbohydrate, calcium, and vitamin C intakes were overestimated, especially in females [11. Some scholars' have reported that FFQs tend to have a larger measurement error in underestimating the energy and nutrient intake when compared to 24-h dietary recalls [12]. And the simplified dietary survey questionnaires tend to underestimate the energy and nutrient intake as compared to the longer questionnaires [13]. In our study, after adjusting the energy, 3R24 was taken as reference for SFFQ. The adjusted correlation coefficients were then calculated. The agreement between SFFQ1 and SFFQ2 was assessed by intra-class correlation coefficients (ICC) of energy-adjusted food and nutrient intake. All analyses were performed using Statistical Package for the Social Science (SPSS) software, version 17.0 (IBM, Armonk, New York, NY, USA). A P value of <0.05 was considered to be statistically significant.

Results

Description of the study population

Approximately 5–8 min were taken to complete one SFFQ under the guidance of investigators. Among the initial 268 participants, 239(89.18%) individuals with a mean age of 44.29(9.12) years have finished both 3R24 and SFFQ1/2. Half of the participants (51.88%) had 3000 + RMB monthly income, but 37.24% of them refused to provide this information. The habit of having breakfast was good, and this accounted for 87% of all participants, which was beyond our expectation. But most of them ate food outside frequently. The characteristics of study population are described in Table 2.

Table 2 Characteristics of study subjects

Reliability of SFFQ1 vs. SFFQ2

Table 3 presented the descriptive statistics by means (standard deviation, SD) of intake of 14 types of food groups and 12 types of nutrients between SFFQ1 and SFFQ2. The mean total energy intake and food intake reported in SFFQ2 were higher than that reported in SFFQ1, except the item of haslet and processed meat products. Compared with SFFQ2, the nutrient intake of vitamin B2 and calcium was slightly higher in SFFQ1. The intake of other nutrients such as carbohydrates, protein, fat, dietary fiber, vitamin C, and iron showed no significant differences between SFFQ1 and SFFQ2. The median energy-adjusted ICC of food groups between SFFQ2 and SFFQ1 was 0.59 (range: 0.49–0.73). The energy-adjusted ICC of nutrients was 0.47(range: 0.39–0.76).

Table 3 Reliability of food groups and nutrients intakes between the SFFQ1 and SFFQ2(n = 239)

Validation of the SFFQ1vs. 3R24

Table 4 reported the descriptive statistics of mean (standard deviation, SD) intake between SFFQ1 and 3R24. Overall, the crude Pearson coefficients between SFFQ1 and 3R24 among different food groups and nutrients ranged from -0.086 to 0.93 and 0.21 to 0.71, respectively. While the energy adjusted Pearson coefficients showed higher correlation than crude coefficients, which ranged from -0.31 to 0.96 and 0.26 to 0.73, respectively. Compared with 3R24, the intake of tubers, poultry, aquatic products, eggs, dairy, phycomycetes and soy products were slightly higher in SFFQ1. The results of validity assessment of SFFQ1 vs. 3R24 are presented in Table 4. Phycomycetes and aquatic products showed weak correlation with the coefficient of less than 0.4. Other four food groups (such as poultry, haslet and processed meat products, tubers, soy products and nuts) showed moderate correlation with the coefficients, ranging from 0.43 to 0.58. Strong correlation with the coefficients of higher than 0.6 was observed among the following eight food groups: cereal, bread and noodles, red meat, dairy, eggs, vegetables, and fruits. The Spearman`s correlation coefficients of nutrients between SFFQ1 and3R24 ranged from 0.19 to 0.71. After adjusting the total energy intake, all correlation coefficients were increased, which ranged from 0.21 to 0.73. All correlations showed statistically significant differences between SFFQ1 and 3R24 (P ≤ 0.05).

Table 4 Validation of food groups and nutrients intakes between the SFFQ1 and 3R24 (n = 239)

Discussion

This study examined the reliability and validity of an SFFQ that assessed the dietary intake among physical examination adults in southwest region of China. The results of usual food intake and correlation coefficients suggested that SFFQ was reproducible and well performed when compared to 3R24.

The validation of food assessment tools is essential to understand the relationships between food and nutrition-related diseases. To assess the validity, although it is always imperfect, a 24 h dietary record is used for comparing widely. In this study, the validity of SFFQ was assessed by comparing the estimates of dietary intake from a 3R24 among adults who was undergoing physical examination in southwest of China. Overall, the SFFQ demonstrated good reproducibility in estimating the intake of food groups. The crude Pearson coefficients for the correlations between daily intake of various food groups derived from the two measures ranged from -0.086 to 0.93 and the mean correlation was 0.44. The adjusted Pearson coefficients for the correlations showed improvement from 0.31 to 0.96, and the mean correlation was 0.63. The correlation coefficients in our study and other studies were found to be very similar for many food groups, showing correlations of 0.19- 0.84 [14,15,16].

The SFFQ and 3R24 showed some differences in their error sources, and the two methods of investigation are sufficiently independent [17]. Both questionnaires are prone to memory bias and have differences in the perception of portion sizes. The 3R24 method is based on open-ended questions, while SFFQ usually involves close-ended questions. The amount of nutritional intake can often be biased by the individuals' inability in accurately estimating the daily intake. Few other individuals showed unwillingness in acknowledging the intake of foods that might be harmful to health, such as internal organs, fatty meat, processed meat products, etc. Therefore, this SFFQ did not provide accurate and precise estimates of some nutrients (such as vitamin B, dietary fiber, protein and fat). However, for foods with low frequency of usual intake, the diet records (DRs) likely underestimates the intake of food. For example, the intake of low-frequency foods obtained by 3R24, such as tubers, poultry, aquatic products, phycomycetes, etc. in this study (in Table 4), was obviously less than those obtained by SFFQ. Although the correlation after adjusting the total energy intake was still higher than 0.3, the analysis and collection of such data seemed to be more reliable by SFFQ method. In this study, the amount of food, energy, and nutrients in repetitive SFFQ2 was increased (Table 3). This might be related to greater recall and higher attention of the subjects when the questionnaire was completed for the first time. Repeated completion of SFFQ might improve the authenticity of SFFQ. A more accurate estimation of these nutrients requires comparison of biomarkers as a reference for further study.

Fruit is the most frequently studied group in the literature and most FFQs. The correlations with regard to fruits generally range between 0.5 and 0.7, [18,19,, 19] and this was similar to those obtained in our study (fruits: r = 0.65). Vegetables, eggs, bread and noodles, dairy and red meat showed stronger correlation (r = 0.96, 0.86, 0.78, 0.71, 0.72), which was consistent with other studies reported [13, 20]. The intake of cereal and soy products was lower than the average levels reported in China, while the intake of fruits, poultry and red meat remained higher. Other foods were similar to those obtained in the earlier studies of China health and nutrition survey (CHNS) [21]. This might be due to certain differences in the dietary and living habits of the population in southwest China The low frequency of intake of certain foods in this area might affect the results of the study. 3R24 was used to analyze, but it does not fully reflect the overall situation ofindividual`s food intake as mentioned in the previous studies [9]. Of course, increasing the days of dietary record can effectively reduce the deviation. Some scholars have collected 7 days or more longer dietary record for research [22]. But this undoubtedly increased the difficulty of the scale, which was not conducive to large-scale data collection in individuals undergoing physical examination.

There are no “gold standard” dietary investigations till date in China. Several types of instruments are used to assess both present and previous diets: the 24 h dietary recall, food-intake record, and food frequency questionnaire [19,20,21,22]. All these instruments had their own advantages and limitations. Which food intake methodology depends on the questions to be probed, the settings and participants, and the outcomes required should be investigated. A best method would be simple and quick, comprehensive and with high resolution, accurate, precise, and amenable to produce efficient and reliable data. Moreover, the main objective of the current study was to develop an easy-to-use FFQ for future epidemiological studies in China. Earlier FFQs that are used in Chinese epidemiological studies included more than 160 food items, such as CHNS [21,22,, 22]. It would take about 30 min to finish a survey, but a lengthy questionnaire is less likely to be completed and returned [23]. The enduring participation rates of respondent effects the validation of the questionnaires [24], as longer questionnaires may cause respondent fatigue and poorer quality of gathered information. So, this might be not suitable for promotion to adults undergoing physical examination. With reference to Chinese Residents Dietary Guidelines’ selections, the FFQ was simplified, which would take 5–8 min to complete a survey, improving the operability and practicability significantly.

Generally the portion sizes are poorly estimated, and the inclusion of portion sizes in FFQs still remained controversial. For complex country such as China in terms of food, it is difficult to collect dietary information accurately using FFQ. Estimating the portion size of foods is difficult for most of the participants [25]. Some investigators have suggested the use of commonly consumed portion sizes for calculating the nutrient intake in FFQ [26]. For validating, it is important to recognize the portion size of food intake accurately. The results of this study inferred that foods with certain size or servings exhibited higher validity. For example, dairy, fruits, vegetables, and eggs, always in a certain portion size showed a higher correlation (Table 4), which was consistent with that of the previous study results [27].

It is important to mention that there are higher number of male participants and their losses (Table 2). In this study, fewer men refused to fill in the questionnaire and most of them answered the questionnaires by telephone. However, it seemed more difficult for women to complete all the questionnaires. The reason for such losses was due to the absence of 3R24, SFFQ incorrect record filling, and/or lack of informed consent. It is possible that this percentage might have influenced the results due to the final sample size.

The major strengths of this study include high participation rate, streamlined questionnaire, data collection by trained interviewers, and use of standard food map for estimating the portion size. An additional strength of this study is the normative design at the time of investigation of 3R24. The three consecutive days 24 h DRs were designed on Saturday, Monday, and Tuesday, which covered two weekdays and one weekend day, accounting for intraindividual variation between day types. Although DRs are recognized as golden standard, recording errors as well as changes in dietary habits are inevitable, especially during the weekend and holiday. The 3R24 collection method used in this study can assist in more objectively collecting the dietary data of participants on work days as well as rest days. This could help in more comprehensively comparing with the data collected by SFFQ. The SFFQ in this study has good reliability and validity in the population. It has a good application prospect in the future for analyzing the relationship between various physical examination indicators and dietary factors. It is imperative to collect health indicators and data of diet questionnaire through physical examination, conduct horizontal analysis and longitudinally monitor the health status of the population. This provides evidence-based practice for adjusting the diet structure and would promote health status of the population in the region.

The purpose of the present SFFQ is to evaluate the dietary patterns, i.e., food groups and nutrients, among adults undergoing physical health examination. The convergent validity and test–retest reliability was evaluated, but the internal consistency and discriminant validity still remained unclear. However, there are several limitations in the design of this SFFQ and validity appraisal. Firstly, it is of short length and consisted of only 14 food groups rather than single food items or dishes. Participants should classify their foods into simplified food groups, which subsequently increase the risk of misclassification. Also analysis of intake of certain special nutrients(such as anthocyanin, lycopene) might cause deviations. In addition, the reference period is only three months, which is more than one year and would be more representative of a person’s long-term dietary intake. There are seasonal variations in dietary and food intake, but the greatest seasonal variation occurs between summer and winter. Therefore, there might be a seasonal difference in the results of SFFQ and 3R24. Secondly, the repeated 24-h recalls were collected within two weeks of SFFQ1 completion. It has become difficult to predict the degree to which the divergences between the data can be attributed to questionnaire bias vs. differences in what individuals have actually ate during the two different time periods. While the unadjusted correlations between the items of the two questionnaires showed statistically significant differences, and some of them remained low. If the data was collected at the same time, then the unadjusted correlations might be either higher or lower. Along with all dietary assessment methods, some potential disadvantages could also be observed with regard to this SFFQ, such as recall bias, overestimation of dietary intake (particularly for rarely-consumed or healthy-perceived foods e.g., fruit and vegetables), bias of current intake and bias of pre-established food listings. Thirdly, the sample size might not reflect the mean energy intake of the population in the present study. Further study is needed for some nutrients (such as saturated and unsaturated fats, animal proteins and plant proteins, dietary fiber, vitamin B2 and vitamin B1). Finally, the problem of extrapolation within and between different food sub-groups in physical health examination adults is worth noting. Regional disparities should be taken into consideration in different areas of China. One question is that how these findings among Chinese in southwest China can be generalizable in other areas. One way to address this question is to conduct more evaluation studies to compare and validate the Chinese food cultural instruments in other areas.

Conclusion

The reliability and validity of SFFQ questionnaire applied to the individuals undergoing physical health examination among Chinese in southwest China in this study was considered acceptable. It can be easily finished in less time when compared to FFQ, which was previously used for investigation. However, before its application in large epidemiological studies in other regions, it is necessary to re-verify the reliability and validation of SFFQ.