Introduction

In recent years, there has been an increasing interest in the use of qualitative research methods in healthcare-related studies, as evidenced by the growing number of published papers in the literature. Qualitative research investigates people’s experiences and the meanings they make of such experiences.¹ In the context of healthcare, qualitative research enables researchers to address questions that may not be easily answered by quantitative methods.² Accordingly, the only way to obtain information about patients’ and their families’ perspectives is to ask, listen, and learn from them, and to incorporate their responses into the delivery of care. Qualitative methods capture patients’ viewpoints and add a dimension to the relevant knowledge base that quantitative research alone cannot acquire.³ Indeed, a qualitative research method is ideal because of its emphasis on an individual’s personal life experiences.⁴

Among the approaches used for qualitative data collection are interviews, observations, and analysis of documents.⁵ Interviews are the most common technique used in healthcare research to gather information.² In particular, focus group discussion (FGD) interviews are preferable since more individuals can be interviewed within a limited amount of time.⁶ Focus group interviews tend to document the “public” instead of the “private” views of individuals.⁷ This technique can be very engaging for participants, especially those from communities in which they often do not have a voice in matters that affect them. Focus groups are ideal for discussions wherein the group moderator has adequate skills, members are knowledgeable, willing, and capable of communicating, and the topic and items are compatible with the interactions among participants.⁸

There are many benefits of using focus groups as a tool in healthcare research. For instance, focus groups can elicit in-depth thoughts and discussions during members’ interactions within the group and can yield richer data, compared to surveys, about the complexities of thoughts and behaviours.⁹ FGDs usually lead to rich stories that most likely would not be told in such detail in a different kind of setting. They can only occur when participants feel comfortable and the group moderator is well prepared.¹⁰ However, like any other method, FGDs can be fraught with shortcomings, such as the quality of the data produced. To minimize this potential issue, the present researchers decided to evaluate the relevance and applicability of the FGD interview guide to ensure its suitability for the exploration of the study objectives. Patton¹ defined an interview guide as a series of topics or broad interview questions that the researcher is free to explore and probe with the interviewee. The advantage of an interview guide is that it helps the interviewer pursue the same basic lines of inquiry with each person interviewed and manage the interviews in a more systematic and comprehensive way.

Reliability and validity are conceptualized as trustworthiness, rigour, and quality in the qualitative paradigm, both of which can increase transparency and decrease the chance of researcher bias in qualitative research.^11-12 The validity of qualitative research means the “appropriateness” of the tools, processes, and data. Specifically, validity measures whether the research question is valid for the desired outcome, the choice of methodology is appropriate for answering the research question, the design is valid for the methodology, the sampling and data analysis are opposite, and finally, the results and conclusions are valid for the sample and context.¹³ One way of evaluating the validity of an instrument is through content validity. Nunnally and Bernstein defined content validity as the extent to which one can generalize from a particular collection of items to all possible items in a broader domain of items to obtain as representative a collection of item material and relevant content as possible.¹⁴

However, establishing validity can present challenges for qualitative researchers.¹⁵ There are several reasons why qualitative researchers may struggle with the notion of validity: the heterogeneity of different methods within the field, high dependence on the applicability of criteria to the research topic, and the importance of the whole research process for establishing its overall quality.¹⁶ Needless to say, experts agree on the need to assure validity, credibility, and reliability in qualitative studies.¹⁷ Therefore, the present study aimed to describe the development and content validation of an FGD interview guide used to explore practices and factors that influence adolescents’ utilization of the NIP when choosing foods.

Materials and Methods

Study Design

Content validation of the FGD interview guide used in this study was performed to evaluate whether items were inclusive and sufficiently reflected adolescent practice, use, and perception of the NIP. This study involved two important processes: development and content validation of the FGD guide.

Development and Content Validation of the FGD Guide

A semi-structured FGD interview guide was developed based on knowledge in the field as well as an extensive review of the literature. The approach was selected because it was flexible and allowed interviewees to provide more information than they would have from other procedures. This type of interview is neither too rigid nor too open; rather, it is a moderate form in which a great amount of data can be generated from the interviewees.¹⁸ The FGD interview guide for this study was developed by the researchers in five steps, as shown in Figure 1.

Figure 1: Steps in the Development of the FGD Interview Guide Click here to View figure

Step 1: Identification of research objectives. This study was carried out to achieve two objectives: (a) to explore the practice of and factors that influence adolescents’ use of the NIP when choosing foods, and (b) to explore adolescents’ need for education.

Step 2: Selection of topics and questions. Questions were carefully developed based on relevant literature and topics that the researchers chose to explore.¹⁹ Particularly, the social cognitive model was used as a guiding framework for formulating the interview questions. The social cognitive theory explains human behaviour according to a three-way, dynamic, reciprocal model in which personal factors, environmental influences, and behaviours continually interact.²⁰ Moreover, the interview questions were carefully selected to accurately address the research questions. Several factors were also taken into account in the preparation of the interview questions, including time constraints, the need to minimize learning effects during the interview, and the need to carefully craft follow-up prompts that would maximize the usefulness of responses while minimizing measurement noise to the greatest extent possible.

Step 3: Development of the guide. The FGD interview guide was divided into four elements to make sure participants were comfortable and relaxed when answering questions. The elements included:

1. A welcome remark that both introduced participants to the study and explained the purpose of the discussion, anonymity, and ground rules.
2. A warm-up session that helped participants to get to know the moderator, assistant, and each member of the group.
3. Interview questions that consisted of an introductory question, a guiding question, and a concluding question. The goal was to select items that were representative of a larger domain, thereby providing general insight into the performance or characteristics of this larger domain of items.
4. A conclusion by the researcher to wrap up the session and mark the end of the discussion.

Thirteen questions were developed by the researchers for this study. Table 1 shows the questions used for the FGD interview.

Step 4: Expert evaluation. A panel of five content experts representing a range of disciplines, including nutritionists, academicians, a qualitative researcher, and a teacher, were invited to evaluate the FGD guide with respect to the relevance, clarity, and comprehensiveness of the interview questions. The experts were selected according to their work experiences and research expertise. Letters of invitation and appointment were sent to the experts by email. The email message contained an attached file titled the FGD Interview Guide Content Validation Form for Expert Panel, consisting 13 items that covered all topics from the FGD interview guide with a description of its purpose and objectives. The experts were directed to evaluate each interview question (including probes) of the interview guide using three response options (i.e., essential, useful, or unnecessary) to indicate its degree of relevance and clarity. Experts were given the opportunity to provide feedback on the questions and suggest additional items.

Step 5: Pilot FGD interview. A similar setting, interview method, and participant criteria as those that will be used in the main study were used by the researchers. A random heterogeneous sample of six healthy adolescents (3 males and 3 females) aged 13 to 14 years from one daily public secondary school were recruited for the pilot study. Parent’s or guardian’s consent was obtained before the interview since all participants were under 18 years old. The location for the interview was decided by the school, subject to the availability of the interviewees and space. The interview was completed during the physical and health education (PHE) class. The interview was timed, and detailed notes were taken by the designated note-taker. Afterwards, a debriefing session was conducted between the moderator and the note-taker to give immediate feedback about the interview. The interview was audio-recorded and transcribed verbatim by the researchers after the session. The main researcher, as the FGD facilitator, and an assistant collected field notes to capture important opinions and observations during the discussion. The transcript was then analyzed using NVivo 11 (QSR International) to identify initial themes.

Statistical Analysis

Expert evaluation feedback was analyzed item by item using a content validity ratio (CVR) according to Lawshe’s formula, CVR=(Ne-N/2)/(N/2), in which the Ne is the number of panellists indicating “essential” and N is the total number of panellists. The value of CVR is determined by Lawshe Table.²¹ According to Lawshe’s formula, if the total of experts is five, the minimum value for CVR is 0.99. From the analysis, any question with negative CVR values and which did not reach CVR of 0.99, especially -1.00 as a score, was removed from the FGD guide. In many situations, it was more efficient to report the overall content validity index (CVI) score than each item’s CVR to determine the content validity of the entire instrument or tool. The CVI is the mean of the CVR values for all the items in the instrument.²² Tilden et al.,²³ and Davis²⁴ suggested CVI values exceeding 0.70 and 0.80, respectively.

Results

Content Validity Ratio

A total of 13 questions on the FGD guide were validated by the expert panel. The CVI value for the FGD guide is 0.938 which is appropriate for a new instrument. However, two questions did not meet the minimum value of 0.99 for CVR: questions one and seven, with a CVR value of 0.6. The researcher, on the other hand, retained both questions with modifications based on the expert panellist feedback and recommendations. Table 2 shows the CVR value for each item in the FGD interview guide.

Table 1: Original Questions for the Focus Group Discussion (FGD)

Expert Feedback and Recommendations 

All the invited experts provided feedback on the FGD interview guide within one month of receiving the invitation email. Their suggestions included reviewing the items, splitting an item into two, retaining an item unchanged, or adding recommended items. Subsequently, seven out of the 13 questions were modified based on the experts’ recommendations. For the first question (Do you know what the NIP is and what do you understand about it?), two experts suggested splitting it into two: (i) Do you know what the NIP is? and (ii) What do you understand about the NIP? Such a modification was suggested to give respondents a chance to think through each question and give more detailed responses to each. In addition, the researchers also separated the fifth question (Please name three information that you searched for when you read the NIP and explain why you chose them) into two questions. For other items, the researchers either retained the original question or changed the wording as recommended by the panel of experts.

Pilot Study

The pilot study was conducted at a public secondary school in Kuala Pilah District, Negeri Sembilan one month before the actual data collection. The pilot FGD was completed within one hour of the interview. From the pilot study analysis, the researchers refined the questions’ wording and structure before initiating the main study. NVivo analysis showed an overall pattern in the participants’ feedback on each topic. It was found that some questions needed substantial probing because participants failed to give further explanations. For example, when the researcher asked the participant about his/her NIP use when purchasing packaged foods or beverages, several probing questions were used, such as “Why do you read or not read the NIP?,” “When do you read the NIP?,” and ‘For what type of food do you read or not read the NIP?” The researchers also identified a couple of words that were used frequently by the participants for each topic, which in turn stimulated additional ideas for probing. Table 3 shows the final version of the interview guide based on expert evaluations and the pilot study. Overall, the final FGD guide consists of 13 items and probes that are suitable for any age group.

Table 2: CVR Value for Each Item in the FGD Interview Guide

Table 3: Final Interview Guide

Discussion

Qualitative research helps us understand human experience and meaning within a given context using text rather than numbers, interpret experience and meaning to generate understanding, and recognize the role of the researcher in the construction of knowledge.²⁵ One question that often arises about qualitative research is related to the reliability of the interpretation and representation of participants’ narratives. Unlike in quantitative designs, there are no statistical tests that can be used to check the reliability and validity of qualitative research.²⁶

In the present study, the FGD interview guide was developed to explore practices and factors that influence adolescents’ use of the NIP in their food choices. Through a systematic process of topic and question development, expert reviews, and a pilot test, we determined the content validity of our FGD interview guide. The formulation of the topic and questions were based on the study objectives, which were to explore adolescents’ use of the NIP in their food choices and to examine their perceptions about the need for NIP education. The social cognitive theory was used as a basis to explore personal, environmental, and behavioural factors underlying adolescents’ understanding and perception of the NIP. Grounded on this theory, the research objectives and FGD topics and questions were formulated.

The developed FGD interview guide was divided into four essential elements: welcome remark, warm-up session, interview questions, and conclusion. Interview guides are important because they set the agenda, provide prompts, and can ensure comparisons across groups. Topics that are non-starters within one group may be barn-burners in another. To this, we need to add a certain degree of probing (or follow-up questions) and moderator involvement. All these factors will determine the number of topics completed during a session.²⁷ Personal experience indicates that when exploring very sensitive and personal issues, the use of pre-existing groups might be advantageous, as there is already a certain level of trust among the group members that will encourage a more honest expression of views.²⁸ We began the development of the interview questions with a review of previous research in the field and then expanded on this research with our questions, which involve topics not yet addressed in the extant literature regarding adolescents. The main questions were then extended to follow-up, warm-up, and closing questions that matched every topic in the guide. Brod et al.,²⁹ stated that, as with new measures, a discussion guide for qualitative interviews with existing measures should begin with a broad, open-ended question, and then proceed to explore the theme of each of the items in the current measure, rather than exploring domains and themes previously identified as potentially important.

Furthermore, the experts’ assessment of the questions enhanced the significance, clarity, and extensiveness of the FGD interview guide. The suggestions and modifications recommended by the experts included rephrasing some questions, separating other questions into two, and adding probes and other relevant items. Consequently, following the recommendations of the panel of experts, seven out of 13 questions were revised. All of the items were accepted by the panel, as they were deemed suitable and sufficient for achieving the study objectives.

Moreover, the pilot study improved the content validity of the FGD interview guide by analyzing adolescents’ use and understanding of the NIP. According to van Teijlingen and Hundley,³⁰ a pilot study is performed to test the adequacy of research tools, assess whether the research protocol is realistic and workable, and train researchers in as many elements of the research process as possible. In this study, the researchers enhanced the main questions by adding probing questions to elicit more thorough and profound responses from the participants. The ideas for probing originated from the interview analysis, which showed an overall pattern in the frequency of words used for each topic in participants’ feedback. The researchers also learned that the interview location and schedule are very important for guaranteeing the quality of data collection. From the pilot study experience, the researchers made sure to properly discuss the suitability and convenience of the location and schedule for the actual research with the school management.

The final version of the interview guide was based on the experts’ evaluations and the pilot study. It consists of 13 items, including probes, that are appropriate for participants of any age. The questions finalized in this guide are sufficient for obtaining insights into adolescents’ understanding and practices and thereby for achieving the study objectives. However, it must be remembered that since in a qualitative interview, the human element in the interaction between the interviewer and interviewee is central to the interview’s success, quantitative notions of reliability cannot meaningfully be applied.³¹ Suffice it to say that the researchers kept this in mind, fully covering each topic and preparing appropriate probes in advance to avoid unwanted complexities during the interview.

The FGD guide was determined to have content validity for future users and participants: research experts, experiential and clinical. The current interview guide is likely to become a useful tool for researchers and policymakers who wish to explore NIP practices and determinants in the general population or examine the effectiveness of public education programs. Further investigation is needed to improve the assessment of procedural knowledge and to test the interview guide in other populations, especially among the main shoppers in a family.

Acknowledgements

The authors would like to thank the State Department of Education, Negeri Sembilan, and the teachers and students at Sekolah Menengah Kebangsaan Za’ba, Kuala Pilah, Negeri Sembilan for their assistance and participation in the study. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict Of Interest

No conflicts of interest.

References

1. Patton M.Q., “Qualitative Evaluation Methods”, California: Sage Publications,
2. Al-Busaidi Z.Q., “Qualitative Research and its Uses in Health Care”, Sultan Qaboos University Medical Journal, 2008; vol. 8, no. 1: 11–19.
3. Williams B., “Understanding qualitative research”, American Nurse Today, 2015; vol. 10, no. 7: 40-42.
4. Miles M., and Huberman M., “Qualitative Data Analysis: An expanded Sourcebook”, California: Sage Publications,
5. Bashir M., Afzal M.T., Azeem M., “Reliability and validity of qualitative and operational research paradigm”, Pakistan Journal of Statistics and Operation Research. 2008; vol. IV, no. 1: 35-45.
6. Ritchie J., Lewis J., Nicholls C.M., Ormston R., “Qualitative Research Practice: A guide for social science students and researchers”, Los Angeles: Sage Publications,
7. Grbich C., “Qualitative Research in Health”, London: Sage Publications,
8. Carey M., “Focus Groups–What Is the Same, What Is New, What Is Next?”Qualitative Health Research, 2016; vol. 26, no. 6: 731-733.
9. Wilkinson S., “Focus groups in health research: Exploring the meanings of health and illness”, Journal of Health Psychology, 1998; vol. 3, no. 3: 329-348.
10. Then K.L., Rankin J.A., Ali E., “Focus group research: what is it and how can it be used?”, CanadianJournal of Cardiovascular Nursing, Winter, 2014; vol. 24, no. 1: 16-22.
11. Golafshani N., “Understanding reliability and validity in qualitative research”, The Qualitative Report, 2003; vol. 8, no. 4: 597-607.
12. Singh A. S., “Conducting case study research in non-profit organisations”, Qualitative Market Research: An International Journal, 2014; vol. 17: 77–84.
13. Leung L., “Validity, reliability, and generalizability in qualitative research”, Journal of family medicine and Primary Care, 2015; vol. 4, no. 3: 324–327.
14. Nunnally J., and Bernstein I., “Psychometric theory”, New York: McGraw-Hill,
15. Cho J., and Trent A., “Validity in Qualitative Research Revisited”, Qualitative Research, 2006; vol. 6, no. 3: 319-340.
16. Kuzmanić M., “Validity in qualitative research: Interview and the appearance of truth through dialogue”, Horizons of Psychology, 2009; vol. 18, no. 2: 39-50.
17. Konradsen H., Kirkevold M., Olson K., “Recognizability: A strategy for assessing external validity and for facilitating knowledge transfer in qualitative research”, Advances in Nursing Science, 2013; 36, no. 2: e66–e76. DOI:10.1097/ANS.0b013e318290209d
    CrossRef
18. Zohrabi M., “Mixed method research: instruments, validity, reliability and reporting”, Theory and Practice in Language Studies, 2013; vol. 3, no. 2: 254-262.
    CrossRef
19. Krueger R.A., “Developing questions for focus groups”, California: Sage Publications, 1998.
    CrossRef
20. Bandura A., “Social Foundations of Thought & Action: A Social Cognitive Theory, Edition 1, Michigan: Prentice Hall, 1986.
    CrossRef
21. Lawshe C., “A quantitative approach to content validity.”Personnel Psychology, 1975; vol. 28, no. 4: 563-575.
    CrossRef
22. Gilbert G.E., and Prion S., “Making sense of methods and measurement: Lawshe’s Content Validity Index. Clinical Simulation in Nursing, 2016; vol. 12: 530-531.
    CrossRef
23. Tilden V., Nelson C., May B., “Use of qualitative methods to enhance content validity”, Nursing Research, 1990; vol. 39, no. 3: 172-175.
    CrossRef
24. Davis L., “Instrument review: getting the most from a panel of experts.”Applied Nursing Research, 1992; vol. 5, no. 4: 194-197.
    CrossRef
25. Petty N.J., Thomson O.P., Stew G., “Ready for a paradigm shift? Part 1: Introducing the philosophy of qualitative research”, Manual Therapy, 2012; vol. 17, no. 4: 267 – 274.
    CrossRef
26. Sutton J., and Austin Z., “Qualitative research: data collection, analysis, and management”, The Canadian Journal of Hospital Pharmacy, 2015; vol. 68, no. 3: 226–231.
    CrossRef
27. Twohig P.L., and Putnam W., “Group interview in primary care research: advancing the state of the art or ritualized research?”Family Practice, 2002; vol. 19, no. 3: 278-284.
    CrossRef
28. Rabiee F., “Focus-group interview and data analysis”, Proceedings of the Nutrition Society, 2004; vol. 63: 655–660.
    CrossRef
29. Brod M., Tesler L.E., Christensen T.L., “Qualitative research and content validity: developing best practices based on science and experience”, Quality of Life Research, 2009; vol. 18, no. 9: 1263–1278.
    CrossRef
30. Van Teijlingen E.R., and Hundley V., “The importance of pilot studies”, Social Research Update, 2001; vol. 35, available at http://hdl.handle.net/2164/157 (accessed 20 November 2017).
31. Prescott F. J., “Validating a long qualitative interview schedule”, Working Papers in Language Pedagogy, 2011; vol. 5: 16-38 available at http://langped.elte.hu/WoPaLParticles/W5Prescott.pdf (accessed 30 June 2017).