Setting Priorities in Childhood Cancer in Low Income Countries Using Nominal Group Technique: Experience from an International Childhood Cancer Forum Exercise in Bangladesh

Background and Objectives: Cancer is one of the major causes of mortality and morbidity worldwide. The incidence of paediatric cancer in particular, in Bangladesh is alarming and most of these patients die without correct diagnosis and adequate medical treatment (MOHFW, 2008). There is a clear disparity in access to care between rural and urban areas (WHO, 2015; Rahman, 2001). There are no established formal childhood cancer registry systems to help inform planning decisions across the country. Most importantly, there are no explicit priorities or methods for identifying such priorities in Low and Middle Income Countries (LMICs). We used a Nominal Group Technique (NGT) method during the International Childhood Cancer Forum (ICCF) for setting priorities. The following two key objectives were addressed: Trialling the NGT in Bangladesh as a priority setting tool; and identify childhood cancer priorities in Bangladesh. Methods: The Nominal Group Techniques (NGT) method was used to elicit information from the participants of ICCF to identify priorities for research and interventions for childhood cancer care in Bangladesh. Participants were divided into four groups. Each group discussed one question each. Two questions focused on cancer research, and the other two focused on interventions. Results: In regards to outcomes, NGT successfully identified the scale of childhood cancer care and identified priorities/action areas to address in Bangladesh. Six priorities were identified and a successful collaboration for implementation has been established with several international organisations. Conclusion: Nominal group technique was found to be an effective tool to identify research and intervention priorities to address childhood cancer in a developing country. For resource limited countries in similar situations, they could benefit from adopting this approach in healthcare settings.


Introduction
Non-communicable diseases (NCDs) are a major cause of morbidity and mortality in Low and Middle Income Countries (LMIC) (Abegunde et al., 2007;WHO, 2018). Cancer in particular is one of the major causes of mortality and morbidity in among the non-communicable diseases in Bangladesh (MOHFW, 2008). If diagnosed at an early stage, with treatment being available, most childhood cancers are highly curable (Ribeiro et al., 2008). Diagnostic and treatment protocols in developed countries have effectively improved childhood cancer survival rates to 75-80%. In Bangladesh, cancer is the sixth leading cause of death (BBS, 2008). This is partly because of the lack of healthcare services, lack of access to services due to does not have the capacity to cope with the burden of childhood cancer. Furthermore, Bangladesh does not have an official childhood cancer registry to help inform the planning process across the country. Therefore, there is an urgent need to identify research priorities, and collation of effective and cost-effective interventions in Bangladesh. Furthermore, there is need for a national, integrated, protocol-based childhood cancer program. We proposed the use of Nominal Group Technique (NGT) to elicit preferences from the participants of an International Childhood Cancer Forum (ICCF) in Bangladesh (2010), to test if it would be suitable for Bangladesh as a priority setting tool; and identify a priority research agenda and intervention for childhood cancer in Bangladesh.

Description of NGT
NGT was developed in the late 1960s in the United States of America (USA) by Van de Ven and has been used as an evaluative tool in medicine, healthcare, nursing, engineering, information systems and other disciplines (Gallaher et al., 1993;Van de Van et al., 1974). Dunnette et al., (1963) describe NGT as a decision-making method for use among groups of many sizes that wish to make decisions quickly with a voting system and take everyone's opinions into account (as opposed to traditional voting, where only the opinion of the majority is considered. Its objective is to generate ideas, which are then discussed and ranked by the group. The group is 'nominal' to the extent that it is highly controlled, and the discussion is allowed only in the later stages of the group process (Gallaher et al.,1993). NGT is especially useful for problem identification, problem solving, and establishing priorities (Ng, 2000). This technique eliminates the social and psychological dynamics of group behaviour, which can inhibit individual creativity and participation in group decisions (DSE, 2007). Participants involved in the NGT participate in a highly structured face-to-face meeting, usually lasting up to two hours (Margaret et al., 2004). Everyone is given a structured opportunity to present ideas independently, privately and subsequently, sharing and voting on them (Corner et al., 2007).
The questions presented to the participants are normally simple and easy to understand to avoid cognitive burden. Wording and grammatical structures which, often affect the levels of abstraction, breath, and the depth of the elicited responses are taken into consideration (Elliott et al., 2002).

How NGT Sessions are Conducted
At the beginning of a session, the group selects a facilitator to ensure that the meeting would be conducted in a timely and orderly manner. The facilitator starts with a brief overview of the exercise and its purpose. Sometimes, all meetings are audio and video recorded with the consent of the participants.
The facilitator asks the group members to record a specific number of high-priority ideas that answer the assigned question. The ideas are then presented as a full list of identified priorities for all the members to see. The facilitator then reads each idea aloud, followed by a discussion, strictly following a rule of non-argument-consultation. Group members then offered their opinion about the new list of ideas and then asked to eliminate extraneous ideas using criteria such as: legal restraints; personal issues; liability issues; actions beyond the scope of the group's authority to implement; and insignificance of the idea.
The facilitator collected all scorecards and shuffled them. The facilitator recorded all votes on the easel and summed the individual rankings to obtain a consolidated score for each suggestion. Finally, the facilitator identified the top five priority suggestions based on the score rankings. This may be repeated several times to achieve consensus.

Rationale for NGT
The rationale for NGT is based on the need for priority setting in healthcare, which is gaining momentum because of increased demand for healthcare services, yet resources are limited (Redman et al., 1997). Two broad approaches are evident in the literature of health research: technical; and interpretive assessments (qualitative). Technical assessments are dominated by quantifiable epidemiologic or other needs and costs data; and interpretive assessments are dominated by the consensus views of informed participants (Loman et al., 2003). Technical approaches tend to express priorities using units for which data are readily available (Loman et al., 2003). In most cases, data are not available with sufficient detail regarding priority setting, just like in Indigenous Australian health (Otim et al., 2014). For example, in Bangladesh, no formal, reliable data exist regarding cancer, especially childhood cancer, due to the absence of a registry system, raising the need for opinion-based processes must be used (Redman et al., 1997).

Application of NGT in health research
Evidence shows that NGT has been used successfully to set priorities in different areas and different countries. Internationally, the NGT has been successfully applied to various areas of health research as an evaluative tool in healthcare (Campbell et al., 2001;Telford et al., 2004). These include medicine and nursing, and non-healthcare sectors such as, engineering, management and behavioural research (Macphail, 2001). Using a simple ranking method to prioritise health conditions, Ustum et al., (1999) found a high level of stability in the ranking of 17 health conditions for 241 key informants from 14 countries.
NGT has been used to identify community health priorities and to rank the health conditions of two rural communities in the Moshi districts of northern Bangladesh for the first time. Thus, the primary aim of this study was to trial the NGT and to identify priorities in childhood cancer in Bangladesh.

Methods
Using the NGT, the ICCF participants were divided into four groups and assigned four areas of research. This was done over a two-day period organised in Dhaka, Bangladesh in March 2010. To set the context, the forum assessed the extent of the childhood cancer problem in the country.

Participants
Researchers, social scientists, service providers, policy makers, national and international cancer experts were participants in the ICCF were allocated into the four groups. Given that the Forum was attended by 120 people from different professional backgrounds, including the Bangladesh Ministry of Health and Family Welfare (MOHFW), national and international cancer research scientists, academics and employees of non-governmental organizations across the country, the selection of appropriate participants was an important part of NGT. According to Van de Van and Delbecq, (1972), 'Individuals must be able to speak in [a] common language', and those who have expertise, experience and insight into the issue at hand should be recruited. With these considerations, 40 participants were selected from the 120 participants. Four NGT groups of 10 participants each, were grouped based on their expertise and experience. By profession participants included medical doctor, health economist, public health specialists, social scientists and epidemiologist.

The NGT questions
The questions presented to the participants were simple and easy to understand to avoid cognitive burden. Wording and grammatical structures, which often affect the levels of abstraction, breath, and the depth of the elicited responses were taken into consideration, to avoid cognitive burden (Elliott et al., 2002). As Delbecq et al., (1975). states, 'NGT is like a microscope. Properly focused by a good question, NGT can produce a great deal of conceptual details about the matter of concern.
For the workshop, the questions were carefully formulated. The initial question was circulated among professionals in Bangladesh and abroad. The four questions were tested with several professionals before being used in the NGT group discussions. Two questions concerned with cancer research, and the other two concerned an intervention program (Table 2).

Conducting NGT Study in the ICCF
At the beginning of the exercise each group selected a chairperson as per the recommended application of NGT ( Van de Ven, 1972). The group members recorded five high-priority ideas that answer the assigned question in score cards. These were collected and shuffled. The votes on the each priority were ranked and summed to obtain a Tanzania (Mukundi et al., 2005). In this study, NGT was applied to a community of lay people, including patients and community leaders, to explore societal value preferences in the ranking of health conditions Mukundi et al., (2004). Redman et al., (1997) also used NGT to identifying priorities for the National Breast Cancer Centre in Australia and found a reasonably high degree of agreement on a number of priorities across groups.
A study in the U.S.A. used NGT to identify and to set priorities for diabetes care in a multi-ethnic state psychiatric hospital (Kahalokula et al., 2004). This was the first qualitative study to understand and address diabetes care issues. Another study in the United Kingdom found that NGT was successful in helping to develop a clinical practice guideline to improve the diagnostic needs assessment and management of dementia at a primary/secondary care clinic in Bristol, U.K (Kahalokula et al., 2004).
Students' choices in physical education have been investigated using NGT with young people in the U.K. and concluded that credible results with reasonable costs can be produced through this process (McPhail, 2001). Another study in the U.K. used this technique to understand patients and professional views on diabetic care in the community. Further, NGT was used successfully in the UK to involve cancer patients on identifying cancer research priorities in the U.K. for the first time, and it evidently showed the importance of patients and public involvement in priority setting (Corner et al., 2007). Robotin et al., (2010) applied NGT to refine initially identified priorities for pancreatic cancer in Australia through literature reviews, focus groups, semi-structured interviews and the Delphi process. NGT excels at generating ideas because it minimises reluctance among participants to express uncommon views and prevents individual ideas dominating at the expense of other important topics (Trickey et al., 1998).
NGT uses secret ballot, which offers a structure in which individuals can support or reject ideas without fear of recrimination. More ideas can be generated quickly from a group of people working individually within a group without cognitive burden on the participants. Furthermore, NGT uses a participatory approach, which makes it easier for the community members to participate and translate the results to action (Loman et al., 2003). It explicitly creates opportunities for each participant to contribute in the initial individual ranking and the voting stages (Walton, 1985). Ng (2000) describes NGT as a powerful tool for increasing a group's creative capacity for generating critical ideas and understanding problems for effective and efficient decision-making.
Considering all these advantages, NGT was chosen as a tool for the ICCF in 2010 in Bangladesh. NGT was useful in setting childhood cancer research priorities in Bangladesh. The literature clearly shows that NGT can generate credible information quickly and at a lower cost by ensuring the active participation of group members.
The literature suggests little or no research in childhood cancer in particular has been undertaken in LMIC. This gap has created an interest in using NGT to help set priorities in childhood cancer research in consolidated score for each suggestion. Finally, the chair identified the top five priority suggestions based on the score rankings.
On the second day, the facilitators of the four groups were asked to present their top five priorities in the plenary session as suggested by Van de Ven and Delbecq (1972). All participants were asked to comments and vote on the 20 priorities (4 groups x 5 priorities per group) as a group (five at a time), which were ranked according to the number of votes. A final list of six priorities was generated during this process (Table 3).
One important aspect of this exercise was the integration of the findings of four different NGT groups. Typically, NGT discussion groups address a single question, even when several NGT groups are involved. In our exercise, four different NGT groups addressed four different questions aimed at identifying research and intervention priorities on childhood cancer. Finally, the findings of the four groups were integrated through consensus voting.

Data Analysis
NGT uses a mixed methods approach (Corner et al., 2007) involving both qualitative and quantitative methods of analysis of data and information. First, each facilitator collected all score cards after the vote exercise and recorded and combined the results on the board to and ranked the responses from one to five, based on the total number of votes for each response. Second, the ranked responses were discussed in the plenary session prior to a final voting exercise and were recorded in a spread sheet. A final count of each response was performed to obtain a list of six priorities based on the total number of votes of individual ideas.

Results
Forty workshop participants were selected among the 120 workshop participants to participate in four NGT sessions. Each group composed of 10 participants. Each of the four NGT groups discussed different questions relating to childhood cancer research and intervention programs. The findings of the four NGTs were discussed in a plenary session, ranked through voting and then summarised as a list of six priorities.
Group one addressed the question: 'What interventions would you suggest to improve the diagnosis, treatment and care for child cancer patients in Bangladesh?'. Their initial 50 ideas/responses were grouped into three general categories: (a) the need for improved infrastructure; (b) the need for financial support (c) the need for human resources. However, after a detailed discussion, elimination of ideas and the voting process, the group agreed on seven distinct priorities. The ranking process indicated that increasing capacity for healthcare providers is the most important issue for the treatment of childhood cancer. This idea scored 45 out of a maximum of 50 points. The other ideas included: affordable treatment facilities practicing standard uniform protocol, availability of inexpensive diagnostic tools at the tertiary level, establish cancer registry to help inform research efforts, establish a center of excellence with international support to provide national and regional care for children with cancer, and public campaign to increase awareness of childhood cancers.
The second group discussed the following question: 'Which social interventions would you suggest for the general population regarding childhood cancer in Bangladesh? The group members generated 45 different ideas relating to social interventions, which were grouped into three broad areas: (a) Media intervention (b) Educational intervention and (c) Infrastructure intervention. After a detailed discussion, the ranking process indicated that an increase in awareness of childhood cancer through the media is the most important social intervention needed. Others include creation of a training, and referral system, targeting stigma related to cancer, early detection and creation of registry infrastructure, and awareness creation among policymakers.
The third group addressed the question of 'What types of clinical research are needed to address childhood cancer in Bangladesh, considering the resources and technology are available'. The responses were grouped in three broad areas: (a) Treatment research; (b) Population research; and (c) Clinical research. After discussion, the risk factors for childhood cancer in Bangladesh ranked as number one priority. Others include strengthening cancer registries in health facilities, performing a comparative study of different types of childhood cancer treatment protocols, early screening of childhood cancers using existing facilities, and community surveillance of childhood cancer were identified as the other priorities.
The fourth group covered the following: 'What types  2 What types of social and economic research are needed to address the current childhood cancer issues in Bangladesh?
3 Which interventions would you suggest to improve the diagnosis, treatment and care for child cancer patients in Bangladesh?
4 Which social interventions would you suggest for the general population regarding childhood cancer in Bangladesh? Table 2. Questions of social and economic research are needed to address the current childhood cancer problem in Bangladesh?' The priorities were grouped in three areas: (a) Social issues; (b) Economic issues; and (c) Political issues.
After discussion, the group identified socio-economic factors that affect early childhood cancer detection is the top priority. Others include: studying low-cost treatment protocols for childhood cancers; studying the distribution of cancer in the population; studying dropouts from cancer treatment; and understanding the role of family, doctors, media, donors, and hospital authorities in overcoming economic and psychological problems in cancer patients.

Integrated list of priorities
The priority lists from each of the four NGT groups were presented in a plenary session (Elliot et al., 2002). The objective of the plenary session was to discuss and identify an integrated list of priorities based on a final vote. All group members participated in the plenary discussion and were asked to provide clarification and highlight the merits of their high-priority ideas. After a detailed discussion, a final vote was performed on the 20 high-priority ideas. The results from the voting process revealed a list of six research and intervention priority areas to address childhood cancer in Bangladesh. A summary of the six highest-ranking priority ideas is presented in Table 3. The results of the ranking process indicated that establishing a comprehensive cancer registry was the top priority among the 20 ideas. Interestingly, three of the four groups identified establishing a cancer registry as a priority in their individual group rankings. For example, NGT Group one and NGT Group two ranked this as the fourth highest priority, and Group three ranked it as the second highest priority. In descending order, the remainder of the top five priority ideas were as follows: increasing capacity; raising awareness; conducting early diagnosis; and studying low-cost therapies. A comparative study on treatment protocols, which was the third highest priority of Group three, was listed as the sixth highest priority. The first priority for Group three (i.e., the study of risk factors for childhood cancer) was not ultimately considered a priority.

Discussion
The purpose of the ICCF was to elicit information and to identify priorities in addressing childhood cancer in Bangladesh. The NGT was used as a method of discussion among a multidisciplinary team of participants, which included healthcare professionals, social scientists, health economists and public health specialists. The NGT method has been employed as an evaluative tool in medicine, health care, nursing and other areas since 1960 (Macphail, 2001). The information collected from this exercise was used to design research projects and an intervention program on childhood cancer. The NGT method demonstrated to be a useful approach to elicit information from a multidisciplinary team efficiently. The process appeared to work well, as no discontent was noticed among the participants.
The technique was easy for the participants to understand, and it required a short amount of time to identify the priorities. The discussion was interactive, and the technique offered the participants greater freedom of choice because it reduced the possibility of domination by any influential or vocal member of the group. The interesting and exciting parts of the process involved generating ideas silently and a secret voting process (Makundi et al., 2005) considers the technique a unique opportunity for discussion and consensus by voting in a group. The process appears to be democratic, and all members had an equal opportunity to express their ideas freely.
Moreover, the final integrated priority was considered acceptable to all. Another advantage of this approach is that ownership of the final decision was given to all participants. This technique identified the foremost problems related to childhood cancer in Bangladesh and showed a relatively high degree of agreement among the participants. The process clearly demonstrates its strength in identifying priorities successfully. For example, the group identified the lack of a cancer registry as the number one priority. Bangladesh does not have a cancer registry system, which is essential for healthcare planning purposes. Similarly, to address the issue of childhood cancer, Bangladesh needs to develop its capacities in physical, financial and human resources, which was identified as the second-highest priority. We believe that the group identified the lack of resources as a priority because the shortage of physical facilities, medical supplies, equipment, qualified personnel, and training areas in public health facilities is well known. This situation is more acute in the case of cancer treatment facilities.
Taking the initiative to create awareness amongst aimed at the general public, policy and decision makers, and caregivers is also important because childhood cancer is not a priority in the government healthcare agenda. For example, the Ministry of Health's national cancer control strategy does not identify childhood cancer as a problem.  It is important that the general public is made aware of the treatment options available so they can benefit from the available resources. An overlap of priorities was found among the NGT groups. For example, the need for a cancer registry was documented by three NGT groups; although it received different ranks in the groups, it was the number one priority in the integrated list after final voting. Similarly, the importance of early detection of cancer was documented as a priority by Groups 2 and 3, and the need for further study on cancer treatment protocols was highlighted by Groups 3 and 4. Both of these priorities were designated as priorities in the final list through consensus voting. Several advantages of NGT were noticed over other qualitative discussion methods, such as focus group discussions. No one individual was able to dominate the discussion. All of the participants had equal opportunities to discuss, generate ideas and vote independently. This conditions are relatively difficult to establish when conducting focus group discussions.
In Conclusion, the issue of childhood cancer has not previously been explored in depth and was not considered a top priority in Bangladesh. Given that Bangladesh is one of the lowest-income countries in the world, where nearly 80% of the population who live in rural areas are less educated and live on a minimal income, it is surprising that socio-economic factors were not considered priority areas for further investigation. This forum was the first time this issue has been discussed at an international level. In addition, this forum represented the first time that the NGT was used as a childhood cancer priority-setting method at the national level. This exercise generated rich insights on children and cancer and presented childhood cancer as a priority agenda to policymakers. The six identified priorities were found to be relevant and important, although other priority areas may need to be addressed. In regards to outcomes, NGT successfully identified the scale of the problem and prioritise actions to address childhood cancer care issues in the country. Eventually, this forum established an effective international collaboration and implemented a twining program in Bangladesh. Countries facing similar situations could benefit from adopting this approach to settings childhood cancer priorities.

Funding statement
Canadian Institutes for Health Research (CIHR) provided funds for organising the Childhood Cancer forum.