Conception of an endodontics questionnaire in undergraduate dental education

The purpose of this study was to create a valid assessing instrument using a set of questions to examine preclinical dental education in endodontics. For this reason, we constructed the “German Endodontology Questionnaire” (GEndoQ), which assesses preclinical dental education in endodontics. In recent years and decades, various curricula in preclinical and clinical endodontic education have been evaluated at both national and international levels and the results published. However, the conception of the questionnaires that have been used have never been discussed or published. The GEndoQ was constructed in fi ve phases using the Delphi technique. In the fi rst phase, the questionnaire was generated according to former questionnaires and publications through the fi rst Delphi round and divided into 10 categories. In phases two and three, different expert panellists, such as specialists in endodontology and attendants of a master’s degree programme in endodontology, used the Delphi technique to confi rm the content validity of GEndoQ. The newest literature was implemented in phase four. In phase fi ve, GEndoQ was fi nally completed after multiple abbreviations were included following feedback from the expert panellists who employed the think-aloud method. Within the fi ve phases, GEndoQ Version 5 was created comprising 49 questions in nine categories. This was done using different answer options. The Likert scale with six possible options to choose from was used the most: 1= don’t agree at all; 2= don’t agree; 3= undecided; 4= agree; 5= fully agree, 6= don’t know. Seven questions could be answered in a free text format, while fi ve questions were in a single-choice format, such as yes/no answers. The GEndoQ is a valid instrument for assessing preclinical dental education in endodontics. Future research will focus on further refi ning and validating the instrument, for example, within a pilot test. Additionally, the questionnaire should be translated into English and validated to make comparisons among international dental faculties easier. Research Article Conception of an endodontics questionnaire in undergraduate dental education S Sacha, D Sonntag, S Rüttermann and S Gerhardt-Szép* Department of Operative Dentistry, Carolinum Dental University Institute, J.W. Goethe University, Frankfurt am Main, Germany Received: 13 May, 2020 Accepted: 28 May, 2020 Published: 29 May, 2020 *Corresponding author: Dr. Susanne Gerhardt-Szép, Professor, MME, Department of Operative Dentistry, Carolinum dental university-Institute GmbH, JW. Goethe University, Theodor-Stern-Kai 7, Frankfurt am Main 60590, Germany, Tel: +49-69-6301-7505; E-mail:


Introduction
with an earlier paper-based survey [1]. The study revealed a great divergence, especially in the teacher-student ratio, time management and teaching methods within British dental faculties. Endodontic education in the United Kingdom has developed positively over the last 20 years [1]. Sonntag, et al. evaluated endodontic training in Germany in 2008 [3]. They noted that preclinical endodontic training varied considerably among German universities due to differences in curricula designs, staff and course content [3]. In 2014, Gerhardt-Szép presented a survey in the context of the 11th conference "Training for Trainers". The survey analysed the divergence in general examination projects in preclinical endodontology [8].
Nevertheless, the conception of an assessing instrument with a set of questions to examine preclinical and clinical dental education in endodontics at the national and international levels has not been presented [2][3][4][5][6][7]. A former paper-based survey by Qualtrough and Dummer from 1997 [7] was used in the studies mentioned above, where the authors stated that no modifi cations to the former paper-based survey had been made [1,[4][5][6]. After 1997, various guidelines for endodontic education [9][10][11] and general guidelines for endodontics [12,13] have been formulated since many changes have occurred in the last decades with regard to endodontic equipment and materials Citation: S [14,15]. Furthermore, the guidelines from the European Society of Endodontology (ESE) [9] and the Association for Dental Education in Europe (ADEE) [10,11] support dental faculties in creating undergraduate endodontic curricula that promote consistent standards within Europe and enhance the quality of patient care in the community [1]. It is also known that considerable differences exist among countries in terms of curricula structure and content as well as the scope of practice generally in dental education [16].
Therefore, the use of contemporary dental assessing instruments for national and international comparison is important. Unfortunately, the current state of research indicates that no validated instrument for examining preclinical dental education in endodontics has been published, nor has the methodology of the conception and validation of such an instrument been explained [1][2][3][4][5][6][7]. In the most recent study about endodontic education in Great Britain, the author described that their survey "was piloted locally to check for question readability, clarity, validity and functionality and time required to complete" [1]. However, the authors did not specify the process. Reviews of literature concerning instruments for evaluating endodontics by general dental practicers worldwide show that little to nothing has been published about the construction and validation of the questionnaires used for their studies [17][18][19][20]. However, plenty can be found for the development and validation of instruments within the dental fi eld [21][22][23] and beyond the horizon of dental education [22][23][24]. Many publications offer general explantations for questionnaire conception and validation, such as following the "seven-step process for designing high-quality questionnaires" given in the Association for Medical Education in Europe (AMEE) Guide: No. 87 [21,[25][26][27] or similar step by step methods [28]. A large number use various methods, such as the Delphi technique [29][30][31][32][33] and the think-aloud method [34][35][36][37], for content validation.
On that account, the main goal of this study was to create an assessing instrument comprised of a set of questions to examine preclinical dental education in endodontics. Therefore, two different methods were applied, the Delphi technique as well as the think-aloud method.

Ethical approval
After consultation with the ethics committee of the Department of Medicine of the Goethe University Frankfurt am Main, it was decided that a vote by the ethics committee was not required (reference 114/2019).

General methodology
The development of the questionnaire consisted of fi ve phases using four Delphi rounds and the think-aloud method ( Figure 1).

Phase 1: Literature review and conception of a discipline-specifi c questionnaire
In phase one, the literature search of English articles published between 1991 and June 2019 was executed using the PubMed database. The search included combinations of keywords such as "endodontics", "education"/"teaching", "survey"/"questionnaire", "preclinical", "undergraduate", and "root canal treatment". These keywords were used with the Boolean operator "AND", although "education" and "teaching", as well as "survey" and "questionnaire" were used as synonyms in the search. In the fi rst step, the abstracts from the literature that was found was examined to determine the relevance of the content of each publication. Non-relevant literature was identifi ed and excluded. For all literature references with a (possible) content reference, the full text of the publication was then searched online. The following inclusion criteria were used in the consideration of publications in order to create the questionnaire: questionnaire/survey requesting the general status quo of endodontic education in the preclinical or/and clinical undergraduate part of dental faculties nationally/ internationally ( Table 1).
The questionnaire was constructed by two experts ( Table 2) based on the data provided in several studies and publications [1,11,14,15,38,39] and the Frankfurt learning objectives for endodontology (FLOE) (Figure 2). The working group named the instrument the "German Endodontology Questionnaire".
In the following article, the questionnaire is titled GEndoQ.
The expert group wrote e-mails, had meetings and discussed their conceptions of the questions. Due to the dental and medical education expertise of one of the authors, the content validity was ensured. GEndoQ Version 1 was created.  Table 2). In this round, the expert received the questionnaire digitally and was asked to ensure context validity and return written feedback directly on the printed questionnaire.    Table 2). Each of them received a printed questionnaire and was asked to improve its grammar, clarity and comprehensibility. They were also asked to ensure objectivity and provide written feedback directly on the printed questionnaire.

Phase 4: Modifi cation of the questionnaire
In the fourth Delphi round, the expert group from the fi rst phase modifi ed and supplemented the third version of GEndoQ using the most recent publication from 2019 concerning undergraduate endodontic education [1].

Phase 5: Final conception of the questionnaire
In the last phase, the panellists, 15 dentists from the previously mentioned Department of Operative Dentistry (specialists in endodontology, graduates of structured further education with endodontological content and participants of a master's degree course in endodontology, participants with and without a doctorate, see Table 2), checked the fourth version of the GEndoQ for reliability, objectivity, content validity and processing time. The study and the questionnaire were presented as a PowerPoint presentation to the panellists.
In this phase, the think-aloud method was used [35].

Phase 1
Using the combination of the keywords "endodontics AND education AND survey AND undergraduate" in the PubMed database, 87 results were returned. For the search "endodontics AND teaching AND survey AND undergraduate", 81 results were returned by PubMed. Twenty-nine results were returned when asking for "endodontics AND teaching AND survey AND preclinical". Following the criteria mentioned above, six publications were considered (Table 1). In addition, two publications of guidelines for endodontic education for undergraduates and guidelines for endodontic treatment in general were chosen (Table 1).
A total of 45 questions with 149 sub-items (sub-items are the possible answer options presented below the questions) were conceived in the fi rst phase of the Delphi round 1. The GEndoQ was divided into ten different categories that are illustrated in Table 3.  single-choice formats, such as yes/no answers. The category III "Learning-teaching settings: Theoretical endodontology" and category IV "Learning-teaching materials: Theoretical endodontology" also asked for the percentage of the used settings and materials. At the end of categories III, IV, VI, VIII, IX and X, the question was asked whether something else was worth mentioning for the specifi c category (Table 4).

Phase 2
In the second phase, the second Delphi round, 22 changes by the panel expert were accepted and the questionnaire was adapted. For example, questions 5 and 10-the sub-item "dentist without specifi c education"-were complemented, while some questions had to be formulated more precisely for better comprehension (Tables 5-7). This resulted in the second Figure 3: GEndoQ Version 1 after phase one. For a better overview, the questionnaire was marked with references numbers to see from which sources the questions and sub-items came. Shown here is category I. General information is illustrated, consisting of 5 questions. Questions 1-3 can be answered in a free text format, question 4 has a single-choice option, and question 5 offers a Likert scale. The questionnaire was divided into 10 categories in order to provide clarity and allow for a schematic response. https://www.peertechz.com/journals/journal-of-dental-problems-and-solutions Citation: S

Phase 4
The third version of the GEndoQ was modifi ed and supplemented by 11 more questions to a total of 55 questions and 218 sub-items (Table 12). A question was added that asked for the time that dental schools dedicate to specifi c topics taught in preclinical endodontic classes: root canal anatomy and pulp histology, pulp pathology and endodontic microbiology, endodontic radiology, endodontic materials, vital pulp therapies, root canal treatment on immature teeth with non-vital pulp tissues, root canal treatment, root canal re-treatment, endodontic surgery, endodontic regeneration, restoration of root-fi lled teeth, bleaching of endodontically treated teeth, dental trauma, endodontic emergencies.     The questions about post-endodontic restoration, the time needed to complete the endodontic treatment and whether there was a special department for endodontology were added to complement the questionnaire. Some questions were supposed to be answered in a free text format; others were in a multiple-choice format.

Phase 5
In phase fi ve, the expert group recommended that some questions and sub-items should be abbreviated in order to shorten the processing time. Therefore, the endodontic learning goals had to be reduced to fi ve major ones, which combined theoretical and practical learning goals. Several questions were changed to free text answer questions instead of the Likert scale, and vocabulary was defi ned more precisely. All in all, 20 changes were suggested Tables 13-14, Figures 8-9 illustrate some examples of the changes made). This step of development guaranteed the objectivity of interpretation. After this phase, GEndoQ was shortened to 49 questions with 99 sub-items (see GEndoQ Version 5 Figures 10-15).

Discussion
The aim of this study was to develop a questionnaire, using various methods, that can evaluate preclinical endodontic education.

Literature review and conception of a discipline-specifi c questionnaire
In the fi rst phase, the expert group conducted a literature review to ensure that the idea of the study was orientated on decisive former publications, following the AMEE Guide No. 87 and the Survey Development Guidance for Medical Education Researchers [25,40]. Its aim was to bring the reader up to date with the literature on a specifi c theme and form the groundwork for future research [41]. Kossioni, et al. described in their methods for developing a questionnaire to measure the clinical learning environment for undergraduate dental students, that a literature review should be conducted [22]. Several other studies have made a similar statement [21,25,[42][43][44]. In this current study, the publication of Sonntag, et al. [3] and the survey of Gerhardt-Szép [8], served as the basis for the questionnaire;  both authors are experts in this study. In addition, the new licensing regulations for dentists (ZApprO) [38], the national competence-based catalogue of learning objectives (NKLZ) [39] and the Frankfurt learning objectives for endodontology (FLOE) were used. The publications using similar questionnaires that were found through the literature review [1,2,[4][5][6][7] were used as orientations for developing the GEndoQ. These were the only publications between 1991 and 2020 provided by the database that included the criteria "questionnaire/survey requesting the general status quo of endodontic education in the preclinical or/and clinical undergraduate part of dental faculties nationally/internationally". Furthermore, the guidelines for endodontic education for undergraduates and guidelines for endodontic treatment in general were added as orientation publications for the questionnaire [9,12]. The European Society of Endodontology (ESE) published undergraduate curriculum guidelines for endodontology in 1992, 2001 [45,46], the latest publication in 2013, and " […] formed a benchmarking reference It was proposed to leave out the size for "Demonstrations" and "Hands-on-exercises", as well as defi ning the word "peer-tutored" (see Tables 8-11). The percentage sign that was missing was added. This fi gure presents a portion of the changes made as an example.   for dental schools and regulatory bodies […]" [9]. Therefore, the ESE guidelines were taken into account to ensure that the questions and sub-items remained relevant.
In this study, the GEndoQ was generated by a team of two individuals using the Delphi technique (explained below). The one individual, besides being a dentist, is an expert in medical education, thereby ensuring content validity.

Content validation using different methods
To evaluate the instrument's content validity, which assesses the relevance of each question and sub-item, different panel experts reviewed the questionnaire in several phases [27]. Baker, et al. stated that "[e]xperts provide an accessible source of information that can be quickly harnessed to gain opinion.
They can often provide knowledge when more traditional https://www.peertechz.com/journals/journal-of-dental-problems-and-solutions Citation: S  research has not been undertaken" [30]. Corresponding to Skulmoski, et al., the expert panellists in the Delphi round should meet four "expertise requirements: i) knowledge and experience with the issues under investigation; ii) capacity and willingness to participate; iii) suffi cient time to participate in the Delphi technique; and iv) effective communication skills" [47]. The use of expert panels is common for assuring content validity of questionnaires [22], [24], [29], [43], [44] and is recommended by the AMEE Guide No. 87 [25].    Table 12: Delphi round 4: Based on a recent publication, the experts from the fi rst Delphi round added 11 more questions to the GEndoQ. The questions and answer options are illustrated in this table.   Table 14: Delphi round 5: Fifteen panel experts who checked for reliability, objectivity, validity and processing time suggested 20 changes. In this phase, the think-aloud method was used. Part II: Six changes are shown.  In the current publication, the experts used the Delphi technique in different phases. The so-called Delphi technique is a systematic, multi-stage decision-making process with resonance, in which experts or groups of experts assess the questionnaire and provide feedback [48][49][50][51]. According to Buckley, et al., the Delphi technique is used when the topic under investigation is not suitable for precise analytical techniques but can benefi t greatly from subjective judgements on a collective basis [52]. He also refers to the possibility of variations in a true Delphi study [53]. In general, the Delphi technique results in multiple iterations to create a consensus of opinions concerning a specifi c topic, using a controlled feedback process [47,54,55].
For the second and third Delphi rounds in the development of the GEndoQ, the experts were selected according to their professional competence [56]. Therefore, the second Delphi round consisted of one expert. This expert, among other qualifi cations, was a specialist in endodontology and an author of an already existing publication on the same topic [3]. According to Powell, et al. representation is assessed by the qualities of the expert panel rather than its numbers [57]. A similar approach was made for the third phase within the third Delphi round evaluating GEndoQ Version 3. Four panel experts were chosen: one specialist in endodontology, one graduate of a structured advanced training course with endodontology content and two participants of a master's degree programme in endodontology. From today's point of view, no standardized number of experts has been recommended for the Delphi technique. The literature has indicated that panel size has ranged from two experts to hundreds of experts [47,56,58,59].
Comparing studies that have evaluated preclinical and clinical education in endodontics, only Al Raisi, et al. [1] stated that they validated their questionnaire, which was a former paper-based survey that had been developed by Qualtrough and Dummer in 1997 [7]. Unfortunately, the validation process of the questionnaire was not part of the publication since no modifi cation had to be made to the almost two-decades-old instrument. Therefore, it is unknown through which method the instrument used by Al Raisi, et al. was validated and why it was decided not to change anything. One might think that modifi cations would have been necessary because various guidelines for endodontic education were formulated after 1997 [9][10][11]. In addition, many changes have occurred with regard to endodontic equipment and materials in the last decades [14,15].
After generating GEndoQ Version 3, the instrument was complemented by 11 questions from the publication of Al Raisi, et al. Part of the questionnaire that Al Raisi, et al. used in their study was illustrated in their publication [1]. Their questionnaire was based on a former paper survey [7]. The expert group of the fi rst phase could, therefore, approximate the questions to the questionnaire of Qualtrough and Dummer from 1997 [7]. This has made it more comparable to the studies mentioned above [2], [4], [5] since those studies were based on the same questionnaire. In addition, the questions and subitems used by Al Raisi, et al. considering preclinical education in endodontics, completed GEndoQ Version 4 after the additional literature review [25,40].
In the last phase, GEndoQ Version 4 and the study was presented in a PowerPoint presentation to an expert group consisting of 15 dentist expert panellists with different qualifi cations (Table 2). After the presentation, the panellists were asked to provide their feedback on GEndoQ version 4 using the think-aloud method, and the verbal feedback was transcribed (Tables 13-14). This method asks the experts to verbalise their thinking during the thought process [60]. The think-aloud method is a scientifi c method that has been used in various disciplines [61]. For example, Zahiri Esfahani, et al. used the think-aloud method to measure effectiveness, learnability, errors and effi ciency characteristics of a picture archiving and communication system [35]. In contrast, Adams, et al. explained the use of the think-aloud interview in order to create and validate a test [34]. Field et al. described that their questionnaire-which assessed the "pan-European practice in relation to curriculum content, teaching and learning strategies and assessment of preclinical dental skills"-had been originally piloted through think-aloud testing [62]. Therefore, the thinkaloud method was an adequate method to gather feedback and make modifi cations to GEndoQ Version 4.

Transferability to national and international levels
The GEndoQ questions and sub-items were based on recent publications in order to make the results more comparable [1,2], [4,7]. It is important to see how and whether dental faculties have developed in the last years and whether they have adapted their endodontic curriculum to the contemporary guidelines of the ESE [9]. Standardisation and quality assurance in both endodontic training and general dentistry are necessary because since 1981, qualifi cations for various health professions, including dentistry, have been mutually recognised throughout the European Union in accordance with EU Directive 81/1057/EEC [63].

Limitations
Concerning the literature review in the fi rst phase, it has to be mentioned that a single database was used for the search. Perhaps more matching publications could have been found if more databases had been used. However, this would have exceeded the time allowed for this project.
The GEndoQ is an instrument for evaluating preclinical education in endodontics for German-speaking dental faculties. The questions have been formulated in the German language. Therefore, the English version of the questionnaire would need to be validated after translating as well.
Finally, as the validation process is not yet complete, the fi nal instrument should be fi eld-tested. Therefore, in future research, the GEndoQ will be tested and applied in real dental faculty environments in order to refi ne further and eliminate any weaknesses that the questionnaire might have.
While this questionnaire has been generated as a paperbased survey, a web-based or mail survey would "offer anonymity and […] afford respondents time to complete the questionnaire at their own pace", according to Cavana et al.

Conclusion
GEndoQ is an instrument to assess preclinical education in endodontics, but the complete validity and reliability has not been assessed. It should be part of further research after fi eldtesting this instrument. This study emphasises the signifi cance of an instrument to assess preclinical education in endodontics.
The goal is to make endodontic curricula comparable with each other on a national or international basis. The fi ve phases show the effort of the conception of our questionnaire. In general, the methodology of such an instrument should not be underrated.