INFORMEG , a new evaluation system for family medicine trainees : feasibility in an Italian rural setting

Introduction: In Italy the course to become a general practitioner (GP) lasts 3 years and includes both theoretical and practical study. Different from the theoretical part, until recently the practical activity has not been assessed at all. The Emilia Romagna Regional Health Authority has developed a special program called INFORMEG (Management of Tutoring during the Triennial Specific Training in General Practice), aimed at assessing primary doctor trainees’ practical skills. INFORMEG includes a list of predefined cases of specific diseases, conditions or health problem, a web application and a smartphone app, aimed at assisting trainee self-management and helping the tutor in the assessment of trainee performance. The Emilia Romagna Regional Health Authority divided the pre-defined cases into three categories (A, B and C) according to their relevance to a trainee’s education and coded them using the International Classification of Primary Care (ICPC). The aim of this project report is to illustrate the implementation of INFORMEG in a rural setting. Methods: Program evaluation took place from 2 May to 31 October 2013 during GPs’ routine clinical activities. The following steps were accomplished during every meeting: (1) consultation recording; (2) identification of the reason for the encounter (RfE); (3) classification of the diagnostic procedure(s) performed (diagnostic/therapeutic/test results/administrative/advice); (4) classification of special procedures called 'practical clinical skills' and (5) elaboration of the final diagnosis after the encounter. Results: The number of cases of specific disease or condition encountered by the trainee were 98 for type A, 57 for type B and 22 for type C. A total of 605 RfEs were collected: 376 for type A cases, 147 for type B cases and 82 for type C cases. A total of 976 procedures were performed during the 6 months: 590 procedures for the type A cases, 271 for type B and 115 for type C.


Introduction
In Italy, the qualification required to work as general practitioner (GP) in the National Health Service is obtained through a course, lasting 3 years, organized by the Regional Health Services.The course includes both theoretical study and practical activity.The practical part consists of 6 months of training at a senior GP's surgery.The theoretical part includes seminars and assessments planned every 6 months under the supervision of the coordinators of the theoretical area.Until recently, the practical activity was not assessed at all.
To overcome this problem, the Emilia Romagna Regional Health Authority has developed a special program called INFORMEG (Management of Tutoring during the Triennial Specific Training in General Practice).INFORMEG includes a list of pre-defined cases, a web application and a smartphone app, and is aimed at assisting trainee selfmanagement and at helping the tutor in the assessment of trainee performance.INFORMEG is based on a system called Patient Encounter Log System (PELS); this system has already been used by the students in Medicine at the Jefferson Medical College, Thomas Jefferson University, Philadelphia 1 .The new PELS system makes use of a personal digital assistant (PDA) and offers several opportunities: students are able, in real time, to review their experiences and share this information with their tutors, fixing, in this way, possible gaps in their education 2 .
A study published by Lopez et al. in 2004 showed how the system is able to produce an objective documentation of the basic procedures performed by students in the fourth year of the Bachelor of Medicine during the compulsory internship in an emergency medicine department 3 .During this internship, the students learned to perform basic procedures such as tracheal intubation, lumbar puncture, insertion of a central venous catheter and suturing of a wound.These procedures were then recorded through the PELS in their PDA.At the end of the internship each student was required to upload their data to a central database managed by the School of Research in Medical Education.The results of this study showed the feasibility of documenting both training in emergency medicine and the performance of basic procedures through the PELS-PDA system.So far this tool has not been used in a rural primary care setting.
The aim of this project report is to illustrate the implementation of INFORMEG in a primary care rural setting.

INFORMEG: professional roles
Three professional roles are necessary in INFORMEG: the trainee, who is enabled to enter the cases' data; the tutor, who assess the trainee's work; and the coordinator, who certifies that a trainee has fulfilled the requirements to pass the exam.Each actor in the training process (trainee, tutor and coordinator) has access to the system via a username and password.

Use of the International Classification of Primary Care to record the data
Patients' data are recorded using an identification code, and include date and site of the encounter and patient medical history.Each case needs a specific diagnosis, which is coded according to the International Classification of Primary Care (ICPC) 4 .The ICPC is quite popular in the primary care setting.It was developed and is constantly updated by the WONCA International Classification Committee.
ICPC has a biaxial structure, with 17 chapters on one axis and 7 components on the other (Fig1).Chapters are based on body systems, with an additional chapter for psychological problems and one for social problems.Each chapter is identified by a single alphabetic code, which is the first character of all rubrics belonging to that chapter.The seven components of each chapter are identified by two-digit numeric codes.Component 1 codes symptoms and complaints, component 2 diagnostic and preventive procedures, component 3 treatment procedures and medication, component 4 test results, component 5 administrative tasks, component 6 referral and other reasons for encounter and component 7 diseases.
Each episode of care includes a reason for encounter (RfE) and a final diagnosis.The RfE(s) is defined as an agreed statement of the reason(s) why a person enters the healthcare system representing the demand for care by that person.The RfE should be recognized by the patient as an acceptable description of the demand for care.ICPC is a simple and essential tool but, at the same time, maintains methodological, epidemiological and semantic rigor 5 .There is a mapping between ICPC encodings and the codes of the International Classification of Diseases (ICD) versions 9 and 10, which allows a dialogue with the organizations that use these encodings in the healthcare environment.ICD is the official classification in many countries; in Italy the version utilized is the ICD 9 CM 2007.To speed up the search for codes and definitions, it is possible to use the compact version, called the ICPC two-pager, which has been translated into several languages including Italian 6 .
For each consultation, the following data were recorded: RfE, procedures performed and final diagnosis.Some special procedures regarded as essential practical manual skills in a rural setting were also recorded 7 .These abilities are those that help practitioners to avoid undue referral to secondary care specialists or casualty departments (Table 1).

Case definition
The term 'case' identifies the development of the health problem presented by the patient and the physician, according to the methodology of the 'episode of care' in the ICPC classification.

The episode of care
The episode of care begins with a pattern of presentation, called for simplicity the RfE; it requires the execution of medical acts, called 'procedures' (medical examination, imaging, prescription, etc.) and ends with a 'diagnosis', established by the physician according to his interpretation of the problem.

Case development
The case develops during one or more 'encounters'.An encounter is defined as contact between the doctor and the patient.During an encounter one or more cases may develop.Each case can develop across one or more encounters, depending on whether the case is considered closed after the first encounter or requires additional encounters to complete its management and investigation 8 .Chronic health problems, such as diabetes, arterial hypertension, dyslipidemia, require regular meetings throughout a patient's life 9,10 .
The list of pre-selected cases was developed by a committee which included all the tutors of the Emilia Romagna region.The resulting cases are divided into three categories by the educational committee (Table 2): Type A cases: These include 10 health problems considered very important and that cannot be ignored.For each of these problems four cases are required, for a total of 40 cases.
Type B cases: These include less relevant health problems (in terms of importance or frequency); 20 cases are required.
Type C cases: These include less frequent and more complex health problems, and 10 cases are required.
Adapted, with permission, from reference 4.  Hence, the total number of cases required during the 6-month internship is 70.The exact number of cases required was decided by the educational committee, which based its decision on the prevalence and relevance of the health problems; this aspect was under evaluation for possible future adjustments.

Study design
The setting of the study was a rural solo practice in the Emilia Romagna region of Italy.
A period of 2 weeks was allowed for the trainee to become acquainted with the ICPC classification.The internships took place from 2 May to 31 October.The trainee sat next to the family doctor and had access to each patient's electronic medical record.Other simple tools available for the trainee were the list of pre-selected health problems to be collected, a notebook and the ICPC two-pager.When the case encountered was one of those included in the preselected list, the trainee started taking notes and asked for patient consent about the collection of data.Afterwards, during the lunch break, the trainee entered the cases into a spreadsheet.
The following steps were accomplished during a meeting: • identification of the reason for the encounter • classification of the diagnostic procedure(s) performed (diagnostic/therapeutic/test results/ administrative/advice) • registration of the special procedures defined as 'important clinical skills' • elaboration of the final diagnosis.
For the electronic clinical records, the tutor was provided with software called SIMEBA (Procexor; http://www.procexor.com/viewdoc.asp?co_id=33), which is structured for the collection of episodes of care as it is classified in the ICPC.

Data analysis
Data were evaluated using descriptive statistics such as counts, percentages and cumulative percentages, as well as measures of central tendency and dispersion such as mean and standard deviation SD).Continuous variables were reported as the mean values ± SD, and categorical variables were reported as percentages.Descriptive statistical analyses were conducted using Statistical Package for the Social Sciences for Windows v22.0 (IBM; http://www.spss.com).

Assessment of INFORMEG
The final assessment of INFORMED was made using a focus group discussion conducted with the main tutor, the coordinator and other tutors involved in the project, at the end of the semester.The feasibility was evaluated using an assessment of the ease of use of the tool, the comprehensiveness of the selected cases and a final discussion about other issues raised by the tutor and the trainee.

Data protection and confidentiality
Subjects' identities were protected and no individual subjects could be identifiable from the data.A subject information sheet with simple straightforward information about the project was given to all patients and doctors involved and an informed consent form was signed by the subjects who agreed to be involved.All subjects encountered were aged 18 years or more.

Ethics approval
Formal ethics approval is not required for this kind of study in Italy.The Emilia Romagna Regional Health Authority was informed about the protocol of this project.The protocol of the study is in compliance with the Helsinki Declaration.

Results
Data included a total of 1536 patients, with a mean age of 54.39 years (SD 19.87), range 9-98 years, with a small prevalence of women (55.92%).Given the paucity of home visit cases, only encounters in the doctor's office have been considered.

Description of cases
A total of 98 type A cases, 57 type B cases and 22 type C cases were recorded.Among the type B cases, we were not able to find health problems such as substance abuse (P19) or advanced malignancy of the breast (X76) and prostate (Y77); for type C, no cases of fear of malignancy (A26) were encountered.The data proved to be normally distributed so mean and standard deviation were used as the measure of central tendency (Table 3).

Analysis of reasons for encounter
A total of 605 RfE were collected: 376 type A cases, 147 type B cases and 82 type C cases.The RfE is the description of the reason(s) that brought the patient to the doctor's surgery.Relational skills are crucial in determining the RfE, and the number of RfEs can be considered a reliable indicator of how much the patient was 'listened to'.
In ICPC all components of the classification can be used as an RfE: symptoms or disorders, diagnosis and procedures.

Analysis of procedures
A total of 976 procedures were performed during the 6 months: 590 procedures for type A cases, 271 for type B cases and 115 for type C cases.Most of the cases collected were type A cases (98 compared to the 40 expected); for type A cases all the pre-selected health problems were encountered.
For the type B cases, 57 were collected (only 20 were officially required); 85% of these health problems were encountered.Health problems such as neoplasia advanced breast and neoplasia advanced prostate and cases of substance abuse were not found.For the type C cases, 22 were collected (10 were required) and 86 of the pre-selected cases were encountered.Table 4 shows the representativeness of the cases encountered.

Discussion
Coding of the cases with the ICPC was a straightforward task and proved to be of high educational value and easy to use.
The number of procedures recorded could be considered as an indirect indicator of comprehensiveness and heterogeneity of the cases 11,12 , while the RfE, as previously stated, is an indirect indicator of the attention to the patient.
One issue raised in the focus group discussion was the absence among the pre-listed health problems of some common arrhythmic conditions such as atrial fibrillation and in particular the management of anticoagulation therapy for stroke prevention in atrial fibrillation.
The cases identified proved to be very illustrative of the complexities of clinical management, confirming the importance of a holistic view of the human condition 13,14 .
The strengths of INFORMEG are that (1) the time available and the organization of the practical training at the tutor's surgery are sufficient and appropriate; (2) the ICPC coding for each act performed during the activity has a high educational value, as it helps the trainee in the construction of the case according to the logical process of family medicine; (3) the large number of procedures employed allows the trainee to develop awareness of the operability of the clinical work, which has been often neglected in the past; (4) the coding for the manual activities and practical skills highlights these abilities -which are essential, especially in a rural setting; (5) the close interaction with the tutor, which is also very rewarding.
It is worth mentioning the high number of total encounters performed in only 6 months 15 .Our surgery was in a rural area with walk-in access.
Among the most comprehensive are The EURACT Educational Agenda of General Practice/Family Medicine 20 and the EURACT Performance Agenda 21 , which are aimed at fostering and harmonizing the competencies of future GPs in different European countries.Core competences worth mentioning are the management process of primary care, the personcentered care, specific skills/abilities in problem solving, the comprehensive approach and community orientation.We believe that INFORMEG may represent an additional contribution that could help in the difficult task of evaluating trainees during their vocational training.
Some parts of INFORMEG need to be amended: the absence among the pre-listed health problems of some common arrhythmic conditions, such as atrial fibrillation and in particular the management of anticoagulation therapy for stroke prevention in atrial fibrillation, and the absence of means to assess the patient-trainee relationship which so far has been evaluated with another tool.

Conclusions
INFORMEG is feasible and easy to use.The pre-selected health problems were almost all addressed, thus confirming the good degree of representativeness of these clinical cases, even in a rural setting.The ICPC coding has a high educational value in helping the trainee in the construction of the case according to the logical process of family medicine 22 .The evaluation of the trainees' ability to perform the specific procedures in Table 1 are of the outmost importance, especially in a rural setting where the long distance from secondary care specialists and casualty departments requires quick and qualified interventions by GPs.

Figure 1 :
Figure 1: Structure of the International Classification of Primary Care.

Table 1 :
Special procedures regarded as essential practical manual skills in a rural setting ICPC, International Classification of Primary Care

Table 2 :
Description of case types A, B and C and number of cases encountered ICPC, International Classification of Primary Care.NOS, not otherwise specified

Table 3 :
Summary of encounters, reasons for encounters and procedures

Table 4 :
Representativeness of the cases encountered