The Delphi survey collected expert opinions on the content of SGs created more than 10 years ago to meet SurSaUD® surveillance objectives. When consensus was found for only a portion of the submitted SGs, other codes suggested by the participants were added.
Selection and participation of experts
SGs are indicators based on clinicians’ consultations and perceptions. It was therefore important to link the monitoring objectives to clinicians’ coding practices, hence their inclusion in the indicator review process, as they provide the medical expertise useful for defining SGs [8]. Almost all experts in the SOS Médecins survey and just over half of the experts in the OSCOUR® survey stated that they were familiar with the SurSaUD® system. This knowledge of the SurSaUD® system may have contributed to the experts’ proper understanding of the monitoring objectives and influenced their opinion in the selection of the composition of SGs. However, it is difficult to assess the impact (positive or negative) on the survey.
This knowledge of the network may also have helped to maintain a high participation rate from the first round, especially for the SOS Médecins survey, but also in the 2nd and 3rd rounds, despite the longer periods than initially planned. Among the volunteers who originally took part in the survey, more than 3 out of 4 experts were registered in the SOS Médecins survey and 1 out of 2 experts in the OSCOUR® survey. A participation rate of over 80% in the 2nd and 3rd rounds for the 2 surveys was observed among the respondents of the 1st round.
The poll was anonymous and the results of each round could not be used to identify anyone’s responses so as not to influence respondents in their future decisions.
survey process
The survey took place in 2019 in three separate rounds over a period of 5 months for SOS Médecins and 9 months for OSCOUR® Content and the number of codes available. The OSCOUR® SG survey included too many codes and subcodes that required discussion of their display up front to make the survey easier to read and understand while optimizing the time required to complete the survey. This initially required unplanned developments to enable user-friendly display of subcodes in tooltips through rollover of diagnostic codes (display method that is used again after each round to return the results). This approach probably had a positive effect on maintaining the participation rate over the course of the survey. In addition, the OSCOUR® survey was suspended during the summer holidays (2 months in total) as some areas are affected by an increase in their tourism-related activities and there is little time left to respond to this type of survey for the ED physicians involved. The extension of the survey duration had negative effects, e.g. B. the higher number of reminders only for SOS-Médecins. Again, this could have resulted in participants’ recall bias, although this was partially compensated for by using the bar graph corresponding to the answer selected in the previous round.
Delphi method for assembling syndromic monitor indicators
Although syndromic surveillance has existed for several years and is widely used [16, 17]there is no reference definition for SGs that would otherwise make it possible to facilitate the exchange or comparison of data between systems and to evaluate performance [8].
To our knowledge, this is the first time that the Delphi method is used to work on the definition and composition of SGs. In existing publications, the method of group consensus after a discussion meeting is often used [7, 8]. Using the Delphi online methodology, a panel of experts working in different geographic areas could be consulted without the need to schedule meetings or travel to them. In addition, in view of the large number of SGs to be examined, several discussion rounds would have been necessary in order to come to a conclusion for all SGs. This would probably have been an obstacle to the participation of several clinical and international experts and their workload would not allow them to be so closely involved in this type of project.
Finally, this approach also measured a percentage of consensus, which was a more objective guide to deciding which codes to keep or not in each SG.
reached consensus level
The SGs for which consensus on codes was already reached in the first round had a specific surveillance objective.
In syndromic surveillance, sensitivity is used to detect the largest number of patients who are likely to be in the early stages of a disease that has not yet been characterized (with few specific signs), while specificity is used to limit investigations in a large number of cases to be identified with similar symptoms [8]. In studies of SG performance, a better positive predictive value is observed when the monitoring target is specific rather than sensitive [7, 8].
The diagnostic codes for SGs used for winter surveillance (bronchiolitis, gastroenteritis) were all retained from the 1st round. The specific target and the small number of codes they encompassed probably facilitated consensus among experts. They have been used in all seasons for many years and are deployed in regional multi-source surveillance, which is widely used, with reports published weekly and meetings with data provider partners where they have been regularly revised. This visibility can also help healthcare professionals to see how useful seasonal monitoring can be, as it is carried out with the aim of helping to organize and adjust the supply of care, which directly benefits clinicians in their day-to-day work. These hypotheses could also explain the results for hyperthermia/heat stroke or insect bite SGs traditionally used in summer surveillance, although both SGs have poorer consensus, particularly the latter, or for trauma and abdominal pain SGs, which are the most frequently found Diagnoses often rank first among the top 10 reasons for seeking emergency medicine.
It is difficult to reach consensus on SGs with sensitive objectives, which more often involve symptoms (diarrhea, abdominal pain, anxiety disorders, stress, etc.). Among them, some codes for impaired general health SG (OSCOUR®) were rejected while others failed to reach consensus. This SG reflects a clinical picture that can have different etiologies and be associated with multiple pathologies, meaning that it can be perceived differently from one doctor to another. This example showed that surveillance objectives were not always clear enough and expert responses did not always match epidemiologists’ expectations. Therefore, participant responses focused on end-of-life or elderly support, while this SG for epidemiologists in charge of health surveillance aimed to predict the sudden deterioration of a patient’s condition (with or without a clearly identified etiology). measure up. .
Another example is the SG for lower respiratory tract infections, for which most codes failed to reach consensus. However, this SG addresses several issues, particularly when monitoring respiratory diseases in winter [18].
SGs with a sensitive target are composed of a variety of diagnostic codes that can pose an obstacle to consensus building. Furthermore, since not all codes and subcodes were displayed, this certainly influenced participants’ choice and could partially explain the lack of consensus for some codes. It is not certain that all participants saw the tooltip displayed when hovering over certain codes with the mouse, i.e. only read part of the subcodes in the selection in response to the set monitoring goals.
There was little discrepancy between the responses from international experts and those from ED specialists. Despite the lack of a reference definition, these results suggest that the development of indicators coincides between countries, thus allowing comparison of observations between international systems, which is a strong point in the case of international threats.
More generally, the results highlighted two limitations of using the full ICD10 classification (40,000 codes) to encode medical diagnoses in ED. First, this classification includes symptoms that should not be used to code medical diagnoses [19]. These symptoms would be more relevant to the coding chief complaint. These symptom codes in the definition of SG may partially explain why consensus was not reached for that SG. Furthermore, unconstrained datasets with a large number of codes available for ED have been shown to provide poor data [20]. The usability of a system is an important factor in the quality of the data we collect [21]. Based on these observations, a study was conducted by the ED syndromic system in the UK. Together with a panel of experts, they proposed a limited list of about 1200 codes based on the SNOMED ontology, without any symptoms in that list [22].
A similar process was started in France to revise the format for collecting ED data. In particular, a new format for ED data proposed three major developments: A list of 1500 ICD10 codes was defined for coding medical diagnoses, rather than the entire ICD10 classification. Symptom codes are removed from this list. A thesaurus for coding the main complaint was also proposed (in the current format, the information was collected in free text without thesaurus). Finally, additional information would be collected to indicate the circumstances of the ED visit. This new format is still under discussion and could be implemented soon.
Review of the SGs and implications for epidemiological surveillance
Although the survey made it possible to add diagnostic codes to SGs that were initially missing and suggested by the experts, other diagnostic codes were not selected and the differences should therefore be discussed with the expert group.
The epidemiological impact of the results of this survey on the composition of SGs should be analyzed by comparing the temporal dynamics of the previous and the new composition for each SG. It would also be relevant to assess the performance of SGs by calculating their sensitivity and specificity in relation to the diagnoses actually reported in the medical record, but such studies are burdensome and expensive at the national level and can only be taken into account to a small extent, be it geographically or in Reference to the selected SG.
