Abstract

Objective: This study aimed to examine the sensitivity and specificity of occupational health doctors (OHDs)’ reading of early-stage pneumoconiosis radiographs. Materials and Methods: A screening test was applied, and 33 OHDs consented to participate in the study. There were atotal of 67 chest radiographs, which consisted of normal and early-stage pneumoconiosis film. The cut-point for disease was set at profusion0/1 and 1/0. Mean sensitivity and specificity for small opacity detection were analyzed. Results: The median sensitivity of ILO profusion of 0/1 or above was 88% (IQR 10.3), and the median sensitivity of film with a 1/0 cutoff pointwas slightly higher at90% (IQR 10.3).The average specificity for ILO profusion of 0/1 or above was 43.3% (SD21.1). Upon increasing the cut-point at profusion1/0, the average specificity increased to 47.0% (SD 20.9). Conclusion: This study showed that occupational health doctors were able to interpret chest radiographs of workers with early-stage pneumoconiosis. This indicates that the development of OHDs’ chest X-ray reading skillsis valuable in order to improve the national pneumoconiosis surveillance system.

1. Introduction

Silicosis is a disease of the pulmonary system, which predominantly occurs in the lung parenchyma. It is caused by inhalation of silica dust or silicon dioxide into the lungs. Silicosis is especially common in workers who work in the sandstone industry, drilling tunnels, in ceramic manufacturing, etc. ¹. Silica dust inhaled into the lungs is deposited in the lung parenchyma, resulting in chronic inflammation and lung fibrosis. Many patients who have fibrosis in lungs will eventually suffer from symptoms of respiratory distress toward the end of their lives ^{1, 2}. Early detection and diagnosis can reduce these problems by limiting patients’ exposure as soon as possible. In some developing countries, such as Thailand, silicosis remains an important occupation-related lung problem ³.Stone gliding, stone sculpture, stone mining, etc. are usually are small or medium enterprisesand most are part of the informal sector. This makes environmental exposure difficult to monitor and control, resulting the need for a medical surveillance program, the aim of which is to discover cases of early-stage pneumoconiosis ⁴.

In Thailand, neither active or passive surveillance systems have performed well ^{5, 6}. The incidence rate of silicosis has increased yearly, whereas in other countries (especially developed countries) it has been in decline ^{7, 8}. This may be due to mainly to better environmental control in developed countries ⁹. In Thailand, the quantity of silica dust in the workplace is not regulated, particularly in the informal sector. In 2012, there were 214 patients admitted to the hospital because of respiratory symptoms caused by silicosis, and Nakhon Ratchasima province had the most cases of any province ⁵.

Thus, it is crucial that silicosis cases are detected early. Researchers have been working to improve the health surveillance system and the tools used therein, but Thailand is still less developed than other countries in this regard. The medical surveillance program in Thailand utilizes a questionnaire about history of silica dust exposure, chest radiographs, and spirometry to detect possible cases of pneumoconiosis ^{6, 10}. Because of their lower cost, chest radiographs play an important role in screening, even though they are less sensitive and specific than computed tomography ¹¹. In Thailand there are not enough B-readers to read chest radiographs for the medical surveillance program ¹². The implementation of an educational course to improve the abilities of occupational health doctors to be able to early detect pneumoconiosis from radiographs would be useful. Previous studies have examined the proficiency of physicians in various fields in detecting pneumoconiosis on radiographs and found that their skills improved after a short training course ¹³. However, those studies did not include occupational health doctors. In addition, the radiographs used for the training did not include cases of early-stage pneumoconiosis. Thus, we aimed to examine the sensitivity and specificity of occupational health doctors’ reading of early-stage pneumoconiosis radiographs in order to improve Thailand’s silicosis surveillance system.

2. Methods

Descriptive study.

The target population consisted of occupational health doctors who were trained as part of the 25^th Short Course in Occupational Medicine at Nopparat Ratchathani hospital from June 3^rd – July 31^st, 2015. Before the application of the exclusion criteria, the study population consisted of 61 OHDs. The samples were then calculated using the estimation of finite population mean equation. The final sample size was 33 OHDs. A simple random sampling technique using Microsoft Excel 2010 was used to select the samples (Figure 1).

We selected standard 14x17 inch chest films recommended in “Development of a disease surveillance system for silicosis and respiratory disorders in stone carving workers exposed to inorganic dust at Sikhiu district in Nakhon Ratchasima province” by K. Silanan (personal communication). There were 315 films of patients who had a history of silica exposure while working as stone carvers. These were films that were agreed upon by three B-readers to be “good” or “acceptable, with no technical defect. Sixty-seven of these test films preserved confidentiality with regard to patient history. Out of these, 42 were of normal (0/0) profusion, fiver had a profusion of 0/1, 10 had a Profusion 1/0, five had a profusion of 1/1, and five had a profusion of > 1/1.

The main outcomes are mean sensitivity and mean specificity of the readers

Sensitivity = the proportion of true positive (both examiner and B-reader; profusion 0/1 and over) for all positive films.

Specificity = the proportion of true negative (both examiner and B-reader; normal 0/0 films) for all negative films.

We graded the sensitivity and specificity according to three categories. Scores of 70.0% and above were determined to be good, 50.0-69.9 were determined to be acceptable, and lower than 50.0were determined to be poor. We also analyzed the accuracy of the reading by categorizing the films into four groups within each subcategory based on descriptive statistics (frequency, mean with SD or medium with IQR, 95% confidence interval).

Correctness = the profusion readers’ answers that matched those of the B-readers or were within one subcategory of the B-readers’ answers. The details are shown in Table 1.

We coordinated with Nopparat Ratchathani Hospital and the Central Chest Institute of Thailand (CCIT) in collecting data on the July 3^rd, 2015. Participants were trained in the basics of radiograph ILO reading for two days by experts from CCIT as part of the 25^th Occupational Medicine Short Course. Before the data were collected, the experts from CCIT explained the basics of radiograph ILO reading. Afterward, a two-phase test was administered, with the first phase consisting of 39 films and second consisting of 28 films. We allowed for two minutes reading each film and recording the results on a standard answer sheet. This process was conducted at CCIT in a room that has no direct light from outside using a standard view box that had been calibrated following the 2011 ILO Guidelines. Participants were anonymized and the completed answer sheets were sent to one of authors for sensitivity and specificity analysis.

We analyzed all data using SPSS version 19. General characteristics (age, work experience) were analyzed using descriptive statistics (frequency, mean with SD, or medium with IQR, 95% confidence interval). Sensitivity and specificity were determined when cut-point was profusion 0/1 and 1/0 by comparing with B-reader’s answer.

All authors have passed a research ethics training course. This research was approved by the office of the Khon Kaen University Ethics Committee in Human Research. All participants were informed before joining to the research that the results would not be reported to public and joining this research would not affect their future career or study.

3. Results

A total of 33 OHDs with various backgrounds participated in the study, 21 of whom were men (63.6%) and 12 of whom were women (36.4%). Their median work experience was 5 years (IQR10), most (44.1%) having three to five years’ experience and 41.2% having more than five years’ experience. Five of the OHDs (21.74%) who participated had experience diagnosing silicosis. Most of the participants were between 25-34 years old (22 OHDs, 66.7%; Table 2).

We found the median sensitivity to detection on 0/1 films and above to be 88.0 with an IQR of 10.3, while the mean sensitivity on 1/0 cut-point films was slightly higher at 90.0 with an IQR of 10.3 (Table 3). The proportion of participants who were categorized as exhibiting “good” sensitivity was 93.9% in both cut-point of disease (Figure 2).

Regardingthe specificity of the reading, the average specificity in detection using 0/1 film was 43.3% (SD21.1), while this number was higher (47.0%; SD 20.9) with a cut-point was1/0 (Table 3). There were four physicians who were categorized as having good specificity using profusion 0/1 cut-point, and three had good specificity when the cut-point was 1/0. However, most had poor specificity at both 0/1 (21 OHDs, 63.6%) and 1/0 (17 OHDs, 51.5%; Figure 2)

In term of correctness, when grouping the films in each subcategory, we found that the average percentage of correct answers was highest at profusion1/1 and>1/1 with 88.3% and 61.2%, respectively. About half of the answers were correct when using films from other subcategories (Table 4).

4. Discussion

This study was aimed at measuring the sensitivity and specificity of readers of early-stage abnormal chest films. There were a total 33 physicians who participated in this study. The median sensitivity of profusion 0/1 cut-point was 92.0% and profusion 1/0 cut-point was 90.0%. The percentage of physicians who were categorized as having good sensitivity was 93.9% at both cut points. However, there were few physicians who were categorized as having good specificity at both cut points (only four at 0/1 and three at 1/0).

Considered in the high sensitivity reading, they demonstrated that occupational health doctors can detect abnormal chest films. In order to fulfill the early detection requirement of the surveillance system for the population at risk, there should be high sensitivity in identifying positive cases for further investigation. The results of this study show that OHDs can detect abnormalities despite low specificity.

In the surveillance program there is further evaluation from B-reader to confirm the abnormality thus we can accept the false positive.

There were a large number of early-stage of abnormal chest films included in the tests that were difficult to differentiate from normal 0/0 films, which accounts for the low specificity. Accordingly, most of the participants answered that the normal 0/0 films were abnormal. Moreover, the films that were selected to be used as test films were originally digital format, and there was too much detail due to converting to analog, as well as overexposure, affecting the judgment of the readers. There were only 67 test films, but the participants were given a total time of 140 minutes to finish. A study conducted by C.S. Lee on the cognitive and behavioral factors that cause errors in reading radiographs found that most radiologists were concerned about missing some abnormality in the films, leading to more false positives ¹⁵.

The sensitivity of readers in determining the level of the disease at a profusion of 0/1 was close to that at a profusion of 1/0. Hnizdo conducted a study that aimed to estimate the sensitivity and specificity of chest radiograph readings by three B-readers compared to autopsy screening for silicosis in 557 subjects who had worked in goldmines ¹⁶. Founding that when the disease was at a profusion of 1/1, the sensitivity was low (23.6-39.3%), and many of the abnormal pathological patients were not categorized as being positive for the disease. In addition, the sensitivity increased when the level of profusion was decreased to 0/1 in patients who were exposed to high levels of silica dust. Therefore, due to the lack of environmental controls in the informal sector, profusion of 0/1 should be an appropriate cut point for the disease to be used in the Nakhon Ratchasima surveillance system.

This study has some limitations. Despite of prudently design of the study and the samples, the chest films that were used in the test were radiographs left over from a previous project. The authors were not being able to change them or improve their quality. However, the authors chose those with the best quality possible from the pool of available films.

5. Conclusion

This study showed that occupational health doctors were able to interpret chest radiographs of workers who had a history of exposure to silica dust with good sensitivity compared with expert NIOSH B-readers after having undergone a short course on basic ILO classification in pneumoconiosis radiographs. There are many patients in Thailand whose chest films are awaiting interpretation, due to a lack of NIOSH B-readers. Developing OHDs’ reading skills will be useful for the national surveillance system.

6. Recommendations

1. The results of this study showed that OHDs can detect abnormal chest radiographs with a high level of sensitivity. A program training OHDs who work in endemic areas for silicosis in the reading of pneumoconiosis radiographs (such as a short course on basic ILO training) should be developed.

2. This study proposed to the surveillance system requirements of providing physicians who can detect large numbers radiographs but cannot represent the appropriate periodic time of x-ray performed to workers, further study about the earliest time of changing in radiograph can fill this gap.

3. Although chest radiographs are important to the silicosis surveillance system, they are part of secondary prevention. A better way to prevent the disease is by protecting workers from exposure to silica dust.

Acknowledgements

The authors would like to thank CCIT for assisting in terms of the provision of staff, view boxes, space, and logistics in the data collection. We would also like thank the Faculty of Medicine’s Department of Community Medicine Division of Occupational Medicine for their support.

Statement of Competing Interests

The author has no competing interests.

References