Validation of A Rapid Tuberculosis PCR Assay for Detection of MDR-TB Patients In Sohag University Hospital
Ahmed Hassan1, Mona Fattouh1,, Iman Atteya1, Hamdi Mohammadeen2, Hydi Ahmed3
1Departments of Medical Microbiology and Immunology, Sohag Faculty of Medicine, Sohag University
2Departments of Chest Diseases, Sohag Faculty of Medicine, Sohag University
3Departments of Clinical pathology, Sohag Faculty of Medicine, Sohag University
Abstract
The aim of this work was to evaluate the value of polymerase chain reaction (PCR) in rapid detection of resistance to rifampin and isoniazid in TB patients. Sixty sputum specimens were collected from MDR-TB patients. In addition to 10 sputum samples from TB patients who were responders to treatment by first line anti-TB drugs. The isolates were identified, cultured and tested for detection of multidrug resistance mutations in ropB gene coding for RIF resistance and katG gene coding for INH resistance using PCR; and were compared to Lowenstein Jensen (LJ) culture drug susceptibility testing (DST) using the Proportion method as the ‘gold standard’. By using proportion method on LJ all 60 sputum samples of MDR patients were resistant to both INH and RIF. By PCR; rpoB gene coding for RIF resistance was detected in 59 clinical isolates. The sensitivity and specificity of mutation of rpoB gene for the 60 specimens were 98.3% and 100% respectively. katG gene coding for INH resistance was detected in 56 clinical isolates. The sensitivity and specificity of mutation of KatG gene for the 60 specimens were 93.3 % and 100 % respectively. The 10 sputum samples of TB patients who were responders to treatment were susceptible to first-line anti-TB drugs on LJ culture DST and no MDR genes were detected by PCR. Our study demonstrated that PCR analysis was rapid, sensitive and specific approach for detection of rpoB and katG genes within 48 hours, which is more rapid than drug susceptibility testing after the culture.
At a glance: Figures
Keywords: Multidrug-resistant, Mycobacterium tuberculosis, polymerase chain reaction (PCR), rpoB, katG
Journal of Applied & Environmental Microbiology, 2014 2 (3),
pp 65-69.
DOI: 10.12691/jaem-2-3-1
Received January 21, 2014; Revised March 03, 2014; Accepted March 18, 2014
Copyright © 2013 Science and Education Publishing. All Rights Reserved.Cite this article:
- Hassan, Ahmed, et al. "Validation of A Rapid Tuberculosis PCR Assay for Detection of MDR-TB Patients In Sohag University Hospital." Journal of Applied & Environmental Microbiology 2.3 (2014): 65-69.
- Hassan, A. , Fattouh, M. , Atteya, I. , Mohammadeen, H. , & Ahmed, H. (2014). Validation of A Rapid Tuberculosis PCR Assay for Detection of MDR-TB Patients In Sohag University Hospital. Journal of Applied & Environmental Microbiology, 2(3), 65-69.
- Hassan, Ahmed, Mona Fattouh, Iman Atteya, Hamdi Mohammadeen, and Hydi Ahmed. "Validation of A Rapid Tuberculosis PCR Assay for Detection of MDR-TB Patients In Sohag University Hospital." Journal of Applied & Environmental Microbiology 2, no. 3 (2014): 65-69.
Import into BibTeX | Import into EndNote | Import into RefMan | Import into RefWorks |
1. Introduction
In 1882, Robert Koch made the landmark discovery that Tuberculosis (TB) is caused by an infectious agent, Mycobacterium tuberculosis (Koch, 1882). The WHO estimates that one-third of the world’s population is infected with Mycobacterium tuberculosis, 9 million new cases of active TB and 2-3 million deaths occur annually; 95% of them in the developing countries (WHO, 2009). According to the WHO estimates, tuberculosis is considered to be the 3rd most important communicable disease problem in Egypt, after schistosomiasis and hepatitis C (WHO, 2009). Multidrug-resistant TB (MDR-TB), defined as culture confirmed resistance to rifampicin and isoniazid, comprises 3% of new and retreatment TB cases in Africa. The presence of mutations in both rpoB and katG confers resistance to the two most powerful 1st line drugs, Rifampicin & Isoniazid respectively, resulting in Mycobacteria tuberculosis that is Multi-drug resistant (WHO, 2011). There are estimated half a million cases of MDR-TB worldwide (Loddenkemper and Hauer, 2010). The mechanism of resistance to RIF involves missense mutations, small deletions or insertions in the rpoB gene encoding the β-subunit of RNA polymerase. Studies from diverse countries have shown that 95–96% of all RIF resistant isolates have mutations within an 81 bp ‘core region’ of rpoB (Yue et al., 2003). In contrast, but perhaps not unexpectedly given its highly complex mechanism of action, the mutations causing INH resistance are located in more than two genes and regions it has been reported that 50–95% of INH-resistant strains contain mutations in codon 315 of the katG gene encoding the catalase-peroxidase (Mokrousov et al., 2002) and 20–35% harbor mutations in the inhA promoter region (Piatek et al., 2000). Molecular (genotypic) DST methods detect resistance associated mutations in target genes of M. tuberculosis, provide results within 1- 2 days and can be performed directly on smear-positive sputum and other clinical samples (Hillemann et al.,2007). Traditional drug susceptibility testing (DST) takes six to eight weeks. Even with the aid of liquid cultures and radiometric methods, the bacteriological culture and DST still take two to three weeks (Loddenkemper and Hauer, 2010). The diagnosis of drug resistant TB (DR-TB) is hampered by the absence of effective and affordable rapid diagnostic techniques for drug sensitivity. In the present study; we tried to evaluate both phenotypic and molecular methods to develop rapid, reliable and accurate methods to detect drug resistance in M. tuberculosis to be applied in high-incidence areas.
2. Patients and Methods
Our study was carried out in Sohag University Hospital during the period from December 2011 to December 2012. Sixty patients with positive morning sputum-smear of M. tuberculosis by acid-fast stain were included in the study; 42 males (70 %) and 18 (30 %) females. Their age range was from 25 to 60 years. All patients were subjected to complete clinical assessment and routine laboratory investigations including; ESR, and CRP. Chest X-rays were done to all patients. Patients had respiratory symptoms and radiological data suggestive of pulmonary tuberculosis, and they did not respond to treatment by first-line anti-TB agents; RIF and INH as there were progressive clinical and / or radiologic findings while on TB therapy and all of them were retreatment pulmonary TB cases. They were admitted to Chest Diseases department in Sohag University Hospital and The Abbasia Chest Hospital, Cairo, Egypt, which is a referral hospital for the treatment of DR-TB patients; they were included to boost the number of MDR-TB specimens in the study. In addition to 10 tuberculous patients responded to treatment by rifampicin and isoniazid first line anti-TB drugs; as there was clinical and / or radiologic findings of improvement while on TB therapy; 7 males (70 %) and 3 females (30%) in the same age group. They were clinically diagnosed as primary pulmonary tuberculosis cases. All the cases included in the study were subjected to sputum smear examination, mycobacterial culture on LJ media, drug-susceptibility testing (DST) and PCR for detection of ropB gene and katG gene coding for multidrug resistance in Mycobacterium tuberculosis.
2.1. SpecimensSeventy acid-fast bacillus (AFB) smear positive sputum specimens enrolled into our study were subjected to decontamination by the N-acetyl-L- cysteine–sodium citrate–NaOH (NALC-NaOH) method (Kent and Kubica, 1985). After centrifugation, sediment was inoculated on Lowenstein-Jensen (LJ) medium and incubated at 37°C for 4 to 8 weeks. The identification of M. tuberculosis complex strains was based on conventional methods, including the optimum temperature and time for growth, colony morphology and Ziehl-Neelsen staining (Sahm and Weissfeld, 2002). Drug susceptibility testing (DST) was done for all specimens using the proportion method.
Drug susceptibility testing (DST):
DST for isoniazid (0.2 μg/ml) and rifampicin (40 μg/ml) was performed using LJ media following the proportion method (PM) (Chanda et al., 2010). The proportion method determines the percentage of growth (number of colonies) of a defined inoculum on a drug-free control medium versus growth on culture media containing the critical concentration of an anti-TB drug. Inoculated colonies were prepared from cultures on LJ media. A few colonies were emulsified in 2 ml of sterile distilled water, where the turbidity level was adjusted to McFarland 1 standard. The suspension was homogenized on a vortex mixer for 1 minute and left to stand for at least 10 minutes to reduce aerosol production in the subsequent manipulations. Standard dilutions of 10–2 and 10–4 were prepared. 0.2 ml aliquot of the 10–2 suspension was used to inoculate all drug-containing LJ tubes, while drug-free control LJ tubes were inoculated with 10–2 and 10–4 suspensions. The tubes were incubated overnight at 37oC in a slanted position with loosened caps. After overnight incubation, the screw caps were tightened and the tubes were further incubated at the same temperature in an upright position. The initial reading of the tubes was performed on the day 28th of incubation. Strains showing susceptibility were again incubated and examined after 6 weeks of incubation before declaring as sensitive. The percentage of resistance (R) was calculated as the ratio between the numbers of colonies on the drug containing media to those on the control medium. For INH and RIF, if R >1%, the isolate was considered resistant (Sethi et al., 2004).
![]() |
Molecular detection of MDR-TB genes by PCR:
Multidrug resistance was evaluated by PCR for detection of mutations in the 81-bp ‘core region’ of the ropB gene coding for RIF resistance and in the codon 315 of the katG gene coding for INH resistance in Mycobacterium tuberculosis.
a) Sample treatment & DNA extraction:
Genomic DNA of the study isolates was extracted for molecular analysis from LJ slopes culture according to the manufacturer's instructions by the use of QIAamp DNA Kits (QIAGEN GmbH, Lot No 127155347).
Primers (Yang et al., 2011):
Oligonucleotide primer sequences used (synthesized by metabion international AG, Germany) were as follows: The 2 oligonucleotide primers A1 and A2 resulting in the amplification of 180 -bp PCR fragments for detection of ropB gene (Rifampicin resistance gene); Primer A1: 5′- GCA TGT CGC GGA TGG AGC -3′. Primer A2: 5′- ACG CTC ACG TGA CAG ACC -3′. The 2 oligonucleotide primers B1 and B2 resulting in the amplification of 230 -bp PCR fragments for detection of katG gene (Isoniazid resistance gene); Primer B1: 5′- GCG GCG GTC GAC ATT -3′. Primer B2: 5′- CTC GAG GAA ACT GTT GTC CC -3′.
b) PCR (Yang et al., 2011):
In a sterile thermal cycler 0.5ml tube, 25 μl PCR reaction mix containing 12.5 μl PCR master mix, 4.5 μl sterile water, 2 μl of each primer and 4 μl of the DNA sample was added. In each set of experiments, a negative control was included. The negative control was prepared by replacing the DNA template with PCR grade water. Amplification of the sample has been performed using a Biometra thermal cycler (T-Gradient PCR system version 4 - Biometra Whatman, Goettingen, Germany). The PCR amplification cycling profile of rpoB gene was 3 min of denaturation at 94°C (1 cycle), followed by 35 cycles of amplification; each of heat denaturation at 94 °C for 60 s, primer annealing at 68 °C for 60 s, and DNA extension at 72 °C for 60 s then one cycle for final extension at 72°C for 5 minutes. The PCR amplification cycling profile of katG gene was 3 min of denaturation at 94°C (1 cycle), followed by 35 cycles of amplification; each of heat denaturation at 94 °C for 60 s, primer annealing at 60 °C for 60 s, and DNA extension at 72 °C for 60 s then one cycle for final extension at 72°C for 10 minutes. The amplified DNA was electrophoresed using 2% agarose gel electrophoresis (Electrophoresis power supply-Biometra Whatman company, Goettingen, Germany), stained with ethidium bromide, and visualized under UV transillumination and photographed.
2.2. Statistical AnalysisQualitative (categorical) data were represented by the number and percent (%).The Chi-square test (X2 value) was used to assess the association between categorical data.
3. Results
The present study was carried out in Sohag University Hospital during the period from December 2011 to December 2012. Sixty MDR-TB patients who were resistant to treatment by INH and RIF; were included in the study; 42 males (70%) and 18 (30%) females. In addition to 10 TB patients who were responders to treatment by INH and RIF; 7 males (70%) and 3 (30%) females. Morning sputum samples were collected from all patients. All positive morning sputum-smears of M. tuberculosis by acid-fast stain resulted in positive growth on LJ media.
3.1. Drug Susceptibility TestingBased on the phenotypic drug susceptibility testing which was performed using LJ media following the proportion method:-
• All the 60 sputum samples of the patients who did not respond to treatment by first-line anti-TB drugs; were resistant to both INH and RIF.
• The 10 M. tuberculosis isolates from TB patients who were responders to treatment by first-line anti-TB therapy were susceptible to both INH and RIF.
3.2. PCR ResultsIn 60 sputum samples with MDR-TB, detection of genes coding for multidrug resistance in Mycobacterium tuberculosis by PCR showing that:
• rpoB gene coding for RIF resistance was detected in 59 clinical isolates from 60 cases who were suffering from RIF DR-TB. The sensitivity and specificity of mutation of rpoB gene for the 60 specimens were 98.3% and 100% respectively. There was no rpoB gene mutations observed by PCR in the 10 clinical isolates of TB patients who were responders to first-line anti-TB drugs.
• katG gene coding for INH resistance was detected in 56 clinical isolates from 60 cases who were suffering from INH DR-TB.The sensitivity and specificity of mutation of KatG gene for the 60 specimens were 93.3 % and 100 % respectively. There was no katG gene mutations observed by PCR in the 10 clinical isolates of TB patients who were responders to first-line anti-TB drugs (Table 1).
Table 1. Comparison between the resistance rates to individual drugs as detected by the DST (PM) and PCR testing of sputum samples of TB patients
• The absence of gene mutations of both rpoB gene and katG gene in the 10 patients who were responders to treatment by INH and RIF, indicated high sensitivity and specificity of PCR which was 100% for both.
4. Discussion
Of the 8.8 million new tuberculosis cases each year, 440,000 are forms of the disease that are multidrug-resistant (WHO, 2011), meaning they cannot be treated with the two primary antibiotics used to treat TB; INH and RIF. The emergence of multidrug-resistant TB (MDR-TB) poses an important threat to TB control as MDR-TB leads to higher mortality and treatment failure rates, and increases periods of transmissibility of the disease (Schneider and Castro, 2003). The success of MDR TB treatment depends upon how quickly a case of TB is identified as drug resistant and whether an effective drug therapy is available. Therefore, we sought that presence of a rapid sensitive test for the diagnosis of MDR-TB among new cases of sputum-positive pulmonary TB is mandatory. Using standardized DST procedures with conventional methods; eight to 12 weeks are required to identify drug-resistant microorganisms on solid media (Pai et al., 2009). In the present study; we tried to evaluate both phenotypic and genotypic methods for detection of drug resistance in M. tuberculosis. We evaluated 70 sputum specimens in our laboratory from December 2011 to December 2012 with culture used as the gold standard. In our study we evaluated the DST using the proportion method which allows precise determination of the proportion of resistant mutants to a certain drug. All the 60 sputum samples of the patients who did not respond to treatment by first-line anti-TB agents were resistant to both INH and RIF. The 10 M. tuberculosis isolates from TB patients who were responders to treatment by first-line anti-TB therapy were susceptible to both INH and RIF. While relatively inexpensive and undemanding of sophisticated equipment, results took weeks. This delayed identification of drug resistance will result in inadequate treatment, which may generate additional drug resistance and continued transmission in the community (Migliori et al., 2008). At an academic institution with a well established DOT infrastructure in New Delhi, suggesting that many patients are dying while awaiting diagnosis (Singla et al., 2009). Resistance of M.tuberculosis to anti-TB drugs is caused by chromosomal mutations in genes encoding drug targets. Rifampin (RIF); one of the principal first-line anti-tuberculosis drugs, inhibits DNA-directed RNA synthesis of M. tuberculosis by binding to the subunit of RNA polymerase. Mutations in the rpoB gene, which codes for the beta subunit of the RNA polymerase, have been shown to be strongly associated with RIF-resistant phenotypes in multiple study populations (Huang et al., 2002). rpoB mutations are more likely segregated in an 81-bp region called the RIF resistance-determining region (RRDR). (Kocagoz et al., 2005). In our study, by PCR we detected ropB gene coding for RIF resistance in 59 cases of the 60 cases of the MDR-TB patients. The sensitivity and specificity of mutation of rpoB gene for the 60 specimens were 98.3% and 100% respectively. Isoniazid (INH) is one of the most effective antimycobacterial agents available for the treatment of TB, which is a prodrug that is converted to its active form in vivo by the katG-encoded M. tuberculosis catalase-peroxidase. Resistance to INH is predominantly associated with mutations in katG particularly at codon 315 (Lavender et al., 2005). In our study, by PCR we detected katG gene coding for INH resistance in 56 cases of the 60 cases of the MDR-TB patients. The sensitivity and specificity of mutation of KatG gene for the 60 specimens were 93.3 % and 100 % respectively. This can be explained by the fact that resistance to INH is located in more than two genes and regions as it has been reported that 50–95% of INH-resistant strains contain mutations in codon 315 of the katG gene encoding the catalase-peroxidase (Mokrousov et al., 2002) and 20–35% harbor mutations in the inhA promoter region (Piatek et al., 2000). A major limitation to molecular genetic detection of drug resistance by any technique is that molecular genetic tests detect only known mutations. Because not all mutations conferring resistance to anti-TB drugs are known; identification of a resistance-associated mutation can be informative, but lack of a mutation in the target sequence must be interpreted with caution (Yang et al., 2005). The absence of gene mutations of both rpoB gene and katG gene in the 10 patients who responded to treatment by first line anti-TB drugs, indicate the high sensitivity and specificity of PCR which was 100% for both. The results obtained on the cultured isolates were promising, but the assay was time-consuming in comparison to the turnaround time by the PCR assay. As, with the application of the PCR assay, the turnaround time to results was 48 hours (times for DNA extraction and PCR amplification, agarose gel electrophoresis), compared to 4 to 6 weeks in the case of the phenotypic culture system. This short turnaround time for diagnosis of multidrug- resistant Mycobacterium tuberculosis plays an important role in guiding standardized treatment regimen.
5. Summary and Conclusion
This study has demonstrated that the PCR Assay for detection of MDR-TB patients has the potential to make a substantial contribution to the early diagnosis of MDR-TB in Sohag University Hospital, and probably elsewhere in the country, and this will result in early isolation and appropriate treatment of many patients and so; will lead to fewer deaths and less transmission of the disease. This test strongly facilitates adequate treatment of MDR-TB patients, long before the results of conventional DST are available. The application of the molecular test directly to clinical material with sufficient bacteria will further speed up the turnaround time for the rapid diagnosis of MDR-TB.
References
[1] | Chanda M, Allan C, and Innocent C. Low occurrence of tuberculosis drug resistance among pulmonary tuberculosis patients from an urban setting, with a long-running DOTS program in Zambia.2010. Tuberculosis Research and Treatment, vol. 2010, Article ID 938178, 6 pages, 2010. | ||
![]() | |||
[2] | Hillemann D, Gerdes RS, and Richter E. Evaluation of Genotype MTBDR plus assay for rifampin and isoniazid susceptibility testing of Mycobacterium tuberculosis strains and clinical specimens. J Clin Microbiol 2007;45:2635e40. | ||
![]() | |||
[3] | Huang H, Jin Q, Ma Y, Chen X, and Zhuang Y. Characterization of rpoB mutations in rifampicin-resistant Mycobacterium tuberculosis isolated in China. Tuberculosis (Edinburgh).2002, 82(2-3): 79-83. | ||
![]() | CrossRef | ||
[4] | Kent P T and Kubica G P. Public health mycobacteriology: a guide for the level III laboratory, p. 64-68. U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control, Atlanta, GA.1985. | ||
![]() | |||
[5] | Kocagoz T, Saribas Z, and Alp A. Rapid determination of rifampin resistance in clinical isolates of Mycobacterium tuberculosis by real-time PCR. J. Clin. Microbiol.2005, 43(12): 6015-60. | ||
![]() | CrossRef | ||
[6] | Koch R. Die Aetiologie der Tuberculosis. Berl. Klinische Wochenschr.1882, 100(12): 539. | ||
![]() | |||
[7] | Lavender C, Globan M, Sievers A, Jacobe HB, and Fyfe J. Molecular characterization isoniazid-resistant Mycobacterium tuberculosis isolates collected in Australia. Antimicrob. Agents. Chemother.2005, 49(10): 4068-4074. | ||
![]() | CrossRef | ||
[8] | Loddenkemper R, and Hauer B. Drug-Resistant Tuberculosis. Dtsch. Arztebl. Int.2010, 107(1-2): 10-19. | ||
![]() | |||
[9] | Migliori GB, Matteelli A, Cirillo D, and Pai M. Diagnosis of multidrug-resistant tuberculosis and extensively drug-resistant tuberculosis: Current standards and challenges. Can J Infect Dis Med Microbiol.2008; 19(2):169-172. | ||
![]() | |||
[10] | Mokrousov I, Narvskaya O, Otten T, Limeschenko E, Steklova L and Vyshnevskiy B. High prevalence of KatG Ser315Thr substitution among isoniazid-resistant Mycobacterium tuberculosis clinical isolates from northwestern Russia, 1996 to 2001. Antimicrob Agents Chemother.2002; 46, 1417-1424. | ||
![]() | CrossRef | ||
[11] | Pai M, Minion J, and Sohn H. Novel and improved technologies for tuberculosis diagnosis: progress and challenges. Clin Chest Med.2009; ![]() | ||
![]() | CrossRef | ||
[12] | Paramasivan CN and Venkataraman P. Drug resistance in 5. tuberculosis in India. Indian J Med Res.2004; 120 : 377-86. | ||
![]() | |||
[13] | Piatek A S, Telenti A, Murray M R, El-Hajj H, Jacobs W R, Kramer F R and Alland D. Genotypic analysis of Mycobacterium tuberculosis in two distinct populations using molecular beacons: implications for rapid susceptibility testing. Antimicrob Agents Chemother.2002; 44, 103-110. | ||
![]() | CrossRef | ||
[14] | Sahm D, and Weissfeld F. Bailey and Scott’s diagnostic microbiology. (11th ed.). St. Louis, MO7 Mosby, pp 3-29.2002. | ||
![]() | |||
[15] | Schneider E, and Castro KG. Tuberculosis trends in the United States, 1992-2001. Tuberculosis (Edinb) 83:21-29.2003. | ||
![]() | |||
[16] | Sethi S, Sharma S, and Sharma SK. Drug susceptibility of Mycobacterium tuberculosis to primary antitubercular drugs by nitrate reductase assay. Indian J Med Res.2004; 120:468-471. | ||
![]() | |||
[17] | Sharma SK and Mohan A. Multidrug-resistant tuberculosis: a 2. menace that threatens to destabilize tuberculosis control. Chest.2006; 130: 261-72. | ||
![]() | |||
[18] | Singla R, Sarin R, and Khalid UK. Seven-year DOTS-Plus pilot experience in India: results, constraints and issues. Int J Tuberc Lung Dis.2009; 13: 976-81. | ||
![]() | |||
[19] | World Health Organization. WHO Report 2011: Global Tuberculosis Control. (2011) Available: https://www.who.int/tb/publications/global_report/ en/. Accessed 2012 Oct 20. | ||
![]() | |||
[20] | World Health Organization. Country cooperation strategy at a glance. Available at: www.who.int / country focus / cooperation_strategy /ccsbrief_egy_en.pdf.2009. | ||
![]() | |||
[21] | World Health Organization. Global Tuberculosis Control - A short update to the 2009 report. | ||
![]() | |||
[22] | World Health organization. Tuberculosis country profiles.2011. Available at : www.who.int/tb/country/data/profile. | ||
![]() | |||
[23] | Yang S, Zhong M, Zhang Y and Wang Y. Rapid detection of rpoB and katG genes from the sputum of multidrug-resistant Mycobacterium tuberculosis by polymerase chain reaction (PCR)-direct sequencing analysis. African Journal of Microbiology Research. 2011; 5(26), 4519-4523. | ||
![]() | |||
[24] | Yang Z, Durmaz R, Yang D, Gunal S, Zhang L, Foxman B, Sanic A, and Marrs CF. Simultaneous detection of isoniazid, rifampin, and ethambutol resistance of Mycobacterium tuberculosis by a single multiplex allele-specific polymerase chain reaction (PCR) assay. Diagnostic Microbiology and Infectious Diseas;.2005, 53; 201-208. | ||
![]() | CrossRef | ||
[25] | Yue J, Shi W, Xie J, Li Y, Zeng E and Wang H. Mutations in the rpoB gene of multidrug-resistant Mycobacterium tuberculosis isolates from China. J Clin Microbiol. 2003;41, 2209-2212. | ||
![]() | CrossRef | ||