Introduction: Traumatic brain injury (TBI) is a serious public health concern in the USA. Each year, TBIs contribute to a total of 52,000 deaths, accounting for 30% of all injury-related deaths and cases of permanent disability. Approximately 5 million survivors of TBI in the USA live with some form of disability [1]. Due to the severity of the brain injuries, some patients will experience a reduced level of consciousness. Early use of pharmacological treatment is fundamental to improve patient outcomes. Background: Providers presently seek guidelines to help them choose the right medication quickly and accurately. A pharmacological toolkit was designed to help providers in the neurology unit to enhance patients’ level of consciousness and improve quality of care following a traumatic brain injury. The theoretical model for this project is the Havelock Theory of Change, which was used to guide the team during the stages of change. The Logic Model was used during the development and implementation of the toolkit. Methods: This project was developed and implemented in the medical neurological unit at the long term acute care and rehabilitation hospital. An educational program was initiated for providers to help them use the evidence based pharmacological toolkit to prescribe neuro-stimulates for patients with TBIs who demonstrated poor levels of consciousness. The Coma Recovery Scale -Revised (CRS-R) scale [2] was used to reassure if effectiveness of the neuropharmacology was feasible to treat poor alertness. The percentage of improvement on CRS-R score of 27.68%, with improvement of 3 points on CRS-R score, is a significant improvement to this QI project. The data was collected over six months and captured the variability reflecting improvement. A post implementation survey was answered by providers to track the benefit and practicability of the tool. Results: The level of satisfaction was high based on the survey response. Conclusion: Use of a neuropharmacological toolkit promises to help providers to treat severe traumatic brain injury in patients experiencing poor level of alertness. In conclusion, following the national brain injury organization recommendations, every institution needs to develop their own guideline to treat TBIs.
Traumatic brain injury (TBI) is a leading cause of disorders of consciousness (DOC), including coma, vegetative state (VS), and minimally conscious state (MCS). 3, 4, 5 Although these disorders are clearly distinct, the recovery of consciousness is recognized as occurring along a clinical continuum that covers a wide range of consciousness gradations, ranging from coma to the restoration of full consciousness. 3 Prolonged DOCs are uncommon but devastating outcomes after severe TBI. Until recently, prognosis and treatment for these patients have been viewed pessimistically. The development of a pharmacological toolkit (See Appendix: Figure 1: Pharmacological Toolkit (See all references about all medications listed used at this Neuro Unit.) is essential to help providers to prescribe medications that optimize patient prognosis and improve efficacy of treatment to enhance consciousness 6.
1.1. Problem DescriptionThe improvement project took place in a neurology medical unit with 26 beds, located in a 180-bed Long Term Acute Care (LTAC) hospital. Most of the medical neuro units’ beds are occupied by adult patients greater than eighteen years of age with severe traumatic brain injuries, and they are classified as disorders of consciousness. Patients at this unit are experiencing poor recovery after acute care stay, and clients are admitted with treatment goals of regaining consciousness, advancing rehabilitation, and as a long-term goal, returning to the community. Patients lacking alertness at this site are unable to progress to more rigorous, advanced rehabilitation, which requires these patients to tolerate more than 3 hours of activity daily. If unable to progress to this level of recovery, they are transitioned to long term care with unclear prognosis of recovery.
The initial plan for treatment of disorders of consciousness (DOC) at this unit consists of eight weeks of aggressive treatment involving neuropharmacology, physical therapy, cognitive analysis and interventions involving a multidisciplinary approach. At this level, providers are challenged to help these patients to make neurological gains such as improved alertness, and physical and cognitive engagement, so that they can progress to a more intensive rehabilitation plan.
Similar to neurology units around the country, this local unit lacked an evidence based neuropharmacologic guideline to benefit providers during clinical decision-making. Before this project was undertaken, local clinicians did not have access to a consistent guideline. This resulted in uncoordinated care, and could result in delayed patient recovery. Patients in the local unit experienced severe complications such as coma, vegetative state, and minimally conscious state. Providers focus on classifying the neurological diagnosis regarding the severity and extensiveness of the brain injuries, using a broad range of etiologies that include hypoxia, varied physical impairments to the brain, ischemic and hemorrhagic stroke, dysautonomia, tumors, and erupted aneurisms. The most common factors contributing to lack of consciousness are the severity of the brain injury, infection, organ failure, seizure, dysautonomia, polymedicine, lack of brain stimulant medication, hydrocephalus, multiple brain hemorrhage, and others. This quality improvement project will provide a neuropharmacological toolkit to guide clinicians when helping patients with post traumatic brain injury to regain consciousness.
Providers at this unit frequently encounter patients with severe post traumatic brain injury, and other central nervous system injuries. If these cases are not properly addressed within a prompt time window, the clinicians will have difficulty in treating impaired consciousness states, which can affect basic vegetative functions (respiration, circulation, sleep-awake cycle) that must be preserved. 4, 7, 8 Therefore, there is growing interest in the possible effect of drugs that act at the level of the central nervous system to promote emergence from disorders of consciousness. 3
Clinicians at this unit know that sporadic cases of dramatic recovery from disorders of consciousness after the administration of various pharmacological agents have been recently supported by several scientific observations 3. These agents include drugs of various classes, which can be grouped into two general categories: central nervous system depressants and central nervous system stimulants. 3, 6, 9 The pharmacological toolkit assists providers in making clinical decisions and prescribing neuro-stimulants and other pharmacological agents to protect the brain and promote alertness in patients with a wide range of disorders of consciousness 6.
Since the hospital’s current Disorder of Consciousness program started four years ago, the use of neuropharmacology to enhance alertness has impacted the quality of care, helped patients to regain alertness, and advanced rehabilitation treatment. In the past four years, fewer patients were discharged to a skilled nursing facility with persistent vegetative stage resulting from severity of the brain injury. The evidence based pharmacology toolkit was implemented at this unit to help providers and team to reduce hospital costs 5. That suggests the importance of an earlier intervention with brain stimulants when treating traumatic brain injury, specifically to treat poor alertness, that has been supported by a recent study 3, 5, 9, 10, 11.
When the medical neuro unit is full to capacity, specifically due to prolonged hospitalization, TBI patients are admitted to different units. The purpose of the medical neuro unit is to enhance alertness and improve goals of care for TBI patients. A multidisciplinary team approach is used to improve the patient’s quality of life as defined by the patient’s and family’s best interests. The team goals are focus on restoring and enhancing neurocognitive functions, which may involve the use of pharmacological approaches in addition to traditional neurohabilitation 9. The main initial goal should be to improve level of consciousness in patients to help patients to advance through the disorder of consciousness program. The evidence based toolkit is a promising tool designed to help providers when prescribing neuropharmacological agents such as: antidepressants, cholinergics, psychostimulants, antipsychotics, and dopaminergic agents. As suggested by Poole 2015 9, this may promote overall quality of care.
1.2. Available KnowledgeThe Centers for Disease Control and Prevention (CDC), based on the National Center for Injury Prevention and Control estimates that 5.3 million U.S. citizens (two percent of the population) live with disability as a result of a traumatic brain injury 1, 10, 11, 12. It is a significant cause of morbidity and mortality and a leading cause of death and disability among young adults. Common causes of TBI include motor vehicle accidents, falls, sports injuries, and violence, and it is recognized increasingly in war zone injuries. 10 In the U.S., approximately two million people will sustain a TBI each year, one quarter of whom will require hospitalization, leading to a conservative direct and indirect cost estimate of $50 billion to $ 100 billion annually 1, 12. A National Institute of Health panel estimates that 2.5 to 6.5 million Americans currently live with traumatic brain injury related disabilities 1, 12. Disabilities arising from traumatic brain injury are classified into four categories: decreased level of consciousness, and neuropsychiatric, neurocognitive, and neurobehavioral sequelae 10.
The literature review suggests that neuropharmacology is an essential tool for medical professionals treating TBI and disorders of consciousness 13. Its suggests that a pharmacological toolkit can help providers to promote patients’ best interest by assisting with prescribing pharmacological agents in a timely manner to enhance level of alertness and improve quality of care. Studies on the use of neuropharmacologic therapies to enhance alertness, on the premise that injury induced derangements in dopaminergic and noradrenergic neurotransmitter systems may be improved through supplementation have had promising results 3, 6, 10, 13. The most used medications to enhance alertness are: amantadine 5, 14, 15, 16, 17, apomorphine 18, zopidem 19, 20, methylphenidate 21, 22, L-dopa/carbidopa, 7, 23 Boswellia Serrata, 24 and bromocriptine. 25 These medications accelerate the pace of functional recovery during active treatment in patients with post-traumatic disorders of consciousness 4, 6, 8.
The benefit of these agents include improving alertness and cognitive impairment in adults with mild to severe traumatic brain injury. The types of intervention include: antidepressants, antipsychotics, dopamine agonists, cholinergics, and psychostimulants, which can modulate the main neurotransmitter systems. These and other agents can play a role in managing the neuropsychiatric, neurocognitive, and neurobehavioral sequelae of injury to the brain. 10
The wide range of factors involved in TBI mandates that providers must have a clear understanding of patients’ degree of impairment, because they must make the appropriate clinical decisions for treatment based on a case by case consideration. This supports the need for a multidisciplinary team approach to improve quality of care. Much of the early care of patients with disorders of consciousness focuses on minimizing presumed adverse influences on recovery and reducing the burden of care while patients remain in these states. 8, 10
The uncertainty involved in treating disorders of consciousness demands that all providers continually maintain an up-to-date clinical routine and analyze patients daily on a case by case basis. Research has demonstrated that survival rates and improvement outcomes for patients are related to the level of severity of the injury as well as adverse factors such as seizure, infection and medication side effects that can complicate patient recovery. The utilization of pharmacological agents that can improve level of consciousness is an emerging national need to improve patient outcomes 5, 8, 20. The multidisciplinary team approach is an essential step to improve quality of care, and the use of neuropharmacology is a favorable step to reach it.
A number of studies have demonstrated promising results when neuropharmacologic therapies are used to enhance consciousness, on the premise that injury induced derangements in dopaminergic and noradrenergic neurotransmitter systems may be improved through supplementation 5, 8. The literature review findings are consistent with the hypothesis that individuals with TBI have altered responsivity to dopamine. 5, 9, 25 This is why it is important to create and implement an evidence based pharmacological toolkit to help medical professionals provide positive outcomes for patients. This toolkit promises to be an effective tool to improve the pace of recovery during the acute rehabilitation stage for patients with post-acute or prolonged post-traumatic disturbances in consciousness. Without proper diagnosis or pharmacological treatment, the patient may remain in persistent vegetative state, may fail to progress with the Disorder of Consciousness Program, and may eventually need to be transferred to a skilled nursing facility. In treating disorders of consciousness, medical professionals must seek to improve the level of consciousness and safeguard the brain 4, 5, 6, 8, 9.
During the study, it was clear that a team approach was strongly encouraged to support the scientific movement nationwide to create a guideline to treat poor levels of consciousness. The Trauma Foundation’s guidelines encourage all U.S. hospitals to create guidelines for treatment of brain injury 11.
Literature searches for this project used the PubMed, Google Scholar, CINAHL, and Medline databases.
1.3. RationaleThe research surrounding disorders of consciousness has grown dramatically over the last decade, with new conceptual developments, greater understanding of natural history and understanding mechanisms, improved assessment techniques, and emerging treatments. 6, 8 This growth in research mandates a proper diagnosis of consciousness level to advance therapy and reduce delays in proper care in order to safeguard the patient’s neurological condition. For instance, there are still error rates as high as 41% when diagnosing the difference between patients in minimally conscious state versus vegetative state 3, 6, 8.
Clinical decision-making needs to be based on evidence based guidelines, and evidence suggests that providers are more likely to translate evidence to practice when they have decision aids to support practice 1, 11, 12. By 2020, is expected that all healthcare agents will base decisions on evidence based practices 1, 11, 12.
The Logic Model (See Appendix: Figure 2: Logic Model Program) provided a theoretical basis for developing and implementing this project helping the microsystem throughout the project phases. The stages of the project are organized according to this methodology. The Havelock Theory guided the microsystem from the contemplation to maintenance stage. The effectiveness of this theory supported providers in prescribing neuropharmacologic agents to enhance alertness under evidence based guidelines.
The Havelock Theory is a linear model that generally resembles Lewin's model, with an emphasis on planning and monitoring. The model takes into account the possibility that people and systems may be resistant to change. Havelock's Theory of Change, consisting of six stages, was used to guide the team. The six stages are: relationship/contemplation, diagnosis, acquire resources for change, select a pathway, establish and accept change, and maintenance and separation. The relationship-based system of change needs to be established during the contemplation phase prior to implementing the project. In that phase, the implementation project was presented to the medical neurologic unit. The team was invited to consider the opportunity to prescribe agents under evidence-based practices. The therapeutic intervention is designated to promote quality of care, promote patients’ best interest, and impact quality of care 26.
During the diagnosis phase, the subject of change must decide whether or not change is needed, or desired by the microsystem 26. The providers and team were educated on how the development and implementation of the toolkit to prescribe neuropharmacologic agents to enhance patient levels of alertness, could support the clinical decisions made by all providers. All providers were presented the CRS-R score 2 reassuring that the toolkit could be useful, where results of the CRS-R indicated improvement in alertness with scores from 4 to 23, proving effectiveness of the toolkit (Table A). One-on-one meetings were scheduled to verify if the providers agreed that the new tool would be helpful to prescribe medication to enhance alertness.
The acquire resources for change phase will begin when providers conclude that a change is necessary 26. A clear understanding of the process of developing solutions begins by gathering as much relevant information as possible about the situation that requires change. The clinicians will start utilizing the toolkit as a clinical resource to enhance patient level of consciousness in treatment of TBI. The fourth stage of Havelock's change theory is when a pathway of change is selected from available options and implemented.
The next phase is the establishment and acceptance of change. Once the change has been put in place, it must be established and accepted. Individuals and organizations are often resistant to change. After change has been accepted, the change process can be declared successful 26. This is defined as the point at which the providers continue to use the toolkit to prescribe neurological agents to enhance alertness.
Maintenance and separation is the final phase. Once the change is successful, the change agent monitors the affected system to make sure that it is successfully maintained 26. The agent applies the survey to track level of acceptance and utilization of the toolkit.
The Logic Model was used to guide the development and implementation of the improvement project to direct its goal. The toolkit was developed with the input of the local microsystem that included a clinical pharmacist, neurologist, site champions, physiatrist, nurse practitioner, and physician (Table B). The pharmacological toolkit was established and presented to the neuro unit. This took place during rounds, multiple meetings, and one-on-one discussion, with the goal of educating staff and providers. Providers can prescribe neuro-stimulants to enhance alertness based on guidelines in the evidence based toolkit.
The logic model was also used to organize the project, set goals to reach positive outcomes. The set goals of the logic model are: short, medium and long term. This model provides guidance for the project during the development through the implementation process. See Appendix Figure 2: Logic Model Program for details on project goals, the evaluation emanates from the outcomes.
1.4. Specific AimThe practice improvement project question is: Will implementation of an evidence-based neuropharmacologic toolkit allow providers to help adult patients with traumatic brain injury improve level of consciousness? This report intends to highlight the urgent need for a guideline to help providers when treating poor level of alertness in patients with post traumatic brain injury, and promote patient and family best interest. Providers experience challenges when treating patients experiencing disorders of consciousness such as coma, vegetative stage, minimum consciousness state, or poor cognition 8, 20. These disorders may block patient recovery, and inhibit opportunities for rehabilitation.
The primary aim of the project is the development and implementation of an evidence-based pharmacological toolkit to help providers enhance level of consciousness in traumatic brain injury patients. The secondary aim of the project is to make the toolkit available to all the providers from the institution, and eventually to publish this project as part of the University of Massachusetts Boston doctorate nurse practitioner program. This evidence-based neuropharmacological toolkit aims to support providers in making clinical decisions in a timely manner.
Context
The local barriers to this project included the lack of existent neuropharmacologic guidelines and timeline constraints due to topic complexity. The unit and facility positively supported the project during the development and implementation. The local medical board committee and the nursing department actively supported the project in a timely manner. The director of the Disorder of Consciousness Program actively supported the project, and no local issues to the project were apparent throughout the project stages.
Intervention
The initial process started by writing the state of science paper, used as a guide to develop and implement a tool to help providers treat poor level of alertness in patients with post traumatic brain injury. This paper was used to select the literature review containing the final eighteen studies of the most commonly used neuropharmacologic drugs used in U.S. and overseas. The project was first approved by the UMB IRB committee as a Quality Improvement project followed by the local facility Quality Improvement committee. Subsequent approval was received from the local facility’s nursing and medical board as well as the capstone committee.
An initial toolkit algorithm was developed, and the team further developed the project with the goal of getting to the implementation phase. As part of the implementation phase, the toolkit was critically evaluated by each provider, and after multiple critiques from each member of the team, the toolkit was finalized. The members of the microsystem included physicians, nurse practitioners, nursing, clinical pharmacist and site champions (Table B). On a weekly basis, the team contributed comments and additions to the toolkit. The project underwent three stages of approval by three different departments: The local facility quality improvement committee, the director of the Disorder of Consciousness program, and the local medical board. After all these steps, the toolkit was officially presented to the unit as part of the unit resource to treat patients with TBIs experiencing poor level of alertness.
The improvement project was implemented from January 2015 to April 2016 (Table C). The neurology unit is the clinical site where the project was developed and implemented. This practice improvement has benefitted the 26 patient-occupied beds under the Disorder of Consciousness Program. Providers answered a post implementation survey to access the benefit and practicability of the tool.
Study of the Intervention
Because the neuropharmacologic toolkit was found effective as an evidence-based tool to guide treatment of post traumatic brain injury patients, specifically those experiencing poor level of alertness, the toolkit was adopted as a unit resource. The project team believed that the toolkit was ideal to the unit and would help providers to make evidence-based clinical decisions. Under the framework of the Logic model (See Appendix: Figure 2: Logic Model Program), the project concluded following the tool development and implementation stage, and the toolkit became part of the unit resource (Table C).
The project received site champion and university chair approval. Site participants actively participated with critiques and suggestions that were considered and implemented during the toolkit implementation stage. A modified CRS-R score was used to measure the efficacy of medications comprising the toolkit, (Table A). Multiple one-on-one meetings with providers and emails exchanged were used to promote education. The toolkit was presented to the nurses as well as the quality improvement and medical committees in several The main goal at this stage was to test the toolkit’s usefulness and to identify necessary changes to improve its utility.
The CRS-R score ranges from 0-23. The purpose of this scale is to assist with differential diagnosis, prognostic assessment and treatment planning in patients with disorders of consciousness. The scale consists of 23 items that comprise six subscales addressing auditory, visual, motor, oromotor, communication and arousal functions. CRS-R subscales are comprised of hierarchically-arranged items associated with brain stem, subcortical and cortical processes 2. The lowest item on each subscale represents reflexive activity while the highest items represent cognitively-mediated behaviors. Scoring was standardized based on the presence or absence of operationally-defined behavioral responses to specific sensory stimuli (See Appendix: Figure 1: Pharmacological Toolkit.) 5 See forms and scale guideline at 2: https://www.coma.ulg.ac.be/images/crs_r.pdf.
After discussing the project with the university chair members and site champion, the clinical pharmacist reviewed each medication in the proposed evidence-based neuropharmacologic toolkit to reassure to all clinicians that they are all safe to be prescribed and are approved by United States Food and Drug Administration. Then the toolkit was reviewed by the Disorder of Consciousness program director and neurologist, followed by a deep review from all the providers from the local unit.
The metrics utilized for this project were the modified CRS-R score and a survey to measure providers’ level of satisfaction with the evidence based pharmacologic toolkit (Table D). The responses from post implementation survey were sent to all providers.
The practice improvement project was finalized following a second survey review after the project was completed. A final electronic toolkit will be used throughout the hospital during the Spring of 2016. A final copy of the neuropharmacologic toolkit is presently part of the unit’s resource to treat patients experiencing poor level of alertness, and by June 2016 it will be part of the hospital resource (Table C).
Measures
The Logic model was used during the development and implementation of the evidence- based neuropharmacologic toolkit, and this model guided this quality improvement project to achieve measureable improvements in the efficiency, effectiveness, performance, accountability, outcomes, and indicators of quality service (See Appendix: Figure 2 Program Logic Model). The model helped build the project clarity to keep providers and team focused on outcomes, using an evidence-based model, and refined design, to collect short, mid and long term goal measurements. Post tool implementation stage, a survey was used to follow the level of applicability with a specific question to determine the provider’s assessment of the usefulness of the toolkit.
During the six-month period of project development through implementation, a chart review, face-to-face discussions with providers and nurses, and a brief patient visit were used to follow providers’ activity when using the toolkit. A random selection of seven cases was chosen to follow a CRS-R response to reassure all providers that the tool appears feasible to treat TBIs until end of the project. This random selection was done to avoid bias. The CRS-R score trending scale was used to follow tool usefulness to support providers while using the level of alertness change following the scores trend (Table A). According (Table A), lower scores represents more severe brain injury. This scale can help providers to follow the clinical decisions and determine how this evidence-based aid may be useful when treating poor level of alertness after a traumatic brain injury. After the post- implementation phase, a survey was sent to verify level of satisfaction from providers as well as identify the areas to improve, and make the toolkit more useful for the providers (Table D). Individual and group meetings were scheduled for qualitative data collection. Meetings with local medical board and the QI committee were also used as part of the qualitative feedback.
Analysis
This practice improvement project used a mix of quantitative and qualitative process measures to assess outcome as described in Table A, C, D, and Figure 2. The quantitative feedback involved chart audits before and after the development and implementation of the evidence-based pharmacological toolkit to assess outcomes.
After the implementation of the toolkit, the CRS-R score was used to measure the majority of the patients’ improvements and to help providers to continue using the evidence-based resource when treating poor level of alertness. A chart review was undertaken of a random selection of seven cases from Sept. 2015 to Feb. 2016 representing the timeline between the development of the neuropharmacologic toolkit to the last data collection in February 2016 (see Table C for project timeline). After discussion with the team, the CRS-R scores were chosen to compare pre and post rather than baseline and post scores because patients had long hospital stays. This could more appropriately reflect patients’ improvement.
The baseline CRS-R for most of the cases ranged from 3-8 upon patient admission to the unit, showing evidence of severe TBI with poor alertness. Patients with baseline initial score ranging from 3-16 as represented in this data, with mean of 9.86±SD 5.58, represents the level of severity of these patients upon admission. After providers started treatment using this evidence-based toolkit, guided to prescribe neurostimulates by patients’ initial CRS-R scores, their scores improved three points, or a mean improvement percentage of 27.68%. This suggested the need for a guideline to help providers offer better care management of traumatic brain injury. These results reassured all providers of the effectiveness of the toolkit and gave providers a high level of satisfaction (Table A and E). The CRS score went up three points post-implementation, or 27.68% improvement. The data were collected over an extended period of time and captured expected variability. Data reflect a trend toward significant improvement. The negative changes represented on Case 1 and Case 7 scores do not reflect a positive trend. These two particular cases had lost most of their brain mass and had very poor prognosis due to the severity of the brain injury. These two cases made significant gains improving alertness despite the low CRS score results. Other factors such as infection and respiratory failure due to tracheostomy interfered with the neurologic progress of both patients, negatively affecting CRS trends.
The CRS-R score scale was used to observe patient improvement during the implementation stage. The small sample size was due to timeline to complete the project. The overall CRS-R score mean rose from 10.84 with a range of 5 to 15 pre-implementation, to 13.39with a range of 5-18 after implementation. The overall CRS score went up 3 points pre-post, which is 27.68% improvement.
After the implementation stage, a survey was applied. The healthcare providers indicated based on their survey response that they felt the study material was well planned and structured, with 100% of satisfaction reported by providers and 86.7% satisfaction from overall responses. Among providers prescribing neuropharmacology, 60% were comfortable and 40% somewhat comfortable with areas needing improvement. The majority, or 87.5% reported that the information in the evidence-based toolkit was effective when prescribing treatment. The providers included physicians, nurse practitioners, psychiatrist, neurologist, and a clinical pharmacist, all with greater than 5 years of experience. Of providers involved in the study, 71% were hospitalists, and 29% were under other specialties (Table B). All providers responded to the survey request, and three participants did not respond after multiple trial to the survey as non-providers (Nurses). Many of the providers delayed responding to the implementation phase of the toolkit due to vacation and scheduling issues, and a total of 11 responded to the post implementation stage survey request.
Ethics/IRB Considerations
Ethical issues: This practice improvement was approved for exemption from review by the University of Massachusetts Boston IRB. The project implemented an evidence-based innovation to improve practice at the local site. The project does not test the safety or efficacy of an intervention, it is not designed to answer a research question or test a hypothesis, and the results are not generalizable beyond the local facility. Thus, the project does not meet the standard definition of human subject research and was deemed exempt from IRB review. The project was approved by the clinical site organization’s compliance office, and it was deemed to meet the criteria for a quality improvement project and subsequently exempt from site IRB. Also, the project underwent review by the site Medical Board and was approved to be used throughout the hospital and will eventually be part of the institution’s resources.
This quality improvement project was executed in a long term acute care inpatient setting. The providers were seeking a guideline to treat poor level of alertness after traumatic brain injury. This project sought to create a tool to assist providers with clinical decision-making when prescribing neuropharmacology.
This quality improvement project was executed from Spring 2014 through April 2016 (Table C). From Spring to Fall of 2014, a systematic review of the literature was completed and the paper was reviewed. To avoid bias in conceptual understanding of the topic, the literature searches for this project used the PubMed, Google Scholar, CINAHL, and Medline.
Capstone project implementation was started in Fall 2015, and the paper was submitted for review in Fall 2016. During November 2014 the capstone proposal was submitted for UMASS IRB approval. In December 2014 the capstone presentation was presented, followed by the committee chair selection and capstone proposal submission. In February 2015, the proposal hearing proceeded, the project underwent clearance from IRB at the local site for approval, and was deemed QI project and approved (Table C).
After clearance from IRB and quality improvement, an educational program at the neurology unit took place, and a pre and post survey monkey was applied to collect data for analysis. Effectiveness of the toolkit was discussed with providers at the neurology unit and team. The post-implementation survey was sent to all 11 participants. Following post-implementation, the project implementation extended from December 2015 to February 2016. However, due to local issues the due date to apply the pre-survey was missed. Providers were educated about the tool, and the pre-survey was not used. After these steps, the second survey was sent to providers and local site champion, the director of the disorder of consciousness program at the local hospital, and to the clinical pharmacist. Based on survey, responses, 99% of providers responded to the post implementation survey. Only 10 providers and one clinical pharmacist answered all questions. The invitations were sent to a total of 12 participants and one never responded despite multiple attempts via email request. Due to conflict of interest, the decision was made to involve only providers and a clinical pharmacist. Therefore, 11 responded, including 6 hospitalists (4 physicians and 2 nurse practitioners) 2 neurologists as specialists, 1 physiatrist, 1 psychiatrist, and 1 clinical pharmacist (Table B).
Along with the provider survey data collection, N=7 patients were randomly selected for chart audits to follow their progress throughout hospital stay. Between September 2015 and February 2016 a post implementation survey was conducted, CRS-R scale results were collected (Table A). These results reassured providers of the usefulness of the tool to support the team in making clinical decisions and to improve quality of care. One-on-one meetings were conducted to assess providers’ confidence to continue considering the evidence-based tool as a valid resource. Concepts were clear to providers during both phases of the survey (Table D) according to survey responses. Survey responses indicated that providers and non-providers approved the toolkit with 100% satisfaction.
A post implementation survey was sent to providers to verify the level of satisfaction and evaluate project aims during project development and implementation. Based on post implementation survey response (Table D), providers responded that they were overall comfortable using the initial toolkit to prescribe medications. Of the respondents, 86.7% expressed satisfaction with the evidence-based neuropharmacologic toolkit, and 100% would refer the tool to another provider. About 87.5% of the providers were comfortable with the information in the toolkit about how to prescribe medications 11.
The CRS score was used to observe patients’ improvements along with providers’ adherence to the toolkit. The CRS scores reassure all providers the applicability of this quality improvement project. Considering the trends of the CRS with the baseline initial score mean of 9.86 before tool implementation, with the changes post minus pre score of three, and reaching a positive outcome of 27.68 % on improvement score, that helped the providers to continue using the tool, reflecting a positive outcome to this quality improvement project. Following the positive outcome of the CRS scores, meeting providers reassuring the quality of care promoted with the tool applicability to treat poor alertness, that made this quality improvement project applicable to this clinical site.
Summary
Traumatic brain injury can disrupt anyone’s life and directly impact family resources as well as negatively impact the economy due to absence from the workforce. TBIs also incur a high cost to the healthcare system. To positively improve outcomes for TBI patients, healthcare institutions must offer the best care possible to speed patient recovery. Providers play a vital role as the front line in making up-to-date care available for patients. Due to the lack of national and international neuropharmacology guidelines for treatment of TBIs, providers are seeking an evidence-based guideline to improve patient alertness, help patients to progress with treatment, and eventually facilitate patients’ return to the community as early as possible with fewer brain lesions and reduced post-treatment complications.
Strengths contributing to this QI project include the providers’ adherence to the guideline. The development and implementation of the neuropharmacologic toolkit has positively impacted the quality of care and will improve patient and family satisfaction 1, 11. It has become a fundamental resource for all providers in the neuro unit, and it will be used hospital-wide to help providers treat poor level of alertness due to severe TBIs. The providers at the intervention site now have a valid resource to support care under evidence-based processes.
According to survey responses, the team and providers are highly satisfied with the evidence-based toolkit and strongly recommend the resource to other professionals to treat patients with post traumatic brain injury experiencing poor alertness. The improvement in the mean CRS-R score supports the efficacy of the toolkit despite small sample size of cases (N=7), indicating that the toolkit is a reliable resource for providers seeking to use an evidence-based guide (Table E). The post survey response indicated positive outcomes such as providers recommending the tool to others, and satisfaction that the evidence-based guide helped to prescribe medication. This evidence enables providers to use the tool with confidence that it will improve quality of care. A future goal is for this tool to be used outside the local institution after it meets all criteria for publication
Interpretation
Traumatic brain injury is a significant health care problem that affects the healthcare system, and socioeconomically affects the whole world. In the United States, approximately 2.5 million patients are affected each year, and the mortality of severe TBI remains as high as 35%. In fact, per the Center of Disease Control and Prevention, traumatic brain injury is a contributing factor to nearly a third (30%) of all injury-related deaths in the United States 1, 12. These statistics underline the urgent need for efficient treatment to improve post-traumatic morbidity and mortality, support efforts to develop effective pharmacological treatment for TBI patients, and outline promising therapeutic avenues in the field to improve quality of care. Severe TBI not only impacts the life patients and their families, it also has a large societal and economic toll. The estimated economic cost of TBI in 2010, including direct and indirect medical costs, is estimated to be approximately $76.5 billion. Additionally, the cost of fatal TBIs and TBIs requiring hospitalization, many of which are severe, accounts for approximately 90% of the total TBI medical costs per the central of Disease Control and Prevention 1, 12. The site’s neurology unit has the challenge to offer the best care possible when treating poor level of alertness in patients suffering post-traumatic brain injury. There is a national and worldwide need for a neuropharmacology guideline to treat TBIs, as this will help providers to properly address patients’ ideal treatment regimens in rehabilitation programs, with the final goal of returning to the community with as few brain injuries and motor lesions as possible.
Observed outcome was consistent with the review of literature, with a selection of the most known recognized studies in neurology in US and overseas in the past fifteen years. Deep analysis of the promising neuropharmacology improvements has significantly improved the lives of many patients experiencing coma, vegetative and minimum conscious state 8. When providers are able to apply evidence-based interventions, quality of care improves. Having a resource available makes it easier for them to apply the guideline.
The high expectation to promote alertness in this unit is fundamental to reassure best recovery and rehabilitation of the patient. The neuro unit provides high quality of care using neuropharmacology, and the use of evidence-based neuropharmacology practices will positively impact and speed the process of enhancing alertness, thereby promoting patient recovery.
Since this project began one and a half years ago, positive cases have progressed, indicating that a neuropharmacological toolkit is an essential resource to help providers to enhance alertness and enable a plan of care for rehabilitation. The key factor is to speed the treatment to reach the best results for patients and their families. Neurological units now have an evidence-based tool to avoid planning delays when treating poor alertness. The toolkit is appreciated by all providers and highly recommended to be used by other professionals and institutions.
Limitations
Risk of bias
The project is a QI project and therefore limited to application to the site in which it was implemented. That said however, the tool that was developed and implemented was evidence-based and could have implications for other comparable sites.
This QI project demonstrated that the evidence-based pharmacological toolkit is a promising resource to help providers to treat lack of alertness when treating patients experiencing poor recovery after severe TBIs 9. Providers no longer need to work without evidence-based practice resources for treating patients with disorders of consciousness. As a long term goal, the toolkit will become an electronic resource for the institution and eventually will be published. Adherence to an evidence-based tool will help providers use a valid resource when prescribing neuropharmacology to enhance alertness 9. About the time this project started, the site was seeking an evidence-based resource to help providers when treating traumatic brain injury, and the tool became a valid resource for this microsystem. As an agent of change with a holistic approach, this project was designed to support future updates, in the hope that new guidelines will emerge in this area to promote the best possible outcomes for patients experiencing poor recovery after traumatic brain injury.
During this project development and implementation, the CRS-R score was used as a metric to reassure all providers and the microsystem as a whole of the feasibility of the medications to improve level of alertness. Based on the mean percent improvement in the CRS score, this guideline reassured all providers of its validity as an evidence-based resource to be used when making clinical decisions to improve quality of care.
Finally, the project significantly impacted the neuro unit and improved the quality of care. Based on the responses from the survey, 86.7% of providers were satisfied with the evidence-based toolkit to treat poor level of consciousness. Also, it was noticed during one- on-one interviews that all providers sought evidence-based guidance in neuropharmacology to treat TBIs. Teamwork and full engagement of the microsystem was recognized during the project stages to promote the success of and positive outcomes resulting from this quality improvement project 4, 6. Many brain injury organizations are recommending that most institutions create their own guidelines for treatment of traumatic brain injury, validating the national need for an evidence- based neuropharmacological toolkit to treat alertness and improve quality of care 11.
[1] | Paul M, Xu L, Wald MM, Coronado VG. Traumatic brain injury. www.cdc.gov/TraumaticBrainInjury. Updated 2010. Accessed 10/16, 2015. | ||
In article | |||
[2] | Giacino J, Kalmar K. Coma recovery scale-revised. The center for outcome measurement in brain injury. COMBI The Center for Outcome Measurement In Brain Injury Web site. https://www.tbims.org/combi/crs. Published May 14, 2016. Updated 2016. Accessed 5/14, 2016. | ||
In article | |||
[3] | Pistoia F, Mura E, Govoni S, Fini M, Sarà M. Awakenings and awareness recovery in disorders of consciousness: Is there a role for drugs? CNS Drugs. 2010; 24(8): 625-638. | ||
In article | View Article PubMed | ||
[4] | Seel RT, Douglas J, Dennison AC, Heaner S, Farris K, Rogers C. Specialized early treatment for persons with disorders of consciousness: Program components and outcomes. Arch Phys Med Rehabil. 2013; 94(10):1908-1923. | ||
In article | View Article PubMed | ||
[5] | Giacino J, Whyte J, Bagiella E, Kalmar K, Childs N, Khademi A. Placebo-controlled trial of amantadine for severe traumatic brain injury. New England Journal of Medicine. 2012; 366(9) (3/2012): 819-26. | ||
In article | |||
[6] | Whyte J. Disorders of consciousness: The changing landscape of treatment. Neurology. 2014; 82(13): 1106-1107. | ||
In article | View Article PubMed | ||
[7] | Ugoya SO, Akinyemi RO. The place of L-dopa/carbidopa in persistent vegetative state. Clin Neuropharmacol. 2010; 33(6): 279-284. | ||
In article | View Article PubMed | ||
[8] | Whyte J, Giacino J. Disorders of consciousness. In: Arciniegas DB, Zasler ND, Vanderploeg RD, Jaffee MS., ed. Management of adults with traumatic brain injury. 1st ed. Washington,DC: American Psychiatric Publishing; 2013:103-124. | ||
In article | View Article | ||
[9] | Dougall D, Poole N, Agrawal N. Pharmacotherapy for chronic cognitive impairment in traumatic brain injury. Cochrane Database of Systematic Reviews. 2015; 12. | ||
In article | View Article | ||
[10] | Talsky A, Pacione LR, Shaw T, et al. Pharmacological interventions for traumatic brain injury. BC Med J. 2010; 53: 26-31. | ||
In article | |||
[11] | Guidelines for the management of severe traumatic brain injury. Brain Trauma Foundation Web site. https://www.braintrauma.org/upload/06/06Guidelines_Management_2007. Updated 06, 2007. Accessed 10/09, 2015. | ||
In article | |||
[12] | Traumatic brain injury rates of TBI-related emergency department visits, hospitalizations, and deaths - united states, 2001-2010. Centers for Disease Control and Prevention Web site. https://www.cdc.gov/traumaticbraininjury/data/rates.html. Updated 2010. Accessed 10/16, 2015. | ||
In article | |||
[13] | Chew E, Zafonte R. Pharmacological management of neurobehavioral disorders following traumatic brain injury. Journal of Rehabilitation Research & Development. 2009; 46(04/30/2009): 851-78. | ||
In article | |||
[14] | Hughes S, Colantonio A, Santaguida PL, Paton T. Amantadine to enhance readiness for rehabilitation following severe traumatic brain injury. Brain Inj. 2009; 19 (14)(12/2005): 19197-206. | ||
In article | |||
[15] | Meythaler JM, Brunner RC, Johnson A, Novack TA. Amantadine to improve neurorecovery in traumatic brain injury-associated diffuse axonal injury: A pilot double-blind randomized trial. J Head Trauma Rehabil. 2002; 17(4):300-313. | ||
In article | View Article PubMed | ||
[16] | Leone H, Polsonetti BW. Amantadine for traumatic brain injury: Does it improve cognition and reduce agitation? J Clin Pharm Ther. 2005; 30(2): 101-104. | ||
In article | View Article PubMed | ||
[17] | Krimchansky BZ, Keren O, Sazbon L, Groswasser Z. Differential time and related appearance of signs, indicating improvement in the state of consciousness in vegetative state traumatic brain injury (VS-TBI) patients after initiation of dopamine treatment. Brain Inj. 2004; 18(11): 1099-1105. | ||
In article | View Article PubMed | ||
[18] | Fridman E, Krimchansky B, Bonetto M, Galperin T, Gamzu E, Leiguarda R. Continuous subcutaneous apomorphine for severe disorders of consciousness after traumatic brain injury. Allied Health; Biomedical; Europe UK & Ireland. 2010; 24(4)(04/2010): 636-41. | ||
In article | |||
[19] | Du B, Shan A, Zhang Y, Chen D, Cai K. Zolpidem arouses patients in vegetative state after brain injury: Quantitative evaluation and indications. PubMed-index for MEDLINE. 2014; 347(3)(03/2014): 178-82. | ||
In article | |||
[20] | Whyte J, Rajan R, Rosenbaum A, Katz D, Kalmar K, Sell R. Zolpidem and restoration of consciousness. American Journal Physician Medical Rehabilitation. 2014; 93(2)(02/2014): 101-13. | ||
In article | |||
[21] | Kim Y, Ko M, Na S, Park S, Kim K. Effects of single-dose methylphenidate on cognitive performance in patients with traumatic brain injury: A double-blind placebo-controlled study. University School of Medicine, Seoul, republic of Korea. 2006; 24(1/2012): 24-30. | ||
In article | |||
[22] | Kim J, Whyte J, Patel S, et al. Methylphenidate modulates sustained attention and cortical activation in survivors of traumatic brain injury: A perfusion fMRI study. Psychopharmacology (Berl). 2012; 222(1): 47-57. | ||
In article | View Article PubMed | ||
[23] | Krimchansky BZ, Keren O, Sazbon L, Groswasser Z. Differential time and related appearance of signs, indicating improvement in the state of consciousness in vegetative state traumatic brain injury (VS-TBI) patients after initiation of dopamine treatment. Brain Inj. 2004; 18(11): 1099-1105. | ||
In article | View Article PubMed | ||
[24] | Moein P, Ford SA, Asnaashari A, et al. The effect of boswellia serrata on neurorecovery following diffuse axonal injury. Brain Inj. 2013; 27(12): 1454-1460: 1454-1460. | ||
In article | |||
[25] | McAllister TW, Flashman LA, McDonald BC, et al. Dopaminergic challenge with bromocriptine one month after mild traumatic brain injury: Altered working memory and BOLD response. J Neuropsychiatry Clin Neurosci. 2011; 23(3): 277-286. | ||
In article | View Article PubMed | ||
[26] | Tyson B. Havelock's theory of change. Bright Hub Project Management Web site. https://www.brighthubpm.com/change-management/86803-havelocks-theory-of-change/. Updated 2010. Accessed 10/06, 2015. | ||
In article | |||
Bromocriptine 5 to 45 mg daily, and one study using a combination of 100 mg of bromocriptine with 100 mg of ephedrine, showed improvement in akinetic mutism, while another study using 5 mg of bromocriptine combined with sensory stimulation led to improvements in patients with vegetative or minimal consciousness (Talsky, 2011).
Levodopa and carbidopa medications suggest that they might be useful in the chronic phase of TBI with diffuse injury and persistent vegetative state (Talsky, 2011). Combining agents found improvements in neuropsychiatric deficits with the daily administration of 25 mg/250 mg of levodopa/carbidopa three times daily, 250 mg of amantadine, and 5 mg of bromocriptine twice daily (Talsky, 2011).
Administration of various pharmacological agents, such as baclofen, zolpidem and amantadine, have been recently supported by intriguing scientific observations. Using the specific drug terms: ‘zolpidem’, ‘baclofen’, ‘lamotrigine’, ‘antidepressants’, ‘dopaminergic agents’, ‘methylphenidate’ and ‘pure stimulants’. Described administration of CNS depressants (zolpidem, lamotrigine and baclofen) and 20 that described administration of drugs with CNS-stimulating properties, including antidepressants, dopaminergic agents and drugs used in attention-deficit hyperactivity disorders (methylphenidate and pure stimulants [amphetamines]) (Pistoia, 2010).
Effect on alertness/arousal using amantadine, methylphenidate and sertraline. Drugs with a pharmacological profile compatible with that of a stimulant of CNS activity and that have been studied in DOC include antidepressants, dopaminergic agents (pramipexole, bromocriptine, amantadine, levodopa and apomorphine), and drugs used to treat attention-deficit hyperactivity disorders (methylphenidate and pure stimulants (amphetamine) (Pistoia, 2010).
A retrospective review of 10 children and adolescents in VS or MCS who were on various dopamine-enhancing medications (amantadine, methylphenidate, pramipexole, bromocriptine, levodopa) showed significant improvement in responses to structured stimuli in a double-baseline serial measure ABA design. Seven of the ten subjects had sustained a TBI (Chew and Zafonte, 2009).
Evidence Based Pharmacological Toolkit for TBI to improve level of consciousness: Logic Model
Project aim: Development and implementation of an evidence based pharmacological toolkit. The toolkit of pharmacological agents will help providers to make clinical decisions using evidence based practices to optimize quality of care.
Situation: Provide evidence-based toolkit with pharmacological agents available for providers’ use to enhance level of consciousness. This toolkit is designed to improve clinical-decision making, and meliorate the health care outcomes.
This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/
[1] | Paul M, Xu L, Wald MM, Coronado VG. Traumatic brain injury. www.cdc.gov/TraumaticBrainInjury. Updated 2010. Accessed 10/16, 2015. | ||
In article | |||
[2] | Giacino J, Kalmar K. Coma recovery scale-revised. The center for outcome measurement in brain injury. COMBI The Center for Outcome Measurement In Brain Injury Web site. https://www.tbims.org/combi/crs. Published May 14, 2016. Updated 2016. Accessed 5/14, 2016. | ||
In article | |||
[3] | Pistoia F, Mura E, Govoni S, Fini M, Sarà M. Awakenings and awareness recovery in disorders of consciousness: Is there a role for drugs? CNS Drugs. 2010; 24(8): 625-638. | ||
In article | View Article PubMed | ||
[4] | Seel RT, Douglas J, Dennison AC, Heaner S, Farris K, Rogers C. Specialized early treatment for persons with disorders of consciousness: Program components and outcomes. Arch Phys Med Rehabil. 2013; 94(10):1908-1923. | ||
In article | View Article PubMed | ||
[5] | Giacino J, Whyte J, Bagiella E, Kalmar K, Childs N, Khademi A. Placebo-controlled trial of amantadine for severe traumatic brain injury. New England Journal of Medicine. 2012; 366(9) (3/2012): 819-26. | ||
In article | |||
[6] | Whyte J. Disorders of consciousness: The changing landscape of treatment. Neurology. 2014; 82(13): 1106-1107. | ||
In article | View Article PubMed | ||
[7] | Ugoya SO, Akinyemi RO. The place of L-dopa/carbidopa in persistent vegetative state. Clin Neuropharmacol. 2010; 33(6): 279-284. | ||
In article | View Article PubMed | ||
[8] | Whyte J, Giacino J. Disorders of consciousness. In: Arciniegas DB, Zasler ND, Vanderploeg RD, Jaffee MS., ed. Management of adults with traumatic brain injury. 1st ed. Washington,DC: American Psychiatric Publishing; 2013:103-124. | ||
In article | View Article | ||
[9] | Dougall D, Poole N, Agrawal N. Pharmacotherapy for chronic cognitive impairment in traumatic brain injury. Cochrane Database of Systematic Reviews. 2015; 12. | ||
In article | View Article | ||
[10] | Talsky A, Pacione LR, Shaw T, et al. Pharmacological interventions for traumatic brain injury. BC Med J. 2010; 53: 26-31. | ||
In article | |||
[11] | Guidelines for the management of severe traumatic brain injury. Brain Trauma Foundation Web site. https://www.braintrauma.org/upload/06/06Guidelines_Management_2007. Updated 06, 2007. Accessed 10/09, 2015. | ||
In article | |||
[12] | Traumatic brain injury rates of TBI-related emergency department visits, hospitalizations, and deaths - united states, 2001-2010. Centers for Disease Control and Prevention Web site. https://www.cdc.gov/traumaticbraininjury/data/rates.html. Updated 2010. Accessed 10/16, 2015. | ||
In article | |||
[13] | Chew E, Zafonte R. Pharmacological management of neurobehavioral disorders following traumatic brain injury. Journal of Rehabilitation Research & Development. 2009; 46(04/30/2009): 851-78. | ||
In article | |||
[14] | Hughes S, Colantonio A, Santaguida PL, Paton T. Amantadine to enhance readiness for rehabilitation following severe traumatic brain injury. Brain Inj. 2009; 19 (14)(12/2005): 19197-206. | ||
In article | |||
[15] | Meythaler JM, Brunner RC, Johnson A, Novack TA. Amantadine to improve neurorecovery in traumatic brain injury-associated diffuse axonal injury: A pilot double-blind randomized trial. J Head Trauma Rehabil. 2002; 17(4):300-313. | ||
In article | View Article PubMed | ||
[16] | Leone H, Polsonetti BW. Amantadine for traumatic brain injury: Does it improve cognition and reduce agitation? J Clin Pharm Ther. 2005; 30(2): 101-104. | ||
In article | View Article PubMed | ||
[17] | Krimchansky BZ, Keren O, Sazbon L, Groswasser Z. Differential time and related appearance of signs, indicating improvement in the state of consciousness in vegetative state traumatic brain injury (VS-TBI) patients after initiation of dopamine treatment. Brain Inj. 2004; 18(11): 1099-1105. | ||
In article | View Article PubMed | ||
[18] | Fridman E, Krimchansky B, Bonetto M, Galperin T, Gamzu E, Leiguarda R. Continuous subcutaneous apomorphine for severe disorders of consciousness after traumatic brain injury. Allied Health; Biomedical; Europe UK & Ireland. 2010; 24(4)(04/2010): 636-41. | ||
In article | |||
[19] | Du B, Shan A, Zhang Y, Chen D, Cai K. Zolpidem arouses patients in vegetative state after brain injury: Quantitative evaluation and indications. PubMed-index for MEDLINE. 2014; 347(3)(03/2014): 178-82. | ||
In article | |||
[20] | Whyte J, Rajan R, Rosenbaum A, Katz D, Kalmar K, Sell R. Zolpidem and restoration of consciousness. American Journal Physician Medical Rehabilitation. 2014; 93(2)(02/2014): 101-13. | ||
In article | |||
[21] | Kim Y, Ko M, Na S, Park S, Kim K. Effects of single-dose methylphenidate on cognitive performance in patients with traumatic brain injury: A double-blind placebo-controlled study. University School of Medicine, Seoul, republic of Korea. 2006; 24(1/2012): 24-30. | ||
In article | |||
[22] | Kim J, Whyte J, Patel S, et al. Methylphenidate modulates sustained attention and cortical activation in survivors of traumatic brain injury: A perfusion fMRI study. Psychopharmacology (Berl). 2012; 222(1): 47-57. | ||
In article | View Article PubMed | ||
[23] | Krimchansky BZ, Keren O, Sazbon L, Groswasser Z. Differential time and related appearance of signs, indicating improvement in the state of consciousness in vegetative state traumatic brain injury (VS-TBI) patients after initiation of dopamine treatment. Brain Inj. 2004; 18(11): 1099-1105. | ||
In article | View Article PubMed | ||
[24] | Moein P, Ford SA, Asnaashari A, et al. The effect of boswellia serrata on neurorecovery following diffuse axonal injury. Brain Inj. 2013; 27(12): 1454-1460: 1454-1460. | ||
In article | |||
[25] | McAllister TW, Flashman LA, McDonald BC, et al. Dopaminergic challenge with bromocriptine one month after mild traumatic brain injury: Altered working memory and BOLD response. J Neuropsychiatry Clin Neurosci. 2011; 23(3): 277-286. | ||
In article | View Article PubMed | ||
[26] | Tyson B. Havelock's theory of change. Bright Hub Project Management Web site. https://www.brighthubpm.com/change-management/86803-havelocks-theory-of-change/. Updated 2010. Accessed 10/06, 2015. | ||
In article | |||