Publications
1. A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports: Algorithm Development and Validation Study
Chen H, Cohen E, Wilson D, Alfred M. (2024). A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports: Algorithm Development and Validation Study
JMIR Hum Factors 2024;11:e53378. https://doi.org/10.2196/53378
ABSTRACT
Adverse events refer to incidents with potential or actual harm to patients in hospitals. These events are typically documented through patient safety event (PSE) reports, which consist of detailed narratives providing contextual information on the occurrences. Accurate classification of PSE reports is crucial for patient safety monitoring. However, this process faces challenges due to inconsistencies in classifications and the sheer volume of reports. Recent advancements in text representation, particularly contextual text representation derived from transformer-based language models, offer a promising solution for more precise PSE report classification. Integrating the machine learning (ML) classifier necessitates a balance between human expertise and artificial intelligence (AI). Central to this integration is the concept of explainability, which is crucial for building trust and ensuring effective human-AI collaboration. This study aims to investigate the efficacy of ML classifiers trained using contextual text representation in automatically classifying PSE reports. Furthermore, the study presents an interface that integrates the ML classifier with the explainability technique to facilitate human-AI collaboration for PSE report classification.
2. Improving patient safety event report classification with machine learning and contextual text representation
Chen, H., Cohen, E., Wilson, D., & Alfted, M. (2023). Improving Patient Safety Event Report Classification with Machine Learning and Contextual Text Representation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 0(0). https://doi.org/10.1177/21695067231193645
ABSTRACT
Adverse events caused by medical errors pose a significant threat to patient safety, with estimates of 251,454 deaths and a cost of $17.1 billion to the healthcare system annually in the United States. Patient safety event (PSE) reports play a vital role in identifying measures to prevent adverse events, but their utility is dependent on the accurate classification of PSE reports. Recent studies have used static natural language processing (NLP) and machine learning (ML) techniques to automate PSE report classification. However, the use of static NLP has limitations in differentiating the meaning of words in disparate contexts, which can lead to inferior classification results. Thus, this study proposes to utilize contextual text representation produced from neural NLP methods to improve the accuracy of PSE report classification. The results suggest that the contextual text representation can further improve the performance of PSE classifiers. The best-performing classifier, a support vector machine trained with contextual text representation (Roberta-base) reaches an accuracy of 0.75 and a ROCAUC score of 0.94, surpassing all ML classifiers trained with static text representations. Furthermore, the confusion matrix of the best classifier exposes latent deficiencies in the PSE reports' classification taxonomy, such as the multi-class nature of PSE and conceptually related event types. The study's findings can save time for PSE reclassification, enhance the learning capabilities of the reporting system, ultimately improve patient safety.
3. Applying an equity lens to hospital safety monitoring: a critical interpretive synthesis protocol
Joanne Goldman, Lisha Lo, Leahora Rotteau, Brian M Wong, Ayelet Kuper, Maitreya Coffey, Shail Rawal, Myrtede Alfred, Saleem Razack, Marie Pinard, Michael Palomo, Patricia Trbovich. (2023). Applying an equity lens to hospital safety monitoring: a critical interpretive synthesis protocol. BMJ Open, 13:e072706. doi: 10.1136/bmjopen-2023-072706
ABSTRACT
Introduction Hospital safety monitoring systems are foundational to how adverse events are identified and addressed. They are well positioned to bring equity-related safety issues to the forefront for action. However, there is uncertainty about how they have been, and can be, used to achieve this goal. We will undertake a critical interpretive synthesis (CIS) to examine how equity is integrated into hospital safety monitoring systems.
Methods and analysis This review will follow CIS principles. Our initial compass question is: How is equity integrated into safety monitoring systems? We will begin with a structured search strategy of hospital safety monitoring systems in CINAHL, EMBASE, MEDLINE and PsycINFO for up to May 2023 to identify papers on safety monitoring systems generally and those linked to equity (eg, racism, social determinants of health). We will also review reference lists of selected papers, contact experts and draw on team expertise. For subsequent literature searching stages, we will use team expertise and expert contacts to purposively search the social science, humanities and health services research literature to support the development of a theoretical understanding of our topic. Following data extraction, we will use interpretive processes to develop themes and a critique of the literature. The above processes of question formulation, article search and selection, data extraction, and critique and synthesis will be iterative and interactive with the goal to develop a theoretical understanding of equity in hospital monitoring systems that will have practice-based implications.
Ethics and dissemination This review does not require ethical approval because we are reviewing published literature. We aim to publish findings in a peer-reviewed journal and present at conferences.
4. Investigating Racial and Ethnic Disparities in Maternal Care at the System Level Using Patient Safety Incident Reports
ABSTRACT
Maternal mortality in the United States is high, and women and birthing people of color experience higher rates of mortality and severe maternal morbidity (SMM). More than half of maternal deaths and cases of SMM are considered preventable. Our research investigated systems issues contributing to adverse outcomes and racial/ethnic disparities in maternal care using patient safety incident reports. We reviewed incidents reported in the labor and delivery unit (L&D) and the antepartum and postpartum unit (A&P) of a large academic hospital in 2019 and 2020. Deliveries associated with a reported incident were described by race/ethnicity, age group, method of delivery, and several other process variables. Differences across racial/ethnic group were statistically evaluated. Almost two-thirds (64.8%) of the 528 reports analyzed were reported in L&D, and 35.2% were reported in A&P. Non-Hispanic white (NHW) patients accounted for 43.9% of reported incidents, non-Hispanic Black (NHB) patients accounted for 43.2%, Hispanic patients accounted for 8.9%, and patients categorized as “other” accounted for 4.0%. NHB patients were disproportionally represented in the incident reports as they only accounted for 36.5% of the underlying birthing population. The odds ratio (OR) demonstrated a higher risk of a reported adverse incident for NHB patients; however, adjustment for cesarean section attenuated the association (OR = 1.25; 95% confidence interval = 1.01–1.54). Greater integration of patient safety and health equity efforts in hospitals are needed to promptly identify and alleviate racial and ethnic disparities in maternal health outcomes. While additional systems analysis is necessary, we offer recommendations to support safer, more equitable maternal care.
5. What's fair is... fair? Presenting JustEFAB, an ethical framework for operationalizing medical ethics and social justice in the integration of clinical machine learning: JustEFAB
ABSTRACT
The problem of algorithmic bias represents an ethical threat to the fair treatment of patients when their care involves machine learning (ML) models informing clinical decision-making. The design, development, testing, and integration of ML models therefore require a lifecycle approach to bias identification and mitigation efforts. Presently, most work focuses on the ML tool alone, neglecting the larger sociotechnical context in which these models operate. Moreover, the narrow focus on technical definitions of fairness must be integrated within the larger context of medical ethics in order to facilitate equitable care with ML. Drawing from principles of medical ethics, research ethics, feminist philosophy of science, and justice-based theories, we describe the Justice, Equity, Fairness, and Anti-Bias (JustEFAB) guideline intended to support the design, testing, validation, and clinical evaluation of ML models with respect to algorithmic fairness. This paper describes JustEFAB's development and vetting through multiple advisory groups and the lifecycle approach to addressing fairness in clinical ML tools. We present an ethical decision-making framework to support design and development, adjudication between ethical values as design choices, silent trial evaluation, and prospective clinical evaluation guided by medical ethics and social justice principles. We provide some preliminary considerations for oversight and safety to support ongoing attention to fairness issues. We envision this guideline as useful to many stakeholders, including ML developers, healthcare decision-makers, research ethics committees, regulators, and other parties who have interest in the fair and judicious use of clinical ML tools.
6. Applications of Extended Reality (XR) in obtaining informed consent: A narrative review
ABSTRACT
Informed consent in healthcare requires patients to have a sufficient understanding of their upcoming procedure before deciding to proceed. Unfortunately, education prior to a surgical procedure is constrained by barriers including poor health literacy, language barriers, one-sided dialogue during consultations, anxiety, and knowledge retention. Extended reality (XR), which includes virtual reality (VR), augmented reality (AR), and mixed reality (MR) has the potential to improve informed consent processes by creating an immersive, interactive, and multimodal sensory experience that supports patient education. The purpose of the study was to review the extant literature on the effectiveness of XR technology in improving patient education, a vital component of informed consent. We screened fifty-two articles and ten relevant papers from PubMed, Scopus, and Compendex, which were included in the review based on our eligibility criteria. We found that VR and AR proved effective in enhancing patient education in eight studies, and thus improving informed consent processes. MR was not utilized in the studies reviewed. The studies were conducted in several countries and positives findings were reported from a broad range of clinical settings and procedures. Though further investigation is needed, this is a promising finding that may encourage health systems to implement similar interventions prior to procedures. The review also provided an overview of the existing XR technology utilized for patient education such as a downloadable mobile application with a virtual chatbot character, and an environment designed to simulate the MRI patient’s perspective. These applications provide immersive and interactive experiences when paired with a head mounted headset such as Google VR Cardboard. The findings also revealed that XR tools are customizable and can be tailored to specific surgical procedures, which makes the potential of implementation applicable to a broader range of settings.
7. Systems engineering-based framework of process risks in perioperative medication delivery
ABSTRACT
Human factors engineering (HFE) approaches can provide important insights into improving patient safety during medication delivery processes. HFE provides guidance on how to model existing processes, which can identify features that are particularly brittle and therefore could be targeted for intervention. The recent use of a Systems Theoretic Process Analysis to identify hazards and controllers within the anaesthesia medication use process is one such example of the value of HFE modelling. However, the complexity of the anaesthesia system leaves no model capable of fully encapsulating all hazards or dangers. Therefore, we must use various HFE models to illustrate and bring understanding to different components of the anaesthesia system. The Systems Engineering Initiative for Patient Safety (SEIPS) model is one such HFE modelling approach that represents the complexities of healthcare systems by describing these systems using the following interacting overarching components: people, tasks, tools, organisation, physical environment, and external environment. Using the SEIPS 2.0 framework, we developed a model to describe the anaesthesia medication delivery process that provides new insight into potential areas ripe for intervention.
8. “One size” doesn't “fit all”: Understanding variability in anesthesia work practices
Biro, J. Neyens, D., Jaruzel, C., Tobin, C., Alfred, M., Coppola, S., Abernathy, J., & Catchpole,
K. (2022). Examining Variability in Anesthesia Work Practices via Vignette-based Interviews.
Human Factors in Healthcare, 100026. https://doi.org/10.1016/j.hfh.2022.100026 https://doi.org/10.1016/j.bja.2022.09.004
ABSTRACT
This work examines the variability in how different anesthesia providers approach patient care to provide insight into the source and necessity of variations in practice, the implications of different individual preferences, and the subsequent consequences on approaches to safety that emphasize standardization.Vignette-based interviews were performed with 18 anesthesia providers to elicit the cognitive processes and strategies that they utilize as they progress through a case. A qualitative data analysis of the transcripts was performed to illuminate the variability in their processes of managing the case.Participants employed converging strategies consistent with prior literature. However, divergence occurs in the application of those strategies; there is variability in participants’ information seeking strategies, problem anticipation, and resulting actions. In some instances, this divergence was minor, representing natural preferences that would inevitably converge in the management of the case. In other instances, this divergence represented major deviations that would not converge throughout the management of the case.The differences in how anesthesia providers approach their work, as identified in this study, call into question whether ‘standardization’ is always the best approach to improve safety in anesthesia. Whatever the cause of variations in individual strategies, when developing new approaches, the variability in anesthesia information seeking, goal setting and anticipation needs to be reflected in policies, practices, protocols, and training. This work reinforces the idea that it is the humans in the system, with their flexibility and expertise, who are the primary source of everyday safety.
9. A better way: training for direct observations in healthcare
ABSTRACT
Direct observation is valuable for identifying latent threats and elucidating system complexity in clinical environments. This approach facilitates prospective risk assessment and reveals workarounds, near-misses and recurrent safety problems difficult to diagnose retrospectively or via outcome data alone. As observers are an instrument of data collection, developing effective and comprehensive observer training is critical to ensuring the reliability of the data collection and reproducibility of the research. However, methodological rigour for ensuring these data collection properties remains a key challenge in direct observation research in healthcare. Although prior literature has offered key considerations for observational research in healthcare, operationalising these recommendations may pose a challenge and unless guidance is also provided on observer training. In this article, we offer guidelines for training non-clinical observers to conduct direct observations including conducting a training needs analysis, incorporating practice observations and evaluating observers and inter-rater reliability. The operationalisation of these guidelines is described in the context of a 5-year multisite observational study investigating technology integration in the operating room. We also discuss novel tools developed during the course our project to support data collection and examine inter-rater reliability among observers in direct observation studies.
10. Human factors integration in robotic surgery
Catchpole, K., Cohen, T., Alfred, M., Lawton, S., Kanji, F., Shouhed, D., ... & Anger, J. (2022).
Human Factors Integration in Robotic Surgery. Human Factors. https://doi.org/10.1177/00187208211068946 https://doi.org/10.1177/00187208211068946
ABSTRACT
Using the example of robotic-assisted surgery (RAS), we explore the methodological and practical challenges of technology integration in surgery, provide examples of evidence-based improvements, and discuss the importance of systems engineering and clinical human factors research and practice. New operating room technologies offer potential benefits for patients and staff, yet also present challenges for physical, procedural, team, and organizational integration. Historically, RAS implementation has focused on establishing the technical skills of the surgeon on the console, and has not systematically addressed the new skills required for other team members, the use of the workspace, or the organizational changes. Human factors studies of robotic surgery have demonstrated not just the effects of these hidden complexities on people, teams, processes, and proximal outcomes, but also have been able to analyze and explain in detail why they happen and offer methods to address them. We review studies on workload, communication, workflow, workspace, and coordination in robotic surgery, and then discuss the potential for improvement that these studies suggest within the wider healthcare system. There is a growing need to understand and develop approaches to safety and quality improvement through human-systems integration at the frontline of care.
​
11. Anaesthesia provider perceptions of system safety and critical incidents in non-operating theatre anaesthesia
Alfred, M., Herman, A., Wilson, D., Neyens, D., Jaruzel, C., Tobin, C., Reves, JG., Catchpole, K. (in press). Anaesthesia providers’ perceptions of system safety and critical incidents in non-operating theatre anaesthesia. British Journal of Anaesthesia. https://doi.org/10.1016/j.bja.2021.12.041
ABSTRACT
Non-operating room anaesthesia (NORA) or non-operating theatre anaesthesia (NOTA) will represent 50% of all anaesthesia cases within a decade. Caring for older, medically complex patients in environments not historically used nor designed for anaesthesia care could create higher rates of mortality and morbidity in comparison with the operating theatre, particularly in cardiology and radiology. Estimates of adverse events in NOTA range from <1% to 2.5%. Given known under-reporting of critical incidents, collecting surveys and narratives of incidents can guide a system safety approach to identify risks prospectively and develop mitigation strategies that reduce the potential for adverse outcomes.
12. Barriers to safety and efficiency in robotic surgery docking.
Cofran, L., Cohen, T., Alfred, M., Kanji, F., Choi, E., Savage, S., ... & Catchpole, K. (2022). Barriers to safety and efficiency in robotic surgery docking. Surgical Endoscopy, 36(1), 206-215. https://pubmed.ncbi.nlm.nih.gov/33469695/
ABSTRACT
The introduction of new technology into the operating room (OR) can be beneficial for patients, but can also create new problems and complexities for physicians and staff. The observation of flow disruptions (FDs)-small deviations from the optimal course of care-can be used to understand how systems problems manifest. Prior studies showed that the docking process in robotic assisted surgery (RAS), which requires careful management of process, people, technology and working environment, might be a particularly challenging part of the operation. We sought to explore variation across multiple clinical sites and procedures; and to examine the sources of those disruptions.
13. Improving health equity through clinical innovation
Alfred, M., & Tully, K. P. (2022). Improving health equity through clinical innovation. BMJ Quality & Safety, 31(9), 634. https://doi.org/10.1136/bmjqs-2021-014540
14. Improving Safety in the Operating Room: Medication Icon Labels Increase Visibility and Discrimination
Lusk, C., Catchpole, K., Neyens, D. M., Goel, S., Graham, R., Elrod, N., Paintlia, A., Alfred, M., Joseph, A., Jaruzel, C., Tobin, C., Heinke, T., & Abernathy, J. H. (2022). Improving safety in the operating room: Medication icon labels increase visibility and discrimination. Applied Ergonomics, 104, [103831]. https://doi.org/10.1016/j.apergo.2022.103831
ABSTRACT
Misreading labels, syringes, and ampoules is reported to make up a 54.4% of medication administration errors. The addition of icons to medication labels in an operating room setting could add additional visual cues to the label, allowing for improved discrimination, visibility, and easily processed information that might reduce medication administration errors. A multi-disciplinary team proposed a method of enhancing visual cues and visibility of medication labels applied to vasoactive medication infusions by adding icons to the labels. Participants were 1.12 times more likely to correctly identify medications from farther away (p < 0.001, AOR = 1.12, 95% CI: 1.02, 1.22) with icons. When icons were present, participants were 2.16 times more likely to be more confident in their identifications (p < 0.001, AOR = 2.16, 95%CI: 1.80, 2.57). Carefully designed icons may offer an additional method for identifying medications, and thus reducing medication administration errors.
15. Using flow disruptions to understand healthcare system safety: A systematic review of observational studies.
Cohen, T. N., Wiegmann, D. A., Kanji, F. F., Alfred, M., Anger, J. T., & Catchpole, K. R. (2022). Using flow disruptions to understand healthcare system safety: A systematic review of observational studies. Applied Ergonomics, 98, 103559. https://doi.org/10.1016/j.apergo.2021.103559
ABSTRACT
This systematic review provides information on the methodologies, measurements and classification systems used in observational studies of flow disruptions in clinical environments. The PRISMA methodology was applied and authors searched two databases (PubMed and Web of Science) for studies meeting the following inclusion criteria: (a) were conducted in a healthcare setting, (b) explored systems-factors leading to deviations in care processes, (c) were prospective and observational, (d) classified observations, and (e) were original research studies published in peer-reviewed journals. Thirty studies were analyzed and a variety of methods were identified for observer training, data collection and observation classification. Although primarily applied in surgery, comparable research has been successfully conducted in other venues such as trauma care, and delivery rooms. The findings of this review were synthesized into a framework of considerations for conducting rigorous methodological studies aimed at understanding clinical systems.
16. Understanding Safety And Systems-factors In Non-operating Room Anesthesia: A Survey.
Herman, A., Alfred, M., Wilson, D., Neyens, D., Jaruzel, C., Tobin, C., ... & Catchpole, K. R. Understanding Safety And Systems-factors In Non-operating Room Anesthesia: A Survey. http://www.asaabstracts.com/strands/asaabstracts/abstract.htm?year=2021&index=1&absnum=6458
ABSTRACT
Non-operating room anesthesia (NORA), or anesthesia delivered outside the traditional operating room (OR) setting, is projected to represent at least 50% of all anesthesia cases over the next decade. NORA cases include those in settings such as the gastroenterology, cardiology, interventional radiology, and psychiatry suites. On average, patients in NORA settings are older and sicker than those seen in the OR, and anesthesia providers are challenged to take care of such patients in new, fast-paced, high-volume environments that may or may not be optimized for anesthesia care. As caseloads increase, these underlying systems issues significantly increase the potential for preventable adverse events. In fact, previous literature has shown that NORA cases may have higher rates of morbidity and mortality associated with anesthesia administration compared to the operating room. Despite this, there is limited literature examining the nature of NORA-related adverse events or quantitative evidence examining how system factors may contribute to such events. In our research, we surveyed anesthesia providers at a large academic medical center to investigate their experiences with near-miss (narrowly avoided events that had the potential to lead to patient harm) and patient harm events in NORA settings
17. Mortality, Morbidity, and System Safety in Non-operating room Anesthesia: A Narrative Review. British Journal of Anesthesia.
Herman, A., Jaruzel, C., Tobin…& Alfred, M. (2021). Mortality, Morbidity, and System Safety in Non-operating room Anesthesia: A Narrative Review. British Journal of Anesthesia. https://www.bjanaesthesia.org/article/S0007-0912(21)00453-0/fulltext
ABSTRACT
Non-operating room anaesthesia (NORA) describes anaesthesia delivered outside a traditional operating room (OR) setting. Non-operating room anaesthesia cases have increased significantly in the last 20 yr and are projected to account for half of all anaesthetics delivered in the next decade. In contrast to most other medication administration contexts, NORA is performed in high-volume fast-paced environments not optimised for anaesthesia care. These predisposing factors combined with increasing case volume, less provider experience, and higher-acuity patients increase the potential for preventable adverse events. Our narrative review examines morbidity and mortality in NORA settings compared with the OR and the systems factors impacting safety in NORA.
18. Room Size Influences Flow in Robotic-Assisted Surgery
Kanji, F., Cohen, T., Alfred, M., Caron, A., Lawton, S., Savage, S., ... & Catchpole, K. (2021). Room Size Influences Flow in Robotic-Assisted Surgery. International Journal of Environmental Research and Public Health, 18(15), 7984.
ABSTRACT
The introduction of surgical technology into existing operating rooms (ORs) can place novel demands on staff and infrastructure. Despite the substantial physical size of the devices in robotic-assisted surgery (RAS), the workspace implications are rarely considered. This study aimed to explore the impact of OR size on the environmental causes of surgical flow disruptions (FDs) occurring during RAS. Fifty-six RAS procedures were observed at two academic hospitals between July 2019 and January 2021 across general, urologic, and gynecologic surgical specialties. A multiple regression analysis demonstrated significant effects of room size in the pre-docking phase (t = 2.170, df = 54, β = 0.017, p = 0.035) where the rate of FDs increased as room size increased, and docking phase (t = −2.488, df = 54, β = −0.017, p = 0.016) where the rate of FDs increased as room size decreased. Significant effects of site (pre-docking phase: p = 0.000 and docking phase: p = 0.000) were also demonstrated. Findings from this study demonstrate hitherto unrecognized spatial challenges involved with introducing surgical robots into the operating domain. While new technology may provide benefits towards patient safety, it is important to consider the needs of the technology prior to integration.
19. Using Flow Disruptions to Examine System Safety in Robotic-Assisted Surgery: Protocol for a Stepped Wedge Crossover Design.
Alfred, M., Cohen, T. N., Cohen, K. A., Kanji, F. F., Choi, E., Del Gaizo, J., ... & Catchpole, K. (2021). Using Flow Disruptions to Examine System Safety in Robotic-Assisted Surgery: Protocol for a Stepped Wedge Crossover Design. JMIR Research Protocols, 10(2), e25284.
ABSTRACT
The integration of high technology into health care systems is intended to provide new treatment options and improve the quality, safety, and efficiency of care. Robotic-assisted surgery is an example of high technology integration in health care, which has become ubiquitous in many surgical disciplines. This study aims to understand and measure current robotic-assisted surgery processes in a systematic, quantitative, and replicable manner to identify latent systemic threats and opportunities for improvement based on our observations and to implement and evaluate interventions. This 5-year study will follow a human factors engineering approach to improve the safety and efficiency of robotic-assisted surgery across 4 US hospitals.
20. Work System Interventions in Robotic Assisted Surgery: A Systematic Review Exploring the Gap Between Challenges and Solutions.
Kanji, F., Catchpole, K., Choi, E., Alfred, M., …& Cohen, T. (2021). Work System Interventions in Robotic Assisted Surgery: A Systematic Review Exploring the Gap Between Challenges and Solutions. Surgical Endoscopy,10.1007/s00464-020-08231. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8058237/
ABSTRACT
The introduction of a robot into the surgical suite changes the dynamics of the work-system, creating new opportunities for both success and failure. An extensive amount of research has identified a range of barriers to safety and efficiency in Robotic Assisted Surgery (RAS), such as communication breakdowns, coordination failures, equipment issues, and technological malfunctions. However, there exists very few solutions to these barriers. The purpose of this review was to identify the gap between identified RAS work-system barriers and interventions developed to address those barriers.
21. Pilot Evaluation of a SmartPhone Application for Teamwork and Communication in Trauma ‘In the Wild’.
Catchpole, K., Privette, A., Roberts, L., Wilson, D., Alfred, M., Carter, B., Woltz, E., Kish, M., and Bruce Crookes B. (2021). Pilot Evaluation of a SmartPhone Application for Teamwork and Communication in Trauma ‘In the Wild’. Human Factors. DOI:10.1177/00187208211021717
ABSTRACT
Disruptions along the trauma pathway that arise from communication, coordination, and handoffs problems can delay progress through initial care, imaging diagnosis, and surgery to intensive care unit (ICU) disposition. Implementing carefully designed and evaluated information distribution and communication technologies may afford opportunities to improve clinical performance. This was a pilot evaluation “in the wild” using a before/after design, 3 month, and pre- post-intervention data collection. Use statistics, usability assessment, and direct observation of trauma care were used to evaluate the app. Ease of use and utility were assessed using the technology acceptance model (TAM) and system usability scale (SUS). Direct observation deployed measures of flow disruptions (defined as “deviations from the natural progression of an procedure”), teamwork scores (T-NOTECHS), and treatment times (total time in emergency department [ED]).
22. Train-the-trainer: Pilot trial for ebola virus disease simulation training
Tobin, C. D., Alfred, M., Wilson, D. A., MenkinSmith, L., Lehman-Huskamp, K. L., Schaefer, J. J., ... & Reves, J. G. (2020). Train-the-trainer: Pilot trial for ebola virus disease simulation training. Education for Health, 33(2), 37. https://www.educationforhealth.net/article.asp?issn=1357-6283;year=2020;volume=33;issue=2;spage=37; epage=45;aulast=Tobin
ABSTRACT
Highly infectious but rare diseases require rapid dissemination of safety critical skills to health-care workers (HCWs). Simulation is an effective method of education; however, it requires competent instructors. We evaluated the efficacy of an internet-delivered train-the-trainer course to prepare HCWs to care for patients with Ebola virus disease (EVD).
23. Work systems analysis of sterile processing: assembly.
Alfred, M., Catchpole, K., Huffer, E., Fredendall, L., & Taaffe, K. M. (2020). Work systems analysis of sterile processing: assembly. BMJ Quality & Safety. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7979531/
ABSTRACT
Sterile processing departments (SPDs) play a crucial role in surgical safety and efficiency. SPDs clean instruments to remove contaminants (decontamination), inspect and reorganise instruments into their correct trays (assembly), then sterilise and store instruments for future use (sterilisation and storage). However, broken, missing or inappropriately cleaned instruments are a frequent problem for surgical teams. These issues should be identified and corrected during the assembly phase.
24. Work systems analysis of sterile processing: decontamination.
Alfred, M., Catchpole, K., Huffer, E., Fredendall, L., & Taaffe, K. M. (2020). Work systems analysis of sterile processing: decontamination. BMJ Quality & Safety, 29(4), 320-328. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7752140/
ABSTRACT
Few studies have explored the work of sterile processing departments (SPD) from a systems perspective. Effective decontamination is critical for removing organic matter and reducing microbial levels from used surgical instruments prior to disinfection or sterilization and is delivered through a combination of human work and supporting technologies and processes. In this paper we report the results of a work systems analysis that sought to identify the complex multilevel interdependencies that create performance variation in decontamination and identify potential improvement interventions.
25. Surgical Performance and the Working Environment
Catchpole, K. R., & Alfred, M. C. (2020). Surgical Performance and the Working Environment. In Human Factors in Surgery (pp. 51-61). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-030-53127-0_6
ABSTRACT
This chapter explores some of the interactions between the surgical working environment, human performance, infection control, and other surgical outcomes. It considers the properties necessary for a room that can appropriately support surgery, from different functions that need to be served, through design considerations and necessary trade-offs, to ventilation, lighting, and acoustics.
26. Illuminating the decision-making strategies of anesthesia providers in challenging cases.
Biro, J., Neyens, D. M., Jaruzel, C., Tobin, C. D., Alfred, M., Coppola, S., ... & Catchpole, K. R. (2020, December). Illuminating the decision-making strategies of anesthesia providers in challenging cases. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 64, No. 1, pp. 653-657). Sage CA: Los Angeles, CA: SAGE Publications.
ABSTRACT
Medication errors and error-related scenarios in anesthesia remain an important area of research. Interventions and best practice recommendations in anesthesia are often based in the work-as-imagined healthcare system, remaining under-used due to a range of unforeseen complexities in healthcare work-as- done. In order to design adaptable anesthesia medication delivery systems, a better understanding of clinical cognition within the context of anesthesia work is needed.
27. Patient Journey Mapping: A Literature Review. In IIE Annual Conference.
Madathil, S. C., Lopes, A. J., & Alfred, M. (2020). Patient Journey Mapping: A Literature Review. In IIE Annual Conference. Proceedings (pp. 937-942). Institute of Industrial and Systems Engineers (IISE). https://www.proquest.com/docview/2511386708?pq-origsite=gscholar&fromopenview=true
ABSTRACT
Quality improvement efforts in healthcare have typically focused on reducing costs, decreasing wait time, or increasing regulatory compliance. Recent emphasis on patient- centered care, coupled with increasing competition among larger consolidated health systems to acquire new patients, provide the impetus to assess and enhance current patients' experience. Patient journey mapping (PJM) is a relatively novel method to evaluate the patient experience during or following the caregiving episodes. This article reviews various research papers on the patient journey mapping process, their methodologies, and outcomes achieved for patients, providers, and the health system.
28. Identifying and Reducing Errors in Perioperative Anesthesia Medication Delivery: The Role of Technology.
Schlee, C., Alfred, M., & Catchpole, K. (2020). Identifying and Reducing Errors in Perioperative Anesthesia Medication Delivery: The Role of Technology. Anesthesia and Analgesia (Vol. 130, No. 5, pp. 648).
29. A work systems analysis of sterile processing: sterilization and case cart preparation. In Structural Approaches to Address Issues in Patient Safety.
Alfred, M., Catchpole, K., Huffer, E., Taafe, K., & Fredendall, L. (2019). A work systems analysis of sterile processing: sterilization and case cart preparation. In Structural Approaches to Address Issues in Patient Safety. Emerald Publishing Limited. https://doi.org/10.1108/S1474-823120190000018008
ABSTRACT
Achieving reliable instrument reprocessing requires finding the right balance among cost, productivity, and safety. However, there have been few attempts to comprehensively examine sterile processing department (SPD) work systems. We considered an SPD as an example of a socio-technical system – where people, tools, technologies, the work environment, and the organization mutually interact – and applied work systems analysis (WSA) to provide a framework for future intervention and improvement.
30. Learning in simulated environments: An assessment of 4-week retention outcomes.
Alfred, M., Neyens, D. M., & Gramopadhye, A. K. (2019). Learning in simulated environments: An assessment of 4-week retention outcomes. Applied ergonomics, 74, 107-117. https://pubmed.ncbi.nlm.nih.gov/30487089/
ABSTRACT
Simulations offer the benefits of a safer and more accessible learning environment, where learners can practice until the point of proficiency. While research into the effectiveness of simulations as learning tools has found tangible benefits, fewer studies have examined retention and differences between high and low fidelity simulations. This research sought to supplement the literature in this domain by investigating whether participants who learned to construct an electrical circuit using a 2D or 3D breadboard simulation could achieve comparable learning, transfer, and retention outcomes to those who learned using a physical breadboard.
31. Industrial conceptualization of health care versus the naturalistic decision-making paradigm: Work as imagined versus work as done.
Catchpole, K., & Alfred, M. (2018). Industrial conceptualization of health care versus the naturalistic decision-making paradigm: Work as imagined versus work as done. Journal of Cognitive Engineering and Decision Making, 12(3), 222-226. https://doi.org/10.1177%2F1555343418774661
ABSTRACT
Quality and safety concerns in health care over the past 20 years precipitated the need to move beyond the traditional view of health care as an artisanal process toward a sociotechnical systems view of performance. The adoption of industrial approaches placed a greater emphasis on standardization of processes and outcomes, often treating humans as the “weak” part of the system rather than valuing their role in holding together complex, opaque, and unpredictable clinical systems. Although some health care tasks can be modeled linearly, others are much more complex. Efforts to reduce variation in clinical reasoning through evidence-based practices have proven problematic by failing to provide a means for context-specific adaptation or to account for the complex and adaptive nature of clinical work. We argue that the current, highly empirical approach to clinical decision making reflects clinical reasoning “as imagined,” whereas the application of the naturalistic decision-making (NDM) paradigm can help reveal clinical reasoning “as done.” This approach will have benefits for improving the conditions for diagnosis; the design of acute, time-pressured clinical work; the identification of deteriorating patients; the development of clinical decision support systems; and many more clinical tasks. Health care seems ready to accept NDM approaches.
32. Comparing learning outcomes in physical and simulated learning environments.
Alfred, M., Neyens, D. M., & Gramopadhye, A. K. (2018). Comparing learning outcomes in physical and simulated learning environments. International Journal of Industrial Ergonomics, 68, 110-117. https://doi.org/10.1016/j.ergon.2018.07.002
ABSTRACT
Quality and safety concerns in health care over the past 20 years precipitated the need to move beyond the traditional view of health care as an artisanal process toward a sociotechnical systems view of performance. The adoption of industrial approaches placed a greater emphasis on standardization of processes and outcomes, often treating humans as the “weak” part of the system rather than valuing their role in holding together complex, opaque, and unpredictable clinical systems. Although some health care tasks can be modeled linearly, others are much more complex. Efforts to reduce variation in clinical reasoning through evidence-based practices have proven problematic by failing to provide a means for context-specific adaptation or to account for the complex and adaptive nature of clinical work. We argue that the current, highly empirical approach to clinical decision making reflects clinical reasoning “as imagined,” whereas the application of the naturalistic decision-making (NDM) paradigm can help reveal clinical reasoning “as done.” This approach will have benefits for improving the conditions for diagnosis; the design of acute, time-pressured clinical work; the identification of deteriorating patients; the development of clinical decision support systems; and many more clinical tasks. Health care seems ready to accept NDM approaches.
33. A Novel Approach to Evaluating Simulation Scenarios.
Alfred, M., Schaefer J, Catchpole K, & Reves JG. A Novel Approach to Evaluating Simulation Scenarios. Anesthesia and Analgesia. 2018; 126 (4):719.
34. Investigating Intraoperative and Intraprofessional Handoffs in Anesthesia.
Jurewicz, K., Alfred, M., Neyens, D. M., Catchpole, K., Joseph, A., & Reeves, S. T. (2018, September). Investigating Intraoperative and Intraprofessional Handoffs in Anesthesia. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 62, No. 1, pp. 469-473). Sage CA: Los Angeles, CA: SAGE Publications.
ABSTRACT
Handoffs occur frequently in healthcare systems, and miscommunications and critical omissions within handoffs have been linked to serious medical errors. Handoff quality is a priority of patient safety initiatives across several international organizations. The majority of previous research in handoffs have focused on postoperative handoffs; yet, there is a need to specifically investigate intraoperative handoffs, especially within individual professions or subspecialties. Each subspecialty within a surgical team may approach handoffs differently. The anesthesia team is especially unique as they take a team-based approach to patient care where multiple anesthesia providers may be involved in one surgical case. We describe an observational study of intraoperative and intraprofessional handoffs in anesthesia. Temporary and permanent handoffs were investigated as well as the contextual elements that influenced handoff procedures. In a sample of 35 video recorded surgeries, a total of 16 handoffs were identified for the maintenance phase of surgery. These handoffs ranged between 33 seconds and 7.42 minutes in duration. Our study revealed variability in intraoperative handoffs during the maintenance phase of anesthesia and emphasizes that intraoperative and intraprofessional handoffs warrant more in-depth examination in order to develop effective strategies or tools for high quality handoffs in anesthesia.
35. A Pilot Trial of Online Simulation Training for Ebola Response Education.
MenkinSmith, L., Lehman-Huskamp, K., Schaefer, J., Alfred, M., Catchpole, K., Pockrus, B., ... & Reves, J. G. (2018). A Pilot Trial of Online Simulation Training for Ebola Response Education. Health security, 16(6), 391-401.
ABSTRACT
This article describes a pilot trial of an internet-distributable online software package that provides course materials and built-in evaluation tools to train healthcare workers in high-risk infectious disease response. It includes (1) an online self-study component, (2) a "hands-on" simulation workshop, and (3) a data-driven performance assessment toolset to support debriefing and course reporting. This study describes a pilot trial of the software package using a course designed to provide education in Ebola response to prepare healthcare workers to safely function as a measurable, high-reliability team in an Ebola simulated environment.
36. An Empirical Study Investigating the Effectiveness of Integrating Virtual Reality-based Case Studies into an Online Asynchronous Learning Environment
Madathil, K. C., Frady, K., Hartley, R., Bertrand, J., Alfred, M., & Gramopadhye, A. (2017). An empirical study investigating the effectiveness of integrating virtual reality-based case studies into an online asynchronous learning environment. Computers in Education Journal, 8(3), 1-10.
ABSTRACT
Widespread use of the Web and other Internet technologies in higher education has exploded in the last decade. Technology such as Virtual Reality (VR) has the potential to improve learning outcomes and student engagement in an active learning environment. This study investigates the extent to which VR-based education enhances learning outcomes and perceived engagement with technical curriculum.
37. Learning in Simulated Environments: A Comparison to Learning in a Physical Environments Using Video Data Analysis.
Alfred, M., Lee, M., Neyens, D. M., & Gramopadhye, A. K. (2017, September). Learning in Simulated Environments: A Comparison to Learning in a Physical Environments Using Video Data Analysis. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 61, No. 1, pp. 1839-1843). Sage CA: Los Angeles, CA: SAGE Publications.
ABSTRACT
Few studies that have evaluated the efficacy of learning in simulated environments have also examined differences in the processes that learners in the simulated environments employed to arrive at the outcomes being measured. In this video analysis study, the researchers sought to understand whether there were differences in the construction process used by participants who learned to construct circuits using a 2D simulation, a 3D simulation, or a physical breadboard and whether these differences subsequently impacted several learning outcomes. The researchers systematically reviewed 30 videos (10 videos per condition) of participants’ circuit construction to identify variations in their comprehension level and their construction procedures. The study found differences in the construction process, odds of successful construction, and construction time for participants in the three conditions that could be attributed to differences in fidelity. These findings provide valuable insights about how learning in simulated environments impact proficiency and learning and can help improve the design of simulated learning environments.
38. Understanding How the Physical Fidelity of the Learning Environment Shapes a Learner’s Mental Model of the Task under Study
Alfred, M., Lee, M., Neyens, D.M, Gramopadhye, A. (2016, June). Understanding How the Physical Fidelity of the Learning Environment Shapes a Learner’s Mental Model of the Task under Study, paper presented at the American Society for Engineering Education Conference, New Orleans, LA. 2016. 10.18260/p.27101.
ABSTRACT
The purpose of this research is to identify differences in the proficiency of students who learned how to construct a circuit on a breadboard under three different levels of physical fidelity – a 2D simulation, a virtual environment and a physical environment. In a previous study, the researchers identified differences in proficiency, defined by construction time, diagram accuracy, and correct circuit construction among participants in the three conditions. However, while providing valuable data about the outcomes achieved by the participants, the results of the study offered little insight into the processes or mechanisms through which the physical fidelity impacted the results obtained. In this follow-up study, we seek to evaluate why students demonstrated significant differences in proficiency between conditions and how these differences related to the fidelity.
39. The Impact of Training Method on Skill Acquisition and Transfer.
Alfred, M., Neyens, D. M., & Gramopadhye, A. K. (2015, September). The Impact of Training Method on Skill Acquisition and Transfer. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 59, No. 1, pp. 1563-1567). Sage CA: Los Angeles, CA: SAGE Publications.
ABSTRACT
Technology based learning tools are gaining traction in both educational and organizational settings. One of the primary advantages of these tools are that they offers a means for students and employees to practice a skill without safety risks and continue to practice until they obtain the necessary level of proficiency. However, there may be differences in the effectiveness of these tools based on the characteristics of the learner. The purpose of this study is to evaluate how an individual’s performance in a task differs depending on the physical fidelity of the learning tool. Additionally, this research examines the effects of cognitive ability, goal orientation, and motivation on performance. Specifically, this study investigates different methods for practicing building electric circuits using a virtual breadboard environment, a 2D breadboard simulation, and a physical breadboard. The study will also assess how the participants’ level of engagement impact performance outcomes using an electrodermal activity (EDA) device. It is expected that the learning tool will have an impact on task performance that is moderated by learner characteristics. The results of this study will have implications on the design and implementation of technology in both student and employee education.
40. An investigation of the factors that predict an Internet user’s perception of anonymity on the web.
Devaraj, S., Alfred, M., Madathil, K. C., & Gramopadhye, A. K. (2015, August). An investigation of the factors that predict an Internet user’s perception of anonymity on the web. In International Conference on Human Aspects of Information Security, Privacy, and Trust (pp. 311-322). Springer, Cham.
ABSTRACT
The growth of the Internet as a means of communication has sparked a need for researchers to investigate the issues surrounding different social behaviors associated with Internet use. Of particular interest is the importance of a user’s perception of anonymity. The independent variables for the study were demographic information, social networking habits and prior negative experience. The dependent variable for this study was perception of online anonymity. Data for this analysis were taken from the Pew Research Center’s Internet & American Life Project’s July 2013 Pew Internet Anonymity Survey. A binomial logistic regression analysis was performed to predict perception of anonymity on the Web. Results indicated that gender, income level, education level, social networking habits and compromised identity are significant in predicting one’s perception of anonymity on the web. Age and prior negative experience were not significant predictors. Differences in technological proficiency and access to the web are two factors believed to have contributed to these results, particularly those related to demographics. The findings from this research could be used to help target demographics with the education and support needed to protect their identity on the web. This study also offers insight about who are more likely to attempt to use the web anonymously and will help further identify the patterns of behavior associated with anonymous web use. This paper calls for further studies to analyze to what extent do the opinions and experiences of friends and relatives impact an individual’s perception of anonymity.