Key Performance Indicators of Benchmarking in Hospital Information Systems

Document Type : Original Article

Authors
Asghar Ehteshami 1
Ahmad Reza Raeisi 2
Maedeh Rashedi 3
1 Assistant Professor, Health Information Management, Health Information Technology Research Center AND Department of Health Information Technology, School of School of Management and Medical Information, Isfahan University of Medical Sciences, Isfahan, Iran
2 Associate Professor, Health Information Management, Health Management and Economic Research Center AND Department of Health Care Management, School of School of Management and Medical Information, Isfahan University of Medical Sciences, Isfahan, Iran
3 MSc, Health Information Technology, Health Information Technology Research Center AND Department of Health Information Technology, School of School of Management and Medical Information, Isfahan University of Medical Sciences, Isfahan, Iran
10.22122/him.v15i1.3341
Abstract
Introduction: Monitoring hospital information systems requires developing key indicators and criteria for efficiency and effectiveness, and consequently fulfillment of organizational objectives, including safety and quality. The purpose of this study was benchmarking of key indicators of hospital information systems.Methods: This research was based on qualitative methodology with two phases. It was carried out in Ahwaz City, Iran, during the years 2015 and 2016. In the first phase, we conducted purposeful sampling using snowball technique. In the second phase, ranking of the key indicators of benchmarking operation of hospital information system (HIS) was conducted using Delphi technique. The method of data analysis in the first phase was based on a thematic content analysis, and primary and secondary themes were extracted through descriptive, interpretative, and explanatory coding. For the second phase, SPSS software was used to analyze quantitative findings and their ranking.Results: After performing 14 semi-structured interviews with the study participants, the key benchmarking indicators for hospital information system were identified with 8 main themes including hardware, software, support services, vendors, output quality, workflow process, design/interface, and costs, and 76 subthemes, prioritized through Delphi method.Conclusion: Benchmarking intended for comparison and improvement of the various hospital systems such as hospital information system, is one of the very helpful instruments in the hands of the managers working in this field. The indicators identified during this research will provide a comprehensive tool for benchmarking of hospital information system. By applying these benchmarks continuously, there is a good possibility to improve the operation of such systems.

Keywords

  1. Kuhn KA, Giuse DA. From hospital information systems to health information systems. Problems, challenges, perspectives. Methods Inf Med 2001; 40(4): 275-87.
  2. Lora A. Call for information, call for quality in mental health care. Epidemiol Psychiatr Sci 2013; 22(1): 9-13.
  3. Verbeke F, Karara G, Nyssen M. Evaluating the impact of ICT-tools on health care delivery in sub-saharan hospitals. In: Lehmann CU, Ammenwerth E, Nohr C, Editors. MEDINFO 2013: Proceedings of the 14th world congress on medical and health informatics, Part 1.Amsterdam, Netherlands: IOS Press; 2013. p. 520-3.
  4. Friedman CP, Wyatt JC, Owens DK. Evaluation and technology assessment. In: Shortliffe EH, Cimino JJ, Editors. Biomedical informatics: Computer applications in health care and biomedicine. Berlin, Germany: Springer Science & Business Media; 2006. p. 403-43.
  5. National Quality Forum. NQF Patient Safety Terms and Definitions [Online]. [cited 2007]; Available from: URL:
  6. https://www.qualityforum.org/Topics/Safety_Definitions.aspx
  7. Otieno GO, Hinako T, Motohiro A, Daisuke K, Keiko N. Measuring effectiveness of electronic medical records systems: Towards building a composite index for benchmarking hospitals. Int J Med Inform 2008; 77(10): 657-69.
  8. Karban P, Kropik P, Kotlan V, Dolezel I. Bayes approach to solving T.E.A.M. benchmark problems 22 and 25 and its comparison with other optimization techniques. Appl Math Comput 2018; 319: 681-92.
  9. Emadi A, Ghoochani M, Zarei J, Mohammadi A, Hemmat M, Valinejadi A. Evaluation of hospital information system performance from the perspective of users in educational hospitals. International Journal of Computer Science and Network Security 2017; 17(7): 336-40.
  10. Sockolow PS, Bowles KH, Rogers M. Health information technology evaluation framework (HITREF) Comprehensiveness as assessed in electronic point-of-care documentation systems evaluations. Stud Health Technol Inform 2015; 216: 406-9.
  11. Jahn F, Winter A. A KPI framework for process-based benchmarking of hospital information systems. Stud Health Technol Inform 2011; 169: 542-6.
  12. Hubner-Bloder G, Ammenwerth E. Key performance indicators to benchmark hospital information systems-a Delphi study. Methods Inf Med 2009; 48(6): 508-18.
  13. Dugas M, Eckholt M, Bunzemeier H. Benchmarking of hospital information systems: Monitoring of discharge letters and scheduling can reveal heterogeneities and time trends. BMC Med Inform Decis Mak 2008; 8: 15.
  14. Ehteshami A. Barcode technology acceptance and utilization in health information management department at academic hospitals according to technology acceptance model. Acta Inform Med 2017; 25(1): 4-8.
  15. Currie LM. Evaluation frameworks for nursing informatics. Int J Med Inform 2005; 74(11-12): 908-16.
  16. Kaplan B, Shaw NT. Future directions in evaluation research: People, organizational, and social issues. Methods Inf Med 2004; 43(3): 215-31.
  17. Janes A, Lenarduzzi V, Stan AC. A continuous software quality monitoring approach for small and medium enterprises. Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE); 2017 Apr. 22-26; L'Aquila, Italy.
  18. Ehteshami A, Sadoughi F, Saeedbakhsh S, Isfahani MK. Assessment of medical records module of health information system according to ISO 9241-10. Acta Inform Med 2013; 21(1): 36-41.
  19. Fernando JI, Dawson LL. The health information system security threat lifecycle: An informatics theory. Int J Med Inform 2009; 78(12): 815-26.
  20. Amiresmaili M, Zarei L, Sheibani E, Arabpur A. Evaluation of the indicators of hospital information system. Health Inf Manage 2013; 10(1): 1-13. [In Persian].
  21. Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente's experience of implementing an electronic medical record: A qualitative study. BMJ 2005; 331(7528): 1313-6.
  22. Lin WT, Shao BBM. The relationship between user participation and system success: A simultaneous contingency approach. Information & Management 2000; 37(6): 283-95.
  23. Mazzoleni MC, Baiardi P, Giorgi I, Franchi G, Marconi R, Cortesi M. Assessing users' satisfaction through perception of usefulness and ease of use in the daily interaction with a hospital information system. Proc AMIA Annu Fall Symp 1996; 752-6.
  24. Lindgaard G, Dillon R, Trbovich P, White R, Fernandes G, Lundahl S, et al. User Needs Analysis and requirements engineering: Theory and practice. Interact Comput 2006; 18(1): 47-70.
  25. McCarthy C, Eastman D. Change management strategies for an effective EMR implementation. Chicago, IL: Healthcare Information and Management Systems Society (HIMSS); 2010.
  26. Farzandi Pour M, Meidani Z, Gilasi HR, Dehghan Banadaki R. Ranking of hospital information systems based on requirements of Iran in 2013. Journal of Modern Medical Information Sciences 2015; 1(1): 1-9. [In Persian].
  27. Palacio C, Harrison JP, Garets D. Benchmarking electronic medical records initiatives in the US: A conceptual model. J Med Syst 2010; 34(3): 273-9.
  28. Shah Moradi L, Ahmadi M, Haghani H. Defining evaluation indicators of health information systems and a model presentation. J Health Adm 2007; 10(28): 15-24. [In Persian].
  29. Hamborg KC, Vehse B, Bludau HB. Questionnaire based usability evaluation of hospital information systems. Electronic Journal of Information Systems Evaluation 2004; 7(1): 21-30.
  30. Sadoughi F, Kimiafar K, Ahmadi M, Shakeri MT. Determining of factors influencing the success and failure of hospital information system and their evaluation methods: A systematic review. Iran Red Crescent Med J 2013; 15(12): e11716.
  31. Madani G, Farzan A, Rabiee M. Patient satisfaction of medical and nursing services. Iran J Nurs Midwifery Res 2004; 9(3).
  32. Ribiere V, LaSalle AJ, Khorramshahgol R, Gousty Y. Hospital information systems quality: A customer satisfaction assessment tool. Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences; 1999 Jan. 5-8; Maui, HI.
  33. Moradi GR, Sarbaz M, Kimiafar KH, Shafiei N, Setayesh Y. The role of hospital information system on dr sheikh hospital performance promotion in Mashhad. Health Inf Manage 2009; 5(2): 159-66. [In Persian].
  34. Aggelidis VP, Chatzoglou PD. Methods for evaluating hospital information systems: A literature review. J Bus 2008; 3(1): 99-118.
  35. Mokhtaripour M, Siadat SA. Information technology in the hospitals of Isfahan: Suggesting a model. Health Inf Manage 2008; 5(1): 8-12. [In Persian].
  36. Kazanjian A, Green CJ. Beyond effectiveness: The evaluation of information systems using a comprehensive health technology assessment framework. Comput Biol Med 2002; 32(3): 165-77.
  37. Borzekowski R. Measuring the cost impact of hospital information systems: 1987-1994. J Health Econ 2009; 28(5): 938-49.
  38. Jahanbakhsh M, Ehteshami A, Shafiei A. Assessment of the hospital information system in compliance with certification commission for healthcare information technology standard at Isfahan University of Medical Sciences' academic hospitals. Int J EducPsychol Res 2016; 2(2): 105-10.
  39. DeLone WH, McLean ER. Information systems success: The quest for the dependent variable. Inf Syst Res 1992; 3(1): 60-95.
  40. Seddon PB. A respecification and extension of the DeLone and McLean model of IS success. Inf Syst Res 1997; 8(3): 240-53.
  41. Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: Human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008; 77(6): 386-98.