Document Type : مقاله مروری نقلی
Author
Assistant Professor, Library and Information Studies, Kharazmi University, Tehran, Iran
Abstract
The present review article aimed to identify, introduce and compare models and frameworks used
for the impact assessment of medical research. Comprehensive searches were conducted in
various databases in order to identify models and frameworks. Those works that proposed a
model or framework were chosen for the review and comparison. Thirteen different models and
frameworks were identified. The level of assessment, items that models evaluate, and the type of
impact they assess showed some differences. In terms of the type of impact, Research Excellence
Framework was the only framework to cover all social, economic, cultural, and scientific aspects.
There is still not a single standard model to assess the impact of medical research. However,
among the proposed models, the payback model appears to be more widely used and accepted.
There is also ambiguity in the utilizations of the terms “impact” and “outcome” and in some
works they have been used interchangeably.
Keywords
1. Jamali HR. A model for research impact assessment of Iranian medical research [Project]. Tehran, Iran: Iranian
Academy of Medical Sciences; 2012.
2. Jamali Mehmoei HR. Research Evaluation: Approaches, Methods and Challenges. Rahyaft 2011; (49): 39-52.
[In Persian].
3. Kostoff RN. Federal research impact assessment: State-of-the-art. Journal of the American Society for
Information Science 1994; 45(6): 440-28.
4. Hanney S, Griffiths P. Ways of assessing the economic value or impact of research: is it a step too far for nursing
research? Journal of Research in Nursing 2011; 16(2): 151-66.
5. Walter I, Nutley S, Davies H. Research impact: A cross sector review literature review [Online]. 2003; Available
from: URL: http://www.st-andrews.ac.uk/~ruruweb/pdf/LSDA%20literature%20review%20final.pdf/
6. Buxton M, Hanney S, Jones T. Estimating the economic value to societies of the impact of health research: a
critical review. Bulletin of the World Health Organization 2004; 82(10): 733-9.
7. Boaz A, Fitzpatrick S, Shaw B. Assessing the impact of research on policy: A review of the literature for a
project on bridging research and policy through outcome evaluation [Project]. London, UK: Kings College
London & Policy Studies Institute; 2008. 2012.
8. Brutscher PB, Wooding S, Grant J. Health Research Evaluation Frameworks, an International Comparison
[Online]. 2008; Available from: URL: http://www.rand.org/pubs/technical_reports/TR629.html/
9. Yazdizadeh B, Majdzadeh R, Salmasian H. Systematic review of methods for evaluating healthcare research
economic impact. Health Research Policy and Systems 2010; 8(1): 6.
10.Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy
1996; 1(1): 35-43.
11.Kalucy L, McIntyre E, Jackson-Bowers E. Primary health care research impact project [Project]. Adelaide,
Austuralian: Flinders University; 2007.
12.Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the NHS Health
Technology Assessment Programme. Health Technol Assess 2007; 11(53): iii-xi, 1. 13. Lavis J, Ross S, McLeod C, Gildiner A. Measuring the impact of health research. J Health Serv Res Policy 2003;
8(3): 165-70.
14.CIHR Framework to Measure the Impact of Health Research [Online]. 2007; Available from: URL:
http://www.vr.se/download/18.5ec4ab911161ebf14dc80005426/CIHR+Borbey.pdf/
15.Kuruvilla S, Mays N, Pleasant A, Walt G. Describing the impact of health research: a Research Impact
Framework. BMC Health Serv Res 2006; 6: 134.
16.Kuruvilla S, Mays N, Walt G. Describing the impact of health services and policy research. J Health Serv Res
Policy 2007; 12 (Suppl 1): S1-31.
17. The societal impact of applied health research Council for Medical Sciences Towards a quality assessment
system [Project]. Amsterdam, Netherlands: Royal Netherlands Academy of Arts and Sciences; 2002.
18. Smith R. Measuring the social impact of research. BMJ 2001; 323(7312): 528.
19.Van Ark G. Societal impact evaluation of research groups: The communication metaphor [Online]. 2007;
Available from: URL: http://www.vr.se/download/18.34261071168fe6a62080001004/ZonMw+van+Ark+2.pdf/
20.HEFCE. Research Excellence Framework. Higher Education Funding Council for England [Online]. 2011
[cited 2011 Jul 7]; Available from: http://www.hefce.ac.uk/research/ref/
21.Weiss AP. Measuring the impact of medical research: moving from outputs to outcomes. Am J Psychiatry 2007;
164(2): 206-14.
22. Foundation K. Logic Model Development Guide. Battle Creek, MI: W.K. Kellogg Foundation; 2003.
23. Liebow E, Phelps J, Van HB, Rose S, Orians C, Cohen J, et al. Toward the assessment of scientific and public
health impacts of the National Institute of Environmental Health Sciences Extramural Asthma Research Program
using available data. Environ Health Perspect 2009; 117(7): 1147-54.
24.MRC. Economic Impact Reporting Framework, Medical Research Council. Medical Research Council UK
[Online]. 2011 [cited 2011 Jul 7]; Available from: URL:
http://www.mrc.ac.uk/Newspublications/Publications/EIRF/index.htm/
25. Trochim WM, Marcus SE, Mâsse LC, Moser RP, Weld PC. The Evaluation of Large Research Initiatives: A
Participatory Integrative Mixed-Methods Approach. American Journal of Evaluation 2008; 29(1): 8-28.
26. The Becker Medical Library Model for Assessment of Research Impact [Online]. 2011 [cited 2011 Jul 7];
Available from: URL: http://becker.wustl.edu/impact/assessment/index.html/
27. Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact. J Med
Libr Assoc 2010; 98(1): 17-23.
28.MSFHR. Performance Measurement and Evaluation Framework. Michael Smith Foundation for Health Research;
2011 [cited 2011 Sep 23]; Available from: URL: http://www.msfhr.org/about/monitoring_evaluation/
29.Canadian Academy of Health Sciences (CAHS). Making an Impact: A Preferred Framework and Indicators to
Measure Returns on Investment in Health Research [Online]. 2009; Available from: URL: http://www.cahsacss.
ca/making-an-impact-a-preferred-framework-and-indicators-to-measure-returns-on-investment-in-healthresearch-
8/
30.ARC. Excellence in Research for Australia (ERA) Initiative, consultation paper: Australian Research Council
[Online]. 2008. Available from: URL: http://www.arc.gov.au/pdf/ERA_ConsultationPaper.pdf/
31. Frank C, Nason E. Health research: measuring the social, health and economic benefits. CMAJ 2009; 180 (5):
528-34.
32. Jamali HR. Evaluation of medical research with focus on research impact. Hakim. [In Press]. [In Persian].
33.Godin B, Dore C. Measuring the impacts of science: beyond the economic dimension. Montreal, ON: CSIIC;
2004.
34.Canadian Health Services Research Foundation I-AI4. Measuring the Impact of Research: What do we know?
[Online]. 2008; Available from: URL: www.http://blog.openmedicine.ca/node/186/
35.Bernstein A, Hicks V, Borbey P, Campbell T. A framework to measure the impact of investments in health
research. OECD Blue Sky II Forum [Online]. 2006 [cited 2006 Sep 25]; Available from: URL:
http://www.oecd.org/dataoecd/10/42/37450246.pdf/
36.Hailey D, Grimshaw J, Eccles M, Mitton C, Adair CE, McKenzie E, et al. Effective Dissemination of Findings
from Research - a compilation of essays [Online]. 2008; Available from: URL:
http://www.ihe.ca/publications/library/2008/effective-dissemination-of-findings-from-research/
Academy of Medical Sciences; 2012.
2. Jamali Mehmoei HR. Research Evaluation: Approaches, Methods and Challenges. Rahyaft 2011; (49): 39-52.
[In Persian].
3. Kostoff RN. Federal research impact assessment: State-of-the-art. Journal of the American Society for
Information Science 1994; 45(6): 440-28.
4. Hanney S, Griffiths P. Ways of assessing the economic value or impact of research: is it a step too far for nursing
research? Journal of Research in Nursing 2011; 16(2): 151-66.
5. Walter I, Nutley S, Davies H. Research impact: A cross sector review literature review [Online]. 2003; Available
from: URL: http://www.st-andrews.ac.uk/~ruruweb/pdf/LSDA%20literature%20review%20final.pdf/
6. Buxton M, Hanney S, Jones T. Estimating the economic value to societies of the impact of health research: a
critical review. Bulletin of the World Health Organization 2004; 82(10): 733-9.
7. Boaz A, Fitzpatrick S, Shaw B. Assessing the impact of research on policy: A review of the literature for a
project on bridging research and policy through outcome evaluation [Project]. London, UK: Kings College
London & Policy Studies Institute; 2008. 2012.
8. Brutscher PB, Wooding S, Grant J. Health Research Evaluation Frameworks, an International Comparison
[Online]. 2008; Available from: URL: http://www.rand.org/pubs/technical_reports/TR629.html/
9. Yazdizadeh B, Majdzadeh R, Salmasian H. Systematic review of methods for evaluating healthcare research
economic impact. Health Research Policy and Systems 2010; 8(1): 6.
10.Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy
1996; 1(1): 35-43.
11.Kalucy L, McIntyre E, Jackson-Bowers E. Primary health care research impact project [Project]. Adelaide,
Austuralian: Flinders University; 2007.
12.Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the NHS Health
Technology Assessment Programme. Health Technol Assess 2007; 11(53): iii-xi, 1. 13. Lavis J, Ross S, McLeod C, Gildiner A. Measuring the impact of health research. J Health Serv Res Policy 2003;
8(3): 165-70.
14.CIHR Framework to Measure the Impact of Health Research [Online]. 2007; Available from: URL:
http://www.vr.se/download/18.5ec4ab911161ebf14dc80005426/CIHR+Borbey.pdf/
15.Kuruvilla S, Mays N, Pleasant A, Walt G. Describing the impact of health research: a Research Impact
Framework. BMC Health Serv Res 2006; 6: 134.
16.Kuruvilla S, Mays N, Walt G. Describing the impact of health services and policy research. J Health Serv Res
Policy 2007; 12 (Suppl 1): S1-31.
17. The societal impact of applied health research Council for Medical Sciences Towards a quality assessment
system [Project]. Amsterdam, Netherlands: Royal Netherlands Academy of Arts and Sciences; 2002.
18. Smith R. Measuring the social impact of research. BMJ 2001; 323(7312): 528.
19.Van Ark G. Societal impact evaluation of research groups: The communication metaphor [Online]. 2007;
Available from: URL: http://www.vr.se/download/18.34261071168fe6a62080001004/ZonMw+van+Ark+2.pdf/
20.HEFCE. Research Excellence Framework. Higher Education Funding Council for England [Online]. 2011
[cited 2011 Jul 7]; Available from: http://www.hefce.ac.uk/research/ref/
21.Weiss AP. Measuring the impact of medical research: moving from outputs to outcomes. Am J Psychiatry 2007;
164(2): 206-14.
22. Foundation K. Logic Model Development Guide. Battle Creek, MI: W.K. Kellogg Foundation; 2003.
23. Liebow E, Phelps J, Van HB, Rose S, Orians C, Cohen J, et al. Toward the assessment of scientific and public
health impacts of the National Institute of Environmental Health Sciences Extramural Asthma Research Program
using available data. Environ Health Perspect 2009; 117(7): 1147-54.
24.MRC. Economic Impact Reporting Framework, Medical Research Council. Medical Research Council UK
[Online]. 2011 [cited 2011 Jul 7]; Available from: URL:
http://www.mrc.ac.uk/Newspublications/Publications/EIRF/index.htm/
25. Trochim WM, Marcus SE, Mâsse LC, Moser RP, Weld PC. The Evaluation of Large Research Initiatives: A
Participatory Integrative Mixed-Methods Approach. American Journal of Evaluation 2008; 29(1): 8-28.
26. The Becker Medical Library Model for Assessment of Research Impact [Online]. 2011 [cited 2011 Jul 7];
Available from: URL: http://becker.wustl.edu/impact/assessment/index.html/
27. Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact. J Med
Libr Assoc 2010; 98(1): 17-23.
28.MSFHR. Performance Measurement and Evaluation Framework. Michael Smith Foundation for Health Research;
2011 [cited 2011 Sep 23]; Available from: URL: http://www.msfhr.org/about/monitoring_evaluation/
29.Canadian Academy of Health Sciences (CAHS). Making an Impact: A Preferred Framework and Indicators to
Measure Returns on Investment in Health Research [Online]. 2009; Available from: URL: http://www.cahsacss.
ca/making-an-impact-a-preferred-framework-and-indicators-to-measure-returns-on-investment-in-healthresearch-
8/
30.ARC. Excellence in Research for Australia (ERA) Initiative, consultation paper: Australian Research Council
[Online]. 2008. Available from: URL: http://www.arc.gov.au/pdf/ERA_ConsultationPaper.pdf/
31. Frank C, Nason E. Health research: measuring the social, health and economic benefits. CMAJ 2009; 180 (5):
528-34.
32. Jamali HR. Evaluation of medical research with focus on research impact. Hakim. [In Press]. [In Persian].
33.Godin B, Dore C. Measuring the impacts of science: beyond the economic dimension. Montreal, ON: CSIIC;
2004.
34.Canadian Health Services Research Foundation I-AI4. Measuring the Impact of Research: What do we know?
[Online]. 2008; Available from: URL: www.http://blog.openmedicine.ca/node/186/
35.Bernstein A, Hicks V, Borbey P, Campbell T. A framework to measure the impact of investments in health
research. OECD Blue Sky II Forum [Online]. 2006 [cited 2006 Sep 25]; Available from: URL:
http://www.oecd.org/dataoecd/10/42/37450246.pdf/
36.Hailey D, Grimshaw J, Eccles M, Mitton C, Adair CE, McKenzie E, et al. Effective Dissemination of Findings
from Research - a compilation of essays [Online]. 2008; Available from: URL:
http://www.ihe.ca/publications/library/2008/effective-dissemination-of-findings-from-research/
)