Emergence and development of the Delphi method: a scientometric perspective

Authors

DOI:

https://doi.org/10.5195/biblios.2018.470

Keywords:

Delphi Method, Delphi Technique, Scientific production, Scientometric, Scopus

Abstract

The present work aims to characterize the development of the Delphi method, based on scientific indicators related to publications with high quality indexes. As a result of an automated search in Scopus, a database of 10341 documents recorded since 1969, has been formed. Among the main results, it is observed an exponential growth of the scientific and technical activity, related with this method. The impact of the published documents becomes verified in an ample heterogeneity of fields of the scientific knowledge. Certain invisible fact-finding schools are identified, determined by complex networks of scientific collaboration. On the other hand, some new areas and development tendencies are also explored. Although the marked variety of terminologies related to the Delphi method hinders the processing of information, it is concluded that the scientometric techniques are effective to characterize the development of this method, even beyond the perspective framework.

Author Biographies

Miguel Cruz Ramírez, Universidad de Holguín - UHO

Doctor en ciencias pedagógicas y profesor titular de la Universidad de Holguín (Cuba). Vicerrector de Investigación y Posgrado. Miembro de la Junta Directiva Nacional de la Sociedad Cubana de Matemática y Computación. Sus intereses científicos están relacionados con la estadística aplicada a la cienciometría y a las investigaciones sociales.

José Alberto Rúa Vásquez, Universidad de Medellín - UDEM

Decano de la Facultad de Ciencias Básicas de la Universidad de Medellín (Colombia). Director del Proyecto Institucional Permanencia con Calidad. Coordinador General del Proyecto “Polya”. Matemático por la Universidad de Antioquia y máster en educación por la Pontificia Universidad Javeriana.

References

Archambault, E., & Larivière, V. (2010). The limits of bibliometrics for the analysis of the social sciences and humanities literature. In World Social Sciences Report (pp. 251-254). International Social Science Council: UNESCO Publishing. http://unesdoc.unesco.org/images/0018/001883/188333e.pdf

Bornmann, L., Thor, A., Marx, W., & Schier, H. (2016). The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute. Journal of the Association for Information Science and Technology, 67(11), 2778-2789. http://dx.doi.org/10.1002/asi.23627

Brown, B. B. (1970). La técnica Delfos. Metodología usada para obtener la opinión de los expertos. Revista Española de la Opinión Pública, 21/22, 217-226 (vid. Rand Corporation Paper P-3925, feb. 1968). http://dx.doi.org/10.2307/40181465

Caron, C., Roche, S., Goyer, D., & Jaton, A. (2008). GIScience journals ranking and evaluation: an international Delphi study. Transactions in GIS, 12(3), 293-321. http://dx.doi.org/10.1111/j.1467-9671.2008.01106.x

Coates, J. F. (1975). In defense of Delphi: A review of Delphi assessment, expert opinion, forecasting, and group process by H. Sackman. Technological Forecasting & Social Change, 7(2), 193-194. https://doi.org/10.1016/0040-1625(75)90058-X

Cruz-Ramírez, M., Escalona-Reyes, M., Cabrera-García, S., & Martínez-Cepena, M. C. (2014). Análisis cienciométrico de las publicaciones educacionales cubanas en la WoS y Scopus (2003-2012). Revista Española de Documentación Científica, 37(3), e058. http://dx.doi.org/10.3989/redc.2014.3.1119

Cummings, J. L., Mega, M., Gray, K., Rosenberg-Thompson, S., Carusi, D. A., & Gornbein, J. (1994). The neuropsychiatric inventory: Comprehensive assessment of psychopathology in dementia. Neurology, 44(12), 2308-2314. http://dx.doi.org/10.1212/WNL.44.12.2308

Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458-467. http://dx.doi.org/10.1287/mnsc.9.3.458

Davidson, P. L. (2013). The Delphi technique in doctoral research: Considerations and rationale. Review of Higher Education & Self-Learning, 6(22), 53-65. https://research.phoenix.edu/sites/default/files/publication-files/The%20Delphi%20technique%20in%20doctoral%20research_1.pdf

Dellinger, R. P., Levy, M. M., Carlet, J. M., Bion, J., Parker, M. M., Jaeschke, R., … Vincent, J.-L. (2008). Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock: 2008. Critical Care Medicine, 36(1), 296-327. http://dx.doi.org/10.1097/01.CCM.0000298158.12101.41

de Villiers, M. R., de Villiers, P. J. T., & Kent, A. P. (2005). The Delphi technique in health sciences education research. Medical Teacher, 27(7), 639-643. http://dx.doi.org/10.1080/13611260500069947

Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014). Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. Journal of Clinical Epidemiology, 67(4), 401-409. http://dx.doi.org/10.1016/j.jclinepi.2013.12.002

Donohoe, H., Stellefson, M., & Tennant, B. (2012). Advantages and limitations of the e-Delphi technique. American Journal of Health Education, 43(1), 38-46. http://dx.doi.org/10.1080/19325037.2012.10599216

Dror, Y. (1970). La predicción de lo políticamente posible. Revista Española de la Opinión Pública, 21/22, 89-98 (vid. Rand Corporation Paper P-4044, apr. 1969). http://dx.doi.org/10.2307/40181461

García-Aracil, A., & Palomares-Montero, D. (2012). Indicadores para la evaluación de las instituciones universitarias: validación a través del método Delphi. Revista Española de Documentación Científica, 35(1), 119-144. http://dx.doi.org/10.3989/redc.2012.1.863

Gargon, E., Gurung, B., Medley, N., Altman, D. G., Blazeby, J. M., Clarke, M., & Williamson, P. R. (2014). Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS ONE, 9(6), e99111. http://dx.doi.org/10.1371/journal.pone.0099111

Gear, C., Koziol-McLain, J., Wilson, D., Rae, N., Samuel, H., Clark, F., & McNeill, E. (2012). Primary healthcare response to family violence: A Delphi evaluation tool. Quality in Primary Care, 20(1), 15-30. https://www.ncbi.nlm.nih.gov/labs/articles/22584364/

Hallowell, M. R., & Gambatese, J. A. (2010). Qualitative research: Application of the Delphi method to CEM research. Journal of Construction Engineering and Management, 136(1), 99-107. http://dx.doi.org/10.1061/(ASCE)CO.1943-7862.0000137

Hasson, F., & Keeney, S. (2011). Enhancing rigour in the Delphi technique research. Technological Forecasting & Social Change, 78(9), 1695-1704. http://dx.doi.org/10.1016/j.techfore.2011.04.005

Herbohn, J., Harrison, S., & Emtage, N. (1999) Response to comments of Don Nicholson, Australian Forestry, 62(3), 285-286. http://dx.doi.org/10.1080/00049158.1999.10674794

Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44(2), 193-295. http://dx.doi.org/10.1007/BF02457380

Kaplan, A., Skogstad, A. L., & Girshick, M. A. (1950). The prediction of social and technological events. Public Opinion Quarterly, 14(1), 93-110. http://dx.doi.org/10.1086/266153

Kauko, K., & Palmroos, P. (2014). The Delphi method in forecasting financial markets. An experimental study. International Journal of Forecasting, 30(2), 313-327. http://dx.doi.org/10.1016/j.ijforecast.2013.09.007

Keeney, S., Hasson, F., & Mckenna, H. (2011). The Delphi Technique in Nursing and Health Research. Oxford: Wiley-Blackwell.

Landeta, J. (1999). El Método Delphi: Una Técnica de Previsión para la Incertidumbre. Barcelona: Ariel.

Landeta, J. (2011). Hybrid Delphi: A methodology to facilitate contribution from experts in professional contexts. Technological Forecasting & Social Change, 78(9), 1629-1641. http://dx.doi.org/10.1016/j.techfore.2011.03.009

Landeta, J., Barrutia, J., Hoyos, J., & Araujo, A. (2014). Initiatives for the improvement of continuous management training. Cuadernos de Gestión, 15(1), 61-92. http://dx.doi.org/10.5295/cdg.130411jl

Lin, V. S., & Song, H. (2014). A review of Delphi forecasting research in tourism. Current Issues in Tourism. http://dx.doi.org/10.1080/13683500.2014.967187

Linstone, H., & Turoff, M. (Eds.). (1975). The Delphi Method: Techniques and Applications. London: Addison-Wesley.

Linstone, H., & Turoff, M. (2011). Delphi: A brief look backward and forward. Technological Forecasting & Social Change, 78(9), 1712-1719. http://dx.doi.org/ 10.1016/j.techfore.2010.09.011

Lotka, A. J. (1926). The frequency distribution of scientific productivity. Journal of the Washington Academy of Sciences, 16(12), 317-323. http://www.jstor.org/stable/24529203

Meshkat, B., Cowman, S., Gethin, G., Ryan, K., Wiley, M., Brick, A., … Mulligan, M. (2014). Using an e-Delphi technique in achieving consensus across disciplines for developing best practice in day surgery in Ireland. Journal of Hospital Administration, 3(4), 1-8. http://dx.doi.org/10.5430/jha.v3n4p1

Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., Stroup, D. F., … Yusuf, S. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. The Lancet, 354(9193), 1896-1900. http://dx.doi.org/10.1016/S0140-6736(99)04149-5

Moreira, M. (2014). Técnica Delphi: metodologia para pesquisas em educação no Brasil. Revista de Educação. 19(1), 63-71. http://periodicos.puc-campinas.edu.br/seer/index.php/reveducacao/article/view/2616

Mullen, P. M. (2003). Delphi: myths and reality. Journal of Health Organization and Management, 17(1), 37-52. http://dx.doi.org/10.1108/14777260310469319

Noack, A. (2007). Energy models for graph clustering. Journal of Graph Algorithms and Applications, 11(2), 453-480. http://dx.doi.org/10.7155/jgaa.00154

Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management, 42(1), 15-29. http://dx.doi.org/10.1016/j.im.2003.11.002

Otte, E., & Rousseau, R. (2002). Social network analysis: a powerful strategy, also for the information sciences. Journal of Information Science, 28(6), 441-453, http://dx.doi.org/10.1177/016555150202800601

Papadimitriou, F., & Kidman, G. (2012). Statistical and scientometric analysis of international research in geographical and environmental education. International Research in Geographical and Environmental Education, 21(1), 11-20. http://dx.doi.org/10.1080/10382046.2012.639153

Powell, C. (2003). The Delphi technique: myths and realities. Journal of Advanced Nursing, 41(4), 376-382. http://dx.doi.org/10.1046/j.1365-2648.2003.02537.x

Price, D. J. S. (1963). Little Science, Big Science. New York: Columbia University Press.

Qureshi, M. I., Bhatti, M. N., Rasli, A. M., Yasir, M., & Zaman, K. (2014). The Delphi method for internationalization of higher education in Pakistan: integrating theory of constraints and quality function deployment. Mediterranean Journal of Social Sciences, 5(20), 2702-2710. http://dx.doi.org/10.5901/mjss.2014.v5n20p270

Rekalde, I., Landeta, J., & Albizu, E. (2015). Determining factors in the effectiveness of executive coaching as a management development tool. Management Decision, 53(8), pp. 1-39. http://dx.doi.org/10.1108/MD-12-2014-0666

Rieger, W. (1986). Directions in Delphi developments: dissertations and their quality. Technological Forecasting & Social Change, 29(2), 195-204. http://dx.doi.org/10.1016/0040-1625(86)90063-6

Robson, N., & Rew, D. (2010). Collective wisdom and decision making in surgical oncology. European Journal of Surgical Oncology, 36(3), 230-236. http://dx.doi.org/10.1016/j.ejso.2010.01.002

Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15(4), 353-375. http://dx.doi.org/10.1016/S0169-2070(99)00018-7

Sackman, H. (1974). Delphi Assessment: Expert Opinion, Forecasting, and Group Process. Santa Monica: Rand Corporation Paper R-1283-PR. http://www.rand.org/pubs/reports/2006/R1283.pdf

Tsai, C.-C., Corley, E. A., & Bozeman, B. (2016). Collaboration experiences across scientific disciplines and cohorts. Scientometrics, 108(2), 505-529. http://dx.doi.org/10.1007/s11192-016-1997-z

Vallejo, M., Ocaña, A., Bueno, Á., Torralbo, M., & Fernández, A. (2005). Producción científica sobre educación multicultural contenida en las bases de datos Social Science Citation Index y Arts y Humanities Citation Index (1956-2003). Revista Española de Documentación Científica, 28(2), 206-220. http://dx.doi.org/10.3989/redc.2005.v28.i2.169

Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84(2), 523-538. http://dx.doi.org/10.1007/s11192-009-0146-3

Vollmar, H. C., Ostermann, T., & Redaèlli, M. (2015). Using the scenario method in the context of health and health care: a scoping review. BMC Medical Research Methodology, 15(1), 1-10. http://dx.doi.org/10.1186/s12874-015-0083-1

von der Gracht, H. A. (2012). Consensus measurement in Delphi studies. Review and implications for future quality assurance. Technological Forecasting & Social Change, 79(8), 1525-1536. http://dx.doi.org/ 10.1016/j.techfore.2012.04.013

Xiao, J., Douglas, D., Lee, A. H., & Vemuri, S. R. (1997). A Delphi evaluation of the factors influencing length of stay in Australian hospitals. International Journal of Health Planning and Management, 12(3), 207-218. http://dx.doi.org/10.1002/(SICI)1099-1751(199707/09)12:3<207::AID-HPM480>3.0.CO;2-V

Zang, L., Fan, N., Hai, Y., Lu, S. B., Su, Q. J., Yang, J. C., … Gao, Y. J. (2015). Using the modified Delphi method to establish a new Chinese clinical consensus of the treatments for cervical radiculopathy. European Spine Journal, 24(6), 1116-1126. http://dx.doi.org/10.1007/s00586-015-3856-7

Published

2018-07-02

How to Cite

Cruz Ramírez, M., & Rúa Vásquez, J. A. (2018). Emergence and development of the Delphi method: a scientometric perspective. Biblios Journal of Librarianship and Information Science, (71), 90–107. https://doi.org/10.5195/biblios.2018.470

Issue

Section

Original

Most read articles by the same author(s)