Emergence and development of the Delphi method: a scientometric perspective

Authors

DOI:

https://doi.org/10.5195/biblios.2018.470

Keywords:

Delphi Method, Delphi Technique, Scientific production, Scientometric, Scopus

Abstract

The present work aims to characterize the development of the Delphi method, based on scientific indicators related to publications with high quality indexes. As a result of an automated search in Scopus, a database of 10341 documents recorded since 1969, has been formed. Among the main results, it is observed an exponential growth of the scientific and technical activity, related with this method. The impact of the published documents becomes verified in an ample heterogeneity of fields of the scientific knowledge. Certain invisible fact-finding schools are identified, determined by complex networks of scientific collaboration. On the other hand, some new areas and development tendencies are also explored. Although the marked variety of terminologies related to the Delphi method hinders the processing of information, it is concluded that the scientometric techniques are effective to characterize the development of this method, even beyond the perspective framework.

Author Biographies

Miguel Cruz Ramírez, Universidad de Holguín - UHO

Doctor en ciencias pedagógicas y profesor titular de la Universidad de Holguín (Cuba). Vicerrector de Investigación y Posgrado. Miembro de la Junta Directiva Nacional de la Sociedad Cubana de Matemática y Computación. Sus intereses científicos están relacionados con la estadística aplicada a la cienciometría y a las investigaciones sociales.

José Alberto Rúa Vásquez, Universidad de Medellín - UDEM

Decano de la Facultad de Ciencias Básicas de la Universidad de Medellín (Colombia). Director del Proyecto Institucional Permanencia con Calidad. Coordinador General del Proyecto “Polya”. Matemático por la Universidad de Antioquia y máster en educación por la Pontificia Universidad Javeriana.

References

Archambault, E., & Larivière, V. (2010). The limits of bibliometrics for the analysis of the social sciences and humanities literature. In World Social Sciences Report (pp. 251-254). International Social Science Council: UNESCO Publishing. http://unesdoc.unesco.org/images/0018/001883/188333e.pdf

Bornmann, L., Thor, A., Marx, W., & Schier, H. (2016). The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute. Journal of the Association for Information Science and Technology, 67(11), 2778-2789. http://dx.doi.org/10.1002/asi.23627

Brown, B. B. (1970). La técnica Delfos. Metodología usada para obtener la opinión de los expertos. Revista Española de la Opinión Pública, 21/22, 217-226 (vid. Rand Corporation Paper P-3925, feb. 1968). http://dx.doi.org/10.2307/40181465

Caron, C., Roche, S., Goyer, D., & Jaton, A. (2008). GIScience journals ranking and evaluation: an international Delphi study. Transactions in GIS, 12(3), 293-321. http://dx.doi.org/10.1111/j.1467-9671.2008.01106.x

Coates, J. F. (1975). In defense of Delphi: A review of Delphi assessment, expert opinion, forecasting, and group process by H. Sackman. Technological Forecasting & Social Change, 7(2), 193-194. https://doi.org/10.1016/0040-1625(75)90058-X

Cruz-Ramírez, M., Escalona-Reyes, M., Cabrera-García, S., & Martínez-Cepena, M. C. (2014). Análisis cienciométrico de las publicaciones educacionales cubanas en la WoS y Scopus (2003-2012). Revista Española de Documentación Científica, 37(3), e058. http://dx.doi.org/10.3989/redc.2014.3.1119

Cummings, J. L., Mega, M., Gray, K., Rosenberg-Thompson, S., Carusi, D. A., & Gornbein, J. (1994). The neuropsychiatric inventory: Comprehensive assessment of psychopathology in dementia. Neurology, 44(12), 2308-2314. http://dx.doi.org/10.1212/WNL.44.12.2308

Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458-467. http://dx.doi.org/10.1287/mnsc.9.3.458

Davidson, P. L. (2013). The Delphi technique in doctoral research: Considerations and rationale. Review of Higher Education & Self-Learning, 6(22), 53-65. https://research.phoenix.edu/sites/default/files/publication-files/The%20Delphi%20technique%20in%20doctoral%20research_1.pdf

Dellinger, R. P., Levy, M. M., Carlet, J. M., Bion, J., Parker, M. M., Jaeschke, R., … Vincent, J.-L. (2008). Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock: 2008. Critical Care Medicine, 36(1), 296-327. http://dx.doi.org/10.1097/01.CCM.0000298158.12101.41

de Villiers, M. R., de Villiers, P. J. T., & Kent, A. P. (2005). The Delphi technique in health sciences education research. Medical Teacher, 27(7), 639-643. http://dx.doi.org/10.1080/13611260500069947

Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014). Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. Journal of Clinical Epidemiology, 67(4), 401-409. http://dx.doi.org/10.1016/j.jclinepi.2013.12.002

Donohoe, H., Stellefson, M., & Tennant, B. (2012). Advantages and limitations of the e-Delphi technique. American Journal of Health Education, 43(1), 38-46. http://dx.doi.org/10.1080/19325037.2012.10599216

Dror, Y. (1970). La predicción de lo políticamente posible. Revista Española de la Opinión Pública, 21/22, 89-98 (vid. Rand Corporation Paper P-4044, apr. 1969). http://dx.doi.org/10.2307/40181461

García-Aracil, A., & Palomares-Montero, D. (2012). Indicadores para la evaluación de las instituciones universitarias: validación a través del método Delphi. Revista Española de Documentación Científica, 35(1), 119-144. http://dx.doi.org/10.3989/redc.2012.1.863

Gargon, E., Gurung, B., Medley, N., Altman, D. G., Blazeby, J. M., Clarke, M., & Williamson, P. R. (2014). Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS ONE, 9(6), e99111. http://dx.doi.org/10.1371/journal.pone.0099111

Gear, C., Koziol-McLain, J., Wilson, D., Rae, N., Samuel, H., Clark, F., & McNeill, E. (2012). Primary healthcare response to family violence: A Delphi evaluation tool. Quality in Primary Care, 20(1), 15-30. https://www.ncbi.nlm.nih.gov/labs/articles/22584364/

Hallowell, M. R., & Gambatese, J. A. (2010). Qualitative research: Application of the Delphi method to CEM research. Journal of Construction Engineering and Management, 136(1), 99-107. http://dx.doi.org/10.1061/(ASCE)CO.1943-7862.0000137

Hasson, F., & Keeney, S. (2011). Enhancing rigour in the Delphi technique research. Technological Forecasting & Social Change, 78(9), 1695-1704. http://dx.doi.org/10.1016/j.techfore.2011.04.005

Herbohn, J., Harrison, S., & Emtage, N. (1999) Response to comments of Don Nicholson, Australian Forestry, 62(3), 285-286. http://dx.doi.org/10.1080/00049158.1999.10674794

Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44(2), 193-295. http://dx.doi.org/10.1007/BF02457380

Kaplan, A., Skogstad, A. L., & Girshick, M. A. (1950). The prediction of social and technological events. Public Opinion Quarterly, 14(1), 93-110. http://dx.doi.org/10.1086/266153

Kauko, K., & Palmroos, P. (2014). The Delphi method in forecasting financial markets. An experimental study. International Journal of Forecasting, 30(2), 313-327. http://dx.doi.org/10.1016/j.ijforecast.2013.09.007

Keeney, S., Hasson, F., & Mckenna, H. (2011). The Delphi Technique in Nursing and Health Research. Oxford: Wiley-Blackwell.

Landeta, J. (1999). El Método Delphi: Una Técnica de Previsión para la Incertidumbre. Barcelona: Ariel.

Landeta, J. (2011). Hybrid Delphi: A methodology to facilitate contribution from experts in professional contexts. Technological Forecasting & Social Change, 78(9), 1629-1641. http://dx.doi.org/10.1016/j.techfore.2011.03.009

Landeta, J., Barrutia, J., Hoyos, J., & Araujo, A. (2014). Initiatives for the improvement of continuous management training. Cuadernos de Gestión, 15(1), 61-92. http://dx.doi.org/10.5295/cdg.130411jl

Lin, V. S., & Song, H. (2014). A review of Delphi forecasting research in tourism. Current Issues in Tourism. http://dx.doi.org/10.1080/13683500.2014.967187

Linstone, H., & Turoff, M. (Eds.). (1975). The Delphi Method: Techniques and Applications. London: Addison-Wesley.

Linstone, H., & Turoff, M. (2011). Delphi: A brief look backward and forward. Technological Forecasting & Social Change, 78(9), 1712-1719. http://dx.doi.org/ 10.1016/j.techfore.2010.09.011

Lotka, A. J. (1926). The frequency distribution of scientific productivity. Journal of the Washington Academy of Sciences, 16(12), 317-323. http://www.jstor.org/stable/24529203

Meshkat, B., Cowman, S., Gethin, G., Ryan, K., Wiley, M., Brick, A., … Mulligan, M. (2014). Using an e-Delphi technique in achieving consensus across disciplines for developing best practice in day surgery in Ireland. Journal of Hospital Administration, 3(4), 1-8. http://dx.doi.org/10.5430/jha.v3n4p1

Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., Stroup, D. F., … Yusuf, S. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. The Lancet, 354(9193), 1896-1900. http://dx.doi.org/10.1016/S0140-6736(99)04149-5

Moreira, M. (2014). Técnica Delphi: metodologia para pesquisas em educação no Brasil. Revista de Educação. 19(1), 63-71. http://periodicos.puc-campinas.edu.br/seer/index.php/reveducacao/article/view/2616

Mullen, P. M. (2003). Delphi: myths and reality. Journal of Health Organization and Management, 17(1), 37-52. http://dx.doi.org/10.1108/14777260310469319

Noack, A. (2007). Energy models for graph clustering. Journal of Graph Algorithms and Applications, 11(2), 453-480. http://dx.doi.org/10.7155/jgaa.00154

Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management, 42(1), 15-29. http://dx.doi.org/10.1016/j.im.2003.11.002

Otte, E., & Rousseau, R. (2002). Social network analysis: a powerful strategy, also for the information sciences. Journal of Information Science, 28(6), 441-453, http://dx.doi.org/10.1177/016555150202800601

Papadimitriou, F., & Kidman, G. (2012). Statistical and scientometric analysis of international research in geographical and environmental education. International Research in Geographical and Environmental Education, 21(1), 11-20. http://dx.doi.org/10.1080/10382046.2012.639153

Powell, C. (2003). The Delphi technique: myths and realities. Journal of Advanced Nursing, 41(4), 376-382. http://dx.doi.org/10.1046/j.1365-2648.2003.02537.x

Price, D. J. S. (1963). Little Science, Big Science. New York: Columbia University Press.

Qureshi, M. I., Bhatti, M. N., Rasli, A. M., Yasir, M., & Zaman, K. (2014). The Delphi method for internationalization of higher education in Pakistan: integrating theory of constraints and quality function deployment. Mediterranean Journal of Social Sciences, 5(20), 2702-2710. http://dx.doi.org/10.5901/mjss.2014.v5n20p270

Rekalde, I., Landeta, J., & Albizu, E. (2015). Determining factors in the effectiveness of executive coaching as a management development tool. Management Decision, 53(8), pp. 1-39. http://dx.doi.org/10.1108/MD-12-2014-0666

Rieger, W. (1986). Directions in Delphi developments: dissertations and their quality. Technological Forecasting & Social Change, 29(2), 195-204. http://dx.doi.org/10.1016/0040-1625(86)90063-6

Robson, N., & Rew, D. (2010). Collective wisdom and decision making in surgical oncology. European Journal of Surgical Oncology, 36(3), 230-236. http://dx.doi.org/10.1016/j.ejso.2010.01.002

Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15(4), 353-375. http://dx.doi.org/10.1016/S0169-2070(99)00018-7

Sackman, H. (1974). Delphi Assessment: Expert Opinion, Forecasting, and Group Process. Santa Monica: Rand Corporation Paper R-1283-PR. http://www.rand.org/pubs/reports/2006/R1283.pdf

Tsai, C.-C., Corley, E. A., & Bozeman, B. (2016). Collaboration experiences across scientific disciplines and cohorts. Scientometrics, 108(2), 505-529. http://dx.doi.org/10.1007/s11192-016-1997-z

Vallejo, M., Ocaña, A., Bueno, Á., Torralbo, M., & Fernández, A. (2005). Producción científica sobre educación multicultural contenida en las bases de datos Social Science Citation Index y Arts y Humanities Citation Index (1956-2003). Revista Española de Documentación Científica, 28(2), 206-220. http://dx.doi.org/10.3989/redc.2005.v28.i2.169

Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84(2), 523-538. http://dx.doi.org/10.1007/s11192-009-0146-3

Vollmar, H. C., Ostermann, T., & Redaèlli, M. (2015). Using the scenario method in the context of health and health care: a scoping review. BMC Medical Research Methodology, 15(1), 1-10. http://dx.doi.org/10.1186/s12874-015-0083-1

von der Gracht, H. A. (2012). Consensus measurement in Delphi studies. Review and implications for future quality assurance. Technological Forecasting & Social Change, 79(8), 1525-1536. http://dx.doi.org/ 10.1016/j.techfore.2012.04.013

Xiao, J., Douglas, D., Lee, A. H., & Vemuri, S. R. (1997). A Delphi evaluation of the factors influencing length of stay in Australian hospitals. International Journal of Health Planning and Management, 12(3), 207-218. http://dx.doi.org/10.1002/(SICI)1099-1751(199707/09)12:3<207::AID-HPM480>3.0.CO;2-V

Zang, L., Fan, N., Hai, Y., Lu, S. B., Su, Q. J., Yang, J. C., … Gao, Y. J. (2015). Using the modified Delphi method to establish a new Chinese clinical consensus of the treatments for cervical radiculopathy. European Spine Journal, 24(6), 1116-1126. http://dx.doi.org/10.1007/s00586-015-3856-7

Downloads

Published

2018-07-02

How to Cite

Cruz Ramírez, M., & Rúa Vásquez, J. A. (2018). Emergence and development of the Delphi method: a scientometric perspective. Biblios Journal of Librarianship and Information Science, (71), 90–107. https://doi.org/10.5195/biblios.2018.470

Issue

No. 71 (2018)

Section

Original

License

Authors who publish with this journal agree to the following terms:

  1. The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
  2. Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
  3. The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons Attribution 4.0 International License or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
    1. Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
    with the understanding that the above condition can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
  4. The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
  5. Authors are permitted and encouraged to post online a prepublication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work. Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
  6. Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
  7. The Author represents and warrants that:
    1. the Work is the Author’s original work;
    2. the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
    3. the Work is not pending review or under consideration by another publisher;
    4. the Work has not previously been published;
    5. the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
    6. the Work contains no libel, invasion of privacy, or other unlawful matter.
  8. The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.

Revised 7/16/2018. Revision Description: Removed outdated link. 

Most read articles by the same author(s)