Keele Research Repository

Kitchenham, BA and Brereton, P (2013) A systematic review of systematic review process in software engineering. Information and Software Technology, 55 (12). 2049 -2075.

Preview

Text B Kitchenham - A systematic review of systematic review process research in software engineering.pdf - Accepted Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (866kB) | Preview

Abstract

Context
Many researchers adopting systematic reviews (SRs) have also published papers discussing problems with the SR methodology and suggestions for improving it. Since guidelines for SRs in software engineering (SE) were last updated in 2007, we believe it is time to investigate whether the guidelines need to be amended in the light of recent research.

Objective
To identify, evaluate and synthesize research published by software engineering researchers concerning their experiences of performing SRs and their proposals for improving the SR process.

Method
We undertook a systematic review of papers reporting experiences of undertaking SRs and/or discussing techniques that could be used to improve the SR process. Studies were classified with respect to the stage in the SR process they addressed, whether they related to education or problems faced by novices and whether they proposed the use of textual analysis tools.

Results
We identified 68 papers reporting 63 unique studies published in SE conferences and journals between 2005 and mid-2012. The most common criticisms of SRs were that they take a long time, that SE digital libraries are not appropriate for broad literature searches and that assessing the quality of empirical studies of different types is difficult.

Conclusion
We recommend removing advice to use structured questions to construct search strings and including advice to use a quasi-gold standard based on a limited manual search to assist the construction of search stings and evaluation of the search process. Textual analysis tools are likely to be useful for inclusion/exclusion decisions and search string construction but require more stringent evaluation. SE researchers would benefit from tools to manage the SR process but existing tools need independent validation. Quality assessment of studies using a variety of empirical methods remains a major problem.

Item Type:	Article
Uncontrolled Keywords:	systematic review, systematic literature review, systematic review methodology, mapping study
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Depositing User:	Symplectic
Date Deposited:	19 Dec 2016 12:21
Last Modified:	24 May 2019 13:56
URI:	http://eprints.keele.ac.uk/id/eprint/2683

Actions (login required)

View Item