Research on the value of peer review is limited by the lack of a validated instrument to measure the quality of reviews.
The aim of this study was to develop a simple, reliable, and valid scale that could be used in studies of peer review.
A Review Quality Instrument (RQI) that assesses the extent to which a reviewer has commented on five aspects of a manuscript (importance of the research question, originality of the paper, strengths and weaknesses of the method, presentation, interpretation of results) and on two aspects of the review (constructiveness and substantiation of comments) was devised and tested.
Its internal consistency was high (Cronbach's alpha 0.84).
The mean total score (based on the seven items each scored on a 5-point Likert scale from 1 to 5) had good test-retest (Kw=1.00) and inter-rater (Kw=0.83) reliability.
There was no evidence of floor or ceiling effects, construct validity was evident, and the respondent burden was acceptable (2-10 minutes).
Although improvements to the RQI should be pursued, the instrument can be recommended for use in the study of peer review.
Mots-clés Pascal : Revue bibliographique, Document publié, Qualité, Evaluation, Validité, Fiabilité, Epidémiologie, Méthodologie, Analyse, Evaluation performance, Homme, Royaume Uni, Europe
Mots-clés Pascal anglais : Bibliographic review, Published document, Quality, Evaluation, Validity, Reliability, Epidemiology, Methodology, Analysis, Performance evaluation, Human, United Kingdom, Europe
Notice produite par :
Inist-CNRS - Institut de l'Information Scientifique et Technique
Cote : 99-0350947
Code Inist : 002B30A01A1. Création : 14/12/1999.