Assessing the reproducibility of the IOTA simple ultrasound rules for classifying adnexal masses as benign or malignant using stored 3D volumes

OBJECTIVE: To analyze the reproducibility of the IOTA simple ultrasound rules for classifying adnexal masses as benign or malignant among examiners with different level of expertise using stored 3D volumes of adnexal masses. STUDY DESIGN: Five examiners, with different levels of experience and blinded to each other, evaluated 100 stored 3D volumes from adnexal masses and looked for the presence or absence of malignant or benign features according to the IOTA definitions. Multiplanar view and virtual navigation were used. All examiners had to assess the 3D volume of each adnexal mass and classify it as benign or malignant. To analyze intra-observer agreement each examiner performed the assessment twice with a two-week interval between the first and second assessments. To analyze the inter-observer agreement, the second assessment from each examiner was used. Reproducibility was assessed calculating the weighted Kappa index. RESULTS: Intra-observer reproducibility was moderate or good for all observers (Kappa index ranging from 0.59 to 0.74). Inter-observer reproducibility was moderate to good (Kappa index range: 0.46-0.67). CONCLUSIONS: The simple rules are reasonably reproducible among observers with different level of expertise when assessed in stored 3D volumes.