TEC Assessment Index

Computer-Aided Detection of Malignancy with Magnetic Resonance Imaging of the Breast

Assessment Program
Volume 21, No. 4
June 2006

Executive Summary

Background
The use of computer-aided detection (CAD) is proposed to supplement radiologists' interpretation of contrast-enhanced magnetic resonance imaging (MRI) of the breast. MRI of the breast is sometimes used as an alternative to mammography or other screening and diagnostic tests because of its high sensitivity in detecting breast lesions, even among those women—for example, younger women and those with denser breasts—in whom mammography is less accurate. However, MRI has a high false-positive rate because of the difficulty in distinguishing between benign and malignant lesions. It is also used to look for more extensive disease in women diagnosed with breast cancer and to gauge the impact of treatment. The CAD systems reviewed in this Assessment are intended to improve the specificity of MRI in detecting or measuring malignant tissue, while maintaining the generally high sensitivity of MRI. The use of CAD may also shorten the time needed to interpret breast MRI images, which currently takes much longer than reading mammograms.

Objective
To assess the evidence on the use of CAD with MRI of the breast by comparing the sensitivity, specificity, and recall rate* of MRI with and without the use of commercially available CAD systems in detecting malignant lesions, evaluating the extent of disease in women with cancer, or gauging the impact of treatment.

Search Strategy
MEDLINE search through March 2006, as well as reviewing reference lists and querying experts in the field.

Selection Criteria
Articles had to compare the sensitivity and specificity of MRI of the breast read with and without the use of CAD systems. The primary focus is on commercially available CAD systems, although some articles on other systems were included if they provided useful information on the potential impact of CAD systems. Additionally, studies had to report on cancer detection based on histological results for at least some of the patients in the sample. Articles on CAD development that did not include independent testing sets or that had fewer than 20 cases were excluded. Selected abstracts were included, but their results should be interpreted with caution.

Main Results
Many of the studies on the use of CAD with MRI of the breast primarily report on the development of CAD systems or testing new CAD approaches. Few of them evaluate commercially available CAD systems. Several of those that do, report on the development and testing of approaches that underlie one of the commercially available systems (3TP); it is not clear to what degree the current 3TP system has or has not been modified compared to these earlier approaches. Although the studies had to have separate testing data sets to be included in this Assessment, these data sets often were enriched with more cancer cases or consisted exclusively of cases in which lesions had been found. As a result, the range of sensitivities and specificities cannot be applied to the populations usually found in a clinical setting. In addition, because many of the studies are retrospective and report primarily on the development and testing of a CAD system, they lack the rigor and generalizability of a large, prospective, well-designed study.

Author's Conclusions and Comments
Unfortunately, the literature on the use of CAD with MRI of the breast was sparse overall, and few studies addressed the specific situations in which CAD with MRI is used in a clinical setting. Many of the few articles and abstracts calculated test characteristics on the basis of lesions and not the number of women or breasts. In a screening population, many women would not have any lesions. Including these women might alter the results. Given MRI's lower sensitivity in detecting ductal carcinoma in situ (DCIS), the mix of DCIS versus masses would affect the calculations of sensitivity and specificity and might affect the impact of the CAD system. There is one article looking at the use of a non-commercial CAD system with MRI among women scheduled for breast-conserving therapy (BCT). About 41% of these women had additional findings (larger or additional lesions), 56% of which were malignant. The results led to changes in treatment when more extensive disease was found. The area under the ROC curve was 0.91±0.04 for the radiologist reading and 0.98±0.04 for the combined radiologist and computerized reading (p=0.03).

The literature as a whole is not clear on how the CAD system is to be used. In the case of CAD with mammography, the radiologist reads the original films first, makes a diagnosis, and then reviews the CAD results. Because CAD is not 100% sensitive, lesions detected both before the use of CAD and after viewing the CAD results may be worked up. In this way, CAD can add to the sensitivity of mammography, but not its specificity. With MRI of the breast, the sensitivity is already high, and the focus is primarily on increasing the specificity. In some articles, it appears that CAD is intended as an adjunct to the initial MRI reading, just as with CAD and mammography. In other articles, it is proposed as a way of speeding up the MRI reading process, and the precise protocol to be followed in reading the MRI images is not clear. Furthermore, unlike in the case of CAD with mammography, in the documents regarding the FDA clearance it does not specify that CAD must be added only after an initial reading of the images alone, although it does say for one system that "Patient management decisions should not be made based solely on the results of the CADstream analysis." Obviously, the impact of CAD on the accuracy of MRI of the breast may depend in part on how the CAD results are incorporated into the reading and diagnostic process.

Based on the available evidence, the Blue Cross and Blue Shield Association Medical Advisory Panel made the following judgments about whether the computer-aided detection of malignancy with MRI of the breast meets the Blue Cross and Blue Shield Association Technology Evaluation Center (TEC) criteria.

1. The technology must have final approval from the appropriate governmental regulatory bodies.

Two CAD systems for use with MRI of the breast have 510(k) marketing clearance from the U.S. Food and Drug Administration (FDA).

2. The scientific evidence must permit conclusions concerning the effect of the technology on health outcomes.

There are no high quality, current published studies of the impact of commercially available CAD systems on the sensitivity and specificity of MRI of the breast. The few studies and abstracts available focus primarily on the development of the CAD system or they include samples of women that are highly selective and usually have far more cases of cancer than would be encountered in a screening population.

3. The technology must improve the net health outcome; and
4. The technology must be as beneficial as any established alternatives.

There is insufficient evidence to assess whether the use of CAD systems would maintain or increase the sensitivity, specificity, and recall rates of MRI of the breast. Given the inability to evaluate these intermediate outcomes, it is not possible to assess the impact of CAD on health outcomes such as treatment success among breast cancer patients or survival.

5. The improvement must be attainable outside the investigational settings.

Whether the use of CAD with MRI of the breast improves outcomes has not been established in the investigational setting.

For the above reasons, computer-aided detection of malignancy with MRI of the breast does not meet the TEC criteria.

*The recall rate is the percentage of patients asked to come back for further evaluation. 

TEC Assessment Index

KEYWORDS: DiagnosticImaging (category) WomensHealth (category) algorithm; artifact; artificial neural networks; biopsy; BIRADS; breast; breast cancer; breast density; Breast Imaging Reporting and Data System; CAD; cancer; Cancer Intervention and Surveillance Modeling Network; carcinoma; carcinoma in situ; CIS; CISNET; computed radiography; computer-aided detection; computer-aided diagnosis; contrast enhancement; DCIS; dense glandular tissue; digital image acquisition; display; ductal carcinoma; ductal carcinoma in situ; dynamic MR; enhancing lesions; FFDM; film mammography; film-screen mammography; full-field digital mammography; imaging; invasive carcinoma; LORAD; magnetic resonance imaging; malignancy; malignant; microcalcifications; MR-guided; MRI; MR imaging; oncology; peripheral enhancement; radiation exposure; radiographic imaging; radiology; screen-film mammography; screening; screening mammography; self-breast examination; SFM; soft-copy reading; spatial contrast uptake; spatial resolution; storage; time-signal intensity; washout; women's health; X-ray