TEC Assessment Index

Use of GeneSearch Breast Lymph Node Assay to Detect Sentinel Node Metastases in Early Stage Breast Cancer

Assessment Program
Volume 22, No. 8
November 2007

Executive Summary

Background

The spread of breast cancer to the axillary lymph nodes is a key prognostic indicator and affects treatment choices. For women with early stage breast cancer and no palpable lymph nodes, sentinel lymph node biopsy (SLNB) is common. Currently, two approaches are used for evaluating sentinel nodes. In the first, the sentinel lymph node is removed, sent to a pathologist, and the surgery is completed. Nodal histology is evaluated, with or without immunohistochemical stains, which takes 1–2 days. A positive result usually leads to a second surgery, where axillary lymph node dissection (ALND) is performed and additional nodes are removed for examination. In the second approach, the sentinel lymph node is removed and sent for evaluation during the surgery. The two most common intraoperative techniques are frozen section histology or imprint cytology, both of which are less accurate than postoperative histology of permanently fixed tissue. Patients with positive lymph nodes generally receive ALND during the same surgery.

The GeneSearch™ breast lymph node (BLN) assay (hereafter, referred to as “GeneSearch” or “the assay”) is proposed as an alternative intraoperative test. Developed and commercialized by Veridex, LLC (Warren, NJ), GeneSearch uses real-time polymerase chain reaction (RT-PCR) to qualitatively evaluate nodal sections for the presence of two gene expression markers, mammaglobin and cytokeratin 19. These genes are expressed at higher levels in breast tissue, but not in normal nodal tissue. The assay is automated and performed in a homogeneous, one-step, fully contained reaction; test results are available in about 35 to 40 minutes. The assay cutoff for positivity is designed to allow detection of metastases that are larger than 0.2 mm in size; however, the assay does not discriminate between micrometastases (i.e., between 0.2 mm and 2 mm) and macrometastases (i.e., larger than 2 mm). The results would allow an intraoperative decision regarding the need for ALND.

Objective

To evaluate the use of the intraoperative GeneSearch™ BLN assay in detecting early stage breast cancer metastases in the sentinel lymph nodes compared to
1. postoperative histology or
2. other commonly used intraoperative evaluation methods, namely, intraoperative frozen section histology or imprint cytology.

Search Strategy

A MEDLINE® search was conducted via PubMed, covering references entered into the database from the year 2000 through December 2007. Text word searching was performed for the terms “GeneSearch” and Veridex. The search was limited to articles in English that discussed treatment of human patients. In addition, clinical data were sought from the U.S. Food and Drug Administration (FDA) and Veridex websites.

Selection Criteria

Articles in peer-reviewed journals or FDA submissions reporting specifically on the GeneSearch assay were selected.

Main Results

FDA-approved tests are extensively reviewed for analytic validity. The GeneSearch BLN assay includes positive and negative controls to ensure reagent quality and instrument performance in each run; control failure invalidates the run. An internal control, consisting of mRNA detection from a constitutively expressed gene in lymph node tissue, monitors the quality of the sample preparation and RT-PCR reaction. A study to ensure replicability of results was also performed.

There is only one published study that provides evidence on the performance of the assay; additional information includes the manufacturer’s submissions to the FDA, materials from the meeting of the Immunological Devices Panel meeting on November 16, 2006, and several abstracts presented at professional society meetings. The published study reports on 416 subjects from 11 sites in the U.S. Patients were those with a diagnosis of breast cancer and who were 18 years or older and who were scheduled for sentinel lymph node biopsies. The GeneSearch assay was performed alongside other tests commonly used at each facility; the GeneSearch results were not used for clinical management. The assay was compared to a reference standard, postoperative histology (primary study outcome), and to other intraoperative techniques where routinely conducted by the participating institution, namely, frozen section histology and imprint cytology. Among the 416 patients with postoperative histology results, 29.1% had positive lymph nodes for metastases, with a range across sites of 14.3% to 45.5%.

Compared to the reference standard, the sensitivity of the GeneSearch assay was 87.6% (95% CI: 80.4–92.9%); the specificity was 94.2% (95% CI: 90.9–96.6%); the positive predictive value was 86.2% (78.8–91.7%); and the negative predictive value was 94.9% (91.7–97.1%). The sensitivity was higher (97.9%; 95% CI: 92.5–99.7%) for patients with macrometastases than for those with micrometastases (56.5%; 95% CI: 34.5–76.8%). The authors assert that the false-positive rate may be lower than reported, because these results may be based on examination of parts of the lymph node that contain metastases, while the postoperative histology slides may come from a different slice of the node where there were no metastases, i.e., differences may be due to sampling distinct parts of the lymph node. While the researchers provide suggestive evidence, further research is needed to confirm or deny this hypothesis.

Among the 319 patients with results for both the assay and frozen section histology, the test performance of each technique compared to the reference standard was as follows:

The FDA analyzed the differences in test characteristics between these approaches using the same data. They concluded that the difference in sensitivity between GeneSearch and frozen section histology was statistically significant (difference=10; 95% CI: 2.5–17.7); as is the difference in the negative predictive value (difference=3.7; 95% CI: 0.8–6.4). The FDA characterized the difference in specificity as borderline significant (difference=-3.5; 95% CI: -7.4–0.0) and the difference in the positive predictive value as insignificant (difference=-7.0; 95% CI: -14.8–1.3). However, this comparison of GeneSearch and the alternative intraoperative techniques was not part of the original protocol. Rather, it was a post-hoc analysis that did not include all participating sites; nor did sites performing alternative tests follow a uniform protocol for conducting the tests and for adjudication of results (as was used for the preplanned analysis). While the results are suggestive, they need additional confirmation. Additional analyses by the FDA suggest that both tests may operate along the same receiver operating characteristic (ROC) curve, in other words, that one test is not better overall than the other one. Data are inadequate to compare the performance of the GeneSearch assay to imprint cytology.

Taking the sensitivity and specificity of the assay in all 416 subjects included in the overall analysis, one can estimate the number of second surgeries avoided if a positive intraoperative assay result lead to immediate ALND and, conversely, the number of unnecessary ALNDs that would be performed because of false-positive assay results. Assuming a population of 1,000 patients, use of the GeneSearch assay to guide intraoperative decision-making regarding ALND would prevent 255 second surgeries and incur 41 unnecessary ALNDs, assuming the study average prevalence of 29.1% patients with positive lymph nodes. The number of second surgeries avoided would fall to 88 if the prevalence were 10%, with 52 unnecessary ALNDs. The number of second surgeries avoided would rise to 438 if the prevalence were 50%, with 29 unnecessary ALNDs. In reviewing the GeneSearch application, the FDA estimated that roughly 11% of patients undergoing ALND develop lymphedema and about the same proportion develop impaired shoulder movement (12–15% for patients undergoing radiotherapy; 7–8% for those not having radiotherapy).

There are several operational issues that add difficulty to the use of the GeneSearch assay, including the need for fresh specimens (rather than putting them in formalin for permanent fixation), the learning curve involved in reducing both the percentage of invalid results (from about 15% initially to 4–8% for more experienced technicians) and the time to perform the test (from 35–55 minutes to 30–35 minutes), and the potentially longer time required to perform the test compared to alternative intraoperative techniques (which take less than 15 minutes). Although not as much of an experienced pathologist’s time is required to evaluate the test results, skill is needed in preparing the specimen.

Author's Conclusions and Comments

Question 1: What are the outcomes (i.e., surgeries avoided vs. unnecessary ALNDs) of using the GeneSearch BLN assay intraoperatively compared to waiting for postoperative histology results before deciding on whom to perform ALNDs?

Does the evidence permit conclusions? The GeneSearch BLN assay has potentially wide use, since a majority of the 180,000 (http://www.cancer.gov/cancertopics/types/breast) new patients per year with breast cancer undergo sentinel lymph node biopsy. A solid evidence base is especially important in this context, because the use of this assay could potentially lead to a major change in practice. There are a number of strengths to the study presented to the FDA, including the clear distinction between the training and test sets and double-reading of the permanent histology slides to increase the reliability of the reference standard. However, there are a number of weaknesses as well:

1. Patient recruitment is not described. For example, were these consecutive SLNB patients? Without this information, it is not possible to assess the potential for spectrum bias in the study results. The wide range of prevalence of positive nodes across sites (14.3% to 43.5%) adds weight to the importance of understanding how the subjects were selected.

2. There was substantial variation in the performance of the assay across sites. Sensitivity varied from 50% to 100%, and specificity ranged from 83.3% to 100%. Therefore, the Immunological Devices Panel questioned whether or not pooling into summary statistics was valid.

3. There are no corroborating studies. The FDA has requested two postapproval studies of the assay, which will substantially increase the evidence base. Other research is also underway, based on presentations at ASCO and other specialty society meetings.

4. GeneSearch does not distinguish between micro- and macrometastases. Although ASCO guidelines currently recommend that ALND be performed if either is detected, they also state that the prognostic significance of micrometastases is unknown, as is the importance of ALND in these patients. If GeneSearch has higher sensitivity because it detects more micrometastases (which is not clear), then the clinical significance of finding these patients may not balance the harms of ALND.

5. There is a learning curve for those conducting this assay. How best to train new staff to perform this test is unknown, as is the impact on patient care if and when the use of this assay is disseminated. The initial failure rate (i.e., invalid test result) was about 15%; it decreased to about 4–8% after personnel gained experience. The failure rate outside a research study is not known. The failure rate was higher with small nodes than with larger ones. GeneSearch proponents suggest that small nodes are less likely to contain metastases. The assay could have been rerun with the leftover homogenate or mRNA for the invalid tests, but this was not routinely done. Doing so would provide a better idea of the impact of these invalid results.

What Is the Net Benefit of the Assay? The key factor in assessing this assay is the trade-off between avoiding a second surgery to remove axillary lymph nodes if the sentinel node is positive versus risking unnecessary ALNDs if the assay produces a false positive result. The reason SLNB is the standard of care in early breast cancer, despite the fact that trials on its impact are still incomplete, is the desire to avoid ALND where possible, given the potential for significant, long-term morbidity. While the researchers conducting the GeneSearch study suggest that many of the false-positive assay results detect real metastases in the additional sections of the node used for the assay, the evidence presented to support this argument is insufficient. Given that GeneSearch was not used for patient treatment decisions in this study, longer-term follow-up of patients with false-positive test results to see if and when axillary metastases develop might be informative, although the utility of this approach may be compromised by the small number of patients involved.

As for the trade-off between avoiding a second surgery and undergoing an unnecessary ALND, patient preferences should play an important role in this decision. The FDA is requiring informed consent for participants in one of the postmarketing studies. Although the number of unnecessary ALNDs is substantially smaller than the number of second surgeries avoided, the “harms” from the former are more common and generally longer lasting than the harms associated with needing a second surgery. The sequelae of ALND can last for years or even a lifetime. The inconvenience of a second surgery, while difficult for patients coping with a serious diagnosis and substantial course of treatment, is likely to be short-lived. For these women, the harms of ALND are unavoidable.

Question 2: How does the sensitivity and specificity of the GeneSearch BLN assay performed intraoperatively during sentinel lymph node biopsy compare to the sensitivity and specificity of alternative intraoperative tests, namely imprint cytology and frozen section histology

Does the evidence permit conclusions? The test set data provided to the FDA is useful in that it provides a direct comparison of the GeneSearch assay versus alternative intraoperative tests on the same patients. Several concerns, on the other hand, are outlined below:

1. The comparison of GeneSearch to alternative, intraoperative tests was not planned; not all study sites performed such tests.

2. The sample size of 29 for the imprint cytology comparison is too small to be meaningful.

3. The alternative tests were not conducted with the rigorous double-reading, review by central laboratory pathologists, and reconciliation of discrepancies applied to the reference standard.

4. The statistical approach to evaluating the difference in the test characteristics between GeneSearch and frozen section histology is not well described.

What Is the Net Benefit of the Assay? The focus here is on which technique—the assay, frozen section histology, or imprint cytology—performs best at avoiding second surgeries versus undergoing unnecessary ALNDs. The data on imprint cytology is inadequate and will not be discussed further. There is also inadequate evidence on whether the GeneSearch assay outperforms frozen section histology. However, an ROC analysis conducted by the FDA suggests that the GeneSearch assay and frozen section histology operate at different points on the same or very similar ROC curves. If this is correct (the analysis is not described in sufficient detail to assess this), then the question is not whether one technique is better than the other overall, but rather what the optimal tradeoff (and therefore assay cutoff) is between false-positive and false-negative results. Given the longer term and potentially more serious sequelae from ALND than from a second surgery, increasing the sensitivity of the test while sacrificing some specificity, as the GeneSearch assay does, may not be optimal.

The GeneSearch assay also provides less information for staging than other intraoperative procedures, since it cannot distinguish between micro- and macrometastases. Nor can it indicate the location of the metastasis (inside or outside of the node). Postoperative histology is therefore required in all cases. It is less crucial when frozen section histology is performed, since pathologists can judge the size of the metastasis and its location from this test, although distortion is possible.

To summarize, the data available is inadequate to assess the clinical utility of the GeneSearch assay compared to either postoperative histology alone or to alternative intraoperative tests such as imprint cytology and frozen section histology. In addition, the balance of benefits versus harms may require higher specificity to avoid unnecessary ALNDs and their sequelae, whereas the GeneSearch design emphasizes sensitivity. Patient preferences should also play a key role in this calculation.

Based on the available evidence, the Blue Cross and Blue Shield Association Medical Advisory Panel made the following judgments about whether the use of GeneSearch breast lymph node assay to detect sentinel node metastases in early stage breast cancer meets the Blue Cross and Blue Shield Association Technology Evaluation Center (TEC) criteria.

1. The technology must have final approval from the appropriate governmental regulatory bodies.

The GeneSearch™ BLN Test Kit was cleared for marketing by the FDA on July 16, 2007.

2. The scientific evidence must permit conclusions concerning the effect of the technology on health outcomes.

There is only one published study on the performance of the GeneSearch assay, which reports on the same study used by the FDA to determine approval. This study has a number of limitations, including the lack of a description of patient recruitment, inadequate descriptions of several analyses performed, substantial variations in test performance across sites, and an ad hoc comparison of the assay to other intraoperative techniques. Additional validation studies are needed that address these issues. The FDA has mandated two postmarketing studies that will also help fill in the gaps. It would also be helpful to have information on patient preferences between avoiding second surgeries versus running the risk of unnecessary axillary lymph node dissection. Therefore, the data available on the GeneSearch assay are insufficient to permit conclusions regarding the effect of this technology on health outcomes.

3. The technology must improve the net health outcome.

The available data are insufficient to determine whether or not the technology improves the net health outcome.

4. The technology must be as beneficial as any established alternatives.

The available data are insufficient to determine whether or not the technology is as beneficial as the established alternatives.

5. The improvement must be attainable outside the investigational settings.

The available data are insufficient to determine whether or not the technology improves the net health outcome and therefore, whether any improvement is attainable outside investigational settings. There appears to have been a learning curve for using this technology, which was addressed in the study through additional training.

Based on the above, the use of GeneSearch breast lymph node assay to detect sentinel node metastases in early stage breast cancer does not meet the TEC criteria.

TEC Assessment Index

KEYWORDS: AJCC; ALND; assay; axilla; axillary; biopsy; BLN; breast cancer; dissection; examination; frozen section; GeneSearch; H&E; histology; immunohistochemistry; imprint cytology; intraoperative; laboratory; lymph node; lymphedema; macrometastases; macrometastasis; markers; metastases; metastasis; micrometastases; micrometastasis; molecular; National Adjuvant Breast and Bowel Project; NCCN; NSABP; oncology; pathology; polymerase chain reaction; real time; reverse; RT-PCR; sentinel lymph node; staging; transcription; tumor; Veridex;