# Measured top-N RAG comparison

## Setup

- Retrieval query: `continuous glucose monitoring problems`.
- Generation question: What recurring problems appear in continuous glucose monitoring reports?
- Corpus: 20,000 sampled openFDA reports in Qdrant; same pinned MiniLM embeddings as the EigenLens demo.
- Retrieval: exact cosine similarity, score threshold 0.4; top-5/10/20/50 are fixed prefixes of one top-50 response.
- Baseline: retrieved full narrative text plus original report IDs only; no clustering tools, topic labels, global counts, reranker, diversification or deduplication.
- Generation: `nvidia/nemotron-3.5-lightning:free`, temperature 0, reasoning effort low, max_tokens 4096; identical instruction template and question for every N.
- Cluster membership was joined afterward using saved EigenLens assignments, not supplied to the baseline model. These clusters are an analytical reference, not ground-truth failure categories.

## Retrieval results

| Top N | Reports in cluster 4 | Unassigned reports | Assigned clusters represented | Distinct exact narrative texts |
| ----: | -------------------: | -----------------: | ----------------------------: | -----------------------------: |
|     5 |                    4 |                  1 |                             1 |                              4 |
|    10 |                    9 |                  1 |                             1 |                              9 |
|    20 |                   19 |                  1 |                             1 |                             14 |
|    50 |                   49 |                  1 |                             1 |                             24 |

The highest-ranked reports concern CGM-versus-blood-glucose-meter inaccuracies. Top-50 contained 49 reports in the same 246-report cluster and one unassigned report. None belonged to the largest 685-report cluster. All 50 report IDs were unique; repeated narratives are distinct source report IDs, not a pagination bug.

The complete above-threshold selection contained 2,581 reports, 13 assigned clusters and 890 unassigned reports. This measures the coverage of the plain cosine top-N selection in this case, not a general claim about all RAG systems.

## Actual generated answers

### Top 5

Across the five provided continuous glucose monitoring reports, a recurring problem is inaccuracy between the CGM and BG meter, with all reported glucose values falling within the D zone of the Parkes error grid. The probable cause of inaccuracy could not be determined via available data in four reports (24795008, 24988367, 25106942, 25668472), which also note that data investigations did not find comparable BG calibration values or found calibrations outside device specifications. Sensor placement was the arm in these four reports, while report 25219799 describes abdomen insertion (off-label) and confirms frequent inaccuracy, additionally noting a difference within the B zone of the Parkes grid. All reports agree no injury or medical intervention was reported. The limited data prevent determining causation or broader frequency.

Input tokens: 1,445. Provider-reported cost: 0. Finish reason: `stop`.

### Top 10

Recurring problems across the provided CGM reports (IDs: 25668472, 24795008, 24988367, 25106942, 25598715, 25325913, 25047628, 25641405) include: inaccuracy between the continuous glucose monitor and blood glucose meter; allegation undetermined with probable cause not determinable via data; reported glucose values falling within the D zone (and in one report, D and C zone) of the Parkes error grid; and data investigation not finding comparable BG calibration values or finding calibrations in accurate range per device specifications. No injury or medical intervention was reported in any case.

A distinct report (25219799) describes frequent CGM-BG meter inaccuracy with the sensor inserted off-label into the abdomen, and the investigation found a difference falling within the B zone (not solely D) of the Parkes error grid.

Wording is repeated across many reports (identical phrasing on sensor placement, undetermined cause, D zone values, calibration comparison issues), indicating template use, while report 25219799 provides distinct evidence of off-label abdomen placement and B zone classification. Evidence is limited to these 10 submitted records; no frequency inferences across unprovided reports are made.

Input tokens: 2,719. Provider-reported cost: 0. Finish reason: `stop`.

### Top 20

Recurring problems in the CGM reports center on inaccuracy between the continuous glucose monitor and blood glucose meter. Across multiple reports, the probable cause of inaccuracy could not be determined via available data (e.g., reports 25668472, 25219799, 24795008). Reported glucose values consistently fall within the D zone of the Parkes Error Grid, with one report noting D and C zones (25075188). Data investigations frequently could not find BG calibration values for comparison or found calibrations outside device specifications (e.g., 24795008, 25047628). No injury or medical intervention was reported in any case (e.g., 25668472, 25219799). Off-label sensor insertion sites—abdomen, arm, and upper buttocks—appear repeatedly (e.g., 25219799, 24795008, 25290634). Some allegations were confirmed due to frequent inaccuracy (25219799), while most were undetermined (e.g., 25668472, 25106942). Repeated wording is common across reports, reflecting standardized investigation templates; distinct evidence includes varying sensor sites, confirmation status, and Parkes zone classifications.

Input tokens: 5,358. Provider-reported cost: 0. Finish reason: `stop`.

### Top 50

Recurring problems across these reports center on CGM–BG meter inaccuracy, with investigation frequently unable to determine cause or compare calibration data. Sensor placement varies: arm is most common; abdomen and upper buttocks are noted as off-label usage in several reports (e.g., 25219799, 25290634, 25488530). The allegation is often undetermined (e.g., 25668472, 24795008), though some are confirmed (e.g., 25165043, 25052749). Reported glucose values predominantly fall in the D zone of the Parkes Error Grid, with confirmed cases sometimes B or C zone (e.g., 25165043, 25186631). Calibration data was typically unavailable or out of accurate range per device specifications. No injury or medical intervention was reported in the majority of cases. The reports largely share repetitive template phrasing but differ in outcome (confirmed/undetermined), sensor site, and Parkes zone assignment. Evidence is limited to the provided narratives; no frequency or causation claims are made beyond what appears in these records.

Input tokens: 12,752. Provider-reported cost: 0. Finish reason: `stop`.

The first top-10 attempt exhausted its 4,096-token completion budget entirely in reasoning and returned no answer. One retry with the same settings succeeded. No alternate model was used.