Archival Image Intelligence
& Sensitive Content Detection

The University of Illinois holds enormous collections of historical archival imagery — photographs, documents, and scans spanning decades of institutional history. Before these collections can be digitized, published, or made publicly accessible, they must be reviewed for sensitive, offensive, or culturally significant content that may require masking, restriction, or specialist review prior to release.

Manual review of collections at this scale is not feasible. Tayler Erbe scoped, designed, and led the development of an automated multimodal AI proof-of-concept to address this directly — building two distinct detection pipelines, a sensitive content taxonomy grounded in archival and legal standards, and a metadata extraction framework aligned with IPTC and Dublin Core archival preservation standards.

The project was developed in active partnership with the RIMS (Records and Information Management) Committee at UIUC and the University Library cataloguing program, with the goal of producing a replicable, scalable detection framework for eventual deployment across the full institutional image archive.

Manual review of the 10,000-image test set identified approximately 8–10 images as genuinely sensitive — a rate of roughly 1% of the corpus. This low base rate is important context for interpreting both classification performance and false positive rates: at 1% prevalence, even a highly accurate model will produce more false positives than true positives in absolute terms.

The semantic method's 16.5% precision on flagged items reflects both the challenge of this low-prevalence setting and the inherent ambiguity of archival content. The 73.4% "borderline" rate suggests that the sensitivity threshold is approximately correct, but that human expert judgment remains essential for a meaningful portion of flagged images.

The most reliable detected categories were Human Remains (skull, skeleton keywords) and Violence/Graphic Content (explosion, fire keywords) — categories where visual descriptions map cleanly to unambiguous taxonomy terms. Categories like Native American Imagery and Historical Racialized Performance proved harder to classify reliably, as descriptions often lacked explicit cultural context cues.

The metadata extraction pipeline is a standalone contribution of this project, independent of content classification. For archival collections, structured, standards-aligned metadata is as important as content moderation — enabling discovery, cataloguing, long-term preservation, and integration with library management systems.

Using tifffile, Pillow (PIL), and iptcinfo3, the pipeline extracts metadata across three levels for every archival image in the collection. All output fields are mapped to IPTC Core and Dublin Core schemas to ensure interoperability with the University Library's cataloguing systems.

The Method 2 LLaVA Direct pipeline was running at ~9 images per minute in production — meaning a full pass over the 12,125-image corpus required roughly 22 hours. This constraint became binding as the project moved from POC to operational scale-up.

A controlled performance characterization on the same NVIDIA L4 GPU isolated the bottleneck as the serving layer rather than the hardware or model. GPU utilization was sitting at 28–31% while p50 latency inflated 8.18× under concurrent requests — the diagnostic signature of request serialization at the Ollama daemon.

Migrating to vLLM 0.6.6 with continuous batching, holding the same model class and identical hardware, recovered the unused capacity: 55.4 images per minute at C=8, with GPU utilization climbing to 97%. A same-model control rerun confirmed the speedup was attributable to the serving architecture rather than a model swap.

The full benchmark methodology, the four supporting charts, and the apples-to-apples model-architecture control are documented in the companion case study.