Wiring iOS CoreML to a Quantized On-Device Reranker for Retrieval-Augmented Generation

Building a two-stage RAG pipeline entirely on-device: a fast ANN retrieval pass using a quantized bi-encoder, followed by a CoreML cross-encoder reranker that rescores top-k candidates — covering INT8 quantization tradeoffs, KV-cache reuse across candidates, and the memory pressure ceiling that determines how many candidates you can rerank before hitting thermal throttling on A16/A17