RAG & LLM Ops.
Private RAG on your data, fine-tuning where it earns its cost, model selection across open-source and frontier, hybrid routing, inference infra, drift monitoring. The plumbing that makes an agent useful.
Retrieval beats model choice. Most of the time.
A team running a 14B model with excellent retrieval beats a team running a 70B model with mediocre retrieval, on both cost and accuracy, in most enterprise tasks we've seen. We spend a disproportionate share of every engagement on retrieval — chunking strategy, hybrid search, reranking, evaluation harnesses.
That said, model choice still matters for the long tail. For reasoning-heavy work, frontier models (Claude Opus, Sonnet, GPT-class, Gemini Pro) in dedicated tenancy still lead. For high-volume routine work, open-weights models running inside your perimeter are usually the right call. Hybrid routing across both is the typical production shape.
The layers of an LLM-Ops engagement.
Retrieval pipeline
Source ingestion, chunking strategy, embedding model selection, hybrid search (vector + BM25), reranking. Evaluation against your real queries.
Model selection
Test-driven model selection on your data, not on public benchmarks. Smallest-model-that-works principle. Quantization where it matters.
Inference infrastructure
vLLM, TGI, llama.cpp, SGLang — picked for the workload. Concurrency planning. GPU choice and reservation strategy.
Hybrid routing
Deterministic routing across open-weights and frontier-in-private-tenancy. Cost, latency, and capability optimized per query class.
Fine-tuning when it pays
LoRA / QLoRA where it matters. Continual pretraining occasionally. We tell you when it isn't worth the effort.
Evaluation & drift
Replayable eval harness over real workflow examples. Drift detection against model upgrades. Quarterly re-evaluation in Managed SLA.
Open weights routinely; frontier in dedicated tenancy when justified.
Open-weights defaults: Llama 3.3 (70B / 8B), Qwen 2.5 (72B / 32B / 14B / 7B), DeepSeek-V3 and R1, Mistral & Mixtral families. We pick based on accuracy on your data, latency budget, and reasoning-trace requirements (auditable reasoning matters more for some workflows than others).
Frontier models, privately: Claude Opus/Sonnet on AWS Bedrock, GPT-class on Azure OpenAI, Gemini on Vertex AI — always through enterprise tenancy contracts with audited no-training, zero-retention, and audit-log delivery. Never through consumer endpoints.
The full framework is in our Private LLM Deployment Guide.
Most LLM-Ops engagements start with a Discovery Sprint.
2-3 weeks, $15-30K, includes retrieval evaluation and a model recommendation.