Context
Standard dependency bots solve detection, not judgment: a new version exists, here is a PR. What they don't solve is whether the changelog behind that version bump matters, or whether the twelve packages that just bumped in the same window are actually one upgrade wearing twelve PRs.
The problem
Two architectural gaps kept surfacing. Synchronous, per-package analysis
couldn't keep the API responsive once changelog fetching and LLM analysis
became the bottleneck. And nothing in a one-PR-per-package model captured that
some upgrades are only safe as a group — a @types/react bump merged ahead of
the React bump it depends on is green on its own PR and broken on main.
What we built
An async work-queue architecture, by design, not as an afterthought. Heavy tasks — changelog fetches, LLM analysis, codebase impact scans — are enqueued to Redis-backed BullMQ queues and processed by seven independent workers, so API handlers return immediately and the UI stays responsive regardless of how long an individual analysis takes. Two separate Redis instances back this: one dedicated to queue coordination, one dedicated to LLM output and changelog caching, so cache pressure and queue throughput never compete for the same resource.
Ecosystem-aware bundling with topological ordering. The bundle recommender recognises 15+ hardcoded ecosystem groups — React and its types, Vue, Angular, Prisma, Next.js, major test runners, ESLint, Babel, Webpack, Vite, GraphQL, Tailwind, Storybook, NestJS — and orders a bundle with Kahn's algorithm so dependency-ordered packages upgrade in the sequence they actually require. Bundle execution runs one job at a time, deliberately, to guarantee two bundles never mutate the same repository concurrently.
A resilient, multi-provider LLM layer. Analysis runs against OpenAI, OpenRouter, Ollama or a local LM Studio endpoint, behind a circuit breaker with a configurable fallback order, so a single provider's outage degrades the pipeline instead of stalling it. Supporting local and self-hosted models in that fallback chain meant hardening the response parser to handle multiple response shapes — including reasoning models that return their answer in a separate field — so the same reliability guarantee holds regardless of which provider in the chain actually answered.
Confidence scoring instead of a binary CI gate. A finished CI run doesn't just report pass or fail — a worker builds a 0–100 confidence score from test pass rate, code coverage, a similarity correlation between failed tests and predicted breaking changes, and the historical success rate of upgrading that component before.
Outcome
Dependency upgrades now move through an async, queue-based pipeline that stays responsive under load, sequences related packages correctly, and keeps running through a single LLM provider's outage — see Upgrade Intelligence.
Lessons learned
Queue the expensive work, keep the request path thin. Any pipeline that mixes a fast user-facing path with slow external calls (LLM analysis, third-party fetches) benefits from the same shape: enqueue, return, process asynchronously.
Bundling is a topology problem before it's a scheduling problem. Getting the dependency order between related packages right makes the scheduling question — sequential versus parallel — a simple, defensible default rather than a source of races.
Build the provider-fallback layer for the least-standardized provider in the chain, not the most. A resilience layer designed only against a single hosted API's response shape breaks the moment a second, compatible-but-not- identical provider joins the fallback order.