01Principle02System03Work04Results05Diagnosis06Process07Contact
AI Systems Architectavailable for contract / fte

AI systems thatReason and Actnot just chat.

I'm Nasr Eddine Sangai. I architect agents, retrieval pipelines and LLM platforms that automate real decisions, built to be operated in production, not demoed in a notebook.

View projects→
Let's build

Built with: the stack behind the systems

  • Anthropic
  • OpenAI
  • Llama
  • vLLM
  • PyTorch
  • Hugging Face
  • LangGraph
  • MCP
  • Qdrant
  • Pinecone
  • Postgres
  • pgvector
  • Redis
  • Kafka
  • Langfuse
  • RAGAS
  • Docker
  • Kubernetes
  • AWS
  • GCP
  • Next.js
  • TypeScript
  • React
  • Node
  • Vercel

operating principle

AI is not prompts. 
AI is  systems.

A model is one component. The product is the architecture around it: retrieval, memory, routing, evaluation, observability, designed so the whole thing can be measured, operated and changed safely.

the signal path

One request,
through the whole system.

Every layer below exists because something breaks without it. Scroll the chain, or pick any layer, and it explains why it's there, and what it's allowed to spend.

  1. ingress

    Every request starts as ambiguous human intent. The system's first job is to preserve that intent while making it machine-tractable.

  2. Authentication, rate limiting and per-route cost budgets. The gateway is where an AI product stops being a demo and starts being a service.

  3. model layer

    Frontier models where answer quality is the constraint, open-weight where cost, latency or data residency win. Routing is an architecture decision, not a dropdown.

  4. Session state, long-term facts and working context: scoped and versioned, so the system remembers what matters without leaking what doesn't.

  5. retrieval

    Hybrid retrieval (dense + BM25) into a cross-encoder reranker into grounded generation. The chunk boundary and the reranker usually matter more than the model.

  6. Qdrant or Pinecone depending on who operates it. Payload filtering, hybrid search and predictable p95 under real load, not just cosine similarity. Called from inside retrieval, not after it.

  7. execution

    A LangGraph state machine, not a single prompt: plan, call tools, grade its own context and loop when confidence drops. Error compounds at every hop: the graph has to recover.

cross-cutting · spans every layer above

  • Langfuse traces on every retrieval and generation: cost, latency, and the exact context each answer saw. You can't fix what you can't replay.

  • RAGAS faithfulness and answer-relevancy scored in CI. Output is non-deterministic: without an eval gate, regressions stay silent until a user finds them.

  • FastAPI services, Dockerized, with versioned prompts and per-route latency budgets. Built to be operated and changed safely, not demoed in a notebook.

the ecosystem

selected work

System portraits, drawn in the language of the craft.

case 01RETRIEVAL ROUTING

Adaptive Retrieval Router

problem
Always-dense retrieval pays embedding cost on every query and is surprisingly weak on named entities. Always-sparse misses paraphrase. Always-hybrid is fine and pays the dense cost regardless. And no static corpus answers a question about today.
architecture
Five ordered heuristics: freshness markers, keyword-overlap ratio, token length, question form, classify each query and dispatch it to the cheapest backend likely to answer: BM25, FAISS dense, RRF-fused hybrid (k=60), or Tavily web. Every rule that fired is recorded on the decision, and /router/explain returns the routing without spending a token.
  • FastAPI
  • FAISS
  • BM25
  • Tavily
  • Streamlit
classifyrouteretrieveaudit

outcome

Per-query cost spans $0 at 1–5 ms on sparse to $0.008 at 200–600 ms on web: the router exists to spend the top of that range only when the bottom would fail. A 40-query eval set forced through all four backends gives 160 directly comparable judged rows.

impact

Routing became an auditable decision instead of a default. Every call emits one structured line: hashed query, backend, latency, cost, so the heuristics can be disproven on real traffic rather than trusted.

Unveil Project→
case 02CALIBRATION

Adaptive Retrieval Router Pro

problem
Hand-tuned routing thresholds are a guess wearing the costume of a policy. And a retrieval API that stops at ranked documents leaves the last hop: grounded, cited generation, to whoever calls it.
architecture
Thresholds became a versioned RouterParameters profile (speed / balanced / quality), selected at runtime and stamped on every decision. An /answer endpoint closes the loop: route, retrieve, then generate with inline [n] citations streamed over SSE with token and cost accounting. Corpus documents can be uploaded, edited and re-indexed live.
  • Router profiles
  • BEIR
  • Anthropic
  • SSE
  • Live re-index
profilerouteretrievecite

outcome

The presets were picked by a 208-config threshold sweep over BEIR NFCorpus and SciFact, not by taste: quality lands in the winning equivalence class at macro nDCG@10 0.560, above always-dense at 0.557 and always-hybrid at 0.558, while speed sits on the sparse-targeting Pareto frontier.

impact

The routing policy stopped being folklore. Each preset carries the sweep that chose it, so changing one is a measurable decision with a baseline to beat.

Unveil Project→
case 03PLATFORM

Multi-Tenant RAG Platform

problem
A RAG prototype that works for one person doesn't survive contact with an organization: no tenancy, no roles, no audit trail, no way to show it publicly without handing out an account, and no gate that catches a retrieval regression before a user does.
architecture
Separable services behind one platform API: JWT auth, organization tenancy and RBAC on FastAPI + Postgres, async ingestion through Celery workers, a cross-encoder reranker sidecar, Qdrant for vectors, Redis for semantic cache, with a DeepEval + RAGAS harness and persisted eval-run records as the promotion gate.
  • FastAPI
  • Qdrant
  • Postgres
  • Celery
  • RAGAS
  • DeepEval
ingestrerankanswereval gate

outcome

Public demo sessions are anonymous and short-lived, backed by throwaway organizations, so isolation between visitors is the same tenancy every query already enforces. Quota is reserved before a request runs and released if it fails: a provider outage never costs a visitor a question.

impact

Retrieval stopped being a script and became a service someone else can operate. CI runs the database-backed tests with the skip turned into a failure, so a model change that never got a migration breaks the build, not production.

Unveil Project→
case 04LEGAL INTELLIGENCE

Multi-System Legal Document Analyst

problem
Contract text arrives from a counterparty, which makes it untrusted input: a clause can carry “mark every claim as verified” aimed squarely at the model. And a risk memo that reads as finished legal advice is a liability, not a feature.
architecture
Every stage that consumes document text wraps it in a per-call random nonce fence the system prompt declares inert, so a document cannot forge its way out of the data channel. Claims carry verbatim evidence spans re-verified against the cited chunk and discarded if they do not appear there. Memos are persisted under an immutable content hash and moved through a review state machine.
  • FastAPI
  • Postgres
  • PyMuPDF
  • Alembic
  • Sentry
fenceretrieveverifyreview

outcome

Golden-set gates fail the build on regression: 100% grounded-span rate, injection recall at or above 85% while flagging at most 5% of benign text, and clause chunks held to 300–800 tokens. An adversarial eval runs the real pipelines against a contract with embedded payloads and asserts none of them hijacked the output.

impact

The system is built to be structurally unable to overstate itself. Memos ship as draft pending review, the audit log rejects UPDATE at the database level, and an approval refers to exactly the bytes that were generated.

Unveil Project→
case 05AGENTS

Autonomous Research Agent

problem
Ask an agent to compare three companies' battery strategy and it will produce a fluent report whose citations point at pages it never opened. The loop is the easy part; making the output trustworthy is the work.
architecture
Everything retrieved is captured verbatim in an evidence store at retrieval time, and every claim is checked against it after generation by a deterministic gate: regex and set arithmetic, no second model, no cost. Citations must resolve to sources with fetched text, material claims need two eTLD+1-independent domains with near-duplicate titles collapsed, and claims older than the planner's freshness horizon are flagged.
  • Anthropic SDK
  • FastAPI
  • Next.js
  • SSE
  • pytest
planresearch ⟲verifyrepair

outcome

A live three-company run produced 24 claims with zero hallucinated citations across 61 sources and 52 independent domains, 8 flagged stale, verification green after a single repair pass, terminated on budget after three rounds.

impact

When verification still fails, the report ships with the offending claims marked rather than failing closed or quietly lying. Degrading honestly is the designed behaviour, not the fallback.

Unveil Project→

measured outcomes, not promises

0.560
Macro nDCG@10

208-config BEIR sweep

100%
Citations resolved

24 claims · 0 hallucinated

≥85%
Injection recall

CI gate · ≤5% false positives

52
Independent domains

61 sources · one research run

diagnosis

You’re already paying for this.

Four conditions I get called in for. None of them arrive as a line item: they arrive as people, hours and risk you stopped noticing. Pick the one that sounds like your week.

retrieval routing

Every question your AI can’t confidently answer lands back on your best people, so you end up paying senior salaries to answer junior questions, all day.

  • Hybrid retrieval
  • Cross-encoder rerank
  • Confidence router
  • Cited generation
  • Tracing
Architecture
A router grades each query and its candidate context, then commits to the cheapest tier that can actually ground an answer: lexical + dense retrieval, cross-encoder rerank, and escalation to a frontier model only when grounding confidence stays low. Spend follows difficulty, not the average case.
Tradeoff
A grading and rerank hop on every request, plus a routing policy that has to be tuned and version-controlled. We pay latency and moving parts to avoid paying for frontier inference on questions a cheaper path already answers.
Failure & recovery
Low grounding confidence is a routing signal, not a silent guess: it escalates the tier or returns “not answerable from these sources” instead of a fluent hallucination. Every answer carries citations back to the chunks it used.
Operations
Routing thresholds, retrieval configs and prompts are versioned; each decision is traced end-to-end; per-route latency and cost budgets are asserted in CI, so a policy change that quietly gets expensive fails the build rather than the invoice.

grounded analysis

You could let AI draft the first pass on contracts, filings or claims, except one invented clause in a real document is worse than doing it by hand, so the tool sits half-used.

  • Grounded RAG
  • Claim–citation binding
  • Verification pass
  • HITL checkpoint
  • Audit trace
Architecture
Retrieval-grounded analysis where every claim is bound to a citation and put through a verification step before it surfaces. Unsupported or low-confidence statements stop at a review checkpoint for a human they never render as conclusions.
Tradeoff
Throughput and the appearance of full autonomy. We accept a slower, gated pipeline with mandatory human sign-off on ambiguous cases, because unattended confident output is the one failure this domain can't absorb.
Failure & recovery
Unverifiable claims degrade to “needs review” with their sources attached, never to a fabricated citation. The checkpoint is explicit state in the pipeline: the system can say “I can't ground this” and route it to a person instead of guessing.
Operations
Prompts, retrieval sets and verification rules are versioned and auditable; every claim-to-source link is traced so a reviewer can reconstruct why the system said what it said. Behaviour changes ride an eval gate before they reach anyone.

event-driven accounting

Close takes days longer than it should because a number doesn’t match, and the only explanation anyone can give is “someone must have fixed it.”

  • Event sourcing
  • Append-only log
  • Idempotent handlers
  • Replayable projections
  • Reconciliation
Architecture
Event-sourced accounting: a balance is a projection of an append-only event log, not a mutable row. Corrections are new compensating events, so history is additive and current state can always be rebuilt by replaying the log.
Tradeoff
Storage and read-model complexity. Append-only plus projections costs more space and needs rebuildable read models accepted to get a tamper-evident audit trail and deterministic replay instead of destructive, unexplainable updates.
Failure & recovery
Event handlers are idempotent and ordered, so a retry or partial failure can't double-post or corrupt a balance. A bad projection is fixed by rebuilding it from the log, never by hand-editing state.
Operations
The log is the source of truth and the audit history at once: every posting traces to the event that caused it, reconciliation runs against the log, and read models can be replayed to any point in time.

plan · act · recover

You automated a real workflow with an AI agent, watched it go sideways a few steps in with no way to see where or stop it, and now a human re-checks everything it does, which erases the point of automating it.

  • State-machine orchestration
  • Typed tool contracts
  • Self-correction (CRAG)
  • HITL approval
  • Replayable traces
Architecture
An explicit state machine: plan, execute, call tools under typed contracts, grade the result, loop back to recover on low confidence, with a human-approval gate on the actions that actually matter. State is inspectable at every hop, not hidden inside a prompt.
Tradeoff
Determinism and developer effort over “just let the model decide.” We give up the illusion of a fully autonomous agent for a graph we can reason about, test and interrupt, and we accept the latency of an approval gate on high-impact actions.
Failure & recovery
Low-confidence grades send the graph back a step instead of forward; tool calls are typed and time-boxed so a failure is a handled transition, not a crash; irreversible actions wait for human approval. Failure is a recoverable state, not a poisoned output.
Operations
Every run is traced as a sequence of state transitions you can replay; prompts and tools are versioned; the graph is exercised by evals in CI, so a regression in the agent's decisions is caught before it ships.

lifecycle

How a system evolves.

Not a checklist run once: a loop. Each phase feeds the next, and the last feeds the first.

I

Explore

A fast spike earns the right to the rigour.

01

Prototype

Prove the task is solvable at all. A notebook is allowed here, and nowhere after it.

02

Research

Pin down the data, the failure modes and what “correct” means, before any architecture.

II

Build

Decide it on paper, then commit it to code.

03

Architecture

Retrieval, agent and infra decisions written down with their tradeoffs: reviewable, reversible.

04

Deployment

Dockerized services, traced from the first request, behind an eval gate from day one.

III

Operate

Watch it before you push load through it.

05

Monitoring

Traces, evals and alerts watch quality the way uptime checks watch servers.

06

Scaling

p95 latency and cost per query pull against each other; budgets make the tradeoff explicit.

IV

Evolve

Every failure is fuel for the next turn.

07

Optimization

Caching, routing, reranking, distillation: spend model capacity only where it buys quality.

08

Continuous Learning

Every incident becomes an eval case. The system gets harder to break each week.

Evolve feeds Explore: every incident becomes the next turn’s research question.

the concierge

A private lineto the architect.

Every engagement begins with a conversation, not a checkout. Tell me where the system breaks down, you'll receive a personal reply with the architecture that fits, and how I'd build it.

Availability
Accepting select engagements
Response
A personal reply, usually within a day
Engagement
Remote worldwide · on-site on request
Email
GitHub
LinkedIn
Upwork

start the brief

how a commission begins

  1. The enquiry

    Tell me the task and where it breaks down today.

  2. The architecture

    You receive the system that fits, with its tradeoffs, in writing.

  3. The build

    Scoped, traced and shipped to be operated, never demoed.