AI Engineer and Technical Product Manager. I ship production GenAI — agents, RAG, NL2SQL, evaluation — and own the product decisions around it. Two years, at a startup small enough that both were the same job.
The products are my employer's; the engineering and the decisions are mine to describe.
Designed a tool-calling layer end to end — how a task is decomposed, which tool runs, what happens when one fails, and where a human is put back in the loop.
Ingestion from documents, URLs and raw text; chunking strategy chosen against a benchmark rather than a default; embedding-model selection; retrieval tuned on real queries.
Schema-aware prompting, query generation and validation, read-only execution, streaming and structured output. Built the pipeline and owned the product through public launch.
Fixed benchmarks with recorded provenance per run — commit, prompt version, model, post-processing version — so a change can be attributed rather than guessed at.
Python services behind auth — streaming responses, webhooks, background workers, caching and usage limits. Containerised and deployed on AWS.
Supervised models taken the whole way — features known at prediction time, a time-based holdout, and evaluation against naive baselines on the task's own metric. Retail sales forecasting scored on RMSPE, classification on product data, experiment runs tracked so a result can be reproduced.
Specs with screen-by-screen UI to API mapping, validated as clickable prototypes before build. Acceptance criteria on every task, scope and launch calls, and the decision about what not to build.
The two bars are the part most demos skip, and the part that decides whether a system survives contact with real users. I have built both.
The habit underneath all of it: I measure before I choose. A model swap proves itself on accuracy against a fixed benchmark before cost enters the conversation.
Open source, and mine — plus the occasional write-up.
I trained a small AI on 5,851 correct database queries and measured it against an untrained copy of itself. It got better at the task and no detectably steadier at it — and the training quietly turned crashes into wrong answers that raise no error at all.
How safe is your local AI setup? A free, read-only security check-up for Ollama and LM Studio, plus a measurement: 32 of 43 popular Ollama models read a different tool prompt than their maker designed, and on qwen2.5 the maker's layout scored 3 to 4 more right in every 100.
Ask an AI the same database question ten times: do you get the same right answer? 14,880 scored AI-written queries show a 13.9-point gap between what a model can do and what it does every time, and that most of the main model's mistakes raise no error at all.
Three AI agents run the finding-and-reporting half of a penetration test: a Planner picks the scanners, an Executor runs them, a Reporter writes the fixes. The AI may choose; code decides what is allowed. Every finding traces back to raw scanner output, and the Planner passed 36 of 36 test cases across three models.
Why a cheaper model doesn't reliably cut the cost of an agent, why the biggest model isn't the answer either, and how to evaluate every node that calls a model on accuracy, latency and cost. With a cost model, a small tool-calling agent built as a test subject, and two measured runs of 240 conversations each.
A habit coach where the model never does arithmetic. Streaks, rates and rankings are pure Python, and the repo has 153 tests; the model is only given the computed statistics, and every answer is checked against them before it is shown.
A scored benchmark harness for PDF text extraction: four hand-verified documents — ten scripts including right-to-left and Indic, three columns, a dense table, a scan — scored on character and word error rate. No parser wins everywhere.
Medical Q&A grounded in a PDF corpus, with every answer citing the pages it came from — and a prompt that tells the model to say "my sources don't cover that" instead of improvising.
Semantic product search where price and rating filters run inside the vector query, served over FastAPI.
A six-week-ahead daily sales forecast for 1,115 stores, built only from inputs known at forecast time and scored on the competition metric against naive baselines. LightGBM: RMSPE 0.1235 against 0.1449 for the best baseline.
Long-form pieces in plain words, each with the code and data behind it.
Running AI on your own computer keeps your chats off other companies' servers. It doesn't make your setup safe. Here are the six weak spots, what researchers and I found in each, and a ten-minute check-up.
A paired test of 1,240 tool-calling questions on qwen2.5: the prompt layout Ollama uses against the one the model's maker designed, read by two different parsers. The maker's layout got 3 to 4 more right in every 100.
Everything below is something I have actually built with. The terms teams ask for most lead each group.
Which model and vendor, RAG or an agent loop, what cost and latency the accuracy is worth, and how exposed a choice leaves us when a provider deprecates it.
Owned AI products end to end while building them — agent capability from prototype to production, NL2SQL, retrieval systems, and the evaluation and observability layers underneath.
Wrote the specs and acceptance criteria the team built against, made the model and architecture calls, and led a cross-functional team of six through to public launch.
Six-month full-time internship on the team's LLM work — retrieval pipelines, prompt and evaluation experiments, and the Python services around them. It led to the full-time offer.
Most people in AI are the engineer or the one deciding what to build. I ended up both, at a startup small enough that the title covered the job rather than describing it. I decide what to build and what not to, turn it into specs engineering can work from, and build the parts I specify — which is why my feasibility calls hold up in the room.
What I want next is more depth on production AI: evaluation, orchestration and reliability as the job rather than the afterthought.