Why it exists
Without production click data, a ranking prototype still needs a way to separate promising behavior from a convincing demo.
Portfolio / CASE STUDY
Private AI evaluation prototype
Ad Classifier.
A synthetic ad ranking and creative review lab that compares a typed model with heuristic and oracle baselines.
MY ROLE
Independent ranking, simulation, and product engineering
What I built
I built a synthetic audience and inventory, a ranking pipeline, and an auction simulator. A review interface lets someone inspect creative judgments and compare ranking arms.
Project in view
Engineering decisions
Separate retrieval from ranking
Candidate retrieval narrows the inventory, then the ranker scores the pool. Page selection discounts repeated topics to diversify the result.
Calibrate with known answers
A synthetic engagement function provides an oracle. A heuristic and an offline stub make it possible to compare behavior under repeatable conditions.
Measure quality and operation together
The prototype tracks calls, latency, token use, and fallback behavior so ranking quality can be considered alongside operating costs.
Capabilities in practice
- AI evaluation design
- Ranking systems
- Synthetic benchmark design
- Model fallback
Status & scope
Private research prototype. Experiments use synthetic ads and audiences. I can discuss the evaluation design on request.
Private repository ad-classifier