USC, Viterbi School of Engineering
AMAR
A multi-agent RAG pipeline that turns plain-language questions into analytical reports.
Sep 2024 – Jun 2025
Framed for
A 50M-events/day multi-agent pipeline on Databricks and Airflow
Orchestrated RAG on Databricks with Airflow DAGs and an MLflow retraining lifecycle, stress-tested at 50M events/day against replayed public clickstream data.
- 50M
- events/day sustained in load test
- ~5 min
- end-to-end pipeline run
- MLflow
- automated retraining + tuning
- Databricks
- Airflow
- MLflow
- RAG
- Multi-Agent
- Python
Cutting analyst report turnaround from an hour to five minutes
Designed a system that lets non-technical stakeholders get custom analytical reports directly from a plain-language question, removing the analyst queue as a bottleneck across three business domains.
- ~1 hr → 5 min
- report turnaround
- 3
- business domains served
- 0
- analyst hand-offs required
- Multi-Agent Systems
- RAG
- Databricks
- Airflow
- MLflow
The problem
Non-technical stakeholders needed custom analytical reports but had to route every request through an analyst, costing roughly an hour per report and gating insight behind a person's queue.
What I built
- 01Built a multi-agent RAG pipeline that parses a plain-language question, plans the analysis, and produces a custom report end to end.
- 02Orchestrated the pipeline on Databricks with Airflow DAGs so runs were scheduled, observable, and reproducible.
- 03Wired automated retraining and parameter tuning through the Databricks MLflow model lifecycle rather than manual redeploys.
- 04Stress-tested at 50M events/day using replayed public clickstream data before trusting the throughput numbers.
Outcome
Report turnaround dropped from roughly an hour of manual analyst work to about five minutes, across financial-reporting, healthcare-diagnostics, and customer-support scenarios.