Staff Data Scientist
2024 – Present
Polly.io · Remote
- Led end-to-end modernization of the analytics platform, migrating 16 production pipelines from legacy batch ETL to Delta Live Tables streaming with Change Data Capture: cutting latency from 90 to 22 minutes, improving data freshness from 2-hour batch windows to 15-minute incremental CDC, and eliminating hundreds of millions of duplicate records across 10B+ daily rows. Designed a 6-level dependency graph across dual orchestration jobs with cadence-decoupled core and reporting tracks.
- Architected a five-module production ML framework for loan volume forecasting: a period-truth computation pipeline (incremental, windowed, and full-overwrite modes), a Unity Catalog Feature Store with lag, rolling, year-over-year, and holiday-effect features with leakage-controlled windowing, Optuna-tuned AutoML across nine model families (XGBoost, LightGBM, CatBoost, OLS, Ridge, Lasso, ElasticNet, ARIMA, ETS) with time-ordered cross-validation and MLflow champion selection, seeded multi-step inference for production-horizon forecasting, and row/batch monitoring with truth-lag handling. Documented in a 63-page TDD.
- Drove six-figure annualized infrastructure cost savings through a Databricks compute audit: DLT cadence optimization (21% reduction, measured Day-1), zombie cluster elimination, and Dev Cluster rightsizing (97% cost reduction).
- Designed and deployed a production capital markets analytics application on Databricks Apps: per-chart LLM interpretations grounded in domain-specific context, streaming AI sidebar chat, AI-generated PDF reporting, and SQL injection prevention via allowlist pattern with OAuth and PAT dual authentication.
- Built a tiered loan status normalization system routing through SQL lookups and cross-client consensus before invoking LLM inference, achieving near-zero per-classification cost at production scale. Improved classification accuracy by enriching LLM prompts with sequential loan history context derived via SQL window functions.
- Applied queueing-theoretic steady-state analysis and Bayesian inference to mortgage pipeline pull-through modeling. Implemented a Dirichlet-Categorical transition model (PyMC, ~400K observed transitions) with conjugate posteriors and credible intervals for sparse-event transitions, enabling confidence-aware path probability estimation and operational bottleneck identification.
- Built an automated analytics knowledge graph spanning 29 production pipelines, enabling instant blast radius analysis, AI-agent dependency lookups, and auto-publishing 1,240+ field definitions to Confluence. Operationalized as a Claude Code custom skill providing a conversational pipeline navigator with authoritative lineage lookups across the full pipeline surface.
- Architected a pull-based Delta Sharing platform enabling external financial clients to access data on demand through recipient-property row filtering and layered security, eliminating file-push workflows entirely. Onboarded 3 external financial clients with zero manual DDL, self-service configuration, and Open Sharing protocol compatibility.