Walt Disney Parks and Resorts Online, Yield & Optimization Manager
Building the testing practice behind Disney’s booking flow
Led the machine-learning testing team, set experimentation standards on Adobe Target, and built the dashboards used to prioritize campaigns across the US, Japan and France.
Result: A/B, MVT and ML tests at global scale
Statistical & Experimentation Methodologies
- Hypothesis Testing: Frequentist (two-proportion z-tests, t-tests, chi-square) and Bayesian experimentation frameworks.
- Experimental Design: Power analysis, sample size estimation, Minimum Detectable Effect (MDE) calculation, and test duration forecasting.
- Variance Reduction & Advanced Techniques: CUPED (Controlled-experiment Using Pre-Experiment Data), stratified sampling, and multi-armed bandits (MAB).
- Bias & Error Mitigation: Sample Ratio Mismatch (SRM) detection, sequential testing / peeking corrections (e.g., alpha spending, always-valid p-values), and multiple testing corrections (Bonferroni, Benjamini-Hochberg).
- Alternative Causal Inference: Difference-in-Differences (DiD), uplift modeling, and propensity score matching (for when randomized controlled trials aren't feasible).
Python & Data Science Stack
- Core Data Manipulation: Pandas, NumPy, Polars (for large-scale event log processing).
- Statistical Modeling: SciPy (scipy.stats), Statsmodels, Pingouin.
- Data Visualization & Storytelling: Matplotlib, Seaborn, Plotly (building automated readout charts and confidence interval plots).
- Data Extraction & Warehousing: SQL (BigQuery, Snowflake, Redshift), Python-to-warehouse connectors (google-cloud-bigquery, SQLAlchemy).
- Workflow & Reproducibility: Jupyter Notebooks, Git/GitHub, automated Python reporting pipelines.
Analytics, Tagging & Experimentation Infrastructure
- Testing Platforms: Optimizely, VWO, Adobe Target, Statsig, LaunchDarkly, or custom feature-flagging architectures.
- Telemetry & Event Tracking: Google Analytics 4 (GA4), Adobe Analytics, Google Tag Manager (GTM), Segment, raw event-stream validation.
- Data Governance: Designing standardized tracking schemas, ensuring clean assignment logging, and auditing pre-test telemetry.
Leadership & Program Strategy
- Experimentation Roadmap Management: Prioritization frameworks (ICE, PIE, RICE), intake workflows, and velocity scaling across product and marketing teams.
- Cross-Functional Alignment: Partnering with engineering, UX/UI, product management, and executive stakeholders to translate business questions into testable hypotheses.
- Governance & Culture: Establishing experimentation Centers of Excellence (CoE), standardizing readout templates, and reducing false-positive deployments.
- Business Impact: Connecting micro-conversions to macro-revenue metrics, holdout group validation, and incremental lift reporting.