Ricky Lee

Mountain View, CA

LinkedInX
RL

About

I am comfortable working with large-scale data and developing and deploying ML models to production, managing every step of the ML model development cycle, from data pipelines and feature engineering to model training, evaluation, deployment, and A/B testing. In the past, I have worked on building DNN-based recommendation models at AppLovin, developed a multi-modal deep semantic embedder for Sponsored Products at Amazon, and led the deployment of open-source LLMs for text generation inference at DreamTavern.

Work Experience

DreamTavern (Raised seed round from Lux Capital, BoxGroup)
ML Consulting

2023 - 2023

ML Consultant

  • Led the deployment of open-source LLMs for text generation inference, which replaced over 90% of OpenAI (gpt-3.5-turbo) API calls and expanded model offerings to over seven, providing better steerability and story generation while achieving comparable cost and Tokens Per Second.
  • Conducted comprehensive testing across multiple cloud providers (AWS, RunPod), GPUs (A100, A40, A6000), LLMs (primarily Llama2-based), and inference frameworks (Hugging Face TGI, vLLM) to find the optimal blend of cost-efficiency, low latency and output quality.
  • Performed qualitative evaluations of model outputs and optimized inference parameters and prompts for seamless integration with various LLM-enabled backend services, ensuring alignment with the desired user experience.

    • AppLovin
    Core Engineering

    2020 - 2023

    Data Scientist

  • Developed tree-based and DNN recommendation models predicting key ad engagement and revenue metrics that power AppDiscovery, a UA product generating over $300M quarterly revenue, using BigQuery, Spark, XGBoost, and PyTorch. [1]
  • Led developments and iterative improvements of ad revenue and event rate models and contributed to the install rate model, increasing advertiser spend and/or profit margins by up to 10% per iteration. Managed deployment and A/B testing for all models.
  • Created a gradient-based feature importance measurement and visualization tool using PyTorch Captum to guide feature and model architecture development for the first launch of deep learning bidder models (Axon 2.0 for AppDiscovery).[2]
  • Implemented ML metric logging and monitored 1,000+ models in production using Weights & Biases and Grafana.

    • Machine Zone (Acq. by AppLovin)
    Marketing Data Science Research

    2019 - 2020

    Data Scientist

  • Automated the aggregation of costs associated with fraudulent marketing channels using Airflow, which saved $500k+(~5% of spend) of monthly cost and replaced manual efforts with auto-generated reports.
  • Designed and implemented new signals for a fraud detection system that multiple teams of marketing analysts used daily for campaign optimization, using Pandas, Spark and MySQL.
  • Led technical communications with ad networks (Unity, etc.) for refund negotiations, by explaining statistical methodologies used for fraud detection.

    • A9.com (Amazon)
    Ad Technology Predictive Modeling

    2017 - 2017

    Graduate Software Development Intern

  • Researched and developed a Deep Semantic Embedding for Sponsored products that jointly embedded query texts and Amazon products for use in ranking, filtering, and other downstream tasks.
  • Implemented a multimodal (text + image) two-tower neural network architecture using TensorFlow, trained on a dataset of 10+ million user purchases with contrastive loss, improving AUC by 2% compared to text-only representations.
  • Created visualization and retrieval demos for in-depth qualitative analysis and final presentation.

    • Education

    Stanford University

    2016 - 2018
    Master of Science in Statistics (Track: Data Science)

    Columbia University

    2012 - 2016
    Bachelor of Science in Applied Mathematics (Cum Laude)

    Skills

    Python
    C++
    Java
    R
    SQL
    Spark
    Hive
    Docker
    Kubernetes
    Airflow
    Grafana
    Google Cloud Platform
    Amazon Web Services
    PyTorch
    TensorFlow
    Keras
    Hugging Face
    XGBoost
    scikit-learn
    Pandas
    Weights & Biases
    Grafana

    Projects

    Neural Network with CUDA & MPI

    Trained an MLP network in C++, parallelizing key operations such as matrix multiplication and back-propagation with CUDA and trained across 4 GPUs and processes using MPI.

    c++
    cuda
    mpi
    profiling

    Fighting Zombies in Minecraft with Deep Reinforcement Learning

    Trained a Minecraft agent to combat zombies with DQN algorithm using TensorFlow on Project Malmo (Minecraft simulation platform).

    python
    tensorflow
    deep q learning