Relativistic Monte Carlo

Xiaoyu Lu, Valerio Perrone, Leonard Hasenclever, Yee Whye Teh, Sebastian J. Vollmer
arXiv preprint arXiv:1609.04388,   October 2016.

(Accepted by Artificial Intelligence and Statistics Conference (AISTATS), 2017)

Abstract

Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCMC) algorithm that generates proposals for a Metropolis-Hastings algorithm by simulating the dynamics of a Hamiltonian system. However, HMC is sensitive to large time discretizations and performs poorly if there is a mismatch between the spatial geometry of the target distribution and the scales of the momentum distribution. In particular the mass matrix of HMC is hard to tune well. In order to alleviate these problems we propose relativistic Hamiltonian Monte Carlo, a version of HMC based on relativistic dynamics that introduce a maximum velocity on particles. We also derive stochastic gradient versions of the algorithm and show that the resulting algorithms bear interesting relationships to gradient clipping, RMSprop, Adagrad and Adam, popular optimisation methods in deep learning. Based on this, we develop relativistic stochastic gradient descent by taking the zero-temperature limit of relativistic stochastic gradient Hamiltonian Monte Carlo. In experiments we show that the relativistic algorithms perform better than classical Newtonian variants and Adam.

Tucker Gaussian Process for Regression and Collaborative Filtering

Hyunjik Kim, Xiaoyu Lu, Seth Flaxman, Yee Whye Teh
arXiv preprint arXiv:1605.07025v2 ,   October 2016.

(Under review by Conference on Neural Information Processing Systems (NIPS), 2017)

Abstract

We introduce the Tucker Gaussian Process (TGP), a model for regression that regularises a Gaussian Process (GP) towards simpler regression functions for enhanced generalisation performance. We derive it using a novel approach to scalable GP learning, and show that our model is particularly well-suited to grid-structured data and problems where the dependence on covariates is close to being separable. A prime example is collaborative filtering, for which our model provides an effective GP based method that has a low-rank matrix factorisation at its core. We show that TGP generalises classical Bayesian matrix factorisation models, and goes beyond them to give a natural and elegant method for incorporating side information.

  • Non-stipendiary Lecturer, Oxford University10/2016 - Present

    Lecturer in Probability and Statistics

  • Business Technical Intern, Google, Ireland07/2015 - 09/2015

    • Collect big data from database using query languages such as SQL.

    • Create competitive analysis and benchmarking study for account hijacking, recommend strategy ad- justments based on findings.

    • Analyse hijacking trends within a specific set of products and develop an action plan based on trends and patterns.

    • Analyse preventable abuse related issues which impact users, and identify core and common prevention focus areas across Product Quality Operation.

    • Partner with engineering teams to improve our hijacking prevention, detection and recovery systems.

    • Build statistical models to select relavant features and predict goodness/badness of clusters of accounts.

    • Delivered excellent presentation and documentation.

  • Quantitative Strategies Summer Analyst, Credit Suisse, London06/2014-08/2014

    • Build pricing models for Calendar Spread Options using Excel and VBA.

    • Perform model calibration and validation, as well as hedging simulation for historical data.

    • Delivered excellent results and received exceptional feedback from managers and colleagues.

  • Power Trading Intern,Gazprom Marketing and Trading, London07/2013-09/2013

    • Build predictive models for bid-offer curves for forecasting in the European power market.

    • Data sampling and manipulation using statistical and programming tools including R and Python.

    • Have received excellent feedback and successfully implemented the pricing model which was in pro- duction.

  • Summer Intern,Guotai Junan Securities, Qingdao, China06/2012-08/2012

    • In charge of assisting with daily business and organising files in a group.

    • Provide customer service and maintain relationships with clients in a fast-paced environment.

    • Improved customers’ satisfaction by 10%.

  • PhD2014 - Present
    Machine Learning

    University of Oxford, New College

  • M.Math.2010 - 2014
    Mathematics and Statistics

    University of Oxford, Lady Margaret Hall

  • 2014
    Clarendon Fund Scholarship, PAG Oxford Scholarship
    • Awarded to the top 3% of accepted graduate students across all disciplines at the University of Oxford
    • Full scholarship for PhD studies
  • 2014
    Royal Statistical Society Prize
    • Ranked top 1st at University of Oxford.
  • 2014
    Gibbs Prize
    • Awarded to the best undergraduate student at Department of Statistics, University of Oxford.
  • 2013
    Department of StatisticsPrize 2013
    • Awarded due to excellent academic performance.
  • 2013
    Top ten finalist, TARGETjobs
    • Top ten finalist for the Mathematics, Economics and Finance Undergraduate Of The Year Award.
  • 2013
    Certificate of Appreciation
    • Awarded due to outstanding volunteer service in Chinglai, Thailand.
  • Language
    • Chinese: Mothertongue
    • English: Fluent
  • IT Skills
    • Intermediate Knowledge: Julia, Python, R, Matlab, Linux, Latex.
    • • Basic Knowledge: Microsoft Offices, VBA.
  • Sports and Arts
    • I am a current member of Oxford University Dancesports Club Beginners Team, dancing for Latin and Ballroom.
    • Achieved Grade 9 (Chinese Musician Association Certificate) in Piano.
  • Volunteering
    • I volunteered at Camillian Social Centre in Chinglai, Thailand in 2013, teaching disabled children Maths, yoga, and social science, as well as organising tailored activities for individuals, which was an invaluable and enjoyable experience for me.
  • Societies
    • Talent Management Team Leader, AIESEC, 10/2012-06/2012
    Responsible for organising recruitment events, socials and running training sessions.
    • LMH(Lady Margaret Hall) Ambassador, 10/2012-06/2012
    Responsible for college tours, Q&As and school visits.