Yan (Rocky) Duan

I'm building intelligent softwares and algorithms for robots at Amazon, Frontier AI & Robotics (FAR) lab. Prior to that I was one of the Co-founders and CTO at covariant.ai.

I obtained my PhD degree from UC Berkeley, advised by Pieter Abbeel. My thesis is Meta Learning for Control.

Previously, I also worked at OpenAI as a research scientist.

I'm interested in reinforcement learning, robotics, unsupervised learning, and meta learning. Also check out my Google Scholar page.

I co-organized the NIPS 2017 Deep RL Symposium and the first Deep RL Bootcamp (now Full Stack Deep Learning Bootcamp).

I served as GSI of the very first offering of the Deep RL Course at UC Berkeley (now a regular offering).

You can reach me at rockyduan94@gmail.com.

Software

• rllab: a framework for developing and evaluating reinforcement learning algorithms. The proposed benchmark problems were later incorporated into OpenAI Gym. Created in 2015, the project remains popular today, and has 2000+ stars and 600+ forks on github.

Preprints

• Adversarial Attacks on Neural Network Policies
  Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, Pieter Abbeel
  [arXiv] [videos]
• RL²: Fast Reinforcement Learning via Slow Reinforcement Learning
  Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, Pieter Abbeel
  [arXiv] [videos]

Publications

• Model-Ensemble Trust-Region Policy Optimization
  Thanard Kurutach, Ignasi Clavera, Yan Duan, Aviv Tamar, Pieter Abbeel
  International Conference on Learning Representations (ICLR), 2018
  [pdf] [videos]
• Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines
  Oral (Top 2%)
  Cathy Wu, Aravind Rajeswaran, Yan Duan, Vikash Kumar, Alexandre M Bayen, Sham Kakade, Igor Mordatch, Pieter Abbeel
  International Conference on Learning Representations (ICLR), 2018
  [pdf] [videos]
• One-Shot Imitation Learning
  Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, Wojciech Zaremba
  Neural Information Processing Systems (NIPS), 2017
  [arXiv] [videos]
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
  Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, Xi Chen, Yan Duan, John Schulman, Filip De Turck, Pieter Abbeel
  Neural Information Processing Systems (NIPS), 2017
  [arXiv] [videos]
• Variational Lossy Autoencoder
  Xi Chen, Diederik P Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, Pieter Abbeel
  International Conference on Learning Representations (ICLR), 2017
  [arXiv]
• Stochastic Neural Networks for Hierarchical Reinforcement Learning
  Carlos Florensa, Yan Duan, Pieter Abbeel
  International Conference on Learning Representations (ICLR), 2017
  [arXiv] [code] [videos]
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
  Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel
  Neural Information Processing Systems (NIPS), 2016
  [arXiv] [code]
• VIME: Variational Information Maximizing Exploration
  Rein Houthooft, Xi Chen, Yan Duan, John Schulman, Filip De Turck, Pieter Abbeel
  Neural Information Processing Systems (NIPS), 2016
  [arXiv] [code]
• Benchmarking Deep Reinforcement Learning for Continuous Control
  Yan Duan, Xi Chen, Rein Houthooft, John Schulman, Pieter Abbeel
  International Conference on Machine Learning (ICML), 2016
  [arXiv] [code] [docs]
• Deep Spatial Autoencoders for Visuomotor Learning
  Chelsea Finn, Xin Yu Tan, Yan Duan, Trevor Darrell, Sergey Levine, Pieter Abbeel
  International Conference on Robotics and Automation (ICRA), 2016
  [arXiv] [videos]
• Motion Planning with Sequential Convex Optimization and Convex Collision Checking
  John Schulman, Yan Duan, Jonathan Ho, Alex Lee, Ibrahim Awwal, Henry Bradlow, Jia Pan, Sachin Patil, Ken Goldberg, Pieter Abbeel
  International Journal of Robotics Research (IJRR), Vol. 33, No. 9, pp. 1251-1270, Aug. 2014
  [pdf]
• Gaussian Belief Space Planning with Discontinuities in Sensing Domains
  Sachin Patil, Yan Duan, John Schulman, Ken Goldberg, Pieter Abbeel
  International Conference on Robotics and Automation (ICRA), 2014
  [pdf]
• Planning Locally Optimal, Curvature-Constrained Trajectories in 3D using Sequential Convex Optimization
  Yan Duan, Sachin Patil, John Schulman, Ken Goldberg, Pieter Abbeel
  International Conference on Robotics and Automation (ICRA), 2014
  [pdf]
• Sigma Hulls for Gaussian Belief Space Planning for Imprecise Articulated Robots amid Obstacles
  Alex Lee, Yan Duan, Sachin Patil, John Schulman, Zoe McCarthy, Jur van den Berg, Ken Goldberg, Pieter Abbeel
  International Conference on Intelligent Robots and Systems (IROS), 2013
  [pdf]