Measuring the Intrinsic Dimension of Objective Landscapes

Measuring the Intrinsic Dimension of Objective Landscapes

  • April 27, 2018
Table of Contents

Measuring the Intrinsic Dimension of Objective Landscapes

In our paper, Measuring the Intrinsic Dimension of Objective Landscapes, to be presented at ICLR 2018, we contribute to this ongoing effort by developing a simple way of measuring a fundamental network property known as intrinsic dimension. In the paper, we develop intrinsic dimension as a quantification of the complexity of a model in a manner decoupled from its raw parameter count, and we provide a simple way of measuring this dimension using random projections. We find that many problems have smaller intrinsic dimension than one might suspect.

By using intrinsic dimension to compare across problem domains, we measure, for example, that solving the inverted pendulum problem is about 100 times easier than classifying digits from MNIST, and playing Atari Pong from pixels is about as hard as classifying CIFAR-10.

Source: uber.com

Tags :
Share :
comments powered by Disqus

Related Posts

DeepMind papers at ICLR 2018

DeepMind papers at ICLR 2018

Between 30 April and 03 May, hundreds of researchers will gather in Vancouver, Canada, for the Sixth International Conference on Learning Representations. Here you will find details of all DeepMind’s accepted papers.

Read More
Introduction to Decision Tree Learning

Introduction to Decision Tree Learning

From Kaggle to classrooms, one of the first lessons in machine learning involves decision trees. The reason for the focus on decision trees is that they aren’t very mathematics heavy compared to other ML approaches, and at the same time, they provide reasonable accuracy on classification problems.

Read More
TDM: From Model-Free to Model-Based Deep Reinforcement Learning

TDM: From Model-Free to Model-Based Deep Reinforcement Learning

While simple, this thought experiment highlights some important aspects of human intelligence. For some tasks, we use a trial-and-error approach, and for others we use a planning approach. A similar phenomena seems to have emerged in reinforcement learning (RL).

Read More