News

Improving Language Understanding with Unsupervised Learning

Improving Language Understanding with Unsupervised Learning

We’ve obtained state-of-the-art results on a suite of diverse language tasks with a scalable, task-agnostic system, which we’re also releasing. Our approach is a combination of two existing ideas: transformers and unsupervised pre-training. These results provide a convincing example that pairing supervised learning methods with unsupervised pre-training works very well; this is an idea that many have explored in the past, and we hope our result motivates further research into applying this idea on larger and more diverse datasets.

Read More
Attacks against machine learning – an overview

Attacks against machine learning – an overview

At a high level, attacks against classifiers can be broken down into three types: Adversarial inputs, which are specially crafted inputs that have been developed with the aim of being reliably misclassified in order to evade detection. Adversarial inputs include malicious documents designed to evade antivirus, and emails attempting to evade spam filters. Data poisoning attacks, which involve feeding training adversarial data to the classifier.

Read More
Why do neural networks generalize so poorly?

Why do neural networks generalize so poorly?

Deep convolutional network architectures are often assumed to guarantee generalization for small image translations and deformations. In this paper we show that modern CNNs (VGG16, ResNet50, and InceptionResNetV2) can drastically change their output when an image is translated in the image plane by a few pixels, and that this failure of generalization also happens with other realistic small image transformations. Furthermore, the deeper the network the more we see these failures to generalize.

Read More
Our cancer preventing genes revealed

Our cancer preventing genes revealed

In our bodies, we all have genes working hard to prevent cancer. If they don’t do their job properly, rogue cells can mutate and develop into the life-threatening disease. The malfunction of one so-called “super tumour suppressor gene” known as p53 causes at least half of all cancers.

Read More
China’s Silicon Valley Faces Meltdown Fears

China’s Silicon Valley Faces Meltdown Fears

Shenzhen is a long way from Silicon Valley. Tech companies are housed in gleaming skyscrapers rather than on rolling campuses; there is scarcely a hoodie to be seen and the Communist Party influence is never far away. Just across the border from Hong Kong and lying in the Pearl River Delta, this metropolis of some 12 million people is home to some of Chinaâs biggest tech players, including the social media giant Tencent valued at over $500 billion and telecom equipment groups ZTE and Huawei.

Read More
Tags