Language model pretraining has led to significant performance gains but careful
comparison between different approaches is challenging. Training is computationally …

Despite considerable advancements with deep neural language models, the enigma of
neural text degeneration persists when these models are tested as text generators. The …

Recent progress in natural language processing has been driven by advances in both
model architecture and model pretraining. Transformer architectures have facilitated …

Over the last several years, the field of natural language processing has been propelled
forward by an explosion in the use of deep learning models. This article provides a brief …

Context. Electroencephalography (EEG) is a complex signal and can require several years
of training, as well as advanced signal processing and feature extraction methodologies to …

We present a large, tunable neural conversational response generation model, DialoGPT
(dialogue generative pre-trained transformer). Trained on 147M conversation-like …

This paper demonstrates that multilingual denoising pre-training produces significant
performance gains across a wide variety of machine translation (MT) tasks. We present …

Deep learning (DL) techniques have obtained remarkable achievements on various tasks,
such as image recognition, object detection, and language modeling. However, building a …

Recent progress in natural language generation has raised dual-use concerns. While
applications like summarization and translation are positive, the underlying technology also …

Large-scale language models show promising text generation capabilities, but users cannot
easily control particular aspects of the generated text. We release CTRL, a 1.63 billion …