A team of more than 30 OpenAI researchers have released a paper about GPT-3, a language model capable of achieving state-of-the-art results on a set of benchmark and unique natural language processing tasks that range from language translation to generating news articles to answering SAT questions. GPT-3 has a whopping 175 billion parameters. By comparison, the largest version of GPT-2 was 1.5 billion parameters, and the largest Transformer-based language model in the world — introduced by Microsoft earlier this month — is 17 billion parameters.
OpenAI released GPT-2 last year, controversially taking a staggered release approach due to fear that the model could be used for malicious purposes. OpenAI was criticized by some for the staggered approach, while others applauded the company for demonstrating a way to carefully release an AI model with the potential for misuse. GPT-3 made its debut with a preprint arXiv paper Thursday, but no release details are provided. An OpenAI spokesperson declined to comment when VentureBeat asked if a full version of GPT-3 will be released or one of seven smaller versions ranging in size from 125 million to 13 billion parameters.
Comments are closed.