five billion. It absolutely was also skilled on quite a bit much more info. But teaching on textual content taken from the internet provides new challenges. GPT-3 soaked up Considerably on the disinformation and prejudice it observed on the web and reproduced it on demand. As OpenAI acknowledged: “Web-trained versions have Online-scale biases.”