In order to train more powerful large language models, researchers use vast dataset collections that blend diverse data from thousands of web sources. But as these datasets are combined and recombined into multiple collections, important information about their origins and restrictions on how they can be used are often lost or confounded in the shuffle.
Transparency is often lacking in datasets used to train large language models, study finds
