When will it all end: how much longer will we have data for training AI?

Modern artificial intelligence models, particularly the widely adopted LLMs (large language models), rely on vast amounts of information, striving to use all existing quality sources for training. Historically, computational power has been the key issue for AI development, but in recent years, the pace of technological progress has begun to outstrip the rate at which new data is created for datasets. With the advent of powerful chips, many researchers have become concerned that a shortage of quality information used for training models is not far off.

June 24, 2024