Katarzyna Petru
As reported by techradar, co-founder of id Software, John Carmack, suggested that a very long fibre optic cable could replace traditional RAM in AI systems. We're talking about as much as 200 km of cable, which with a bandwidth of 256 Tb/s would be able to 'store' around 32 GB of data in transmission. According to Carmack, AI models rely on predictable access patterns to weights, so one can imagine architecture without traditional DRAM, where data is constantly streamed to cache via a looped fibre optic cable. This is a response to the growing issue of memory limitations in data centres. For now, it’s just a concept, but it shows an alternative direction for the development of AI infrastructure.
the future of AI memory under scrutiny
The current memory market is under enormous pressure. Demand driven by AI significantly exceeds supply, and it's not just about server RAM in data centres, but also memory in accelerators and graphics cards. This is why John Carmack is looking for alternatives in the form of fibre optic infrastructure, which could alleviate the strain on traditional memory modules and reduce the domino effect on prices visible today across the entire PC segment.
However, the very idea has serious barriers. A massive amount of fibre would be needed, and maintaining signal quality in such a loop is not trivial. Carmack also points to a more down-to-earth option, which is combining multiple cheap flash modules into a parallel high-bandwidth system, provided a fast interface is created that directly cooperates with AI accelerators.
This solution seems closer to implementation, but it requires cooperation between GPU manufacturers and storage memory producers. Meanwhile, the RAM crisis may last for many more months, so the search for new architectures for AI models is no longer an experiment but has become a necessity.
source: techradar.com
