Katarzyna Petru
As reported by techradar, co-founder of id Software, John Carmack, suggested that very long fiber optic cables could replace traditional RAM in AI systems. We're talking about up to 200 km of cable, which at a bandwidth of 256 Tb/s would be able to "store" about 32 GB of data in transmission. According to Carmack, AI models take advantage of predictable patterns in weight access, so one can envision an architecture without traditional DRAM, where data is continuously streamed to cache through looped fiber optic cables. This is a response to the growing problem of memory constraints in data centers. For now, it’s a concept, but it shows an alternative direction for the development of AI infrastructure.
The Future of AI Memory Under the Microscope
The current memory market is under immense pressure. Demand driven by AI significantly exceeds supply, and this is not only about server RAM in data centers but also about memory in accelerators and graphics cards. This is why John Carmack is looking for alternatives in the form of fiber optic infrastructure that could alleviate the burden on traditional memory modules and reduce the domino effect on prices seen today throughout the PC segment.
However, the very idea has serious barriers. A massive amount of fiber would be needed, and maintaining signal quality in such a loop is not trivial. Carmack also points to a more down-to-earth option, which is to combine many inexpensive flash modules into a parallel high-throughput system, provided a fast interface is created that works directly with AI accelerators.
This solution seems closer to realization, but it requires cooperation between GPU manufacturers and storage memory. Meanwhile, the RAM crisis may last for many more months, which is why searching for new architectures for AI models is no longer an experiment but a necessity.
source: techradar.com
