Katarzyna Petru
As reported by TechRadar, co-founder of id Software, John Carmack, suggested that a very long fibre optic cable could replace traditional RAM in AI systems. This concerns up to 200 km of cable, which, at a bandwidth of 256 Tb/s, could "store" around 32 GB of data in transit. According to Carmack, AI models utilise predictable patterns of access to weights, so one can imagine an architecture without traditional DRAM, where data is continuously streamed to cache via a looped fibre optic cable. This is a response to the growing problem of memory limitations in data centres. For now, it’s a concept, but it shows an alternative direction for the development of AI infrastructure.
the future of AI memory under scrutiny
The current memory market is under immense pressure. Demand driven by AI significantly exceeds supply, and this applies not only to server RAM in data centres but also to memory in accelerators and graphics cards. That is why John Carmack is looking for alternatives in the form of fibre optic infrastructure that could offload classic memory modules and reduce the domino effect on prices currently visible throughout the PC segment.
However, the very idea has serious barriers. A huge amount of fibre would be required, and maintaining signal quality in such a loop is not trivial. Carmack also points to a more down-to-earth option, which is to combine many cheap flash modules into a parallel high-throughput system, provided that a fast interface is created that directly works with AI accelerators.
This solution seems closer to realisation, but requires cooperation between GPU and storage manufacturers. Meanwhile, the RAM crisis may last for many more months, which is why searching for new architectures for AI models is no longer an experiment, but rather a necessity.
source: techradar.com
