Following the impressive surge in NVDA’s stock price in recent months, the question on investors’ minds is: which company holds the potential to become the next NVDA? The answer is companies that can offer solutions to the “Memory Wall” challenge in Generative AI cloud workloads.
What is a Memory Wall?
Many of the Generative AI Large Language Models available today are built upon Google’s Transformer system. This system utilizes a Neural Network composed of modules that analyze a wide range of contextual information and historical data. To effectively handle these modules, the Neural Network Algorithm requires a stored cache of information that the model can reference and process. Once this information is processed and saved, the algorithm can proceed to handle subsequent modules. However, the Neural Network Algorithm might have to revisit the previous module in order to reprocess information. Consequently, the Algorithm moves back and forth between modules, which requires a substantial amount of memory. Memory on chips is severely constrained due to space constraints, hence, the Neural Network Algorithm must access supplementary memory off-the-chip package. Unfortunately, once the Algorithm ventures off-the-chip for additional memory, the process slows down significantly as off-chip communication operates at a fraction of the speed compared to on-the-chip communication. Furthermore, during the time when the Algorithm is retrieving data from the off-chip memory, the Processor remains inactive and utilizes only a fraction of its capacity, typically ranging from 10% to 20%. This also results in a significant waste of power as the Processor is still consuming power. This inefficiency can lead to financial losses amounting to millions of dollars across various model training tasks and exponentially increase the training time of a Generative AI Large Language Model. A Memory Wall refers to this computer architecture issue that is a significant performance bottleneck between the processor and the memory for training Generative AI Large Language Models.
The silicon companies that could provide a solution to this “Memory Wall” problem are attracting the next phase of investments in Generative AI stock rally.
#GenerativeAI #semiconductors #AI #stockstowatch