According to chip giant Intel, which unveiled details of upcoming processors on Thursday during its annual “Architecture Day” ritual, the processing of neural networks for artificial intelligence is becoming a major part of the workload of every type of chip.
In June, Koduri assumed leadership of Intel’s newly formed Accelerated Computing Unit, as part of a broad reorganization of Intel’s executive leadership under CEO Pat Gelsinger.
Koduri asserted that by speeding up the matrix multiplications at the heart of neural networks, Intel will have the fastest chips for machine learning, deep learning, and any other type of artificial intelligence processing. As part of Architecture Day, Intel demonstrated how its upcoming stand-alone GPU, Ponte Vecchio, outperformed Nvidia’s A100 GPU in a common benchmark neural network task, running the ResNet-50 neural network to categorize images from the ImageNet library of photographs.
Intel claims in the demonstration that Ponte Vecchio, in pre-production silicon, can process over 3,400 images in one second, breaking previous records of 3,000 images. That is used to train neural networks. Ponte Vecchio is able to predict over 43,000 images in a single second in the area of inference, where a trained neural net makes predictions, surpassing what it claims is the current top score of 40,000 images per second.
Intel’s Xeon chips have traditionally dominated the AI inference market, but Nvidia has been making inroads. Intel has a small share of the neural network training market, whereas Nvidia dominates the field with its GPUs.
The architecture day focuses on Intel’s roadmap for how its chips’ circuit design will lay out transistors and functional blocks on the chip, such as arithmetic logic units, caches, and pipelines. Many details about the new CPUs have already been revealed by Intel, including at last year’s Architecture Day. Years before the processors are ready to roll off the line, the company must get software designers thinking about and working on them.
Among the new information released today is that the new CPUs will use a hardware structure known as the “Thread Director.” To relieve the operating system of some of its responsibilities, the Thread Director takes control of how execution threads are scheduled to run on the processor in a way that adjusts to factors such as energy consumption.
Another new detail is the way the chips will use memory bandwidth technologies. Alder Lake, for example, will support PCIe Gen 5 and DDR 5 memory interfaces, according to the information released. Sapphire Rapids, Intel’s upcoming data center processor, includes methods for spreading data across both DDR 5 main memory and high-bandwidth HBM memory without the application software needing to know anything about either chip. This enables both memory capacity and memory bandwidth to steadily increase for AI workloads that require both memory and I/O.
Sapphire Rapids is an example of how Intel is increasingly seeing physical chip construction across multiple substrates as a competitive advantage. The use of multiple physical tiles, rather than a single monolithic semiconductor die, for example, makes use of what Intel refers to as its embedded multi-die interconnect bridge.
Because of the limitations of Moore’s Law’s traditional scaling of transistor size, Koduri believes it is critical to take advantage of other advantages that Intel can bring to chip manufacturing, such as stacking multiple die within a package.
He was referring to the observation made by Robert Dennard, an IBM scientist, in the 1970s that as more and more transistors are packed into a square area of a chip, the power consumption of each transistor decreases, resulting in a more power-efficient processor. Dennard Scaling, like Moore’s Law, is thought to be effectively extinct.
At the same time, some of Intel’s components will be manufactured by Taiwan Semiconductor Manufacturing, which also supplies Intel’s competitors. This move to selectively outsource is an extension of Intel’s current use of outsourced transistor production. It’s what CEO Gelsinger refers to as Intel’s “IDM 2.0” strategy.
When asked how Intel’s various innovations might change the way neural networks are built, Koduri said that the various types of processors now proliferating at Intel and elsewhere will have to cooperate much more and function less independently in order to share tasks.
