In 1997 the IBM Deep Blue supercomputer defeated world chess champion Garry Kasparov. It was a groundbreaking demonstration of supercomputer expertise and a primary glimpse into how high-performance computing would possibly someday overtake human–degree intelligence. Within the 10 years that adopted, we started to make use of synthetic intelligence for a lot of sensible duties, reminiscent of facial recognition, language translation, and recommending motion pictures and merchandise.
Quick-forward one other decade and a half and synthetic intelligence has superior to the purpose the place it could actually “synthesize information.” Generative AI, reminiscent of ChatGPT and Secure Diffusion, can compose poems, create paintings, diagnose illness, write abstract studies and laptop code, and even design built-in circuits that rival these made by people.
Super alternatives lie forward for synthetic intelligence to change into a digital assistant to all human endeavors. ChatGPT is an efficient instance of how AI has democratized the usage of high-performance computing, offering advantages to each particular person in society.
All these marvelous AI functions have been resulting from three components: improvements in environment friendly machine-learning algorithms, the supply of huge quantities of information on which to coach neural networks, and progress in energy-efficient computing via the development of semiconductor expertise. This final contribution to the generative AI revolution has acquired lower than its justifiable share of credit score, regardless of its ubiquity.
During the last three a long time, the main milestones in AI had been all enabled by the modern semiconductor expertise of the time and would have been not possible with out it. Deep Blue was applied with a mixture of 0.6- and 0.35-micrometer-node chip-manufacturing expertise. The deep neural community that gained the ImageNet competitors, kicking off the present period of machine studying, was applied with 40-nanometer expertise. AlphaGo conquered the sport of Go utilizing 28-nm expertise, and the preliminary model of ChatGPT was educated on computer systems constructed with 5-nm expertise. The latest incarnation of ChatGPT is powered by servers utilizing much more superior 4-nm expertise. Every layer of the pc programs concerned, from software program and algorithms right down to the structure, circuit design, and system expertise, acts as a multiplier for the efficiency of AI. However it’s honest to say that the foundational transistor-device expertise is what has enabled the development of the layers above.
If the AI revolution is to proceed at its present tempo, it’s going to wish much more from the semiconductor business. Inside a decade, it’s going to want a 1-trillion-transistor GPU—that’s, a GPU with 10 instances as many units as is typical at present.
Advances in semiconductor expertise [top line]—together with new supplies, advances in lithography, new forms of transistors, and superior packaging—have pushed the event of extra succesful AI programs [bottom line]
Relentless Progress in AI Mannequin Sizes
The computation and reminiscence entry required for AI coaching have elevated by orders of magnitude previously 5 years. Coaching GPT-3, for instance, requires the equal of greater than 5 billion billion operations per second of computation for a whole day (that’s 5,000 petaflops-days), and three trillion bytes (3 terabytes) of reminiscence capability.
Each the computing energy and the reminiscence entry wanted for brand new generative AI functions proceed to develop quickly. We now have to reply a urgent query: How can semiconductor expertise preserve tempo?
From Built-in Units to Built-in Chiplets
Because the invention of the built-in circuit, semiconductor expertise has been about cutting down in function measurement in order that we will cram extra transistors right into a thumbnail-size chip. At the moment, integration has risen one degree larger; we’re going past 2D scaling into 3D system integration. We are actually placing collectively many chips right into a tightly built-in, massively interconnected system. This can be a paradigm shift in semiconductor-technology integration.
Within the period of AI, the potential of a system is immediately proportional to the variety of transistors built-in into that system. One of many important limitations is that lithographic chipmaking instruments have been designed to make ICs of not more than about 800 sq. millimeters, what’s known as the reticle restrict. However we will now lengthen the scale of the built-in system past lithography’s reticle restrict. By attaching a number of chips onto a bigger interposer—a bit of silicon into which interconnects are constructed—we will combine a system that comprises a a lot bigger variety of units than what is feasible on a single chip. For instance, TSMC’s chip-on-wafer-on-substrate (CoWoS) expertise can accommodate as much as six reticle fields’ value of compute chips, together with a dozen high-bandwidth-memory (HBM) chips.
HBMs are an instance of the opposite key semiconductor expertise that’s more and more essential for AI: the flexibility to combine programs by stacking chips atop each other, what we at TSMC name system-on-integrated-chips (SoIC). An HBM consists of a stack of vertically interconnected chips of DRAM atop a management logic IC. It makes use of vertical interconnects known as through-silicon-vias (TSVs) to get indicators via every chip and solder bumps to kind the connections between the reminiscence chips. At the moment, high-performance GPUs use HBMextensively.
Going ahead, 3D SoIC expertise can present a “bumpless different” to the traditional HBM expertise of at present, delivering far denser vertical interconnection between the stacked chips. Current advances have proven HBM take a look at constructions with 12 layers of chips stacked utilizing hybrid bonding, a copper-to-copper reference to a better density than solder bumps can present. Bonded at low temperature on high of a bigger base logic chip, this reminiscence system has a complete thickness of simply 600 µm.
With a high-performance computing system composed of a lot of dies working giant AI fashions, high-speed wired communication could shortly restrict the computation velocity. At the moment, optical interconnects are already getting used to attach server racks in information facilities. We’ll quickly want optical interfaces based mostly on silicon photonics which can be packaged along with GPUs and CPUs. This can enable the scaling up of energy- and area-efficient bandwidths for direct, optical GPU-to-GPU communication, such that lots of of servers can behave as a single big GPU with a unified reminiscence. Due to the demand from AI functions, silicon photonics will change into one of many semiconductor business’s most essential enabling applied sciences.
Towards a Trillion Transistor GPU
As famous already, typical GPU chips used for AI coaching have already reached the reticle subject restrict. And their transistor depend is about 100 billion units. The continuation of the pattern of accelerating transistor depend would require a number of chips, interconnected with 2.5D or 3D integration, to carry out the computation. The mixing of a number of chips, both by CoWoS or SoIC and associated superior packaging applied sciences, permits for a a lot bigger whole transistor depend per system than might be squeezed right into a single chip. We forecast that inside a decade a multichiplet GPU may have greater than 1 trillion transistors.
We’ll have to hyperlink all these chiplets collectively in a 3D stack, however luckily, business has been in a position to quickly scale down the pitch of vertical interconnects, rising the density of connections. And there’s loads of room for extra. We see no cause why the interconnect density can’t develop by an order of magnitude, and even past.
Power-Environment friendly Efficiency Pattern for GPUs
So, how do all these revolutionary {hardware} applied sciences contribute to the efficiency of a system?
We are able to see the pattern already in server GPUs if we take a look at the regular enchancment in a metric known as energy-efficient efficiency. EEP is a mixed measure of the vitality effectivity and velocity of a system. Over the previous 15 years, the semiconductor business has elevated energy-efficient efficiency about threefold each two years. We consider this pattern will proceed at historic charges. It is going to be pushed by improvements from many sources, together with new supplies, system and integration expertise, excessive ultraviolet (EUV) lithography, circuit design, system structure design, and the co-optimization of all these expertise components, amongst different issues.
Particularly, the EEP improve shall be enabled by the superior packaging applied sciences we’ve been discussing right here. Moreover, ideas reminiscent of system-technology co-optimization (STCO), the place the totally different purposeful components of a GPU are separated onto their very own chiplets and constructed utilizing the most effective performing and most economical applied sciences for every, will change into more and more crucial.
A Mead-Conway Second for 3D Built-in Circuits
In 1978, Carver Mead, a professor on the California Institute of Know-how, and Lynn Conway at Xerox PARC invented a computer-aided design methodology for built-in circuits. They used a set of design guidelines to explain chip scaling in order that engineers might simply design very-large-scale integration (VLSI) circuits with out a lot information of course of expertise.
That very same type of functionality is required for 3D chip design. At the moment, designers have to know chip design, system-architecture design, and {hardware} and software program optimization. Producers have to know chip expertise, 3D IC expertise, and superior packaging expertise. As we did in 1978, we once more want a typical language to explain these applied sciences in a approach that digital design instruments perceive. Such a {hardware} description language offers designers a free hand to work on a 3D IC system design, whatever the underlying expertise. It’s on the way in which: An open-source commonplace, known as 3Dblox, has already been embraced by most of at present’s expertise firms and digital design automation (EDA) firms.
The Future Past the Tunnel
Within the period of synthetic intelligence, semiconductor expertise is a key enabler for brand new AI capabilities and functions. A brand new GPU is now not restricted by the usual sizes and kind components of the previous. New semiconductor expertise is now not restricted to cutting down the next-generation transistors on a two-dimensional airplane. An built-in AI system might be composed of as many energy-efficient transistors as is sensible, an environment friendly system structure for specialised compute workloads, and an optimized relationship between software program and {hardware}.
For the previous 50 years, semiconductor-technology growth has felt like strolling inside a tunnel. The street forward was clear, as there was a well-defined path. And everybody knew what wanted to be completed: shrink the transistor.
Now, we have now reached the top of the tunnel. From right here, semiconductor expertise will get more durable to develop. But, past the tunnel, many extra potentialities lie forward. We’re now not certain by the confines of the previous.
