Breaking the Memory Wall In AI
In the long unfolding arc of technology innovation, artificial intelligence (AI) looms as immense. In its quest to mimic human behavior, the technology touches energy, agriculture, manufacturing, logistics, healthcare, construction, transportation and nearly every other imaginable industry – a defining role that promises to fast track the fourth Industrial Revolution. And if the industry oracles have it right, AI growth will be nothing shy of explosive.
Internet of Things (IoT) and 4G/5G, both key AI enablers, are expected to account for more than 75 percent of device connections by 2025. In order to process the colossal amount of data central to the promise of AI, from over 30 billion connected devices worldwide, the industry needs to now break through the limits of a key technology: memory.
Memory As A Critical AI Bottleneck
The challenge for memory begins with performance. Historically, every decade, the gains in computer performance have outpaced improvements in memory speed by 100 times, and over the past 20 years that gap has grown. The result of this trend is that memory has bottlenecked computer and AI performance.
The industry has responded by developing several methods to implement memory systems on AI chips, with each being specialized for different performance needs and each requiring different trade-offs. Among the front runners:
- On-chip memory delivers the highest bandwidth and power efficiency but is limited in capacity.
- HBM (High Bandwidth Memory) offers both very high memory bandwidth and density.
- GDDR offers good tradeoffs between bandwidth, power efficiency, cost and reliability.
Since 2012, AI training capability has grown 300,000 times. This growth outpaced Moore’s law by 25,000 times, doubling every 3.5 months, a blistering pace compared to the 18-month doubling cycle of Moore’s law. The staggering recent improvements have been attributed to parallel computing capacity and new application-specific silicon like Google’s Tensor Processing Unit (TPU).
Specialized silicon architectures and parallel engines are one key to sustaining future gains in compute performance and combating the slowing of Moore’s Law and the end of power scaling. By rethinking the way processors are designed for certain markets, chip makers can develop dedicated hardware capable of operating with between 100 and 1,000 times greater energy efficiency than general purpose processors to overcome another big limiter to scaling compute performance: processing power.
For its part, the memory industry can improve performance by signaling at higher data rates and using stacked architectures like HBM for greater energy efficiency and performance, and by bringing compute closer to the data.
Memory scaling for AI
A key challenge is scaling memory for artificial intelligence (AI) applications. Growing consumer demand for better voice, gesture, facial recognition experiences, and more immersive virtual reality and augmented reality interactions rise in importance. Offering these experiences requires more computing capability from high-performance computing (HPC) to make big data analytics possible, to run the analytics themselves, as well as machine learning processes that generate meaningful insights using AI and machine intelligence.
Emerging machine learning applications include classification and security, medicine, advanced driver assistance, human-aided design, real-time analytics, and industrial automation. With 75 billion IoT-connected devices generating data expected by 2025, there will be no shortage of data to analyze. For example, the wings alone on a new Airbus A380-1000 feature over 10,000 sensors. Mountains of this data are stored in massive data centers on magnetic hard drives, then transferred to DRAM before moving to SRAM within the CPU for the hand off to the compute hardware for analysis.
With stored data growing at an exponential clip, the question is how to make sure all other memory systems are able to handle the flood of data. AMD’s answer is a chiplet architecture featuring eight smaller chips around the edge that drive the compute and a large chip in center that doubles the IO interface and memory capability to double chip bandwidth.
AMD has also moved from a legacy GDDR5 memory chip configuration to HBM (high-bandwidth memory) to bring memory bandwidth closer to the GPU to process AI applications more efficiently. The HBM provides much higher bandwidth while reducing power consumption. Compared to DRAM, AMD’s HBM delivers a much faster data rate and far greater memory density than DRAM to put memory closer to the GPU so AI applications are processed more efficiently.
Over the next decade, look for more performance improvements from multi-chip architectures, innovations in memory technology and integration, aggressive 3D stacking and streamlined system-level interconnects. The industry will also continue to drive performance gains in devices, compute density and power through technology scaling.
Although the total volume of data is expected to grow exponentially from proliferating technologies such as 5G, IoT, and autonomous driving applications, magnetic tapes will likely still remain the mainstream data storage medium over the next 10 years. This recent assertion is supported by the idea that magnetic tape storage capacity is expected to grow 30% per year, compared to only 10% for hard disc storage, according to IBM.
At a recent IBM forum in Taipei, the company’s Greater China CTO said that magnetic tapes, after around 50 to 60 years of development, have been a preferred medium to store data mainly because they are cheaper than hard discs and also boast a longer storage life, being able to maintain the security of its data for between 30 and 50 years.
Many data center operators choose magnetic tapes to build their backup data files, but in order to meet different usage needs for cold data and hot data, new storage architectures will need to become available in the future, including a mixed use of flash, tape, and FLAPE (flash + tape).
Data storage industry sources claim that between 70 and 80% of all data stored at data centers are stored with magnetic tapes, although hard discs are increasingly adopted whenever costs permit. Because of this, IBM has developed the Linear Tape Open (LTO) initiative in cooperation with HP and Seagate, and is now the bellwether of magnetic tape technology, which has advanced to LTO-8.
IBM and Sony have also recently teamed up to break the areal density record for the magnetic tape medium, successfully cramming 201 billion bits of uncompressed data into each square inch of tape.
Industry news sources cited Seagate’s estimates that global data volumes will grow from 33ZB in 2017 to over 175ZB by 2025, predicting a growth factor of over 500% within the next eight years.
Intel Showcases New Chip Packaging Powers
Packaging has perhaps never been a hotter subject in popular discourse. Since Moore’s Law no longer seems to provide the impact it once did, another path to better computing is by connecting chips together more tightly within the same package.
At the recent Semicon West event, Intel showcased three new research efforts related to packaging. The first combines two existing technologies to more tightly integrate chiplets, which are smaller chips linked together in a package to form the kind of system that would normally be produced as a single chip. The second effort provides power delivery efficiencies by adding dies at the top of a 3D stack of chips. The final research effort is an improvement on Intel’s chiplet-to-chiplet interface, known as the Advanced Interface Bus (AIB).
The first effort, named “Co-EMIB,” is essentially a way of combining two existing Intel packaging technologies: EMIB (for embedded multidie interconnect bridge) and Foveros. The former bridges two chiplets over a short distance by using a small piece of silicon embedded in a package’s organic substrate. The interconnect lines on silicon can be made narrower than on the organic substrate and can be packed together more tightly to form a high-bandwidth chip-to-chip connection. This method has been used to produce systems like Intel’s Stratix 10 FPGA, which is actually an FPGA chiplet linked to two high-bandwidth DRAM and four high-speed transceiver chiplets in the same package.
Foveros is Intel’s 3D chip-stacking technology. This technology allows die-to-die connections of just 50 micrometers distance, then leading to high-bandwidth vertical connections. Through-silicon vias (or TSVs), conductors that pass vertically through the silicon of the bottom die, then connect the stack to the package substrate.
Combining the two into Co-EMIB lets two or more Foveros stacks communicate through high-density EMIB bridges to build more complex systems. Since connections are only micrometers apart, using an organic substrate that is hard to make perfectly planar, and a fairly large area to pattern, it became quite difficult.
“The scale of it becomes more and more critically [dependent] on how you can hold all your dimensional tolerancing through the assembly process,” says Johanna Swan, a fellow at Intel’s components research and technology development group. “The process tricks become more important in order to manage the size of structures. We’re able to show there’s a path for maintaining that dimensional stability over a larger area.”
The second research effort, Intel’s Omnidirectional Interconnect (ODI), essentially allows for EMIB-like vertical connections. These are larger than typical through-silicon vias, about 70 micrometers across versus an ordinary TSV’s 10 micrometers. Large diameter makes them especially well-suited to deliver power to the top die within a 3D stack. “As you scale that area, you get cleaner, more efficient power delivery,” Swan added.
MDIO, the product of the third effort, should be available in 2020 according to Intel’s Semicon West presentation. It offers 200 gigabytes per second per millimeter of chip edge versus AIB’s 63 GB/s-mm, and it uses 0.50 picojoules per bit versus AIB’s 0.85. Intel compared MDIO to TSMC’s LIPINCON technology, which is also expected in 2020 and delivers 67 GB/s-mm at about the same picojoules per bit.
Intel R&D claims that it will continue to try to increase the number of bumps—the solder ball on/off ramps from a chip— which are available in a given area. Ultimately, getting rid of solder is their primary goal with these research efforts. The intermetallic interface between the solder and the copper interconnects limits current, so chip manufacturers are now exploring a technology known as “hybrid bonding,” which uses a dielectric material and heat to connect one chip’s copper pads to another without using solder.
Vinyls are a simple chemical group, yet they are industrially indispensable. They have some of the broadest practical applications in solution-oriented chemical engineering. In the form of vinyl resin polymers or copolymers, they have the potential to be utilized for an almost limitless set of purposes. For this reason, products derived from these vinyl resins are ubiquitous.
Vinyl resin polymers are the most commercially-visible vinyl resin solutions used. These include polyvinyl acetate (PVAc) and polyvinyl chloride (PVC). Polymerized in water, polyvinyl acetate (PVAc), is an emulsion with widely useful physical qualities which allow for all sorts of coating and adhesive solutions. The most obvious of these is PVA glue, also known as Elmer’s glue, but its full range of applications varies considerably.
More varied still in application is polyvinyl chloride (PVC). Produced globally in immense quantities, PVC is the most popular vinyl resin polymer and the world’s second biggest selling plastic today. A considerable contributing factor in this is almost certainly a financial one of cost-effectiveness. Synthesized from ethylene and chlorine, PVC is one of the most easy plastics to produce. It’s also less susceptible to changing prices of oil, since its chemical composition isn’t exclusively based on hydrocarbons, but includes chlorine as well.
Logistically, polyvinyl chloride is convenient for the production of commodities with a small cost to large benefit ratio. Besides this, PVC’s preeminent properties of strength, durability, corrosion-resistance and lightness have guaranteed its salience in chemical engineering as applied to industrial scale production. Its low cost make it an economical choice for many manufacturers. Accordingly, PVC resin has become, both economically and literally, the very substance of markets like glazing, cladding, facing, plumbing and drainage, as well as the production of consumer-facing food and drink containers, to say nothing of many thousands of other applications.
The largest producer of PVC in the United States is Shintech which, as a subsidiary of Shin-Etsu, is a sister company of MicroSi. Globally, Shin-Etsu itself is the largest single producer, providing roughly 30% of the world’s supply of PVC. In 2018, Shintech announced another expansion of its PVC production, with plans for an additional production facility in the near future. Other major producers of PVC are making similar plans.
The outlook is similar for vinyl resin polymers in general. The market for vinyl resin products grows reliably year on year. This will only continue as producers across the globe expand their production capabilities and entire industries grow up around the demand for products derived from vinyl resins. In the United States, the major producers of these vinyl resins are planning for more of the same — a future where their vinyl polymer products have more applications than ever before.
Today, Intel recognized 35 suppliers for their exceptional commitment to quality in 2018. The suppliers have collaborated with Intel to implement innovative process improvements and serve with the highest level of integrity, while providing superior products and services. Intel has three levels of supplier recognition: the Supplier Continuous Quality Improvement (SCQI) award, the Preferred Quality Supplier (PQS) award and the Supplier Achievement Award (SAA).
The PQS award recognizes suppliers who relentlessly pursue excellence and conduct business with resolute professionalism. To qualify for PQS status, suppliers must exceed high expectations and uncompromising performance goals while scoring at least 80 percent on an integrated report card that assesses performance throughout the year. Suppliers must also achieve 80 percent or greater on a challenging continuous improvement plan and demonstrate solid quality and business systems.
“Companies receiving this year’s Intel Preferred Quality Suppliers award have demonstrated excellence in customer satisfaction and have been critical in enabling our technology and manufacturing roadmaps,” said Mike Mayberry, senior vice president and general manager of Technology Development at Intel. “These suppliers play a crucial role in our relentless pursuit of Moore’s Law as we collaborate to deliver the technology foundation for the future of computing.”
- Shin Etsu Chemical Co., Ltd.*: silicon wafers, mask blanks, thermal insulating materials, underfill and photoresist
The distribution partnership covers the territory of New England, New York, New Jersey, Pennsylvania, Virginia and parts of Ohio.
This includes the Solbin vinyl chloride-vinyl acetate copolymers for solvent-based inks and coatings, Vinyblan water-based vinyl chloride, vinyl acetate and acrylic acid ester copolymers, and Chaline silicone acrylic hybrid resins.
“Shin-Etsu MicroSi has very unique water-based technology for the inkjet and specialty coatings markets,” states Bob Whiteley, EVP, JNS-SmithChem. “Their proprietary silicone acrylic hybrid polymers have applications for rubber and plastics, and their solvent-based vinyl copolymer resins continue to be used in specialty coatings ink applications for film and foil where performance is needed.”
“JNS-Smithchem has a strong presence in the coatings market, and their people understand specialty materials and how to bring them to market,” added Ed Nichols, marketing manager for Shin-Etsu MicroSi. “We are excited to be working with them especially when it comes to introducing our newer resin technologies.”
The distribution partnership covers the territory of New England, New York, New Jersey, Pennsylvania, Virginia and parts of Ohio.
Thermal interface materials are integral to any consideration of thermal management solutions. They not only improve reliability and longevity of components without any sacrifice in performance, but also reduce thermal management costs overall by making components easier to cool. In smaller markets including, on the one hand, enterprise consumers seeking to improve the efficiency and longevity of high-performance electronics for industrial purposes and, on the other, private enthusiasts seeking solutions for overclocking personal computers, there’s clearly a specific interest in thermal interface materials.
Even so, demand for effective cooling solutions like thermal interface materials isn’t a demand limited to these sectors or to markets for niche technologies. It’s spreading and will continue to spread from these pre-existing sectors to other consumer groups and to emerging markets throughout the world, particularly in Asia and the Indian subcontinent, as we approach the end of the decade.
This is because thermal management is becoming a more general concern. It becomes an increasing challenge for consumer electronics as a simple result of the increasing speed, energy demands and miniaturization of processors, integrated circuitry and other electronic components. Smaller, faster components run hotter and have less surface area to dissipate heat, therefore demanding more in the way of thermal management. Anywhere a certain standard of performance is expected of increasingly smaller and smaller devices, high quality thermal management becomes inevitable. Today, that’s virtually everywhere. This is why the market for thermal interface materials is one which is growing without any sign of slowing down.
The most obvious thermal management solution for cooling components like processors or heatsinks is the careful application of thermal interface materials like thermal grease or paste. For thermal management solutions, most cooling needs can be met by thermal grease products like those in Shin-Etsu MicroSi’s silicone-based product lines (e.g. G765). Thermal grease is specifically beneficial for applications like cooling surfaces of components which are uneven or interrupted by minute gaps, as well as being generally convenient for its heat dissipation properties.
However, this isn’t the only type of thermal interface material available or appropriate for specific thermal management needs. Besides thermal paste, thermal interface materials include thermally conductive adhesive or double-sided thermal tape, an important subset of thermal management solutions. Applied to clean, even surfaces, thermal tape like Shin-Etsu MicroSi’s TC-SAS-T offers benefits comparable to those of thermal grease.
Moreover, thermal tape has specific advantages of its own. These include physical stability at high temperature, even distribution of cooling material and therefore even dissipation of heat, firm adhesion without mechanical aids like screws, and functioning as an electrical insulator while still effectively conducting and dissipating heat. This is crucial for the safe yet effective cooling of electronic components. In many cases, its use for these purposes is more convenient than thermal paste for the same purposes.
Thermal tape can both firmly affix and evenly cool critical components like heat sinks. It’s susceptible to neither irregular elongation nor loss of adhesion. Double-sided thermal tape can more cleanly, neatly and easily affix surfaces to one another. Similarly, it can be removed about as cleanly, neatly and easily and re-used or re-worked without significant loss of adhesion, making thermal tape a viable and effective heat management option.
We can expect interesting things in the semiconductor industry this year. The forecast for the year ahead is hard-going. The source of this is an inventory oversaturation on the supply side of semiconductor products, specifically the supply of processors currently held by chip manufacturers. Market growth has slowed and may continue to slow in the next few months. Most predictions are for a market correction and sustained decline in profits for several quarters.
Industry news is challenging, yet also promising
After intense growth in 2017 and 2018, some of what’s being reported for 2019 is challenging, but there is good news as well. Adjustment of inventory will, at any rate, be a challenging but rewarding process in both the short and long term. For example, oversupply makes manufacturers more competitive. When the market is saturated and prices drop as we’re currently seeing in memory modules for example, consumers are riddled with choice. They’ll be even more discerning than usual and will go with the best product at the best price.
This is, of course, good news for industry leaders in semiconductor materials including Shin-Etsu. Suppliers providing semiconductor materials at the beginning of the supply-chain – like silicon, lithography materials, epoxy resin and thermal interface materials – will continue to benefit from judicious consumer preferences at the market-end of the chain.
Ways to overcome include 5G, IoT, Ai and beyond
This doesn’t solve the supply problem as a whole, however. So, how can the semiconductor industry move back out of its bear-pit and recover its dynamism? The good news: a number of developments will quickly and fundamentally change the landscape for the better. Given the current state of the market, these couldn’t be emerging at a better time.
The first is 5G technology. Smartphones and related technology have influenced the fortunes of the industry in ways distinct from pre-smartphone years. These mobile and mobile-friendly technologies presented new challenges and opportunities for the semiconductor industry. 5G is, in a sense, an extension of this. American mobile carriers will be supporting 5G in major cities throughout 2019.
Yet 5G does not render 4G (or 4G LTE) completely obsolete. For the foreseeable future, both technologies will co-exist in smartphones and other mobile devices so the leftover supply of semiconductors for the latter technology won’t have to be scrapped on Day Zero as a result of 5G implementation. Rather, the demand for 5G devices at the consumer end-market may even boost sales of 4G chips upstream for use in those same devices. Both will still, by and large, require the same semiconductor materials. This benefits the whole supply-chain.
Internet of Things as an evolving market
Another development is the so-called Internet of Things (IoT). Intel in particular has leaned into this market and is already benefiting from it. This pivot may well see Intel become the big winner over the next several quarters. The industry will be able to seize new opportunities in this still-evolving market, if manufacturers can provide semiconductor solutions to suit the IoT market, one based in the variation and interactivity of home devices, not to mention interaction with cloud storage and outside networks.
It’s even possible that the further development of this technology will soon dovetail with that of 5G, generating a new market of roaming, always-on devices interacting in completely new ways. In any case, meeting the demands of this market will require a diversified approach to semiconductors and semiconductor materials by industry leaders.
There’s another promising development for the semiconductor industry, albeit more distant: artificial intelligence (AI). AI is still a marginal market and is somewhat far from practical consumer applications, but it’s receiving increasing interest and investment. Processors being developed for AI are intended for completely different operations than those we currently use in most devices, so manufacturer inventory of chips and other technologies being developed for this purpose is unaffected by the oversupply of normal processors. AI is certainly some way off from being a fully-fledged technology relevant to consumer applications, but the demand for this technology is already making itself felt. The possibilities when practically integrated, are endless.
Industry shift is necessary to provide opportunities for future growth
The Internet of Things and 5G will likely reach maturity much sooner. Both of these will push the industry into new challenges, but also yield new solutions and new rewards. These will likely hasten the recovery of the semiconductor market. More importantly, they’ll prompt a fundamental shift in the entire industry by providing additional opportunities for limitless potential.
If you are looking to apply thermal interface material, also known as thermal paste, there are a few things you’ll want to keep in mind to ensure that you are maximizing product efficiency and ensuring proper heat dissipation.
Do remember to clean first.
Start with a clean working area. It is best to lay both thermal surfaces face up. Use a dry cloth or paper to wipe away any loose debris or surface material. Cotton balls or cotton swabs are also recommended.
You may wipe both surfaces with alcohol; do be sure to let it dry completely. Alcohol can also be an effective way to remove old thermal paste from your components.
It’s important to note that old thermal paste shouldn’t be re-used, so if you do separate the heatsink, make sure and wipe up all old materials, clean thoroughly, and apply new thermal paste before reattaching.
Do make sure you have a completely clear surface.
Never underestimate the importance of cleaning both surfaces; dirt, oil, debris or residue can all interfere with cooling performance.
Remove all stickers and labels. Without a clear surface, there’s no way to maintain proper heat dissipation. To avoid having to repeat this process, it’s important to clear everything away the first time.
Don’t put on too much thermal paste
If you only take away one of these tips, we recommend that it be this one. When you apply too much thermal paste, depending on the quality of the paste you’ve selected, it can actually have the opposite effect of what you intend.
When you apply too much thermal paste, it can act like an insulator. At best case, this may make the paste ineffective, and at worst case, you may damage components via overheating.
Make sure that you apply a thin layer of paste to your component. Remember, the entire point of the thermal paste is to fill in the tiny gaps between the two components. When applied properly, the paste should facilitate a better transfer of thermal energy.
Don’t choose a mediocre thermal paste
This goes without saying. There are many downsides to choosing a subpar thermal paste, including expensive equipment repairs and/or replacements. The benefits of selecting a quality thermal paste, such as ShinEtsu’s X23-7783D are numerous.
Benefits include reduced and/or zero curing time, faster processor speeds, and prolonged longevity of components.
Automotive semiconductors represent a $24 billion industry, with a growth rate of 8% annually. Electrified drivetrains will soon fuel the largest semiconductor industry expansion to date, using power metal oxide semiconductor field-effect transistors and the like to carry power from the motor while supporting crystal-level tuning. Innovative fabrication doesn’t always produce positive effects, though. Computer chip manufacturing requires workers to handle hundreds of different chemicals, many of which are reproductive toxins, cancer-causing chemicals, and mutagens.
25 years ago, a chip maker’s production line began to suffer dire effects from exposure. A 1986 study discovered double the miscarriage rate along a semiconductor assembly line. The resulting taskforce found that this wasn’t an isolated problem. It had changed the lives of thousands.
New Materials, New Risks
The average vehicle has $350 worth of semiconductor content, and 80% of it now requires new materials like scandium. A sturdy occupational health policy is thus critical. Shin Etsu has one of the most robust safety policies in the industry. To manage specialist hazards, advanced risk assessment strategies including HAZOP and FMEA are in use. HAZOP examines design intent along two flow diagrams to find out if existing protection is enough. FMEA is engineer-designed to spot failures in risk mitigation.
Shin Etsu’s safety goals and targets are constantly reviewed, and managers are properly trained to uphold safety standards. Staff are educated, and an open door policy is maintained for those who want to discuss safety concerns. When health issues arise, the company investigates them thoroughly, creating suitable strategies to mitigate risks.
In an industry as complex as semiconductor manufacturing, static safety practices aren’t enough. They need to be constantly reassessed and improved upon—a process that Shin Etsu actively maintains. An industry that’s rising to thrilling new heights cannot afford losses, especially those relating to its workers.