Beyond Silicon: Nurturing AI SoCs with IP

While artificial intelligence (AI) is not a new technology, it wasn’t until 2015 that a steep hike in new investments made advances in processor technology and AI algorithms possible. Beyond simply seeing it as an academic discipline, the world began to take notice of this scientifically proven technology that could exceed human capabilities. Driving this new generation of investment is the evolution of AI in mainframes to embedded applications at the edge, leading to a distinct shift in hardware requirements for memory, processing, and connectivity in AI systems-on-chip (SoCs).

In the past ten years, AI has emerged to enable safer automated transportation, design home assistants catered to individual user specifications, and create more interactive entertainment. To provide these useful functions, applications have increasingly become dependent on deep-learning neural networks. Compute-intense methodologies and all-encompassing chip designs power deep learning and machine learning to meet the demand for smart everything. The on-chip silicon technology must be capable of delivering advanced math functions, fueling unprecedented real-time applications such as facial recognition, object and voice identification, and more.

Defining AI

There are three fundamental building blocks that most AI applications follow: perception, decision-making, and response. Using these three building blocks, AI has the capacity to recognize its environment, use input from the environment to inform itself and make a decision, and then, of course, act on it. The technology can be broken up into two broad categories: “weak AI or narrow AI” and “strong AI or artificial general intelligence.” Weak AI is the ability to solve specific tasks, while strong AI includes the machine’s capability to resolve a problem when faced with a never-before-seen task. Weak AI makes up most of the current market, while strong AI is considered a forward-looking goal the industry hopes to employ within the next few years. While both categories will yield exciting innovations to the AI SoC industry, strong AI opens up a plethora of new applications.

Machine vision applications are a driving catalyst for new investment in AI in the semiconductor market. An advantage of machine vision applications that utilize neural network technology is increased accuracy. Deep learning algorithms such as convolutional neural networks (CNNs) have become the AI bread and butter within SoCs. Deep learning is primarily employed to solve complex problems, such as providing answers in a chatbot or a recommender function in your video streaming app. However, AI has wider capabilities that are now being leveraged by everyday citizens.

The evolution of process technology, microprocessors, and AI algorithms has led to the deployment of AI in embedded applications at the edge. To make AI more user-friendly for broader markets such as automotive, data centers, and the internet of things (IoT), a variety of specific tasks have been implemented, including facial detection, natural language understanding, and more. But looking ahead, edge computing — and more specifically, the on-device AI category — is driving the fastest growth and bringing the most hardware challenges in adding AI capabilities to traditional application processors.

While a large chunk of the industry enables AI accelerators in the cloud, another emerging category is mobile AI. The AI capability of mobile processors has increased from single-digit TOPS to well over 20 TOPS in the past few years. These performance-per-watt improvements show no signs of slowing down, and as the industry steadily nears the point of data collection in edge servers and plug-in accelerator cards, optimization continues to be the top design requirement for edge device accelerators. Due to the limited computing power and memory that some edge device accelerators possess, the algorithms are compressed to meet power and performance requirements, all while preserving the desired accuracy level. As a result, designers have had no choice but to increase the level of compute and memory. Not only are the algorithms compressed, but given the huge amount of data being generated, there is only capacity for the algorithms to focus on designated areas of interest.

While the appetite for AI steadily increases, there has been a noticeable uptick in non-traditional semiconductor companies investing in technology to solidify their place among the innovative ranks. Many companies are currently developing their own ASICs to support their individual AI software and business requirements. Implementing AI in SoC design does not come without many challenges.

See also: Stanford Unveils New Flexible AI Chip

The AI SoC Obstacle Course

The overarching obstacle for AI integration into SoCs is that design modifications to support deep learning architectures have a sweeping impact on AI SoC designs in both specialized and general-purpose chips. This is where IP comes into play; the choice and configuration of IP can determine the final capabilities of the AI SoC. For example, integrating custom processors can accelerate the extensive math that AI applications require.

SoCs designers face a variety of other challenges when balancing specific computing requirements with the implementation of deep learning capabilities:

• Data connectivity: CMOS image sensors for vision and deep learning AI accelerators are key examples of the real-time data connectivity needed between sensors. Once compressed and trained, an AI model will be prepared to carry out tasks through a variety of interface IP solutions.
• Security: As security breaches become more common in both personal and business virtual environments, AI offers a unique challenge in securing important data. Protecting AI systems must be a top priority for ensuring user safety and privacy as well as for business investments.
• Memory performance: Advanced AI models require high-performance memory that supports efficient architectures for different memory constraints, including bandwidth, capacity, and cache coherency.
• Specialized processing: To manage massive and changing compute requirements for machine and deep learning tasks, designers are implementing specialized processing functions. With the addition of neural network abilities, SoCs must be able to manage both heterogeneous and massively parallel computations.

Charting AI’s Future Path for SoCs

To sort through trillions of bytes of data and power tomorrow’s innovations, designers are developing chips that can meet the advanced and ever-evolving computational demand. Top-quality IP is one key to success, as it allows for optimizations to create more effective AI SoC architectures.

This SoC design process is innately arduous as decades of expertise, advanced simulation, and prototyping solutions are necessary to optimize, test, and benchmark the overall performance. The ability to “nurture” the design through necessary customizations will be the ultimate test in determining the SoC’s viability in the market.

Machine learning and deep learning are on a strong innovation path. It’s safe to anticipate that the AI market will be driven by demand for faster processing and computations, increased intelligence at the edge, and, of course, automating more functions. Specialized IP solutions such as new processing, memory, and connectivity architectures will be the catalyst for the next generation of designs that enhance human productivity.