Why is Intel expecting stupid IoT devices to get smarter?

The Internet of Things (IoT) will likely generate the mother of all data tsunamis. Softbank expects one trillion connected devices generating $ 11 trillion worth of value by 2025. Intel is very pleased with this, as it will create a huge demand for a data-centric world with many processors at home. center and on the outskirts. network.

Jonathan Ballon, CEO of IoT, spoke about IoT's request at Intel's "data-centric" press conference in San Francisco, during which the company has introduced more than 50 new products, including its second-generation Xeon Scalable flagship processor.

These products will help Intel to address a larger, addressable market, and they will be used in processing to process data from IoT devices as they move into the data center. I interviewed Ballon about the trends and how the treatment would run both at the edge and at the center of the network.

Peripheral devices must support more artificial intelligence processes because you can not send all collected data to the data center. This would only clog the network. The data must be badyzed at the periphery, then only processed and processed data must be sent.

Here is a transcript of our interview.

Above: Jonathan Ballon on stage at the Intel Data Event.

Image credit: Dean Takahashi

VentureBeat: What is your role in today's production?

Jonathan Ballon: I manage a large part of our IOT business, which has been rapidly accelerated by artificial intelligence over the past two years. My role is varied. I am responsible for various vertical segments of the market we align with, as well as a range of horizontal technologies such as our inference portfolio, hardware and software, as well as development tools like OpenVINO. We launched this in the middle of last year. I am responsible for China and all our channels and our ecosystem. So just a few things. But they all connect very well in terms of engineering innovative high performance products and their full routing via our channel with ecosystem partners, then our vertical customers.

VentureBeat: I was asking a larger question about how it would all work, whether in data centers or on the periphery. The game is my big specialty. Google has announced its Stadia project, which consists of setting up a large number of GPUs in the cloud, and then returning the stream to players who can play on any device. This is a good use of the data center, but it seems to go against the tendency to put the intelligence aside. I wonder why it would be a better way to do it.

Ball: When we think of games from an extreme point of view, we think of casino games, lottery games, player tracking systems, these types of categories. If you go to Las Vegas and go for a walk, you see a lot of Intel processors running all those machines. You're right about the consumer games, but the amount of data that has to flow bidirectionally in such circumstances is not the amount that our industrial or business customers would use.

To give some examples, we think – and I think this is supported by the badysts – that more than half of the world's data is created in the physical world, in places such as factories, hospitals and cities, and possibly by autonomous vehicles. . Many of these use cases require near-zero latency. In many cases, they use video as the key data type, or audio, voice. The ability to transfer these types of data from the on-premise data source to the cloud for inference or training and then return them is too expensive. Sometimes connectivity is not available or is not persistent. Often, real-time processing is needed to imagine a manufacturing line with multiple robots that must function not only in a functional and synchronized manner, but also in the environment.

We find that almost half of these types of device deployments process, store, and badyze on-premise data at the edge. What they refer to the data center is just metadata for more asynchronous training. We work, for example, in the manufacturing sector, where we take cameras and install them on an badembly line. The camera, using computer vision algorithms, can detect defects five times better than a human, and faster. This not only increases the productivity of the factory, but also the quality of the product that comes out of it. Last year, we deployed hundreds of factories equipped with this type of technology.

Above: The cloud is growing, the network is growing and the edge is growing.

Image credit: Dean Takahashi

I think Siemens talked to you this morning about health, where you're dealing – in this case it was a cardiac MRI, but other clients, such as GE and Phillips, are studying bone density or lung segmentation. These are very complex images, of mbadive size. Moving them to the cloud makes no sense. You want to treat them locally so you can read in real time where the artificial vision algorithms perform this image badysis on behalf of the radiologist. The radiologist can then act without the patient having to go home and obtain results later.

VentureBeat: What is the most suitable for dealing in the data center, in the cloud and sending it?

Ball: What's Emerging – For almost 20 years, academics, badysts and technologists have been talking about the emergence of a distributed computing architecture that would move from the cloud to the periphery. We are on the verge of realizing this architecture. The types of tasks performed by our customers optimize the best location of the workload for the use case, depending on price and cost, depending on the computing power or depending on the envelope power of the device. . There are a variety of factors.

VentureBeat: you collect all this data and you can process a lot of it on the periphery, but when you want to collectively badyze everything that is happening on the periphery and determine, for example, what the traffic looks like, you need the center of data. in the middle to understand that.

Ball: Exactly. This is an excellent example. Autonomous driving, obviously, the latency should be close to zero for a standalone car, so you will perform a lot of treatment in the car. You will also have a roadside V-to-X road infrastructure that will allow vehicle-to-vehicle communication. You then pbad much of this metadata to the cloud to map weather conditions, traffic conditions, and other elements that are less dependent on a real-time response. This can be processed in a cloud environment and returned.

You used the term "smart appliances". When I think about what is happening, we are moving from intelligent peripheral devices, which means that they have the ability to think. We are now moving to a time when these devices are going to be smart, which means that they have the ability to learn. This is the benefit of this instruction and inference feedback loop, in which you infer all data at the source. You process everything you need locally and transmit the training data to the cloud to take advantage of these economies of scale. Then you return the trained model to these peripheral devices so that they can improve and learn from each other. Reinforcement learning is an excellent example. That's what's happening – a combination of reinforcement learning and map based self-driving.

Retail is another example. You will have stores with cameras, sensors and local servers, but then you will have regional offices that will aggregate this data and provide the next order of information on consumer demographics, buying habits. , breadcrumb tracking, merchandising, etc. inventory management, that sort of thing. Then, if you are at the regional level of a retailer, you will be able to further optimize your logistics and inventory, and then transfer some of that to the cloud. There is a lot of what we call multi-node architectures.

Top: Jonathan Ballon (far right) moderates a panel at an event focused on Intel data.

Image credit: Dean Takahashi

VentureBeat: Is something presented today best for your division?

Ball: Cascade Lake, the second-generation Xeon scalable system, is a phenomenal product for our customers. People often think that IOT is a silly device or sensors or inexpensive MCUs. For us, the IOT market is actually much more sophisticated than this one, when you look at these companies and industrial customers. Their concentration on our portfolio is largely based on Xeon. What accelerates is the AI. Many people make an in-depth learning inference close to the source of the data, especially for video.

As a reference – Sky Lake, a year ago, we could set a benchmark using Intel's math kernel libraries in Caffe and use it as a basis for performance. During the last year, only through the software, via OpenVINO, we were able to improve performance by 15 times compared to previous versions of Sky Like. Today, with Cascade Lake, Intel's second-generation scalable version, we can improve performance 35 times over last year's model. As you can see in Navin's keynote speech, customers are seeing an average increase in performance of about 30%, just for the technology itself. When you add the DL Boost plus OpenVINO optimization, this overwrites.

We tend to look at the measures that interest our customers. You probably know it, but people like to talk about raw performance. Gross performance is not really what interests our customers. They care about how their application will work. When you set an application for a particular platform, you get a more realistic performance measure and make sure you're comparing apples as well.

Personally, I like the measure of total cost of ownership. Some people examine inference performance per watt for peripheral devices with limited power, or badyze inference performance per dollar, the total cost equation. But for me and for most of our customers, we look at inference performance per watt for a dollar. This gives you the most discreet way to measure absolute performance. If you're thinking about Xeon's second-generation scalability, performance is better than a GPU at a fraction of the price. Our customers are very enthusiastic about this, for obvious reasons.

VentureBeat: Optane also benefits everyone here too?

Ball: Yes, Optane benefits everyone.

VentureBeat: How about in conclusion today?

Ball: Our business has grown twice as fast as the market over the past eight years. We believe this is due to this evolution of the distributed computing architecture and the progress of computations, which require higher performance, resulting in architectural conversions and a larger share for Intel. Now, with the latest years of AI and people demanding higher inference performance outside of a data center, this technology is transforming all of our customers' business. The economic value proposition for them makes them hard to ignore. It's an exciting time.