AI will impact many areas of IoT, including jobs. Chuck Byers, CTO of the Industry IoT Consortium, joins Ryan Chacon on the IoT For All Podcast to discuss how AI is affecting IoT. They talk about the role of AI in IoT, how AI models are trained, how IoT can use generative AI, the impact AI will have on IoT-adjacent technologies such as edge computing, bias in AI models, and the future of AI and IoT together.

About Chuck Byers

Charles (Chuck) Byers is CTO of the Industry IoT Consortium. He works on the architecture and implementation of edge computing systems, common platforms, media processing systems, drone delivery infrastructure, and the Internet of Things. Previously, he was CTO of Valqari, a Principal Engineer and Platform Architect with Cisco, and a Bell Labs Fellow at Alcatel-Lucent.

Interested in connecting with Chuck? Reach out on LinkedIn!

About Industry IoT Consortium

The Industry IoT Consortium has over 100 member companies working to deliver transformative business value to industry, organizations, and society by accelerating adoption of a trustworthy Internet of Things.

Key Questions and Topics from this Episode:

(00:09) Chuck Byers and the Industry IoT Consortium

(01:28) The role of AI in IoT

(04:26) How are AI models trained?

(07:46) Generative AI and IoT

(10:55) How will AI impact IoT-adjacent technologies?

(12:41) Bias in AI models

(15:52) Future of AI and IoT together

(21:01) Learn more and follow up

Transcript:

- [Ryan] Welcome Chuck to the IoT For All Podcast. Thanks for being here this week.

- [Chuck] My pleasure. 

- [Ryan] Yeah, it's great to have you. Let's kick this off by having you give a quick introduction about yourself and the organization you're with. 

- [Chuck] I have a Master's degree in electrical engineering from Wisconsin, and I taught the computer control and instrumentation class there for a few semesters, so I'm pretty familiar with the details of sensors, actuators, edge computing, control, and so forth.

I worked at Bell Labs as a Bell Labs Fellow for about 22 years, where I worked on switching and access and wireless infrastructure. I was at Cisco for about 10 years working on media processing, analytics, IoT, and edge computing. I've been CTO of a couple of organizations, a company called Valqari that makes drone package delivery systems, heavily dependent upon AI and machine vision.

And most recently in the organization I'm representing today is the Industry IoT Consortium, which is one of the programs of the Object Management Group. We're a consortium of over a hundred member companies interested in the internet of things as a mechanism for digital transformation and trustworthy networks.

I have 135 US patents, three dozen of which more or less are somehow related to AI technologies and applications. Happy to be here. 

- [Ryan] Yeah. It's great to have you. So let's talk about AI a little bit here then. So when we're talking about the IoT industry and AI playing a role, what types of AI or what elements of AI are particularly important to the internet of things?

- [Chuck] It's really about autonomy and automation in the IoT world. So, we're really interested in taking the readings from bunches of sensors, maybe readings that would overwhelm a human. Twenty camera images or a thousand pressure sensors at once, how's a human going to look at those gauges, right? So we're going to read those in. We're going to apply various kinds of algorithms. Some of them might be heuristic based, meaning there's a rule for if the pressure goes over this, change that valve. Or they could be based on a machine learning, artificial intelligence algorithm, where we know what that particular factory or refinery or locomotive is supposed to be doing.

We know what the normal situations are, and we can detect abnormal situations by departure from that model, and then the AI can further recommend how to adjust the actuators in order to make that IoT system come back into performance line. Those would be some examples. A lot of hype recently on the so called large language model or generative AI.

ChatGPT being the prime example of that hype. That really involves trying to emulate human creativity. And there are applications for that in artificial intelligence and machine learning in IoT as well because we, for example, have a lot of Python code to write, and there've been excellent reports of good results writing Python code from plain text paragraph that write me Python code that reads these sensors and processes it thus and does an actuation. That's something that we can never hire enough programmers to do for 50 billion sensor points. AI might be able to write that code for us. That's one example. Another example really is the user interface. If I'm driving in my self driving car and the, let's say the ride is a little rough. I might say to it ride is a little rough. Can you as AI do something about that? And then the AI will look at suspension parameters and try to find a better road or whatever it's got to do in order to improve that situation. The human didn't know anything about the physical plant involved with that. They got no idea what the pressure of the shock absorbers ought to be, but the AI does.

And the AI can translate the human language into a machine understandable context, and it can therefore apply that to its learning models and know what parameters to adjust in the device. That's a really important example. 

- [Ryan] No, absolutely. That's fantastic. And when it comes to the models or the data itself, I guess two things.

Where is the data coming from and how are the models being trained? Because I think those two things are interesting for our audience just to understand. Obviously with IoT, we're talking about being able to collect data, different data than we maybe had before using sensors. So once we have that data, how are those models being improved upon, being trained and so forth?

Is there other data that maybe we're not thinking about that is playing a role here? 

- [Chuck] As much data as we can get is the short answer from as many sources as we can get over as wide a timescale as we can get. So there are historians right now who really just look at sensors and record what's going on. The black box of a factory.

What it's basically doing is recording everything, and if something goes bad, there's a quality problem or a safety problem or whatever, those historians have months, years, perhaps decades in the case of something like an oil refinery, of data about the performance and readings from all of those thousands of sensors that are monitoring that thing.

And that is something that we can use. We can designate for the entirety of 2021, that refinery worked perfectly, but in January of 2022, it had a weird hiccup, and what we can do is look back on the historian and learn from what caused that hiccup, and then try to detect that as a trend that we can try to mitigate before it happens a second time.

That would be an interesting thing to do. And that data comes from historians. Another source of data might be from the the physics models involved with it. So if I'm trying to model, for example, the anti-lock brakes of a locomotive, I know how much the mass of the train is. I know what the coefficient of friction under the steel wheels is.

I know how much power I can apply at braking and therefore I can probably use that information as training data in the artificial intelligence engines that are running that anti-lock brakes in future locomotives. The ultimate physics simulation is sometimes what we call a digital twin, which is where we have a full complex system. It could be something like a city. It could be something like an aircraft carrier, something as complex as that. We try to simulate all the different electrical, optical, physical characteristics of that thing and use that physics to predict its behavior.

And we can potentially predict its behavior much faster than real time. So if we want to know what's going to be happening on an aircraft carrier a second from now, I might be able to run a thousand simulations between now and a second from now in order to look at all kinds of different scenarios and determine the state of the device.

That is a way that we can train AI. If we can run all these different scenarios and digital twins. What happens if there's a low voltage event? What happens if the wind is blowing too fast, whatever it is, we can apply all those scenarios to the digital twin, use the real physics to determine how that system would likely react, and then use that as training information. We, for example, probably wouldn't want to simulate an oil refinery if one of the blow down drums had an explosion because that's a million dollar repair, if it's, if we did it really, but what we can do is we can simulate that, and we can use that as a way to train the model of what happens if that explosion is imminent. That's useful.

- [Ryan] And you mentioned this earlier a little bit but talking about generative AI and how an AI, sorry, an IoT system can take the output from generative AI and basically create value for business. Can you elaborate on that a little bit more and just talk about how that potentially works or will work?

- [Chuck] Generative AI, especially the large language model versions, are trained with a huge corpus of data. In the case of ChatGPT and the GPT 3.5 model, the most famous one that's out there today, although GPT-4.0 is being used to great effect by Microsoft, that one was trained in 2021 or early 22 at the cost of something approaching $50 million dollars.

And it was trained based on pretty much the entire written output of the human race as it's available, at least on the internet. And that let's ChatGPT take your seed phrase and kind of figure out what word comes next. That's what it does. That's all it does is it knows the words that it said so far, and then it figures out what would come next if the entire training corpus was put to work on what it knows about the stimulus that you gave it. Examples of how that might be applied to IoT is we, one other thing about Chat is that because it's expensive to train these models, they take three times, 10 to the 23rd, explanation point, if you know what that means, the of what's called flops, floating point operations, to train the GPT-3.5 model. That, if you had 82 racks of the best GPUs in the world, they could calculate that model about once, it would take about a week to calculate that model. So if you dedicated that, those 82 racks, a hundred million dollars worth of GPUs, to training your large language model, that means that about once a week, you can refresh that model with what's fresh on the internet.

And ChatGPT 3.5, you can do an interesting experiment. Ask it about the dangers of Chinese balloons. And it will send you back information about choking hazards and heavy metal contamination in the latex and dangers to wildlife. But it doesn't know about surveillance balloons flying over the Great Lakes because it was trained well before those news events were on everybody's mind for months and months.

So there's, think about what that means to training AI. What happens if the data that I'm using for that conversational model doesn't know the current events that happened in the last, say, year. And how does that screw up the AI's usefulness or what problems and dangers does it put into the system?

It may not know, for example, that a interstate highway collapsed in Philadelphia, and it might try to route you right through there, right? Self driving car doesn't know that collapsed because it was trained well before that. Those kinds of problems, that's a kind of a contrived example, but those kinds of problems are going to be predominant in large language models that are too expensive to train continuously. 

- [Ryan] How do you see the generative AI working with other technologies that are oftentimes being utilized in IoT solutions like machine vision, AR, VR, edge computing? I know we talked about edge AI in the past and things like that, but how is that all coming together?

- [Chuck] The models are generally trained in the cloud where you have lots of computing available, and you don't care if it takes a few milliseconds or a few hours longer than you expected. But when you run the inference, you take that model, and you apply the sensor data or apply the human inputs to it, you want that to run fairly quickly.

So you may decide to use that on more distributed computing resources than the cloud. You might drive it into content delivery networks like the caching engines that supply Netflix. There's edge computing there. You might put it in what's called MEC, multi axis edge computing. That's an ETSI standard for computers that are typically located at the base of 5G cell towers.

Those are nicely distributed around the landscape. There's, you can even run edge computing and edge gateways or mobile edge devices or even human portable edge devices that could actually run some of those more simple inference phases. So what you want to do is you want to put the inference engine, the thing that's applying the model and making the decisions, you want to put it at the right depth of the network from the cloud all the way down to some kind of endpoint device so that you have the right amount of computation capabilities there, the right amount of power and cooling and all that stuff, but you want to get as deep as you possibly can into that network so that you eliminate the latency in the network bandwidth and the potential for hacking and privacy violations and all that. The deeper in the network the AI is inferring, the better off you generally are under those circumstances. 

- [Ryan] What have you seen as far as how the different biases and things that are happening with the models, obviously, this is a big discussion and there's plenty of ways to discuss or talk about it. But just from your perspective, how are these biases playing a role? How are they being thought about? How are they being adjusted, fixed, minimized with how it's impacting potentially it working without an IoT solution.

- [Chuck] Yeah, bias in training models and training data into those models is an enormous problem. And in fact, it's entirely possible that a significant portion of those people who are worried about losing their jobs due to AI automation and autonomous systems are likely going to be able to be employed in trying to unbias the training data for some of these AI models. There's lots of well understood machine vision bias positions.

For example, people with darker skin are have much less fidelity of their facial recognition than those with lighter skin because the algorithms were trained and developed apparently by folks with lighter skin. That's a bias that's got, that kind of thing has got to get removed, but there are even more insidious versions of those biases that could exist in IoT systems.

There might be a bias towards the sunny day training data because 99 percent of the time the factory is working properly and plunking out the right equipment and the right products at high quality. But for the 1 percent that it's not, that 1 percent may not be enough represented in the training data to allow the AI to have a broad unbiased view of all the possible operation modes of that factory, good and bad. That's a thing that's going to require a lot of thought. The digital twin approach that I mentioned before lets us investigate those failing and abnormal scenarios without actually producing tons of bad product. Those are some of the mechanisms that we can use to do unbias.

There will be humans involved in cleaning data. There'll be humans involved in saying this picture has no trespassers in it, where this picture has a coyote in it, and this picture has three human trespassers that probably are a real problem. But it's really hard for the AI to take those images and figure out what's in them without a human interpreting those contexts. So there'll be a lot of crowdsourcing kind of work being done in terms of training those images. In fact, the CAPTCHAs that you sometimes use as if you're trying to go to a website, and it wants to prove that you're a human, show me all the things with traffic signals. You may have gotten that one. That's actually going into AI training data. You as a human identifying those are using that data where all those traffic signals are to train the AIs that are running self driving cars. Isn't that interesting? So you're getting double duty out of those, you're getting double duty out of that, proving that you're human, and also throwing a lot of different images into a training model that the distributed crowd is validating. 

- [Ryan] Let me ask you this before we wrap up here, one of the last things I wanted to touch on is as we move forward with AI getting more integrated closely into the IoT space, what does the future look like with AI and IoT coming more closely together? 

- [Chuck] One thought is that government regulation, especially in the United States, European Union, and China, will have significant impacts on what AI is allowed to do and what kind of training data is acceptable for that AI. That government regulation might retard the development of some of these things by a year or so.

But I think that might not be all bad. Waiting until we have some, what we sometimes called guardrails in the business, some rules for what's acceptable and what's not acceptable in terms of technologies and applications of those technologies, that will be, that'll be something that needs to get done.

So that's one thing that I think might be in the future, and one of the big unknowns in the future is how much is government regulation going to impact the deployment wide-scale AI? Other things, I think that large language models are very important to the way that humans are going to be doing work. And any human who sits at a desk and does a job that you could have described on a post-it note, they're gone. They're replaced by AI, right? So there's plenty of humans, and lawyers think about that, they're probably not doing a job that can be described in a post-it note. But if you can be, you might want to start retraining yourself to be more in AI data wrangling or testing validation of these systems because you're going to get replaced. These are people who do data entry, clerks, anybody who types something in off of a piece of paper, forget it, they're gone. A lot of that stuff, a lot of those jobs do tend to exist in IoT networks. The swivel chair people who sit there and manage those networks, they watch for the, wait for the red signal to come up on the dashboard, and then they dispatch a human to go, and you'll change that battery or fix that fiber cable, whatever the problem might be.

Those folks, I think, could probably be replaced by various kinds of expert systems and conversational AI systems. And as a result, that might be a deal. I don't know where customer support's going to be. Right now, when I get an automated customer support system, I push zero to see if a human will come on, and then I hang up.

- [Ryan] We're starting this AI podcast, and we actually, one of our first guests, we were talking about how these, we started off talking about enterprise assistance and then turned into chatbot conversations and just being able to create that experience to be something that people feel way more comfortable and trusting to engage with and don't do exactly that, push to get to a human because the cost and the expenses that go into training people and maintaining a sales staff is pretty high. So how can these new tools, these new models help customer support become more efficient and do the job better than needing individuals and humans every step of the way. So, it's very fascinating to see how that's going to evolve because everyone listening to this interacts with that kind of experience on a regular basis 

- [Chuck] Five years from now, people like me sitting here trying to make my technology device work on hold with the help desk, they're going to prefer AI because AI is instantly available. AI is always polite. They have an accent that is perhaps the one that you chose with your slider. If you want somebody who talks with a British accent, you can do that if that's easier for you. And they're going to be more knowledgeable than 90 percent of the folk.

So what you're going to have is the AI doing the triage and for the 10 percent that the AI doesn't have high confidence that it knows the answer to, it will abridge that information, it will send it to a human, and it will attach your conversation to that human. You don't have to go through anything that you told the AI because that's all on that human screen already. That kind of thing is inevitable, and I think what that lets us do is get those 50 billion IoT devices that the planet is supposed to have by the end of this decade, get them rolled out faster without having to rely on a bunch of humans in swivel chairs typing IP addresses and a bunch of more humans in swivel chairs with headphones on trying to troubleshoot the people whose garage door opener won't connect to the internet. That stuff is going to be AI driven, and it's an enabling technology, but it does have a social cost because the folks that used to have those moderate to good jobs sitting in those swivel chairs are going to be systematically replaced.

- [Ryan] Really appreciate your time, Chuck. And thank you so much for being here for our audience, who is looking to learn more about the organization and follow up on this conversation, anything like that. What's the best way to do that? 

- [Chuck] Connect to iiconsortium.org. That's the Industry IoT Consortium dot org. And there's a resources page that has a whole bunch of fundamental documents that you can download for free.

One of them is about IoT based AI engines, and I think you'll find that very useful. There's other ones about cybersecurity and trustworthiness and other things that I think are useful. There's also an Apply for Membership page, and we have excellent deals for startups, and not too bad a deal for small, medium, and large businesses, depending upon your revenue, we'll charge you a modest annual fee, but you get a lot out of it.

You get the opportunity to hear what's being talked about in terms of future reference architectures, future best practices, maturity models, all that stuff. And you also have the opportunity to influence our organization as we invent the future. So if you have a particular technology that you love, a particular way of doing things, a protocol that you'd like to see a deep implementation of, we are the place that's making those decisions and trying to deploy it to the entire IoT industry. 

- [Ryan] Well, Chuck, thank you again so much for your time, and I'm very excited to get this out to our audience.

- [Chuck] Thank you so much. Good luck to the audience and your IoT journeys. Take care.