Ahead in the Cloud: Building an AI cloud platform for all with Nvidia's Shanker Trivedi

Chad Watt: Welcome to Ahead in the Cloud, where business leaders share what they've learned on their cloud journey. I'm Chad Watt, Infosys Knowledge Institute researcher and writer. I'm here today with Shanker Trivedi, Senior Vice President of Enterprise Business with NVIDIA. Shanker's job is to take NVIDIA's accelerated computer platform and know-how to big businesses of all technology proficiencies. Welcome, Shanker.

Shanker Trivedi: Hi, Chad. Nice to talk to you.

Chad Watt: NVIDIA makes this bold statement that AI is the greatest technology force of our time. Briefly tell me why you guys make that assertion.

Shanker Trivedi: Yeah, I mean, if you think about what AI is, it's basically the ability of a machine, a computing stack, if you will, to look at and understand complex data, large amounts of complex data and make decisions which are, uh, quite difficult decisions. And so, when you think about that, you know, in terms of, the ability to do things that you couldn't do before, it is truly transformational. Right?

It basically means that you can embed it in every single business process in every single consumer service and, the machine, will be, does things better than a human being can. The machine can look at images, videos, it can understand, you know, text, it can speak to you, it can look at medical images and all of these things can be done better than a human being. Or at least as good as, you know, um, the most intelligent human being. So that's really transformational, and it's gonna affect all of us.

Chad Watt: So, now, we've mentioned the GPU. This all starts with the GPU and that's NVIDIA's heritage, as we discussed. It's a hardware component. Do you consider NVIDIA to be a hardware company or more of a platform business today?

Shanker Trivedi: Yeah. So, we make the world's best hardware, the GPU, the DPU and now the, the CPU and systems and racks and data centers, but we are not a hardware company. We're not even just a software company, we're a computing company, and most important of all, we're a platform company. A platform company starts with developers and outreaching to developers. We are the only AI computing company that works with all of the other AI companies, and that has a very wide base of developers and ISVs that are supporting the platform.

Chad Watt: You have a rich set of application software for AI that's been developed within NVIDIA. How do you collect that catalog that the power users, the developers have put together and make it accessible to people who are less proficient with technologies, people who aren't power users, people who are just new to AI?

Shanker Trivedi: Yeah. So, what we do is we take all of our body of software work, um, you know, we have more than 400 SDK, software development kits, and we make it available, um, in a, in a very public forum called developer.nvidia.com to every single developer, uh, at no cost. Right? So that's the first step. And then, we encourage all of the developers to publish all that work.

A so we have, you know, um, hackathons and, and, you know, workshops and training classes everywhere around the world and all of this body comes together in, in something which is the world's biggest developer conference called GTC. A- and so, that's the foundational work. Now what happens is that, you know, people need to say, "Okay, so this is good, now I, you know, I need to started on, my own project." So we, we make it easy for people to then consume this software.

Many years ago, we actually put all of this body in a cloud-native way, so you could just run it on any, any single cloud provider, uh, for free. And we call that body of work the NVIDIA GPU Cloud, NGC. And so, today, on NGC, we have hundreds of software containers, we have, trained models, we have model training scripts, we have, workflows, we have Helm charts, all of which are designed to make it easy for, you know, a reasonably well-educated, engineer to put together, a full AI workflow.

And then, you know, for our Enterprise customers who tend to have, you know, need somebody to help them, we have created two pieces of software. One is called NVIDIA AI Enterprise and one is called NVIDIA Omniverse Enterprise, which make it easy for customers to buy fully supported software, including previous versions and so on, with a proper enterprise levels of support for those customers.

Chad Watt: Could you do this without cloud?

Shanker Trivedi: Uh, you could, and, you know, the beauty of saying your software is cloud-native is you can run it everywhere. You can run it in the public cloud, you can run it on a private cloud, you could run it on your own computer, you can run it on your PC. You can even run it, in our case, on a device. So we are, you know, embedded in many, many devices. You know, out in hospitals and in warehouses and other sort of factories and so on. So it doesn't require the cloud, but it is helpful if it's cloud-native.

Chad Watt: So being cloud-native gives you that, versatility, but then you could put it in some sort of edge AI solution where it can stand on its own, in a warehouse in a [inaudible 00:05:52] and be self-sufficient, or you can, use it in some sort of, uh, kind of universal cloud setup, in a public cloud wherever you like, then.

Shanker Trivedi: Correct.

Chad Watt: NVIDIA, you talk about turning data centers into AI factories. Uh, is a, probably a pithier way of saying, than what I said. How do you turn a data center into an AI factory?

Shanker Trivedi: Yeah, so the first thing is this notion of a factory. Right? That's quite important. What a factory does is it, it takes raw materials, uses some machinery and some processes to create finished goods. That's what a factory does. So an AI factory, in the same way, will take the raw material, which is data, and then the machines are software robots, right? Think of these as software robots, they're machines that, work on the data, right? In some sort of way, and the finished product is some sort of intelligence.

A more intelligent process, a new way of doing things, a better way of doing something. So it's, and here's the interesting thing. As the world changes, right? As the data changes, the factory has to operate continuously in order to keep the intelligence current. It's just like if we, as human beings are not, you know, lifelong learners, we get stale and rusty and ultimately irrelevant. In the same way, an AI factory has to be kept current and relevant.

And, of course, all of these software robots sit on some kind of large, powerful data center, computing stack. And so we call that, an NVIDIA SuperPOD or an NVIDIA SuperCloud, depending on whether it's on prem or on the cloud.

Chad Watt: Shanker, what sort of companies should be pursuing AI?

Shanker Trivedi: Yeah, so, I mean, I think the answer is every single company should pursue AI, but the most important thing is to work on the projects which are going to deliver the most amount of return on investment.

Chad Watt: How do you govern that?

Shanker Trivedi: So, well, let me, before I talk about governance, because governance happens only after you do something, let's first talk about what does a good ROI mean? You know, I, the other day, I was with an industrial automation company and we're discussing where they should focus their energy on and their people said, "Oh, you know, look, we're gonna do this HR recruitment project and use AI to screen all the resumes and it's a really powerful project."

I said, "What's gonna be your ROI on this? You know, how many recruiters, how many recruits do you have, add up the total number of recruits, you know, the best you're gonna save is the, the entire cost of recruitment." Right? And it amounts to maybe, you know, uh, $1,000,000. And, they're an industrial automation company. They have large capital assets. The best ROI for them is the [inaudible 00:08:52] of their asset. Imagine if you own a power generation system. You should use AI to improve the [inaudible 00:08:59] of the power generator.

That's what's gonna give you the most amount of return on investment. So it's super important to choose projects that are going to move the needle. Then, people say, "Hey, Shanker, but, you know, what if it fails?" I said, "Well, if it fails, at least you'll have learnt something that's gonna be of value to your core business instead of something that was of value to, you know, a non-core part of your business." So it's super important to choose the right projects with the right ROI.

Chad Watt: I don't know if you know this, but in our data and AI radar project that I just finished up, the single best functioning use case was using AI for predictive maintenance on you know, very large, heavy assets like wind turbines and jet engines, that sort of thing. It, it was a very, very business-focused, you know, capital-intensive asset and that's where it was working really well.

Shanker Trivedi: Yeah, I mean, you know, Siemens Energy has, has built a digital twin of their heat reduction steam generator. It's fully, you know, and, and they can in detail, predict exactly when to bring the machine down, where exactly the maintenance crews are gonna work on, reduce the amount of maintenance time and thereby maximize the amount of productive time. That's a massive ROI.

Chad Watt: Right. And it's very clear and concrete, this is not AI that's, uh, this is not AI that's hard to discern, this is very clear, um, clear, very useful. Very useful.

Shanker Trivedi: Another area, by the way, is many, you know, the hyperscalers. The big in- consumer internet companies, everyone thinks, "Oh, you know, AI is for them." And, of course, it is. And they think it's because they have these vast amounts of data and therefore they can apply these fancy models and they have computing power. Now here's the most interesting thing that I'm finding with our Enterprise customers. The amount of data is not as important. You know?

Knowing your data is super important, having good data scientists is super important and when I speak to them, the problem of diversity of data and removal of bias and, and the accuracy of the model can be fixed by synthetic data generation. This is a new domain within AI where the computer will produce the data, and the beauty of the computer producing the data is that, by definition, it can be unbiased. And by definition, it can go into corner cases. We've learnt a lot by doing our own self-driving car, of the value of synthetic data generation.

Chad Watt: Got it. Tell me, let's back up real quick. Tell me why is, uh, NVIDIA making your own self-driving car?

Shanker Trivedi: So, you know, we want to, obviously, so if you think about one of the main uses of AI is autonomous vehicles. And if you wanna be a platform, you need to make a, you know, you need to go understand it end to end. You can't have a platform for autonomous vehicles or, or autonomous robots if you don't build your own vehicle and your own robot. Right, that's the starting point. You know? We're engineers.

You've gotta see what actually happens. We don't want people just, you know, talking theoretical stuff on a PowerPoint. So, we built, but we're not in the business of producing, like, a huge fleet of cars, we don't need 1,000 cars. Because we're a simulation company, we realized that we can just have a relatively few number of cars, but based on having those, we can actually then artificially generate a huge body of, uh, synthetic data and use that data to train the engine of the car.

Chad Watt: You can have a small fleet of cars and then simulate, create synthetic data to make that small fleet look like a large fleet of cars.

Shanker Trivedi: Look like a very large fleet of cars. You know, uh, this is our gaming heritage as well, right? Notice, you know, in gaming, we create artificial worlds and then we can simulate different climates and different geographies. And nowadays, the simulations are real. We're talking about real geospatial coordinates with real, you know, wind and temperature and climate and so on.

Chad Watt: We're heading into 2023 and you've described some amazing things. Where do you see the next major breakthrough in AI? Is it in this kind of metaverse side of things? Is it the language processing? Data management? Data processing?

Shanker Trivedi: Yeah, there are three areas that I think come to mind immediately. Um, so the first thing is, large language models. Um, today the language models have become so good, you know, and we have language models which are, like, 500,000,000,000 parameters, are trained with a language model, that does it better than human being. And so, what's happening now is you can use these language models as a foundation to create a company-specific, a domain-specific or a process-specific language model to do customer service, for example.

And, this language model will, imagine you were a bank, it knows the language of the bank, the products of your bank and so on. For example, at UF Health System, we've trained our big model, which we call NeMo LLC, and we've made a bio version of it called BioNeMo, and we worked with the University of Florida Health System to understand electronic health records.

And so, now, when the doctor speaks into, you know, does their notes, or when the doctors is reading the notes, the machine helps the doctor to identify patterns from previous discussions with doctors or to, to provide new, um, health records, you know, when you're putting in the, uh, details of the most recent visit, it can annotate and enhance the work that the doctor's doing. And this, you know, this is gonna be prevalent everywhere.

Uh, you know, you can think about so many different languages itself and so many different domains and so many different types of processes, so I think that's gonna be huge. Large language models becoming production.

Second, generative AI. So, generative AI is where the computer produces the data. And so, I already talked about synthetic data generation and that, you know, right now, uh, there's a lot of excitement in the, in the Twittersphere and in the consumer area because, you know, um, OpenAI's GPT has now been opened up and people are saying, "Wow, the computer is writing these amazing essays just with a few prompts. Isn't that incredible?" Or, "The computer is creating these new images just by saying, you know, do me something in this, of this style."

And, the thing which is interesting in all of that is, what's the use going to be in businesses? And so, um, let me give you examples. For example, the same technology which we've had for quite a while, uh, we're, you know, we're working with a number of people who are clinicians and they are using it to enhance, uh, their understanding of medical images and create better models in a federated, HIPAA-compliant, completely privacy environment, so that you can better understand, you know, cancers and, and, and, you know, just with a, a, a simple biopsy, you get a lot more insight than you could get before, because the machine is now better at doing it.

So, that, you know, these are sort of, you know, I think the generative AI is, gonna be, I don't know in your industry, uh, Chad, you know, like, whether they're gonna write the scripts for you or what, you know, no, or, but we'll see. We'll see what happens.

And then, the third one, which is, uh, a really interesting one, is the use of recommender systems. So, today, you know, the best consumer internet companies have amazing recommender systems which allow you to go from, you know, millions, hundreds of millions of items in a product catalog into the 10 best things that you should buy right now based on, you know, whatever data they have about you. And that's an example of a recommender system.

Now, this technology is becoming democratized. And so, all of the e-commerce providers and all of the retailers now will have access to these very high-performance, very, very good recommender systems that will give people the ability to do the next best action. And notice we're in a new world where supply chain and logistics is also pretty important, so the same recommender system can recommend supply routes and sources of supply and logistics to do a better job. So I think recommender systems are gonna have a incredible impact in 2023.

Chad Watt: Because we had such a disruption and such a pattern change, not in just in supply chain, but in, also in language models. We speak differently now than we did before the COVID pandemic, we order and behave differently, uh, the supply chain models that predated that didn't really work. And we went through some chaos, but, let me just ask you, are we gonna have better models coming out the backside of this now that we've had this reset?

Shanker Trivedi: I believe so, I mean, I think that the forecasting models have become so much better as a result of, um, you know, the, the situations that we've had in the pandemic. Uh, people are paying a lot more attention to, uh, diversity and, um, resilience of the supply chain. And, in fact, an AI can actually help you assess that risk and, and work through all the different scenarios so that, you know, uh, you as a decision-maker, um, have way better insight before you make certain decisions on your supply chain.

Chad Watt: Very fascinating, very fascinating. Thanks, Shanker. AI's not new, it's just newly accessible. Newly democratized, I guess I would say. What industry, think, let's bring it back to business. What industry stands the most to gain from AI? And I'll give you some parameters, in the next, uh, year or two.

Shanker Trivedi: So, manufacturing automation, the value for quality control on a recall, for example, is just giant. If you reduce your type one errors, you know, um, by 50%, the cost of doing, you know, the recall, the warranty replacement's just giant.

Chad Watt: And we're talking about assets, you know, talk about assets that have, really, a describable value right now. It's, you could put a dollar figure on this immediately. Very fascinating. What industry right now is doing the best with AI?

Shanker Trivedi: I think healthcare is doing amazing work with AI. You know, um, we could not get to the vaccinations without the kind of computing that's being provided. The healthcare industry has changed from a wet lab, um, uh, kind of environment into using computers to do the simulation.

Chad Watt: That's really something. That's really something. So you have a business leader who wants to understand AI. They're new to it completely. Where do you recommend she or he start?

Shanker Trivedi: The three things in NVIDIA, one, read all of our blog posts. Number two, we get these incredible, training classes, we call them DLI, they're all free, basically, off the web or a nominal charge. And, number three, we have this, NVIDIA LaunchPad. So if you actually have a project, you can just, you know, click on nvidia.com and we will make a launch, we will provide you a curated lab for your project for up to three weeks. NVIDIA LaunchPad's incredible.

Chad Watt: NVIDIA LaunchPad, we'll check it out. Thank you, Shanker Trivedi, very much for your insights today. This podcast is part of our collaboration with MIT Tech Review, in partnership with Infosys Cobalt. Visit our content hub on technologyreview.com to learn more about how businesses across the globe are moving from cloud chaos to cloud clarity.

Be sure to follow Ahead in the Cloud wherever you get your podcasts. You can find more details in our show notes and transcripts at infosys.com/iki. It's in our podcast section. Thanks to our producers, Catherine Burdette, Christine Calhoun and Yulia DeBari. Dode Bigley is our audio technician. I'm Chat Watt with the Infosys Knowledge Institute. Until next time, keep learning and keep sharing.

Shanker Trivedi: Thank you, Chad.

Chad Watt: Thank you, sir.