May 7, 2024 • 30 mins
For as long as humans have used computers, we have sought to make them faster, smarter, and more capable. So what new potential can harnessing the power of AI in PCs promise? What if every computer could use AI technology to unleash new capabilities that will benefit anyone and everyone who uses a computer. Plus, imagine data security that AI PCs can offer that Cloud AI cannot. Intel Vice President Robert Hallock helps us explore how computers equipped with artificial intelligence–like the new Intel Core Ultra Processor–are transforming productivity and IT operations.
Learn more about how Intel is leading the charge in the AI Revolution at intel.com/AIeverywhere
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:12):
For as long as humans have used computers, we have
sought to make them faster, smarter, and more capable. But
what happens when we harness the power of artificial intelligence
with new PC hardware and software? And what if every
computer could use AI technology to unleash new capabilities that
will benefit anyone and everyone who uses a computer.
Speaker 2 (00:34):
What can I help you with?
Speaker 1 (00:36):
This is not a distant dream. Each day it becomes
more and more of a reality. AI is no longer
exclusively running in the cloud. It's increasingly finding its way
into all computers, enabling business users to do more and
be more with PC technology from Intel. In this episode,
(00:56):
we'll be focusing on defining the AIPC and what represents
for the future of this transformative technology. We'll also explore
what the growth of AI running directly on the PC
versus the cloud means and how this AIPC revolution changes
how we work, live and create. Join us as we
(01:16):
take a journey into the future of AI computing, where
machines are not just tools but partners in our endeavors.
Welcome to Technically Speaking, an Intel podcast produced by iHeartMedia's
Ruby Studio in partnership with Intel. Hey the I'm gram class.
(01:37):
Joining us today is Robert Hallock, the VP and General
manager of Client AI and Technical Marketing at Intel. Here
has a long history with product development and PCs. Welcome
to the show, Robert.
Speaker 2 (01:52):
Thanks for having me.
Speaker 1 (01:53):
Good to be here. Yeah, I just want to the
start off by saying, there seems to be a lot
of talk around AI, and it seems to always revolve
around cloud based or software as a service type business models.
How Intel is taking a different route. Well, but can
you give us an overview of what an AIPC means
(02:13):
at Intel and can you help us understand some of
the benefits of running AI directly on a PC versus
in the cloud.
Speaker 2 (02:22):
Yeah, a lot of people have been exposed to AI
through the cloud, and these are services like chat, GPT
or Dolly three where you can create a piece of
content from a text description. And that's one kind of
AI that's called generative AI is a category pros and
cons to that doing it in the cloud. The pro
(02:44):
is the models are huge, right. They essentially have the
Internet's collective knowledge of data to produce an answer or
a picture, and that's highly detailed. But one of the
cons is that they're not very specific to you what
you are doing on your PC, the information on your computer,
(03:05):
or the images that you care about. So that's sort
of motivating to bring AI capabilities locally to the PC
where you don't need an Internet connection to use these capabilities.
And that's sort of the tip of the iceberg, because
when you move AI to the PC without a cloud connection,
(03:25):
you can also do new types of AI workloads that
would be too big or simply can't run in the cloud.
So a good example of that is in teleconferencing, lots
of people use background blurring or background pictures, and we
can actually make that a lot more energy efficient. We
can actually give you hours of battery life back by
(03:48):
using AI on that use case. So the whole point
of AI is that there's a wave of software coming
that uses AI to improve performance or to reduce power consumption.
So you'll get better performance from your system and longer
battery life as well. And this will continue to grow
over the next five years.
Speaker 1 (04:06):
And Robert I was wondering if you could please describe
to the audience what the current state of the art
is with PC architecture. You know, we've got CPUs, which
are central processing units that make computing possible, and you've
got GPUs, which are graphical processing units that are instrumental
for machine learning, gaming applications, video editing. So how does
(04:28):
that actually differ from the concept of an AIPC.
Speaker 2 (04:32):
That's a great question. So, yeah, users may have heard
about an AIPC at this point, and at the basic level,
this is just a new generation computer with hardware inside
that is capable of accelerating an AI based workload and
previous hardware, let's say early twenty twenty three computers, you
(04:53):
can run AI workloads, but they will fall back to
the CPU cores, which are not as fast and not
as energy efficient as running that same workload on the
new accelerators in an AIPC. And the way we're doing
it at Intel is actually the CPU cores themselves. We've
(05:14):
added AI accelerating capabilities to those cores. The GPU that's
built into our processor also has accelerators, and then we've
also added an entirely new component called the NPU or
neural processing unit, which sits next to the CPU and
the GPU and is now its own third category of
(05:34):
acceleration on the device. And different workloads in AI, or
different features in AI run best on one of those
three engines, so you kind of need all three to
do this well. And our new product, Intel Core Ultra
is our first AIPC processor, and so if you see
that name Core Ultra, you know that it has the
(05:57):
AI accelerators to run these features well. But at its route,
it is a PC that has AI specific hardware.
Speaker 1 (06:04):
And I'm quite interested in I guess the thinking behind
at Intel of this shift towards this new architecture. Was
it something that was internally driven or did you see
some conversations and discussions with your key customers and clients
driving that sort of shift.
Speaker 2 (06:24):
This is very much a software industry driven transition, and
CPU vendors like Intel were innovating to keep pace with
what's going on in the software environment. And that's kind
of a key thing I'd want to stress. You know,
maybe you're on the fence about AIPC or you wonder,
you know, how long is this thing going to stick around.
This is one of those cases where it's both the
(06:44):
hardware industry and the software industry agreeing that this is
the right thing to do for performance and features and power.
Because I truly believe that over the next three to
five years we will reach this point of general acceptance
in AI, where it's whether you know it or not,
widely diffused. In most of the applications you're working on
(07:06):
or working with, many of the features that you value
will use AI again transparently in the background. But this
is very much a collaborative, industry wide effort from all
the hardware makers, all the big software vendors. This is
here to stay for sure. Let's pause for second here
to reiterate Robert's point. Right of adoption is always a
(07:29):
crucial aspect of any new technology, and that's certainly true
of what we're discussing today with AIPCS. It's important to
understand that the leaders in the field of computing, both
in the hardware and software domain, have already begun down
this revolutionary path and it doesn't seem like there's any
roadblocks in site. With that in mind, I asked Robert
about the benefits of moving AI workloads from the cloud
(07:52):
to the PC, especially when it comes to issues of
data privacy, latency, and connectivity. You just touched couple that
are really important to local AI. It's that data privacy
or data security. We've all read about information going up
to a cloud resource of some kind, and you don't
really know what's going to happen with your data or
(08:13):
your request after that. And that's not to say there's
anything malicious implied. You just don't know. So I'm sure
people in corporation, certainly at Intel, when you go to
a generative AI website, it says, hey, be careful what
you enter into this textbox. And so moving this stuff
offline gives you a couple of things. Yes, you get
chain of custody over the data. It's private, right, it's
(08:35):
working on your information offline, right, so it doesn't have
to go to the cloud. That's a big one. Cost
is another component. A lot of the most powerful AI
services online, you know, ten to twenty bucks a month,
and it's not cheap to have several of those. The
last that is interesting is cloud servers are intrinsically pricey.
(09:03):
I'm not saying they're expensive, but for AI as a
genre of software to truly take off and thrive in
all the way the software vendors want it to, it
has to reach the local PC. It has to reach
you know, tens of millions of users, hundreds of millions
of users on a local device and that's sort of
(09:25):
that critical mass install base that takes this effort to
the next level. Cloud was sort of one point zero
of AI, and now we're trying to, you know, for
one of a better term, create the two point zero
where lots of people have access to this and it's
widely available and in a couple of years Intel alone
we want to get one hundred million accelerators for AI
(09:47):
into the hands of people.
Speaker 1 (09:49):
And do you have any examples of at this early
stage of actually pushing the boundaries of using this sort
of power in a local PC to do some really
interesting work.
Speaker 2 (10:03):
One for an enterprise that we've been tinkering with is
a technology called rag rag, and it's the idea where
you have a language model running offline on the user's PC.
But this RAG component can scan your documents, your corporate information,
(10:24):
and then specialize the LM to be for you, your work,
your knowledge. So just as an example, we scanned one
state's DMV manual, which hundreds of pages long for the
Department of Motor Vehicle manual and now you can ask
very specific procedural and legal questions about the subject of
(10:47):
that manual and it'll spit back highly accurate answers for you.
And if you extend out out words, protecting and promoting
institutional knowledge is hard. You might have that employ that's
been there for twenty years and has all that institutional
knowledge in their head and if they leave, it goes
with them. But a RAG model could synthesize that knowledge
(11:11):
for people, so a new employee could just ask a
question in a text box and get an accurate answer
about what that company's working on or what this feature does.
And that's just huge for business.
Speaker 1 (11:25):
Yeah, you know, I'm from a small business sort of background.
My dad has a small business. And the fact that
you can bring this power to the small and micro
businesses as well without having to pay these cloud based
prices and also in conjunction with using some of the
open source type software that I know Intel is very
(11:46):
supportive of. I'm just really excited to see the little guys,
you know, be able to compete with some of the
technology that the big boys have.
Speaker 2 (11:54):
Absolutely, AI is, at the end of the day, a
force multiplier for a person. Right like at the root,
AI is designed to save time writing meeting minutes or
email summaries or drafting in outline. These are all just
like time consuming tasks for people, and they don't require
(12:16):
skill per se, but it's it's time consuming, and so
being able to offload that to a digital assistant that
can just sort of ninety percent or ninety five percent
do that for you allows you to refocus your efforts
back to something else that is more productive and more
worthy of your time. And especially for a small business,
administrative overhead, bureaucratic overhead is hard, it's time consuming. I
(12:39):
myself own a single member LLC and administrative stuff takes
up a ton of my time. Like I'd love to
outsource that to AI.
Speaker 1 (12:49):
That's right. In terms of Intel's history with past, you know,
technological revolutions, I'm reminded of Intel's initiative to get Wi
Fi into every your laptop and that was code named Centrino,
and it's interesting to hear that again. Intel are trying
to push new technology so that it's ubiquitous, and that's
(13:13):
exactly what they're doing with these aipcs. And in ten
twenty years time, I think that aipowered PCs will be
so ubiquitous that we won't think anything of it. I'd
like Robert your thoughts on that and more insights into
the way Intel is evolving their strategy and Intel's role
(13:34):
in this. Yeah.
Speaker 2 (13:35):
Actually, Centrino is a really nice analogy because that's sort
of what we're trying to do with these AI accelerators.
It's not a huge tweak to the configuration of a
system design because most of the work happens inside the CPU,
and there are very few external requirements that would change
(13:56):
a system designed to make this possible. Right, So it's
a system vendor could theoretically update last year's chassis to
have a new CPU and that would confer the benefits
from Core Ultra and an AI workloads and Centrino not
much different, right. You're adding a Wi Fi chip and
an antenna to the system.
Speaker 1 (14:17):
Yes, But for those.
Speaker 2 (14:18):
Of you who weren't around during the Centrino days, it
used to be very common that a laptop would not
have Wi Fi, which seems sort of unimaginable now, but
that's because Intel made this massive effort to make it
common and we all sort of take it for granted now.
Another analogous moment for me is the addition of graphics
(14:39):
in the processor. I was a hardware reviewer when that
started happening, and I remember the conversations back then like
why are we doing this? You can't even play a
game on it, what's the point? This is just going
to make CPUs more expensive, YadA, YadA, YadA. Today web
pages are rendered by your graphics card orphics accelerator in
(15:00):
the processor, your browser uses it. It's everywhere. So both of
those are very foundational examples for me, because I see
AI as being quite analogous in both respects. But I
think history will bear out that this was a pivotal
moment and AI will be very, very widespread, just like
graphics and just like Wi Fi.
Speaker 1 (15:23):
Coming up next on Technically Speaking and Intel podcast.
Speaker 2 (15:28):
Being familiar with how prompts work, it's going to be
a key business skill, and I think there'll be a
real advantage for employees who know how to engineer a
good prompt to get a great result quickly.
Speaker 1 (15:42):
We'll be right back after a brief message from our
partners that Intel welcome back to Technically Speaking. I'm here
now with Robert Hallett getting back to I guess more
(16:06):
on the business side of things. Maybe if you can
talk a little bit about our friends in the IT
department having to manage the security and all these sort
of things. Yeah, what's some of the benefits they could
look forward to in terms of managing these types of
aipcs in their enterprise.
Speaker 2 (16:22):
I think it'll make every ITDM happy that at the
end of the day, these AI applications are nothing more
than endpoint applications. You download them, you install them, and
they're from software vendors, and I'm sure they'll be breakout
ISVs that come under the scene as a result of
this AI transformation, but largely speaking, trusted vendors doing new work.
(16:43):
And because it's entirely offline, your user has custody of
the data and the information, which is no different from
any other application today. So it's not like this transformation
comes part and parcel with like a radical transformation and
endpoint management, which would make it way harder. From a
security point of view, AI has some very interesting benefits
(17:08):
for security models. So now this is the ITDM point
of view. We recently with Dell and CrowdStrike, and if
people don't know what CrowdStrike is, it's, amongst many things,
an endpoint security solution which is specifically designed to help
prevent threats that don't attack files. Many of the tax
(17:29):
that go into a system now aren't like a virus
that infects a file, it's actually resident in memory. They're
fileless attacks, and these are way harder to detect and prevent.
So CrowdStrike uses convolutional neural networks, which is a simpler
form of neural network or AI, to monitor the real
(17:52):
time conditions of the system and see if something unusual
is happening. And this experiment moves these convolutional neural network
models or CNNs to that neural processing unit or the
NPU in coraltrum. It did a couple things when we
did that. First, it gave the processor cores twenty percent
(18:13):
performance back. The second thing it did it made the
security model slightly smaller, so it had a smaller memory
footprint now so user gets some RAM back. And it
also improved the accuracy of the model because they can
make the model computationally bigger, right because it has its
own dedicated accelerator now to run on. And so security
(18:36):
got better and crowdstreg is a very popular solution, and
there are other solutions in the pipe for phishing detection,
which is notoriously hard pattern matching problem. And that's just
two examples of the way security can be enhanced by
offloading to an AI specific accelerator.
Speaker 1 (18:56):
Yeah, and we talked a little bit about you know,
businesses and enterprises adopting these new aipcs. Is anything special
that IT departments and organizations need to do to prepare
for this.
Speaker 2 (19:09):
I'll say there's probably three things that an ITDM would
want to think about if they intend to use large
language models or just generative AI. Those workloads are pretty
sensitive to memory bandwidth, so you wouldn't normally think about
memory bandwidth in a system purchase, but making sure that
(19:30):
it has two memory sticks over one. For example, right,
if you want to get sixteen gigs a RAM, make
sure that's two by eight instead of one by sixteen,
Just as an example, that will dramatically improve the performance
of the LLM, because these language models are fundamentally limited
or enhanced by the performance of the memory subsystem. Outside
(19:52):
of that, for other forms of AI in the year ahead,
there is a calculation called TOPS or terra operations per second,
which is sort of a ballpark for how much AI
performance a device can give you. Software makes or breaks
the acquisition of that TOPS figure. So I can give
(20:16):
you a billion TOPS, and if I had a very
poor software stack under that you would never see the
billion tops. So there's a new level of knowledge the
ITDM needs to develop on sort of software stack robustness
underneath that rating. Okay, so you have to be familiar
(20:37):
with what Intel's doing in the software space, what its
competitors are doing in the software space to really understand
whether or not you're going to get a good experience
out of the device you're purchasing. So that's number two,
and number three I think would be to say that
itdms will probably be faced with a lot of advocacy
for the NPU as an AIX, but it's important to
(21:01):
understand that the software industry broadly also wants to use
GPU and CPU as well. So if you make an
upgrade decision that is all in on NPU performance and
you didn't check on the GPU or the CPU, you
may be out in the cold on performance or power
(21:22):
efficiency for these Frankly a large number of workloads that
use graphics and CPU for AI acceleration. Those are the
three things that I would say are new or different
in this era, But overall it is something that you
can integrate into your upgrade cycle piecemeal. YEP, Newer devices
(21:43):
will be faster. I mean, they always are, but it's
not like you're missing out on a new feature, right,
We're going to make them faster, But the features are
delivered by the software, are not the hardware. You can
kind of get in at any point and get the
goodness of AI, which is pretty cool as well.
Speaker 1 (21:59):
Yeah, yeah, And are you helping the software vendors, I
guess compile their code so that it will help them
really utilize that hardware underneath.
Speaker 2 (22:10):
Yeah, that's the secret of AI. And I'll start with
an analogy of PC gaming, which I think is a
lot more familiar to people. So at the bottom, you
have a piece of hardware, a graphics card, and they
tell you, you know, it's certain terra flops or gigaflops of performance,
but everybody knows that really depends on how well optimized
(22:32):
the game engine is, how well optimized the game you're
running is, how good the graphics drivers are. AI is
no different. AI accelerators are actually exposed in direct X
in Windows as a GPU without display outputs, so the
system sees them as essentially graphics cards, and instead of
(22:57):
game engines you have AI models will have features and
apps just like games you even have run times or
an environment where the code is running in there's a
DirectX run time for graphics, there is equivalent run times
for AI. So in many respects, the AI software stack
looks and works a lot like the gaming software stack.
(23:21):
And so if people think about all the times that
a GPU that's supposed to be faster on paper didn't
live up to that number because of one software reason
or another, that is a possible reality for AI as well.
And that's why it's so important to make your decision
(23:41):
not just on tops or what the accelerator is, but
on the robustness of the software underneath, because that's where
it really happens.
Speaker 1 (23:52):
Regular listeners to the show Mike remember back in season one,
we divided a whole episode on the skills that workers
will need in order to take advantage of the kinds
of AI tools we discussed today. AI technology can only
expand our expectations of what's possible if we understand the
most effective ways to use it. So I asked Robert
(24:12):
for his thoughts and what workers should focus on to
take advantage of this new wave of technology, and he
began his answer with something just about everyone does on
the Internet, every day.
Speaker 2 (24:23):
Ooh, that's a good one. I actually want to start
briefly at web searching. You know, web searching, a good,
well composed query is a learned skill. We have all
encountered people that haven't quite learned that skill, and they
get bad search results and they're frustrated with a search engine.
And not a lot of people teach that skill because
(24:44):
it's its own language of sorts. You know, you want
to freeze things to a search box differently than how
I would ask it out loud. So that takes me
to AI, where you are still engineering a search of sorts.
It's called a prompt now, but it's a search and
(25:04):
the skill is in like, how do you craft that
prompt to get the result that you're looking for? You
have to be somewhat specific, and you have to understand
the limitations of the AI model you're working with. So
let's take an image creation model. Some of them only
support what's called positive prompts. You have to describe the
(25:26):
things that you want and it will kind of omit
anything else that you're not asking for explicitly. Others support
negative prompts, and if you're working with a positive only
image service, you could easily have users saying I want this,
this and this, but not that, that and that. Yeah,
the not operator the AI engine's not going to understand it,
(25:49):
so it's going to give you the things in the
not category. In the picture.
Speaker 1 (25:54):
Yeah, it's like saying I want a picture of a zoo,
but no elephants.
Speaker 2 (25:58):
Right, and it doesn't nderstand the word no, So in.
Speaker 1 (26:01):
A positive one, it'll just have elephants.
Speaker 2 (26:03):
Yeah, you get all elephants exactly right. So understanding this
is a real skill. Understanding like what the AI model
or by distant extension, what the game engine will let
you do is really important, and how to be specific
and understanding how to provide follow up commentary to tweak
(26:26):
the result to get what you're looking for.
Speaker 1 (26:28):
This is a.
Speaker 2 (26:28):
Whole category of skill that is not taught, not widely
known in business or even amongst users because this is
so new. But being familiar with how prompts work is
going to be a key business skill, and I think
there'll be a real advantage for employees who know how
to engineer a good prompt to get a great result quickly.
Speaker 1 (26:51):
So going to take a bit of a step back
and look far in the horizon in the sort of
the three to five seven year time range. What's Intel's
plan for this journey into AI.
Speaker 2 (27:06):
I think at top level, i'd want to acknowledge that
I understand the skepticism or the I don't understand of AI.
But where we're going is like a complete industry transformation.
Intel believes by twenty twenty seven or twenty twenty eight
that eighty percent of all the computer sold will have
AI accelerators inside. And by the end of this year,
(27:30):
we want at least one hundred different software developers partnered
with Intel. We want to deliver three hundred or so
different AI features into the marketplace through all of those companies,
help them optimize it, deliver it, market it. We want
to bring one hundred million accelerators into the space by
the end of twenty twenty five, and so just between
(27:53):
now and twenty twenty seven twenty eight, we're talking about
zero to eighty percent of the market in under four years,
which is an extraordinary velocity. And as of right now,
Intel has the largest number of accelerators, the most number
of applications in the market, and sort of below the scenes,
(28:13):
all the enabling tools and softwares and frameworks, we also
have the most of those as well, So we want
to be the scale provider for AI, you know, the
biggest install based and we want to help to succeed
because that's what software vendors are asking us to do.
So that's the longest short of it.
Speaker 1 (28:30):
That's great. I think I'll leave it there. Thanks Robert.
Thanks my deepest thanks to Robert Hallock for his invaluable
contributions to today's episode of Technically Speaking. Robert's passion and
enthusiasm has convinced me that we are on the brink
of a revolutionary technology that could transform humanity. We've all
(28:51):
used PCs both in our personal and professional lives, but
the leap to aipowered processes represents a once in a
generation advancement. Imagine having your own personal AIS system that
continually adapts to your needs and priorities. Admittedly, this is
a prospect I'm still trying to fully grasp, but I
especially appreciate that this technology will operate locally on my PC,
(29:15):
keeping my data private and secure. I'm also particularly interested
in how technology can empower the underdog to do amazing things.
The garage based tech entrepreneur building the next world beating
app the corner cafe owner getting that extra hour per
day to spend with their family, and the first year
design student creating beautiful and innovative art. With AI enhanced
(29:37):
pieces readily accessible, I believe we can greatly advance human prosperity.
The future looks bright. Ground. Next episode will continue to
unpact the involvement of AI to create a more accessible
and livable city for everyone. Join us again on Tuesday,
April twenty third for another enlightening discussion here on Technically
(29:59):
Speaking and Intel podcast. Technically Speaking was produced by Ruby
Studio from iHeartRadio in partnership with Intel and hosted by
me Graham Class. Our executive producer is Molly Sosher, our
EP of Post Production is James Foster, and our supervising
producer is Nikia Swinton. This episode was edited by Sierra
(30:22):
Spreen and was written by Molly Sosher and Nick Ferschall.
For as long as humans have used computers, we have
sought to make them faster, smarter, and more capable. But
what happens when we harness the power of artificial intelligence
with new PC hardware and software? And what if every
computer could use AI technology to unleash new capabilities that
will benefit anyone and everyone who uses a computer.
Speaker 2 (00:34):
What can I help you with?
Speaker 1 (00:36):
This is not a distant dream. Each day it becomes
more and more of a reality. AI is no longer
exclusively running in the cloud. It's increasingly finding its way
into all computers, enabling business users to do more and
be more with PC technology from Intel. In this episode,
(00:56):
we'll be focusing on defining the AIPC and what represents
for the future of this transformative technology. We'll also explore
what the growth of AI running directly on the PC
versus the cloud means and how this AIPC revolution changes
how we work, live and create. Join us as we
(01:16):
take a journey into the future of AI computing, where
machines are not just tools but partners in our endeavors.
Welcome to Technically Speaking, an Intel podcast produced by iHeartMedia's
Ruby Studio in partnership with Intel. Hey the I'm gram class.
(01:37):
Joining us today is Robert Hallock, the VP and General
manager of Client AI and Technical Marketing at Intel. Here
has a long history with product development and PCs. Welcome
to the show, Robert.
Speaker 2 (01:52):
Thanks for having me.
Speaker 1 (01:53):
Good to be here. Yeah, I just want to the
start off by saying, there seems to be a lot
of talk around AI, and it seems to always revolve
around cloud based or software as a service type business models.
How Intel is taking a different route. Well, but can
you give us an overview of what an AIPC means
(02:13):
at Intel and can you help us understand some of
the benefits of running AI directly on a PC versus
in the cloud.
Speaker 2 (02:22):
Yeah, a lot of people have been exposed to AI
through the cloud, and these are services like chat, GPT
or Dolly three where you can create a piece of
content from a text description. And that's one kind of
AI that's called generative AI is a category pros and
cons to that doing it in the cloud. The pro
(02:44):
is the models are huge, right. They essentially have the
Internet's collective knowledge of data to produce an answer or
a picture, and that's highly detailed. But one of the
cons is that they're not very specific to you what
you are doing on your PC, the information on your computer,
(03:05):
or the images that you care about. So that's sort
of motivating to bring AI capabilities locally to the PC
where you don't need an Internet connection to use these capabilities.
And that's sort of the tip of the iceberg, because
when you move AI to the PC without a cloud connection,
(03:25):
you can also do new types of AI workloads that
would be too big or simply can't run in the cloud.
So a good example of that is in teleconferencing, lots
of people use background blurring or background pictures, and we
can actually make that a lot more energy efficient. We
can actually give you hours of battery life back by
(03:48):
using AI on that use case. So the whole point
of AI is that there's a wave of software coming
that uses AI to improve performance or to reduce power consumption.
So you'll get better performance from your system and longer
battery life as well. And this will continue to grow
over the next five years.
Speaker 1 (04:06):
And Robert I was wondering if you could please describe
to the audience what the current state of the art
is with PC architecture. You know, we've got CPUs, which
are central processing units that make computing possible, and you've
got GPUs, which are graphical processing units that are instrumental
for machine learning, gaming applications, video editing. So how does
(04:28):
that actually differ from the concept of an AIPC.
Speaker 2 (04:32):
That's a great question. So, yeah, users may have heard
about an AIPC at this point, and at the basic level,
this is just a new generation computer with hardware inside
that is capable of accelerating an AI based workload and
previous hardware, let's say early twenty twenty three computers, you
(04:53):
can run AI workloads, but they will fall back to
the CPU cores, which are not as fast and not
as energy efficient as running that same workload on the
new accelerators in an AIPC. And the way we're doing
it at Intel is actually the CPU cores themselves. We've
(05:14):
added AI accelerating capabilities to those cores. The GPU that's
built into our processor also has accelerators, and then we've
also added an entirely new component called the NPU or
neural processing unit, which sits next to the CPU and
the GPU and is now its own third category of
(05:34):
acceleration on the device. And different workloads in AI, or
different features in AI run best on one of those
three engines, so you kind of need all three to
do this well. And our new product, Intel Core Ultra
is our first AIPC processor, and so if you see
that name Core Ultra, you know that it has the
(05:57):
AI accelerators to run these features well. But at its route,
it is a PC that has AI specific hardware.
Speaker 1 (06:04):
And I'm quite interested in I guess the thinking behind
at Intel of this shift towards this new architecture. Was
it something that was internally driven or did you see
some conversations and discussions with your key customers and clients
driving that sort of shift.
Speaker 2 (06:24):
This is very much a software industry driven transition, and
CPU vendors like Intel were innovating to keep pace with
what's going on in the software environment. And that's kind
of a key thing I'd want to stress. You know,
maybe you're on the fence about AIPC or you wonder,
you know, how long is this thing going to stick around.
This is one of those cases where it's both the
(06:44):
hardware industry and the software industry agreeing that this is
the right thing to do for performance and features and power.
Because I truly believe that over the next three to
five years we will reach this point of general acceptance
in AI, where it's whether you know it or not,
widely diffused. In most of the applications you're working on
(07:06):
or working with, many of the features that you value
will use AI again transparently in the background. But this
is very much a collaborative, industry wide effort from all
the hardware makers, all the big software vendors. This is
here to stay for sure. Let's pause for second here
to reiterate Robert's point. Right of adoption is always a
(07:29):
crucial aspect of any new technology, and that's certainly true
of what we're discussing today with AIPCS. It's important to
understand that the leaders in the field of computing, both
in the hardware and software domain, have already begun down
this revolutionary path and it doesn't seem like there's any
roadblocks in site. With that in mind, I asked Robert
about the benefits of moving AI workloads from the cloud
(07:52):
to the PC, especially when it comes to issues of
data privacy, latency, and connectivity. You just touched couple that
are really important to local AI. It's that data privacy
or data security. We've all read about information going up
to a cloud resource of some kind, and you don't
really know what's going to happen with your data or
(08:13):
your request after that. And that's not to say there's
anything malicious implied. You just don't know. So I'm sure
people in corporation, certainly at Intel, when you go to
a generative AI website, it says, hey, be careful what
you enter into this textbox. And so moving this stuff
offline gives you a couple of things. Yes, you get
chain of custody over the data. It's private, right, it's
(08:35):
working on your information offline, right, so it doesn't have
to go to the cloud. That's a big one. Cost
is another component. A lot of the most powerful AI
services online, you know, ten to twenty bucks a month,
and it's not cheap to have several of those. The
last that is interesting is cloud servers are intrinsically pricey.
(09:03):
I'm not saying they're expensive, but for AI as a
genre of software to truly take off and thrive in
all the way the software vendors want it to, it
has to reach the local PC. It has to reach
you know, tens of millions of users, hundreds of millions
of users on a local device and that's sort of
(09:25):
that critical mass install base that takes this effort to
the next level. Cloud was sort of one point zero
of AI, and now we're trying to, you know, for
one of a better term, create the two point zero
where lots of people have access to this and it's
widely available and in a couple of years Intel alone
we want to get one hundred million accelerators for AI
(09:47):
into the hands of people.
Speaker 1 (09:49):
And do you have any examples of at this early
stage of actually pushing the boundaries of using this sort
of power in a local PC to do some really
interesting work.
Speaker 2 (10:03):
One for an enterprise that we've been tinkering with is
a technology called rag rag, and it's the idea where
you have a language model running offline on the user's PC.
But this RAG component can scan your documents, your corporate information,
(10:24):
and then specialize the LM to be for you, your work,
your knowledge. So just as an example, we scanned one
state's DMV manual, which hundreds of pages long for the
Department of Motor Vehicle manual and now you can ask
very specific procedural and legal questions about the subject of
(10:47):
that manual and it'll spit back highly accurate answers for you.
And if you extend out out words, protecting and promoting
institutional knowledge is hard. You might have that employ that's
been there for twenty years and has all that institutional
knowledge in their head and if they leave, it goes
with them. But a RAG model could synthesize that knowledge
(11:11):
for people, so a new employee could just ask a
question in a text box and get an accurate answer
about what that company's working on or what this feature does.
And that's just huge for business.
Speaker 1 (11:25):
Yeah, you know, I'm from a small business sort of background.
My dad has a small business. And the fact that
you can bring this power to the small and micro
businesses as well without having to pay these cloud based
prices and also in conjunction with using some of the
open source type software that I know Intel is very
(11:46):
supportive of. I'm just really excited to see the little guys,
you know, be able to compete with some of the
technology that the big boys have.
Speaker 2 (11:54):
Absolutely, AI is, at the end of the day, a
force multiplier for a person. Right like at the root,
AI is designed to save time writing meeting minutes or
email summaries or drafting in outline. These are all just
like time consuming tasks for people, and they don't require
(12:16):
skill per se, but it's it's time consuming, and so
being able to offload that to a digital assistant that
can just sort of ninety percent or ninety five percent
do that for you allows you to refocus your efforts
back to something else that is more productive and more
worthy of your time. And especially for a small business,
administrative overhead, bureaucratic overhead is hard, it's time consuming. I
(12:39):
myself own a single member LLC and administrative stuff takes
up a ton of my time. Like I'd love to
outsource that to AI.
Speaker 1 (12:49):
That's right. In terms of Intel's history with past, you know,
technological revolutions, I'm reminded of Intel's initiative to get Wi
Fi into every your laptop and that was code named Centrino,
and it's interesting to hear that again. Intel are trying
to push new technology so that it's ubiquitous, and that's
(13:13):
exactly what they're doing with these aipcs. And in ten
twenty years time, I think that aipowered PCs will be
so ubiquitous that we won't think anything of it. I'd
like Robert your thoughts on that and more insights into
the way Intel is evolving their strategy and Intel's role
(13:34):
in this. Yeah.
Speaker 2 (13:35):
Actually, Centrino is a really nice analogy because that's sort
of what we're trying to do with these AI accelerators.
It's not a huge tweak to the configuration of a
system design because most of the work happens inside the CPU,
and there are very few external requirements that would change
(13:56):
a system designed to make this possible. Right, So it's
a system vendor could theoretically update last year's chassis to
have a new CPU and that would confer the benefits
from Core Ultra and an AI workloads and Centrino not
much different, right. You're adding a Wi Fi chip and
an antenna to the system.
Speaker 1 (14:17):
Yes, But for those.
Speaker 2 (14:18):
Of you who weren't around during the Centrino days, it
used to be very common that a laptop would not
have Wi Fi, which seems sort of unimaginable now, but
that's because Intel made this massive effort to make it
common and we all sort of take it for granted now.
Another analogous moment for me is the addition of graphics
(14:39):
in the processor. I was a hardware reviewer when that
started happening, and I remember the conversations back then like
why are we doing this? You can't even play a
game on it, what's the point? This is just going
to make CPUs more expensive, YadA, YadA, YadA. Today web
pages are rendered by your graphics card orphics accelerator in
(15:00):
the processor, your browser uses it. It's everywhere. So both of
those are very foundational examples for me, because I see
AI as being quite analogous in both respects. But I
think history will bear out that this was a pivotal
moment and AI will be very, very widespread, just like
graphics and just like Wi Fi.
Speaker 1 (15:23):
Coming up next on Technically Speaking and Intel podcast.
Speaker 2 (15:28):
Being familiar with how prompts work, it's going to be
a key business skill, and I think there'll be a
real advantage for employees who know how to engineer a
good prompt to get a great result quickly.
Speaker 1 (15:42):
We'll be right back after a brief message from our
partners that Intel welcome back to Technically Speaking. I'm here
now with Robert Hallett getting back to I guess more
(16:06):
on the business side of things. Maybe if you can
talk a little bit about our friends in the IT
department having to manage the security and all these sort
of things. Yeah, what's some of the benefits they could
look forward to in terms of managing these types of
aipcs in their enterprise.
Speaker 2 (16:22):
I think it'll make every ITDM happy that at the
end of the day, these AI applications are nothing more
than endpoint applications. You download them, you install them, and
they're from software vendors, and I'm sure they'll be breakout
ISVs that come under the scene as a result of
this AI transformation, but largely speaking, trusted vendors doing new work.
(16:43):
And because it's entirely offline, your user has custody of
the data and the information, which is no different from
any other application today. So it's not like this transformation
comes part and parcel with like a radical transformation and
endpoint management, which would make it way harder. From a
security point of view, AI has some very interesting benefits
(17:08):
for security models. So now this is the ITDM point
of view. We recently with Dell and CrowdStrike, and if
people don't know what CrowdStrike is, it's, amongst many things,
an endpoint security solution which is specifically designed to help
prevent threats that don't attack files. Many of the tax
(17:29):
that go into a system now aren't like a virus
that infects a file, it's actually resident in memory. They're
fileless attacks, and these are way harder to detect and prevent.
So CrowdStrike uses convolutional neural networks, which is a simpler
form of neural network or AI, to monitor the real
(17:52):
time conditions of the system and see if something unusual
is happening. And this experiment moves these convolutional neural network
models or CNNs to that neural processing unit or the
NPU in coraltrum. It did a couple things when we
did that. First, it gave the processor cores twenty percent
(18:13):
performance back. The second thing it did it made the
security model slightly smaller, so it had a smaller memory
footprint now so user gets some RAM back. And it
also improved the accuracy of the model because they can
make the model computationally bigger, right because it has its
own dedicated accelerator now to run on. And so security
(18:36):
got better and crowdstreg is a very popular solution, and
there are other solutions in the pipe for phishing detection,
which is notoriously hard pattern matching problem. And that's just
two examples of the way security can be enhanced by
offloading to an AI specific accelerator.
Speaker 1 (18:56):
Yeah, and we talked a little bit about you know,
businesses and enterprises adopting these new aipcs. Is anything special
that IT departments and organizations need to do to prepare
for this.
Speaker 2 (19:09):
I'll say there's probably three things that an ITDM would
want to think about if they intend to use large
language models or just generative AI. Those workloads are pretty
sensitive to memory bandwidth, so you wouldn't normally think about
memory bandwidth in a system purchase, but making sure that
(19:30):
it has two memory sticks over one. For example, right,
if you want to get sixteen gigs a RAM, make
sure that's two by eight instead of one by sixteen,
Just as an example, that will dramatically improve the performance
of the LLM, because these language models are fundamentally limited
or enhanced by the performance of the memory subsystem. Outside
(19:52):
of that, for other forms of AI in the year ahead,
there is a calculation called TOPS or terra operations per second,
which is sort of a ballpark for how much AI
performance a device can give you. Software makes or breaks
the acquisition of that TOPS figure. So I can give
(20:16):
you a billion TOPS, and if I had a very
poor software stack under that you would never see the
billion tops. So there's a new level of knowledge the
ITDM needs to develop on sort of software stack robustness
underneath that rating. Okay, so you have to be familiar
(20:37):
with what Intel's doing in the software space, what its
competitors are doing in the software space to really understand
whether or not you're going to get a good experience
out of the device you're purchasing. So that's number two,
and number three I think would be to say that
itdms will probably be faced with a lot of advocacy
for the NPU as an AIX, but it's important to
(21:01):
understand that the software industry broadly also wants to use
GPU and CPU as well. So if you make an
upgrade decision that is all in on NPU performance and
you didn't check on the GPU or the CPU, you
may be out in the cold on performance or power
(21:22):
efficiency for these Frankly a large number of workloads that
use graphics and CPU for AI acceleration. Those are the
three things that I would say are new or different
in this era, But overall it is something that you
can integrate into your upgrade cycle piecemeal. YEP, Newer devices
(21:43):
will be faster. I mean, they always are, but it's
not like you're missing out on a new feature, right,
We're going to make them faster, But the features are
delivered by the software, are not the hardware. You can
kind of get in at any point and get the
goodness of AI, which is pretty cool as well.
Speaker 1 (21:59):
Yeah, yeah, And are you helping the software vendors, I
guess compile their code so that it will help them
really utilize that hardware underneath.
Speaker 2 (22:10):
Yeah, that's the secret of AI. And I'll start with
an analogy of PC gaming, which I think is a
lot more familiar to people. So at the bottom, you
have a piece of hardware, a graphics card, and they
tell you, you know, it's certain terra flops or gigaflops of performance,
but everybody knows that really depends on how well optimized
(22:32):
the game engine is, how well optimized the game you're
running is, how good the graphics drivers are. AI is
no different. AI accelerators are actually exposed in direct X
in Windows as a GPU without display outputs, so the
system sees them as essentially graphics cards, and instead of
(22:57):
game engines you have AI models will have features and
apps just like games you even have run times or
an environment where the code is running in there's a
DirectX run time for graphics, there is equivalent run times
for AI. So in many respects, the AI software stack
looks and works a lot like the gaming software stack.
(23:21):
And so if people think about all the times that
a GPU that's supposed to be faster on paper didn't
live up to that number because of one software reason
or another, that is a possible reality for AI as well.
And that's why it's so important to make your decision
(23:41):
not just on tops or what the accelerator is, but
on the robustness of the software underneath, because that's where
it really happens.
Speaker 1 (23:52):
Regular listeners to the show Mike remember back in season one,
we divided a whole episode on the skills that workers
will need in order to take advantage of the kinds
of AI tools we discussed today. AI technology can only
expand our expectations of what's possible if we understand the
most effective ways to use it. So I asked Robert
(24:12):
for his thoughts and what workers should focus on to
take advantage of this new wave of technology, and he
began his answer with something just about everyone does on
the Internet, every day.
Speaker 2 (24:23):
Ooh, that's a good one. I actually want to start
briefly at web searching. You know, web searching, a good,
well composed query is a learned skill. We have all
encountered people that haven't quite learned that skill, and they
get bad search results and they're frustrated with a search engine.
And not a lot of people teach that skill because
(24:44):
it's its own language of sorts. You know, you want
to freeze things to a search box differently than how
I would ask it out loud. So that takes me
to AI, where you are still engineering a search of sorts.
It's called a prompt now, but it's a search and
(25:04):
the skill is in like, how do you craft that
prompt to get the result that you're looking for? You
have to be somewhat specific, and you have to understand
the limitations of the AI model you're working with. So
let's take an image creation model. Some of them only
support what's called positive prompts. You have to describe the
(25:26):
things that you want and it will kind of omit
anything else that you're not asking for explicitly. Others support
negative prompts, and if you're working with a positive only
image service, you could easily have users saying I want this,
this and this, but not that, that and that. Yeah,
the not operator the AI engine's not going to understand it,
(25:49):
so it's going to give you the things in the
not category. In the picture.
Speaker 1 (25:54):
Yeah, it's like saying I want a picture of a zoo,
but no elephants.
Speaker 2 (25:58):
Right, and it doesn't nderstand the word no, So in.
Speaker 1 (26:01):
A positive one, it'll just have elephants.
Speaker 2 (26:03):
Yeah, you get all elephants exactly right. So understanding this
is a real skill. Understanding like what the AI model
or by distant extension, what the game engine will let
you do is really important, and how to be specific
and understanding how to provide follow up commentary to tweak
(26:26):
the result to get what you're looking for.
Speaker 1 (26:28):
This is a.
Speaker 2 (26:28):
Whole category of skill that is not taught, not widely
known in business or even amongst users because this is
so new. But being familiar with how prompts work is
going to be a key business skill, and I think
there'll be a real advantage for employees who know how
to engineer a good prompt to get a great result quickly.
Speaker 1 (26:51):
So going to take a bit of a step back
and look far in the horizon in the sort of
the three to five seven year time range. What's Intel's
plan for this journey into AI.
Speaker 2 (27:06):
I think at top level, i'd want to acknowledge that
I understand the skepticism or the I don't understand of AI.
But where we're going is like a complete industry transformation.
Intel believes by twenty twenty seven or twenty twenty eight
that eighty percent of all the computer sold will have
AI accelerators inside. And by the end of this year,
(27:30):
we want at least one hundred different software developers partnered
with Intel. We want to deliver three hundred or so
different AI features into the marketplace through all of those companies,
help them optimize it, deliver it, market it. We want
to bring one hundred million accelerators into the space by
the end of twenty twenty five, and so just between
(27:53):
now and twenty twenty seven twenty eight, we're talking about
zero to eighty percent of the market in under four years,
which is an extraordinary velocity. And as of right now,
Intel has the largest number of accelerators, the most number
of applications in the market, and sort of below the scenes,
(28:13):
all the enabling tools and softwares and frameworks, we also
have the most of those as well, So we want
to be the scale provider for AI, you know, the
biggest install based and we want to help to succeed
because that's what software vendors are asking us to do.
So that's the longest short of it.
Speaker 1 (28:30):
That's great. I think I'll leave it there. Thanks Robert.
Thanks my deepest thanks to Robert Hallock for his invaluable
contributions to today's episode of Technically Speaking. Robert's passion and
enthusiasm has convinced me that we are on the brink
of a revolutionary technology that could transform humanity. We've all
(28:51):
used PCs both in our personal and professional lives, but
the leap to aipowered processes represents a once in a
generation advancement. Imagine having your own personal AIS system that
continually adapts to your needs and priorities. Admittedly, this is
a prospect I'm still trying to fully grasp, but I
especially appreciate that this technology will operate locally on my PC,
(29:15):
keeping my data private and secure. I'm also particularly interested
in how technology can empower the underdog to do amazing things.
The garage based tech entrepreneur building the next world beating
app the corner cafe owner getting that extra hour per
day to spend with their family, and the first year
design student creating beautiful and innovative art. With AI enhanced
(29:37):
pieces readily accessible, I believe we can greatly advance human prosperity.
The future looks bright. Ground. Next episode will continue to
unpact the involvement of AI to create a more accessible
and livable city for everyone. Join us again on Tuesday,
April twenty third for another enlightening discussion here on Technically
(29:59):
Speaking and Intel podcast. Technically Speaking was produced by Ruby
Studio from iHeartRadio in partnership with Intel and hosted by
me Graham Class. Our executive producer is Molly Sosher, our
EP of Post Production is James Foster, and our supervising
producer is Nikia Swinton. This episode was edited by Sierra
(30:22):
Spreen and was written by Molly Sosher and Nick Ferschall.
Host
Graeme Klass
Popular Podcasts
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy And Charlamagne Tha God!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Decisions, Decisions
Welcome to "Decisions, Decisions," the podcast where boundaries are pushed, and conversations get candid! Join your favorite hosts, Mandii B and WeezyWTF, as they dive deep into the world of non-traditional relationships and explore the often-taboo topics surrounding dating, sex, and love. Every Monday, Mandii and Weezy invite you to unlearn the outdated narratives dictated by traditional patriarchal norms. With a blend of humor, vulnerability, and authenticity, they share their personal journeys navigating their 30s, tackling the complexities of modern relationships, and engaging in thought-provoking discussions that challenge societal expectations. From groundbreaking interviews with diverse guests to relatable stories that resonate with your experiences, "Decisions, Decisions" is your go-to source for open dialogue about what it truly means to love and connect in today's world. Get ready to reshape your understanding of relationships and embrace the freedom of authentic connections—tune in and join the conversation!