January 23, 2025 • 48 mins
Unlock the secrets of video streaming technology with our special guest, Romain Bouqueau, a key contributor to the GPAC framework and CEO of MotionSpell. We promise that by tuning in, you'll gain a deep understanding of multi-threaded performance challenges in FFmpeg, especially when pushing the limits with complex tasks like 4K HEVC encoding. Romain brings us behind the scenes, explaining how threading and scheduling come into play, and why issues like input-output management and memory can make or break your streaming success. Together, we also explore how giants like YouTube and Vimeo optimize these technologies to deliver seamless streaming experiences.
Our journey doesn't stop there. We dissect the strengths of FFmpeg, GStreamer, and GPAC, focusing on codec performance and their unique roles in the streaming ecosystem. From the efficiency of x264 and x265 codecs to the power-saving potential of Gravitons, learn why companies like MulticoreWare are at the forefront of performance enhancements. We also share insights into the evolving world of FFmpeg's infrastructure and GStreamer's superior handling of bottlenecks, shedding light on the importance of choosing the right tools for your streaming needs without the hassle of constant compilation for different CPUs.
Finally, we delve into media packaging standards such as HLS, DASH, and the emerging CMAF format. The discussion underscores the critical role of ISO BMFF and highlights tools like GPAC's Compliance Warden in maintaining interoperability. We tackle the risks of using under-maintained open-source projects like Bento4 and explore the supportive communities that provide commercial support, ensuring you stay ahead in the fast-paced world of multimedia technology. With an eye on the future, we touch on advancements like VVC support and consider the potential for user-friendly interfaces to simplify complex processes. Join us for an episode packed with insights and expert guidance on navigating the ever-evolving landscape of video streaming.
Stay tuned for more in-depth insights on video technology, trends, and practical applications. Subscribe to Voices of Video: Inside the Tech for exclusive, hands-on knowledge from the experts. For more resources, visit Voices of Video.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:07):
Voices of Video.
Voices of Video.
Voices of Video.
Voices of Video.
Speaker 2 (00:17):
Welcome to NetEnt's Voices in Video, where we
explore criticalstreaming-related topics with
the experts who create andimplement new technology-related
solutions.
If you're watching and havequestions, please post them as a
comment.
On whichever platform you'rewatching on, we'll answer live
if time permits.
Today's episode is all aboutencoding and packaging, and
we'll be discussing the GPACopen-source technology framework
(00:41):
recently licensed by Netflixfor their live streams.
Specifically, I'll speak withRomain Bocou on his birthday,
and he's a main contributor toGPAC and the founder and CEO of
MotionSpell, the licensing armof GPAC.
And the reason we're talkingwith Romain today relates back
to an experience we had withFFVMPEG.
(01:02):
Here at NetEnt, we started doingsome testing with some of our
cards and the cards performedwell up to a point.
When the FFmpeg operationsbecame more and more complex, we
saw that the throughput droppedsignificantly, even though
capacity in the cards andcapacity in the CPU was fine.
So we started researchingalternative solutions.
(01:23):
We found GStreamer and we alsofound GPAC.
Romain was kind enough to joinus today and talk about these
and many other applications ofboth programs and just
describing the technologies,where they fit, what they
perform and how to considerwhich ones to use for your
application.
So let's start there, romain,thanks for joining us.
Speaker 1 (01:45):
Thank you, John.
Speaker 2 (01:49):
So why don't we just jump into the FFmpeg issue?
What's going on withmulti-threaded performance with
FFmpeg?
What are the symptoms, what arethe causes and how do we work
around it?
And take it a bite at a time,don't try and jump into all that
.
Speaker 1 (02:00):
So we're going to talk a lot about multi-threads,
multi-threading.
So first, what is a thread?
So a thread is a sequence ofinstructions, things that you
want to do sequentially, so youkind of do them in parallel.
If you want to do things inparallel because you have many
cores on your computers, forexample, so your computer is
(02:22):
able to compute in parallel,then you need to have several
threads, and these threads mostof the time are not all the time
, but most of the time are saidkernel threads.
It's an object that isunderstood by the kernel of your
operating system, that'shandled by a component that is
called the scheduler.
(02:43):
So the scheduler analyzes whichthreads are running, which
threads are eligible forcomputing, and it handles all of
that.
More specifically, on FFmpeg,for your question, ffmpeg is a
multi-threaded application andso there are several levels
where it's multi-threaded.
(03:03):
There can be like a codec thatcan be multi-threaded when you
execute, for example, x364 orx365.
As a library inside FFmpeg,they create their own thread,
but also FFmpeg creates its ownthread, the application FFmpeg.
Speaker 2 (03:20):
So what was I experiencing when we saw a drop
in throughput?
So what was I experiencing whenwe saw a drop in throughput
with and let me just you knowone of the typically the
operation was large file in, say4K, and then a pretty complex
encoding letter out, say five orsix rungs, and testing HEVC,
because it's 4K output and youknow H.264 doesn't make sense.
(03:41):
What's happening in a?
You know it's a pretty simplecommand line, right, we do it
all the time, but you knowwhat's happening that causes
throughput to drop.
Speaker 1 (03:49):
It's not an easy question.
So first, I think there's beenan effort in FFmpeg to try to
bring more threads at theapplication level.
So that's what I was saying.
There were not so many threads,I think, in FFmpeg-5,.
One of the big changes is thatnow, when you have multiple
outputs, each of these outputsrun in a separate thread, which
(04:14):
means that, for example, if youhave two outputs connected to
two different servers, there isa problem on one of the outputs.
The second one is not hung, sothey can run in parallel.
When it comes to 4K, it's areally huge question.
I think there are many factorsthat affect performances when
(04:36):
you're trying to reach thelimits of computers.
There are threadings, of course.
If you're not threaded enough,or if you have too many threads
or so maybe that's somethingthat we may want to talk about,
then you're not going to run atyour maximum speed.
But there is also IOs, right?
(04:56):
If you don't deal with IOscorrectly, then you can be
limited by your IOs.
So the input-output, and thereis also some memory issues, for
example.
So we're going to compare maybewith the other frameworks, but
in FFmpeg the queues are reallylarge.
If you queue a high number ofdecoded frames that are pretty
(05:19):
large by themselves, then it cantake several gigabytes of
memory.
So RAM is cheap.
But if you don't have enoughRAM, then it's take several
gigabytes of memory.
So RAM is cheap, but if youdon't have enough RAM, then it's
going to write to the disk andthen it's going to be super slow
.
Speaker 2 (05:32):
Interesting.
How do you mitigate that?
And if you're working with acommand string as opposed to, I
guess more programmatically, isthere any way you can mitigate
against that?
Speaker 1 (05:42):
So FFmpeg is two parts, I think, for most of the
people.
There are some libraries likelibavcodex for the codex,
libavformat for the demuxers,muxers, network protocols and a
bunch of others, and then thereis the FFmpeg application.
There are other applications inFFmpeg.
There is FFplay, which is asimple player.
(06:04):
There is FFprobe to inspectyour content, and the efforts
that I was talking about are onthe FFmpeg part, which means
that other people were able toleverage some of the benefits
that the FFmpeg developers aretrying to implement, because
they directly integrated theFFmpeg libraries.
(06:27):
So, when you consider bigcompanies like Vimeo, youtube or
other projects like GPAC, weintegrate the libraries and we
could investigate, with someimprovements on the performances
.
Ffmpeg is really great for thisbecause you can plug at many
points, like for the IOs or forthe memory allocators or for the
(06:48):
threading, and you can do yourown stuff.
Not everything is solved on theFFmpeg site but, as I said,
they've been working on it, soexciting.
Speaker 2 (06:55):
Version 6 came out right in the middle of these
problems and I basically justused the same command lines and
tried it on version six and Ifound no difference.
So you know, I thought that oneof the one of the goals of
version six was bettermulti-threaded performance.
What happened?
Am I just you know what changedwith between five and six and
(07:18):
why didn't I see any performancedifference between those two
versions?
With these type of of ofencoding projects, Okay, yeah,
that's a bit technical.
Speaker 1 (07:28):
In version 5, as I said, they have separated the
different outputs, so now youcan run several outputs in
parallel and if there was aproblem on one, then it's not
going to create problems on theother output.
In version 6, it's acontinuation of this and they've
been working on this for atleast a year.
(07:50):
I could find comments relatedto better threading in FFmpeg,
and at the beginning of the year, the FFmpeg developers at the
FOSDEM Open Source Conferencesaid that the effort would last
for the whole year, so probablyyou will need to test with
FFmpeg-7 next year.
Speaker 2 (08:08):
You talked a moment ago about too many cores.
Go into that and then maybedescribe how machines, how you
should configure an encodingmachine better for 4K as
compared to, say, 1080p-typeprojects.
Speaker 1 (08:23):
There are maybe many factors on this.
If you put too many threads,the problem is that in your
operating system, in the kernel,in the scheduler that we talked
about, then the scheduler saysit's called preemptive.
It allocates you with slices oftime, typically tens of
milliseconds, and each tens ofmilliseconds is going to look if
(08:46):
there are some new eligiblestasks so that everybody can have
some time to execute.
If you put too many threads,there are two kind of problems.
The first one is that youswitch all the time.
It's called a context switch.
The price is pretty low, buteach time you have to save your
registers, you have to save abunch of stuff and switch to the
(09:11):
other threads, so that you know, instead of spending time doing
some computation, you'respending time switching tasks.
And the second problem is thatwe're most of the time we are
CPU bound and so we rely heavilyon caches.
And when you change coressometimes you break the cache
(09:33):
and so again you have aperformance penalty.
But this is something that isreally really difficult to
measure.
Speaker 2 (09:39):
Can you give any advice at all?
I mean, if I'm doing, you know,say with, let's stick with
software because you know that'sthe most generic case.
But if I'm doing, you know, saywith, let's stick with software
because you know that's themost generic case.
But if I'm encoding 4k videos,is there a benefit to going to
128k or 128 cores, or should I?
You know, should I stick with32 or does it?
Does it matter?
Or you know, more jobs, lessjobs.
(10:00):
What's your sense of the bestmachine for high volume 4K
transcoding Again, vod, it's alot.
Speaker 1 (10:09):
I think the number of threads on your computer should
be a couple of times the numberof physical cores that are
available on your CPU, and ifyou make some graphics related
to this, you're going to seethat it scales almost linearly
up to a certain point.
That being said, it alsodepends on the application.
(10:30):
As I mentioned, there are, youknow, codecs can create in
FFmpeg.
Ffmpeg the application createsits own codecs, but then the
application may create their owncodecs.
You have codec specific options, for example for x264 or x265
or AVC, hevc encoding or anyencoder where you can try to
(10:52):
tune.
So most of the time the encoderthemselves provide an auto,
automatic parameter so that ittries to scale by itself,
because only the applicationknows if it scales better with
twice the number of cores forthe number of threads, or just
(11:13):
the number of cores plus acertain number because they use
this for scheduling or whatever.
And again, as I said, it's notonly CPU cores.
If you saturate your bandwidthfor the memory, if you saturate
your IOs, for example becauseyou're reading from the network,
or you're reading from a diskthat is not fast enough because
you launched several tasks atthe same time, then it can be a
(11:35):
problem.
That's really.
Really measuring performancesis something that is really not
easy and sometimes with smallchanges, like going from you,
going from FFmpeg, theapplication, to another
framework that uses FFmpeg aslibraries, you can see big
differences.
Speaker 2 (11:53):
Just kind of an aside .
I recently did some workanalyzing the CPUs that were
available on AWS, so I looked atIntel versus Graviton versus
AMD.
What's your sense of you know?
What I found was that Gravitonwas best for X.264 and AMD was
best for X.265.
Are those general conclusionsyou can reach, or do you have
(12:15):
different thoughts as to whichCPUs do the best job with those
codecs?
Speaker 1 (12:19):
I can say that we use a lot and we advocate for, you
know, gravitons and equivalentsbecause they are more power
efficient and also that'sbecause the FFmpeg developers
developing the codecs they'vemade for years a tremendous job
in implementing all the assemblyacceleration that were needed.
Speaker 2 (12:43):
I found that true with X.264, but X.265 was very
bad with Graviton and that wasincluding a very new version
from multicore wear thatactually improved performance by
300 or 400%.
So any comment there?
Speaker 1 (13:01):
Very different projects, right?
X.264 is really community-basedpassionate people and X.264 is
really community-basedpassionate people and x265 is
maintained by multi-coreware.
And even though there ispassion in companies, especially
on video codecs, you can havereally interesting discussions
with them, it's not the sameright You're in front.
(13:24):
X264 was made by people thatwere genius, I would say even
crazy at some point.
So for me, totally differentperspective.
Speaker 2 (13:32):
What about compiling different versions of ffmpeg for
the different CPUs?
Do you need to take it to thatlevel, or is that not necessary?
Speaker 1 (13:41):
I don't think that's necessary, but yeah, you could
do this.
We see not necessary.
I don't think that's necessary,but yeah, you could do this.
We see.
You know you always haveproblems with operations, like
operational problems.
You can come into a situationwhere you'd need to have several
builds or using severalversions even of a specific tool
.
That's a possibility, yes.
Speaker 2 (14:01):
So let's transition over to JPEG and let me kind of
set it up.
I mean, I know FFmpeg.
You know it's really good onthe encoding, transcoding side,
a little bit less good on thepackaging side.
We use GStreamer as kind of aworkaround for some of the
bottlenecks that we you knowthat I described a few minutes
ago.
That work pretty well.
And then you know everybody.
(14:23):
You know, I've known you foryears and known about gpac for
years, but you really came toprominence when netflix licensed
the technology for their livestreams.
Give me, give me an overview ofyou know, just really high
level, about where you see thosethree projects, including, you
know, ffmpeg, gstreamer andgpacpack, what they do, how they
(14:44):
fit in together, what they'regood at, what they're not so
good at.
Speaker 1 (14:48):
FFmpeg.
As I said, they are thelibraries and these people are
implementing codecs protocols.
It's a really nice toolbox thatis really, really complete.
There has been a huge effort onit.
Really successful project, Iwould say, critical
infrastructure for the industrytoday.
(15:08):
And then there is the FFmpegapplication that people use.
People like it because theinterface is quite stable.
You can reuse a lot of yourcommon lines from years ago and
they still work, your commandlines from years ago and they
still work, which is not thecase at the library level where
(15:29):
you have to implement the API,which is the interface.
At the coding level, it changesfrom time to time, so that's
more work to go.
On the libraries.
You can get some of thebenefits.
For most people you can use theFFmpeg command line libraries,
gpack, so we are alsoimplementing some standards and
(15:54):
some streaming protocols.
So we're really experts on thisDemuxing, muxing, packaging,
encrypting, streaming.
We are active atstandardization, so we have a
lot of R&D.
That's the difference, forexample, with FFmpeg and inside
GPAC.
We have our own code and weintegrate it with other
(16:14):
libraries, and FFmpeg is a bigGPAC.
We have our own code and weintegrate it with other
libraries and FFmpeg is a bigone.
For us.
It's a big dependency, and weleverage a lot of FFmpeg.
For our customers, our users ingeneral, gstreamer is more like
an aggregator.
There is no real code beingimplemented in FFmpeg, even
though it has changed over time,but its ambition is to
(16:38):
aggregate everything into thesekind of what they call elements,
which are kind of modules thatyou connect together in a big
graph.
So GStreamer is really good atthis.
You mentioned that GStreamercould handle some of the
bottlenecks that you had withFFmpeg.
The framework is really morecomplex and, for example, we
(17:00):
were talking about saturatingthe memory.
Gstreamer has nice mechanismsfor this.
It pre-allocates the buffers.
It never allocates on the fly,which is also something that
FFmpeg or GPAC do.
But with GStreamer, when youhave a queue, a buffer, when you
are high in a buffer or low ina buffer, it's going to emit a
(17:22):
message, and so I was talkingabout the scheduler.
This is something that isimportant At the GStreamer level
.
They can decide to execute orto not execute some of their
elements just because the bufferlevel is high or not high, and
so you avoid saturating, forexample, your memory usage or
(17:43):
whatever, because when a bufferis full, there is nobody that's
able to push data to you, forexample.
So that's a really preciseexample of where a GStreamer may
have better performances.
A GPAC is also quite flexibleand we have all kinds of I think
intelligent mechanisms to tryto handle performances in the
(18:04):
best way.
But again, you know it's magicby default.
There is a lot of automationand if you don't like it you can
disable it, and there is a lotof flags to try to control your
performances, depending on.
You know the whole environment.
Jpeg may not be aware of.
It's not like when you runthese tools.
There is no like uh, somethingthat is benchmarking your system
(18:26):
and then says, okay, the bestflags for you are this or this.
It's just saying how many cpucores do I have or how much
memory do I have, and it doesn'tconsider the fact that you're
running a browser at the sametime.
You know using already a lot ofmemory or you know other
considerations.
Speaker 2 (18:43):
Let's look at the individual, I guess, components
of what I want from the encoderpackager.
I mean it feels like again,ffmpeg is very strong in the
encode side and it's very usable, very accessible.
Pretty weak on the packagingside.
You know, and I guess you knowFFmpeg is over here.
I don't know where GStreamer ison packaging.
I know that you're prettysophisticated there.
(19:04):
I know there are projects likeBento 4 that are primarily
packaging.
So you know, just trying tounderstand that, and I think
you've given us a good start,what can you tell us about
GStreamer's packagingcapabilities?
Speaker 1 (19:19):
As I said, they write little code, so basically
they're more of an aggregator.
So you know they don't have aGPAC integration, for example,
and most of their packagingrelies on FFmpeg.
So mostly they have thepackaging capability of FFmpeg,
with a few other components onWebRTC et cetera, where they
(19:41):
wrote their own code.
But otherwise I thinkG-Streamer has some particular
markets.
It's more on the hardware side,which is why I'm not surprised
that a company like NetInt wouldtry to use it.
But when it comes to puresoftware workflows I think
(20:02):
FFmpeg is way, way more popularand it's been years that we've
tried so historically, GPAC ismore about file-based packaging
with MP4Box.
So MP4Box is the main tool ofJPEG.
But then people had to encodetheir content with FFmpeg, dump
it and package it.
(20:22):
I think it was okay a couple ofyears ago, where people wouldn't
trust open-source software todo some live.
But now things have changed.
So it's really difficult to beintegrated as a third party to
FFmpeg.
I hope that we can change thatin the near future as near as
possible for me, because FFmpegis super popular.
(20:45):
But we decided to do thereverse.
We said, OK, we can buildsomething that we think is
leveraging FFmpeg and which isbetter than FFmpeg, and so we
have a new application in GPAC,which is simply called GPAC,
like the FFmpeg application inFFmpeg.
That allows you to buildcomplex workflows and it's used
(21:05):
for live by many people.
We have a lot of users on thisand, as I said, because we're
doing some standardization and alot of R&D in JPEG, it's a mix
of a lot of things.
It's a player, it's an analyzer, it's a content generator so
(21:26):
you can encode, you can package,you can stream, you can dump,
we embed an HTTP server, so itdoes a lot of things.
Speaker 2 (21:34):
Taking a step in a different direction for a moment
.
What are you seeing on thepackaging side regarding
utilization?
I mean, we've heard about CMAPfor years, we know Dash, we know
HLS.
What are you seeing as trendsin packaging utilization among
the people that you're workingwith?
Speaker 1 (21:52):
I said that we were doing standardization.
We are advocating for ISO BMFF.
So that's the MPEG-4 fileformat.
And so, because we are instandardization, because it's
pretty active, because they havethis open source software like
GPAC, when it comes to openstandardization, mpeg systems
(22:12):
and the MPEG file format arereally really, really powerful.
Iso BMFF is everywhere.
The QuickTime file format onthe prediction side, cmap,
fragmented MP4, mp4, everythingthat's related also to
encryption with commonencryption dash, hls that's
fragmented MP4, cmap For storage, there is MP4.
(22:37):
I think there's a huge consensus, contrary to what happens in
codecs right now, where thesituation is pretty fragmented.
That's the same for inputcontribution protocols, where
people still use RTMP.
They use also SRT, but theydon't know exactly where to move
.
When it comes to file formats,it's pretty stable.
Iso BMFF is everywhere and Ibelieve that JPEG is the best
(23:00):
tool.
But again, you can try it, it'sopen source.
There is a lot of nice featuresI think that people are not so
aware of.
If you take a raw H.264 file,for example, or H.265 file that
you dump from your encoder, ifyou package it in MP4, the MP4
(23:21):
is going to be smaller than yourinput because we eliminate all
the redundancy and, in addition,you can get some indexation of
the content and you can have alot of nice features that come
with.
You know packaging your content.
Speaker 2 (23:39):
Well, how does that translate to HLS, dash or CMAP?
I mean, does that mean dash?
Or you know it feels like HLSis still the leader, you know,
years after standards-basedapproaches became available.
Are you saying that's not true?
Or I don't remember whatBitmoven said in their developer
survey?
I haven't looked at that in awhile.
But people, they have to outputone of those packaging
(24:03):
standards.
What are you saying in terms ofsupport?
Speaker 1 (24:06):
HLS has dominated for years.
It's going to continue todominate, but that's the truth.
But it's not important, right?
The way you handle yourmanifest is just a really tiny
part compared to what you'redoing with the rest of the data,
which is the majority ofeverything.
And then, there again, there isa convergence on CMAP, which is
(24:26):
ISO BMFF based.
So I think we're pretty closenow that there is a convergence
also on the encryption, becauseyou know that you know
everything that was, you knowindustry-based was aes, ctr and
apple said okay, but cb, cbc andcbcs in particular are.
You know where we want to headto.
(24:47):
So there is a consensus rightnow to go to cmap and for the
encryption, cb.
So I think the convergence ishere.
We are really near the pointwhere you would have only one
media format to distribute.
Speaker 2 (25:00):
And that is ISO BMFF with different packaging.
And that's the whole promise ofCMAP is.
You know, the manifest is sucha small portion of it, it really
doesn't matter.
Speaker 1 (25:10):
You know it's messy, because when you, you know, when
you consider so I say ISO BMFF,which is maybe a technical term
for most of the audience that'sthe base standard that is
edited at MPEG and you canimagine that CMAP is a profile
of it.
So it's going to restrict thetoolbox.
I would have liked it to berestricted to a point where you
(25:31):
say, okay, I need to packagethis and you know exactly what's
going to be out, like we havefor, you know, bit accuracy,
like we have for codecs when youtake an encoding stream, you
know exactly where you're goingto have as an output.
It's not that true anymore.
There are, you know, a fewdiscussions on that, but
basically that's what you have,but still with CMath, not
(25:54):
restricted anymore.
So there is a lot of freedom.
And when there is freedom, ofcourse interoperability is much,
much more difficult.
But again, in GPAC, separatefrom the main GPAC repository,
we have a repository calledCompliance Warden and the
Compliance Warden is all aboutcompliance on ISO BMFF and maybe
(26:15):
one day we'll get some sponsorsto have the full ISO BMFF and
CMAP stack so that people canactually have some better
compliance on this.
Speaker 2 (26:29):
So let's take a deeper look at JPEG you said
it's open source, you said it'scommand line.
A deeper look at gpac you saidit's open source, you said it's
command line.
Um codex that it handles.
You know pretty much thestandard.
You know h.264, hbc, av1.
Any limitations there?
Speaker 1 (26:42):
a lot more.
You can mix.
You can mix everything, uh, allcodecs.
You know you could dash anycodec.
You can dash also masrovska,webm, you know whatever.
You can put special flags to beinteroperable with basically
everything.
That's the chance we have.
We have pretty big users, notonly Netflix, and it gives us a
(27:03):
wide, wide range of applicationand interoperability.
I think that a lot of people,because we're doing R&D, for
example, vvc we've had someintegration with VVC for three
years.
So basically, when you want topackage VVC, there is a high
chance that you come to GPAC andyou try GPAC.
So it's not a real strategyfrom us to try to attract users,
(27:29):
but this is something that ispretty effective compared to
other packages.
You were mentioning Bento4.
So Bento4 has only onemaintainer who doesn't work, who
hasn't worked in packaging forlike six or seven years.
So for me, there is a lot of,you know, packaging projects
that are really good, used bythe industry, but there is a
(27:49):
real, real risk when you haveonly one maintainer and the
project is not activelymaintained anymore.
So that's why you know we weretalking about that there are
different strategies, I think inopen source and projects like
FFmpeg or GPAC you know theycome from a real pain point,
from users, passionate peoplewho you know that's my case.
(28:13):
That's my case I was.
I needed this solution.
I started to contribute to GPACand then I made a company
because I wanted, because I'mpassionate I think people can
see that I'm passionate about itI tried to find a way to make
it a living and also with GPAClicensing you know, the
commercial arm of GPAC insideMotionSpell to allow other
(28:33):
people also to be able to liveoff their contributions.
Okay, ffmpeg, they have thesame right.
They have FF Labs now they had abooth at NAB and for G-Swimmer
they were these two companies,fluendo and then Collabora.
I think that's really importantto give the industry confidence
that they are real peoplesupporting the project and
understanding their needs.
It's really important to givethe industry confidence that you
(28:53):
know they are real people, youknow, supporting the project and
understanding their needs.
It's not a bunch of teenagerkids, you know, in their garage,
because sometimes that's thefeeling that you can have If you
go, for example, on the FFFBugtracker.
You know you can be, you know,have a wrong image, but still
they are trying to buildsomething for the industry.
Speaker 2 (29:15):
I mean, they're tremendously talented
programmers.
What about?
So, looking back at JPEG, whatabout HDR?
I mean, you know Dolby supportshuge.
Hdr 10 plus is huge.
What do you?
What are your capabilitiesthere?
Speaker 1 (29:27):
We support everything .
We have a very generic supportand the latest release, that was
last December, had a specialfocus on HDR.
So we support everything static, dynamic metadata, all kinds of
metadata.
You can re-inject your ownmetadata.
You can fix the metadata alongthe stream because sometimes
(29:47):
there is a lot of streams thatare just broken.
Let's frame it this way.
So we support everything.
And, again, we have a reallydeep integration with FFmpegs
internal.
That allows us to take, likethe row, you know, the matrix
coefficients and the color spaceetc.
And to map it into somethingthat would be recognized as
(30:09):
Dolby Vision or HDR10 or HDR10plus, etc.
Where does the Dolby licensingcome in?
Or HDR10+, et cetera.
Speaker 2 (30:15):
Where does the Dolby licensing come in?
And I guess, taking a step back, can I do Dolby Vision with
FFmpeg.
I know I can do HDR10+, but canI do Dolby with FFmpeg?
And what about the licensing?
Is that just device decode sideor is that on the encode side
as well?
Speaker 1 (30:34):
I don't know.
I know they support it on theencode side.
They support the metadata.
Like you know, they have somespecial side data that comes
with the packets of the data andthey support it.
That's all I can say.
I don't know more about this.
But that's sufficient for GPACto package it.
Speaker 2 (30:52):
You know, one of the things that's kind of been
interesting to me is the compareand contrast for commercial
software like you can get from anumber of vendors.
You know Harmonic comes to mind.
You know what are the pros andcons of open source versus, you
know, buying a commercialversion.
That's going to have a prettyextensive UI, it's going to have
a lot of testing, it's going tohave people you can call and
(31:13):
get support from pretty easily.
What's your sense of thatdecision?
Speaker 1 (31:17):
Well, I think there are several reasons why you
would want to do that.
I had the chance to talk at anAB streaming summit with Netflix
and they explained that they doit first because it's open
standards.
They don't see how they couldcompete on this.
It's not their goal now.
They are pushing contributionsthere.
(31:38):
They want to compete on otherthings, not these open standards
.
And I think Netflix made ademonstration when it migrated
to JPEG.
They moved super fast, like yousee Bandersnatch.
You see evolving a newer, newgeneration audio codecs.
They announced an ad-based tier.
(32:03):
They moved to live.
I don't know any company ableto move that fast.
So I'm not saying it's 100%because they are using open
source or whatever.
But clearly that's points wherepackaging is crucial, right?
How do you make sure, if you'redoing interactive, how do you
make sure that you can stitchyour content?
Do you know any commercial toolthat says, okay, I can stitch
(32:25):
and create scenarios on this.
If you don't control exactlywhat a packager does, you cannot
do this.
So, and again, when there isnew codecs, when there are other
things using an open sourcetool, you know you can leverage
the work that has already beendone by the community.
So I think that's important.
But, that being said, netflixis a pretty innovative companies
(32:49):
, so we have the chance to havethem like, as you know, sponsors
, and also they pay us when theyneed, you know, integration of
new tools, new codecs orwhatever you know.
For us it's really easy to workwith you know such a company
and we understand that it's it'sreally easy to work with such a
company and we understand thatit's kind of easy customers.
Speaker 2 (33:08):
I mean, they've got a phenomenal engineering team and
I guess certainly a lot ofpublishers don't have the
engineering resources in-housethat Netflix does.
But let's move to Netflix.
Tell us what it is you licenseto them and what it's being used
for at Netflix.
Speaker 1 (33:28):
They gave a number.
It's like what was it?
It was a million.
I don't remember the scale, butthat's millions of content
every month that are beingpackaged.
Basically, they packageeverything.
They mentioned that, as, as Isaid, on the production side, uh
, they handle the contents, uh,or at least some of the content,
with gpac, because they don'tcontrol 100 percent of the.
(33:50):
You know the formats that thatcomes in.
Also previews for thejournalists.
Uh, you know packaging thecontent for their 200 million
viewers.
So my understanding is thatthey package everything.
But again, when it comes to openstores, the magic is that you
don't know what your users do.
We don't put telemetry in ourtools.
(34:12):
That's pure freedom.
So I don't know what Netflix isactually working on If they
don't send me an email.
Or when we work with customers,we have acceptance tests.
So that's tests, basically,that we both execute.
So that motion spell for GPACwould execute and Netflix would
execute, and we agree that thisshouldn't be broken, otherwise
(34:35):
there would be a problem.
We send them an email, or whenthey want to upgrade their
version of GPAC, they would haveto execute this list of tests
to make sure, you know, to givethem confidence.
So that's the only way for usto know how a customer actually
uses GPAC.
Speaker 2 (34:52):
So is it for live?
Is it for live NBOD, or youjust don't know?
Speaker 1 (34:56):
I don't know.
I know for VOD, for sure wewere not able to talk about live
for the NAB presentation, but Ihope that you know, if there is
some GPAC involved, we can alsoshare things.
They have a really greattechnical blog, like many of the
companies, and so it would begreat if they could communicate
on it.
But you know, I don't know,maybe this needs to mature also.
(35:19):
I don't know, maybe this needsto mature also.
I'm not into, as I said,netflix or any other customers
internally.
We're a pure technical vendorno-transcript.
(35:46):
That's something that you maywant to do to get some support.
You may want to support us.
Also, if you use us and you'resuccessful, you may want to
support us so that the projectcontinues.
It's sustainable, basically,but there are some sometimes.
The open source, the GNU LGPLlicense, is a bit tricky for
(36:07):
companies to deal with.
Like you know, you cannotforbid reverse engineering and
this type of things that youknow you look twice if you want
to accept such clauses.
So we're in a position to do alicense, gpac, and so we can
emit commercial licenses.
(36:27):
But these licenses we're notlooking specifically into
details.
What is important for us is thatpeople have a solution.
We're building really aplatform and I'm pretty sure I
have all the statistics, ofcourse I'm pretty sure GPAC is
going to be the it's.
A majority of the packaging nowis done with GPAC and so I'm
(36:49):
pretty sure, as I said, becausewe are on open standards, I'm
pretty sure that you know thereis not going to be a lot of
competition on this.
I think it's going to stabilize.
We have ISO BMFF, which is areally good technical basis, and
once the innovation slows downand I think it's going to happen
on where we are?
(37:10):
I hope so.
So there's still the question,as I said, to the contribution
protocols.
We're not on it and there is areal problem.
There is also the problem ofwhat's going to replace MPEG-TS
exactly.
So now we went into HTTPdelivery with OTT and ISO BMFF
is, you know, pretty good, butthat's not a streaming protocol,
(37:31):
right?
So there are still things Ithink to you know, figure out on
the transport level.
But you know that's the future.
I think ISO BMFF is going tostay.
It's going to stay for 20 or 30years and after that, who knows
?
Speaker 2 (37:46):
Give me the compare and contrast with Bento 4,
because it seems like youextended back into FFmpeg for
more of the transcodingoperation but that primarily
you're being used as a packager.
Is Bento 4 your most, I guess,the most pressing competitor at
this point, or the one that'smost obvious?
Speaker 1 (38:08):
I think a lot of people in the industry use Bento
4.
Some of them realize that it'snot maintained anymore and
that's a technological risk, butthat's not the only risk that
they face.
As I said, it's a problem.
You go into open source andthen the project is not
maintained anymore.
You know you're happy with theproject, but then you know there
(38:29):
is a new video codex thatappears.
Who's going to implement this?
If you have nobody tocoordinate the developments,
what happens?
So I again, that's what I think.
A lot of people like GPAC andthey go to see us because there
is an active community.
It's an active project.
We're always ahead of time.
Everything is easy, everythingis flexible, there is a
(38:54):
commercial arm.
So I think that maybe the nextmove for us is also to help
customers to migrate from Bento4to us, just because at some
point there is no solution.
There is another competitor,pretty big, in the same area.
That's Shaka Packager.
The two main maintainers arenot working on it.
There is a lot of people whocame to sales and said what do
(39:16):
we need to do?
It's a Google project, we needto de-Google-ize it and then we
need to maintain it.
And when there are new codecs,who's going to do this?
Even for big companies, therewas no inside resources.
Netflix is the proof of that.
They had their own packager andthey decided to move to open
source, and then some of thecompanies who went to the wrong
(39:37):
they couldn't know to an opensource project that isn't
maintained anymore.
Now what do they do?
Either they invest in somethingelse, like JPEG, or they need
to maintain the tool that is notmaintained anymore, or create a
community around it, orwhatever.
So I think it's also ourresponsibility to help people
(39:59):
migrate easily to us.
Speaker 2 (40:02):
Okay, I should note.
I mean I went and I'm doing atalk at Mile High next in a
couple of weeks and I looked atthe.
I mean there are some responsesto questions posed on the Bento
, for, you know, whatever theresource is for support, I don't
know if there's any newdevelopment.
There have been responseswithin the last two or three
(40:23):
weeks, so I don't know what thatmeans.
You know.
I just want to mention thatbecause I don't want to.
I don't want to.
I'm not saying it's not betterto switch over to your tool.
I'm just saying, you know,there are signs of life there
that may be different than whatit is you're expressing.
Speaker 1 (40:38):
I think people can.
There are things that you canhave a look at, like how many
open issues in a project, howmany pending pull requests.
It gives you, you know, a senseof, you know is the project
active?
Are the people you know, arethere enough people to you know,
deal with the activity of theproject or whatever.
(40:58):
I think that for people youknow comparing open source
projects, most of the project orwhatever.
I think that for people youknow comparing open source
projects, most of the timethey're comparing the stars,
which is on GitHub, when theprojects are on GitHub, which is
a good indicator of thepopularity of a project.
But then look at the openissues.
Speaker 2 (41:14):
What about?
You know I'm actually lookingat a GPAC now to integrate into
the same talk.
What are the educationalresources that you have?
You know that's going to beimportant both for new
implementers and for peopleswitching over from Shaka and
Bento 4.
You know how developed arethose capabilities on your
website.
Speaker 1 (41:31):
The website is pretty old, it's just.
It dispatches users.
That's how that's all it does.
We have a pretty good wikithat's hosted on GitHub.
So if you make searches onGitHub directly, you can find a
lot of things.
And if you read the readme inthe front page, we give a lot of
pointers to our test, our buildboat.
(41:52):
There is a doxygen fordevelopers.
But yeah, again, I think thewiki is the best entry point for
people.
And then on the common line, wehave some really good magic.
If you make gpack minus h andyou put whatever word, it's
going to output everything thatis related to this word.
And by reading the doc you willlearn that we have several
(42:16):
levels of verbosity on the help.
So, as I said, we want to makeit magic.
I think that for a lot ofpeople, you know common lines
can feel painful and thelearning curve is really, really
important.
What I would like is that in thefuture we have a graphical
interface.
I think that would be great forpeople to see.
(42:37):
You know what's happening, etcetera, because multimedia is a
lot of magic.
You know, you have H.264, it'sAnnex B and then someone has to
take care of this.
That's the same AAC, implicitsignaling, you know ADTS, latm.
That's a nightmare when youdon't look into the details.
It can be a nightmare and Ithink people would, you know,
(43:00):
educate themselves by having toyou know some, you know
graphical interface, showing tothem what is the actual work
that's being done, and Iwouldn't do that only for users,
I would do that so that youknow we can also, you know, keep
talents and, you know, make itless, you know, less complex.
I think talent is, is, is, youknow, how do you say?
(43:23):
Talent retention is somethingthat the whole industry should
look at, because there is a lotof, you know, gifted people
leaving our industry after acouple of years and we need more
of this.
Speaker 2 (43:34):
We have a question Can you describe the extent of
your VVC support and I guess,vvc?
I know Fraunhofer had an issuewith getting their VVC
implementation into FFmpeg, sowhich version are you supporting
and how does that work?
Speaker 1 (43:49):
Yeah, we're pretty up to date because we have a close
integration with the people ofOpenVVC, which is an open source
VVC decoder.
They have some integration inFFmpeg and so we're following
the latest developments of thestandard.
If that's not the case, justopen an issue on our bug tracker
(44:10):
.
But we're pretty up to date.
There is MPEG this week and sowe're going to make an update if
there are fixes to the spec oradditions.
Speaker 2 (44:19):
Another question on your inspector.
What can your inspector do andhow is it different from, say,
MediaInfo?
Speaker 1 (44:25):
Yeah, okay.
So I think MediaInfo is a greatinspector.
You take MediaInfo, the name ofthe file, you can go common
line, you can go with thegraphical interface.
It gives you immediately anoverview.
But one thing that it doesn'tdo is inspect the packets like
deep inspection of each of yourpackets, and GPAC does that.
(44:45):
We have several levels ofverbosity, and so it can display
general information, and Iwould like, as I told you that I
want, to improve that, maybegraphical interface so that
people can navigate into thecontent and then you can go deep
, deep, navigate into thecontent and then you can go deep
, deep, deep, deep, up to thepoint of every bit that's been
(45:07):
parsed.
Netflix had an anecdote aboutthis that JPEG was really,
really picky and going reallydeep in the bit streams and
actually it was reporting, orthey were able to report, errors
on some standards where fileswere already widespread.
(45:28):
And then you have what do youneed to do in this situation?
Do you need to modify thestandard or do you need to ping
the other implementation and say, hey, you should modify what
you're doing?
Speaker 2 (45:42):
One last question what is MotionSpell?
I think you mentioned that atthe start and we just got a
question in that relates to thatis what are the licensing terms
for GPAC and FFmpeg?
So what is MotionSpell and whatare the licensing terms for
GPAC and FFmpeg?
Speaker 1 (45:59):
So MotionSpell is the company that holds the
commercial arm of gpac and we'realso making products, most most
coming from the gpac technologyor related, so we have things
that are related.
Now we have an ott livesubtitle inserter for people you
know having a live ott streambut you know they need to add
(46:21):
titles on the top of it and theydon't know how to do it and, as
I said, also related tocompliance.
We're doing a lot of work withthe biggest companies in the
industry.
I think that's pretty important.
And regarding the license, thelicense of FFmpeg and GPAC are
the same, that's LGPL V2.1 plus,and my chance at MotionSpec is
(46:44):
that I went to see every of thecopyright holder in GPAC and
they signed an agreement with meand I can put a commercial
license which is basically yourterms.
We propose basic terms and youcan modify them to suit your
needs.
Speaker 2 (46:58):
Another question came in what's the status of
mp4boxjs?
Is that being maintained?
Speaker 1 (47:03):
It's not actively maintained and I think that's so
.
No, it's not true.
We're in the same situation asBento 4, right, we're merging
the pull request, but we're notactively developing it, and
there's a reason for that.
There is WebAssembly now andyou can build JPEG and run it in
your browser, so I think it'sgoing to be duplicated within a
(47:26):
couple of years.
That being said, in the middle,I think MP4Boxjs is a really,
really nice tool that's reallypopular and people use it,
coordinated with WebCodecs, andit can transform their content
in the browser.
That's really amazing.
Speaker 2 (47:41):
Pleasure seeing you at NAB, pleasure working with
you on this and I'm sure we'llbe in touch over the next year.
I'm going to need some helpwith the presentation I talked
about.
I'll be in touch and thank youfor participating today.
Speaker 1 (47:52):
Yeah, my pleasure.
Thank you, Jan.
Thank you everybody forattending, and if you have
questions, I'm always here.
Voices of Video.
Voices of Video.
Voices of Video.
Voices of Video.
Speaker 2 (00:17):
Welcome to NetEnt's Voices in Video, where we
explore criticalstreaming-related topics with
the experts who create andimplement new technology-related
solutions.
If you're watching and havequestions, please post them as a
comment.
On whichever platform you'rewatching on, we'll answer live
if time permits.
Today's episode is all aboutencoding and packaging, and
we'll be discussing the GPACopen-source technology framework
(00:41):
recently licensed by Netflixfor their live streams.
Specifically, I'll speak withRomain Bocou on his birthday,
and he's a main contributor toGPAC and the founder and CEO of
MotionSpell, the licensing armof GPAC.
And the reason we're talkingwith Romain today relates back
to an experience we had withFFVMPEG.
(01:02):
Here at NetEnt, we started doingsome testing with some of our
cards and the cards performedwell up to a point.
When the FFmpeg operationsbecame more and more complex, we
saw that the throughput droppedsignificantly, even though
capacity in the cards andcapacity in the CPU was fine.
So we started researchingalternative solutions.
(01:23):
We found GStreamer and we alsofound GPAC.
Romain was kind enough to joinus today and talk about these
and many other applications ofboth programs and just
describing the technologies,where they fit, what they
perform and how to considerwhich ones to use for your
application.
So let's start there, romain,thanks for joining us.
Speaker 1 (01:45):
Thank you, John.
Speaker 2 (01:49):
So why don't we just jump into the FFmpeg issue?
What's going on withmulti-threaded performance with
FFmpeg?
What are the symptoms, what arethe causes and how do we work
around it?
And take it a bite at a time,don't try and jump into all that
.
Speaker 1 (02:00):
So we're going to talk a lot about multi-threads,
multi-threading.
So first, what is a thread?
So a thread is a sequence ofinstructions, things that you
want to do sequentially, so youkind of do them in parallel.
If you want to do things inparallel because you have many
cores on your computers, forexample, so your computer is
(02:22):
able to compute in parallel,then you need to have several
threads, and these threads mostof the time are not all the time
, but most of the time are saidkernel threads.
It's an object that isunderstood by the kernel of your
operating system, that'shandled by a component that is
called the scheduler.
(02:43):
So the scheduler analyzes whichthreads are running, which
threads are eligible forcomputing, and it handles all of
that.
More specifically, on FFmpeg,for your question, ffmpeg is a
multi-threaded application andso there are several levels
where it's multi-threaded.
(03:03):
There can be like a codec thatcan be multi-threaded when you
execute, for example, x364 orx365.
As a library inside FFmpeg,they create their own thread,
but also FFmpeg creates its ownthread, the application FFmpeg.
Speaker 2 (03:20):
So what was I experiencing when we saw a drop
in throughput?
So what was I experiencing whenwe saw a drop in throughput
with and let me just you knowone of the typically the
operation was large file in, say4K, and then a pretty complex
encoding letter out, say five orsix rungs, and testing HEVC,
because it's 4K output and youknow H.264 doesn't make sense.
(03:41):
What's happening in a?
You know it's a pretty simplecommand line, right, we do it
all the time, but you knowwhat's happening that causes
throughput to drop.
Speaker 1 (03:49):
It's not an easy question.
So first, I think there's beenan effort in FFmpeg to try to
bring more threads at theapplication level.
So that's what I was saying.
There were not so many threads,I think, in FFmpeg-5,.
One of the big changes is thatnow, when you have multiple
outputs, each of these outputsrun in a separate thread, which
(04:14):
means that, for example, if youhave two outputs connected to
two different servers, there isa problem on one of the outputs.
The second one is not hung, sothey can run in parallel.
When it comes to 4K, it's areally huge question.
I think there are many factorsthat affect performances when
(04:36):
you're trying to reach thelimits of computers.
There are threadings, of course.
If you're not threaded enough,or if you have too many threads
or so maybe that's somethingthat we may want to talk about,
then you're not going to run atyour maximum speed.
But there is also IOs, right?
(04:56):
If you don't deal with IOscorrectly, then you can be
limited by your IOs.
So the input-output, and thereis also some memory issues, for
example.
So we're going to compare maybewith the other frameworks, but
in FFmpeg the queues are reallylarge.
If you queue a high number ofdecoded frames that are pretty
(05:19):
large by themselves, then it cantake several gigabytes of
memory.
So RAM is cheap.
But if you don't have enoughRAM, then it's take several
gigabytes of memory.
So RAM is cheap, but if youdon't have enough RAM, then it's
going to write to the disk andthen it's going to be super slow
.
Speaker 2 (05:32):
Interesting.
How do you mitigate that?
And if you're working with acommand string as opposed to, I
guess more programmatically, isthere any way you can mitigate
against that?
Speaker 1 (05:42):
So FFmpeg is two parts, I think, for most of the
people.
There are some libraries likelibavcodex for the codex,
libavformat for the demuxers,muxers, network protocols and a
bunch of others, and then thereis the FFmpeg application.
There are other applications inFFmpeg.
There is FFplay, which is asimple player.
(06:04):
There is FFprobe to inspectyour content, and the efforts
that I was talking about are onthe FFmpeg part, which means
that other people were able toleverage some of the benefits
that the FFmpeg developers aretrying to implement, because
they directly integrated theFFmpeg libraries.
(06:27):
So, when you consider bigcompanies like Vimeo, youtube or
other projects like GPAC, weintegrate the libraries and we
could investigate, with someimprovements on the performances
.
Ffmpeg is really great for thisbecause you can plug at many
points, like for the IOs or forthe memory allocators or for the
(06:48):
threading, and you can do yourown stuff.
Not everything is solved on theFFmpeg site but, as I said,
they've been working on it, soexciting.
Speaker 2 (06:55):
Version 6 came out right in the middle of these
problems and I basically justused the same command lines and
tried it on version six and Ifound no difference.
So you know, I thought that oneof the one of the goals of
version six was bettermulti-threaded performance.
What happened?
Am I just you know what changedwith between five and six and
(07:18):
why didn't I see any performancedifference between those two
versions?
With these type of of ofencoding projects, Okay, yeah,
that's a bit technical.
Speaker 1 (07:28):
In version 5, as I said, they have separated the
different outputs, so now youcan run several outputs in
parallel and if there was aproblem on one, then it's not
going to create problems on theother output.
In version 6, it's acontinuation of this and they've
been working on this for atleast a year.
(07:50):
I could find comments relatedto better threading in FFmpeg,
and at the beginning of the year, the FFmpeg developers at the
FOSDEM Open Source Conferencesaid that the effort would last
for the whole year, so probablyyou will need to test with
FFmpeg-7 next year.
Speaker 2 (08:08):
You talked a moment ago about too many cores.
Go into that and then maybedescribe how machines, how you
should configure an encodingmachine better for 4K as
compared to, say, 1080p-typeprojects.
Speaker 1 (08:23):
There are maybe many factors on this.
If you put too many threads,the problem is that in your
operating system, in the kernel,in the scheduler that we talked
about, then the scheduler saysit's called preemptive.
It allocates you with slices oftime, typically tens of
milliseconds, and each tens ofmilliseconds is going to look if
(08:46):
there are some new eligiblestasks so that everybody can have
some time to execute.
If you put too many threads,there are two kind of problems.
The first one is that youswitch all the time.
It's called a context switch.
The price is pretty low, buteach time you have to save your
registers, you have to save abunch of stuff and switch to the
(09:11):
other threads, so that you know, instead of spending time doing
some computation, you'respending time switching tasks.
And the second problem is thatwe're most of the time we are
CPU bound and so we rely heavilyon caches.
And when you change coressometimes you break the cache
(09:33):
and so again you have aperformance penalty.
But this is something that isreally really difficult to
measure.
Speaker 2 (09:39):
Can you give any advice at all?
I mean, if I'm doing, you know,say with, let's stick with
software because you know that'sthe most generic case.
But if I'm doing, you know, saywith, let's stick with software
because you know that's themost generic case.
But if I'm encoding 4k videos,is there a benefit to going to
128k or 128 cores, or should I?
You know, should I stick with32 or does it?
Does it matter?
Or you know, more jobs, lessjobs.
(10:00):
What's your sense of the bestmachine for high volume 4K
transcoding Again, vod, it's alot.
Speaker 1 (10:09):
I think the number of threads on your computer should
be a couple of times the numberof physical cores that are
available on your CPU, and ifyou make some graphics related
to this, you're going to seethat it scales almost linearly
up to a certain point.
That being said, it alsodepends on the application.
(10:30):
As I mentioned, there are, youknow, codecs can create in
FFmpeg.
Ffmpeg the application createsits own codecs, but then the
application may create their owncodecs.
You have codec specific options, for example for x264 or x265
or AVC, hevc encoding or anyencoder where you can try to
(10:52):
tune.
So most of the time the encoderthemselves provide an auto,
automatic parameter so that ittries to scale by itself,
because only the applicationknows if it scales better with
twice the number of cores forthe number of threads, or just
(11:13):
the number of cores plus acertain number because they use
this for scheduling or whatever.
And again, as I said, it's notonly CPU cores.
If you saturate your bandwidthfor the memory, if you saturate
your IOs, for example becauseyou're reading from the network,
or you're reading from a diskthat is not fast enough because
you launched several tasks atthe same time, then it can be a
(11:35):
problem.
That's really.
Really measuring performancesis something that is really not
easy and sometimes with smallchanges, like going from you,
going from FFmpeg, theapplication, to another
framework that uses FFmpeg aslibraries, you can see big
differences.
Speaker 2 (11:53):
Just kind of an aside .
I recently did some workanalyzing the CPUs that were
available on AWS, so I looked atIntel versus Graviton versus
AMD.
What's your sense of you know?
What I found was that Gravitonwas best for X.264 and AMD was
best for X.265.
Are those general conclusionsyou can reach, or do you have
(12:15):
different thoughts as to whichCPUs do the best job with those
codecs?
Speaker 1 (12:19):
I can say that we use a lot and we advocate for, you
know, gravitons and equivalentsbecause they are more power
efficient and also that'sbecause the FFmpeg developers
developing the codecs they'vemade for years a tremendous job
in implementing all the assemblyacceleration that were needed.
Speaker 2 (12:43):
I found that true with X.264, but X.265 was very
bad with Graviton and that wasincluding a very new version
from multicore wear thatactually improved performance by
300 or 400%.
So any comment there?
Speaker 1 (13:01):
Very different projects, right?
X.264 is really community-basedpassionate people and X.264 is
really community-basedpassionate people and x265 is
maintained by multi-coreware.
And even though there ispassion in companies, especially
on video codecs, you can havereally interesting discussions
with them, it's not the sameright You're in front.
(13:24):
X264 was made by people thatwere genius, I would say even
crazy at some point.
So for me, totally differentperspective.
Speaker 2 (13:32):
What about compiling different versions of ffmpeg for
the different CPUs?
Do you need to take it to thatlevel, or is that not necessary?
Speaker 1 (13:41):
I don't think that's necessary, but yeah, you could
do this.
We see not necessary.
I don't think that's necessary,but yeah, you could do this.
We see.
You know you always haveproblems with operations, like
operational problems.
You can come into a situationwhere you'd need to have several
builds or using severalversions even of a specific tool
.
That's a possibility, yes.
Speaker 2 (14:01):
So let's transition over to JPEG and let me kind of
set it up.
I mean, I know FFmpeg.
You know it's really good onthe encoding, transcoding side,
a little bit less good on thepackaging side.
We use GStreamer as kind of aworkaround for some of the
bottlenecks that we you knowthat I described a few minutes
ago.
That work pretty well.
And then you know everybody.
(14:23):
You know, I've known you foryears and known about gpac for
years, but you really came toprominence when netflix licensed
the technology for their livestreams.
Give me, give me an overview ofyou know, just really high
level, about where you see thosethree projects, including, you
know, ffmpeg, gstreamer andgpacpack, what they do, how they
(14:44):
fit in together, what they'regood at, what they're not so
good at.
Speaker 1 (14:48):
FFmpeg.
As I said, they are thelibraries and these people are
implementing codecs protocols.
It's a really nice toolbox thatis really, really complete.
There has been a huge effort onit.
Really successful project, Iwould say, critical
infrastructure for the industrytoday.
(15:08):
And then there is the FFmpegapplication that people use.
People like it because theinterface is quite stable.
You can reuse a lot of yourcommon lines from years ago and
they still work, your commandlines from years ago and they
still work, which is not thecase at the library level where
(15:29):
you have to implement the API,which is the interface.
At the coding level, it changesfrom time to time, so that's
more work to go.
On the libraries.
You can get some of thebenefits.
For most people you can use theFFmpeg command line libraries,
gpack, so we are alsoimplementing some standards and
(15:54):
some streaming protocols.
So we're really experts on thisDemuxing, muxing, packaging,
encrypting, streaming.
We are active atstandardization, so we have a
lot of R&D.
That's the difference, forexample, with FFmpeg and inside
GPAC.
We have our own code and weintegrate it with other
(16:14):
libraries, and FFmpeg is a bigGPAC.
We have our own code and weintegrate it with other
libraries and FFmpeg is a bigone.
For us.
It's a big dependency, and weleverage a lot of FFmpeg.
For our customers, our users ingeneral, gstreamer is more like
an aggregator.
There is no real code beingimplemented in FFmpeg, even
though it has changed over time,but its ambition is to
(16:38):
aggregate everything into thesekind of what they call elements,
which are kind of modules thatyou connect together in a big
graph.
So GStreamer is really good atthis.
You mentioned that GStreamercould handle some of the
bottlenecks that you had withFFmpeg.
The framework is really morecomplex and, for example, we
(17:00):
were talking about saturatingthe memory.
Gstreamer has nice mechanismsfor this.
It pre-allocates the buffers.
It never allocates on the fly,which is also something that
FFmpeg or GPAC do.
But with GStreamer, when youhave a queue, a buffer, when you
are high in a buffer or low ina buffer, it's going to emit a
(17:22):
message, and so I was talkingabout the scheduler.
This is something that isimportant At the GStreamer level
.
They can decide to execute orto not execute some of their
elements just because the bufferlevel is high or not high, and
so you avoid saturating, forexample, your memory usage or
(17:43):
whatever, because when a bufferis full, there is nobody that's
able to push data to you, forexample.
So that's a really preciseexample of where a GStreamer may
have better performances.
A GPAC is also quite flexibleand we have all kinds of I think
intelligent mechanisms to tryto handle performances in the
(18:04):
best way.
But again, you know it's magicby default.
There is a lot of automationand if you don't like it you can
disable it, and there is a lotof flags to try to control your
performances, depending on.
You know the whole environment.
Jpeg may not be aware of.
It's not like when you runthese tools.
There is no like uh, somethingthat is benchmarking your system
(18:26):
and then says, okay, the bestflags for you are this or this.
It's just saying how many cpucores do I have or how much
memory do I have, and it doesn'tconsider the fact that you're
running a browser at the sametime.
You know using already a lot ofmemory or you know other
considerations.
Speaker 2 (18:43):
Let's look at the individual, I guess, components
of what I want from the encoderpackager.
I mean it feels like again,ffmpeg is very strong in the
encode side and it's very usable, very accessible.
Pretty weak on the packagingside.
You know, and I guess you knowFFmpeg is over here.
I don't know where GStreamer ison packaging.
I know that you're prettysophisticated there.
(19:04):
I know there are projects likeBento 4 that are primarily
packaging.
So you know, just trying tounderstand that, and I think
you've given us a good start,what can you tell us about
GStreamer's packagingcapabilities?
Speaker 1 (19:19):
As I said, they write little code, so basically
they're more of an aggregator.
So you know they don't have aGPAC integration, for example,
and most of their packagingrelies on FFmpeg.
So mostly they have thepackaging capability of FFmpeg,
with a few other components onWebRTC et cetera, where they
(19:41):
wrote their own code.
But otherwise I thinkG-Streamer has some particular
markets.
It's more on the hardware side,which is why I'm not surprised
that a company like NetInt wouldtry to use it.
But when it comes to puresoftware workflows I think
(20:02):
FFmpeg is way, way more popularand it's been years that we've
tried so historically, GPAC ismore about file-based packaging
with MP4Box.
So MP4Box is the main tool ofJPEG.
But then people had to encodetheir content with FFmpeg, dump
it and package it.
(20:22):
I think it was okay a couple ofyears ago, where people wouldn't
trust open-source software todo some live.
But now things have changed.
So it's really difficult to beintegrated as a third party to
FFmpeg.
I hope that we can change thatin the near future as near as
possible for me, because FFmpegis super popular.
(20:45):
But we decided to do thereverse.
We said, OK, we can buildsomething that we think is
leveraging FFmpeg and which isbetter than FFmpeg, and so we
have a new application in GPAC,which is simply called GPAC,
like the FFmpeg application inFFmpeg.
That allows you to buildcomplex workflows and it's used
(21:05):
for live by many people.
We have a lot of users on thisand, as I said, because we're
doing some standardization and alot of R&D in JPEG, it's a mix
of a lot of things.
It's a player, it's an analyzer, it's a content generator so
(21:26):
you can encode, you can package,you can stream, you can dump,
we embed an HTTP server, so itdoes a lot of things.
Speaker 2 (21:34):
Taking a step in a different direction for a moment
.
What are you seeing on thepackaging side regarding
utilization?
I mean, we've heard about CMAPfor years, we know Dash, we know
HLS.
What are you seeing as trendsin packaging utilization among
the people that you're workingwith?
Speaker 1 (21:52):
I said that we were doing standardization.
We are advocating for ISO BMFF.
So that's the MPEG-4 fileformat.
And so, because we are instandardization, because it's
pretty active, because they havethis open source software like
GPAC, when it comes to openstandardization, mpeg systems
(22:12):
and the MPEG file format arereally really, really powerful.
Iso BMFF is everywhere.
The QuickTime file format onthe prediction side, cmap,
fragmented MP4, mp4, everythingthat's related also to
encryption with commonencryption dash, hls that's
fragmented MP4, cmap For storage, there is MP4.
(22:37):
I think there's a huge consensus, contrary to what happens in
codecs right now, where thesituation is pretty fragmented.
That's the same for inputcontribution protocols, where
people still use RTMP.
They use also SRT, but theydon't know exactly where to move
.
When it comes to file formats,it's pretty stable.
Iso BMFF is everywhere and Ibelieve that JPEG is the best
(23:00):
tool.
But again, you can try it, it'sopen source.
There is a lot of nice featuresI think that people are not so
aware of.
If you take a raw H.264 file,for example, or H.265 file that
you dump from your encoder, ifyou package it in MP4, the MP4
(23:21):
is going to be smaller than yourinput because we eliminate all
the redundancy and, in addition,you can get some indexation of
the content and you can have alot of nice features that come
with.
You know packaging your content.
Speaker 2 (23:39):
Well, how does that translate to HLS, dash or CMAP?
I mean, does that mean dash?
Or you know it feels like HLSis still the leader, you know,
years after standards-basedapproaches became available.
Are you saying that's not true?
Or I don't remember whatBitmoven said in their developer
survey?
I haven't looked at that in awhile.
But people, they have to outputone of those packaging
(24:03):
standards.
What are you saying in terms ofsupport?
Speaker 1 (24:06):
HLS has dominated for years.
It's going to continue todominate, but that's the truth.
But it's not important, right?
The way you handle yourmanifest is just a really tiny
part compared to what you'redoing with the rest of the data,
which is the majority ofeverything.
And then, there again, there isa convergence on CMAP, which is
(24:26):
ISO BMFF based.
So I think we're pretty closenow that there is a convergence
also on the encryption, becauseyou know that you know
everything that was, you knowindustry-based was aes, ctr and
apple said okay, but cb, cbc andcbcs in particular are.
You know where we want to headto.
(24:47):
So there is a consensus rightnow to go to cmap and for the
encryption, cb.
So I think the convergence ishere.
We are really near the pointwhere you would have only one
media format to distribute.
Speaker 2 (25:00):
And that is ISO BMFF with different packaging.
And that's the whole promise ofCMAP is.
You know, the manifest is sucha small portion of it, it really
doesn't matter.
Speaker 1 (25:10):
You know it's messy, because when you, you know, when
you consider so I say ISO BMFF,which is maybe a technical term
for most of the audience that'sthe base standard that is
edited at MPEG and you canimagine that CMAP is a profile
of it.
So it's going to restrict thetoolbox.
I would have liked it to berestricted to a point where you
(25:31):
say, okay, I need to packagethis and you know exactly what's
going to be out, like we havefor, you know, bit accuracy,
like we have for codecs when youtake an encoding stream, you
know exactly where you're goingto have as an output.
It's not that true anymore.
There are, you know, a fewdiscussions on that, but
basically that's what you have,but still with CMath, not
(25:54):
restricted anymore.
So there is a lot of freedom.
And when there is freedom, ofcourse interoperability is much,
much more difficult.
But again, in GPAC, separatefrom the main GPAC repository,
we have a repository calledCompliance Warden and the
Compliance Warden is all aboutcompliance on ISO BMFF and maybe
(26:15):
one day we'll get some sponsorsto have the full ISO BMFF and
CMAP stack so that people canactually have some better
compliance on this.
Speaker 2 (26:29):
So let's take a deeper look at JPEG you said
it's open source, you said it'scommand line.
A deeper look at gpac you saidit's open source, you said it's
command line.
Um codex that it handles.
You know pretty much thestandard.
You know h.264, hbc, av1.
Any limitations there?
Speaker 1 (26:42):
a lot more.
You can mix.
You can mix everything, uh, allcodecs.
You know you could dash anycodec.
You can dash also masrovska,webm, you know whatever.
You can put special flags to beinteroperable with basically
everything.
That's the chance we have.
We have pretty big users, notonly Netflix, and it gives us a
(27:03):
wide, wide range of applicationand interoperability.
I think that a lot of people,because we're doing R&D, for
example, vvc we've had someintegration with VVC for three
years.
So basically, when you want topackage VVC, there is a high
chance that you come to GPAC andyou try GPAC.
So it's not a real strategyfrom us to try to attract users,
(27:29):
but this is something that ispretty effective compared to
other packages.
You were mentioning Bento4.
So Bento4 has only onemaintainer who doesn't work, who
hasn't worked in packaging forlike six or seven years.
So for me, there is a lot of,you know, packaging projects
that are really good, used bythe industry, but there is a
(27:49):
real, real risk when you haveonly one maintainer and the
project is not activelymaintained anymore.
So that's why you know we weretalking about that there are
different strategies, I think inopen source and projects like
FFmpeg or GPAC you know theycome from a real pain point,
from users, passionate peoplewho you know that's my case.
(28:13):
That's my case I was.
I needed this solution.
I started to contribute to GPACand then I made a company
because I wanted, because I'mpassionate I think people can
see that I'm passionate about itI tried to find a way to make
it a living and also with GPAClicensing you know, the
commercial arm of GPAC insideMotionSpell to allow other
(28:33):
people also to be able to liveoff their contributions.
Okay, ffmpeg, they have thesame right.
They have FF Labs now they had abooth at NAB and for G-Swimmer
they were these two companies,fluendo and then Collabora.
I think that's really importantto give the industry confidence
that they are real peoplesupporting the project and
understanding their needs.
It's really important to givethe industry confidence that you
(28:53):
know they are real people, youknow, supporting the project and
understanding their needs.
It's not a bunch of teenagerkids, you know, in their garage,
because sometimes that's thefeeling that you can have If you
go, for example, on the FFFBugtracker.
You know you can be, you know,have a wrong image, but still
they are trying to buildsomething for the industry.
Speaker 2 (29:15):
I mean, they're tremendously talented
programmers.
What about?
So, looking back at JPEG, whatabout HDR?
I mean, you know Dolby supportshuge.
Hdr 10 plus is huge.
What do you?
What are your capabilitiesthere?
Speaker 1 (29:27):
We support everything .
We have a very generic supportand the latest release, that was
last December, had a specialfocus on HDR.
So we support everything static, dynamic metadata, all kinds of
metadata.
You can re-inject your ownmetadata.
You can fix the metadata alongthe stream because sometimes
(29:47):
there is a lot of streams thatare just broken.
Let's frame it this way.
So we support everything.
And, again, we have a reallydeep integration with FFmpegs
internal.
That allows us to take, likethe row, you know, the matrix
coefficients and the color spaceetc.
And to map it into somethingthat would be recognized as
(30:09):
Dolby Vision or HDR10 or HDR10plus, etc.
Where does the Dolby licensingcome in?
Or HDR10+, et cetera.
Speaker 2 (30:15):
Where does the Dolby licensing come in?
And I guess, taking a step back, can I do Dolby Vision with
FFmpeg.
I know I can do HDR10+, but canI do Dolby with FFmpeg?
And what about the licensing?
Is that just device decode sideor is that on the encode side
as well?
Speaker 1 (30:34):
I don't know.
I know they support it on theencode side.
They support the metadata.
Like you know, they have somespecial side data that comes
with the packets of the data andthey support it.
That's all I can say.
I don't know more about this.
But that's sufficient for GPACto package it.
Speaker 2 (30:52):
You know, one of the things that's kind of been
interesting to me is the compareand contrast for commercial
software like you can get from anumber of vendors.
You know Harmonic comes to mind.
You know what are the pros andcons of open source versus, you
know, buying a commercialversion.
That's going to have a prettyextensive UI, it's going to have
a lot of testing, it's going tohave people you can call and
(31:13):
get support from pretty easily.
What's your sense of thatdecision?
Speaker 1 (31:17):
Well, I think there are several reasons why you
would want to do that.
I had the chance to talk at anAB streaming summit with Netflix
and they explained that they doit first because it's open
standards.
They don't see how they couldcompete on this.
It's not their goal now.
They are pushing contributionsthere.
(31:38):
They want to compete on otherthings, not these open standards
.
And I think Netflix made ademonstration when it migrated
to JPEG.
They moved super fast, like yousee Bandersnatch.
You see evolving a newer, newgeneration audio codecs.
They announced an ad-based tier.
(32:03):
They moved to live.
I don't know any company ableto move that fast.
So I'm not saying it's 100%because they are using open
source or whatever.
But clearly that's points wherepackaging is crucial, right?
How do you make sure, if you'redoing interactive, how do you
make sure that you can stitchyour content?
Do you know any commercial toolthat says, okay, I can stitch
(32:25):
and create scenarios on this.
If you don't control exactlywhat a packager does, you cannot
do this.
So, and again, when there isnew codecs, when there are other
things using an open sourcetool, you know you can leverage
the work that has already beendone by the community.
So I think that's important.
But, that being said, netflixis a pretty innovative companies
(32:49):
, so we have the chance to havethem like, as you know, sponsors
, and also they pay us when theyneed, you know, integration of
new tools, new codecs orwhatever you know.
For us it's really easy to workwith you know such a company
and we understand that it's it'sreally easy to work with such a
company and we understand thatit's kind of easy customers.
Speaker 2 (33:08):
I mean, they've got a phenomenal engineering team and
I guess certainly a lot ofpublishers don't have the
engineering resources in-housethat Netflix does.
But let's move to Netflix.
Tell us what it is you licenseto them and what it's being used
for at Netflix.
Speaker 1 (33:28):
They gave a number.
It's like what was it?
It was a million.
I don't remember the scale, butthat's millions of content
every month that are beingpackaged.
Basically, they packageeverything.
They mentioned that, as, as Isaid, on the production side, uh
, they handle the contents, uh,or at least some of the content,
with gpac, because they don'tcontrol 100 percent of the.
(33:50):
You know the formats that thatcomes in.
Also previews for thejournalists.
Uh, you know packaging thecontent for their 200 million
viewers.
So my understanding is thatthey package everything.
But again, when it comes to openstores, the magic is that you
don't know what your users do.
We don't put telemetry in ourtools.
(34:12):
That's pure freedom.
So I don't know what Netflix isactually working on If they
don't send me an email.
Or when we work with customers,we have acceptance tests.
So that's tests, basically,that we both execute.
So that motion spell for GPACwould execute and Netflix would
execute, and we agree that thisshouldn't be broken, otherwise
(34:35):
there would be a problem.
We send them an email, or whenthey want to upgrade their
version of GPAC, they would haveto execute this list of tests
to make sure, you know, to givethem confidence.
So that's the only way for usto know how a customer actually
uses GPAC.
Speaker 2 (34:52):
So is it for live?
Is it for live NBOD, or youjust don't know?
Speaker 1 (34:56):
I don't know.
I know for VOD, for sure wewere not able to talk about live
for the NAB presentation, but Ihope that you know, if there is
some GPAC involved, we can alsoshare things.
They have a really greattechnical blog, like many of the
companies, and so it would begreat if they could communicate
on it.
But you know, I don't know,maybe this needs to mature also.
(35:19):
I don't know, maybe this needsto mature also.
I'm not into, as I said,netflix or any other customers
internally.
We're a pure technical vendorno-transcript.
(35:46):
That's something that you maywant to do to get some support.
You may want to support us.
Also, if you use us and you'resuccessful, you may want to
support us so that the projectcontinues.
It's sustainable, basically,but there are some sometimes.
The open source, the GNU LGPLlicense, is a bit tricky for
(36:07):
companies to deal with.
Like you know, you cannotforbid reverse engineering and
this type of things that youknow you look twice if you want
to accept such clauses.
So we're in a position to do alicense, gpac, and so we can
emit commercial licenses.
(36:27):
But these licenses we're notlooking specifically into
details.
What is important for us is thatpeople have a solution.
We're building really aplatform and I'm pretty sure I
have all the statistics, ofcourse I'm pretty sure GPAC is
going to be the it's.
A majority of the packaging nowis done with GPAC and so I'm
(36:49):
pretty sure, as I said, becausewe are on open standards, I'm
pretty sure that you know thereis not going to be a lot of
competition on this.
I think it's going to stabilize.
We have ISO BMFF, which is areally good technical basis, and
once the innovation slows downand I think it's going to happen
on where we are?
(37:10):
I hope so.
So there's still the question,as I said, to the contribution
protocols.
We're not on it and there is areal problem.
There is also the problem ofwhat's going to replace MPEG-TS
exactly.
So now we went into HTTPdelivery with OTT and ISO BMFF
is, you know, pretty good, butthat's not a streaming protocol,
(37:31):
right?
So there are still things Ithink to you know, figure out on
the transport level.
But you know that's the future.
I think ISO BMFF is going tostay.
It's going to stay for 20 or 30years and after that, who knows
?
Speaker 2 (37:46):
Give me the compare and contrast with Bento 4,
because it seems like youextended back into FFmpeg for
more of the transcodingoperation but that primarily
you're being used as a packager.
Is Bento 4 your most, I guess,the most pressing competitor at
this point, or the one that'smost obvious?
Speaker 1 (38:08):
I think a lot of people in the industry use Bento
4.
Some of them realize that it'snot maintained anymore and
that's a technological risk, butthat's not the only risk that
they face.
As I said, it's a problem.
You go into open source andthen the project is not
maintained anymore.
You know you're happy with theproject, but then you know there
(38:29):
is a new video codex thatappears.
Who's going to implement this?
If you have nobody tocoordinate the developments,
what happens?
So I again, that's what I think.
A lot of people like GPAC andthey go to see us because there
is an active community.
It's an active project.
We're always ahead of time.
Everything is easy, everythingis flexible, there is a
(38:54):
commercial arm.
So I think that maybe the nextmove for us is also to help
customers to migrate from Bento4to us, just because at some
point there is no solution.
There is another competitor,pretty big, in the same area.
That's Shaka Packager.
The two main maintainers arenot working on it.
There is a lot of people whocame to sales and said what do
(39:16):
we need to do?
It's a Google project, we needto de-Google-ize it and then we
need to maintain it.
And when there are new codecs,who's going to do this?
Even for big companies, therewas no inside resources.
Netflix is the proof of that.
They had their own packager andthey decided to move to open
source, and then some of thecompanies who went to the wrong
(39:37):
they couldn't know to an opensource project that isn't
maintained anymore.
Now what do they do?
Either they invest in somethingelse, like JPEG, or they need
to maintain the tool that is notmaintained anymore, or create a
community around it, orwhatever.
So I think it's also ourresponsibility to help people
(39:59):
migrate easily to us.
Speaker 2 (40:02):
Okay, I should note.
I mean I went and I'm doing atalk at Mile High next in a
couple of weeks and I looked atthe.
I mean there are some responsesto questions posed on the Bento
, for, you know, whatever theresource is for support, I don't
know if there's any newdevelopment.
There have been responseswithin the last two or three
(40:23):
weeks, so I don't know what thatmeans.
You know.
I just want to mention thatbecause I don't want to.
I don't want to.
I'm not saying it's not betterto switch over to your tool.
I'm just saying, you know,there are signs of life there
that may be different than whatit is you're expressing.
Speaker 1 (40:38):
I think people can.
There are things that you canhave a look at, like how many
open issues in a project, howmany pending pull requests.
It gives you, you know, a senseof, you know is the project
active?
Are the people you know, arethere enough people to you know,
deal with the activity of theproject or whatever.
(40:58):
I think that for people youknow comparing open source
projects, most of the project orwhatever.
I think that for people youknow comparing open source
projects, most of the timethey're comparing the stars,
which is on GitHub, when theprojects are on GitHub, which is
a good indicator of thepopularity of a project.
But then look at the openissues.
Speaker 2 (41:14):
What about?
You know I'm actually lookingat a GPAC now to integrate into
the same talk.
What are the educationalresources that you have?
You know that's going to beimportant both for new
implementers and for peopleswitching over from Shaka and
Bento 4.
You know how developed arethose capabilities on your
website.
Speaker 1 (41:31):
The website is pretty old, it's just.
It dispatches users.
That's how that's all it does.
We have a pretty good wikithat's hosted on GitHub.
So if you make searches onGitHub directly, you can find a
lot of things.
And if you read the readme inthe front page, we give a lot of
pointers to our test, our buildboat.
(41:52):
There is a doxygen fordevelopers.
But yeah, again, I think thewiki is the best entry point for
people.
And then on the common line, wehave some really good magic.
If you make gpack minus h andyou put whatever word, it's
going to output everything thatis related to this word.
And by reading the doc you willlearn that we have several
(42:16):
levels of verbosity on the help.
So, as I said, we want to makeit magic.
I think that for a lot ofpeople, you know common lines
can feel painful and thelearning curve is really, really
important.
What I would like is that in thefuture we have a graphical
interface.
I think that would be great forpeople to see.
(42:37):
You know what's happening, etcetera, because multimedia is a
lot of magic.
You know, you have H.264, it'sAnnex B and then someone has to
take care of this.
That's the same AAC, implicitsignaling, you know ADTS, latm.
That's a nightmare when youdon't look into the details.
It can be a nightmare and Ithink people would, you know,
(43:00):
educate themselves by having toyou know some, you know
graphical interface, showing tothem what is the actual work
that's being done, and Iwouldn't do that only for users,
I would do that so that youknow we can also, you know, keep
talents and, you know, make itless, you know, less complex.
I think talent is, is, is, youknow, how do you say?
(43:23):
Talent retention is somethingthat the whole industry should
look at, because there is a lotof, you know, gifted people
leaving our industry after acouple of years and we need more
of this.
Speaker 2 (43:34):
We have a question Can you describe the extent of
your VVC support and I guess,vvc?
I know Fraunhofer had an issuewith getting their VVC
implementation into FFmpeg, sowhich version are you supporting
and how does that work?
Speaker 1 (43:49):
Yeah, we're pretty up to date because we have a close
integration with the people ofOpenVVC, which is an open source
VVC decoder.
They have some integration inFFmpeg and so we're following
the latest developments of thestandard.
If that's not the case, justopen an issue on our bug tracker
(44:10):
.
But we're pretty up to date.
There is MPEG this week and sowe're going to make an update if
there are fixes to the spec oradditions.
Speaker 2 (44:19):
Another question on your inspector.
What can your inspector do andhow is it different from, say,
MediaInfo?
Speaker 1 (44:25):
Yeah, okay.
So I think MediaInfo is a greatinspector.
You take MediaInfo, the name ofthe file, you can go common
line, you can go with thegraphical interface.
It gives you immediately anoverview.
But one thing that it doesn'tdo is inspect the packets like
deep inspection of each of yourpackets, and GPAC does that.
(44:45):
We have several levels ofverbosity, and so it can display
general information, and Iwould like, as I told you that I
want, to improve that, maybegraphical interface so that
people can navigate into thecontent and then you can go deep
, deep, navigate into thecontent and then you can go deep
, deep, deep, deep, up to thepoint of every bit that's been
(45:07):
parsed.
Netflix had an anecdote aboutthis that JPEG was really,
really picky and going reallydeep in the bit streams and
actually it was reporting, orthey were able to report, errors
on some standards where fileswere already widespread.
(45:28):
And then you have what do youneed to do in this situation?
Do you need to modify thestandard or do you need to ping
the other implementation and say, hey, you should modify what
you're doing?
Speaker 2 (45:42):
One last question what is MotionSpell?
I think you mentioned that atthe start and we just got a
question in that relates to thatis what are the licensing terms
for GPAC and FFmpeg?
So what is MotionSpell and whatare the licensing terms for
GPAC and FFmpeg?
Speaker 1 (45:59):
So MotionSpell is the company that holds the
commercial arm of gpac and we'realso making products, most most
coming from the gpac technologyor related, so we have things
that are related.
Now we have an ott livesubtitle inserter for people you
know having a live ott streambut you know they need to add
(46:21):
titles on the top of it and theydon't know how to do it and, as
I said, also related tocompliance.
We're doing a lot of work withthe biggest companies in the
industry.
I think that's pretty important.
And regarding the license, thelicense of FFmpeg and GPAC are
the same, that's LGPL V2.1 plus,and my chance at MotionSpec is
(46:44):
that I went to see every of thecopyright holder in GPAC and
they signed an agreement with meand I can put a commercial
license which is basically yourterms.
We propose basic terms and youcan modify them to suit your
needs.
Speaker 2 (46:58):
Another question came in what's the status of
mp4boxjs?
Is that being maintained?
Speaker 1 (47:03):
It's not actively maintained and I think that's so
.
No, it's not true.
We're in the same situation asBento 4, right, we're merging
the pull request, but we're notactively developing it, and
there's a reason for that.
There is WebAssembly now andyou can build JPEG and run it in
your browser, so I think it'sgoing to be duplicated within a
(47:26):
couple of years.
That being said, in the middle,I think MP4Boxjs is a really,
really nice tool that's reallypopular and people use it,
coordinated with WebCodecs, andit can transform their content
in the browser.
That's really amazing.
Speaker 2 (47:41):
Pleasure seeing you at NAB, pleasure working with
you on this and I'm sure we'llbe in touch over the next year.
I'm going to need some helpwith the presentation I talked
about.
I'll be in touch and thank youfor participating today.
Speaker 1 (47:52):
Yeah, my pleasure.
Thank you, Jan.
Thank you everybody forattending, and if you have
questions, I'm always here.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.