This is Self-Directed Research. Our
hosts, James and Amos,
entertain each other with
weekly hyper-focused technical deep
dives. Amos is up this
week with "Black Magic Fusion
is Weird." Make sure to like, follow, and
subscribe wherever you find
us, and visit sdr-podcast.com/episodes
for all the presentations,
videos, show notes, and transcripts.
And as usual, stay tuned to the end to
hear more about Depot,
the sponsor of this episode.
James Munns: I am so ready.
I am so ready. Amos, I am so
ready. Well, I'm trying to look at what
your slides are, but I've turned my my
Jitsi performance all the way down
because it was struggling
and now yeah, the letters are--
Amos Wenger: I need to show the other screen.
Yes. So. James. I have-- I have
exactly one worry It's that we don't talk
about things that are relatable enough on
this podcast. And this I think-- I
think that people want to hear about
technologies that they use every day and
this is why --
today's episode is about migrating from
AppleScript to Lua. Something that I
think everyone can relate to
James Munns: Two languages so many people
have written I would argue a
Amos Wenger: I would argue a
bunch of people have that exposure to Lua
if only to like mod a game or something
James Munns: Yeah, Lua.
I feel like the two places it pops up are
for me at least-- well I guess three
because video games, Nginx and a lot of--
it's one of the few languages that
people will put on embedded systems and
Has a decent chance at running it like
very very small embedded systems. So like
it's a weird palette of places that I've
seen Lua. But I guess it makes sense of
like I need fast scripting
glue kind of stuff.
Amos Wenger: Yes. The actual title of the episode is "Black
Magic Fusion is weird" and that's an
understatement. I'm so excited James
because finally we're gonna have a short
Amos episode because I only-- can you guess
how many slides I have?
James Munns: You said that at the front of some
episodes that have
gone a very solid hour.
Amos Wenger: I only have 61 slides. So
this is this is gonna be
Amanda Majorowicz: Whoopsie, again again-- quote.. I can
quote this. I can splice it all together
Be like, "It's gonna be short!
I only have a bajillion slides."
Amos Wenger: But you'll see there's this like little
games we can play, we can try to guess
things, there's like text encoding going
on. It's very fun, but mostly it's
because I can't help myself. And because I...
We have a special guest today.
His name is Sherlock, he is my white cat. He's uh,
he's meowing at the window and I'm not
gonna do anything about it
James Munns: Hello Sherlock!
Amos Wenger: He's gonna calm down
and also cannot hear you. I cannot be
helped but I can be nerd sniped and so I....
I'm going to do something about the cat.
James Munns: What's the French
onomatopea for cats meowing?
Amos Wenger: Miaou, it's M-I-A-O-U, "miaou"
Yeah, so it's kind of the
same sound as meow. Yeah.
Amanda Majorowicz: You almost pronounce all of the vowels
actually... surprisingly.
James Munns: So Amos, why do you find yourself
understanding Lua for Black Magic Fusion?
Amos Wenger: This is a sequel episode actually
Previously on Self-Directed Research.
See this is why I have 61 slides, because I
have a slide "Previously, on SDR." James:
Where can people find the slides?
James Munns: They can find it on
sdr-podcast.com/episodes
Amos Wenger: Good luck editing this Amanda
James Munns: Amanda.
I'm sorry.
We're gonna be a nightmare
Amanda Majorowicz: I'm ready
Amos Wenger: So, previously on Self-Directed
Research I talked about how I make videos
because this is not a podcast. This is a
thinly veiled marketing vessel for the
both of us. And so I explained that I can
make screenshots of code and import that
into DaVinci Resolve and then in
Fusion I can use like a 3D Keyer node to
remove the background so that I can then
composite that with some shots of my face
and other backgrounds other elements and
animations and whatnot. But that it's not
ideal. Because you get kind of messy edges
all around the text. It's already
composited, the text is already composited
on something, some color and so when
you remove the color even when you choose
your color wisely or black and you're
putting a dark background.
It's just not ideal. We talked about how
actually in some browsers like Safari, we
recently learned we had viewer mail but
not-- listener mail? Like, we got some mail!
Someone said, "Oh actually you can also do
that in Chrome with the Puppeteer API and
it's much simpler." But I would like-- the
whole episode was actually on Safari.
There's a capture screenshot command that
takes a DOM node and then you can get
that entire node no matter how big it is
with completely transparent background
and it's perfectly
crisp and clean and nice
James Munns: Just what you wanted
Totally unintentionally.
Amos Wenger: Yeah... and so then we talked about
AppleScript and Automator and all
sorts of things like that. And how
AppleScript is weird. It's a very
entertaining episode actually. When
reviewing it I was like,
"Wow, we're-- we're interesting."
James Munns: This is the-- this is the "automating macOS"
one if I remember correctly...
Amos Wenger: Yes, it is.
Here's the thing: some people have pointed
out, and they are correct! And I am
helpless that Fusion can render text, of
course because... you can have titles
you can just-- you can have a... what's called
a Text+ node, so you can even get some
formatting in there.
So here we have cantaloupe
salmon in the Berkeley Mono.
James Munns: Is this what you ate for lunch or is this just a
like a diceware...
Amos Wenger: It's the name of macOS colors from the picker
Because the next thing I want to show--
James Munns: Thos are two pretty colors, yeah.
Amos Wenger: Yeah is that you can change a bunch of
things in the Text+ node.
You can change the size, the
tracking, line spacing, various
alignment properties, and you can animate
writing on the text so that it comes
progressively as if someone was writing
it. And you can add multiple shading
elements. And something that took me
forever to figure out: you can also have
character level styling. So the way it
works is the least intuitive thing in the
world. So you first type in some text in
your text node and then you right-click
in the text control in the sidebar
inspector thingy, you right-click, you
click character level styling and then
you go in modifiers... And then in modifiers
you then select- not in the text input
that you just clicked, or the one that's
a modifier that's an almost exact copy
of that- you select in the preview window
So you do like a rectangular selection in
the preview window and you get those
little green sci-fi highlights
Like, I don't know...
James Munns: Character divider boxes?
Amos Wenger: Outlines, yeah yeah.
Yeah, and that's your selection. And so
much of this will make sense in a minute,
but when I first discovered it, I was quite
puzzled. But yeah, you can-- in the same
Text+ node you can have different
colors, you can have different sizes, you
can have different-- all the controls you
see here. Even alignments and--
James Munns: Instead of trying to make two text boxes
that have different colors and you just
like shove them next to each other and hope that they're aligned.
Amos Wenger: Exactly, that would be my nightmare.
Yeah. Editing this
is really bad. Once you've added
styles, if you then click in the middle of
salmon for example, and you try adding
more letters It will be the right color
So it kind of extends the range that is
styled but if you copy and paste for
example, it does not
retain styling or anything.
This is not WYSIWYG,
This is not what you see is what you get
It is what you get is what you get.
James Munns: For anyone interested in debugging that
can go back to our episode about
clipboards where you could probably use
one of the tools we
mentioned in there to figure out exactly--
Amos Wenger: We're getting to-- James! No spoilers!
James Munns: Am I spoiling things? Okay.
Amos Wenger: You absolutely are, but I like that. I
think that that's the sign of a
well-designed slide deck is that you can
guess where I'm taking
you. So can we automate that?
Well, let's talk about the different ways
DaVinci Resolve and
Fusion can be automated. So: Yes.
Yes.
The short answer is yes, but there's many
different ways you can do that. One way is
Fuses. And this is essentially... Yeah, you
can just script this thing with Lua.
It wasn't always there in Fusion, I'm just
now realizing that 'fuses' and 'fusion' are
the same word. Wow. Coffee is a great drug.
James Munns: They're like... I
forget what -phone it is.
It's not a homophone, but they're like
'fusion' and 'fuses' are... I don't even actually
know if they share a root for anything,
because fusion seems nuclear to me and
then fuses sound electrical to me.
Amos Wenger: Maybe?
I don't... do we have time
for an etymology break?
James Munns: No, I don't-- keep going.
Well, we gotta run , we gotta run.
Amanda will put the Wikipedia page in
the links below and we'll never know the
answer until we go
and review the episode.
Amos Wenger: All right.
All right.
All right, so I'm talking about Fusion
and Resolve kind of interchangeably and
that's because today they are integrated.
DaVinci Resolve is like three or four
different programs in a trench coat.
One of them is Fusion. One of them is the
audio thing... what is it called?
Fairlight. The Fairlight audio tab.
There's just different tabs or pages in
the application. But they used to be or
they also still are sometimes-- Fusion I
think is still available as a standalone
thingy. And Fusion is very very old. It was
made for MS-DOS in 1987.
It's older than me! By Eyeon.
James Munns: I thought it was a newer-- like for some
reason, it registered as like-- it's
probably because when it became a
public tool or something like that, but
it registers to me as the new
one versus all the other ones.
Amos Wenger: No, it's the first Windows version
was in 1996. They only added a timeline so
like you could do animated renders in
1997. And then they
added Lua Fuses in 2007.
I don't remember-- I don't have it in my
notes like what year Blackmagic, which is
the company behind DaVinci Resolve,
acquired the Fusion software.
I think it's 2014 off the top of my head.
But do not quote me on that. Now Fusion
has been used in a ton of big movies and
TV shows, including Sin City.
I don't know if you... You know Sin City? And
Lost and American Horror Story. I cannot
not say American Horror Story in the FX
voice. Like, "Next week on American Horror
Story." That's just how I have to say it
You can tell that Fusion is old because
we're looking at this Wikipedia... Like
early Wikipedia clone... like
what is it called, MediaWiki?
James Munns: VFXPedia.
Amos Wenger: Yeah, called VFXPedia.
Well, we're actually looking at as a
mirror maintained by the wonderful people
at We Suck Less, a phpBB forum, hosted at
steakunderwater.com, and which shows you
a lovely SQL error if you try to
register. But it still works.
I got in. I got us a listing of what a
Fuse looks like. And you can actually
do quite a bit with it. In this
example, you can draw some text.
So this is what we're interested in
actually. It's just Lua
code as a font manager.
You can scan the directory, you can assign
things. And drawstring here that we see on
the screen is-- you can get the slides at
sdr-podcast.com/episodes or on YouTube
if you're if you're listening to that
elsewhere, you can just go on YouTube and
have the slides synchronized.
James Munns: I do upload the
videos to Spotify as well.
Amos Wenger: We do?
James Munns: For a very long time now!
Amos Wenger: I don't use Spotify.
James Munns: So drawstring, not like the pajamas.
Amos Wenger: Drawstring is not a built-in. Drawstring
is a Lua function that calls out to
FreeType. And it has interesting comments like:
'Get the character or glyph' which is
followed immediately by a call to
something called bite.
So that does not...
inspire confidence In terms of text--
James Munns: It's-- it rings from a time
before Unicode was... a thing.
Amos Wenger: Yeah... Then it says 'Not really sure why we
multiply this by 10, we just do' smiley
face emoticon. Yeah, and then right
James Munns: Love that.
Amos Wenger: Yeah. And then
right before, the crucial bit: there's a comment
that says 'I think this renders the shape
we are interested in' and this is the code
sample that Blackmagic gives you if you
want to learn about Fuses. They also have
like the unofficial PDF documentation as
of July 2022. So it only
took you know... you do the math
James Munns: 25 years?
Amos Wenger: No, 2007 to 2022 but they acquired it in
2014- like okay sure. So Fuses could work.
Right? Because you can you can do
arbitrary rendering. But what are we gonna
do? Are we gonna like design a control
where we paste in some- some Rust code and
then write some Lua code to parse the
Rust into an AST and like... whatever?
Or like have some sort of markup language?
Like parse the HTML instead and then
render that or...? There's still questions.
James Munns: I don't know. I watched you fall down the
hole of AppleScript.
I really... you could go so many ways with this.
Amos Wenger: Yeah, but no. I decided not use Fuses. In fact, I
got the source code and I got the manual
text rendering and layouting thing as a
last pass before we recorded this. I didn't
actually pursue that avenue at all.
Another way you can extend DaVinci
Resolve is OpenFX filters, which is
actually a standard as far as I can tell.
And it's... a bit like VST, but for
video. VST is for--
a standard for audio effects
James Munns: Yeah, I like audio effects in digital
audio workstations and things like that.
The one person I know who makes VSTs kind
of hates VST or like every platform makes
it hard to keep things portable. But it is
the standard way of
writing audio plugins basically.
Amos Wenger: Yeah, and it's a really fun
little microcosm because you buy a DLL
essentially from someone and then they
always have these
wonky UIs, I don't know.
It's a fun little
universe. OpenFX filters:
here's the, here's the read the docs
Sphinx docs guide thingy.
Here's what it looks like.
It's the same thing as the Lua thing, but
it's in C++ so you can actually do per
pixel thingy. They're using in Fusion for
like blur, box blur, directional blur,
Gaussian blur, lens blur, mosaic blur,
radial blur, zoom blur: all sorts of like
color space transforms. All the basic
visual effects are actually OpenFX
plugins that are
shipped with DaVinci Resolve.
So it's not the
best solution for that.
I don't even know how it would work,
because you would have to have some sort
of like color generator node and then
drag your effect on top of that and then
you would have a separate kind of-- yeah
sure you. I mean, it's native right?
You could ship whatever you could have
your own copy of FreeType in there. You
can-- you could do all the rendering all
the, um... you could link against some Rust code
to actually do the highlighting... you could
do everything. But at the end of the day,
it's... it's awkward to composite because
your output is something like a 4k canvas.
Right? If you have code that is taller
than that, then you need to have the
scroll parameter or something be a
parameter of your effect, of your plugin.
You can't take that and put it into
Fusion and like have it be part of the
graph and animate it. I don't know, it's... I
have not pursued this road either.
James Munns: If you wanted to do a giant Star
Wars scroll of your code...
Amos Wenger: I mean it happens more often than you think.
I have a lot of code.
James Munns: I believe it.
Amos Wenger: Can you write OpenFX filters
in Rust? Of course you can someone made
ofx-rs. And it's
exactly what you'd expect.
It's ugly because it it has to interface
with C++ and uh... But it's better because it's--
it's more readable to me,
let's say, because it's Rust.
James Munns: Hmm. Is it using like CXX
or something like that?
Amos Wenger: I have no idea. I just checked that it
worked, glad that it existed--
James Munns: That's the nice thing about of a crate: you don't
have to think about it.
It just worked and I did--
Amos Wenger: I don't have to.
James Munns: I'm sorry if I just cursed you with the
desire for knowledge.
Amos Wenger: No, no, no, I'm getting wiser James. And
we are only halfway through. Another thing
you can do is that
resolve has a scripting API.
So that's another place where you can use
Lua or Python interchangeably. I'm not
exactly sure how they did that but... and
the documentation for that is
non-existent. They have a readme.txt
somewhere and someone parsed it and
generated like a nice website out of it.
And it still has almost nothing. I think
the only thing that I've seen that uses
it- that I've used- is AutoCut that does
something like export your whole project
to a WAV file, then analyze it elsewhere,
generate cut points and like basically
cuts out the silences. And it does that by
using the scripting API to tell Resolve:
Okay, move here. Do
blade cut. Select this etc.
So that's what it's for.
It's for like moving clips around on the
timeline, changing properties, automating
stuff like that. So I imagine like big
productions, if they have to mass retag
some clips or something, that's what
they would use. But you know what's fun
It's that everything in Fusion- like every
scene- is actually Lua.
So this is also a callback to- and you
called it James- but this is a callback to
our clipboard episode. If you go into
Fusion and you copy the the Text+
nodes that we were- like that I was showing
earlier- you can see it doesn't
immediately register to me as Lua, but
it is. Actually--
James Munns: Okay, yeah, I would say Lua-- I've had to tweak a
couple times but I'm not really fluent in
Lua, but it's got a very like somewhere
between JavaScript and
Python kind of... kind of vibe to it.
Amos Wenger: Lua's weird. I had to use it a bunch
when I worked at itch and I never really...
just the one index
arrays I just never could
But it it has a jit. It's light enough to
integrate, it's been around forever...
Yeah, there's a reason
it's almost everywhere.
And yeah the scene description- like the
node graphs in Fusion are Lua.
So when you copy them you get a bunch of
text readable things which explains a lot
about the performance overall by the way,
and the stability of the software, which
is sometimes not ideal. But it's also very
nice because you can see that cantaloupe
salmon is here in the character level
styling node. Kind of need
to zoom in a little bit.
There you go.
And that's-- for the colors,
we also have something that
is pretty easily reversible.
I think what happened there, the
nerdsniping process, is that I just
noticed that you got text out of it, and I
figured you could probably paste it back
in or like generate something and paste
it back in. But I had no idea-- I saw all
these numbers and I got scared and I
posted it on Bluesky or something and
someone was like: Wait a minute.
It's not scary at all! 2401 is red, 2402 is
green and 2403 is blue. As you can see.
Like I put the color picker next to it
And you can see: yeah, one is 255, 0.49 is
126, 0.47 is 121. So yeah, okay.
So the first element in the array 2401 is
the attribute number or ID or something
like the property ID. Then we have 11 and
16 are start and end. And then value is
the value. And you have different
properties. You can have font, you can have
font variants, you can have a bunch of
different things. But we're just
interested in colors right now.
And so the plan becomes: okay, so we have
an example scene, right?
We've added some Text+
nodes in Fusion.
We have had this complete Lua printout
and we can just generate our own and then
paste it back into Fusion. That's-- and then...
and then profit. But there is a few issues...
There's at least two important details
that got in the way. Because this is
exactly what I did. I'll show you what it
looks like later, but
James Munns: We're falling down the
hole with you right now.
Amos Wenger: Yes yes yes. Um, can you James--
Can you predict what some
of the details might be?
James Munns: Well, you said earlier that when you
copied and pasted it back in, it didn't
have the second one. So when you did the
like fancy text box formatting, it was
just cantaloupe and not cantaloupe salmon
or the salmon was cantaloupe If that
makes sense. So i'm guessing that's going
to be an issue here.
Amos Wenger: So... yes, and no. So you don't paste it into
the text input. You paste it into the node
graph editor. The way it works is that I
generate my own entire node graph and
then I have to select all the existing
node graph, delete that... And then paste my
own in place. But there
is an interesting bug here.
I didn't even think to document it. It's
that when you delete the entire node
graph and you paste another entire node
graph: because the media output just disappeared
It just never shows anything anymore in the
preview tab until you switch to a
different page and switch back
So you don't get to see what it actually...
You don't get a preview of what you just
pasted in until you switch back to the
edit page and then back to a Fusion.
But uh, no the uh, the first detail that's
annoying is that RGB doesn't really mean
anything. You need to kind of have a
reference. It goes from zero to one,
sure. But... what does it mean? And there's
three different places we can see color
And none of them agree with each other.
There's the macOS color picker, there's
the Fusion color control, and then there's
the Resolve render preview window. And
they all show distinct values. This is my
favorite slide. But we can see that.
James Munns: Oh goodness .
Amos Wenger: On the preview
We have 255, 175, 171.
On the macOS color picker.
We have 255, 148, 140. It's even darker on
the-- in the Fusion inspector which is 255,
115, 115. The preview is closest to the uh,
actual export. Which is good.
At least you can kinda trust that. If you
export the video, the color-- I did color
picking on the- on the final file and it
is actually what the
preview is. But if you remember--
James Munns: We're gonna learn more
about color spaces now?
Amos Wenger: Not a lot. Don't worry.
Let me get my slide.
Yeah, the macOS color picker
corresponded well with the values here,
which does not correspond to what you
actually see on screen.
So I... Honestly almost gave up at this
point because I was like I don't even
freaking know-- and I made a whole freaking
article slash video about
color: "Color is best effort."
James Munns: Mm-hmm.
Amos Wenger: Which is my worst performing video of the
past 10 videos by the way. A good like humbling
reminder that you know, not every video
is a hit. But I like it. You think I would
figure it out I would go like: is it red
709 because that's the
color space of the project.
Is it SRGB that would make sense for a
web, that would make sense for like a
system color picker.
Is it linear?
I don't I don't know.
Uh, what I know is I assumed that the RGB
CSS properties that i'm converting from
on my website-- this is some code for my
website-- are sRGB and i'm assuming that
Fusion wants something linear. And it's
it's wrong by some measure. But not enough
for me to care anymore.
I have-- I have spent all my cycles on this.
I have no more cycles to spend and uh,
it's close enough. The second detail is
how to define tech ranges and this one I
did not expect because it's 2025 and I
keep forgetting that some
software is actually a 35 year old.
James Munns: This is gonna be bytes and not glyphs
and which means
terrible UTF8 things can happen?
Amos Wenger: So glyph-- glyph is... glyph
is making me sad angry.
Glyph is not a thing really
James Munns: I'm using the
incorrect Unicode word. There--
You're right, there's a-- there's a very
specific word for what
we should use for this
Amos Wenger: Frapheme cluster.
James Munns: Yeah, it's a grapheme
cluster That's exactly it.
Is glyph the go word for--?
Amos Wenger: No, no, no, no, no go has runes, which
are different still. I
forget what they are.
But yeah, the question is like: how do
we deal with emojis for example?
Like text ranges will get into but emojis
are another problem because normally
operating systems... I don't know, if you
have a text editor you say, "Okay I want
Ariel font," and then you write some text.
Okay, it's gonna render the text in Ariel
But if you have an emoji in the middle
it's gonna use the system emoji font
Which is a color font in the middle of
all that. It's gonna have different widths
different like dimensions, metrics, whatnot.
And it's gonna do that seamlessly and
that's like emojis show differently on
different platforms, and that's because
the operating system knows how to render
emoji. But Fusion does not. It will just
render--
James Munns: Excellent, excellent.
Amos Wenger: garbage and it
does not support color fonts either, so
even if you bring your own like Noto
color emoji font, it's just not gonna
know what to do with it. I don't know what
FreeType actually does with fonts like
these but... It just doesn't work.
Fortunately Noto Emoji exists and it's
a version that is not a color font.
It's a... what word i'm looking for...
James Munns: Black and white outline kind of?
Amos Wenger: It's a monochrome emoji font... in, like, in
text style. Another little detail:
I have nerd fonts in my terminal and in
my like text editor with um...
This is Oh My Zsh or something?
This is actually Starship. Starship.rs.
James Munns: Ah, okay.
Amos Wenger: Yeah.
And... same issue here.
James Munns: So is this using
ligatures or things like that?
Amos Wenger: Uh, it's not it's using private use area
characters. So again, they would just
show up as like the
'don't have a glyph for that'
glyph in DaVinci Resolve
James Munns: There's a word for that Unicode.
It's not toasts, but it's
the square that has like the--
Amos Wenger: Character replacement box or something?
James Munns: Yeah, yeah, but it's just got the the hex
value of what-- what
exists inside of that space
Amos Wenger: I wish that were the case.
That would be so much easier. But at least
not in-- not in DaVinci Resolve. On my
website I have a patched version of
Berkeley Mono and so there's no need to
do anything special. And also in CSS you
can have like for this range like you
know the range for nerd fonts characters.
Just use that other font. It's all built
in. The web is actually very good at
rendering text surprisingly turns out.
Yeah, there's a symbols nerd font which
only has the symbols and so technically
if you were to go through some text that
I captured from the terminal you could
detect what ranger are an emoji because
an emoji is not just one right?
That's never the case
It's time for a game.
The only way to learn is to play. Let's
play a little game called.
What's the length of this string?
James Munns: Oh, no. This isn't quite as bad as what
date is this in JavaScript?
Yeah, this one's pretty bad
Amos Wenger: There's a bunch of like .wtf websites that
popped up recently. One of them was
about Python fstrings the other one was--
Yeah yeah, the date-- the
JavaScript date thing is
James Munns: just like Sam
Who did the
JavaScript date library thing.
Amos Wenger: Yeah, so we'll start easy.
We'll start with a dollar sign and we'll
count four different things.
We'll count grapheme clusters, Unicode
code points, UTF-16
code units, and UTF-8 bytes.
James Munns: Uh, let's see if this is an ASCII dollar
sign, it would be one grapheme
cluster, one unicode code point, one UTF-16
code unit and one UTF-8 byte?
Amos Wenger: Yes, that is correct.
So that's four points for you.
James Munns: Because we're in the ASCII subset of of Unicode
Amos Wenger: Do you want a bonus point to
tell us what the code point is?
James Munns: Oh, I don't know.
No, I'd have to Google it.
Amos Wenger: Well, it's plus 24.
It's hex 24.
But yeah, that's all correct. It's exactly--
the length of one dollar sign is one.
Everybody agrees.
Honestly, we should
have stopped there probably
Next one.
James Munns: Oh no.
Amos Wenger: A crab emoji.
James Munns: Crab? It is gonna be one grapheme cluster
One Unicode code point. It's gonna be
Amos Wenger: Correct
James Munns: one UTF-18 code unit and it's
gonna be three UTF-8 bytes
Amos Wenger: Before you go any further, I want to tell
you what the code point is so that it can
help you. The Unicode code
point for the crab is 1F980.
James Munns: 1F980 yeah, so it'd be... I mean it's gonna
be three UTF-8 bytes, right?
Amos Wenger: No, it's gonna be two UTF-16 code units
and four UTF-8 bytes.
Because the code point is too big.
It don't fit. It don't fit in one UTF-16
code unit. So you have a surrogate pair.
It's called that way.
James Munns: Oh goody
Amos Wenger: And uh, a surrogate pair-- I just, i'm saying
it weird and now I have the the fruit, the
pear, like a surrogate pear on my mind.
James Munns: This is my surrogate pear!
Amos Wenger: Amanda please edit a pear.
James Munns: It's a surrogate pear.
Amos Wenger: Exactly. I should
have used a pear emoji.
[[James Munns] Yeah, there you go.
Amos Wenger: And uh for UTF-8 it's
a little bit more complicated.
There's very good tools, there's UTF-8
visualizers. I linked two of them.
They're gonna be in the show notes
because Amanda is the best. Yeah, we have
four UTF-8 bytes because again, it
doesn't fit in three
James Munns: Yeah, I always forget that the UTF-8
format is slightly more complicated than
like the Protobuf varint format. Like
normally in Protobuf varints and actually
the same way Postcard works, you just use
one bit as the continuation bit whereas
in UTF-8 they use what is
it four bits in the first
byte and then two in each of the
continuation? Like it uses
significantly more for metadata
Amos Wenger: That's what it seems to show.
Yeah.
Actually the other UTF-8 visualizer
seems to indicate that-- you get four bytes
set to one and then one zero.
Do you see it in the corner?
They disagree on how many leading zeros
we have that are part of the...
James Munns: Because it uses how many ones you have
before the first zero to basically count
there. So like the 1111 zero, the zero is
the terminator for the string of one bits.
So it does actually take five bits to say
there are four bytes in this sequence
Amos Wenger: But why do we need to know how many bytes
there are in the sequence?
Can we just use continuation bytes?
Ah, well, no, okay.
Well, no.
Yeah.
James Munns: Yeah, I think it's
probably a detection thing.
I'm not entirely sure. It's like a clear
way that yes, this is
the beginning of a thing
Amos Wenger: UTF-8 is fun.
The hard part of UTF-8 is... I don't know.
Normalization and stuff. Big Unicode
tables, stuff like that.
But UTF-8 itself is just,
it's just pretty neat. All right.
Next, next question.
James: a pair, an emoji pair of scissors
How many grapheme
clusters, how many code points?
James Munns: I'm gonna guess all the same numbers as
the crab Just to be wrong.
Amos Wenger: Just to be wrong, of course!
James Munns: Yeah... yeah, yeah.
Amos Wenger: So this time there's not one
there's two Unicode code points
James Munns: Excellent, but it's a
big Unicode code point
Amos Wenger: Has two UTF-16 code units, but six UTF-8
bytes for some reason. This time each code
point fits in one UTF-16 code unit, but
UTF-8 goes from like-- how many
bytes do we have? Four to six.
Because... Because it's like three and three
I'm assuming? I don't know, I haven't
decoded those. Do you know why there's two
code points, you have any idea?
James Munns: Is it the left half of the scissors?
Is it just two knives glued together and
that's actually scissors?
Amos Wenger: That's not as out there
of a guess as it sounds
James Munns: It's not, it's not. UTF-8 does some weird
things sometimes.
Amos Wenger: But no. It's an image variation selector. The
problem is that the pair of scissors
existed before emoji. So if you just do
pair of scissors, then you get the text
version is sideways for some reason.
James Munns: Get the lo-fi-- lo-fi text scissors.
Amos Wenger: The monochrome outline version the the Noto
Emoji without color. And then you add an
image variation selector after that, and
you get an emoji pair of scissors with
colors and everything instead.
James Munns: Is this how like
things like the-- oh, okay.
I won't spoil it
Amos Wenger: You can also select the other-- like this
is only one of the selectors. You can also
select the text version if you wanted to.
And so I guess if you only have 2702 like
only the pair of scissors, I guess it's
not wrong to render it as emoji?
I'm not exactly sure
James Munns: Yeah, there's a bunch of stuff that like
it's allowable fall downs in Unicode. We
need to just have Manish on is what we
need to have like... Someone who's in the
Unicode Consortium and knows all of these
things of like, there are certain things that
are allowable and then there's certain
ways to say something's not allowable.
Yeah, this is beyond my ken.
Amos Wenger: Well, don't worry
because we're reaching the end.
Here's the last one.
James Munns: This is the best one.
Yeah, because this is-- Okay, okay.
I know that the like combined family
emoji is a modifier because it's like
combining multiple emojis.
I don't know if it's quite like person
plus person plus child plus child. You're
giving me the like Japanese symbol
version. Not the like color emoji one that
has like cartoon looking people
Amos Wenger: I thought it would be
cartoon looking people.
I don't know why it shows this
James Munns: Interesting.
Amos Wenger: Google Docs is just being weird.
James Munns: So i'm gonna guess this is i'm guessing
this is four grapheme clusters and like
seven unicode code points
and like 14 bytes
or something like that.
Amos Wenger: Okay, you're already wrong on the
first one. How many
shapes do you identify?
James Munns: Is it four? because it's the four adults
and two children or it's usually two
adults and two children
Amos Wenger: How many clusters do
you identify?
James Munns: In a box?
Amos Wenger: That's just the one.
James Munns: I don't know.
Amos Wenger: This is one cluster, right?
It's one, you can select it.
James Munns: Oh, right, right.
Amos Wenger: It's one-- like if you do left or right
you're not going to be in the middle of
that thing. You can be on the left of it
and on the right of it
It's one grapheme cluster
James Munns: Doing like the zero width join of
multiple emoji characters, it becomes
one cluster is the join of all the things
that have been joined together. So I guess
that's why you were saying the difference
between grapheme clusters and code points...
because it's one cluster with like five
Unicode code points?
Does that include the zero width joins
and stuff like that?
Amos Wenger: I have- I have to zoom in
my speaker notes to cheat
No, there's so many so many freaking code
points in there. So because you have to
have some sort of joiner, right? Between them.
James Munns: The glue that holds the family together.
Amos Wenger: Exactly and you have to repeat it between
everything. So you actually have like dad,
zero width joiner, mom, zero width joiner,
child A, zero width joiner, child B and then
a selector- a variant selector. You have
eight Unicode code points because like
four family members, three joiners,
one selector. And then you have
James Munns: 28 bytes
Amos Wenger: And then you have
like however many code units and bytes it
takes to encode this freaking mess.
James Munns: Plus there's variants for all this like
you can have adult man plus adult man
plus female child plus male child or
whatever for all of it. Like you can
customize-- and then I think you can also
add skin color variants and things like
that? You can make the
family emoji a lot of bites.
Yeah.
Amos Wenger: Yeah.
Yeah, you can. I actually have no idea if
the renderers allow all combinations or
if they just hard-coded a bunch of them
but yeah. So in JavaScript if you have an
array literal and then you do ellipsis of
like that string literal family emoji, you
see the individual code points kind of
you see the male face here with joiner
female face here with joiner, etc, etc.
We're almost there.
We almost have something that works
We have only 10 slides left.
How do we identify emojis in a string?
We have to know the boundaries of an
emoji so that we can tell
Fusion to use our emoji font.
How do you do that?
James Munns: Uh, since you have Rust code you just do
not is ASCII, is ASCII
character or something on each?
Amos Wenger: No because emojis are like a very small-
relatively speaking- subset of all
non-ASCII things. We still want like
diacritics, we still want... There's a lot of
things that are not in the emoji font.
James Munns: How do we do this?
Amos Wenger: How do we do this?
First of all, I was pleasantly surprised
to learn that there's a standard browser
API. Also, this is not Rust code.
This is all TypeScript.
Uh, this is standard
segmenter API in the intl for
international namespace thingy.
You have to give it a locale. So I just
choose English but you can segment it.
James Munns: Wait a minute.
When do we go from Lua
When do we go from Lua or Python or Rust
or C++ to TypeS cript?
How did-- how do we bring another language
into this uh, this cluster?
Amos Wenger: Well, we need to skip ahead so I can show
you. But basically the way it works
Is that on my website-- like I want to make
a video or rather my editor is now tasked
with making a video. All they have to do
is go on my website, run my website
locally and then there's a little button
on each code block. And it opens this
little preview with a transparent image
of my face for scale
In that-- this is still in the browser. So
you click the button and it whoop you
click the button and it
opens this preview here.
James Munns: Oh, so your website...
Amos Wenger: And then this is content editable.
Yeah, you're in the browser.
You can still edit the thing--
James Munns: So your website is generating Lua
the describes text.
Amos Wenger: Exactly.
James Munns: I love it!
Amos Wenger: Yeah, it could be done server
side. But here you get to remove things
that are not relevant because actually
Fusion is really slow at rendering text.
So you can just remove a paragraph that
you don't need or like a code block or
something. You can highlight something
like select something with a mouse and
then press space and it's gonna dim
everything else and just highlight that
bit. That used to take forever. In Fusion,
I used to have to like add rectangles for
masks and stuff like just to highlight
some bit of code, but now it's really
just that. And then you you copy it
without the highlight and with the
highlight, so you have two different clips
and then you do a cross fade between
those two different clips and it works
James Munns: So this means your
website's now a compiler?
Amos Wenger: Essentially, yes, I think so?
James Munns: Excellent!
Amos Wenger: Basically it was really hard to
like first you segment things into
grapheme clusters, and then for each
grapheme cluster you run that-- there's an
emoji regular expression, small version
that identifies emojis and then if it's
an emoji you get the Noto Emoji font. If
it's something that looks like a nerd
font thing, you get the nerd font symbols
font. I discovered that Berkeley Mono is
missing some Greek symbols. So it's just
things that-- there's a lot of fallbacks
everywhere, and it takes a freaking dumb
font renderer for you to realize that
you're missing them.
James Munns: There's a lot of glue
Amos Wenger: There is. It's still kind of a nightmare
because the size of everything is wrong.
I have an actual preview here. Uh, which
I you would need to zoom in a lot to see
what's going on. Amanda! We'll do it in
post! But basically the emojis don't have
the same width as everything else if you
had an emoji in a table... That's a problem
for a lot of terminal emulators and code
editors and whatnot. Yeah, if you have an
emoji somewhere you can misalign
everything after it, but at least it works.
James Munns: I ran into this not with uh... not
with emojis but Departure Mono in odd
font sizes- so like 17
point font, 19 point font-
there's a specific-- it's not even an emoji
It's the Unicode like filled arrow and I
use a diagramming tool called Monodraw
which makes diagrams and has arrows and
lines and stuff like that, but it it
makes it as text. But when I copied and
pasted it into my editor, all of the boxes
that had the arrows left or right in them
would become unaligned, even though i'm
using a monospace font in a monospace
text editor. But I realized that if
I change my font from 17 to 18 or
whatever, then all of a sudden they align
again! So, luckily Departure Mono is an
open source font, which means I went and
opened a GitHub issue and i'm like, "I
don't-- I don't know how fonts work, but
something terrible is going on..." So it's
not just emojis that run into this
problem. A character, even a monospace font
one character width
is not one character width.
Amos Wenger: I mean maybe...
For you, it's probably just
precision and calculation.
I think it's not right.
James Munns: I'll send you a
link to the issue, and I guess we'll put
it in the show notes,
but it's a fun error
Amos Wenger: But I'm pretty happy with
the results of my thing.
You can see this is a screenshot of
Fusion. You can see that there's an emoji--
The emoji is is rendered as an emoji in
the text input on the right like you see
the crab emoji with colors and everything
but that's because that's a qd text box
On the left you have the preview from Fusion.
James Munns: So the operating systems rendering
can do it, but Fusion's
rendering can't do it.
Amos Wenger: Yeah, but it's very funny because now I
understand why we have this futuristic
Matrix green outline of the text. I put it
like to the left of the canvas on purpose
so we could see the outline and see that
It's actually vector.
Because it is actually vector.
This is what happens. It's using FreeTtype
to get a vector shape out of a font and
it's very slow, because for every single
character-- I imagine what the Text+
node is doing is not too far from their
Fuse sample that we saw earlier, which is
a very basic and bad way to render text. I
imagine that's why it's slow because it
shouldn't be that slow to render text
James Munns: I guess it's one of those things you
don't run into that-- there aren't that
many people putting a lot of text... on
there. Like, a little text.
Sure.
I wonder how bad people's... the credit
sequences are to render for a lot of
people. I feel like that's the most text
heavy part of an entire movie if people
are using this for movies.
Amos Wenger: That's true. I imagine there's specialized
software for that like they do the render
elsewhere. Like i've been watching a lot
of Corridor Crew, the VFX people react to
VFX in movies and they use a
patchwork of techniques. It's such a mess.
It's so practical.
It's like... I don't know. As a developer,
It hurts me, but I
also see how... I don't know.
It's fun to see them make
things happen with not much.
Anyway, it all works. It's already like--
James Munns: Hell yeah.
Amos Wenger: my solution for generating
Fusion node graphs. I've already used in
the video "Introducing facet: Reflection
for Rust" and uh, it's great! It's like that
plus wearing a cap so that I-- shooting
with green screen again is such a huge
time save. I like started working with
Sekun for video
editing who's credited now
James Munns: Wait, is the cap just to keep your hair from
Amos Wenger: Yes.
James Munns: causing like
fringing with the green screen?
Amos Wenger: Yeah.
Yeah, yeah, it's not a style choice.
It's not a statement.
James Munns: Ah. Nice.
Amos Wenger: It's just for--
James Munns: It's like the video version of
hair nets for food prep.
You know what I mean?
Amos Wenger: Did you ever watch any Any Austin video?
The guy who's like
James Munns: Austin video?
Amos Wenger: Any Austin.
Amanda Majorowicz: I have yeah.
Amos Wenger: No.
Yeah, he um, I haven't
Amanda Majorowicz: I like that guy.
Amos Wenger: He does restaurant reviews of Skyrim, he
asked like where does the
water come from in Zelda?
He's like can a 747 really land in the
GTA5 airport, and the answer is no.
Oh, I think I've seen clips from... yeah.
Amanda Majorowicz: He's also from Minnesota.
Amos Wenger: He is!
And he's always wearing hair clips.
Amanda Majorowicz: It's so great.
Yeah, hair clips, reppin'.
Amos Wenger: I stopped feeling bad about-- my wearing
a cap when I saw him wearing hair clips.
It's such a... I don't know. But who
cares, the content is amazing.
He's going around Delfino Plaza wondering
which NPC's have a job and
which don't.
James Munns: Mario Sunshine?
Amos Wenger: Yeah.
James Munns: Okay.
Amos Wenger: Yeah, he's measuring the unemployment
rate in various video
games. That's what he does.
It's amazing.
James Munns: Alright, now
I have a bunch of
YouTube videos to go watch.
Amos Wenger: Yes you do.
James Munns: Is there a special key combo
I can press on your website to see this,
or is it only on the local render view?
Amos Wenger: It's only on the local render view and
the TypeScript code that does all that is
not even bundled in the production
version. So... I'm sorry
James Munns: Boo. In case everyone else wanted their
immediate Fusion node
text generator application
Amos Wenger: It's so custom made...
I thought it would be much more
work. I must admit, I vibe coded a bunch of
this. I had to stop when-- like for the two
details I mentioned, like for the color
and the text extense thingy.
I tried to debate with Claude code about
Like how Unicode works
and I was like: okay.
I have to actually understand
the code that's going on now.
James Munns: You don't have like tree sitter in there
somewhere already or
is it already formatted?
I guess the website's already doing
syntax highlighting Using some kind of
Tree-sitter plugin or something? And then...
Amos Wenger: Yes.
James Munns: You get the output of that and then
you know the colors and then you--
Amos Wenger: Exactly.
James Munns: you generate from that. So it's already
done by your website. So you just hop on
that train and just spit out from the
colored text to generate the
Lua that's necessary for this
Amos Wenger: Yes, so the various code blocks, including...
highlighting code is not that exciting.
Everyone's doing it. The thing I'm really
excited about is that I'm capturing
terminal sessions with color.
That's much harder. In both cases they end
up being a soup of like i tags I think
with different classes and those
different classes have different colors
in light mode and dark
mode. And so I force dark mode
I think in that preview thing. And I use
computed style map, I
think, to grab the CSS colors.
James Munns: Nice!
Amos Wenger: I think I had a whole video or
episode-- at some point,
I talked about how display p3, blah,
blah, blah... I went back to sRGB for all
the colors just because of that
James Munns: Yeah, because it works in more places or?
Amos Wenger: I moved from Catppucin to Melange and
from Iosefka to Berkeley Mono so...
James Munns: Mmm. For fonts?
Amos Wenger: Lots of changes. Yeah.
James Munns: I think it was you who for a long time
your Twitter profile was just, "The font is
Iosefka" or whatever and I know-- I think
Eliza started doing the same thing for
like, "The font is Berkeley Mono."
Amos Wenger: Mhm, yep.
James Munns: think it was when you were maybe starting
to make videos and it was just probably
the most common question everyone had for
a video was, "What is that font?"
Amos Wenger: I still get it.
Yeah.
Yeah, you make like, uh, I don't know 30
minute video about something intense and
people are like, "What's the font?"
James Munns: There's an old drawing-- or, not old,
it's still current. There's a tool on iPad
called Concepts which is really nice for
drawing things with like an Apple pencil
and things like that if you're drawing
actual diagrams and stuff, Concepts is
excellent. But every time I posted
something where I was excited, I just
figured out how something works and I
drew a diagram of it so I could remember
how it's supposed to work- the only
engagement I ever got on that was, "What is
that diagramming tool?"
And-- so I was just like, I need to just put
"It's Concepts," in every-- like no engagement
with the actual thing that I figured out
or I was excited about, it's just, "What is
that diagramming tool?"
JAmos Wenger: It's really hard.
James Munns: And so I feel deeply for the-- like you
spent all this time on it and it's like, "Oh,
what's the pretty font?" Yeah.
Amos Wenger: I occasionally, very occasionally...
it's not like everyone
asked me, "How do you be famous?"
No, it's like- occasionally people ask me
about making content and videos like that.
And I have no idea what to tell them
because even like at the scale that I'm
at, I'm really happy. It took a bunch of
work to getting to 50k subscribers.
I'm excited about color I figured about a
bunch of stuff about it.
I already said it's a complete flop. YouTube
algorithm is, uh, is harsh. Sometimes it just
doesn't want to show it to people. It
shows it to people a little bit, and if
it's not an instant hit, if everybody's
not immediately into it,
it's like, "Nah, it's not working.
Sorry." You get it
James Munns: Yeah. The other thing that I see people do
all the time is changing the title like
seven times in the first like week that
it's out, and I'm sure that's just like
engagement hacking of seeing who clicks
on it immediately and things like that or
they change the thumbnail and whatever. We
just kind of publish it and we go,
"Oh, well, it'll do as it does."
Amos Wenger: Yeah... Well, that's all
I have for you today.
James Munns: I'm glad you got text pretty. At some
point, your website is just going to
become the ultimate YouTube authoring
tool. Like it's just going to generate the
project for you and do
the recording at some point.
Amos Wenger: I've kind of given up the idea that it's
going to be useful for someone other than
me. At this point. And
it's been liberating.
James Munns: Yeah!
Amos Wenger: But like I don't expect anyone to
use the Fusion thing. If anything, I'd like
to move to something else for motion
design, but I don't know. I'm just so
embedded into Resolve right now.
Anyway, that's all!
James Munns: I love it.
This episode is sponsored by Depot,
the build acceleration
platform that's on a mission
to make all builds near instant.
If you're tired of watching your builds
and GitHub actions crawl
like the modern day equivalent
of paint drying, give Depot's GitHub
Actions runners a try.
They're up to 10 times faster with
unlimited concurrency,
faster caching, support for
Linux, macOS and Windows,
and they plug right into
other Depot optimizations
like accelerated container image builds
and remote caching for
Bazel, Turborepo, Gradle and more.
Depot is built by developers who are
tired of wasting time
waiting on builds instead of shipping.
It's made for teams
that wanna move faster
and stay focused on
what actually matters.
That's why companies
like PostHog use Depot
to cut build times from over three hours
to just three minutes,
saving tens of thousands
of build hours every week.
Start your free seven
day trial at depot.dev
and let them know we sent you.