Blackmagic Fusion is weird

James Munns: I am so ready. I am so ready. Amos, I am so ready. Well, I'm trying to look at what your slides are, but I've turned my my Jitsi performance all the way down because it was struggling and now yeah, the letters are--

Amos Wenger: I need to show the other screen. Yes. So. James. I have-- I have exactly one worry It's that we don't talk about things that are relatable enough on this podcast. And this I think-- I think that people want to hear about technologies that they use every day and this is why -- today's episode is about migrating from AppleScript to Lua. Something that I think everyone can relate to

James Munns: Two languages so many people have written I would argue a

Amos Wenger: I would argue a bunch of people have that exposure to Lua if only to like mod a game or something

James Munns: Yeah, Lua. I feel like the two places it pops up are for me at least-- well I guess three because video games, Nginx and a lot of-- it's one of the few languages that people will put on embedded systems and Has a decent chance at running it like very very small embedded systems. So like it's a weird palette of places that I've seen Lua. But I guess it makes sense of like I need fast scripting glue kind of stuff.

Amos Wenger: Yes. The actual title of the episode is "Black Magic Fusion is weird" and that's an understatement. I'm so excited James because finally we're gonna have a short Amos episode because I only-- can you guess how many slides I have?

James Munns: You said that at the front of some episodes that have gone a very solid hour.

Amos Wenger: I only have 61 slides. So this is this is gonna be

Amanda Majorowicz: Whoopsie, again again-- quote.. I can quote this. I can splice it all together Be like, "It's gonna be short! I only have a bajillion slides."

Amos Wenger: But you'll see there's this like little games we can play, we can try to guess things, there's like text encoding going on. It's very fun, but mostly it's because I can't help myself. And because I... We have a special guest today. His name is Sherlock, he is my white cat. He's uh, he's meowing at the window and I'm not gonna do anything about it

James Munns: Hello Sherlock!

Amos Wenger: He's gonna calm down and also cannot hear you. I cannot be helped but I can be nerd sniped and so I.... I'm going to do something about the cat.

James Munns: What's the French onomatopea for cats meowing?

Amos Wenger: Miaou, it's M-I-A-O-U, "miaou" Yeah, so it's kind of the same sound as meow. Yeah.

Amanda Majorowicz: You almost pronounce all of the vowels actually... surprisingly.

Blackmagic Fusion is weird

James Munns: So Amos, why do you find yourself understanding Lua for Black Magic Fusion?

Amos Wenger: This is a sequel episode actually Previously on Self-Directed Research. See this is why I have 61 slides, because I have a slide "Previously, on SDR." James: Where can people find the slides?

James Munns: They can find it on sdr-podcast.com/episodes

Amos Wenger: Good luck editing this Amanda

James Munns: Amanda. I'm sorry. We're gonna be a nightmare

Amanda Majorowicz: I'm ready

Amos Wenger: So, previously on Self-Directed Research I talked about how I make videos because this is not a podcast. This is a thinly veiled marketing vessel for the both of us. And so I explained that I can make screenshots of code and import that into DaVinci Resolve and then in Fusion I can use like a 3D Keyer node to remove the background so that I can then composite that with some shots of my face and other backgrounds other elements and animations and whatnot. But that it's not ideal. Because you get kind of messy edges all around the text. It's already composited, the text is already composited on something, some color and so when you remove the color even when you choose your color wisely or black and you're putting a dark background. It's just not ideal. We talked about how actually in some browsers like Safari, we recently learned we had viewer mail but not-- listener mail? Like, we got some mail! Someone said, "Oh actually you can also do that in Chrome with the Puppeteer API and it's much simpler." But I would like-- the whole episode was actually on Safari. There's a capture screenshot command that takes a DOM node and then you can get that entire node no matter how big it is with completely transparent background and it's perfectly crisp and clean and nice

James Munns: Just what you wanted Totally unintentionally.

Amos Wenger: Yeah... and so then we talked about AppleScript and Automator and all sorts of things like that. And how AppleScript is weird. It's a very entertaining episode actually. When reviewing it I was like, "Wow, we're-- we're interesting."

James Munns: This is the-- this is the "automating macOS" one if I remember correctly...

Fusion can render text!

Amos Wenger: Yes, it is. Here's the thing: some people have pointed out, and they are correct! And I am helpless that Fusion can render text, of course because... you can have titles you can just-- you can have a... what's called a Text+ node, so you can even get some formatting in there. So here we have cantaloupe salmon in the Berkeley Mono.

James Munns: Is this what you ate for lunch or is this just a like a diceware...

Amos Wenger: It's the name of macOS colors from the picker Because the next thing I want to show--

James Munns: Thos are two pretty colors, yeah.

Amos Wenger: Yeah is that you can change a bunch of things in the Text+ node. You can change the size, the tracking, line spacing, various alignment properties, and you can animate writing on the text so that it comes progressively as if someone was writing it. And you can add multiple shading elements. And something that took me forever to figure out: you can also have character level styling. So the way it works is the least intuitive thing in the world. So you first type in some text in your text node and then you right-click in the text control in the sidebar inspector thingy, you right-click, you click character level styling and then you go in modifiers... And then in modifiers you then select- not in the text input that you just clicked, or the one that's a modifier that's an almost exact copy of that- you select in the preview window So you do like a rectangular selection in the preview window and you get those little green sci-fi highlights Like, I don't know...

James Munns: Character divider boxes?

Amos Wenger: Outlines, yeah yeah. Yeah, and that's your selection. And so much of this will make sense in a minute, but when I first discovered it, I was quite puzzled. But yeah, you can-- in the same Text+ node you can have different colors, you can have different sizes, you can have different-- all the controls you see here. Even alignments and--

James Munns: Instead of trying to make two text boxes that have different colors and you just like shove them next to each other and hope that they're aligned.

Amos Wenger: Exactly, that would be my nightmare. Yeah. Editing this is really bad. Once you've added styles, if you then click in the middle of salmon for example, and you try adding more letters It will be the right color So it kind of extends the range that is styled but if you copy and paste for example, it does not retain styling or anything. This is not WYSIWYG, This is not what you see is what you get It is what you get is what you get.

James Munns: For anyone interested in debugging that can go back to our episode about clipboards where you could probably use one of the tools we mentioned in there to figure out exactly--

Amos Wenger: We're getting to-- James! No spoilers!

James Munns: Am I spoiling things? Okay.

Can we automate that?

Amos Wenger: You absolutely are, but I like that. I think that that's the sign of a well-designed slide deck is that you can guess where I'm taking you. So can we automate that? Well, let's talk about the different ways DaVinci Resolve and Fusion can be automated. So: Yes. Yes. The short answer is yes, but there's many different ways you can do that. One way is Fuses. And this is essentially... Yeah, you can just script this thing with Lua. It wasn't always there in Fusion, I'm just now realizing that 'fuses' and 'fusion' are the same word. Wow. Coffee is a great drug.

James Munns: They're like... I forget what -phone it is. It's not a homophone, but they're like 'fusion' and 'fuses' are... I don't even actually know if they share a root for anything, because fusion seems nuclear to me and then fuses sound electrical to me.

Amos Wenger: Maybe? I don't... do we have time for an etymology break?

James Munns: No, I don't-- keep going. Well, we gotta run , we gotta run. Amanda will put the Wikipedia page in the links below and we'll never know the answer until we go and review the episode.

Amos Wenger: All right. All right. All right, so I'm talking about Fusion and Resolve kind of interchangeably and that's because today they are integrated. DaVinci Resolve is like three or four different programs in a trench coat. One of them is Fusion. One of them is the audio thing... what is it called? Fairlight. The Fairlight audio tab. There's just different tabs or pages in the application. But they used to be or they also still are sometimes-- Fusion I think is still available as a standalone thingy. And Fusion is very very old. It was made for MS-DOS in 1987. It's older than me! By Eyeon.

James Munns: I thought it was a newer-- like for some reason, it registered as like-- it's probably because when it became a public tool or something like that, but it registers to me as the new one versus all the other ones.

Amos Wenger: No, it's the first Windows version was in 1996. They only added a timeline so like you could do animated renders in 1997. And then they added Lua Fuses in 2007. I don't remember-- I don't have it in my notes like what year Blackmagic, which is the company behind DaVinci Resolve, acquired the Fusion software. I think it's 2014 off the top of my head. But do not quote me on that. Now Fusion has been used in a ton of big movies and TV shows, including Sin City. I don't know if you... You know Sin City? And Lost and American Horror Story. I cannot not say American Horror Story in the FX voice. Like, "Next week on American Horror Story." That's just how I have to say it You can tell that Fusion is old because we're looking at this Wikipedia... Like early Wikipedia clone... like what is it called, MediaWiki?

James Munns: VFXPedia.

Amos Wenger: Yeah, called VFXPedia. Well, we're actually looking at as a mirror maintained by the wonderful people at We Suck Less, a phpBB forum, hosted at steakunderwater.com, and which shows you a lovely SQL error if you try to register. But it still works. I got in. I got us a listing of what a Fuse looks like. And you can actually do quite a bit with it. In this example, you can draw some text. So this is what we're interested in actually. It's just Lua code as a font manager. You can scan the directory, you can assign things. And drawstring here that we see on the screen is-- you can get the slides at sdr-podcast.com/episodes or on YouTube if you're if you're listening to that elsewhere, you can just go on YouTube and have the slides synchronized.

James Munns: I do upload the videos to Spotify as well.

Amos Wenger: We do?

James Munns: For a very long time now!

Amos Wenger: I don't use Spotify.

James Munns: So drawstring, not like the pajamas.

Amos Wenger: Drawstring is not a built-in. Drawstring is a Lua function that calls out to FreeType. And it has interesting comments like: 'Get the character or glyph' which is followed immediately by a call to something called bite. So that does not... inspire confidence In terms of text--

James Munns: It's-- it rings from a time before Unicode was... a thing.

Amos Wenger: Yeah... Then it says 'Not really sure why we multiply this by 10, we just do' smiley face emoticon. Yeah, and then right

James Munns: Love that.

Amos Wenger: Yeah. And then right before, the crucial bit: there's a comment that says 'I think this renders the shape we are interested in' and this is the code sample that Blackmagic gives you if you want to learn about Fuses. They also have like the unofficial PDF documentation as of July 2022. So it only took you know... you do the math

James Munns: 25 years?

Amos Wenger: No, 2007 to 2022 but they acquired it in 2014- like okay sure. So Fuses could work. Right? Because you can you can do arbitrary rendering. But what are we gonna do? Are we gonna like design a control where we paste in some- some Rust code and then write some Lua code to parse the Rust into an AST and like... whatever? Or like have some sort of markup language? Like parse the HTML instead and then render that or...? There's still questions.

James Munns: I don't know. I watched you fall down the hole of AppleScript. I really... you could go so many ways with this.

Amos Wenger: Yeah, but no. I decided not use Fuses. In fact, I got the source code and I got the manual text rendering and layouting thing as a last pass before we recorded this. I didn't actually pursue that avenue at all. Another way you can extend DaVinci Resolve is OpenFX filters, which is actually a standard as far as I can tell. And it's... a bit like VST, but for video. VST is for-- a standard for audio effects.

OpenFX filters

James Munns: Yeah, I like audio effects in digital audio workstations and things like that. The one person I know who makes VSTs kind of hates VST or like every platform makes it hard to keep things portable. But it is the standard way of writing audio plugins basically.

Amos Wenger: Yeah, and it's a really fun little microcosm because you buy a DLL essentially from someone and then they always have these wonky UIs, I don't know. It's a fun little universe. OpenFX filters: here's the, here's the read the docs Sphinx docs guide thingy. Here's what it looks like. It's the same thing as the Lua thing, but it's in C++ so you can actually do per pixel thingy. They're using in Fusion for like blur, box blur, directional blur, Gaussian blur, lens blur, mosaic blur, radial blur, zoom blur: all sorts of like color space transforms. All the basic visual effects are actually OpenFX plugins that are shipped with DaVinci Resolve. So it's not the best solution for that. I don't even know how it would work, because you would have to have some sort of like color generator node and then drag your effect on top of that and then you would have a separate kind of-- yeah sure you. I mean, it's native right? You could ship whatever you could have your own copy of FreeType in there. You can-- you could do all the rendering all the, um... you could link against some Rust code to actually do the highlighting... you could do everything. But at the end of the day, it's... it's awkward to composite because your output is something like a 4k canvas. Right? If you have code that is taller than that, then you need to have the scroll parameter or something be a parameter of your effect, of your plugin. You can't take that and put it into Fusion and like have it be part of the graph and animate it. I don't know, it's... I have not pursued this road either.

James Munns: If you wanted to do a giant Star Wars scroll of your code...

Amos Wenger: I mean it happens more often than you think. I have a lot of code.

James Munns: I believe it.

Amos Wenger: Can you write OpenFX filters in Rust? Of course you can someone made ofx-rs. And it's exactly what you'd expect. It's ugly because it it has to interface with C++ and uh... But it's better because it's-- it's more readable to me, let's say, because it's Rust.

James Munns: Hmm. Is it using like CXX or something like that?

Amos Wenger: I have no idea. I just checked that it worked, glad that it existed--

James Munns: That's the nice thing about of a crate: you don't have to think about it. It just worked and I did--

Amos Wenger: I don't have to.

James Munns: I'm sorry if I just cursed you with the desire for knowledge.

Amos Wenger: No, no, no, I'm getting wiser James. And we are only halfway through. Another thing you can do is that resolve has a scripting API. So that's another place where you can use Lua or Python interchangeably. I'm not exactly sure how they did that but... and the documentation for that is non-existent. They have a readme.txt somewhere and someone parsed it and generated like a nice website out of it. And it still has almost nothing. I think the only thing that I've seen that uses it- that I've used- is AutoCut that does something like export your whole project to a WAV file, then analyze it elsewhere, generate cut points and like basically cuts out the silences. And it does that by using the scripting API to tell Resolve: Okay, move here. Do blade cut. Select this etc. So that's what it's for. It's for like moving clips around on the timeline, changing properties, automating stuff like that. So I imagine like big productions, if they have to mass retag some clips or something, that's what they would use.

It's Lua all the way down!

Amos Wenger: But you know what's fun It's that everything in Fusion- like every scene- is actually Lua. So this is also a callback to- and you called it James- but this is a callback to our clipboard episode. If you go into Fusion and you copy the the Text+ nodes that we were- like that I was showing earlier- you can see it doesn't immediately register to me as Lua, but it is. Actually--

James Munns: Okay, yeah, I would say Lua-- I've had to tweak a couple times but I'm not really fluent in Lua, but it's got a very like somewhere between JavaScript and Python kind of... kind of vibe to it.

Amos Wenger: Lua's weird. I had to use it a bunch when I worked at itch and I never really... just the one index arrays I just never could But it it has a jit. It's light enough to integrate, it's been around forever... Yeah, there's a reason it's almost everywhere. And yeah the scene description- like the node graphs in Fusion are Lua. So when you copy them you get a bunch of text readable things which explains a lot about the performance overall by the way, and the stability of the software, which is sometimes not ideal. But it's also very nice because you can see that cantaloupe salmon is here in the character level styling node. Kind of need to zoom in a little bit. There you go. And that's-- for the colors, we also have something that is pretty easily reversible. I think what happened there, the nerdsniping process, is that I just noticed that you got text out of it, and I figured you could probably paste it back in or like generate something and paste it back in. But I had no idea-- I saw all these numbers and I got scared and I posted it on Bluesky or something and someone was like: Wait a minute. It's not scary at all! 2401 is red, 2402 is green and 2403 is blue. As you can see. Like I put the color picker next to it And you can see: yeah, one is 255, 0.49 is 126, 0.47 is 121. So yeah, okay. So the first element in the array 2401 is the attribute number or ID or something like the property ID. Then we have 11 and 16 are start and end. And then value is the value. And you have different properties. You can have font, you can have font variants, you can have a bunch of different things. But we're just interested in colors right now. And so the plan becomes: okay, so we have an example scene, right? We've added some Text+ nodes in Fusion. We have had this complete Lua printout and we can just generate our own and then paste it back into Fusion. That's-- and then... and then profit. But there is a few issues... There's at least two important details that got in the way. Because this is exactly what I did. I'll show you what it looks like later, but

James Munns: We're falling down the hole with you right now.

Amos Wenger: Yes yes yes. Um, can you James-- Can you predict what some of the details might be?

James Munns: Well, you said earlier that when you copied and pasted it back in, it didn't have the second one. So when you did the like fancy text box formatting, it was just cantaloupe and not cantaloupe salmon or the salmon was cantaloupe If that makes sense. So i'm guessing that's going to be an issue here.

Amos Wenger: So... yes, and no. So you don't paste it into the text input. You paste it into the node graph editor. The way it works is that I generate my own entire node graph and then I have to select all the existing node graph, delete that... And then paste my own in place. But there is an interesting bug here. I didn't even think to document it. It's that when you delete the entire node graph and you paste another entire node graph: because the media output just disappeared It just never shows anything anymore in the preview tab until you switch to a different page and switch back So you don't get to see what it actually... You don't get a preview of what you just pasted in until you switch back to the edit page and then back to a Fusion.

Detail 1: what does RGB 0-255 mean?

But uh, no the uh, the first detail that's annoying is that RGB doesn't really mean anything. You need to kind of have a reference. It goes from zero to one, sure. But... what does it mean? And there's three different places we can see color And none of them agree with each other. There's the macOS color picker, there's the Fusion color control, and then there's the Resolve render preview window. And they all show distinct values. This is my favorite slide. But we can see that.

James Munns: Oh goodness .

Amos Wenger: On the preview We have 255, 175, 171. On the macOS color picker. We have 255, 148, 140. It's even darker on the-- in the Fusion inspector which is 255, 115, 115. The preview is closest to the uh, actual export. Which is good. At least you can kinda trust that. If you export the video, the color-- I did color picking on the- on the final file and it is actually what the preview is. But if you remember--

James Munns: We're gonna learn more about color spaces now?

Amos Wenger: Not a lot. Don't worry. Let me get my slide. Yeah, the macOS color picker corresponded well with the values here, which does not correspond to what you actually see on screen. So I... Honestly almost gave up at this point because I was like I don't even freaking know-- and I made a whole freaking article slash video about color: "Color is best effort."

James Munns: Mm-hmm.

Amos Wenger: Which is my worst performing video of the past 10 videos by the way. A good like humbling reminder that you know, not every video is a hit. But I like it. You think I would figure it out I would go like: is it red 709 because that's the color space of the project. Is it SRGB that would make sense for a web, that would make sense for like a system color picker. Is it linear? I don't I don't know. Uh, what I know is I assumed that the RGB CSS properties that i'm converting from on my website-- this is some code for my website-- are sRGB and i'm assuming that Fusion wants something linear. And it's it's wrong by some measure. But not enough for me to care anymore. I have-- I have spent all my cycles on this. I have no more cycles to spend and uh, it's close enough. The second detail is how to define tech ranges and this one I did not expect because it's 2025 and I keep forgetting that some software is actually a 35 year old.

Detail 2: how do we define text ranges?

James Munns: This is gonna be bytes and not glyphs and which means terrible UTF8 things can happen?

Amos Wenger: So glyph-- glyph is... glyph is making me sad angry. Glyph is not a thing really

James Munns: I'm using the incorrect Unicode word. There-- You're right, there's a-- there's a very specific word for what we should use for this

Amos Wenger: Frapheme cluster.

James Munns: Yeah, it's a grapheme cluster That's exactly it. Is glyph the go word for--?

Amos Wenger: No, no, no, no, no go has runes, which are different still. I forget what they are. But yeah, the question is like: how do we deal with emojis for example? Like text ranges will get into but emojis are another problem because normally operating systems... I don't know, if you have a text editor you say, "Okay I want Ariel font," and then you write some text. Okay, it's gonna render the text in Ariel But if you have an emoji in the middle it's gonna use the system emoji font Which is a color font in the middle of all that. It's gonna have different widths different like dimensions, metrics, whatnot. And it's gonna do that seamlessly and that's like emojis show differently on different platforms, and that's because the operating system knows how to render emoji. But Fusion does not. It will just render--

James Munns: Excellent, excellent.

Amos Wenger: garbage and it does not support color fonts either, so even if you bring your own like Noto color emoji font, it's just not gonna know what to do with it. I don't know what FreeType actually does with fonts like these but... It just doesn't work. Fortunately Noto Emoji exists and it's a version that is not a color font. It's a... what word i'm looking for...

James Munns: Black and white outline kind of?

how do we deal with nerdfonts?

Amos Wenger: It's a monochrome emoji font... in, like, in text style. Another little detail: I have nerd fonts in my terminal and in my like text editor with um... This is Oh My Zsh or something? This is actually Starship. Starship.rs.

James Munns: Ah, okay.

Amos Wenger: Yeah. And... same issue here.

James Munns: So is this using ligatures or things like that?

Amos Wenger: Uh, it's not it's using private use area characters. So again, they would just show up as like the 'don't have a glyph for that' glyph in DaVinci Resolve

James Munns: There's a word for that Unicode. It's not toasts, but it's the square that has like the--

Amos Wenger: Character replacement box or something?

James Munns: Yeah, yeah, but it's just got the the hex value of what-- what exists inside of that space

Amos Wenger: I wish that were the case. That would be so much easier. But at least not in-- not in DaVinci Resolve. On my website I have a patched version of Berkeley Mono and so there's no need to do anything special. And also in CSS you can have like for this range like you know the range for nerd fonts characters. Just use that other font. It's all built in. The web is actually very good at rendering text surprisingly turns out. Yeah, there's a symbols nerd font which only has the symbols and so technically if you were to go through some text that I captured from the terminal you could detect what ranger are an emoji because an emoji is not just one right? That's never the case It's time for a game. The only way to learn is to play. Let's play a little game called. What's the length of this string?

Let's play: what's the length of this string?

James Munns: Oh, no. This isn't quite as bad as what date is this in JavaScript? Yeah, this one's pretty bad

Amos Wenger: There's a bunch of like .wtf websites that popped up recently. One of them was about Python fstrings the other one was-- Yeah yeah, the date-- the JavaScript date thing is

James Munns: just like Sam Who did the JavaScript date library thing.

Amos Wenger: Yeah, so we'll start easy. We'll start with a dollar sign and we'll count four different things. We'll count grapheme clusters, Unicode code points, UTF-16 code units, and UTF-8 bytes.

James Munns: Uh, let's see if this is an ASCII dollar sign, it would be one grapheme cluster, one unicode code point, one UTF-16 code unit and one UTF-8 byte?

Amos Wenger: Yes, that is correct. So that's four points for you.

James Munns: Because we're in the ASCII subset of of Unicode

Amos Wenger: Do you want a bonus point to tell us what the code point is?

James Munns: Oh, I don't know. No, I'd have to Google it.

Amos Wenger: Well, it's plus 24. It's hex 24. But yeah, that's all correct. It's exactly-- the length of one dollar sign is one. Everybody agrees. Honestly, we should have stopped there probably Next one.

James Munns: Oh no.

Amos Wenger: A crab emoji.

James Munns: Crab? It is gonna be one grapheme cluster One Unicode code point. It's gonna be

Amos Wenger: Correct

James Munns: one UTF-18 code unit and it's gonna be three UTF-8 bytes

Amos Wenger: Before you go any further, I want to tell you what the code point is so that it can help you. The Unicode code point for the crab is 1F980.

James Munns: 1F980 yeah, so it'd be... I mean it's gonna be three UTF-8 bytes, right?

Amos Wenger: No, it's gonna be two UTF-16 code units and four UTF-8 bytes. Because the code point is too big. It don't fit. It don't fit in one UTF-16 code unit. So you have a surrogate pair. It's called that way.

James Munns: Oh goody

Amos Wenger: And uh, a surrogate pair-- I just, i'm saying it weird and now I have the the fruit, the pear, like a surrogate pear on my mind.

James Munns: This is my surrogate pear!

Amos Wenger: Amanda please edit a pear.

James Munns: It's a surrogate pear.

Amos Wenger: Exactly. I should have used a pear emoji.

James Munns: Yeah, there you go.

UTF-8 Visualizer

Amos Wenger: And uh for UTF-8 it's a little bit more complicated. There's very good tools, there's UTF-8 visualizers. I linked two of them. They're gonna be in the show notes because Amanda is the best. Yeah, we have four UTF-8 bytes because again, it doesn't fit in three

James Munns: Yeah, I always forget that the UTF-8 format is slightly more complicated than like the Protobuf varint format. Like normally in Protobuf varints and actually the same way Postcard works, you just use one bit as the continuation bit whereas in UTF-8 they use what is it four bits in the first byte and then two in each of the continuation? Like it uses significantly more for metadata

Amos Wenger: That's what it seems to show. Yeah. Actually the other UTF-8 visualizer seems to indicate that-- you get four bytes set to one and then one zero. Do you see it in the corner? They disagree on how many leading zeros we have that are part of the...

James Munns: Because it uses how many ones you have before the first zero to basically count there. So like the 1111 zero, the zero is the terminator for the string of one bits. So it does actually take five bits to say there are four bytes in this sequence

Amos Wenger: But why do we need to know how many bytes there are in the sequence? Can we just use continuation bytes? Ah, well, no, okay. Well, no. Yeah.

James Munns: Yeah, I think it's probably a detection thing. I'm not entirely sure. It's like a clear way that yes, this is the beginning of a thing

Amos Wenger: UTF-8 is fun. The hard part of UTF-8 is... I don't know. Normalization and stuff. Big Unicode tables, stuff like that. But UTF-8 itself is just, it's just pretty neat. All right. Next, next question. James: a pair, an emoji pair of scissors How many grapheme clusters, how many code points?

James Munns: I'm gonna guess all the same numbers as the crab Just to be wrong.

Amos Wenger: Just to be wrong, of course!

James Munns: Yeah... yeah, yeah.

Amos Wenger: So this time there's not one there's two Unicode code points

James Munns: Excellent, but it's a big Unicode code point

Amos Wenger: Has two UTF-16 code units, but six UTF-8 bytes for some reason. This time each code point fits in one UTF-16 code unit, but UTF-8 goes from like-- how many bytes do we have? Four to six. Because... Because it's like three and three I'm assuming? I don't know, I haven't decoded those. Do you know why there's two code points, you have any idea?

James Munns: Is it the left half of the scissors? Is it just two knives glued together and that's actually scissors?

Amos Wenger: That's not as out there of a guess as it sounds

James Munns: It's not, it's not. UTF-8 does some weird things sometimes.

Amos Wenger: But no. It's an image variation selector. The problem is that the pair of scissors existed before emoji. So if you just do pair of scissors, then you get the text version is sideways for some reason.

James Munns: Get the lo-fi-- lo-fi text scissors.

Amos Wenger: The monochrome outline version the the Noto Emoji without color. And then you add an image variation selector after that, and you get an emoji pair of scissors with colors and everything instead.

James Munns: Is this how like things like the-- oh, okay. I won't spoil it

Amos Wenger: You can also select the other-- like this is only one of the selectors. You can also select the text version if you wanted to. And so I guess if you only have 2702 like only the pair of scissors, I guess it's not wrong to render it as emoji? I'm not exactly sure

James Munns: Yeah, there's a bunch of stuff that like it's allowable fall downs in Unicode. We need to just have Manish on is what we need to have like... Someone who's in the Unicode Consortium and knows all of these things of like, there are certain things that are allowable and then there's certain ways to say something's not allowable. Yeah, this is beyond my ken.

Amos Wenger: Well, don't worry because we're reaching the end. Here's the last one.

James Munns: This is the best one. Yeah, because this is-- Okay, okay. I know that the like combined family emoji is a modifier because it's like combining multiple emojis. I don't know if it's quite like person plus person plus child plus child. You're giving me the like Japanese symbol version. Not the like color emoji one that has like cartoon looking people

Amos Wenger: I thought it would be cartoon looking people. I don't know why it shows this

James Munns: Interesting.

Amos Wenger: Google Docs is just being weird.

James Munns: So i'm gonna guess this is i'm guessing this is four grapheme clusters and like seven unicode code points and like 14 bytes or something like that.

Amos Wenger: Okay, you're already wrong on the first one. How many shapes do you identify?

James Munns: Is it four? because it's the four adults and two children or it's usually two adults and two children

Amos Wenger: How many clusters do you identify?

James Munns: In a box?

Amos Wenger: That's just the one.

James Munns: I don't know.

Amos Wenger: This is one cluster, right? It's one, you can select it.

James Munns: Oh, right, right.

Amos Wenger: It's one-- like if you do left or right you're not going to be in the middle of that thing. You can be on the left of it and on the right of it It's one grapheme cluster

James Munns: Doing like the zero width join of multiple emoji characters, it becomes one cluster is the join of all the things that have been joined together. So I guess that's why you were saying the difference between grapheme clusters and code points... because it's one cluster with like five Unicode code points? Does that include the zero width joins and stuff like that?

Amos Wenger: I have- I have to zoom in my speaker notes to cheat No, there's so many so many freaking code points in there. So because you have to have some sort of joiner, right? Between them.

James Munns: The glue that holds the family together.

Amos Wenger: Exactly and you have to repeat it between everything. So you actually have like dad, zero width joiner, mom, zero width joiner, child A, zero width joiner, child B and then a selector- a variant selector. You have eight Unicode code points because like four family members, three joiners, one selector. And then you have

James Munns: 28 bytes

Amos Wenger: And then you have like however many code units and bytes it takes to encode this freaking mess.

James Munns: Plus there's variants for all this like you can have adult man plus adult man plus female child plus male child or whatever for all of it. Like you can customize-- and then I think you can also add skin color variants and things like that? You can make the family emoji a lot of bites. Yeah.

Amos Wenger: Yeah. Yeah, you can. I actually have no idea if the renderers allow all combinations or if they just hard-coded a bunch of them but yeah. So in JavaScript if you have an array literal and then you do ellipsis of like that string literal family emoji, you see the individual code points kind of you see the male face here with joiner female face here with joiner, etc, etc. We're almost there. We almost have something that works We have only 10 slides left.

How do you identify emojis

Amos Wenger: How do we identify emojis in a string? We have to know the boundaries of an emoji so that we can tell Fusion to use our emoji font. How do you do that?

James Munns: Uh, since you have Rust code you just do not is ASCII, is ASCII character or something on each?

Amos Wenger: No because emojis are like a very small- relatively speaking- subset of all non-ASCII things. We still want like diacritics, we still want... There's a lot of things that are not in the emoji font.

James Munns: How do we do this?

Amos Wenger: How do we do this? First of all, I was pleasantly surprised to learn that there's a standard browser API. Also, this is not Rust code. This is all TypeScript. Uh, this is standard segmenter API in the intl for international namespace thingy. You have to give it a locale. So I just choose English but you can segment it.

James Munns: Wait a minute. When do we go from Lua When do we go from Lua or Python or Rust or C++ to TypeS cript? How did-- how do we bring another language into this uh, this cluster?

Amos Wenger: Well, we need to skip ahead so I can show you. But basically the way it works Is that on my website-- like I want to make a video or rather my editor is now tasked with making a video. All they have to do is go on my website, run my website locally and then there's a little button on each code block. And it opens this little preview with a transparent image of my face for scale In that-- this is still in the browser. So you click the button and it whoop you click the button and it opens this preview here.

James Munns: Oh, so your website...

Amos Wenger: And then this is content editable. Yeah, you're in the browser. You can still edit the thing--

James Munns: So your website is generating Lua the describes text.

Amos Wenger: Exactly.

James Munns: I love it!

Amos Wenger: Yeah, it could be done server side. But here you get to remove things that are not relevant because actually Fusion is really slow at rendering text. So you can just remove a paragraph that you don't need or like a code block or something. You can highlight something like select something with a mouse and then press space and it's gonna dim everything else and just highlight that bit. That used to take forever. In Fusion, I used to have to like add rectangles for masks and stuff like just to highlight some bit of code, but now it's really just that. And then you you copy it without the highlight and with the highlight, so you have two different clips and then you do a cross fade between those two different clips and it works

James Munns: So this means your website's now a compiler?

Amos Wenger: Essentially, yes, I think so?

James Munns: Excellent!

emoji-regex-xs

Amos Wenger: Basically it was really hard to like first you segment things into grapheme clusters, and then for each grapheme cluster you run that-- there's an emoji regular expression, small version that identifies emojis and then if it's an emoji you get the Noto Emoji font. If it's something that looks like a nerd font thing, you get the nerd font symbols font. I discovered that Berkeley Mono is missing some Greek symbols. So it's just things that-- there's a lot of fallbacks everywhere, and it takes a freaking dumb font renderer for you to realize that you're missing them.

James Munns: There's a lot of glue

Amos Wenger: There is. It's still kind of a nightmare because the size of everything is wrong. I have an actual preview here. Uh, which I you would need to zoom in a lot to see what's going on. Amanda! We'll do it in post! But basically the emojis don't have the same width as everything else if you had an emoji in a table... That's a problem for a lot of terminal emulators and code editors and whatnot. Yeah, if you have an emoji somewhere you can misalign everything after it, but at least it works.

James Munns: I ran into this not with uh... not with emojis but Departure Mono in odd font sizes- so like 17 point font, 19 point font- there's a specific-- it's not even an emoji It's the Unicode like filled arrow and I use a diagramming tool called Monodraw which makes diagrams and has arrows and lines and stuff like that, but it it makes it as text. But when I copied and pasted it into my editor, all of the boxes that had the arrows left or right in them would become unaligned, even though i'm using a monospace font in a monospace text editor. But I realized that if I change my font from 17 to 18 or whatever, then all of a sudden they align again! So, luckily Departure Mono is an open source font, which means I went and opened a GitHub issue and i'm like, "I don't-- I don't know how fonts work, but something terrible is going on..." So it's not just emojis that run into this problem. A character, even a monospace font one character width is not one character width.

Amos Wenger: I mean maybe... For you, it's probably just precision and calculation. I think it's not right.

James Munns: I'll send you a link to the issue, and I guess we'll put it in the show notes, but it's a fun error

Amos Wenger: But I'm pretty happy with the results of my thing. You can see this is a screenshot of Fusion. You can see that there's an emoji-- The emoji is is rendered as an emoji in the text input on the right like you see the crab emoji with colors and everything but that's because that's a qd text box On the left you have the preview from Fusion.

James Munns: So the operating systems rendering can do it, but Fusion's rendering can't do it.

Amos Wenger: Yeah, but it's very funny because now I understand why we have this futuristic Matrix green outline of the text. I put it like to the left of the canvas on purpose so we could see the outline and see that It's actually vector. Because it is actually vector. This is what happens. It's using FreeTtype to get a vector shape out of a font and it's very slow, because for every single character-- I imagine what the Text+ node is doing is not too far from their Fuse sample that we saw earlier, which is a very basic and bad way to render text. I imagine that's why it's slow because it shouldn't be that slow to render text

James Munns: I guess it's one of those things you don't run into that-- there aren't that many people putting a lot of text... on there. Like, a little text. Sure. I wonder how bad people's... the credit sequences are to render for a lot of people. I feel like that's the most text heavy part of an entire movie if people are using this for movies.

Amos Wenger: That's true. I imagine there's specialized software for that like they do the render elsewhere. Like i've been watching a lot of Corridor Crew, the VFX people react to VFX in movies and they use a patchwork of techniques. It's such a mess. It's so practical. It's like... I don't know. As a developer, It hurts me, but I also see how... I don't know. It's fun to see them make things happen with not much. Anyway, it all works. It's already like--

James Munns: Hell yeah.

Amos Wenger: my solution for generating Fusion node graphs. I've already used in the video "Introducing facet: Reflection for Rust" and uh, it's great! It's like that plus wearing a cap so that I-- shooting with green screen again is such a huge time save. I like started working with Sekun for video editing who's credited now

James Munns: Wait, is the cap just to keep your hair from

Amos Wenger: Yes.

James Munns: causing like fringing with the green screen?

Amos Wenger: Yeah. Yeah, yeah, it's not a style choice. It's not a statement.

James Munns: Ah. Nice.

Amos Wenger: It's just for--

James Munns: It's like the video version of hair nets for food prep. You know what I mean?

Amos Wenger: Did you ever watch any Any Austin video? The guy who's like

James Munns: Austin video?

Amos Wenger: Any Austin.

Amanda Majorowicz: I have yeah.

Amos Wenger: No. Yeah, he um, I haven't

Amanda Majorowicz: I like that guy.

Amos Wenger: He does restaurant reviews of Skyrim, he asked like where does the water come from in Zelda? He's like can a 747 really land in the GTA5 airport, and the answer is no. Oh, I think I've seen clips from... yeah.

Amanda Majorowicz: He's also from Minnesota.

Amos Wenger: He is! And he's always wearing hair clips.

Amanda Majorowicz: It's so great. Yeah, hair clips, reppin'.

Amos Wenger: I stopped feeling bad about-- my wearing a cap when I saw him wearing hair clips. It's such a... I don't know. But who cares, the content is amazing. He's going around Delfino Plaza wondering which NPC's have a job and which don't.

James Munns: Mario Sunshine?

Amos Wenger: Yeah.

James Munns: Okay.

Amos Wenger: Yeah, he's measuring the unemployment rate in various video games. That's what he does. It's amazing.

James Munns: Alright, now I have a bunch of YouTube videos to go watch.

Amos Wenger: Yes you do.

James Munns: Is there a special key combo I can press on your website to see this, or is it only on the local render view?

Amos Wenger: It's only on the local render view and the TypeScript code that does all that is not even bundled in the production version. So... I'm sorry

James Munns: Boo. In case everyone else wanted their immediate Fusion node text generator application

I vibe coded a bunch of it

Amos Wenger: It's so custom made... I thought it would be much more work. I must admit, I vibe coded a bunch of this. I had to stop when-- like for the two details I mentioned, like for the color and the text extense thingy. I tried to debate with Claude code about Like how Unicode works and I was like: okay. I have to actually understand the code that's going on now.

James Munns: You don't have like tree sitter in there somewhere already or is it already formatted? I guess the website's already doing syntax highlighting Using some kind of Tree-sitter plugin or something? And then...

Amos Wenger: Yes.

James Munns: You get the output of that and then you know the colors and then you--

Amos Wenger: Exactly.

James Munns: you generate from that. So it's already done by your website. So you just hop on that train and just spit out from the colored text to generate the Lua that's necessary for this

Amos Wenger: Yes, so the various code blocks, including... highlighting code is not that exciting. Everyone's doing it. The thing I'm really excited about is that I'm capturing terminal sessions with color. That's much harder. In both cases they end up being a soup of like i tags I think with different classes and those different classes have different colors in light mode and dark mode. And so I force dark mode I think in that preview thing. And I use computed style map, I think, to grab the CSS colors.

James Munns: Nice!

Amos Wenger: I think I had a whole video or episode-- at some point, I talked about how display p3, blah, blah, blah... I went back to sRGB for all the colors just because of that

James Munns: Yeah, because it works in more places or?

Amos Wenger: I moved from Catppucin to Melange and from Iosefka to Berkeley Mono so...

James Munns: Mmm. For fonts?

Amos Wenger: Lots of changes. Yeah.

James Munns: I think it was you who for a long time your Twitter profile was just, "The font is Iosefka" or whatever and I know-- I think Eliza started doing the same thing for like, "The font is Berkeley Mono."

Amos Wenger: Mhm, yep.

James Munns: think it was when you were maybe starting to make videos and it was just probably the most common question everyone had for a video was, "What is that font?"

Amos Wenger: I still get it. Yeah. Yeah, you make like, uh, I don't know 30 minute video about something intense and people are like, "What's the font?"

James Munns: There's an old drawing-- or, not old, it's still current. There's a tool on iPad called Concepts which is really nice for drawing things with like an Apple pencil and things like that if you're drawing actual diagrams and stuff, Concepts is excellent. But every time I posted something where I was excited, I just figured out how something works and I drew a diagram of it so I could remember how it's supposed to work- the only engagement I ever got on that was, "What is that diagramming tool?" And-- so I was just like, I need to just put "It's Concepts," in every-- like no engagement with the actual thing that I figured out or I was excited about, it's just, "What is that diagramming tool?"

JAmos Wenger: It's really hard.

James Munns: And so I feel deeply for the-- like you spent all this time on it and it's like, "Oh, what's the pretty font?" Yeah.

Amos Wenger: I occasionally, very occasionally... it's not like everyone asked me, "How do you be famous?" No, it's like- occasionally people ask me about making content and videos like that. And I have no idea what to tell them because even like at the scale that I'm at, I'm really happy. It took a bunch of work to getting to 50k subscribers. I'm excited about color I figured about a bunch of stuff about it. I already said it's a complete flop. YouTube algorithm is, uh, is harsh. Sometimes it just doesn't want to show it to people. It shows it to people a little bit, and if it's not an instant hit, if everybody's not immediately into it, it's like, "Nah, it's not working. Sorry." You get it

James Munns: Yeah. The other thing that I see people do all the time is changing the title like seven times in the first like week that it's out, and I'm sure that's just like engagement hacking of seeing who clicks on it immediately and things like that or they change the thumbnail and whatever. We just kind of publish it and we go, "Oh, well, it'll do as it does."

Amos Wenger: Yeah... Well, that's all I have for you today.

James Munns: I'm glad you got text pretty. At some point, your website is just going to become the ultimate YouTube authoring tool. Like it's just going to generate the project for you and do the recording at some point.

Amos Wenger: I've kind of given up the idea that it's going to be useful for someone other than me. At this point. And it's been liberating.

James Munns: Yeah!

Amos Wenger: But like I don't expect anyone to use the Fusion thing. If anything, I'd like to move to something else for motion design, but I don't know. I'm just so embedded into Resolve right now. Anyway, that's all!

James Munns: I love it.