WEBVTT

00:00:13.666 --> 00:00:16.125
This is Self-Directed Research. Amos and

00:00:16.125 --> 00:00:17.416
James are back with weekly

00:00:17.416 --> 00:00:18.666
hyper-focused deep dives.

00:00:19.208 --> 00:00:20.541
Make sure to like, follow, and subscribe

00:00:20.541 --> 00:00:22.375
wherever you find us to stay up to date,

00:00:22.375 --> 00:00:25.291
and as usual, visit sdr-podcast.com slash

00:00:25.291 --> 00:00:26.541
episodes for all the presentations,

00:00:27.041 --> 00:00:29.458
videos, show notes, and transcripts. This

00:00:29.458 --> 00:00:30.916
episode is sponsored by Depot.

00:00:31.333 --> 00:00:32.666
Listen at the end for more information.

00:00:33.083 --> 00:00:34.333
It's Amos' turn to present

00:00:34.333 --> 00:00:36.125
this week with Automating macOS.

00:00:44.208 --> 00:00:46.416
Hello everyone, how you doing today?

00:00:46.833 --> 00:00:47.333
Hi, Amos.

00:00:47.666 --> 00:00:48.541
Hi, James.

00:00:48.541 --> 00:00:49.416
I'm doing pretty good.

00:00:49.875 --> 00:00:51.166
Hi, Amanda, thanks for

00:00:51.166 --> 00:00:52.708
the thumbs up and the wave.

00:00:53.166 --> 00:00:54.958
Today we're gonna talk about automating

00:00:54.958 --> 00:00:58.375
Mac OS, subtitle, the quest for

00:00:58.375 --> 00:01:00.541
transparent, DOM element screenshots.

00:01:01.000 --> 00:01:03.916
So as you can see, I'm chasing cheap

00:01:03.916 --> 00:01:05.291
clicks for this season.

00:01:05.625 --> 00:01:07.500
I'm shooting for something that I know

00:01:07.541 --> 00:01:08.750
everyone will be into.

00:01:09.166 --> 00:01:09.958
Does this mean we get a

00:01:09.958 --> 00:01:11.666
pogger's face on the--

00:01:11.916 --> 00:01:13.041
We absolutely do, James.

00:01:13.750 --> 00:01:14.041
Excellent,

00:01:14.416 --> 00:01:15.458
I'll get right on it.

00:01:15.875 --> 00:01:17.916
So this is a thinly

00:01:17.916 --> 00:01:19.208
veiled promotion for myself.

00:01:19.208 --> 00:01:22.416
I've been making articles and videos at

00:01:22.416 --> 00:01:23.458
the same time lately

00:01:23.458 --> 00:01:25.291
because I have two audiences.

00:01:25.291 --> 00:01:26.583
I have people who read my articles, I

00:01:26.583 --> 00:01:27.750
have people who watch me on YouTube.

00:01:28.208 --> 00:01:30.208
And so far I've had to choose whenever

00:01:30.208 --> 00:01:31.791
there's something interesting, whether

00:01:31.791 --> 00:01:32.541
I'm gonna make an article

00:01:32.541 --> 00:01:33.875
about it or a video about it.

00:01:33.875 --> 00:01:35.500
And making videos takes forever.

00:01:35.958 --> 00:01:38.333
I need to do the research anyway, and I

00:01:38.333 --> 00:01:39.833
need to, visuals for the articles usually

00:01:39.833 --> 00:01:40.916
make diagrams and everything.

00:01:41.541 --> 00:01:43.666
So it's annoying.

00:01:44.333 --> 00:01:46.541
And also I used to write video scripts in

00:01:46.541 --> 00:01:48.208
something like Google Drive so I can

00:01:48.208 --> 00:01:50.375
share them and get feedback in line and

00:01:50.375 --> 00:01:51.958
then I can collaborate with other people.

00:01:52.416 --> 00:01:54.750
But I really missed just writing the way

00:01:54.750 --> 00:01:57.041
I write my articles which is with a code

00:01:57.041 --> 00:01:58.708
editor like Zed, just write markdown.

00:01:59.250 --> 00:02:00.541
And I made a really

00:02:00.541 --> 00:02:01.541
nice surfer for myself.

00:02:01.541 --> 00:02:02.125
You can drag and drop

00:02:02.125 --> 00:02:03.333
images directly in the browser.

00:02:03.625 --> 00:02:05.125
There's this nice syntax highlighting

00:02:05.125 --> 00:02:05.916
that I've already talked

00:02:05.916 --> 00:02:08.166
about in a recent video.

00:02:08.458 --> 00:02:10.708
And there's gonna be a bunch of visuals

00:02:11.083 --> 00:02:12.125
for this podcast episode.

00:02:12.125 --> 00:02:12.541
I'm sorry.

00:02:13.000 --> 00:02:14.208
I apologize for the people who think

00:02:14.208 --> 00:02:15.375
podcasts are mostly audio.

00:02:15.625 --> 00:02:16.000
They're not.

00:02:16.375 --> 00:02:16.958
You should watch--

00:02:17.375 --> 00:02:18.708
Where can they find those visuals?

00:02:19.166 --> 00:02:21.916
They can find those visuals on

00:02:21.916 --> 00:02:23.875
SDR-podcast.com slash

00:02:23.875 --> 00:02:25.208
episodes if you want to.

00:02:25.375 --> 00:02:26.541
There's a link.

00:02:26.750 --> 00:02:27.791
All right, I don't need to say slash

00:02:27.791 --> 00:02:28.750
episodes all the time.

00:02:28.750 --> 00:02:29.750
But the point is yeah, you can get the

00:02:29.750 --> 00:02:31.166
PDF, you can get the slides, or you can

00:02:31.166 --> 00:02:33.166
watch those on YouTube where we might get

00:02:33.166 --> 00:02:34.125
ad money at some point.

00:02:34.500 --> 00:02:37.166
And you'll have those timed for you so

00:02:37.166 --> 00:02:38.500
you won't even have to flip them.

00:02:38.500 --> 00:02:40.708
The first visual is of course, a picture

00:02:40.708 --> 00:02:43.791
of my blog, which I redesigned a couple

00:02:43.791 --> 00:02:44.666
months ago, I think.

00:02:44.958 --> 00:02:46.916
And it's the article version of catching

00:02:46.916 --> 00:02:49.083
up with Async Rust, which you need to be

00:02:49.083 --> 00:02:52.166
a patron to read fully right now.

00:02:52.666 --> 00:02:54.000
But then also that

00:02:54.000 --> 00:02:55.791
particular one was a double feature.

00:02:55.791 --> 00:02:58.000
I released a video at the same time that

00:02:58.000 --> 00:02:58.750
I released the article,

00:02:59.125 --> 00:03:00.541
and the video is on YouTube.

00:03:00.541 --> 00:03:01.125
So it's available for

00:03:01.125 --> 00:03:02.500
everyone to watch right now.

00:03:02.833 --> 00:03:04.541
And so yeah, for six months, only the

00:03:04.541 --> 00:03:06.000
article video is only for sponsors and

00:03:06.000 --> 00:03:07.125
that's kind of my new business model.

00:03:07.125 --> 00:03:09.791
And it's been kind of working for me.

00:03:09.833 --> 00:03:12.958
But it's a lot of work to do both, not

00:03:12.958 --> 00:03:13.875
quite twice as much.

00:03:14.291 --> 00:03:15.000
Especially when you put so much

00:03:15.000 --> 00:03:16.416
production into both of them.

00:03:16.791 --> 00:03:18.416
I would say you have some of the cleanest

00:03:18.416 --> 00:03:20.166
videos and articles that I've seen.

00:03:20.708 --> 00:03:21.125
You're really,

00:03:21.708 --> 00:03:21.916
James.

00:03:23.416 --> 00:03:24.541
Making me feel bad.

00:03:25.041 --> 00:03:25.916
Cause for me, writings are

00:03:25.916 --> 00:03:27.166
usually in a fever dream.

00:03:27.166 --> 00:03:29.541
If someone poked my brain in just the

00:03:29.541 --> 00:03:31.541
right way that all the neurons lined up

00:03:31.541 --> 00:03:33.583
and I just vomit all the ideas into a

00:03:33.583 --> 00:03:35.458
blog post at once, and then I have to get

00:03:35.458 --> 00:03:37.416
it all out in one sitting, or otherwise I

00:03:37.416 --> 00:03:38.875
sit on the idea for too long.

00:03:39.250 --> 00:03:40.166
Whereas yours are like

00:03:40.166 --> 00:03:42.250
a crafted work of art.

00:03:42.250 --> 00:03:44.708
It's, well this is completely off topic

00:03:44.708 --> 00:03:45.666
and I have 61 slides.

00:03:46.041 --> 00:03:49.208
But yeah, it's been hard to get into a

00:03:49.208 --> 00:03:51.291
mode where I come back to older drafts

00:03:51.291 --> 00:03:52.708
and I'm able to iterate on them.

00:03:53.166 --> 00:03:54.458
Actually all the content I'm releasing

00:03:54.458 --> 00:03:57.708
early, like in Q1 2025, are old pieces

00:03:57.708 --> 00:03:59.791
that I've been drafting throughout 2024.

00:04:00.041 --> 00:04:02.375
But due to unforeseen life stuff

00:04:02.375 --> 00:04:04.625
happening, I couldn't finish any of them.

00:04:04.625 --> 00:04:05.250
And I just had this

00:04:05.250 --> 00:04:06.250
chains of dependencies.

00:04:06.541 --> 00:04:08.208
Like if I improve my blog to

00:04:08.208 --> 00:04:09.666
do this, then I can do that.

00:04:09.916 --> 00:04:10.583
Oh, but to talk about

00:04:10.583 --> 00:04:11.625
this, and you talk about that.

00:04:11.625 --> 00:04:13.166
And I just had like five or six different

00:04:13.166 --> 00:04:14.750
drafts chained up to each other.

00:04:15.083 --> 00:04:17.291
I got out of the hole recently and I

00:04:17.291 --> 00:04:18.416
started releasing stuff again.

00:04:18.416 --> 00:04:19.250
And now I have a kind

00:04:19.250 --> 00:04:20.083
of a monthly schedule.

00:04:20.750 --> 00:04:22.000
It's like the opposite of bankruptcy

00:04:22.208 --> 00:04:22.791
where now you're just

00:04:22.791 --> 00:04:24.583
delivering everything at once.

00:04:24.583 --> 00:04:27.583
Yes, but I'm excited to finish cleaning

00:04:27.583 --> 00:04:29.125
up my old drafts and actually write some

00:04:29.125 --> 00:04:30.083
fresh material again.

00:04:30.083 --> 00:04:31.583
Because for me, it's old news.

00:04:31.583 --> 00:04:33.500
Some of those drafts were 11 months old,

00:04:33.500 --> 00:04:35.083
so I'm bored of them.

00:04:35.083 --> 00:04:36.166
But anyway, so this is the

00:04:36.166 --> 00:04:37.041
article, this is the video.

00:04:37.375 --> 00:04:40.166
And how do you do that video?

00:04:40.166 --> 00:04:41.416
If you'll notice there's a

00:04:41.416 --> 00:04:42.500
little bit of code in the video.

00:04:42.875 --> 00:04:44.375
It's kind of on purpose, I show code in

00:04:44.375 --> 00:04:46.041
videos because yeah, it's awkward.

00:04:46.333 --> 00:04:47.458
So pay for the article.

00:04:47.458 --> 00:04:49.708
That's the whole thing.

00:04:49.708 --> 00:04:51.208
You can still pause, you can zoom in.

00:04:51.208 --> 00:04:52.000
You can like, I try

00:04:52.000 --> 00:04:52.833
to make it big enough.

00:04:52.833 --> 00:04:53.916
But I want it to be clean.

00:04:54.166 --> 00:04:55.375
The gray background that you

00:04:55.375 --> 00:04:56.541
see is not a gray background.

00:04:56.916 --> 00:05:00.000
It's a black piece of clothing behind me.

00:05:00.458 --> 00:05:02.125
And with the color grading, it's almost

00:05:02.125 --> 00:05:03.791
uniform, but not quite.

00:05:04.208 --> 00:05:05.875
And so the code that's

00:05:05.875 --> 00:05:07.875
on there is composited.

00:05:07.875 --> 00:05:08.958
It's a transparent PNG.

00:05:09.375 --> 00:05:10.541
It's composited with the background.

00:05:11.000 --> 00:05:12.375
You'll notice there's no outline, there's

00:05:12.375 --> 00:05:14.458
no background around the code sample.

00:05:14.708 --> 00:05:16.375
It's not just the rectangle screenshot,

00:05:16.375 --> 00:05:17.333
which is what I used to do.

00:05:17.333 --> 00:05:20.333
So I used to just use initially the

00:05:20.333 --> 00:05:21.916
built-in macOS screenshot tool and then

00:05:21.916 --> 00:05:23.500
CleanShot X, which is, "Hey,

00:05:23.583 --> 00:05:24.791
CleanShot, please sponsor me.

00:05:24.791 --> 00:05:26.416
I will sell the frick out of

00:05:26.416 --> 00:05:27.875
your product because I love it."

00:05:27.958 --> 00:05:30.041
Well, yeah, you already did to me.

00:05:30.458 --> 00:05:31.041
So good job.

00:05:31.083 --> 00:05:31.625
I did.

00:05:31.875 --> 00:05:32.375
Do you like it?

00:05:32.958 --> 00:05:34.541
I mean, yeah, I think it's nice.

00:05:34.583 --> 00:05:34.833
Right.

00:05:34.875 --> 00:05:36.458
I can do stuff.

00:05:36.458 --> 00:05:37.333
I can record things.

00:05:37.583 --> 00:05:39.083
So many nice little apps on Mac.

00:05:39.875 --> 00:05:40.916
It's like 20 bucks a pop.

00:05:41.166 --> 00:05:42.791
So you take a rectangle and then you open

00:05:42.791 --> 00:05:44.833
it in an app like Affinity Photo, for

00:05:44.833 --> 00:05:46.458
example, Affinity sponsor m — No, I

00:05:46.458 --> 00:05:48.708
don't like their product that much.

00:05:48.708 --> 00:05:50.208
They're just not subscription like Adobe.

00:05:50.750 --> 00:05:51.458
Then you just remove the

00:05:51.458 --> 00:05:52.833
background, but it's always bad.

00:05:52.833 --> 00:05:54.500
It's always bad because the text is

00:05:54.500 --> 00:05:57.458
already composited with some solid color.

00:05:57.916 --> 00:05:59.750
If it's something dark and you're putting

00:05:59.750 --> 00:06:00.500
it back on something

00:06:00.500 --> 00:06:01.541
dark, it's kind of okay.

00:06:02.125 --> 00:06:04.416
But you can see in this visual, you can

00:06:04.416 --> 00:06:06.500
choose the chroma tolerance.

00:06:06.500 --> 00:06:08.291
There's a bunch of settings you can

00:06:08.291 --> 00:06:09.708
adjust, but you're still gonna have some

00:06:09.708 --> 00:06:10.875
of those darker pixels

00:06:11.000 --> 00:06:12.958
because it had a gray background.

00:06:12.958 --> 00:06:14.166
We could see some red text and you can

00:06:14.166 --> 00:06:16.416
see the pixels on the outer edge are not

00:06:16.416 --> 00:06:17.375
just more transparent

00:06:17.791 --> 00:06:19.000
because of anti-aliasing.

00:06:19.000 --> 00:06:21.291
They're also darker and kind of gray

00:06:21.291 --> 00:06:22.125
instead of being red.

00:06:22.708 --> 00:06:23.416
I think the industry

00:06:23.416 --> 00:06:24.791
term for these are jaggies.

00:06:25.291 --> 00:06:27.000
Yes, jagged edges.

00:06:27.666 --> 00:06:28.208
I think so too.

00:06:28.583 --> 00:06:31.083
Another technique you can do is directly

00:06:31.083 --> 00:06:32.291
inside of DaVinci Resolve.

00:06:32.875 --> 00:06:34.583
DaVinci Resolve integrates Fusion, which

00:06:34.583 --> 00:06:36.333
is their After Effects competitor.

00:06:36.708 --> 00:06:39.708
They're like, what is it even called?

00:06:40.166 --> 00:06:41.208
Special effects suite.

00:06:41.708 --> 00:06:44.458
It's a node-based compositing solution

00:06:45.166 --> 00:06:48.375
and it has a thing called a 3D keyer.

00:06:48.708 --> 00:06:50.291
The way it works is actually funny.

00:06:50.666 --> 00:06:52.541
I'm just now realizing because they make

00:06:52.541 --> 00:06:54.166
a color cube and then they measure the

00:06:54.166 --> 00:06:55.791
distance between the color you give it.

00:06:56.083 --> 00:06:58.083
And like, you define an area within the

00:06:58.083 --> 00:06:59.583
cube that is like, I wanna remove that.

00:06:59.583 --> 00:07:00.500
And then it goes, okay.

00:07:00.541 --> 00:07:02.041
And so you can select a color space.

00:07:02.041 --> 00:07:03.166
Colors are fascinating.

00:07:03.166 --> 00:07:04.166
I'm gonna write about it eventually.

00:07:04.625 --> 00:07:06.291
I've tried, but the draft is like two

00:07:06.291 --> 00:07:07.708
hours long and I'm like, I'm

00:07:07.708 --> 00:07:08.750
not sure about half of this.

00:07:08.750 --> 00:07:09.875
Yeah, you can do it directly in DaVinci

00:07:09.875 --> 00:07:10.875
Resolve, but then you have

00:07:10.875 --> 00:07:12.708
to do this for everything.

00:07:12.708 --> 00:07:14.250
And back then I didn't know you could

00:07:14.250 --> 00:07:15.333
copy Fusion composition

00:07:15.333 --> 00:07:16.416
from one clip to the other.

00:07:16.750 --> 00:07:18.083
I didn't know about a lot of things.

00:07:18.083 --> 00:07:19.666
So it was a lot of just like waiting for

00:07:19.666 --> 00:07:21.583
the tab to open, adding a new node,

00:07:21.791 --> 00:07:24.125
dragging on the background, changing the

00:07:24.125 --> 00:07:26.583
sensitivity, dragging the Despill slider

00:07:26.583 --> 00:07:29.125
up, noticing that if you do that on a

00:07:29.125 --> 00:07:30.583
screen of GitHub, for example,

00:07:30.583 --> 00:07:31.750
it'll make the buttons hollow.

00:07:32.291 --> 00:07:33.583
Like if you go too far with the slider,

00:07:33.750 --> 00:07:34.833
it'll start removing.

00:07:35.458 --> 00:07:37.000
So yeah, so this only really works if you

00:07:37.000 --> 00:07:38.625
take screenshots of dark backgrounds and

00:07:38.625 --> 00:07:40.708
put it back on another dark background.

00:07:40.708 --> 00:07:42.375
And even then, if you look closely, which

00:07:42.375 --> 00:07:44.000
I always do, then it's not great.

00:07:44.000 --> 00:07:45.833
I have a bunch of, well, the results are

00:07:45.833 --> 00:07:47.708
not that bad, as you can see on the

00:07:47.708 --> 00:07:48.750
slides that you can find on

00:07:48.750 --> 00:07:51.083
SDR-podcast.com slash episodes.

00:07:51.541 --> 00:07:53.291
But as you can see, if you put it on a

00:07:53.291 --> 00:07:54.958
white background, there's a

00:07:54.958 --> 00:07:56.958
lot of, not just jagged edges.

00:07:56.958 --> 00:07:58.791
You can see it gets darker near the edge

00:07:58.791 --> 00:08:00.708
of the ladder and then goes straight to

00:08:00.708 --> 00:08:01.708
pure white background

00:08:01.708 --> 00:08:02.541
and it's pretty bad.

00:08:03.250 --> 00:08:04.333
So this is not what we want, but

00:08:04.333 --> 00:08:06.708
eventually I discovered in my journey of

00:08:06.708 --> 00:08:08.541
just replacing everything with Apple

00:08:08.541 --> 00:08:09.791
products because I can finally afford

00:08:09.791 --> 00:08:12.375
them and they work most of the time, that

00:08:12.375 --> 00:08:14.250
in Safari, you can right-click inspect

00:08:14.250 --> 00:08:16.333
element and from the dev tools,

00:08:16.583 --> 00:08:17.666
right-click an element

00:08:17.875 --> 00:08:18.708
and do capture screenshots.

00:08:19.041 --> 00:08:19.958
I'm not sure if Chrome

00:08:19.958 --> 00:08:21.166
offers the same thing.

00:08:21.166 --> 00:08:21.916
It probably does.

00:08:22.166 --> 00:08:23.666
Their web engines have kind of diverged,

00:08:23.666 --> 00:08:25.291
so I don't know who has what feature.

00:08:25.958 --> 00:08:27.916
But what it does is give you a clean PNG

00:08:28.208 --> 00:08:29.041
that's not composited

00:08:29.041 --> 00:08:29.958
with any background.

00:08:30.583 --> 00:08:32.750
And then I did the

00:08:32.750 --> 00:08:34.916
compositing test on this slide as well.

00:08:34.916 --> 00:08:36.250
You can see it on a white background and

00:08:36.250 --> 00:08:37.333
on a dark background.

00:08:37.625 --> 00:08:39.916
Well, it's a dark syntax highlighting

00:08:39.916 --> 00:08:41.291
scheme, so it doesn't really work on

00:08:41.291 --> 00:08:42.625
white, but there's none of

00:08:42.625 --> 00:08:43.708
those edges that we saw before.

00:08:44.000 --> 00:08:46.083
So if this weren't an auditory medium,

00:08:46.291 --> 00:08:47.291
this would be a great demonstration.

00:08:47.541 --> 00:08:49.041
Again, you should really just go on

00:08:49.041 --> 00:08:50.500
YouTube or find the slides for yourself

00:08:50.500 --> 00:08:51.291
to see the difference

00:08:51.583 --> 00:08:52.416
because it's quite striking.

00:08:52.958 --> 00:08:53.458
But then the question

00:08:53.458 --> 00:08:55.000
is, can we automate this?

00:08:55.000 --> 00:08:56.750
Because it's neat that I found a way.

00:08:57.166 --> 00:08:59.083
What I ended up doing is that I now have

00:08:59.083 --> 00:09:00.250
a production tracker.

00:09:00.250 --> 00:09:01.791
So I have columns for everything I need

00:09:01.791 --> 00:09:03.166
to do for an article.

00:09:03.375 --> 00:09:04.833
And the first column is research and

00:09:04.833 --> 00:09:06.916
write everything, and then shoot

00:09:06.916 --> 00:09:10.583
everything, and then do a rough cut, and

00:09:10.583 --> 00:09:12.500
then collect material, visuals that I'm

00:09:12.500 --> 00:09:13.291
going to put into a video.

00:09:13.291 --> 00:09:14.458
So I have so many different columns.

00:09:14.458 --> 00:09:16.458
At the end, I have manual subtitling.

00:09:16.458 --> 00:09:19.000
I have, well, correcting auto subtitles.

00:09:19.916 --> 00:09:20.708
Done with Whisper, whatever.

00:09:20.875 --> 00:09:22.458
And now I just had a column for

00:09:22.458 --> 00:09:23.958
collecting all of those.

00:09:23.958 --> 00:09:25.625
I would open up the article of my blog

00:09:25.791 --> 00:09:26.833
and just scroll through the page.

00:09:27.041 --> 00:09:28.458
And then just go through the dev tool and

00:09:28.458 --> 00:09:30.250
click capture screenshot one by one and

00:09:30.250 --> 00:09:31.416
put them all into a folder.

00:09:31.416 --> 00:09:33.083
That's not great, but it's still better

00:09:33.083 --> 00:09:34.958
than switching back and forth between

00:09:34.958 --> 00:09:36.625
being in timeline editing mode in DaVinci

00:09:36.625 --> 00:09:39.291
Resolve and going back to Safari to

00:09:39.291 --> 00:09:40.208
capture more things.

00:09:40.333 --> 00:09:41.958
It's a very sit down with a beverage in a

00:09:41.958 --> 00:09:42.875
podcast kind of thing.

00:09:43.125 --> 00:09:44.916
Okay, this is the next 20 minutes of my

00:09:44.916 --> 00:09:47.000
life is to just sit down and do this.

00:09:47.000 --> 00:09:47.708
Absolutely, yes.

00:09:47.916 --> 00:09:49.625
And you can mess up because if you

00:09:49.625 --> 00:09:51.791
accidentally select part of it, then the

00:09:51.791 --> 00:09:53.458
selection, the highlights will be also

00:09:53.458 --> 00:09:54.625
part of the screenshot, which has

00:09:54.625 --> 00:09:55.500
happened to me many times.

00:09:55.500 --> 00:09:56.625
Or if you select the slightly the wrong

00:09:56.625 --> 00:09:58.250
node, just like the parent or a child,

00:09:58.250 --> 00:10:00.333
you're gonna get only one word of code or

00:10:00.333 --> 00:10:01.625
you're gonna get the whole page.

00:10:01.958 --> 00:10:03.541
Have you ever tried to import a PNG

00:10:03.750 --> 00:10:05.416
that's several 10,000s of

00:10:05.416 --> 00:10:07.083
pixel high into DaVinci Resolve?

00:10:07.083 --> 00:10:07.750
It's very unhappy.

00:10:08.416 --> 00:10:09.083
I can imagine.

00:10:09.375 --> 00:10:10.875
It's pissy about it too.

00:10:10.875 --> 00:10:12.208
I don't know, it shows a dialogue with

00:10:12.208 --> 00:10:13.875
the GPU error 72 or something.

00:10:15.250 --> 00:10:16.375
It doesn't even tell you what's wrong.

00:10:16.375 --> 00:10:16.541
It's just

00:10:16.541 --> 00:10:17.833
computer said no.

00:10:17.875 --> 00:10:18.916
You done goofed.

00:10:19.166 --> 00:10:19.416
Yeah.

00:10:19.791 --> 00:10:21.041
You know what you did, mister.

00:10:21.541 --> 00:10:22.333
It's really bad.

00:10:22.791 --> 00:10:24.791
So can we automate this is the question I

00:10:24.791 --> 00:10:25.375
eventually asked

00:10:25.375 --> 00:10:27.375
myself because I was bored.

00:10:27.375 --> 00:10:28.000
I get bored.

00:10:28.000 --> 00:10:28.708
This is my issue.

00:10:28.708 --> 00:10:30.125
This is why I couldn't get things done is

00:10:30.125 --> 00:10:31.833
because I started doing manual work and

00:10:31.833 --> 00:10:32.833
I'm like, I hate this.

00:10:32.833 --> 00:10:33.125
This sucks.

00:10:33.458 --> 00:10:34.333
I cannot afford to pay

00:10:34.333 --> 00:10:35.416
someone to do this for me yet.

00:10:36.083 --> 00:10:37.458
And this is what saved me.

00:10:37.458 --> 00:10:40.333
Because if I had paid people to do the

00:10:40.333 --> 00:10:42.208
workflow that I used to do, say two years

00:10:42.208 --> 00:10:44.583
ago, I would have a team of 12 people

00:10:44.583 --> 00:10:45.916
doing repetitive work.

00:10:46.375 --> 00:10:47.250
That's bad.

00:10:47.250 --> 00:10:48.708
But instead, because I couldn't afford

00:10:48.708 --> 00:10:50.625
to, I had to just make the workflow

00:10:50.625 --> 00:10:52.250
better, which is just like when you're

00:10:52.250 --> 00:10:53.500
chasing a bug and you're looking at the

00:10:53.500 --> 00:10:54.875
wrong places, you're making the rest of

00:10:54.875 --> 00:10:56.833
the code so much nicer and more

00:10:56.833 --> 00:10:58.666
auditable, more debuggable, more tested,

00:10:59.000 --> 00:11:00.166
even if the bug was not

00:11:00.166 --> 00:11:01.041
there in the first place.

00:11:01.041 --> 00:11:02.791
So I iterated a bunch on this workflow.

00:11:03.458 --> 00:11:04.875
And one of my thoughts was, can we just

00:11:04.875 --> 00:11:05.750
make a Safari extension?

00:11:06.583 --> 00:11:08.333
Surely if you make a Safari extension, I

00:11:08.333 --> 00:11:10.333
understand that the webpage itself cannot

00:11:10.333 --> 00:11:11.791
call into the dev tools.

00:11:11.791 --> 00:11:12.666
That would be bad.

00:11:13.250 --> 00:11:14.250
Because through the dev tools, you can

00:11:14.250 --> 00:11:15.708
probably access private user data and

00:11:15.708 --> 00:11:16.750
like, control part of the OS.

00:11:16.958 --> 00:11:17.750
I understand there's

00:11:17.750 --> 00:11:18.958
sandboxing going on here.

00:11:19.416 --> 00:11:20.958
But as an extension, surely you're able

00:11:20.958 --> 00:11:23.291
to, because you're able to add a whole

00:11:23.291 --> 00:11:24.916
different tab to the dev tools, you can

00:11:24.916 --> 00:11:26.500
make your own dev tools for something

00:11:26.500 --> 00:11:27.750
like React or

00:11:27.750 --> 00:11:28.791
whatever front-end framework.

00:11:29.500 --> 00:11:32.250
So surely there's a way to control that

00:11:32.250 --> 00:11:33.166
screenshot functionality,

00:11:33.458 --> 00:11:34.500
but I couldn't find anything.

00:11:35.416 --> 00:11:36.500
Documentation around Safari

00:11:36.500 --> 00:11:37.875
DevTools is really sparse.

00:11:38.125 --> 00:11:40.375
Maybe I searched wrong, but I searched

00:11:40.375 --> 00:11:42.416
for half an hour and I kept finding like,

00:11:42.416 --> 00:11:44.500
no, there's nothing, good luck.

00:11:45.083 --> 00:11:47.583
My only solution so far was just the

00:11:47.583 --> 00:11:48.500
right click on a DOM node

00:11:48.500 --> 00:11:50.083
and choose Capture Screenshot.

00:11:50.583 --> 00:11:52.916
There is the only thing I could see in

00:11:52.916 --> 00:11:54.291
the public API, because I did eventually

00:11:54.291 --> 00:11:57.000
find the API docs is get screenshots of

00:11:57.000 --> 00:11:59.458
visible area with a completion handler or

00:11:59.458 --> 00:12:00.666
as an async function.

00:12:00.958 --> 00:12:03.666
I don't know how Swift works.

00:12:04.541 --> 00:12:07.666
But obviously this actually grabs the

00:12:07.666 --> 00:12:08.958
background as well, because it's

00:12:08.958 --> 00:12:10.041
screenshot a visible area.

00:12:10.041 --> 00:12:12.250
You give it just a rectangle and then it

00:12:12.250 --> 00:12:13.583
gives you everything that's in there.

00:12:13.791 --> 00:12:14.958
So that's not super helpful.

00:12:14.958 --> 00:12:15.541
That's not what I want.

00:12:16.166 --> 00:12:17.500
Then I thought about WebDriver.

00:12:18.041 --> 00:12:19.291
I don't know if you've ever written tests

00:12:19.583 --> 00:12:21.083
for the front-end of a web application.

00:12:21.375 --> 00:12:23.625
It's not a good time, but you have tools.

00:12:24.166 --> 00:12:26.250
It used to be a super hack, just like,

00:12:26.625 --> 00:12:30.083
well, the Chrome DevTools, we kind of

00:12:30.083 --> 00:12:31.500
know what the protocol is.

00:12:31.500 --> 00:12:33.083
We've kind of reversed it or something,

00:12:33.083 --> 00:12:34.208
or it's all open source anyway.

00:12:34.500 --> 00:12:35.625
Someone's read the Chromium sources.

00:12:35.958 --> 00:12:39.500
What if we have some test suite send the

00:12:39.500 --> 00:12:40.791
same thing that the Chrome DevTools

00:12:40.791 --> 00:12:43.166
would, and it kind of simulates clicking

00:12:43.166 --> 00:12:45.375
around the webpage and typing tests and

00:12:45.375 --> 00:12:46.666
whatnot and waiting for elements to

00:12:46.666 --> 00:12:47.833
appear and click on them.

00:12:47.833 --> 00:12:49.000
I think it's still a bit of a mess.

00:12:49.375 --> 00:12:51.375
But there's a standard now, so you have a

00:12:51.375 --> 00:12:52.458
reason to be mad about it now.

00:12:53.291 --> 00:12:55.958
The WebDriver standard includes a take

00:12:55.958 --> 00:12:58.541
element screenshots, HTTP method.

00:12:59.250 --> 00:13:02.666
But again, it takes a rectangle, a region

00:13:02.666 --> 00:13:04.333
to capture, and so it

00:13:04.333 --> 00:13:06.416
does include the background.

00:13:06.791 --> 00:13:08.916
And it's not like it's a bug.

00:13:09.166 --> 00:13:12.208
I can't report this to the browser maker.

00:13:12.541 --> 00:13:13.708
They're just following the standard.

00:13:13.916 --> 00:13:14.833
They can't unilaterally

00:13:14.833 --> 00:13:16.333
decide to make that transparent.

00:13:16.916 --> 00:13:18.000
I was gonna say, for most people, that's

00:13:18.000 --> 00:13:18.708
usually what they want.

00:13:19.541 --> 00:13:21.750
The way I've seen this before is for

00:13:21.750 --> 00:13:24.000
regression testing and things like that,

00:13:24.000 --> 00:13:25.208
where usually that's what you want in a

00:13:25.208 --> 00:13:27.375
bug report, is you're like, "I want this

00:13:27.375 --> 00:13:28.708
chunk of the thing "where there's

00:13:28.708 --> 00:13:30.125
supposed to be a log and element, "and

00:13:30.125 --> 00:13:31.583
there's not one, or it's sideways or

00:13:31.583 --> 00:13:33.291
upside down now, "and I wanna see the

00:13:33.291 --> 00:13:35.041
whole thing where, I don't know."

00:13:35.083 --> 00:13:36.791
I know, I'm the weird one out.

00:13:36.791 --> 00:13:38.500
I can't think of anything else that you

00:13:38.500 --> 00:13:39.541
would want, you know what I mean?

00:13:39.541 --> 00:13:41.166
Other than the fact that that was, you

00:13:41.166 --> 00:13:42.708
found, you happened to find in one

00:13:42.708 --> 00:13:44.583
minute, you were like, "It does exactly

00:13:44.583 --> 00:13:46.500
what I want, "almost accidentally."

00:13:47.000 --> 00:13:48.708
This is probably the stage of the episode

00:13:48.708 --> 00:13:50.291
where people are asking, "Wait a minute,

00:13:50.750 --> 00:13:52.916
Amos is using this "to export some code

00:13:52.916 --> 00:13:53.750
blocks essentially."

00:13:54.041 --> 00:13:57.041
So I'm using Safari to render just

00:13:57.041 --> 00:13:58.250
colored text essentially.

00:13:58.541 --> 00:14:00.083
I could do that elsewhere, right?

00:14:00.083 --> 00:14:01.375
I could write a bit of code that takes

00:14:01.375 --> 00:14:03.708
the thing, run the tree sitter, gives me

00:14:03.708 --> 00:14:04.583
the colors and whatnot.

00:14:05.208 --> 00:14:06.458
Well-- Amos, where should people send

00:14:06.458 --> 00:14:07.958
their suggestions of how they

00:14:07.958 --> 00:14:09.250
think your workflow should be?

00:14:09.916 --> 00:14:10.125
Definitely.

00:14:11.750 --> 00:14:12.083
(Both Laughing)

00:14:12.666 --> 00:14:14.333
But I'm also using it for diagrams.

00:14:14.875 --> 00:14:16.541
Rendering SVG is not as easy as it

00:14:16.541 --> 00:14:18.208
sounds, especially because of rich text

00:14:18.208 --> 00:14:19.083
formatting, it's actually

00:14:19.083 --> 00:14:20.333
HTML elements inside of SVG.

00:14:20.541 --> 00:14:21.875
So the short version is I'm not just

00:14:21.875 --> 00:14:24.333
doing code, I'm also doing things that I

00:14:24.333 --> 00:14:25.958
could technically do outside of browsers,

00:14:25.958 --> 00:14:27.208
but I really don't want to.

00:14:27.583 --> 00:14:29.375
And I like things looking exactly the way

00:14:29.375 --> 00:14:30.125
they look on the web.

00:14:30.625 --> 00:14:32.750
So, next option, build

00:14:32.750 --> 00:14:34.125
webkit slash Chromium yourself.

00:14:34.333 --> 00:14:35.125
The problem is--

00:14:35.166 --> 00:14:36.375
You know, I was gonna give you credit

00:14:36.375 --> 00:14:37.708
because I said it's probably easier to

00:14:37.708 --> 00:14:39.416
figure out how to make the browser do

00:14:39.416 --> 00:14:40.458
something that you wanna do

00:14:40.458 --> 00:14:41.500
than write your whole engine.

00:14:41.500 --> 00:14:43.541
Like I totally understand, you know, what

00:14:43.541 --> 00:14:44.958
kind of yak shaving would that be to

00:14:44.958 --> 00:14:46.250
build your whole rendering engine?

00:14:46.250 --> 00:14:47.791
I know other people in Rust who do that,

00:14:47.916 --> 00:14:49.208
but I was gonna give you huge credit,

00:14:49.208 --> 00:14:50.708
like you gotta draw the line somewhere

00:14:50.708 --> 00:14:51.875
and just drive the thing that exists.

00:14:52.125 --> 00:14:53.541
And then you drop a slide that says,

00:14:53.583 --> 00:14:55.666
build webkit or Chromium yourself.

00:14:55.666 --> 00:14:56.916
No, no, no, no, no, no, no, no.

00:14:56.958 --> 00:14:57.291
I'm gonna--

00:14:58.000 --> 00:14:58.333
You misunderstood.

00:14:58.583 --> 00:15:00.208
I mean compile from their sources.

00:15:00.416 --> 00:15:02.333
I mean not implement.

00:15:02.375 --> 00:15:04.666
No, I still, I feel like there's patches

00:15:04.666 --> 00:15:05.500
coming in that I'm

00:15:05.500 --> 00:15:08.708
about to have opinions on.

00:15:08.833 --> 00:15:10.541
There aren't, I don't know.

00:15:10.541 --> 00:15:11.375
Cause I don't even know if

00:15:11.375 --> 00:15:12.333
they have the functionality.

00:15:12.708 --> 00:15:15.333
I booted up Chromium, I looked for APIs,

00:15:15.333 --> 00:15:16.833
they weren't anything automatable.

00:15:17.125 --> 00:15:19.083
So like, even if you can do it from their

00:15:19.083 --> 00:15:20.416
dev tools, I'm not interested cause I'm

00:15:20.416 --> 00:15:21.833
at the same point essentially.

00:15:21.833 --> 00:15:22.416
But yeah, building

00:15:22.416 --> 00:15:23.750
Chromium takes a long time.

00:15:24.291 --> 00:15:25.625
It's worse than building LLVM.

00:15:26.250 --> 00:15:27.916
Working on Chromium is one of the reasons

00:15:27.916 --> 00:15:30.208
that Google has a bunch of different

00:15:30.208 --> 00:15:32.291
build tools and build farms and whatnot.

00:15:32.291 --> 00:15:33.416
It's just gigantic.

00:15:33.416 --> 00:15:34.375
There's so many dependencies.

00:15:34.708 --> 00:15:36.083
I'm not sure I have enough RAM in this

00:15:36.083 --> 00:15:37.208
house to build it honestly.

00:15:37.708 --> 00:15:38.708
And the human average age

00:15:38.708 --> 00:15:40.333
expectancy is like 80 something.

00:15:40.833 --> 00:15:42.208
So I, you know, this is

00:15:42.208 --> 00:15:43.333
just not an option for me.

00:15:43.541 --> 00:15:44.083
If it was for a

00:15:44.083 --> 00:15:45.166
commercial project, maybe.

00:15:45.708 --> 00:15:47.666
But for me, I looked for something else.

00:15:47.666 --> 00:15:48.958
And then I remembered

00:15:48.958 --> 00:15:50.708
that I'm on Apple now.

00:15:50.708 --> 00:15:52.333
I'm not switching back and forth between

00:15:52.333 --> 00:15:54.041
windows and my MacBook.

00:15:54.666 --> 00:15:56.375
Everything's on macOS.

00:15:56.958 --> 00:15:58.708
So they have a tool called Automator,

00:15:58.958 --> 00:16:00.125
which looks like this.

00:16:00.375 --> 00:16:02.625
You can find this slide, say it, no, I'll

00:16:02.625 --> 00:16:05.250
stop. sdr-podcast.com slash episodes.

00:16:05.583 --> 00:16:06.833
It's a little app that

00:16:06.833 --> 00:16:08.625
lets you build workflows.

00:16:08.916 --> 00:16:10.625
And I recently actually built a simple

00:16:10.625 --> 00:16:12.083
workflow that I've been wanting forever.

00:16:12.083 --> 00:16:13.416
I didn't realize you could just use

00:16:13.416 --> 00:16:14.500
Automator to do that.

00:16:14.541 --> 00:16:16.250
It's a quick action so I can right click

00:16:16.250 --> 00:16:18.083
a PDF in the finder and

00:16:18.083 --> 00:16:19.791
render each page as a separate PNG.

00:16:20.000 --> 00:16:21.041
That's actually something I do a lot

00:16:21.041 --> 00:16:23.458
while editing videos because I made a

00:16:23.458 --> 00:16:25.375
video where I talk about the NSA's Memory

00:16:25.375 --> 00:16:26.375
Safety Report, for example.

00:16:26.666 --> 00:16:28.125
It's just seven pages.

00:16:28.541 --> 00:16:29.875
What I used to do is open it in Preview,

00:16:30.375 --> 00:16:31.875
make it big enough so that the resolution

00:16:31.875 --> 00:16:33.291
is kind of OK, take

00:16:33.291 --> 00:16:34.958
screenshots, and then cut everything.

00:16:34.958 --> 00:16:35.583
It's just bad.

00:16:35.583 --> 00:16:36.875
Or you can just figure out what the

00:16:36.875 --> 00:16:38.291
command line is and go through several

00:16:38.291 --> 00:16:39.500
tools and blah, blah, blah, blah.

00:16:39.583 --> 00:16:40.208
I hated that.

00:16:40.541 --> 00:16:42.541
And also, Preview has an Export as PNG

00:16:42.541 --> 00:16:44.083
option, but it exports

00:16:44.083 --> 00:16:46.416
an Animate PNG, an APNG.

00:16:47.125 --> 00:16:47.708
It does.

00:16:48.250 --> 00:16:49.583
James, you're shaking your head, but I

00:16:49.583 --> 00:16:50.875
swear to God, it does.

00:16:51.083 --> 00:16:51.875
No, I believe you.

00:16:51.875 --> 00:16:53.041
It's a confused face.

00:16:53.750 --> 00:16:55.000
Like, oh, God, why?

00:16:55.625 --> 00:16:56.583
I believe you.

00:16:56.583 --> 00:16:58.125
I expected an option like export each

00:16:58.125 --> 00:16:59.708
page as an individual PNG, but no.

00:16:59.958 --> 00:17:01.708
However, they do have a built-in render

00:17:01.708 --> 00:17:04.541
PDF pages as images, hidden in Automator,

00:17:04.541 --> 00:17:06.000
which everyone keeps forgetting about.

00:17:06.000 --> 00:17:06.958
Even on Apple forums,

00:17:06.958 --> 00:17:07.583
they're like, oh, right.

00:17:07.583 --> 00:17:08.125
That's the thing.

00:17:08.583 --> 00:17:08.916
Forgot.

00:17:09.416 --> 00:17:10.291
So Automator is fun.

00:17:10.625 --> 00:17:12.125
You, the listener, may have not have

00:17:12.125 --> 00:17:13.583
heard of Automator before, but you may

00:17:13.583 --> 00:17:15.583
have heard of Shortcuts, which Apple

00:17:15.583 --> 00:17:16.291
teaches you about

00:17:16.291 --> 00:17:17.625
every major version of iOS.

00:17:18.041 --> 00:17:19.000
So this is that, but

00:17:19.000 --> 00:17:20.333
for Mac OS, pretty much.

00:17:20.916 --> 00:17:22.583
And you can record something.

00:17:22.583 --> 00:17:24.291
So you can hit the Record button, then do

00:17:24.291 --> 00:17:26.000
something on your computer, and then play

00:17:26.000 --> 00:17:27.291
it back, and it'll do the thing.

00:17:27.708 --> 00:17:29.750
But that doesn't work for me, really,

00:17:29.750 --> 00:17:31.333
because for me, it's opening the DevTools

00:17:31.333 --> 00:17:33.291
and then scrolling through each element

00:17:33.291 --> 00:17:34.500
that matches the selector.

00:17:34.750 --> 00:17:37.458
Like, I don't think-- I don't know

00:17:37.458 --> 00:17:38.750
exactly what it records.

00:17:39.000 --> 00:17:39.791
Is it going through

00:17:39.791 --> 00:17:40.791
the element hierarchy?

00:17:41.083 --> 00:17:42.375
Is it based on screen position?

00:17:43.125 --> 00:17:44.708
It just doesn't work for me.

00:17:45.291 --> 00:17:46.041
Enter AppleScript.

00:17:46.875 --> 00:17:47.125
Yay.

00:17:47.500 --> 00:17:49.875
So initially, what motivated me to

00:17:49.875 --> 00:17:52.291
actually make that slide, make that

00:17:52.291 --> 00:17:53.666
presentation here, was that

00:17:53.666 --> 00:17:54.958
AppleScript is freaking weird.

00:17:55.208 --> 00:17:55.875
So we're going to talk about

00:17:55.875 --> 00:17:56.833
AppleScript for a hot minute.

00:17:57.166 --> 00:17:59.000
It's a language, a natural language,

00:17:59.458 --> 00:18:01.833
developed by Apple, that

00:18:01.833 --> 00:18:03.916
first appeared 32 years ago.

00:18:04.416 --> 00:18:05.333
I'm older than

00:18:05.333 --> 00:18:06.916
AppleScript, but not by much.

00:18:07.500 --> 00:18:08.541
I was born in 1990.

00:18:08.750 --> 00:18:09.833
It first appeared in 1993.

00:18:10.291 --> 00:18:11.500
Typing discipline is weak.

00:18:11.500 --> 00:18:13.416
I'm reading from the Wikipedia info box.

00:18:14.083 --> 00:18:15.708
Typing discipline is weak, of course.

00:18:15.708 --> 00:18:17.208
It appeared in System 7, apparently.

00:18:17.375 --> 00:18:19.666
I have never, in my life, run System 7.

00:18:19.916 --> 00:18:21.083
I've never run Mac OS 8.

00:18:21.708 --> 00:18:23.375
I've seen screenshots of Mac OS 9.

00:18:23.375 --> 00:18:24.125
I'm pretty sure it ran in

00:18:24.125 --> 00:18:25.291
my browser or something.

00:18:25.291 --> 00:18:25.541
No.

00:18:26.083 --> 00:18:26.291
Yeah.

00:18:26.833 --> 00:18:29.041
And it is extremely weird.

00:18:29.250 --> 00:18:30.333
Let's go through some of the things.

00:18:30.833 --> 00:18:32.500
AppleScript was released in October 1993

00:18:32.750 --> 00:18:35.625
as part of System 7.1.1, the first major

00:18:35.625 --> 00:18:36.750
upgrade to System 7.

00:18:37.500 --> 00:18:39.333
QuarkXPress-- I don't know if-- James,

00:18:39.333 --> 00:18:40.625
you recognize that product name?

00:18:41.000 --> 00:18:43.708
It rings a bell, but I never used it.

00:18:43.708 --> 00:18:44.041
It's a layout thing.

00:18:44.041 --> 00:18:45.708
When you make a magazine, you need to do

00:18:45.708 --> 00:18:46.541
the layout of the thing?

00:18:46.541 --> 00:18:47.416
You would use that.

00:18:47.416 --> 00:18:49.666
I should ask Amanda, although if it's a

00:18:49.666 --> 00:18:50.583
journalism thing, maybe.

00:18:50.750 --> 00:18:52.208
But it's a very long ago.

00:18:52.208 --> 00:18:52.708
I don't think you were

00:18:52.708 --> 00:18:54.416
doing journalism in 93, Amanda.

00:18:54.416 --> 00:18:54.458
I

00:18:54.458 --> 00:18:55.666
definitely was not.

00:18:55.916 --> 00:18:56.791
And so I got--

00:18:56.791 --> 00:18:57.750
I'm sure it's still relevant.

00:18:58.375 --> 00:18:59.541
Journalism, that is me.

00:18:59.541 --> 00:18:59.708
Wait.

00:18:59.916 --> 00:19:00.291
Oh, no.

00:19:00.583 --> 00:19:00.916
Not yet.

00:19:01.500 --> 00:19:03.041
QuarkXPress was one of the first major

00:19:03.041 --> 00:19:03.750
software applications

00:19:03.750 --> 00:19:04.875
that supported AppleScript.

00:19:04.875 --> 00:19:06.458
This, in turn, led to AppleScript being

00:19:06.458 --> 00:19:08.625
widely adopted within the publishing and

00:19:08.625 --> 00:19:10.291
pre-press world, often tying

00:19:10.291 --> 00:19:11.666
together complex workflows.

00:19:11.875 --> 00:19:13.333
This was a key factor in retaining the

00:19:13.333 --> 00:19:15.625
Macintosh's dominant position in

00:19:15.625 --> 00:19:17.416
publishing and pre-press, even after

00:19:17.416 --> 00:19:18.750
QuarkXPress and other publishing

00:19:18.750 --> 00:19:19.333
applications were

00:19:19.333 --> 00:19:20.375
ported to Microsoft Windows.

00:19:20.875 --> 00:19:22.416
And I had no idea.

00:19:22.958 --> 00:19:24.583
I remember wondering as a

00:19:24.583 --> 00:19:26.708
kid, why do people buy Macs?

00:19:26.708 --> 00:19:27.791
Because it's so expensive.

00:19:28.291 --> 00:19:29.708
And now I have two

00:19:29.708 --> 00:19:31.916
decent reasons, at least.

00:19:32.875 --> 00:19:33.458
There's more.

00:19:34.041 --> 00:19:35.958
But one thing I love is you just click

00:19:35.958 --> 00:19:36.666
the Help menu and you

00:19:36.666 --> 00:19:37.666
start typing something out.

00:19:37.708 --> 00:19:39.708
And it tells you every menu item that

00:19:39.708 --> 00:19:40.458
exists in the application.

00:19:40.708 --> 00:19:41.166
That's pretty good.

00:19:41.458 --> 00:19:43.333
I used to complain that there's a global

00:19:43.333 --> 00:19:44.500
menu, essentially, depending on which

00:19:44.500 --> 00:19:46.333
application is focused, the global menu

00:19:46.333 --> 00:19:47.166
bar at the top changes.

00:19:47.458 --> 00:19:48.791
But I think it's great now, because you

00:19:48.791 --> 00:19:49.833
can just discover functionality.

00:19:50.291 --> 00:19:51.833
And DaVinci Resolve all the time, I'm

00:19:52.041 --> 00:19:53.291
clicking the element and going to Help

00:19:53.291 --> 00:19:53.833
and being like, is

00:19:53.833 --> 00:19:54.791
there a thing to do this?

00:19:54.791 --> 00:19:56.750
And it's either showing you the menu item

00:19:57.166 --> 00:19:58.333
or directing you to the Help page.

00:19:58.708 --> 00:20:01.916
So yeah, AppleScript lets you tell

00:20:01.916 --> 00:20:03.000
applications to do things.

00:20:03.458 --> 00:20:05.875
And apparently, that was enough for the

00:20:05.875 --> 00:20:07.541
publishing and pre-press industry.

00:20:08.125 --> 00:20:10.875
And instead of writing code to do their

00:20:10.875 --> 00:20:13.125
thing, they wrote this weird natural

00:20:13.125 --> 00:20:15.125
scripting language to press

00:20:15.125 --> 00:20:18.250
buttons and move files around.

00:20:18.583 --> 00:20:19.541
I don't know if you've ever seen the

00:20:19.541 --> 00:20:21.208
AppleScript editor before.

00:20:21.750 --> 00:20:22.500
But it looks like this.

00:20:23.041 --> 00:20:24.250
So James, can you describe what you're

00:20:24.250 --> 00:20:25.083
seeing on screen right now?

00:20:25.291 --> 00:20:25.916
Tell application

00:20:25.916 --> 00:20:27.708
Safari activate and tell.

00:20:28.041 --> 00:20:30.250
Oh, man, this is from a time.

00:20:30.458 --> 00:20:30.875
It is.

00:20:31.166 --> 00:20:32.166
System event Safari, JoJo.

00:20:32.416 --> 00:20:34.500
So you go to Safari, go

00:20:34.500 --> 00:20:35.583
to the JavaScript console.

00:20:35.916 --> 00:20:37.291
What stands out the most?

00:20:37.708 --> 00:20:38.625
Can you see a difference between the

00:20:38.625 --> 00:20:39.750
first block and the second block?

00:20:40.333 --> 00:20:40.833
One's purple?

00:20:41.666 --> 00:20:41.875
Exactly.

00:20:42.083 --> 00:20:43.416
One's purple and monospace.

00:20:44.083 --> 00:20:45.541
As it turns out-- I have a slide about

00:20:45.541 --> 00:20:48.291
this-- the purple text is uncompiled.

00:20:48.875 --> 00:20:50.000
It's using Menlo regular,

00:20:50.208 --> 00:20:51.208
which is a monospace font.

00:20:51.208 --> 00:20:52.083
Everything's in size 12.

00:20:52.416 --> 00:20:53.666
And the other things are Verdana.

00:20:53.958 --> 00:20:56.291
So actually, the compiled code is sans

00:20:56.291 --> 00:20:57.666
serif, but it's not monospace.

00:20:58.625 --> 00:21:00.625
They're not kidding around with the

00:21:00.625 --> 00:21:01.750
natural language thing.

00:21:01.750 --> 00:21:03.083
They're like, no, it's just a document

00:21:03.083 --> 00:21:05.083
with some colors and italics.

00:21:05.083 --> 00:21:05.666
Don't worry about it.

00:21:06.458 --> 00:21:06.833
It's easy.

00:21:07.416 --> 00:21:09.375
It says, set split group to first UI

00:21:09.375 --> 00:21:11.000
element of target window whose role

00:21:11.000 --> 00:21:12.083
description is split group.

00:21:12.083 --> 00:21:13.125
It's not programming.

00:21:13.125 --> 00:21:13.958
It's just English.

00:21:13.958 --> 00:21:14.333
It's fine.

00:21:14.958 --> 00:21:15.500
You can do this.

00:21:15.833 --> 00:21:19.291
This is of that era of hypercard.

00:21:19.916 --> 00:21:21.333
And there's all those other ones where it

00:21:21.333 --> 00:21:22.958
was the people who took the right amount

00:21:22.958 --> 00:21:24.875
of drugs were programmers at that point,

00:21:24.875 --> 00:21:26.708
where they're like, everything is--

00:21:27.208 --> 00:21:29.166
everything was sort of modeled as like,

00:21:29.166 --> 00:21:30.916
OK, we want interactive programs.

00:21:31.166 --> 00:21:31.833
And people aren't going to

00:21:31.833 --> 00:21:33.000
write programs and run them.

00:21:33.000 --> 00:21:33.958
It's going to be an interactive

00:21:33.958 --> 00:21:35.291
conversation with your computer.

00:21:35.666 --> 00:21:37.791
And there was this whole genre of--

00:21:37.791 --> 00:21:39.583
It was low code before low code, yeah.

00:21:39.875 --> 00:21:40.125
Yeah.

00:21:40.583 --> 00:21:42.666
And Oberon and all of those kind of

00:21:42.666 --> 00:21:44.416
things came out of this same kind of era.

00:21:44.416 --> 00:21:47.875
And then it just kind of lost to most

00:21:47.875 --> 00:21:50.791
regular compile your application and run

00:21:50.791 --> 00:21:51.916
it and those

00:21:51.916 --> 00:21:53.541
traditional scripting type things.

00:21:53.541 --> 00:21:56.166
But there's this whole 90s genre of--

00:21:56.625 --> 00:21:58.958
especially on Macs, where there was just

00:21:58.958 --> 00:22:00.625
this software that was like, people are

00:22:00.625 --> 00:22:01.708
going to write human

00:22:01.708 --> 00:22:03.041
language and it's going to work.

00:22:03.833 --> 00:22:04.541
And you should make sure that everything

00:22:04.541 --> 00:22:06.625
you can do with the GUI can be automated

00:22:06.625 --> 00:22:07.583
through this kind of thing.

00:22:07.583 --> 00:22:08.458
I'm going to stop here because you're

00:22:08.458 --> 00:22:09.750
going to go through my entire slide.

00:22:09.750 --> 00:22:11.583
Just talking about how excited you are.

00:22:11.583 --> 00:22:12.375
I know nothing about

00:22:12.375 --> 00:22:13.208
hypercard, by the way.

00:22:13.208 --> 00:22:14.208
But I know I think it's one of the

00:22:14.208 --> 00:22:16.333
ancestors of Apple scripts or like the

00:22:16.333 --> 00:22:17.333
whole automated thing.

00:22:17.958 --> 00:22:18.166
Yeah.

00:22:18.375 --> 00:22:20.166
So yeah, so you write code and it shows

00:22:20.166 --> 00:22:21.416
up at monospace purple.

00:22:21.791 --> 00:22:22.791
And then you hit Save.

00:22:23.000 --> 00:22:23.750
And if you didn't make any

00:22:23.750 --> 00:22:25.083
mistakes, it highlights it.

00:22:25.583 --> 00:22:26.208
This is just--

00:22:27.250 --> 00:22:28.416
what's the format named?

00:22:28.833 --> 00:22:29.583
I'm going to-- I want to

00:22:29.583 --> 00:22:30.708
say PDF, but that's not it.

00:22:31.125 --> 00:22:31.416
RTF.

00:22:31.416 --> 00:22:32.125
This is just RTF.

00:22:32.375 --> 00:22:34.333
You can copy and paste it into Safari,

00:22:34.708 --> 00:22:35.500
and you'll get the colors.

00:22:35.708 --> 00:22:36.416
I know because this is

00:22:36.416 --> 00:22:37.333
how I made the next slides.

00:22:37.875 --> 00:22:39.833
So how do you write Apple script?

00:22:40.125 --> 00:22:41.416
You tell applications to do things.

00:22:41.416 --> 00:22:42.208
In this case, you can tell

00:22:42.208 --> 00:22:44.083
application Safari to activate.

00:22:44.083 --> 00:22:45.166
What is activating an application?

00:22:45.375 --> 00:22:46.916
It's bringing its windows to the front,

00:22:46.916 --> 00:22:49.166
essentially, and having the top menu bar.

00:22:49.166 --> 00:22:50.333
It's focusing it, basically.

00:22:50.625 --> 00:22:51.458
But focusing is different

00:22:51.458 --> 00:22:52.125
because it's for Windows.

00:22:52.500 --> 00:22:53.708
So in my case, first I

00:22:53.708 --> 00:22:55.208
want to activate Safari.

00:22:55.958 --> 00:22:58.041
And oh, also, everything you can do with

00:22:58.041 --> 00:22:59.375
an application is

00:22:59.375 --> 00:23:01.666
actually-- there's metadata for that.

00:23:01.666 --> 00:23:03.708
There's definitions for that

00:23:03.708 --> 00:23:05.375
that are in the app bundle.

00:23:05.708 --> 00:23:07.583
Because in case you didn't know, on

00:23:07.583 --> 00:23:10.125
macOS.app-- Safari.app,

00:23:10.125 --> 00:23:11.583
for example-- are folders.

00:23:11.916 --> 00:23:13.041
They're app bundles or folders.

00:23:13.500 --> 00:23:14.416
So you can just go in there

00:23:14.416 --> 00:23:16.083
and look at what they have.

00:23:16.333 --> 00:23:18.833
And in the contents slash resources slash

00:23:18.833 --> 00:23:22.750
Safari.sdef file, it's just an XML file

00:23:22.750 --> 00:23:23.916
that defines-- lists

00:23:23.916 --> 00:23:24.958
everything you can do.

00:23:25.166 --> 00:23:27.333
So if, for example, they've bound that to

00:23:27.333 --> 00:23:28.791
other languages, and you can generate

00:23:28.791 --> 00:23:31.083
types and methods and whatnot, there's

00:23:31.083 --> 00:23:32.791
different definitions for everything you

00:23:32.791 --> 00:23:34.250
can do with any applications from those

00:23:34.250 --> 00:23:36.375
XML files, which is absolutely great.

00:23:36.375 --> 00:23:36.875
The one thing you can do

00:23:36.875 --> 00:23:37.583
is you can click menus.

00:23:38.208 --> 00:23:39.791
For example, I have here, tell

00:23:39.791 --> 00:23:40.875
application system events.

00:23:41.166 --> 00:23:42.875
This is how you do synthetic

00:23:42.875 --> 00:23:44.625
inputs, keyboard and mouse.

00:23:45.333 --> 00:23:47.583
Tell application process Safari, click

00:23:47.583 --> 00:23:49.916
menu item, quote, show JavaScript

00:23:49.916 --> 00:23:52.250
console, of menu, quote,

00:23:52.250 --> 00:23:53.708
develop, of menu bar one.

00:23:54.125 --> 00:23:54.750
So there's a lot of

00:23:54.750 --> 00:23:56.291
menu bar one, menu bar 37.

00:23:56.625 --> 00:23:57.416
It gets complicated.

00:23:57.416 --> 00:23:58.291
We'll get into it quickly.

00:23:58.583 --> 00:24:00.000
But this is how it works.

00:24:00.041 --> 00:24:01.250
I don't know what happens if you switch

00:24:01.250 --> 00:24:03.166
the system language, and then suddenly

00:24:03.166 --> 00:24:04.916
it's a "Montrez la console de

00:24:04.916 --> 00:24:06.625
développement JavaScript" instead of

00:24:06.625 --> 00:24:07.625
show JavaScript console.

00:24:07.916 --> 00:24:08.500
I don't know where we have to go.

00:24:08.500 --> 00:24:09.333
You have to try this now.

00:24:09.958 --> 00:24:10.750
I will not.

00:24:11.333 --> 00:24:12.541
So that does an exercise to

00:24:12.541 --> 00:24:14.416
the here listener, listener.

00:24:14.625 --> 00:24:15.250
Maybe, yeah, maybe

00:24:15.250 --> 00:24:16.750
somebody else will try it.

00:24:17.500 --> 00:24:19.416
Please report back as a YouTube comment.

00:24:19.666 --> 00:24:20.208
We need the engagement.

00:24:20.500 --> 00:24:21.208
So what else can you do?

00:24:21.208 --> 00:24:22.666
You can send keystrokes.

00:24:23.166 --> 00:24:25.500
There's a keystroke command, which in

00:24:25.500 --> 00:24:28.625
this case, I'm using to type JavaScript

00:24:28.625 --> 00:24:30.166
code into the DevTools.

00:24:30.500 --> 00:24:31.291
How very meta.

00:24:31.291 --> 00:24:32.791
I'm using one scripting language to type

00:24:32.791 --> 00:24:33.791
another scripting language

00:24:33.791 --> 00:24:35.208
into a DevTool of a browser.

00:24:35.708 --> 00:24:38.875
You can do key code to simulate a key

00:24:38.875 --> 00:24:40.250
press that isn't a string.

00:24:40.791 --> 00:24:42.125
So keystroke is for

00:24:42.125 --> 00:24:43.000
strings for typing text.

00:24:43.583 --> 00:24:44.750
Keycode 36, for example, is enter.

00:24:45.250 --> 00:24:46.083
And then you can wait

00:24:46.083 --> 00:24:48.541
because welcome to GUI automation.

00:24:49.000 --> 00:24:49.625
You will wait.

00:24:50.041 --> 00:24:52.166
You will do some things and try to

00:24:52.166 --> 00:24:54.125
estimate how soon after that it is safe

00:24:54.125 --> 00:24:54.833
to do the next thing.

00:24:55.125 --> 00:24:57.041
If you're lucky, you can actually query

00:24:57.041 --> 00:24:58.666
the state of the GUI and wait for

00:24:58.666 --> 00:24:59.750
something to pop up or something.

00:25:00.291 --> 00:25:02.166
But there's a lot of looping and waiting

00:25:02.166 --> 00:25:03.500
for something to pop up.

00:25:03.500 --> 00:25:04.916
And if it doesn't try the action again

00:25:04.916 --> 00:25:06.083
and undo and whatever,

00:25:06.083 --> 00:25:07.041
there's a lot of cleanup code.

00:25:07.041 --> 00:25:07.708
There's going to be a link

00:25:07.708 --> 00:25:09.458
to the code I ended up with.

00:25:09.458 --> 00:25:10.250
And it's very defensive.

00:25:10.583 --> 00:25:12.000
And the thing that makes all this

00:25:12.000 --> 00:25:13.958
possible is accessibility.

00:25:14.333 --> 00:25:16.416
I have a slide that says A11y.

00:25:16.416 --> 00:25:17.416
If that's the first time you

00:25:17.416 --> 00:25:18.833
see it, it's just accessibility.

00:25:19.125 --> 00:25:20.666
It's A11 editors and then y.

00:25:20.666 --> 00:25:22.208
Just like we have I18n for

00:25:22.208 --> 00:25:26.208
internationalization and L10n for

00:25:26.208 --> 00:25:28.291
localization, which are different things.

00:25:28.750 --> 00:25:31.416
All of this is possible because-- I don't

00:25:31.416 --> 00:25:33.625
know which caused which.

00:25:33.875 --> 00:25:37.083
They both benefit from every user

00:25:37.083 --> 00:25:39.458
interface element being part

00:25:39.458 --> 00:25:41.375
of a tree that you can query.

00:25:41.625 --> 00:25:43.875
There are accessibility APIs where you

00:25:43.875 --> 00:25:45.125
can list what's going

00:25:45.125 --> 00:25:45.791
on with an application.

00:25:46.083 --> 00:25:48.083
So what we have here on this slide is the

00:25:48.083 --> 00:25:49.000
accessibility inspector

00:25:49.000 --> 00:25:50.333
that's built into macOS.

00:25:50.750 --> 00:25:52.500
And you have this little

00:25:52.500 --> 00:25:54.083
crosshair button that you can click.

00:25:54.083 --> 00:25:55.125
And then you can point at

00:25:55.125 --> 00:25:56.500
any UI element anywhere.

00:25:57.083 --> 00:25:58.625
And it's going to tell you exactly where

00:25:58.625 --> 00:26:00.541
it is in the UI hierarchy.

00:26:01.208 --> 00:26:02.458
Yeah, I've definitely seen similar things

00:26:02.458 --> 00:26:05.291
for Selenium tests where working with

00:26:05.291 --> 00:26:06.458
some people who are doing front-end

00:26:06.458 --> 00:26:08.500
testing but also accessibility things

00:26:08.500 --> 00:26:09.791
that companies have worked at where they

00:26:09.791 --> 00:26:11.791
said, usually if you're doing the right

00:26:11.791 --> 00:26:13.750
things for accessibility, it also makes

00:26:13.750 --> 00:26:16.083
automation and testing much easier

00:26:16.291 --> 00:26:17.708
because there's already anchors and

00:26:17.708 --> 00:26:20.083
metadata and stuff for all the things you

00:26:20.083 --> 00:26:20.958
would need for this web

00:26:20.958 --> 00:26:22.208
driver testing as well.

00:26:22.208 --> 00:26:24.416
Yes, and everything has to have labels

00:26:24.416 --> 00:26:27.000
and roles and some description that can

00:26:27.000 --> 00:26:28.375
be used by a screen reader.

00:26:28.625 --> 00:26:30.541
There's a button on that accessibility

00:26:30.541 --> 00:26:33.125
inspector thing that is the Play button.

00:26:33.625 --> 00:26:37.166
And this reads what voiceover the macOS

00:26:37.166 --> 00:26:40.166
screen reader built in would read if that

00:26:40.166 --> 00:26:40.958
element were active.

00:26:41.250 --> 00:26:42.958
And then there's also little arrows,

00:26:42.958 --> 00:26:44.500
which is what happens when you navigate

00:26:44.500 --> 00:26:45.833
to the next of the previous element.

00:26:45.958 --> 00:26:47.041
And similarly, there's actions.

00:26:47.333 --> 00:26:47.958
So for a button,

00:26:47.958 --> 00:26:48.875
there's the press action.

00:26:48.875 --> 00:26:51.541
For a window, there's a raise or lower.

00:26:52.083 --> 00:26:55.916
Everything that you can do without a

00:26:55.916 --> 00:26:57.541
visual pointing device or something like

00:26:57.541 --> 00:26:58.916
that, without actually looking at the

00:26:58.916 --> 00:27:00.666
screen, you can control all of this by

00:27:00.666 --> 00:27:02.625
just listening and saying, yes, no,

00:27:02.625 --> 00:27:03.625
maybe, click, whatever.

00:27:04.208 --> 00:27:05.583
And so actually, what we can see in the

00:27:05.583 --> 00:27:06.833
accessibility inspector, you're missing

00:27:06.833 --> 00:27:07.916
out if you're not looking at the slides,

00:27:07.916 --> 00:27:09.000
where you can see the whole hierarchy

00:27:09.000 --> 00:27:10.416
starting from Safari, which is an

00:27:10.416 --> 00:27:12.208
application, and then catching up with

00:27:12.208 --> 00:27:13.500
Async Rust, the standard window.

00:27:13.833 --> 00:27:15.500
And then split group, tab group, group,

00:27:15.791 --> 00:27:17.208
group, scroll area, HTML content,

00:27:17.708 --> 00:27:19.541
console, tab panel, log,

00:27:19.750 --> 00:27:21.375
group, and then finally text.

00:27:21.375 --> 00:27:22.541
And it's 10 as the result

00:27:22.541 --> 00:27:24.458
of evaluating elems.length.

00:27:24.750 --> 00:27:26.500
Are you using Safari because it works the

00:27:26.500 --> 00:27:27.958
best with Apple Script, or does this

00:27:27.958 --> 00:27:29.958
work-- I assume probably everything works

00:27:29.958 --> 00:27:30.750
mostly the same in

00:27:30.750 --> 00:27:31.916
Firefox or Chrome as well?

00:27:32.291 --> 00:27:33.500
I would hope so.

00:27:33.875 --> 00:27:34.458
I didn't check.

00:27:34.458 --> 00:27:36.666
I think all browsers should be pretty

00:27:36.666 --> 00:27:38.166
good with that, at least on Mac.

00:27:38.166 --> 00:27:38.750
I know the situation

00:27:38.750 --> 00:27:39.916
is complicated on Linux.

00:27:40.333 --> 00:27:41.583
But at least on Mac and Windows, I would

00:27:41.583 --> 00:27:44.208
assume that all browsers expose the DOM

00:27:44.208 --> 00:27:47.083
as this accessibility hierarchy as well,

00:27:47.083 --> 00:27:48.166
because you need to be able to navigate

00:27:48.166 --> 00:27:49.833
what's in the web page using

00:27:49.833 --> 00:27:51.208
the built-in OS screen reader.

00:27:51.750 --> 00:27:52.416
So I haven't checked.

00:27:53.000 --> 00:27:55.208
I'm using Safari because-- and I'm

00:27:55.208 --> 00:27:56.500
ashamed to admit that Safari is actually

00:27:56.500 --> 00:27:57.750
my primary browser, James.

00:27:58.666 --> 00:27:59.708
Nothing wrong with that, especially if

00:27:59.708 --> 00:28:00.458
you like battery life.

00:28:01.083 --> 00:28:02.500
But coming out, y'all

00:28:02.500 --> 00:28:03.375
have something to see.

00:28:03.375 --> 00:28:04.041
I'm using Safari.

00:28:04.500 --> 00:28:05.000
I don't know.

00:28:05.333 --> 00:28:07.791
I don't like Chrome anymore.

00:28:08.291 --> 00:28:09.416
Chrome was cool when it came out.

00:28:09.666 --> 00:28:10.750
I'm going to move on.

00:28:10.833 --> 00:28:13.083
So you have that whole hierarchy that you

00:28:13.083 --> 00:28:14.250
see in the accessibility inspector.

00:28:14.708 --> 00:28:18.000
And then you can turn it into AppleScript

00:28:18.000 --> 00:28:20.791
with great efforts and, in

00:28:20.791 --> 00:28:22.791
my case, the help of one LLM.

00:28:23.750 --> 00:28:25.791
So you would set target window to first

00:28:25.791 --> 00:28:28.125
window, set split group to first UI

00:28:28.125 --> 00:28:30.041
element of target window, whose role

00:28:30.041 --> 00:28:31.166
description is split group.

00:28:31.166 --> 00:28:31.958
And then you keep going.

00:28:31.958 --> 00:28:33.583
You go down the hierarchy like that,

00:28:33.791 --> 00:28:36.791
hoping that it is actually the first UI

00:28:36.791 --> 00:28:37.708
element whose role

00:28:37.708 --> 00:28:38.625
description is split group.

00:28:38.625 --> 00:28:39.750
Or if you have multiple split groups,

00:28:39.833 --> 00:28:41.208
then you need to count them or you need

00:28:41.208 --> 00:28:42.625
to filter them by something else.

00:28:42.625 --> 00:28:43.208
And there's only so many

00:28:43.208 --> 00:28:44.166
fields you can filter by.

00:28:44.375 --> 00:28:45.291
Role description is a good one.

00:28:45.708 --> 00:28:47.083
But some just don't have a

00:28:47.083 --> 00:28:48.083
lot-- there's two groups.

00:28:48.083 --> 00:28:49.166
I don't know what those groups are for.

00:28:49.625 --> 00:28:51.208
It's just kind of fragile.

00:28:51.541 --> 00:28:53.333
If Safari releases a major update, I will

00:28:53.333 --> 00:28:54.833
most likely have to update my scripts.

00:28:55.250 --> 00:28:56.458
And then finally, at the end, we have log

00:28:56.458 --> 00:28:57.916
value of text element as integer.

00:28:58.250 --> 00:29:01.291
And then you can see that in the script

00:29:01.291 --> 00:29:02.541
editor is kind of an IDE.

00:29:02.541 --> 00:29:04.083
So when you hit Save, it compiles.

00:29:04.083 --> 00:29:05.291
We've seen it formats the code.

00:29:05.291 --> 00:29:06.416
And then also, it shows

00:29:06.416 --> 00:29:07.708
messages as you run the script.

00:29:08.208 --> 00:29:11.041
And this is kind of not great to go from

00:29:11.041 --> 00:29:12.083
that accessibility inspector

00:29:12.083 --> 00:29:13.791
to writing that code by hand.

00:29:13.791 --> 00:29:15.125
And there used to be a better way.

00:29:15.125 --> 00:29:16.875
It was called UI browser.

00:29:17.416 --> 00:29:20.000
It was a third-party application made by

00:29:20.000 --> 00:29:22.375
Bill Cheeseman, which-- OK.

00:29:22.916 --> 00:29:24.208
Yeah, Bill Cheeseman.

00:29:24.666 --> 00:29:25.625
Kind of famous

00:29:25.625 --> 00:29:27.250
independent Mac OS developer.

00:29:27.750 --> 00:29:28.666
He did a bunch of apps.

00:29:29.208 --> 00:29:31.291
And he made an app called UI browser.

00:29:31.666 --> 00:29:33.583
It kind of overlaps with the

00:29:33.583 --> 00:29:34.291
accessibility inspector.

00:29:34.541 --> 00:29:36.208
You can still-- you can inspect elements.

00:29:36.208 --> 00:29:37.083
You see the whole hierarchy.

00:29:37.458 --> 00:29:38.208
It's displayed in a

00:29:38.208 --> 00:29:39.375
slightly different format.

00:29:39.375 --> 00:29:41.458
But then there's a button where you can

00:29:41.458 --> 00:29:43.416
just generate Apple script from the thing

00:29:43.416 --> 00:29:44.625
that you pointed to.

00:29:44.625 --> 00:29:45.916
So in this case, it's get value of static

00:29:45.916 --> 00:29:47.125
text, blah, blah, blah.

00:29:47.333 --> 00:29:49.500
But you can see of group 24 of group 1,

00:29:49.500 --> 00:29:52.333
see, you still have to tune it a bit.

00:29:52.625 --> 00:29:53.458
On that last slide,

00:29:53.458 --> 00:29:55.083
that's very small talk.

00:29:55.333 --> 00:29:57.083
There's the small talk graphical browser.

00:29:57.333 --> 00:29:58.833
This is another one of those ones of the

00:29:58.833 --> 00:30:01.875
same contemporary era of that, where all

00:30:01.875 --> 00:30:02.583
of your functions

00:30:02.583 --> 00:30:03.708
would be compiled objects.

00:30:03.708 --> 00:30:04.583
And you get this browser

00:30:04.583 --> 00:30:05.541
of all of your functions.

00:30:05.541 --> 00:30:07.208
So if you wanted to call functions, you

00:30:07.208 --> 00:30:08.875
could browse through it almost like a

00:30:08.875 --> 00:30:10.833
docs.rs But you could click on it and

00:30:10.833 --> 00:30:11.625
drag it into your

00:30:11.625 --> 00:30:13.000
program and stuff like that.

00:30:13.000 --> 00:30:14.708
So it's-- Everything was inspectable.

00:30:14.708 --> 00:30:16.333
It smells very much of that age.

00:30:16.875 --> 00:30:17.208
It does.

00:30:17.708 --> 00:30:20.291
Unfortunately, UI browser version 3 was

00:30:20.291 --> 00:30:23.291
end of life'd on October 17, 2022.

00:30:23.291 --> 00:30:23.958
So I missed the boat.

00:30:23.958 --> 00:30:25.000
You cannot buy it anymore.

00:30:25.583 --> 00:30:27.708
That's why my screenshot says free trial,

00:30:28.125 --> 00:30:29.125
because I don't have a license.

00:30:29.666 --> 00:30:31.041
I have the statement here, the current

00:30:31.041 --> 00:30:32.583
release will not be updated.

00:30:32.916 --> 00:30:34.083
The website will close.

00:30:34.083 --> 00:30:35.458
It will no longer be possible to download

00:30:35.458 --> 00:30:37.583
or purchase UI browser, and product

00:30:37.583 --> 00:30:38.750
support will no longer be available.

00:30:39.208 --> 00:30:40.250
UI browser has been a

00:30:40.250 --> 00:30:41.291
labor of love for me.

00:30:41.291 --> 00:30:43.708
It's sole developer for almost 20 years.

00:30:44.041 --> 00:30:46.625
Now that I'm 79 years old, it is time to

00:30:46.625 --> 00:30:48.041
bring this good work to a conclusion.

00:30:48.500 --> 00:30:49.125
Little cheese man.

00:30:49.625 --> 00:30:51.666
It's hard to be mad at Bill, honestly.

00:30:52.083 --> 00:30:52.291
79?

00:30:52.833 --> 00:30:53.916
OK, you get to retire.

00:30:54.375 --> 00:30:54.583
Fine.

00:30:54.791 --> 00:30:55.833
From open source maintenance.

00:30:55.833 --> 00:30:56.333
Oh, it was not even

00:30:56.333 --> 00:30:56.958
open source, actually.

00:30:57.208 --> 00:30:57.916
Well, OK.

00:30:57.916 --> 00:30:59.333
So there's a UI browser for--

00:30:59.333 --> 00:31:00.125
That means he was 60

00:31:00.125 --> 00:31:01.458
when he started making it.

00:31:01.458 --> 00:31:02.000
That's correct.

00:31:02.250 --> 00:31:03.625
And then retired at 79.

00:31:04.250 --> 00:31:04.666
You know what?

00:31:04.666 --> 00:31:05.666
It's never too late.

00:31:05.791 --> 00:31:06.708
If you think you're

00:31:06.708 --> 00:31:07.458
too old, no, you're not.

00:31:08.125 --> 00:31:08.416
Except Bill.

00:31:08.833 --> 00:31:09.125
Bill's no.

00:31:09.750 --> 00:31:10.791
Bill gets to retire.

00:31:10.791 --> 00:31:11.125
You don't.

00:31:11.125 --> 00:31:12.458
As he retired, like, he

00:31:12.458 --> 00:31:14.041
stopped working on UI browser 3.

00:31:14.375 --> 00:31:16.291
He also released a partially complete

00:31:16.291 --> 00:31:18.791
port of UI browser to Swift from

00:31:18.791 --> 00:31:19.541
Objective-C, I

00:31:19.541 --> 00:31:21.208
imagine, called UI browser 4.

00:31:21.458 --> 00:31:25.208
And I actually compiled it and tested it.

00:31:25.500 --> 00:31:26.541
It's not functional.

00:31:26.541 --> 00:31:27.875
It's missing pretty important things.

00:31:27.875 --> 00:31:29.708
And from the day that it was open

00:31:29.708 --> 00:31:31.375
sourced, I haven't seen any major

00:31:31.375 --> 00:31:32.750
activity on forks or whatever.

00:31:32.750 --> 00:31:33.833
So I don't know what

00:31:33.833 --> 00:31:34.666
people are doing now.

00:31:35.000 --> 00:31:35.583
Actually, I do.

00:31:35.583 --> 00:31:36.041
That's a lie.

00:31:36.416 --> 00:31:37.291
Actually, what they're doing is they're

00:31:37.291 --> 00:31:39.250
just pirating the UI browser 3, because

00:31:39.250 --> 00:31:40.041
it still kind of works.

00:31:40.208 --> 00:31:42.583
So there's a file in library logs--

00:31:42.958 --> 00:31:45.708
that's called k6 whatever-- that has the

00:31:45.708 --> 00:31:47.500
ROT-13 version of your full

00:31:47.500 --> 00:31:49.041
name in an XML dictionary.

00:31:49.583 --> 00:31:50.375
And you can just

00:31:50.375 --> 00:31:51.583
essentially remove that file.

00:31:51.583 --> 00:31:52.208
And it's like you never

00:31:52.208 --> 00:31:53.166
started the trial again.

00:31:53.166 --> 00:31:54.458
And that's what the forums say to do,

00:31:54.458 --> 00:31:55.250
because what are you going to do?

00:31:55.250 --> 00:31:56.958
You cannot buy the software anymore.

00:31:57.208 --> 00:31:58.958
The newer version doesn't work, because

00:31:58.958 --> 00:31:59.833
there's no support anyway.

00:32:00.208 --> 00:32:00.666
So it's not like

00:32:00.666 --> 00:32:01.625
they're missing out on sales.

00:32:01.625 --> 00:32:02.916
They're not willing to sell it anymore.

00:32:02.916 --> 00:32:04.458
So people have turned to piracy.

00:32:04.958 --> 00:32:05.958
Or just don't care

00:32:05.958 --> 00:32:07.166
about accessibility anymore.

00:32:07.166 --> 00:32:09.708
But the UI browser 3 works on Sequoia

00:32:09.708 --> 00:32:12.416
15.2, but it's all crashy.

00:32:12.666 --> 00:32:13.875
So yeah, a bunch of

00:32:13.875 --> 00:32:15.750
slides showing UI browser 4.

00:32:16.208 --> 00:32:18.083
It looks nicer, but most of

00:32:18.083 --> 00:32:19.125
the buttons don't do anything.

00:32:19.583 --> 00:32:21.083
It is at this point in my journey that I

00:32:21.083 --> 00:32:22.833
discovered that actually some things are

00:32:22.916 --> 00:32:24.083
exposed in Safari DevTools.

00:32:24.416 --> 00:32:26.041
There is, for example, a screenshot

00:32:26.041 --> 00:32:27.458
method that I didn't know about.

00:32:27.708 --> 00:32:29.375
Because I had a problem in my automation.

00:32:29.583 --> 00:32:31.500
You still need to somehow scroll through

00:32:31.500 --> 00:32:33.458
the DOM and right click on the node and

00:32:33.458 --> 00:32:34.458
select the menu item

00:32:34.458 --> 00:32:35.333
that's capture a screenshot.

00:32:35.583 --> 00:32:36.583
But actually, no, you

00:32:36.583 --> 00:32:37.416
don't have to do that.

00:32:37.416 --> 00:32:39.750
You can just type screenshot and pass a

00:32:39.750 --> 00:32:41.750
reference to a DOM element right there in

00:32:41.750 --> 00:32:42.916
the DevTools console.

00:32:43.416 --> 00:32:45.125
It gives an image with a checkerboard

00:32:45.125 --> 00:32:46.250
background, so you know it's

00:32:46.250 --> 00:32:47.166
transparent, which is great.

00:32:47.666 --> 00:32:49.416
But then you still have to right click it

00:32:49.666 --> 00:32:51.250
and save it as an image.

00:32:51.583 --> 00:32:53.458
But at least it's predictable.

00:32:54.166 --> 00:32:57.458
You can now write some JavaScript that is

00:32:57.458 --> 00:33:00.041
going to go-- what are we looking at?

00:33:00.041 --> 00:33:01.208
You can write some AppleScript that

00:33:01.208 --> 00:33:02.291
writes some JavaScript.

00:33:02.666 --> 00:33:03.333
That's exactly it.

00:33:03.583 --> 00:33:04.500
That's exactly it.

00:33:05.041 --> 00:33:06.708
You write-- well, so what I ended up

00:33:06.708 --> 00:33:08.541
doing is the JavaScript part I just have

00:33:08.541 --> 00:33:10.583
as part of the JavaScript sources for my

00:33:10.583 --> 00:33:11.958
website so that I get type scripting and

00:33:11.958 --> 00:33:12.708
all the nice things.

00:33:13.291 --> 00:33:15.458
Also, just because it's slow to have

00:33:15.500 --> 00:33:18.166
AppleScript emit keyboard events to enter

00:33:18.166 --> 00:33:19.833
one by one character by character

00:33:19.833 --> 00:33:20.833
JavaScript code in your

00:33:20.833 --> 00:33:21.916
DevTools, it's just slow.

00:33:22.250 --> 00:33:24.000
So it is part of my bundle.

00:33:24.375 --> 00:33:27.250
And the AppleScript just calls a function

00:33:27.416 --> 00:33:29.083
and sets some globals essentially.

00:33:29.083 --> 00:33:30.166
And then we just iterate through every

00:33:30.166 --> 00:33:31.125
element that has to be screenshot.

00:33:31.541 --> 00:33:32.083
And this is the

00:33:32.083 --> 00:33:33.916
AppleScript that ends up doing it.

00:33:34.458 --> 00:33:35.750
You can see a bunch of syntax.

00:33:36.041 --> 00:33:38.000
Like string concatenation is using the ampersand

00:33:38.208 --> 00:33:39.208
operator, for example.

00:33:39.458 --> 00:33:41.166
The most striking thing for me is that

00:33:41.166 --> 00:33:44.291
parentheses are only used for precedence

00:33:44.291 --> 00:33:46.875
or grouping or priority of-- for

00:33:46.875 --> 00:33:48.250
instance, there's the set last log item

00:33:48.250 --> 00:33:50.208
to last UI element of blah, blah, and

00:33:50.208 --> 00:33:50.708
then the parentheses.

00:33:51.041 --> 00:33:52.291
These actually don't do

00:33:52.291 --> 00:33:53.958
anything, as far as I can tell.

00:33:54.375 --> 00:33:56.333
But it's supposed to read as a sentence.

00:33:56.333 --> 00:33:57.958
It's like last UI element of log element

00:33:57.958 --> 00:33:58.708
whose role description

00:33:58.708 --> 00:34:00.250
is-- whose role description?

00:34:00.500 --> 00:34:01.916
Role description is in two

00:34:01.916 --> 00:34:02.958
words There's space between them.

00:34:03.208 --> 00:34:04.208
But it's the same color.

00:34:04.208 --> 00:34:05.250
You can see it's just the name of a

00:34:05.250 --> 00:34:06.166
property that just happens

00:34:06.166 --> 00:34:07.083
to have a space in there.

00:34:07.541 --> 00:34:08.541
This is not the final

00:34:08.541 --> 00:34:09.250
version of the code.

00:34:09.458 --> 00:34:11.791
It performs action ax show menu to

00:34:11.791 --> 00:34:13.833
trigger the context menu, then adds a

00:34:13.833 --> 00:34:15.791
little delay, and then does the down

00:34:15.791 --> 00:34:18.291
arrow key code and then enter.

00:34:18.625 --> 00:34:20.625
I found a better way after

00:34:20.625 --> 00:34:22.208
finishing that slide deck.

00:34:22.416 --> 00:34:23.666
But it's a lot of that.

00:34:23.666 --> 00:34:25.791
It's just like if you can find that menu

00:34:25.791 --> 00:34:27.750
and click Save Image, then

00:34:27.750 --> 00:34:28.916
it shows up at the dialogue.

00:34:29.166 --> 00:34:29.750
And it's already

00:34:29.750 --> 00:34:30.666
focused on the file name.

00:34:30.666 --> 00:34:32.083
So you can just type the file name and

00:34:32.083 --> 00:34:33.291
press Enter and hope

00:34:33.291 --> 00:34:33.916
that everything works.

00:34:34.458 --> 00:34:36.625
It took me forever to debug this.

00:34:36.958 --> 00:34:39.666
OK, I spend an entire day of running it.

00:34:39.666 --> 00:34:41.708
And it goes through 75 images.

00:34:41.708 --> 00:34:42.333
And then it breaks.

00:34:42.333 --> 00:34:43.708
Because it got in some weird state.

00:34:43.708 --> 00:34:44.375
Something took a little

00:34:44.375 --> 00:34:45.500
longer than it usually does.

00:34:45.750 --> 00:34:47.125
I had to do a lot more accessibility

00:34:47.125 --> 00:34:49.041
inspection just to make sure that the UI

00:34:49.041 --> 00:34:50.208
state is what I think it is.

00:34:50.583 --> 00:34:51.625
A lot of looping and

00:34:51.625 --> 00:34:52.708
delaying and retrying.

00:34:53.250 --> 00:34:55.166
And then after that, after all of that, I

00:34:55.166 --> 00:34:57.041
figured out that I never needed to write

00:34:57.041 --> 00:34:58.166
Apple Script in the first place.

00:34:58.500 --> 00:35:01.250
Because there's a thing called JXA, which

00:35:01.250 --> 00:35:02.666
is JavaScript for automation.

00:35:03.333 --> 00:35:04.250
Because of course, there's

00:35:04.250 --> 00:35:05.916
not just Apple Script, right?

00:35:05.916 --> 00:35:06.916
There's also JavaScript.

00:35:07.375 --> 00:35:09.333
There's also Perl, Python,

00:35:09.750 --> 00:35:12.125
Ruby, and TCL from TCL TK.

00:35:12.625 --> 00:35:12.958
Yeah.

00:35:13.750 --> 00:35:15.208
I'm quoting from Wikipedia here.

00:35:15.208 --> 00:35:17.041
"The Macintosh versions of Perl, Python,

00:35:17.041 --> 00:35:19.291
Ruby, and TCL all support native means of

00:35:19.291 --> 00:35:20.416
working with Apple Events

00:35:20.458 --> 00:35:21.750
without being OSA components.

00:35:22.250 --> 00:35:24.458
Also, JXA, so JavaScript for automation,

00:35:25.041 --> 00:35:27.208
also provides an Objective-C and C

00:35:27.208 --> 00:35:28.833
language foreign language interface."

00:35:29.208 --> 00:35:30.375
James, do you realize what that means?

00:35:30.916 --> 00:35:31.541
You're going to write

00:35:31.541 --> 00:35:33.708
Rust that FFI's into JXA?

00:35:34.125 --> 00:35:34.791
No.

00:35:35.208 --> 00:35:35.500
No.

00:35:35.500 --> 00:35:37.041
But that means you can write an entire

00:35:37.041 --> 00:35:38.708
Apple native Mac OS

00:35:38.750 --> 00:35:40.125
application in Apple Script.

00:35:40.125 --> 00:35:40.833
And people have.

00:35:41.125 --> 00:35:42.041
And they've sold them.

00:35:42.458 --> 00:35:43.791
You can write the entire thing.

00:35:43.791 --> 00:35:45.041
Because you can call out to

00:35:45.041 --> 00:35:47.625
any Mac API from Apple Script.

00:35:47.958 --> 00:35:49.250
Because there's this Objective-C bridge.

00:35:49.250 --> 00:35:50.583
Are they all like automator

00:35:50.583 --> 00:35:51.583
tools and things like that?

00:35:51.583 --> 00:35:52.583
Just a regular app.

00:35:52.583 --> 00:35:53.083
Because you can turn

00:35:53.083 --> 00:35:54.750
Apple Scripts into app.

00:35:54.791 --> 00:35:56.791
And you can call any Mac OS API from it.

00:35:57.208 --> 00:35:57.958
So you know.

00:35:58.416 --> 00:36:00.000
Because Objective-C is closer to

00:36:00.000 --> 00:36:02.583
scripting language than C.

00:36:02.750 --> 00:36:04.250
Let's say they already had this concept

00:36:04.250 --> 00:36:07.041
of objects and sending messages and being

00:36:07.041 --> 00:36:09.166
able to do reflection and iterating on

00:36:09.166 --> 00:36:10.125
properties and whatever.

00:36:10.708 --> 00:36:11.416
So you can do that

00:36:11.416 --> 00:36:12.333
from Apple Script as well.

00:36:12.750 --> 00:36:13.625
So they just made a bridge.

00:36:14.458 --> 00:36:15.791
Isn't that something?

00:36:16.333 --> 00:36:18.125
So when we publish this, what

00:36:18.125 --> 00:36:19.833
would you be most excited by?

00:36:19.833 --> 00:36:21.708
Like someone from the Safari team who

00:36:21.708 --> 00:36:22.833
tells you exactly how to

00:36:22.833 --> 00:36:23.875
do this out of the box?

00:36:24.375 --> 00:36:26.291
Someone who's like the last person

00:36:26.291 --> 00:36:29.250
maintaining Apple Script inside of Apple.

00:36:29.250 --> 00:36:31.083
Or Bill Cheeseman.

00:36:31.500 --> 00:36:34.166
Oh, I would love an autograph from Bill.

00:36:34.166 --> 00:36:35.250
I don't know if he's still around.

00:36:35.625 --> 00:36:37.083
But Bill, if you're out there, hit me up.

00:36:37.083 --> 00:36:38.125
Let's not jinx it.

00:36:38.125 --> 00:36:38.500
We can chat.

00:36:39.083 --> 00:36:41.458
Well, the reason I had this slide is that

00:36:41.458 --> 00:36:42.750
you would think that JXA is

00:36:42.750 --> 00:36:44.166
this old deprecated thing.

00:36:44.166 --> 00:36:45.916
Just like on Microsoft, you still have

00:36:45.916 --> 00:36:48.458
JScript, which is not exactly JavaScript.

00:36:49.083 --> 00:36:49.250
OK.

00:36:49.333 --> 00:36:50.250
I don't know if you've ever-- sometimes

00:36:50.250 --> 00:36:52.416
you see-- when you run old-ish

00:36:52.416 --> 00:36:54.333
applications, you see dialogues pop up.

00:36:54.333 --> 00:36:56.000
And it's a script error.

00:36:56.291 --> 00:36:58.083
But that's because it's an old, weird

00:36:58.083 --> 00:37:00.583
version of JavaScript being run by a very

00:37:00.583 --> 00:37:02.708
old, outdated engine built into Windows

00:37:02.708 --> 00:37:04.375
that they ship because they have to--

00:37:05.041 --> 00:37:05.666
backward compatibility.

00:37:05.875 --> 00:37:07.208
I think I've heard of that.

00:37:07.708 --> 00:37:10.083
I've never had to debug that before, so--

00:37:10.083 --> 00:37:12.125
I assume JXA was like this because Apple

00:37:12.125 --> 00:37:13.833
Script seems somewhat maintained.

00:37:14.125 --> 00:37:15.416
I think, essentially, they have to.

00:37:15.708 --> 00:37:18.458
Because again, the freaking pre-press and

00:37:18.458 --> 00:37:20.333
publishing industry is reliant on it.

00:37:20.333 --> 00:37:21.791
It's a selling point of Macs.

00:37:21.833 --> 00:37:23.291
They can never retire it now.

00:37:23.291 --> 00:37:23.833
It's too late.

00:37:24.333 --> 00:37:26.208
But I thought JXA was just like, oh, they

00:37:26.208 --> 00:37:28.291
tried it in OS X Yosemite.

00:37:29.041 --> 00:37:30.625
Didn't really take off

00:37:30.625 --> 00:37:31.916
because it's annoying.

00:37:32.125 --> 00:37:34.458
You lose a lot of the niceties-- We're

00:37:34.458 --> 00:37:35.291
going to get into that--

00:37:35.875 --> 00:37:36.875
of writing Apple Script.

00:37:36.875 --> 00:37:38.916
There's no IDE integration or anything.

00:37:38.916 --> 00:37:39.833
You can write JavaScript.

00:37:39.833 --> 00:37:40.750
You get access to the same thing.

00:37:40.750 --> 00:37:41.916
But you're completely on your own.

00:37:42.250 --> 00:37:43.791
But it is giving you the latest

00:37:43.791 --> 00:37:44.708
JavaScript feature set.

00:37:44.708 --> 00:37:45.583
But that Safari has.

00:37:45.583 --> 00:37:46.958
So you can run exactly the same code that

00:37:46.958 --> 00:37:47.708
you can run in Safari,

00:37:47.708 --> 00:37:48.416
which is actually great.

00:37:48.416 --> 00:37:49.541
So you get to do modern JavaScript.

00:37:49.958 --> 00:37:52.750
This is an example of part of the script

00:37:52.750 --> 00:37:53.833
written as JavaScript.

00:37:54.375 --> 00:37:56.333
You can see I tried to get some typing.

00:37:56.333 --> 00:37:57.208
There are some TypeScript

00:37:57.208 --> 00:37:58.166
types for the-- Yeah, I was going

00:37:58.166 --> 00:37:59.375
to say those look like TypeScript.

00:38:00.083 --> 00:38:00.291
Yeah.

00:38:00.291 --> 00:38:00.500
Yeah.

00:38:01.000 --> 00:38:02.958
You can see there's a shebang up top to

00:38:02.958 --> 00:38:04.958
run it using OSA script, which is the

00:38:04.958 --> 00:38:05.791
command line interface.

00:38:06.041 --> 00:38:06.750
Because I've shown the

00:38:06.750 --> 00:38:07.916
script editor, which is a GUI.

00:38:07.916 --> 00:38:09.458
But you can also just have a plain text

00:38:09.458 --> 00:38:10.875
file somewhere and then

00:38:10.875 --> 00:38:12.250
pass it to OSA script.

00:38:12.250 --> 00:38:12.875
And it still works.

00:38:12.875 --> 00:38:14.541
It just compiles on-demand I guess.

00:38:14.541 --> 00:38:16.916
Because when you save from the script

00:38:16.916 --> 00:38:18.708
editor, it saves the compiled version

00:38:18.708 --> 00:38:20.125
alongside with the sources.

00:38:20.125 --> 00:38:20.583
But the point is, when

00:38:20.583 --> 00:38:21.750
you execute it, it's faster.

00:38:22.083 --> 00:38:23.125
It doesn't have to recompile it every

00:38:23.125 --> 00:38:23.875
time you execute it.

00:38:23.875 --> 00:38:25.416
It's like, what, what are they, Python,

00:38:25.416 --> 00:38:27.250
PyO files, that kind of thing?

00:38:27.250 --> 00:38:27.875
The first time you

00:38:27.875 --> 00:38:29.041
run it, it compiles it.

00:38:29.041 --> 00:38:30.208
And then it holds on to it.

00:38:30.208 --> 00:38:31.458
So you don't have to compile it again the

00:38:31.458 --> 00:38:32.333
second time you run it.

00:38:32.333 --> 00:38:33.458
Something like that.

00:38:34.125 --> 00:38:35.333
Yeah, I guess so.

00:38:35.583 --> 00:38:35.750
Yeah.

00:38:36.083 --> 00:38:37.625
But 30 years before, I guess.

00:38:37.625 --> 00:38:39.166
I'm assuming compiling Apple's script

00:38:39.208 --> 00:38:41.208
made a huge difference back when

00:38:41.208 --> 00:38:42.500
computers were a lot slower.

00:38:43.208 --> 00:38:44.375
And so you can see the difference.

00:38:44.375 --> 00:38:45.125
You're looking at this JavaScript.

00:38:45.125 --> 00:38:46.291
You can see we're doing

00:38:46.291 --> 00:38:47.083
essentially the same thing.

00:38:47.125 --> 00:38:47.833
You're clicking on a menu.

00:38:48.291 --> 00:38:49.416
You're just accessing properties.

00:38:49.750 --> 00:38:50.875
What's aggravating is that you cannot

00:38:50.875 --> 00:38:52.458
actually enumerate properties like you

00:38:52.458 --> 00:38:53.750
would on regular JavaScript objects.

00:38:53.750 --> 00:38:54.375
Because these are not

00:38:54.375 --> 00:38:55.416
JavaScript objects that exist.

00:38:55.708 --> 00:38:56.416
They're just proxies

00:38:56.416 --> 00:38:57.500
for something that exists.

00:38:57.708 --> 00:38:59.041
You can't go through all properties.

00:38:59.041 --> 00:38:59.625
You have to know what

00:38:59.625 --> 00:39:00.291
properties are named.

00:39:00.625 --> 00:39:02.125
And you have to try accessing something.

00:39:02.125 --> 00:39:02.500
And then it just

00:39:02.500 --> 00:39:03.625
throws if it doesn't exist.

00:39:03.833 --> 00:39:05.291
So you have a bunch of try-catch blocks.

00:39:05.291 --> 00:39:06.458
But you have all the same stuff, like

00:39:06.458 --> 00:39:07.500
keystroke, keycode,

00:39:07.500 --> 00:39:09.125
delay, clicking on menus.

00:39:09.625 --> 00:39:11.250
You're just indexing into things instead

00:39:11.250 --> 00:39:13.208
of saying item zero or item one.

00:39:13.458 --> 00:39:14.666
Oh, yeah, because Apple's script is

00:39:14.666 --> 00:39:16.833
one-based also, one-based indexing.

00:39:16.875 --> 00:39:17.750
But in JavaScript, it's

00:39:17.750 --> 00:39:18.833
zero-based because of course.

00:39:19.250 --> 00:39:20.541
And yeah, you don't get

00:39:20.541 --> 00:39:22.041
DevTools at all if you use JXA.

00:39:22.291 --> 00:39:24.416
It just says error minus

00:39:24.416 --> 00:39:26.708
1728 cannot get object.

00:39:26.708 --> 00:39:28.500
It doesn't even tell you the line.

00:39:28.500 --> 00:39:29.833
I was not expecting a full backtrace.

00:39:30.250 --> 00:39:30.750
I'm not greedy.

00:39:31.166 --> 00:39:32.458
I just wanted to do the line of the

00:39:32.458 --> 00:39:33.791
script that actually barfed.

00:39:33.791 --> 00:39:35.916
Just a crumb of context, please.

00:39:36.208 --> 00:39:36.750
It does not.

00:39:37.000 --> 00:39:38.833
Even if you run it directly from the

00:39:38.833 --> 00:39:40.208
script editor, which you can, you can

00:39:40.208 --> 00:39:40.791
actually open

00:39:40.791 --> 00:39:41.958
JavaScript in the script editor.

00:39:42.458 --> 00:39:42.875
Doesn't work.

00:39:43.458 --> 00:39:44.208
Doesn't tell you anything.

00:39:44.958 --> 00:39:46.375
My last couple of slides are just funny,

00:39:46.375 --> 00:39:47.666
cursed things that people are doing.

00:39:48.125 --> 00:39:51.833
Some people are running OSA script from

00:39:51.833 --> 00:39:54.583
JXA to run Apple script because there are

00:39:54.583 --> 00:39:55.416
some things that Apple

00:39:55.416 --> 00:39:57.208
script can do, but JXA cannot do.

00:39:57.500 --> 00:40:00.375
So if you're writing JXA, you can tell

00:40:00.375 --> 00:40:05.458
system events to do shell scripts, OSA

00:40:05.458 --> 00:40:07.000
script, Apple script, something.

00:40:07.708 --> 00:40:08.958
Basically, yeah, I don't know.

00:40:09.166 --> 00:40:12.291
It's apparently impossible in JXA to get

00:40:12.291 --> 00:40:13.500
the front window of the

00:40:13.500 --> 00:40:14.208
frontmost application.

00:40:14.416 --> 00:40:16.750
You can iterate through all the windows

00:40:16.750 --> 00:40:17.500
of the application, but you

00:40:17.500 --> 00:40:18.291
don't know which one is front.

00:40:18.500 --> 00:40:22.083
So you can run OSA script as an external

00:40:22.083 --> 00:40:23.708
process to run some Apple script and

00:40:23.708 --> 00:40:24.416
evaluate to something

00:40:24.416 --> 00:40:25.500
and read it back into it.

00:40:25.500 --> 00:40:26.166
It's a terrible hack.

00:40:26.666 --> 00:40:28.250
This is a stack overflow answer that has

00:40:28.250 --> 00:40:29.333
zero upvotes as it should.

00:40:29.875 --> 00:40:30.458
Oh, this is the thing

00:40:30.458 --> 00:40:31.041
I was talking about.

00:40:31.041 --> 00:40:32.125
It's Apple script, ObJc.

00:40:32.791 --> 00:40:34.333
It's a co-code development software

00:40:34.333 --> 00:40:36.541
framework called Apple script slash

00:40:36.541 --> 00:40:38.333
Objective C or ASOC.

00:40:38.708 --> 00:40:40.500
It's part of the Xcode package since Mac

00:40:40.500 --> 00:40:42.666
OS X, Mac OS X, no leopard.

00:40:43.291 --> 00:40:44.958
It allows Apple scripts to use co-code

00:40:44.958 --> 00:40:46.166
classes and methods directly.

00:40:46.500 --> 00:40:48.333
So yeah, you can make an

00:40:48.333 --> 00:40:50.041
entire app just with Apple script.

00:40:50.041 --> 00:40:50.458
Should you?

00:40:50.875 --> 00:40:52.916
I don't think so, but you can.

00:40:53.458 --> 00:40:55.166
This is a comment on some forum that I

00:40:55.166 --> 00:40:57.833
found very soothing as I spend the entire

00:40:57.833 --> 00:40:59.250
day debugging my automation.

00:40:59.666 --> 00:41:01.583
Someone says, "I started programming in

00:41:01.583 --> 00:41:04.041
1962 and since then have written in about

00:41:04.041 --> 00:41:06.458
15 languages, but this GUI scripting is

00:41:06.458 --> 00:41:08.291
the most vexing I've ever encountered."

00:41:08.291 --> 00:41:09.125
So I don't think this

00:41:09.125 --> 00:41:10.208
is the actual Tim Burton.

00:41:10.291 --> 00:41:11.791
That's just a pseudonym.

00:41:12.291 --> 00:41:14.125
But my heart goes out to you.

00:41:14.250 --> 00:41:15.958
You can also send me an autograph if

00:41:15.958 --> 00:41:17.208
you're listening to this.

00:41:17.208 --> 00:41:18.208
Anyway, I got it working.

00:41:18.458 --> 00:41:20.875
I'm going to have a link to my code just

00:41:20.875 --> 00:41:21.958
so you can see what it looks

00:41:21.958 --> 00:41:23.208
like because it's really weird.

00:41:23.208 --> 00:41:24.666
I just never heard of it before.

00:41:25.291 --> 00:41:27.083
I forgot automation was a thing.

00:41:27.708 --> 00:41:29.708
The accessibility thing is super powerful

00:41:29.708 --> 00:41:31.708
and I'm excited that so

00:41:31.708 --> 00:41:33.458
much of Mac OS is accessible.

00:41:33.708 --> 00:41:36.458
I haven't tried the full screen reader

00:41:36.458 --> 00:41:38.166
experience yet, like actually trying to

00:41:38.166 --> 00:41:40.333
get things done just with my eyes

00:41:40.333 --> 00:41:41.375
completely closed or something.

00:41:41.958 --> 00:41:44.375
But it gives me hope.

00:41:44.666 --> 00:41:46.583
I hope to get more into that because I've

00:41:46.583 --> 00:41:47.750
been meaning to get into accessibility

00:41:47.750 --> 00:41:49.541
stuff for a long time.

00:41:49.541 --> 00:41:51.166
And so this was kind of a soft intro for

00:41:51.166 --> 00:41:52.166
me for something that

00:41:52.166 --> 00:41:53.833
actually saves me a lot of time.

00:41:53.833 --> 00:41:54.875
So I got it working.

00:41:55.250 --> 00:41:56.541
It captured all the images.

00:41:57.166 --> 00:41:58.000
And then of course I noticed

00:41:58.000 --> 00:41:59.500
some typos and some diagrams.

00:41:59.500 --> 00:42:02.000
So I had to manually grab a bunch again.

00:42:02.000 --> 00:42:03.208
But it was super helpful.

00:42:03.958 --> 00:42:04.375
So yay.

00:42:05.000 --> 00:42:06.666
I was going to say, have you improved the

00:42:06.666 --> 00:42:08.291
accessibility of your website now to make

00:42:08.291 --> 00:42:10.250
it easier for your own scripting and

00:42:10.250 --> 00:42:11.125
automation purposes?

00:42:11.458 --> 00:42:13.833
No, I think the accessibility of my

00:42:13.833 --> 00:42:16.291
website is not terrible.

00:42:16.750 --> 00:42:18.916
I've gotten some, because I have some

00:42:18.916 --> 00:42:22.583
readers who are using screen readers.

00:42:22.875 --> 00:42:24.416
So they give me feedback sometimes.

00:42:24.416 --> 00:42:25.875
When I changed the component, I had this

00:42:25.875 --> 00:42:27.083
part switcher for series

00:42:27.083 --> 00:42:28.500
that was not accessible.

00:42:28.916 --> 00:42:29.708
So I changed it.

00:42:29.708 --> 00:42:30.375
I added titles and

00:42:30.375 --> 00:42:31.166
descriptions and whatnot.

00:42:31.500 --> 00:42:32.958
When you prototype a website, you use the

00:42:32.958 --> 00:42:34.375
default browser controls for a bunch of

00:42:34.375 --> 00:42:35.333
things like buttons and

00:42:35.333 --> 00:42:36.250
drop downs and whatnot.

00:42:36.250 --> 00:42:37.916
And then if you replace that with React

00:42:37.916 --> 00:42:39.375
components, they might not be accessible.

00:42:39.708 --> 00:42:42.000
There are some components like the whole

00:42:42.000 --> 00:42:45.000
React ARIA component suite is designed to

00:42:45.000 --> 00:42:46.833
be accessible from the ground up.

00:42:46.833 --> 00:42:48.375
But if you're just throwing something

00:42:48.375 --> 00:42:51.416
together quickly, it's really easy to

00:42:51.416 --> 00:42:54.666
lose track of things that matter to

00:42:54.666 --> 00:42:56.583
non-sighted or low-sighted users.

00:42:57.125 --> 00:42:57.541
Very cool.

00:42:57.750 --> 00:42:58.833
It speaks to AppleScript.

00:42:59.166 --> 00:43:00.666
It strikes me as one of those tools that

00:43:00.666 --> 00:43:02.958
they've tried to kill for 20 years now.

00:43:03.500 --> 00:43:05.625
And none of the successors have reached

00:43:05.625 --> 00:43:08.333
the power of the original one of it was

00:43:08.333 --> 00:43:10.458
just the right amount of imperfect but

00:43:10.458 --> 00:43:13.000
useful that it still has

00:43:13.000 --> 00:43:15.500
existed since 93 or whatever.

00:43:16.625 --> 00:43:18.791
It's hard for me to hate on it because it

00:43:18.791 --> 00:43:19.916
lowered the bar, which means

00:43:19.916 --> 00:43:21.125
more people get to do stuff.

00:43:21.375 --> 00:43:22.166
And that's beautiful.

00:43:22.875 --> 00:43:24.250
I think that's a great note to end on.

00:43:24.666 --> 00:43:26.375
I'm so sorry.

00:43:26.666 --> 00:43:27.500
I keep trying.

00:43:27.958 --> 00:43:29.333
I will just do fewer slides.

00:43:29.833 --> 00:43:30.375
I'm so sorry.

00:43:31.000 --> 00:43:33.416
I also have not hit the download button

00:43:33.416 --> 00:43:34.625
once during this whole recording.

00:43:34.875 --> 00:43:35.708
But I did now.

00:43:35.958 --> 00:43:36.416
So we're good.

00:43:38.708 --> 00:43:40.958
(upbeat music)

00:43:45.083 --> 00:43:46.541
This episode is sponsored by Depot,

00:43:46.750 --> 00:43:47.541
the build acceleration

00:43:47.541 --> 00:43:48.833
platform that's on a mission

00:43:48.833 --> 00:43:50.416
to make all builds near instant.

00:43:50.833 --> 00:43:52.166
If you're tired of watching your builds

00:43:52.166 --> 00:43:53.375
and GitHub actions crawl

00:43:53.375 --> 00:43:54.666
like the modern day equivalent

00:43:54.666 --> 00:43:56.666
of paint drying, give Depot's GitHub

00:43:56.666 --> 00:43:57.666
Actions runners a try.

00:43:58.166 --> 00:43:59.583
They're up to 10 times faster with

00:43:59.583 --> 00:44:00.791
unlimited concurrency,

00:44:00.875 --> 00:44:02.208
faster caching, support for

00:44:02.208 --> 00:44:03.666
Linux, macOS and Windows,

00:44:04.000 --> 00:44:04.916
and they plug right into

00:44:04.916 --> 00:44:05.958
other Depot optimizations

00:44:06.250 --> 00:44:07.916
like accelerated container image builds

00:44:08.083 --> 00:44:09.041
and remote caching for

00:44:09.041 --> 00:44:11.208
Bazel, Turborepo, Gradle and more.

00:44:11.666 --> 00:44:13.166
Depot is built by developers who are

00:44:13.166 --> 00:44:14.166
tired of wasting time

00:44:14.166 --> 00:44:15.625
waiting on builds instead of shipping.

00:44:15.958 --> 00:44:16.750
It's made for teams

00:44:16.750 --> 00:44:17.708
that wanna move faster

00:44:18.000 --> 00:44:18.708
and stay focused on

00:44:18.708 --> 00:44:19.541
what actually matters.

00:44:20.083 --> 00:44:20.750
That's why companies

00:44:20.750 --> 00:44:22.208
like PostHog use Depot

00:44:22.208 --> 00:44:24.083
to cut build times from over three hours

00:44:24.083 --> 00:44:25.166
to just three minutes,

00:44:25.458 --> 00:44:26.541
saving tens of thousands

00:44:26.541 --> 00:44:27.833
of build hours every week.

00:44:28.166 --> 00:44:29.125
Start your free seven

00:44:29.125 --> 00:44:30.500
day trial at depot.dev

00:44:30.500 --> 00:44:31.666
and let them know we sent you.

00:44:32.708 --> 00:44:33.500
(upbeat music)