Categories
Podcasts

Josh Goldberg

Josh Goldberg talks game development, Full Screen Mario, his new book, Learning TypeScript, and more.

Show Notes: https://catchingup.dev/podcasts/josh-goldberg/

Catching Up With Web Performance
Catching Up With Web Performance
Josh Goldberg
/

Links

Video

Transcript

Tanner:

Look at the standing desks. We’re so ready to go.

Josh:

Posture is important. Messed up my back not paying attention. Now I’ve learned the hard way.

Tanner:

Hello, everybody, and welcome back to Catching Up With Web Performance, a podcast about learning web performance. I have today a special guest, Josh Goldberg. Josh, how’s it going, man?

Josh:

Hey, I’m excited to be here. Thanks for having me on. I’m good. It’s a Friday, so I’m almost checked out, but this is still in my happy place. Happy space. How are you?

Tanner:

This is the perfect time for us to chat. I love chatting when checked out.

Josh:

Oh, yeah. I have no idea who I am, what I’ve done in the past or what perf is. So this will be a fun one.

Tanner:

We are gonna have the most interesting story of all time. So we’ve already chatted a bit about, you know, hey, simple, simple episodes here. We just want to talk about stories, like I’m going to ask you, “What’s your first web performance memory?” And we’ll just see where memory lane takes us.

Josh:

That sounds great. My first memory is when I didn’t even think I wanted to be a programmer. It was in high school. I was really into Photoshop, which is what I thought I’d do before I switched. I had a portfolio website where I used to make like little forum signatures back in the day when everyone was on forums on the internet and you could have this little, whatever, 100 by 300, 200 by 400, little image underneath all your posts.

Josh:

And I would Photoshop them, and I wanted a website, and I had a gallery of all the signatures I’d made for other people, and I put the full image in there. But some of these were like animated GIFs that were like really big, and so the page took forever to load. So learning that you need to use thumbnails was key.

Josh:

It was really nice.

Tanner:

So there you are, doing forums, and then dropping massive images and realizing, wow, this—and did you realize it yourself or did like your friends tell you, “Oh, this is slow” and then you fixed it?

Josh:

I realized it myself. At the time I didn’t know about network throttling or device emulation or any of that. I was just like, “Oh, this this page takes too long. Why is that?” And then I Googled around a little bit and I saw, wow, this takes like, whatever, several megabytes of images. I was also doing the occasional wallpaper, so you have like a full desktop resolution in there. And then I found, I don’t know, probably like a Google post or a Stack Overflow question about it, and I was really pleased. It felt good. Like I felt like when you lose, like when you—I don’t want to say “lose weight” because not everyone is trying—but like when you get your target weight or you get that outfit, that’s just really good for you.

Josh:

I felt lean and mean, felt really good.

Tanner:

So then going from there, like, did you immediately say, “I’m going to become a performance engineer”? Or like, I’m sure you have a different story arc here. Like where does it go from there after forums and thumbnails?

Josh:

Yeah, first I had to decide to be a programmer, which I at first did not think was a possibility.

Tanner:

Right, when did you switch from Photoshop?

Josh:

Maybe junior or senior year of high school? Probably senior year. I took Java, a computer science course in high school, and then AP Java, or AP computer science for the New Yorkers out there. And that was fun. I still didn’t really think I was cut out for it, but I got into RPI, which is a pretty good school that was ten minutes away from where I grew up.

Josh:

And, “Okay,” I thought. “Maybe after all, this is what I’m supposed to do.” And also, I looked at the expected salaries for a graphic designer or similar, and computer science was in my future. So I went to school four years for computer science. Didn’t really know what I wanted to do. Was pretty sure I didn’t want to do web or frontend, that seemed like, you know, not real programing, like it’s just JavaScript, whatever, it doesn’t have types.

Josh:

Fast forward to now where I have been on web performance and web platform teams and I wrote a book on TypeScript and am a big advocate for good modern tooling. You have my book?

Tanner:

You mean this book?

Josh:

Yay! Thank you.

Tanner:

Yeah, I was actually very excited to get it. I’ve been sitting on the bench for quite a while with TypeScript. People make fun of me at work, and I’m the curmudgeon who’s like, “I’ll get to it.” You, sir. You’re the reason now. We can thank Josh Goldberg for Tanner converting.

Josh:

That makes me so happy. To all Tanner’s coworkers: you’re welcome. And to Tanner, thank you for getting my book.

Tanner:

Anytime. Everybody else, you’ll see a link somewhere. Maybe.

Josh:

Show notes, chat, etc. But yeah, I did C++ in college because that was the standard curriculum. I liked it. We even had a course where I did C in Assembly and that felt really satisfying. I liked controlling the bits and the bytes. But all my side projects were things I wanted to do on the web, and on the web you have to do performance.

Josh:

And my first internship in tech was doing the website for a neuroscience lab, Schalk Lab, over in Albany. Very cool stuff. I also got to put neuroscience on my resume. I did the website and it was this big Drupal—which is a very, very powerful, good PHP framework akin to Joomla or WordPress—and the homepage had like, whatever, like 3 or 4 megabytes of assets in total, which I trimmed down to I think only like 1.2 or 1.3 megabytes, which is still way more than I would want now in 2022.

Josh:

But at the time in 2013, I was like, you know what it’s good enough. I didn’t really know what I was doing. But yeah, that was kind of my introduction to web performance as a semi sort of becoming professional software developer frontend person.

Tanner:

Yeah.

Josh:

Good times.

Tanner:

So we started doing forums, optimizing images. Then you get into neuroscience and more optimizing, you know, it’s all file size, like “I’m going to bring this down.” What happens next? Because I feel like this is still—how many years ago was this?

Josh:

This was 2013 or 20—summer 2012, actually, my apologies. And then 2013. I did the internship two summers. It’s a fun place to work.

Tanner:

Right. So we’re pushing a decade now, like there’s even so much more here. So have you…

Josh:

What happened next?

Tanner:

Yeah, I’m going to keep asking the “What happened next?” question.

Josh:

Till we get to this podcast. “Well, you see, we reached out on Twitter and…” After that, I did a project called Full Screen Mario, which, unlike me, has its own Wikipedia page. It was a full on JavaScript, not TypeScript, remake of the original Super Mario Brothers. The people at the lab thought it was hilarious. I got some press, it went viral, and then Nintendo took it down after a little more than 2.5 million unique visitors, which I’m very proud of, because it was a ripoff of the original Mario. But it had a lot good going for it.

Josh:

It was full screen, so it was not just a port of the original Assembly, it was a re-implementation in what is now a very terrible game engine. At the time it was just code. It didn’t use like Phaser or Excalibur or any other really good game engine. It also had all the original 32 levels, along with mods and a developer console., so you could like do FSM.setMap() or similar. It also had a level editor and a full screen, or excuse me, a random map generator. That was cool.

Josh:

And performance was very important. I originally had terrible performance. I still don’t have good performance on the old versions of the game before Nintendo also took down the repository. Fun fact: the original version of the game—I thought this was so clever—used HTML elements for everything. Mario, pipes, cloud scenery, everything was an HTML element that got moved around on the screen.

Tanner:

It was even semantic!

Josh:

It was semantic, yeah, aria-hidden and everything. No, I didn’t know anything about semantic HTML or accessibility at the time. Yeah, no, it was HTML. I thought, “Oh yeah, I’m using the platform.” Eventually, late in the development, I switched it to use canvas. Never got around to learning to WebGL, but canvas. And then that’s where performance became tolerable, where you could actually play the game.

Josh:

But it still never really got good performance. I’ve recently been tinkering around with it. I have this website I’ve been trying to get release-ready, ChooseYourFramework.dev. I eventually took the Full Screen Mario code, split out a game engine, and am now off and on (mostly off) writing Full Screen Pokémon, which is exactly what it sounds like.

Josh:

So ChooseYourFramework.dev is that scene where you get to choose what your starter is, but with like Solid and Angular and React. But like the game engine still isn’t very performant, and at the time it was even worse. So optimizing things like reducing the number of JS files, making sure that assets could download in serial and not be like JS would require an image that could have been downloaded on the initial HTML. Little things like that, really. That’s when I picked them up.

Tanner:

This would be a great place maybe to unpack a little more. Because I’ve been trying to keep track of, we’re optimizing images in the beginning, then we’re watching our file size, getting a bit more, I don’t know, intelligent, I guess, or robust, about how we measure performance. And then you’re talking about game performance, like… Unpack a little bit more about what does the performance—like you feel like this game is janky, or something. There’s something about this game that you feel, and then somehow that ties over to a really raw, technical change or implementation.

Tanner:

So maybe when you think, “My Mario game is not performant,” what kind of things are you talking about?

Josh:

Great question. Let’s talk about a game, or a game engine. How does that actually work? A game runs at a certain FPS, frames per second, or a tick interval, as I sometimes call it. So every, let’s say 16.667 milliseconds, what you see on the screen updates. That’s… is that 30? 30 FPS? I can never do the math.

Tanner:

Math.

Josh:

Let’s say that’s 30 FPS. So every second you do 30 frames. So 60 seconds divided by—or sorry, 1000 milliseconds in a second divided by 30 frames is the number of milliseconds between each frame. So let’s say that you have 16 milliseconds in order to do all the logic that has to happen. Or maybe it’s 32 point something, yeah, 32 milliseconds for a 30 FPS game to run.

Josh:

So you have to do everything in your game between the ticks in, let’s say 16 or 32 milliseconds. So that involves physics. So determining whether Mario is touching the thing, the thing has fallen off the edge, the goomba, the gooba, whatever. It involves scrolling. So if Mario is further to the right than whatever the threshold is, the middle of the screen, you have to move everything over.

Josh:

It involves spawning. If you’re moving stuff over, you have to spawn stuff at, say, the right side of the screen. And it involves graphics. If Mario is now whatever number of ticks after the last walking animation change, you have to move him to the next sprite for walking. If you have destroyed an enemy, switch it from the happy sprite to the dead sprite, and so on.

Josh:

And then you have to redraw all those on the screen, rendering. All that has to happen in 16 or 32 or whatever milliseconds. Game engines are really impressive things that are able to do that for agnostic inputs. And when you consider that like, I play Halo once in a while because I still have nostalgia for that beautiful game series and marathon, like we’re rendering 3D shenanigans with lighting and shaders and all this stuff.

Josh:

So like me as this little amateur programmer, who’s not even learning JavaScript in college, like that was hard. So a lot of the things that I learned were to cache the heck out of everything I could. Like instead of caching, or excuse me, instead of computing the data for a particular sprite whenever it’s requested, I will compute that the first time it’s requested and cache what that sprite data is supposed to be.

Josh:

I’ll try not to make too many new objects. Like don’t use empty object, use like a shared empty object variable that’s set to an empty object. Which is not actually a technique I implemented in Mario, but now I know to do. So like don’t create new things that the garbage collector is going to have to garbage collect frequently, stuff like that.

Tanner:

Yeah.

Josh:

Have you played with these types of real time things before? I’m curious.

Tanner:

I’m dreaming. I aspire to one day actually make my own game. You know, for the past couple of years I’ve been stuck in WordPress land still.

Josh:

Oh boy, WordPress. That’s a fun one.

Tanner:

Oh yeah. My stories are not nearly as exciting. You know, like finding a place where somebody had a for loop, there are 500 results in the post query, and then they’re doing an additional query each time in the for loop and the template. Like that kind of stuff.

Josh:

Oh, yeesh. Yeah.

Tanner:

But I don’t have to deal with physics. I’m not spawning people. I’m not, you know, trying to wreck goombas. So I have a dream one day to get into…

Josh:

To get into the goombas?

Tanner:

Let me make a game. Ha! Get into the goombas, exactly.

Josh:

Yeah. There is actually one interesting point with physics where, I still haven’t done this, but physics is like an N squared operation in that you have, let’s say, N number of things on the screen and then you want to compare them to each of the other possible things they could collide with. So if your game is nice and easy, things are always like a rectangle or a circle where it’s relatively easy to compute whether the 2D things intersect.

Josh:

What’s harder is when you have rectangles and circles, because then you have harder math. So thank God in Mario everything, it’s made for like the old NES or whatever, so it’s all rectangles. But I originally had an N squared “for of for of” in there, or “for in” at the time.

Tanner:

Yeah.

Josh:

So now it’s not quite using a quadtree, which is like the common, what I’ve heard of is like a really good data structure, which lets you determine whether two things are near each other. But it uses quadrants, which is like a grid of everything in the map. Excuse me, everything visible on the screen is then like a certain, like one up to four parts of the grid, and they only do physics like “Am I touching this against things that share a quadrant or cell?” It’s not great, but it gets the game working-ish.

Tanner:

I’ve just, I’ve literally never conceived the impact of choosing between squares and circles. I feel like there’s a goldmine here of things that we should just dig out. Like, what are the other seemingly inconsequential things that can impact the performance of a game?

Josh:

Ooh, whether things are allowed to overlap. Overlapping computation is hard. Like let’s say that you’re running forward with Mario and you’re in the big version, you’ve had a mushroom, and then you press down. Mario slides, he doesn’t just stop, and then if he slides underneath a brick and then you not, you un-press down, he’ll pop up. But like, what if like, does he kill the brick? What if he’s now between two bricks? Does he killed both bricks? So like the physics there is that each time you have to figure out what is his overlap with the things that he’s potentially overlapping with and push him a little bit to the side, and that actually was really inefficient for a while in FSM. I had to optimize it a bunch.

Josh:

And then also I just didn’t get the game physics right. I didn’t think to look at how the actual game implemented physics, so there are just these horrible discrepancies.

Tanner:

“I just did it from memory!”

Josh:

Yeah. I had an emulator and a Game Boy and I just ran him side by side and like, “Oh yeah, these seem right.”

Tanner:

Dig in a bit for me. What is, what is this “FSM”?

Josh:

Oh, I’m sorry. Full Screen Mario, FSM.

Tanner:

Okay, Full Screen Mario, FSM. We’re all on the same page. What was that you optimized?

Josh:

The overlapping physics? So for a while I was having a check whether he was overlapping bricks or other solids in this weirdly inefficient way. I don’t even know how I got that to be inefficient. I think I was doing like quadratic equations or some weird thing I’d found on the internet and didn’t understand. But eventually I made it simpler and then it stopped slowing the game down every time you slid underneath a brick in World 1-2.

Tanner:

This is actually a great segue. Where do you learn this stuff? Like how do you even, do you just “Oh, yep. I googled quadratic formulas.”

Josh:

Yeah, at the time I was fresh out of high school-ish, a couple years, so I still had some algebra in there. A lot of it was just googling. I didn’t know how to write a game engine. I made the choice not to use one because I wanted to learn, which was a very good decision for my learning and a very bad decision for the game.

Tanner:

Trade-offs.

Josh:

Yeah, trade-offs always. Always in programing and performance, trade-offs. But yeah, just a lot of Googling, Stack Overflow.

Tanner:

About to delve into philosophy here. Trade-offs.

Tanner:

Are there any particular resources or things, and even since then, that have been really helpful for you learning?

Josh:

The Chrome docs are always really good, Developer.Chrome.com has like a whole section for performance. Also just using the dev tools was really important. I think the dev tools are how I determined that my ticks were slow. I was taking more than whatever it was, 32 milliseconds per tick. Twitter is also weirdly useful. That’s how we met, that’s how I’ve learned and met a lot of other people and things. Yeah, Twitter and the Chrome dev tools and their docs. Great stuff.

Tanner:

I feel like this is a shot in the dark here. Is there a book out there somewhere, a video series, a particular like, “Hey, here’s how to make an efficient game,” or just a game in general?

Josh:

Oh, I’m sure there are. I wish I’d read them.

Tanner:

Me too. But if you want to learn TypeScript, now for the low price of… I don’t know how much.

Josh:

Yeah, LearningTypeScript.com.

Tanner:

Amazing.

Josh:

What about you? I’m curious what your performance resources have been that you come to mind with

Tanner:

Lately—and this is gonna be a side tangent answer. Lately I’ve been learning statistics, because… Well, here’s the back story. Performance I never really considered a thing before. I just kind of thought “good code,” right? Like, “Let’s code and do good. Do your code good.” And then Core Web Vitals happened, and it was like, “Oh, metrics! Field data! Percentiles!” And percentiles, in particular. Like I never took a stats class. I don’t know how I passed through school without that. But you know, long story short, Core Web Vitals coming out sent me down a rabbit hole to understand all the things. And I’ve come to a point where I’m going off the beaten path. Like I’ve got, you know, let’s see…

Tanner:

Here are two good ones. We got Jeremy Wagner’s Web Performance In Action and Ilya Grigorik’s High Performance Browser Networking. And there’s the classic High Performance Websites from Steve Souders. That one actually, I was surprised it’s stood the test of time. And it’s so simple, it’s just 14 rules. “Let’s learn about these 14 things.” You know, if you can kind of gloss over some of the maybe older specific items, like “Let’s talk about ETags.” I don’t know who talks about ETags still.

Josh:

I don’t even remember what those are. At some point I knew.

Tanner:

Yeah, it’s basically a cache control header. So the principles that came out of that, though, those still stand the test of time. This is my long roundabout way of saying I think there are a lot of good books out there, but I don’t know that any of them quite feel cohesive enough. Like none of them quite lay the groundwork the way that I would hope to see.

Tanner:

And I think I read something similar in the preface of your Learning TypeScript book where, you know like, “Hey, we’ve got a lot of good TypeScript resources out here,” right? Like the TypeScript handbook. But there’s still this kind of missing foundational layer. And you can find it through experience by going and talking to people on Twitter and Google. But like, is there a place I could just go? Maybe? To get all of this?

Josh:

Yeah.

Tanner:

I haven’t found it yet, but I’ve got my bookshelf.

Josh:

I think that’s one of the fundamental problems with educational content for developers. All of our areas, even the ones that seem more sedentary, like C++, are constantly changing, including C++. And it takes a long time to have good, well known resource of any particular type commonly available. Like let’s take textbooks as an example. You can find good textbooks, but they have to come through reputation, through people who’ve used them and have confirmed that they’re good, which takes time to build up. But then inherently you need to update that every few years.

Josh:

And that’s what happened with O’Reilly. They had a really good set of TypeScript textbooks a few years ago, and now they’ve seen that the TypeScript field has changed, that TypeScript as a language has kind of morphed in how people use it and what features are commonly recommended for or against. So now Learning TypeScript is new. It’ll see its moment in the sun and then in, whatever, five years, forever in web tech, you will either have to make a new edition to update it for the vastly different landscape of ESM and Bun and whatever. Or it’ll just be one of those books that like, “Oh yeah, I learned this. It was good, just, you know, ignore these 12 chapters and whatever because they’re no longer relevant.” Yeah.

Tanner:

Yep, yep. Man, I could go off on a whole tangent about the statistics stuff. And maybe we can weasel it in here somehow. My quest, I guess, has left me with, I don’t know, that abstract question. Like, what even is web performance? Like what is this thing that we’re trying to talk about or teach? I’m curious, do you have any thoughts? I sideways asked it when we were talking about FSM, like how did you feel that performance and how did you get to coding stuff from there. But performance, what is it?

Josh:

It’s that feeling. As a user, I have actions I want to take and I don’t want to have to wait. Whether it’s loading a page, the most commonly thought of one, or interacting with something on the page, another commonly thought of one, or maybe waiting for the game to update since I pressed right and now my character should be running. All these things.

Josh:

I don’t want users to wait. I as a user hate waiting so much. I grow up these days as a human with the internet. Why should I wait for this? What is this, the seventies? So that to me is web performance, not lagging behind the user’s intent.

Tanner:

Yeah, what’s so interesting about that is it feels like a relatively clear boundary if you just say, or rather, when you focus on time. And say like, “Performance, web performance, is about time… time, time, time, time, time.” And then Core Web Vitals—I don’t know how far down the rabbit hole you’ve gone here. Layout shifts really threw a monkey wrench in that mental model for me. Like okay, I’ve got so much, “I don’t want to wait, I don’t want to wait, I want it to be fast.” But also, “Don’t bounce things around the screen.” And that just kind of, “Fling!” You know?

Josh:

Honestly, I think the Core Web Vitals are one of the best things that have happened to web dev, ever. Like making it not just a soft thing that we as engineering people can take to our stakeholders, say, “Hey, performance sucks, we should do something about it. I don’t think the users like it,” to, “Our SEO is suffering, our page looks bad for other developers, and we need to do something about it.”

Josh:

Here are the concrete metrics that are actually good metrics, like layout shift. It’s wonderful. But when you think about it in the terms of something even more meta, not just “time waiting for” but “not getting in the way of users,” then that’s great because layout shift really does get in the way of users. There are still websites today, including Twitter, that will shift things dynamically in ways that sometimes can’t be caught by the current or upcoming set of web vitals.

Josh:

So not getting in the way of users, of that performance is a huge concern.

Tanner:

I really, really, really like that definition.

Josh:

Yay!

Tanner:

You know, it covers so much ground in one little sentence there. “Not getting in the way of users.” Or at least that is a good outline for a goal. Maybe not a definition for what performance is, but definitely like the objective, or maybe the goal. Like, “The reason I do performance is so that we don’t get in their way.” Performance is best when it’s invisible.

Josh:

For sure. What is the phrase? I’m not coining this, I’m quoting. “It’s like oxygen.” What does that come from? “You only notice it when you don’t have it.” What is the original? Is that about performance? I don’t remember.

Tanner:

Let’s go to the Googles. Oxygen… “Freedom is like oxygen. You only notice it when it’s gone.”

Josh:

Yeah. Web performance, freedom, all these wonderful things.

Tanner:

One and the same, performance and freedom.

Josh:

Yeah.

Tanner:

Maybe picking up a little more on this idea of like, “What is this thing?” You had a job somewhere in here in the past however many years at Microsoft, I believe.

Josh:

That is correct.

Tanner:

There was a sprinkling of performance in the job description. But what was this job that you had at Microsoft and how, performance what?

Josh:

Good question. I interned Microsoft 2014 in the summer. Great experience. I highly recommend the Microsoft internship. A lot of fun, learned a lot. I worked on not too much performance, web performance, critical stuff. I worked on an internal search engine that would take data from a few places like LinkedIn or Yammer, a thing, like it, I don’t even know how to describe it. It’s a thing enterprises use to let people talk to each other, closer to Facebook than Slack. And then like support forums and stuff and support agents could use that to determine whether a question was already answered or who was the best person to talk to. It was internal to one of the business teams, Dynamics AX. The web performance story there wasn’t huge because I did a frontend, but it was like a very basic MVP, and then we have this search engine, Lucene, that handled a lot of the ranking and then I just kind of took results in from different sources. Perf wasn’t a thing, like it wasn’t enough data to be slow.

Josh:

But then when I went full-time, I was moved almost immediately to Dynamic CRM, another business unit, and then the super secret awesome project we were working on got canceled and I was like, “I don’t wanna work on business software. I’m not interested in any of this.” They just cancelled.

Josh:

So I moved over to a much more exciting for me team. By the way, I say “for me” because there are a lot of really cool problems being solved by business software and I don’t want to disparage that entire swath of our industry. Not a good fit for me. I moved to a project called Sway.

Josh:

Sway is awesome. Sway is not something Microsoft has invested in in the last few years seriously, but I really wish they would. It’s something called intent-based authoring. You put in your content, text, images, tweet embed, whatever, and a few suggestions of like, “Oh yeah, this should be a presentation or a long flowing vertical document,” and it makes a document, or presentation, or whatever for you that you can easily switch as a user between those different views. And it’s accessible by default, like super duper accessible. It has like accessibility insights that the Word team later famously copied, which was great.

Josh:

I love spouting little bits of gossip from like internal Microsoft from like five years ago that no one cares about anymore. “Sway did it first!” “What is Sway?” No one knew about it. They never marketed it heavily, we barely had t-shirts. And eventually the team got merged and moved into Word Web. But while I was on Sway, I was already passionate about web and JavaScript and performance, so that was the team that I was on in Sway for a while. For the first, I think two thirds of it, I was on performance or similar working on things like speeding up image loading.

Josh:

One really cool thing was that Sway implemented its own tick system. And like how Full Screen Mario did, where like every 16 milliseconds it would check for updates to the screen because everything was super duper dynamic based off of like a representation of the user’s stuff. So I found a way to turn off that check thing if the user hadn’t interacted with the page for a while, which is really common when someone was giving a presentation. And doing so didn’t change the visible user experience, but it really, really helped with the battery consumption of the thing, which is another side to performance that no one really thinks about often, like battery usage when I’m in semi-idle mode. That made me really happy. We never actually shipped that, by the way. There was a bug that I could never fix, which was really annoying.

Tanner:

But still, the idea!

Tanner:

The idea!

Tanner:

The prototype, it was there. See, there’s another good one, though, battery life. I feel like with the books and going around talking about performance like, “Time, time, time, time, time—but also layout shifts!—time, time, time, time, time—but also battery!” I’m so curious about these interesting tangents where it goes, “Oh, you know, there’s this other thing where we’re trying not to get in the user’s way.” In that example, how did you find out that it saved battery life? Like do you guys have a battery life dashboard or were you running machines 60 miles an hour to see? Or like how’d you know?

Josh:

No, it was just a user complaint that their laptops would overheat or similar, when they were presenting in a hot space, or something like that. And then I went to the dev tools and saw that we were heavily executing JavaScript every frame, which was the only reasonable explanation for this. And then when I found a way to stop running JavaScript every frame, we never got a confirmation in production because we never shipped, but I’m pretty, I’m pretty sure based off of my basic experimentation that it fixed it. Which is another great thing with performance. Sometimes it just, you know, you don’t, you can’t be scientific, it just seems better, which is the worst thing in the world. But what are you gonna do?

Tanner:

This, by the way, is the whole reason that I got that statistics book. Because there’s a whole other side to performance, and maybe this could be all software, but definitely with performance, where measurement, like fundamentally, even more fundamental than time, performance is about measurement and measuring things. And then you say, “Well, how do you know?” Like deep at the core of performance is, “How do you know this is better?” You know, like are you going to chef’s kiss taste the website to know that the flavor was better? Or like, how do you know?

Josh:

Sprinkle some image, or whatever, defer tags. I have had plenty of changes where I thought it was faster, it felt faster, and then I was scientific, I took whatever, ten, twenty recordings, got the mean, median distribution, and found that no, I had made it worse. I needed to not do what I was trying to do. Happens plenty of times in performance.

Tanner:

Do you have an example? Is there one that comes to mind where it felt better and then everything actually was worse?

Josh:

Yeah. So the browser has a finite number, limited set number, I think it’s seven or nine or something depending on the browser and network, etc., of parallel requests it will make at once. So if you put too many preload tags in there, the browser may start preloading stuff and that stuff might not be necessary to preload.

Josh:

So actually I’m working on my little ChooseYourFramework.dev toy project site, which is on a forked version of my Pokémon stuff, and I tried—I’m as we speak this week working on the “create the production build” part of the game engine tooling. And in that production build tooling I had it preload all the assets that might get lazy loaded. The way it works now is first the game loads first, that’s the most important thing, and then asynchronously eventually the HTML menus for like options and mods and stuff load after it, because that’s not necessary. People don’t click the options within a half second of loading.

Josh:

So I preloaded that, but then that meant that the production version was loading all these things in preload, which then made later stuff that was more important take longer to load. So by removing preload tags, I improved game load performance. It was horrible. I hated it. I was very upset about it.

Tanner:

But you were so confident in the beginning.

Josh:

Yeah, preload tags are always the right strategy.

Tanner:

Always!

Josh:

What could possibly go wrong here?

Tanner:

Pretty sure Tim Kadlec posted something about a site that had like 1000 preload tags. I think I saw Matt Z at DebugBear did something similar. But I’m curious, when was the moment? Because you did it, you were so confident. It goes out live, like, when was the time that you said, “Oh crap, I really messed up”?

Josh:

Like ten seconds later. This was over the weekend, or recently. I didn’t ship. This is not a shipped project. I’m saying it now but I’m not like announcing it on Twitter or anything. I was just like, “Oh, this should work.” And then just for fun, I checked out how it impacted performance, which I thought would be a pleasant thing, and no, I had to undo the changes because they made it worse.

Tanner:

And just on a lark, you open dev tools and like hit the, or how did you do it?

Josh:

Dev tools, yeah. I recorded page load. The dev tools show you the nice visualization of load, DOMContentLoaded, etc., with Fast 3G throttling. But by now I’ve obtained in a good way the instinct of, if I make any changes that probably impacts performance, I will check it out. I will test it because I’ve learned the hard way time and time again that things don’t always make stuff better when you think they will. And this was one of them.

Tanner:

Man, I feel like this should become a running segment of the show. “What was a time that you felt so confident and then you realized you were blatantly wrong?”

Josh:

Yeah, yeah.

Tanner:

Great stories. Thinking of being a scientist, this may be a stretch, but I feel like you had a talk sometime in the past two years. What is this, Achieving Zero Frontend Crashes In Production? I haven’t been able to watch the whole thing, so maybe can you give me like a snippet of what was that talk about?

Josh:

Sure. It’s not about performance, but it is about monitoring and telemetry and team dynamics, which is something that I think if we continue down my personal history, we’ll have to talk about.

Tanner:

Now’s the time.

Josh:

And it’s zero with quotes around it, air quotes. So it’s impossible to have zero bug crashes in production. The idea was there are a series of things, there’s this user acquisition funnel of bugs, series of things that code has to go through to hit production. Therefore, a series of things bugs have to go through to make it from production.

Josh:

And the earlier in the funnel you go, the more bugs there are. And things that stop bugs early in the funnel, such as static typing with a language such as TypeScript or a linter such as ESLint, the more bugs they catch. Later in the funnel you get things like unit tests, you get user reports, you get regression testing, end-to-end tests, stuff like that.

Josh:

And I think it really coolly, in a nice way, mirrors the concept of the user acquisition funnel. Someone finds an ad, clicks, signs up, etc. So trying to emphasize each of those individual pieces of the funnel as valid on their own and important is good because that way your team doesn’t overfocus on one and underfocus on another.

Josh:

One area I think a lot of teams that I’ve seen underfocus on is telemetry from production using a service like LogRocket, Sentry, etc., to see bugs. Oh shoot, what was the really popular one? Datadog! Codecademy, later in my employment there I moved to Datadog, which had these really nice dashboards for backend errors, where you could look at all the logs and see correlations and stuff. And then their frontend errors dashboards were at the time still in beta. It was like, come on. This is real. But we were able to hack together some custom dashboards and therefore get a list of all the top bugs, that it was, say, all browsers or just Chrome.

Josh:

So then the talk talks about reduction techniques of going through the highest hitting issues, figuring out what they are when it’s really hard to determine what’s actually causing the bug, and so on and so forth. So yeah, would recommend that type of methodology and thinking.

Tanner:

I feel like there’s so many parallels, though, and I find it fascinating that you said this talk is not about performance. But so much of what you’re saying, I’m like, “Oh, telemetry.” Like field data is, that’s telemetry, like that’s what that is. Bugs that could be showstoppers. I mean, like, well, I guess is this site reliability engineering, would we call that? Is that the label we should put on this talk? Or where does this fit in the world of software?

Josh:

I don’t know. It’s amorphous. I guess you could, if you wanted to, put it under site reliability engineering, SREs. But I would put it as like a web platform concern, at least in the way that Codecademy teams are structured. Where I was—I’m no longer there now, the team still exists—we had a web platform team that dealt with things like cross-team, architectural decisions and best practices, standards, tooling setups. And part of that tooling setup thing meant setting up ways for teams to monitor their bugs, and then throwing bugs at teams when they caused them and didn’t fix them soon. So that’s a whole other political discussion of how do you set up a team to be the “yelling at you” team for other teams without them being instantly hated. Which, I actually thought we did a pretty good job of.

Tanner:

Of being tactful naggers?

Josh:

Yeah, well, there was a lot of learning for me. This was also my first time being senior and then also my first time later on being staff, which is…

Tanner:

Congratulations.

Josh:

Thanks. Senior II for other companies, startup senior though I should note.

Josh:

The construction of the web platform team was really good. We proved ourselves as a team by fixing a lot of issues with the dev tooling. We helped convert to TypeScript. We fleshed out the design system, which meant it took a lot less effort for other people to do things like use a button, which a lot of people appreciated. And we set up tooling for errors. But then also there was back of the mind, like, the nagging. And sometimes I went overboard, I will admit. It was very much a learning experience of what, like, should you publicly post in a team’s Slack channel that they are causing more bugs than everyone else combined and need to fix their stuff? Probably not. Should you remind them when you’ve told them privately three times to fix it? Depends on the team. Depends on your relationship with the team, really. I’d say, as a learning from there, know your relationship with the team and know the team’s relationship with everyone else. It’s easy to blame but hard to be productive.

Tanner:

Good advice. I feel like this is another theme, and why it’s been exciting to talk so far. Because I feel like performance, like this concept of performance again, there’s so much we can, if it’s not the same, we’re at least overlapping. There’s so much we can learn. If you wanna say, “I want to talk about web performance,” well, learn from the game developers of how they optimize things, right? Learn from the site reliability engineers of how you monitor and keep your site running efficiently. And then even into some of that more cultural political stuff, like, know your boundaries, know your relationships, know when to wag your finger and when to come alongside and help.

Josh:

Yeah, it all blends together. I mean, there’s so many cross-cutting, overlapping concerns and good techniques. Like I read a long time ago for the first time How to Win Friends and Influence People, which is just a great book, and that’s helped me immensely, not just with being able to hold a conversation, something I struggled with, in many ways still do. But like working with other teams, understanding how to get buy-in from stakeholders. Maybe don’t just tell them about how important web perf is, maybe relate it to the things they care about. Like things like that. Plus one.

Tanner:

Yeah man, this has been a lot of fun so far.

Tanner:

Up on my list of questions, I’ve got two, my two closers. Question number one, what is something that you find particularly interesting or difficult about web performance? And then, closer question is like, hey, what are you doing now? How can people follow you, learn more?

Josh:

One thing that I’ve found both interesting and difficult with web performance is third party scripts. They are the single biggest issue with website loading that I have seen. And it could be what I call “second party,” like another team inside a big company. But scripts that aren’t owned by your team. Teams generally are pretty good, at least I’ve worked with a lot of really good dev teams, so in my experience teams have been pretty good at caring about performance.

Josh:

There are always more things you can do, especially if they don’t yet have a dedicated perf or platform team. But like we, let’s say at Microsoft, we worked hard on performance for Sway and the landing page for Sway. And then the unified header and footer team, UHF, made this godawful header that I am unafraid to publicly bash because I hate it. It’s the bane of my existence. It lazy loads in eventually and pushes the rest of the page down for all users who might need to confirm cookies. Or at the time it was, you know, you’re on Edge advertise—excuse me, you’re on Chrome advertise Edge. Like all of our hard work on making the page load fast pfft, just pushed down on the page, hurt me.

Josh:

And then at Codecademy—and by the way, this is not me bashing on people without giving them any credit. Like there’s a good reason why people put these things. UHF, unified header and footer, was important for standardizing Microsoft site visual appeal, and consistency, and accepting cookies in a reasonable way cross-domain, cross-site, etc.

Josh:

For Codecademy, our big problem was Codecademy does a lot of marketing, which is really important for a small to mid-sized startup, or whatever you would call a ten year old startup. Like that’s really important stuff, but we were loading like LinkedIn, Bing, Google, Quora, like all these random tags at the beginning of the page. So one of my biggest happy moments with web performance in my career is that we worked with other teams to make a unified, GDPR-compliant, so tracking protection, data, stuff, compliant solution—it’s open source on Codecademy’s GitHub—where we wait a second after page load, then load the GDPR scripts that we use, I forgot what the third party provider is called, and then based off the GDPR thing we load like Segment and other scripts that load and stuff, which was just a huge improvement, because no longer were marketing teams adding raw HTML tags with blocking scripts through dev teams that were pressured to let them. It was just all done through one GDPR-compliant, loads after the page is done thing. The downside then is that you lose like the first second of user activity. Heavens forbid someone click a link in the first second of the page loading and you lose the attribution. That like never happens. So I was really proud of that and it helped performance a lot.

Tanner:

That’s fantastic. How about you? What are you doing now? Are you affiliated? Are you employed or what’s the game?

Josh:

The game is fun. I am not affiliated or employed with any company. I am a full-time open source developer, so in large parts I’ve actually moved away from performance and towards platform. I work on open source projects in the TypeScript ecosystem. So if your website uses TypeScript, you probably use something I’ve contributed to, such as TypeScript itself, or TypeScript ESLint, which is the tooling that lets you run ESLint on TypeScript code and also provides a lot of really good rules.

Josh:

I wrote the book that we talked about earlier, Learning TypeScript, which I would plead with the audience to buy because I don’t have a job and it helps me. And also rate five stars on Amazon because that really helps me too. You can sponsor me on GitHub. I’m Joshua K Goldberg on GitHub, Twitch, Twitter and dot com. I also have a blog and other projects, and I stream on Twitch twice a week.

Josh:

So sponsor TypeScript ESLint. You can find us on TypeScript-ESLint.io, sponsor me, all that good stuff. And if you have like open source TypeScript things you want my help with, like projects or questions, I’m always happy to help with things. That’s me!

Tanner:

Awesome! Well hey, Josh, thanks so much for coming on the show. Loved the time. Hope we get to do it again.

Josh:

Yeah, likewise. This was fun. Have fun with the new podcast stuff. Thanks Tanner, bye!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.