16. Pioneers. with Tim Neutkens Next.js
Tim Neutkens
Piotr Karwatka: [00:00:44] Hello, everyone! In today's interview we are going to talk about the front-end frameworks, gem stack, and open source. I couldn't have a better guest than Tim Neutkens, the lead engineer of the Next JS framework. The framework that grew alongside the gem stack became the de facto standard for react based applications. Hi Tim, thank you for accepting my invitation.
Tim Neutkens: [00:01:06] Hey, thanks for having me.
Piotr Karwatka: [00:01:07] How did you join the Next JS team?
Tim Neutkens: [00:01:09] So I found out about Next JS shortly after it was released, because it was part of the Vercel community back then. I have been contributing to Hyper, which is this terminal built on web technology, so Electron, HTML, CSS and 3X actually, React and Redux. And I've been contributing to that and then Vercel, and then the Next JS. At the time I was working mostly on PhD websites, mostly Magento actually, so mostly e-commerce. I worked at this agency that builds a lot of Magento sites in the Netherlands. And I found out about Next JS and I found it pretty interesting. So I started contributing to that as well, next to contributing to Hyper. And I basically kept contributing to it in my free time. Cause I was still working at this agency and then I did that for about a year or so. Eventually, after a few months of contributing, the chairman, the CEO of Vercel reached out to me and said: “Hey, would you be interested in joining Vercel to work on next year, as in some other things inside of the company?”. So I was like, yes, sure! Let's talk about it. So they flew me out to San Francisco to basically attend the first Vercel conference back then. I got to meet the team and all that. I eventually decided to not go for it at the time. I was 19, I believe… Which might sound crazy, but I wasn't ready for it yet.
Piotr Karwatka: [00:03:09] When was it, Tim? How long ago was it when you got this offer?
Tim Neutkens: [00:03:14] Three and a half years ago. April of 2017 or so. So I basically declined the offer which still may sound crazy. But then after a few months of contributing more and more to the Next JS in my free time. So I was working full-time at this agency and then also I basically kept concerning the Next JS in my free time. Besides that other 40 hours per week as well, cause I was just super into it. So I kept doing it like boot first issues, helping out the community, like all that stuff. And it was just a really good way for me to learn about JavaScript, like new JS stuff, all the things that we didn't do at work. Cause we were mostly a PHP shop, I learned in my free time. Then after a few months I was like: “maybe I should still entertain that offer that Cheryl made me?”. So I reached out and was like: “hey, if you still want to, I would potentially be interested still”. And this is like super open-ended, right? Cause I didn't know what the situation with hiring and everything was at the time. And then Cheryl got back to me and he was like: “yeah, if you'd be interested, we can probably work something out”. And that's basically when I decided to join Vercel to work on Next JS. There was basically one full time contributor at the time. And then we were two, and then this other full-time contributor Erin Noda went on to work on other things inside of Vercel. Then I basically started working on Next JS full time at the end of 2017.
Piotr Karwatka: [00:05:26] That's amazing. So you started your career really early, like you said that you got this offer three years ago, you were 19 years old and already used to work at this agency. So does it mean you started your career at the age of 16?
Tim Neutkens: [00:05:45] I was very fortunate because I was doing a practical education pretty much. In the Netherlands you have a system where there's multiple levels of education and you can work yourself towards higher education. So I was doing this practical education part that is before doing a bachelor. It lasts two or four years and then you get to do a bachelor, for example. It's like I could go into software engineering or something like that. But what happened was in my last year of education, I basically got to do an internship and by accident I ended up at this agency that builds PHP websites like I said. They were getting into building WordPress sites, doing custom PHP stuff, they were really big up and they also did a lot of Magento websites. So I was very fortunate to do an internship there for something that I wasn't actually going to school for, per se. So I wasn't doing a programming education, but I still got to do programming there. So I started there after a few months, the CEO of that company came up to my desk and he was like: “hey, can you go to a meeting room with me?”. And I thought either I did something wrong or something is going well. So I walked into the room and he was like: “yeah, you're doing super well!”. All the other people there, even the senior developers were like “yeah, we need to hire this person”. And he was basically like: “If you want to think about it, we'd like to hire you if you graduate from the education that you're doing”. So I got two choices back then. Either I go to do a bachelor in computer science or I go to work and I get a little of practical experience and keep developing my skills in that way. So eventually I decided to take the job. I joined the team full-time, but I was already doing a full-time internship, but I turned the teams full-time as an employee. I think it's 2015 or so, I was 18 at the time. Then they kept giving me work that helped me develop my own skills. I basically went from doing WordPress websites, building out those for smaller customers to building some of the largest e-commerce websites in the Netherlands also Europe, actually.
Piotr Karwatka: [00:08:58] Sounds amazing.
Tim Neutkens: [00:09:00] Yeah. So then I was doing mostly Magento sites. And then eventually I rolled into open source NJS. That got me hired at Vercel.
Piotr Karwatka: [00:09:15] Awesome. It sounds really cool. Let's talk about the Vercel and Next JS. How about the core team working with you? On the Primo, can you say more on this? You said that there were two other person working on the courtroom when you joined, right?
Tim Neutkens: [00:09:36] Yeah. So when I joined Vercel, it was really a small team, I believe it was only like 10 or 12 people or so. There was one person working on Next JS full time. And then there were a bunch of other people working on the Vercel platform and all that.
Piotr Karwatka: [00:10:02] What was first, Tim - the serverless hosting platform or the framework?
Tim Neutkens: [00:10:08] We went through some iterations of the platform. Vercel started in 2015, first we started with doing Docker hosting, then just hosting container. It's like you give us your code, we host it, that's it. Then we went into doing serverless Docker, which was basically like we brewed up the containers just in time. And then we tried to run your code as fast as possible. And that brings a lot of different issues of course, because you have cold bots, scalability. You need to scale out these containers for your customers. Customers wanting to run workloads indefinitely. So keeping their containers around forever and all that. In 2018, we named the new iteration of the platform, which floats completely serverless. It would just basically mean that you host serverless functions and stack with generators files and the Vercel platform automatically hosts those and generates the right files for you. So you don't even have to think about it, like, okay, I'm building a serverless application, you're just building an access application or a Gatsby or Next application. And then the platform automatically transforms the output of those frameworks into serverless functions, static files, demo, CDN and all that. So the platform was first but to build out the platform, you need to have a dashboard. So in 2016, Kushermo and Nayuki, the fires of Cell, basically wanted to build a dashboard that was highly interactive. So this starts to build this framework called N4 back then, which was sort of like Next JS, but not exactly like that outcome yet. It didn't use React. It was using some custom templating language. And you had to write a lot of plate code. There were a few iterations of that, where they built out this framework, that looked a lot like Next JS, but wasn't there yet. And then eventually, Kushermo decided to take this bet on trying to use React for that framework. They replaced the custom templating language with React and React components and truly built around adepth concept where you have a page directory and every file inside of a page directory is a separate route. So it's almost like page speed which is also why it's affected me. Cause I was doing a lot of PHP development back then. It was pretty easy to reason about it. You create a page directory, you put an index to the task export area component and JIRA and you’re off to go. So that was the reason why they created Next JS. It was basically the internal framework that they used to build the dashboards, the Vercel platform. And it basically evolved from that. So it was taking an approach similar to what created React up at the time, but it was more so a framework focus. There was no EGX function. Like you just had these like good defaults to get started. And then everything was built on top of that.
Piotr Karwatka: [00:14:11] That was very interesting. Thank you for this explanation. Maybe let's switch back to this question about the courting for a little while, because we got off route a little bit. It would be great to hear about your colleagues.
Tim Neutkens: [00:14:28] Yeah, definitely. So at the beginning of last year we decided to grow out the team that wrestle Next JS full-time, so we hired four more people. My team right now is five people that work on Next JS full-time, managing the community, triaging issues, creating pull requests, emerging pull requests, all that. So my team handled the full Next JS repository. We manage the whole thing. We fix bugs, we create new features. We write RFCs, all of that. So that's currently four people besides myself. Basically we manage the end to end experience. So it's most likely that if you created an issue on the Next JS repo, you'll talk to one of the people in my team or myself and we try to interact as much as we can with the community. I also tend to reach out to people, talk to them, find out the issues that they ran into, bugs they had that they haven't reported. And we try to make that experience as good as possible.
Piotr Karwatka: [00:15:54] Gotcha. That sounds amazing. So actually you have two core products at Vercel, one is the hosting platform, second is the framework. How much it helped Next JS to have this out of the box deployment universal and vice versa? I mean hybrid pages, image optimization, and super simple serverless functions are big selling points for Next JS. And that wouldn't be that simple to configure on different platforms without having the Vercel hosting.
Tim Neutkens: [00:16:27] Yeah, for sure. Even looking back at what I was explaining, we went from Docker containers to fully serverless hosting and there's a big gap between those. You have to think about all these different constraints. There is a cold boot if you use serverless and also when you're using the painters, but slightly different. Scalability is automated in a serverless environment and all that. And basically when Next JS was created, it was just a simple node server. Your rednecks start, it boots up this node server. You could compare that to Express. It's actually not using Express, but it’s sort of like that, you have to read the script, node server, boots out. And then the node servers hosted on local host retiremsent, and people can request that. And then in your Docker container you just forward port 80 to port 3000 and you're done. Then you still have to scale it out manually. You have to create more containers if the load balances, all that, which is fairly complicated, to be honest. But it's definitely doable. It's also still how many people host their own Xs applications. But we also wanted to evolve this towards a better solution. So we set out to evolve. When we started evolving the platform, we realized that we also had to evolve Next with it because at the time Next was just that node server and it couldn't host things in the serverless environments, for example. So when we introduced the serverless target that license to export the application into sort of Lenda function rappers that are not AWS Lendas per se, but it's this interoperability layer that just takes in requests and response from node.js and it answer the request there if you call it. Next has evolved with the platform, but also the platform evolved with JS. So when we started introducing more and more features, for example automatic static optimization which allowed pages that don't have blocking data fetching requirements to be statically generated automatically. That is what we needed to evolve the platform as well, because we wanted to statically host those files as well. We wanted to serve those from a CDN. If you request the resell homepage, which doesn't have blocking data factoring requirements, it should come from a recent close to you. For me, that would be answer to them for example, right? So that was one of the things that we wanted to handle as well on the platform. So the platform evolved with Next JS. And then even more. Earlier this year we introduced two new data fetching methods, one for blocking data requirements, which is. Cut server-side prompts. That means every time you do a request, you get a specifically rendered version of the page for that user, because it generates it on demand every time. This is a lot like how looks fabrics, right? You do a request to get a response back. And it's always unique to that user. But we also want to do static generation. So we also introduced get static prompts and we've got static prompts that you can do data fetching at build time or at request time and then cash it automatically. And that's what we introduced like in April, I believe, from full-time mine to next.js 9.3. and it allowed us to create this hybrid model for a next step application where you can have static pages, you can have automatic static optimized pages. So that would be a home page that doesn't have any data fetching requirements. And then you could also do service on renderings still. And then you can combine all of these in the same application. That means that your home page could be automatic static optimization, your blog could be fed in from data source, a CMS, for example. But still statically generated using that setting prompts and then your dashboards that are highly interactive could be either clients that rendered or service that rendered based on what your preferences are, for those applications.
Piotr Karwatka: [00:21:14] Gotcha. Let me paraphrase what you just described. Actually having the platform helped you a lot, especially in the, let's say early days, putting Node.js on production is a thing and I know what I'm saying. After VueStorefront we are getting a lot of enquiries just about that. Serverless and especially Vercel change it a lot. I remember when I first used it just using the now command in CLI was like magic, it just worked. It was so awesome. But now you are also handling a lot of different use cases, actually from what you said, most of the cases you can imagine statically generated pages, dynamically generated pages and hybrids. So that's really cool. I'm just wondering if you have any stats, how many people are using Next without the Vercel.
Tim Neutkens: [00:22:24] There's quite a lot of people that don't use Vercel when hosting Next JS. So this, especially with larger companies that have certain requirements or have our internal DevOps team and all that. They could be so big that they have their own data centers. Then they host their own like Next JS instances on that data center. An example of self host, the Next JS could be like Lyft, for example. But we also see a lot of really large Next JS applications like Alexa Top 100 where they are really like the largest sites on the internet, like Twitch, for example, or QQ, which is the largest news site in China. Any Alexa ranking, like QQ is crazy high, it's like above Instagram and all that. So it's like in the top 10 somewhere I believe in the fifth place or so. And they're using Next JS in production, scaled out to delivering millions and millions and millions of users. So definitely self hosting Next JS is definitely possible. And then we also see really large companies like Barstool Sports or HashiCorp that use Next JS in production and Vercel as well. So they do massive amounts of traffic, and that Debrick said fine as well. You can definitely host your total production website on Vercel.
Piotr Karwatka: [00:24:17] That's really awesome. What is the coolest feature you are most excited about in Next JS?
Tim Neutkens: [00:24:23] This year we spend a lot of time on making static site generation, really good, the same level as any other static site generator, but also providing more features than you would generally get from a static site generator because we can own the end to end, we regenerate the pages, we could also enhance them client-site. We can give you things that generally you see your aesthetic lead-generating, fully statically generated. It’s not as ideal as it could be because for example, if you're using a CMS to statically generate pages, you want your editor to still be able to see the changes they make. Inside of the website without having to redeploy everything right. So we introduced this thing called preview mode where you can basically say like, okay, like set this cookie to access API routes. And then you basically like those editors that go to this specific route to get the cookie can bypass the state generation and just get immediately rendered the immediately service, that rendered page that does the static data searching for you. So what that means is you can see, even at a third goes to a headless CMS, or any CMS really, and clicks on preview this page, than go to this API routes and then automatically end up on the page they wanted to preview with the content they changed without having all other users go to this, like change this content, basically. So that was one part of it. And then the other part is like, how can we prevent people from doing a full rebuild. When this editor actually sass, like, okay, I made a typo, I removed like one character from the page. Now I click save what is going to happen, as to rebuild the whole application, like that takes maybe like five minutes. And it could also be a critical change that you have to make in some application. So we introduced this incremental state generation approach where if you make this change is going to propagate into the application without having to do a full rebuilt.
Piotr Karwatka: [00:26:54] Somehow integrated with the tools like CMS that they are telling the system what change or how does it work?
Tim Neutkens: [00:27:03] It uses the still or revalidate approach. And if you're not familiar with this, it's a BRC. That basically allows a page to be regenerated in the background. Without having other users hit a still page or actually hitting the server directly, immediately. So what that means is generally, if you're doing a caching surgery and to have the surfs that render content cached, it's going to be cashed. But at some point it expires. And then when it expires you actually… if there's thousands of people going to a certain page, It would actually block on all those thousand people, being, like service not rendered on demand. And you know, obviously that one tab in general, because it would then hammer your database, it would hammer your application. You might have availability issues and all that. So what similar elevates, helps with, is basically saying, okay, like I say, every second dispatch needs to be regenerated. So it's always up to date sort of. And then when a user comes in, they get a still version of the page pack so that the cash version that was previously there, and then in the background, the server will regenerate the page and replace the cashed entry. So what that means is the end user will always get a cash version of the page, but you can still do regeneration in the background. And that's what Next JS has license to do, but just setting like one property and you don't have to like, learn about, pasted to be catching primitives and like figuring out which values are the correct ones. On the side of Next JS, that's just setting a property called free validates and setting it to a timeline value. So it's either like one second or a minute, or even longer, if you want to. And then once a, a user comes in after that timeout is exceeded, it will regenerate it in the background. And then the new page will be served to all users that come after that one user.
(...)
