Gatsby, the Open Source React-Based Static Site Generator, Gets Commercial Backing
“To paraphrase Tolstoy, all fast websites are alike, but all slow websites are slow in different ways.” — Gatbsy creator Kyle Mathews.
SSGFS: static site generator fatigue syndrome. It’s a thing, people. With five hundred plus SSGs roaming the wild these days — everything from Ace, written in Ruby, to Zodiac, written in awk and sh — SSG-related announcements likely generate a collective ¯\_(ツ)_/¯ amongst developers.
But not all static site generators are created equal. While all SSGs share the same goal — uniting data with templates in order to create assets to be served statically via a content delivery network — they also all face the same set of real-world challenges. Such as creating a “glue layer” between templates and services, scaling to extremely large/complex sites, and how to enable fast incremental builds.
A platform that solves all these challenges, resolving everything behind the scenes so that developers can just, you know, make stuff, all sounds like a front-end web dev’s happy daydream of applying cloud-native architecture to website builds. But. What if? What if we could build sites using that same model of snapping together components, data structures and design systems like so many Legos? Kind of like CI/CD for websites…
Well, it turns out this daydream is on its way to reality.
Meet the Gatsbys
GatsbyJS is a React-based SSG that first debuted in 2015. Built by Kyle Mathews and fully open source, Gatsby upended the SSG ecosystem by combining best-practice front-end development techniques (route-based code splitting, PRPL, service workers, and offline support) with dynamic data integrations via a rich set of plugins (WordPress, Drupal, Contentful, MongoDB, etc.), queried at compile-time. Best of all was when Gatsby v1.0 shipped with connectors to PHP workhorse CMSs like Drupal and WordPress. Meaning that non-techies could still comfortably push content in the familiar way, while devs got to make use of React’s component model. Developers jumped on Gatsby like birthday kids on cupcakes: Gatsby is already in use by literally tens of thousands of developers, and downloaded nearly 500,000 times each month.
Today, a second Gatsby joins the party. Mathews partnered with longtime friend (and Gatsby core contributor) Sam Bhagwat to bring forth Gatsby Inc. The impetus to create a company around open-source Gatsby came because, Mathews said, he realized that in order to reach full potential as a “real-time stream processing system, only applied to website builds,” Gatsby would need custom-built cloud support. And the way to make that happen was to build their own Gatsby-centric startup.
Islands in the Stream
Gatsby operated on the novel approach of taking the core tenets of stream processing (“Where you precompute all the state that you need it so reads — or, in this case, webpage loads — are super fast,” explained Mathews) and inventing a practical way to model static sites around them. It was a revolutionary notion: applying modern software engineering practices and tools to building websites. Even so, however, developers accelerating their projects by using Gatsby still had plenty of work to do around using Gatsby.
React’s robust toolset was undeniably helpful, as was Gatsby’s open source community dedicated to creating and maintaining plugins and libraries to optimize the SSG’s superpowers. Inevitably, though, the build stage is a barrier in every static site generator — a list that currently includes Gatsby — because whenever even a single change gets made, you have to recompute the whole site. Unsurprisingly, this makes scaling inherently tough.
Bhagwat and Mathews realized wanted they wanted was for web devs to simply be able to push code and have everything work — just like in serverless and hosted databases. Essentially, giving front-end devs the power to build websites by gluing together services quickly and seamlessly — an ability software developers, our cousins-in-tech, pretty much get to take for granted these days. This was followed by a second realization: even given the fact that Gatsby 1.0 has been built to take advantage of just this model, it simply could not be achieved without specially built cloud software.
To deliver this specially built software, Gatsby needed a specially built company devoted to bringing its innovative, apply-cloud-native-principals-to-web-development possibilities to real-world actuality.
“With Gatsby Inc, we’re building specialized cloud infra to do incremental, parallelized builds — taking the current Gatsby stream processing/cloud-native architecture and pushing it even farther,” Bhagwat — now head of enterprise (and also co-founder) for the new company — explained. The goal is “to practically eliminate the build step entirely: You’ll just make a change, any change — code change or content change — and it will go live immediately.”
“That is completely unique among SSGs, and it’s a technical advance that will vastly improve DX/UX!” Bhagwat added.
As it turns out, Silicon Valley agreed. Gatsby Inc.quickly found backing from San Francisco-based Trinity Ventures. Trinity, which specializes in supporting cloud native startups, recognized that highly interactive, bandwidth-intensive enterprise websites are current industry best practice. (It doesn’t help that so many of these are also single-page apps [SPAs]). While of course, the best-looking sites are typically also the slowest to load. Aaand every e-commerce company’s bottom line is directly related to their website’s load time.
“Gatsby flips this customer conundrum on its head by creating state-of-the-art websites that are also lightning fast,” said Trinity partner Dan Scholnick. “This solved a real need, so I’m not surprised that they’ve seen such extraordinary adoption throughout the OS world. I haven’t witnessed this level of fervor since Docker’s early days.”
Progress Is, Well, Progressive
Scholnick’s words ring true to every front-end web developer who has ever struggled to get a JavaScript-powered single-page app to show up in search results (using the escaped_fragment “feature”) — i.e., most of us. One of the (many) reasons we are especially captivated by Gatsby’s “progressive” experience is, when you run a Gatsby build, the output files are rendered in HTML. What this means is that, for search engines, all of the content is sitting there ready to index right from the initial fetch. Client-side, as soon as the page finishes loading, a React app gets mounted in the root div. Thus users enjoy the same rich browsing experience of an SPA, only without spending quality time watching the beachball spinner wend its way through a 5MB JavaScript file.
Welcome to the brave new world of progressive web apps, or PWAs. This is fundamental to Gatsby’s architecture: Gatsby was intentionally designed, said Mathews, to be a PWA website framework.
“Google does a lot of research about how to make fast websites, and PWA is sort of an umbrella term for these patterns,” he explained. “So with Gatsby, we just asked ourselves, why not bake these patterns, all these things that make a website fast, into a website framework?”
Basically, a PWA is a native app delivered via the web. Like a native mobile app, it has a shell allowing for app-style interaction and navigating — but with no need to download it from an app store. Fully self-contained, PWAs run right in the browser. They can load instantly, even in areas with bad coverage/connectivity, thanks to service workers — JavaScript-based client-side proxies — which allow them to load instantly, since already resident upon a device. Then, thanks to pre-caching, the app keeps itself up to date at all times, displaying the most recent version on launch — and rendering an instant and reliable user experience. Mathews explains,
There’s a close analogy to just-in-time manufacturing ideas. Companies found that the way to be the most responsive to customers is to actually avoid doing work ahead of time. When they did do work ahead of time this would paradoxically slow them down as the speculative work would get in the way of getting the work done that’s actually necessary (resource contention). For both manufacturing and web apps, there’s high inventory cost (unused code takes up memory) and a premium on responsiveness. The car customer wants their new car yesterday and the web app consumer wants their app running immediately. Any work you do ahead of time because “they might need it” gets in the way of the app being responsive to the user.
With both, you want to wait until the user asks for something and then work quickly to get it to them as fast as possible.
Ultimately, concluded Mathews, “The basic goal is only to make browsers do work that they have to do in order to achieve your goals for your site.”
So how does Gatsby actually put this PWA performance into the developer’s toolbox?
That’s for Gatsby to know, and devs to never have to worry about. “You shouldn’t have to think about performance — your site should just be fast. Your framework should just make your site fast,” said Mathews.
It’s not like the Gatsby team is trying to hide anything about how it all works. It is an open source project after all. They genuinely just want to let developers focus on the things that need to be custom-created for a website — the design and UI/UX — while taking care of the rest on our behalf. An arrangement most devs are delighted to live with.
Transparency
At this point, Gatsby Inc. is extremely new. A work in progress, even. Most startups would not be making formal announcements at such an early stage in their existence, but once again Gatsby is changing the game.
As original creator of Gatsby, Kyle Mathews wants to let everyone know that Gatsby Inc. will not in any way mean a move away from open source. Gatsby, the company, is building cloud services for Gatsby, the SSG — which itself will remain fully open source. As will most of what Mathews and team hope to build beyond the original platform. Even though this “what” is largely abstraction at this point, Mathews wants to go about launching a private company in an essentially open-source way.
Perhaps, in part, because getting financial backing for the startup has allowed Gatsby to actually bring on board several of its core open source contributors.
“Creating this company means we can invest even more effort in improving Gatsby core and its surrounding ecosystem,” said Mathews. “Able to work with open end upstreams to make targeted improvements benefiting not only Gatsby itself but the whole front-end ecosystem.”
