TL;DR: we replaced Webpack with Vite. We’ve been happier ever since and our users got some value out of it as well. We built and open-sourced our own Flask-Vite integration; you can find it on PyPI: canonicalwebteam.flask-vite.
This article is the second in a two part series that covers why we did it and how. If you’re here for the juicy technical details, keep reading; otherwise if you’re curious about the reasons behind this choice, you can read about them in part one.
What’s a “backend integration” for Vite?
Integrating a traditional bundler – like Webpack – with Flask is trivial, doing the same for Vite is a bit more complicated due to the design of the tool. A simple bundler has a list of source files called “entry points” that it uses to build the application for production use; each entry point gets processed and transformed into a bundle. When running in development mode, the bundler watches these entry points and their imports and it rebuilds the bundles on every change. This means that, from the backend’s point of view, the bundles behave effectively the same across development and production.
Vite on the other hand operates in two very different ways: in production it behaves like a traditional bundler, but in development it runs an asset server that doesn’t bundle the source files. This means that the backend must distinguish between these two modes and generate the contents of <script> tags accordingly when rendering HTML.
The easy part, development mode
Vite’s asset server works by processing resources on demand when it receives a request with a path that matches a file in the project directory. It loads the requested file, processes it and sends the result to the browser as an ES module; as the browser parses the resulting script’s import statements, each dependency triggers a network request to the asset server, leading to a recursive exploration of the dependency tree of the source file. It sounds complicated – and it technically is – but loading resources using Vite’s asset server is actually quite easy. For example, all it takes to load static/js/app.tsx is adding a script tag that points to it through the asset server’s URL inside the HTML template:
<script type="module" src="http://localhost:5173/static/js/app.tsx"></script>
An interesting detail to highlight is that Vite can also process styles, either linked to by the HTML via <link rel="stylesheet"> or imported by JavaScript. These two methods expect different file formats, so the browser generates network requests with different Accept headers. The asset server responds accordingly: in the first case the request has Accept: text/css so the response will be a proper CSS file, in the latter case the request has Accept: */* to which the asset server responds with an ES module that injects a <style> tag with the stylesheet’s contents.
To avoid writing the script and link tags manually for every resource, we can create a custom Flask template function that takes a file path as an argument, builds the associated asset server URL and generates the appropriate HTML tag based on the file’s extension:
{{ vite_import("static/js/app.tsx") }}
<!-- ↑ becomes ↓ -->
<script type="module" src="http://localhost:5173/static/js/app.tsx"></script>
{{ vite_import("static/css/styles.scss") }}
<!-- ↑ becomes ↓ -->
<link rel="stylesheet" href="http://localhost:5173/static/css/styles.scss">
The hard part, production mode
What happens in production, where there is no asset server? We would have our bundles with the processed content of the source files, but the bundles’ paths wouldn’t match the source files’ – and it’s not like we would have access to the source files anyway. What happens when a bundle has dependencies? What if multiple bundles share a dependency? What if the dependencies are so big they split into multiple modules?
Vite solves all of these problems with the manifest.json file; its contents represent the dependency graph of the bundles, mapping each source file path to the associated bundle, marking entry points and linking them to the assets they depend on. Here’s an example of what a manifest.json file could look like:
{
"static/js/app.tsx": {
"isEntry": true,
"name": "app",
"src": "static/js/app.tsx",
"file": "app.js",
"imports": ["_lib.js"],
"css": ["assets/app-styles.css"]
},
"static/js/another-app.tsx": {
"isEntry": true,
"name": "another-app",
"src": "static/js/another-app.tsx",
"file": "another-app.js",
"imports": ["_npm-deps.js"]
},
"_lib.js": {
"file": "chunks/lib.js",
"name": "lib",
"imports": ["_npm-deps.js"]
},
"_npm-deps.js": {
"file": "chunks/npm-deps.js",
"name": "lib"
}
}
Putting this example in the form of a graph, it would look something like this:
When running in production mode, the vite_import helper parses the contents of the manifest.json file and maps the source files’ paths to the bundle paths it uses to build the HTML tags. Emphasis is placed on “tags”, plural: if a source file’s imports include stylesheets or if its dependencies have been bundled in separate modules, the associated stylesheets and bundles should also be linked in the document. To do this, after finding the bundle associated to the entry point, vite_import recursively explores the dependency tree of the bundle, creating <link rel="stylesheet"> and <link rel="modulepreload"> tags for all .css and .js dependencies it encounters along the way.
Taking the example manifest.json shown above, a vite_import call and its results would look something like this:
{{ vite_import("static/js/app.tsx") }}
<!-- ↑ becomes ↓ -->
<script type="module" src="/static/dist/app.js"></script>
<link rel="stylesheet" href="/static/dist/assets/components.css">
<link rel="modulepreload" href="/static/dist/chunks/lib.js">
<link rel="modulepreload" href="/static/dist/chunks/npm-deps.js">
Notice how the import function looks the same way in production and in development mode. Pretty nifty, huh?
The clever part, preventing bugs
An interesting byproduct of the way Vite works is that in development mode tracking entry points is pretty much irrelevant. The asset server only cares about the source file path in the requests it receives, so any file within the project directory can be served regardless of whether it’s an entry point or not. This however isn’t true in production mode, where Vite builds the bundles starting from the entry points, meaning that an empty entry points array in the configuration file is an immediate failure at build time.
While filling out this array isn’t a complex operation, it is redundant: the backend’s templates already have to link the files via vite_import, so we are effectively tracking the same data twice. And just as you’d expect, duplicated data means we have to make sure the two copies match: linking a file in the templates but not doing the same in the entry points configuration means that the site will work correctly in development, but will break silently in production! It would be nice if we could use the information from the templates to list the entry points and add them to the Vite configuration automatically in some way. If only there was some sort of API that would allow us to hook into Vite and edit the configuration programmatically at build time…
Oh, would you look at that, Vite’s plugin API has a config hook that allows us to edit the configuration object programmatically at build time! With a simple plugin, we can grep across all HTML templates looking for invocations of the vite_import function, parse the source file paths and inject them into the configuration just before the actual build process starts. Not sure if other backend integrations do something similar – I couldn’t find any that did – but it’s a neat way to avoid issues caused by missing entry points or obsolete file paths after refactors, and it also allows us to get rid of some of the tooling-related busywork which is always great.
The trivial part, using it yourself
Speaking about tooling-related busywork… To help out anyone else who’s trying to use Flask together with Vite and to spare them the effort of going through what’s described above, we released the Flask extension we built as a separate package for everyone to use. You can find it on PyPI under the name canonicalwebteam.flask-vite. It’s not perfect, and it might be too focused on snapcraft.io’s needs to be immediately usable for everyone, so community feedback will be incredibly valuable for developing it further.
As for the Vite plugin, for the moment it lives in our Vite configuration file; feel free to take a look. We plan to also release it as a separate NPM package, but that is still work in progress. We want to get rid of the implicit dependency on grep in order to support platforms that don’t ship it by default – yes, apparently they do exist and they’re also quite widespread.
So what now?
As expected, upgrading the tooling didn’t fix all our problems, but it did help us address them faster and with more confidence. Project startup times went from over 50 seconds to just under 1, build times went from 40 seconds to 6; most importantly, thanks to the automatic bundle splitting, duplicated dependencies across bundles stopped being a thing, so bundle sizes were instantly reduced by 54%.
With bundle size not being a pressing issue anymore, we were able to merge the publisher and enterprise features into the same SPA, improving UX and reworking much of the application layout code in the process. At this point the application structure was much easier to reason about, so we were able to improve its performance using lazy loading and code splitting via dynamic imports and React.lazy. This – and some other small tweaks – translated into a 32% decrease in JavaScript downloaded on first load and a 36% improvement on average across Lighthouse metrics. It’s almost as if good DX led to good software or something…
Obviously, good software is never good enough, so there’s still much work to be done, bugs to fix and things to improve; it’s unlikely that tooling will be one of them though, at least for the foreseeable future. So if you’re eager for a sequel to this blog post, don’t hold your breath. I’ll see you in 2035 for “Bringing snapcraft.io’s tooling into the 2030s”.