An Old-Fashioned Technology Stack
In the world of modern front-end, the architecture of Magento 2's client-side would be considered outdated. This isn't super surprising, though: development work on Magento 2 first started in 2010. Quite a bit has changed on the web in the last 9 years!
To help put that in context, here are some random notes about the web of 2010 to help frame things:
- Steve Jobs declared war on Flash
- The term Responsive Web Design was coined
- RequireJS was still a year away from a 1.0 release
- AngularJS and KnockoutJS were first released
- Browserify (first node module bundler) was still a year away from being announced
- Internet Explorer 8 was the latest version of everyone's favorite browser
- Google Chrome was 2 years old and was only a desktop browser
It's not uncommon for web apps with a long shelf life to begin getting slower as more features are added: Magento is not an exception.
If you don't have time to read the linked articles, I'd like to call out one quote in particlar from the always helpful Addy Osmani:
The Waterfall Problem
Writing modules in this format ensures that RequireJS can load modules both asynchronously and in the correct order. The benefits here are primarily:
- Opportunities for race conditions when loading > 1 file decrease
- Code can be written in a modular fashion, leading to easier testing and long-term maintenance.
This modularity, though, can be devastating to performance. As an application grows and dependency chains become deeper, this leads to a waterfall effect when loading the files.
To illustrate the problem, consider a small app consisting of 5 AMD modules.
Foo is the entry point for the application, this creates the following network waterfall:
Unfortunately, the browser cannot discover a dependency until it has both downloaded and executed the parent. In this example, the browser cannot begin downloading
Foo has completed.
To see this problem on a larger scale, here is an extremely zoomed and blurred out example of the waterfall effect when loading a stock Magento 2 homepage:
Bundling - The Wrong Way
To help put this in context, let's compare the page weight of 2 identical product pages in Magento 2.3.2, with and without bundling:
The built-in bundling feature allows you to configure the number of separate bundles it generates. However, this feature does not make a difference, because all scripts are loaded in a blocking fashion in the
eval at runtime. Hiding code from the browser until the last minute like this bypasses a significant number of performance optimizations present in modern browsers.
Paper Cuts Add Up
All these problems add up very quickly, even in a store with no modifications. As an example, I set up a store on Magento 2.3.2, and recorded the performance of loading the site on a mid-tier mobile device on an average 3G network, both with and without bundling. I think the results are telling:
Bundling - The Manual Way
The Magento DevDocs has an article that provides instructions to manually bundle a store with per-page bundles, by writing code to scrape the pages of a store in development or staging.
This process can yield good results, but the manual work involved tends to not scale for large stores or teams, and is extremely error prone. It also has the same blocking problem as the built-in bundling feature.
Just use webpack
Believe me when I say I'd much prefer to use an existing tool for this. In 2018 I invested a pretty significant amount of time working on a webpack plugin that attempted to shim some necessary functionality.
Progress was made, but at a certain point the cost to build and maintain the webpack plugin began to outweigh the cost of writing something that better fit Magento's architecture.
The only other way to move the Magento platform onto a modern bundler would require some significant breaking changes for both store owners and extension authors. Although this is tempting for the sake of performance, there is already a team working on a modern rebuild of the storefront. I just cannot justify large breaking changes when people are planning to start migrating their stores in the next few years.
For posterity's sake, here's an incomplete list of features that are not compatible with modern bundlers:
mage-init and x-magento-init
Magento is a server-rendered application, so most markup is typically written in
.phtml partials. It's common for developers to open a script tag and interface directly with the module registry via
window.define. These can be included on any page, at any time.
RequireJS was patched in the core of Magento with functionality that allows a developer to intercept requests for modules, and mutate or replace the result.
A Magento package can include a file named
requirejs-config.js, which is merged with all other packages' RequireJS configurations. A
deps key can be included, which signals that a file should be included and executed on all pages.
Magento provides (another) abstraction for reusable UI widgets. These have a special
template property that points to an HTML file, which is loaded via RequireJS.
I'm excited to say that these efforts were fruitful! Based on the progress so far, I'm confident we're headed in the right direction.
How Baler Works
Baler has 2 separate strategies for optimization
Baler then follows each entry points' dependencies recursively, building up a directed graph of all dependencies that can be statically analyzed. The graph is then flattened, sorted, and minified into a single bundle.
Finally, Baler creates a new version of
requirejs-config.js for a theme, with instructions that tell the RequireJS runtime to not fetch already-bundled modules 1 by 1. This is the trick that allows the core bundle to be loaded asynchronously, with no risks of race conditions for existing stores.
Baler parses and analyzes all
Not enough information is available during this stage to determine which templates are used together, so bundling of modules here is at high risk of shipping dead or duplicate code.
Instead, Baler takes advantage of HTTP/2 multiplexing and delivers these modules individually. The trick here is that a package (
Magento_Baler) can inject
preload tags into the
head, effectively flattening the graph and eliminating the waterfall effect. This means you ship a few more bytes due to loss in compression (larger blobs of text compress better), but you make up those bytes by not shipping any modules not used on the current page.
The (automated) process works roughly like this:
- Request comes in to Magento store
- Magento's layout rules run, yielding a final layout
- A list is collected of all
.phtmlfiles in the layout that will be rendered to the client
- Using the manifest generated by
preloadtags to the
headof the document, to give the browser an early signal for all modules in the graph. Modules already in the core bundle are excluded
State of Baler
The "Core Bundle" functionality is already fully implemented in Baler. This means that it's already capable of delivering scripts used on all pages of a store asynchronously in the
Until the "Dynamic Dependencies" work has been completed, dynamic dependencies will continue to load in with the waterfall effect. However, this shouldn't stop you from giving it a shot today. All testing done so far shows improvements on most metrics compared to built-in bundling or no bundling.
The following stats were collecting from a Magento 2.3.2 store, with sample data installed, visiting the "Echo Fit Compression Short" product record:
|Unbundled||X||495 KB||1.5 MB||176|
|Bundled||2.1 MB||7.6 MB||10|
|Baler||X||400 KB||1.2 MB||84|
I've skipped including runtime loading/execution measurements in this post for now, to instead encourage you to start playing with Baler yourself!
If you're ready to get started, the alpha guide is waiting for you.
- The built-in bundling feature in Magento is bad for performance, and should be disabled
- Waterfall loading of modules isn't great
- Baler is a new tool to tackle both these problems
- Magento is pretty cool for letting me burn up so much time looking for a solution to this, even after multiple failed attempts