The Web Application Architecture - Intermediate Stage

Part 2: Solving Part 1

In order to discuss more intermediate topics of the proper architecture for a web application, we have to operate on the assumption that your application has already accomplished the fundamentals.

The problem is that the current frog-in-a-pot front end ecosystem has convinced most developers and - by extension - their companies that they have accomplished the fundamentals, and they're ready to move on to more advanced topics. I don't believe this is remotely true. However, I'm not your dad, so I can't stop you from reading this.
Consider, though, that if your development environment has a mandatory build system, or if any of your source code has specific signaling just for your bundler, or if you use React, Vue, Apollo, or any Flux implementation, you cannot possibly have accomplished even one of the three items necessary for the fundamentals of proper web application architecture. These are by no means the full list of tools and practices, but they are certainly some of the most popular.

That statement probably causes a visceral reaction in you, perhaps even disgust and anger. I am genuinely sorry to make you feel this way. Unfortunately, the level to which web developers have incorporated the tooling that has built up in their toolbelt over time into the core of their identities is alarming, and we can't keep dancing around the fact that the tooling is a huge part of the problem.
If you've incorporated these tools into your identity, you are not the problem! You are valuable and useful! 🙂 But we do have to look at the cold truth about our tooling, processes, and architectures.

So: read on - nobody can stop you. Just know that if you don't truly nail the fundamentals, anything else you do will be built on a foundation of sand.

It's the fun part

The silver lining here is that once you've nailed the fundamentals of de-complexity, robust foundations, and inter- & intra-app communication, everything else becomes much easier.
Removing complexity and establishing a standard way everything "talks" removes much of the difficulty of removing, adding, and upgrading things. You're then left to work on the actual problems that further your software's goals.
But wait! you exclaim, my tooling also allows me to focus on my core business problems instead of all the other junk!
But does it? In every "bog-standard" ecosystem I've worked in, developers almost never solved novel problems or dramatically improved overall application performance or created uniquely elegant solutions to dramatically improve customer experience.
Almost without fail, most developers spend the vast majority of their time fitting pre-defined solutions to inflexible libraries and systems.

So when you escape from that cycle - the one where everything gets worse and worse over time until you can't recover it any more and you either leave the company or are forced into a full rewrite - web software development gets fun again.

In the first part, I was intentionally vague about how to solve these problems. It's possible the right solution for your situation doesn't look like the solution for mine.
However, architecture is about blueprints, so here's one with more specifics.

There will always be software tools

You could absolutely 100% roll your own solutions to every problem, but it's unlikely that you should for the most complex parts of an application.
That complexity should be irreducible (for example: updating the DOM), and not self-inflicted.

There must always be systems

The original sin of most of the terrible software in modern web development is that they are tools that try to solve problems that can only be solved by systems.
What's the difference?
Tools plug into specific parts of systems. Systems are ideas. You can encapsulate a system purely in documentation. You should probably also have some code that implements the system, but it isn't technically necessary.

De-complexifying systems

Here it is: What does de-complexifying my application look like?

  • Anything that solves more than one problem at a time is too complex.
  • Anything that deviates significantly from your platform is too complex.

It should be pretty clear why those statements are true, but here's a quick explainer.
If something solves two problems at the same time ("problem" here are classes of application-level concerns, like say: requesting information from an API, request caching, and data binding), any issue with one problem becomes something that affects all of the other problems.
You have an issue with how data binding into your components work? Now you also have a problem with how network requests to your API work. That intertwinedness should actively raise your blood pressure.
In similar fashion, when something deviates significantly from your platform, you become less able to adapt to underlying changes to that platform. This is the problem with all of the VDOM frameworks in use today: they deviated significantly from the platform. The platform has a native component model and they're unable to even come close to the performance or flexibility provided by the platform.
Experienced engineers warned of this a decade or more ago. Code written in defiance of platform norms is zombie code: already dead and a danger to everything around it.

Simple front end application structure

Recall that simple is not the same as easy. Simple systems separate from each other so that they can interoperate more seamlessly by agreeing on standardized interfaces. This allows them to act as discrete black-box systems instead of having their internals deeply intertwined.


You could break most web applications down into these four (really three) parts:

Four green rectangles are in a horizontal line with some space between each. The are labeled, in order, 'Data', 'Business Logic', 'Display Logic', and 'UI'. The rectangles are connected with arrows. 'Data' has a double-headed arrow connecting to 'Business Logic'. 'Business Logic' is connected with a single-head arrow pointing to 'Display Logic'. 'Display Logic' is connected with a single-head arrow pointing to 'UI'. 'UI' is connected back to 'Display Logic' with a separate single-headed arrow labeled 'UX Reactivity'.
A basic web application has three (or four) parts: Data (retrieval, storage), Business Logic, and Display Logic (which results in part four: the UI). The UI sends signals back to the display logic based on user input.


It could help to better define these parts of the application, so we can rename each more precisely:

The four green rectangles from previously now have updated labels. Respectively, they are: 'Data', 'Application Value Proposition', 'Pages, Components, & Component State', and 'DOM'. Additionally, the arrow between 'Application Value Proposition' and 'Pages, Components, & Component State' is double-headed. In a more generic system, 'Business Logic' would unidirectionally drive 'Display Logic', but with this more specific outline, 'Pages, Components, & Component State' should likely have a feedback loop to the 'Application Value Proposition'.
A more precise web application has four parts: Data; the Application's Value Proposition (why does it exist?); Pages, Components, & Component State; and DOM. The DOM sends signals back to the component or page based on user input.

Specifically correct (separation of the parts)

In these images, each part of the application is connected directly to another part. This type of connection is how most applications today are built, and it encourages deeply broken interactions and spaghetti code. This is not only because there aren't clear boundaries about where things should go, but also because it fundamentally doesn't matter: the application's UI is virtually indistinguishable from its core value proposition. If you put application logic directly in a UI component, it doesn't matter, because the UI is the business logic.
This is - of course - madness (See De-complexifying systems, just above).
A robust, scalable architecture requires that each part of a system operate in a predictable way. To keep predictability to a maximum without complex interdependencies causing literal exponential growth of potential outcomes (see: Enumerate, Don't Booleanate for an exploration of explosive exponential growth of interdependent state on a micro scale - within one component), each part should attach to a standardized interface.
You could call this "Signals" or "Actions" or "Events" or "Messages," but whatever they are, the idea is that the application has a backbone of messages passing back and forth, and when a relevant message is noticed, something occurs. Critically, this is the only way the application should operate. Every behavior and data change should be able to be followed through a clear history of messages being passed. I have talked about this multiple times before. I also built gravlift to be this carrier backbone for any system.

The same four green rectangles as previously are still present, and labeled the same. The two spaces between 'Data', 'Application Value Proposition', and 'Pages, Components, & Component State' are much wider and their connecting arrows are gone. In those spaces, and below the line of horizontal green boxes are two new blue boxes, each labeled 'Messenger'. Each has two double-headed arrows. The two on the left 'Messenger' box connect to 'Data' and 'Application Value Proposition' respectively. The two on the right 'Messenger' box connect to 'Application Value Proposition' and 'Pages, Components, & Component State' respectively. This shows how the standardized messaging interface is the go between 'shuttle' for all application behaviors.
To standardize on communication in an application where each part is separate from the others (the goal!) we add a Messenger, which shuttles standardized message packets between the Data, Value Proposition, and Page/Component parts of the application. The DOM remains an artifact of the Page/Component part of the application and does not directly interact with the core structure of the application, only sending reactivity signals back to its parent Page/Component.

Different types of display logic

The "Pages, Components, & Component State" section is doing a lot of work, so we can split that up appropriately.

The four green rectangles and two blue rectangles from previously have been split up. Everything left of the second blue 'Messenger' rectangle remains the same: 'Data' and 'Application Value Proposition' still connect through the first 'Messenger' box. 'Application Value Proposition' still connects to the second 'Messenger' box. The final two green boxes have been replaced with a set of four new green boxes in a square layout to the right of the second 'Messenger' box. Clockwise from the top left, the new green boxes are 'Components', 'DOM', 'DOM', and 'Pages'. 'Components' and 'Pages' each connect to the second 'Messenger' box with their own double-headed arrow. 'Components' adopts the configuration from the previous image and has a single-headed arrow connecting to the first 'DOM' box. That 'DOM' box has a separate single-headed arrow labeled 'UX Reactivity' that connects back to 'Components'. The 'Pages' box connects to both 'Components' and the second 'DOM' box with unidirectional, single-headed arrows. This indicates that 'Pages' can render their own DOM and other components, but components can't render pages. Only components have UX Reactivity.
Pages and components are two sides of a coin, but they have slightly different usecases. Primarily, pages can render components and DOM. Components can only render other components and DOM. Only components describe UX reactivity. DOM is still the artifact of these two classes of display logic.

Defining data

For many people, defining data is difficult. This is made especially difficult because many tools improperly conflate the concepts of application state (what is the application doing right now) and core application data (what is the information that any user is here to see). These tools incorrectly treat these two concepts as the same, and store them the same way and provide the same interfaces for accessing and mutating them.
The clear defining line is "If you wouldn't store it in your database, it's not data." Loading indicators, which sections of a page have been collapsed by the user, which temporary view modification is enabled, and more are all examples of data you wouldn't store in your database. These are component-level state concerns and don't belong in data.
Data is a broad abstraction of three separate things. Your application's value proposition should not be aware of these distinctions; that's why the Data interface is separated by the standardized Messenger.

  • Browser Storage
  • Single-Concept Utilities
  • API
The previous image's structure remains. The 'Data' box now has a trident arrow to three purple boxes. They are labeled 'Browser Storage', 'Single-Concept Utilities', and 'API'.
The Data interface blends access to a back end API, browser storage (for local caching), and single-concept utilities.

The "single-concept utilities" above requires a bit of explanation. It is very common to need to do certain things with known types of data that should happen based on raw data at this low level of the application. For example, you may be building an app that allows users to view and rate dogs (they are all 12/10). After the application gets data from the API, it may need to import the normalize function from Dog.js and make sure to smooth over any potential rough edges in the data from the API. Dog.js might also provide some helpers like makeNew for when someone is submitting a new dog, and the system needs a "default" Dog value for them to fill in.
The two-fold key is that these utilities are tightly focused (for example, just Dog.js, not normalizeData.js) and they are limited to data. There's a whole part of the app for the value proposition code, this area should just be for unavoidable data mungeing.

More elaborate components and component DOM

Components hold a bit of information (the state of the application, which we covered earlier in that we know it is not data). Component DOM also updates regularly, which means we should define how that happens.

Everything from the previous image remains intact. A purple box labeled 'Finite State Machine' is connected to 'Components' with a single-headed arrow. Two purple boxes labeled 'Surgical Updates' and 'Web Components' are connected to the Components 'DOM' box with single-headed arrows.
Application state is encapsulated in component Finite State Machines so that it can be both catalogued and repeated. DOM updates are done as little as possible - updating only what's necessary, not recomputing and re-rendering the entire tree. Components use the web-native component model.

A blueprint for long-lasting, high-performing applications

And so we have arrived at the simple application architecture. Had you followed each of the principles in part 1, you may have arrived at a similar architecture. It is not necessarily easy, but the best things never are. It is undoubtedly elaborate, but most meaningfully complex web applications are elaborate. In this structure, the complexity is kept from seeping into every corner of the code.

  1. The application is split into four discrete parts:
    1. Data
    2. the application's Value Proposition (the Business Logic)
    3. the display logic, split between Pages and Components
    4. the rendering artifact of that logic: the UI, which returns reactive signals back to the components
  2. Communication is passed among the Data, Value Proposition, and Pages/Components by a message system with a standardized format
  3. Data reflects database information. It is not application state. The Data application layer abstracts:
    • accessing the API
    • Browser Storage (local cache, for one example)
    • Single-concept utilities, which help normalize and interact with data constructs
  4. Components use Finite State Machines to catalogue non-exponential lists of application states, the non-data type of information your application must manage.
  5. Component DOM updates are managed by the family of Web Components browser features (like Shadow DOM, custom elements, and constructable stylesheets). These updates are not full-component re-computes (like VDOM libraries), but instead maintain connections to live versus static parts of the template, and only update those parts that have received changes.