🤖🚫 AI-free content. This post is 100% written by a human, as is everything on my blog. Enjoy!

The web architecture ladder

June 24, 2021 in Engineering philosophy

There are many approaches to creating a website or application. In summary, anything that you want to be custom, costs more – more money, more efforts, more risks. When starting a project, it’s important to focus your efforts on the ground-breaking, value-creating bits, and save on everything else. Even when money is not an issue, spending development time on boilerplate that’s the same as everyone else’s is demoralizing and wastes the most precious resource – time.

I organized the approaches in a “ladder” of architecture complexity. The higher you climb, the more you pay, but the more capabilities you gain.

Rung 1 – Content on someone else’s platform

The ground level – something for everyone

Everyone starts here. And it’s no small achievement of Web 2.0 that anyone, from your kids to your grandma, can post something online.

Advantages

• Available literally to anyone.
• Free.
• Social features built in.

What is it good for

If your business is not on the internet, this is the best place to be. You can happily share knowledge, engage people, and sell products and services, without investing in web architecture at all.

Key names

Social media. Blog sites. Online marketplaces. Video hosting. Photo hosting. It’s everywhere, and we all rely on it.

Limitations

• Appearance is limited by the platform.
• Search and discoverability is limited by the platform.
• Often content is organized by time and older posts disappear into oblivion.
• Often funded with ads shown along your content.
• It is hard or impossible to leave a platform with your content and audience.

Rung 2 – Site builder

Site builders allow you to construct a site, on your domain, with your content.

Advantages

• Your own domain is the foundation of web presence.
• A wide choice of designs that all look well.

What is it good for

It’s perfect for building your brand on the internet. You don’t need a programmer or designer. But you still get a professional looking site. Plus, with a domain you own, you are free to change your architecture later, and keep the audience and backlinks.

And if you are making a more complicated web app, it’s worth using a site builder for your blog and support pages. Usually, those are an entirely separate project from the web app, and few assets are reused.

Key names

• Squarespace, Wix, Wordpress.com.

Limitations

• You can only choose from designs provided by the site builder. Note that this is only a limitation if you have a custom design ready. If you didn’t hire a professional designer, this is an advantage.
• You can’t meaningfully “program”. So, your site’s functionality is limited to what the builder offers. Modern site builders include advanced features like forums and storefronts.
• Content export can be complicated or impossible.

Rung 3 – Static site

Now entering the realm of programming. Starting from here, all approaches require some engineering work.

A static site is a collection of files served from the server as is. It’s called a “static” site because there is no backend logic. Static sites are pre-built. These days, we have utilities called static site generators that do the tedious task of HTML templating and page organization.

Advantages

• You can host anything.
• Full control over content. Content is stored in a simple, accessible way.
• Full control of site presentation.
• Easily automated – so you can integrate the build into other scripts.
• Content can be fully generated, and not entered manually.

What is it good for

Static sites are a good fit for automatically generated content. For example, documentation sites, or for presentation of script results. Maybe you’re running data analysis and want results to be posted online automatically.

Oh, and they are the approach of choice for engineers aspiring to write articles – my blog is built with Hugo and deployed to Vercel.

Key names

• Hosting: Firebase, Vercel
• Technologies: Hugo, Jekyll

Limitations

• There is no backend logic. Which, ultimately, means that your visitors cannot change anything on the site for other people to see. Only you can alter the site, and only at build time.
• You spend much more effort versus a site builder for much plainer and unimpressive results.

Rung 3.1 – Static site, refreshed by an automated script

Static sites are built by a script. If the content is also generated by a script, you can run the script automatically, on a cloud service.

For one example, I host a private site that filters RSS feeds. It was initially a Ruby app that processes feeds on-request, caching in Redis. It required an app server and a database. I replaced it with a scheduled AWS Lambda that filters the feeds daily and stores the output feeds on AWS S3. The feeds are served as a static site from S3. Now it costs nothing to run and nothing to maintain.

What is it good for

If you plan a similar service that scrapes, analyzes, and digests sources, consider the script approach.

Key names

• Scheduled GitHub Actions, or scheduled functions in AWS or Google Cloud.

Rung 3.2- Static site enhanced with JavaScript

JavaScript brought a revolution to the web. The crux of the revolution: instead of running logic on the server – costly and limited – you can run it in the client’s browser – for free, and effortlessly “scaling” with the number of users.

Because a JavaScript application is delivered as a simple file, it can be served from any static site hosting.

What is it good for

Even on a static site with no backend, with a bit of JavaScript, you can build interesting data visualizations, search, and other interactive capabilities – as long as all the source data is provided at build time.

Thanks to JavaScript, you can give convenient access to a large volume of data, by fetching it from static files (like you would from an API).

Key names

• D3, Lunr

Rung 4 – Single-Page Application, working with data stored on client device

A “Single-Page Application” is a frontend JavaScript application that functions without reloading the page. Typically, it is deployed on a static page. You can certainly deploy it on a static site.

As I mentioned in 3.2, you can build compelling JavaScript logic. What I didn’t mention is that the logic can handle user data, as long as it’s handled entirely in the browser.

If you want to process user’s data, the easiest way is to use some form of storage available in the browser:

• LocalStorage
• file uploads
• cookies
• IndexedDB
• URL hashes

Advantages

• No registration required – people can start using your app right away.
• No privacy concerns – data stays on the device.

What is it good for

Such an application can solve one-off tasks for the user. It’s also good for sensitive data that people might not want to upload to a server.

Key names

• React, PouchDB

Limitations

• User data is volatile
• Switching browsers, or reinstalling, or simply purging local settings might unwittingly wipe your app’s data.
• No publishing from users.

Rung 5 – SPA, working with data on third-party storage

All right, finally, what if users should be able to share or publish their data. Or, as a minimal case, what if you want a user to store and sync their data for themselves?

Then you need some backend storage. The simplest way to get backend storage is a third-party database-as-a-service like Firebase Firestore, or a document storage like Google Drive. User accounts in this case belong to the third-party service, you offer a login – typically via OAuth – and show their data through your app’s interface.

Advantages

• Allow users to share and publish data.
• Full-featured sync across devices without effort.
• Security better than your own backend.
• Scales without effort.

Key names

• Firebase Firestore, Google Drive

Limitations

• Data schema and access is limited by the third-party API.
• There is always some vendor lock-in for data. In some cases (like using a user’s Google Drive), you will not be able to migrate the data later.

Rung 6 – SPA, enhanced with some cloud functions

Cloud functions are bits of code that run on an abstracted cloud backend. These days, most static site hostings also allow running cloud functions. Cloud functions can fill the missing bits of backend logic that can’t be solved with storage. For example, they can handle payments, or send mails.

What is it good for

For when you need backend logic for one or two features. Typically, it has to do not with logic but with secret data – like API keys.

Key names

• Firebase, Vercel, Cloudflare Workers

Limitations

• Cloud functions are not as performant as a backend.
• They are slower to start up.
• The pricing is per-request, and you abuse cloud functions they get unreasonably expensive.

Be careful not to overdo cloud functions.

Rung 7 – Backend application, hosted on a Platform-As-A-Service

Finally, if your product is going to include some really intricate backend logic that is not just sync and access control and the occasional third-party API call, then you require a backend application.

When I said Rails is King, I meant it’s the king up here – if you can’t avoid using it. And I urge you to avoid Rails unless it’s absolutely necessary.

As a Ruby on Rails developer, of course, I’ve seen many applications that can only be done with a full backend. And earlier in my career, I made the mistake of not considering simpler solutions. When you start planning from a backend standpoint, it seems like there is plenty of business logic to implement. But much business logic can be placed on the frontend just as well as the backend. And a frontend-only app is simpler to develop and support. So now, I try my best to avoid writing mundane backend apps.

Advantages

• At this rung, you can finally serve dynamic HTML and avoid JavaScript! Therefore, if you don’t like SPAs, you have to skip from 3.1 to here to make your app.
• Full control over server responses.
• Requests are cheap (so you can serve APIs, for example).
• PaaS provides a simple deploy process.
• PaaS ensures a base level of reliability.
• Scaling is easy.

Key names

• Ruby on Rails and Heroku. That’s all you need. And I also like fly.io

Limitations

• Scaling is expensive.
• Features limited by the PaaS – for example, you might be limited to HTTP/HTTPS only, or not every kind of database can be deployed.

Rung 8 – Backend application, hosted on own server

Now entering the domain of DevOps. To achieve the following approaches, you need to know how servers work, how to secure them, how to keep them running, how to monitor… it’s really a massive domain!

The same backend application can also be deployed to a Linux server. When I started web development some 15 years ago, this was the state of the art – rent a virtual server, set up a database, deploy your app there. It seems easy enough – the app already runs on your developer machine. But trust me, keeping an app running, reliably, 24/7, without manual interventions is not the same.

These days, I would skip this rung and go to cloud infrastructure.

Advantages

• Host non-web servers (game servers, mail servers)
• Host any kind of applications – databases, services.
• Generally cheaper than PaaS, but it depends.
• Ultimate flexibility in setup (in the bounds of one machine)

What is it for

Legacy servers and hobbyists. Or a hosting provider – but that’s not a web app.

Key names

I suggest going with Linode.

Limitations

• Reliability – with your own server, you are responsible for keeping it up and recovering from crashes. It’s much more complicated than it seems.
• Scaling – a server has a natural limit to its capacity. Once you hit that limit, you will need to re-think your infrastructure.
• Security – you are responsible for keeping the server secure and doing damage control when the app gets exploited and credentials get leaked.

Rung 9 – cloud infrastructure

By “cloud infrastructure” I mean a backend assembled from services of a cloud provider such as AWS. It’s the place for large-scale projects, where everything is possible. The ultimate flexibility can cost more than any of the other approaches, but at this level you hopefully have the budget.

Advantages

• Unlimited scaling with little effort.
• Good inherent security – resources are isolated.
• Flexible access control – both for resources and for developers.
• When you get used to it, assembling cloud resources is simpler than setting up a Linux machine, and has fewer risks involved.
• At a certain scale becomes cheaper than a PaaS.

What is it for

Any modern application requiring a backend that doesn’t fit into the mold of a PaaS, or has outgrown a PaaS by scale, or has DevOps resources to set up the infrastructure.

Key names

The big three – Amazon Web Services, Google Cloud, Azure. I’m an AWS man, myself.

Use Terraform for configuration management.

Limitations

• You will get lost if doing setup manually through web UIs. Then you end up with a messy infrastructure that’s impossible to replicate. Do use Terraform.
• There is no standard to most of the cloud constructs. So, you’ll have to learn vendor-specific terms like “Fargate” or “Elastic IP”.
• Costs are hard to estimate upfront, and can run high.
• Some resources are not available in small sizes and are expensive for a project that’s just beginning.

Rung 10 – dedicated hardware

Hosting on your hardware – the final frontier

To be honest, I never went this far in my line of work. I bet any solution in this space is highly custom, and you only do it out of necessity.

You can also shortcut to “dedicated hardware” by hosting sites on your home/office machine. This I have experience with. In my opinion, the shortcut is not justified: yes, you save on hosting costs. But the support costs are still high, uptime is worse, and with own hardware, you gravitate to more complex solutions than you need. I would only run a hobby project on a local machine, if I don’t care whether it is up or not. So, it’s an infrastructure dead end.

Conclusion

If you’re a web developer, don’t be stuck with the technology you “majored at”. Explore other possibilities. As somebody who once made a blog powered by Rails, I can assure you – no matter how much you love an approach, it’s not the best for every occasion.

In my opinion, the most promising approach for a developer right now is single-page applications with a stock backend for sync. You can develop many interesting apps and solve many human needs with this. No one really cares what your backend is, so why spend time and money on it.

Liked the post? Treat me to a coffee