This sprint we’re going to focus on why and how we break up software into different components, such as a frontend and backend.

We will do this through a worked example. We will build a quote website, which lets people get quotes, or add quotes to a list.

We will supply most of the code for you. You should read the code and make sure you understand it. Then you should copy it onto your computer, run it, and experiment with it. Later, you will need to deploy it to the internet, and you will need to modify the code yourself.

Next sprint, you will be building your own website. It will be similar, but not the same. You will build an interactive chat room, where multiple users can chat in real-time. We will not supply the code. You will need to apply all of the skills you’re learning this sprint to write and deploy the code yourself.

A lot of this sprint focuses on limitations 🧶 🧶 Limitations A limitation is something we can’t do, or which is hard to do. and trade-offs 🧶 🧶 Trade-offs A trade-off is when we gain some benefit, but it also has some cost/draw-back.

There isn’t always a clear correct choice between two approaches - often they have different benefits and draw-backs. .

When writing software, we are always thinking about limitations and trade-offs. We always have to make choices. Like whether to use a for-loop or map. Or whether we want users to register a username that only they can use, or whether users can use any name they want.

Sometimes the choices we make are obvious. But a lot of the time there are advantages and disadvantages to different approaches.

Let’s use that example of user registration.

What advantages and disadvantages can you think of each option?

Neither approach is obviously better - they have different advantages and disadvantages. Sometimes those advantages and disadvantages are not about the same topic.

Often when thinking about limitations and trade-offs we are sorting requirements by importance, we are finding the places that problems are happen, and we’re choosing where things belong. There isn’t always one correct answer - we can often build a system many different ways.

Read about the Internet

Check you have achieved each learning objective listed on this page.

You’ve made static 🧶 🧶 Static Static has a lot of different meanings in different contexts.

Here we mean: The same files are always served from a server to a browser - no extra processing is done by the server to work out how to respond to a request. websites, both as part of the “world-wide web” reading, and in previous courses/modules.

These were HTML pages, which perhaps included some JavaScript. The JavaScript made them interactive. But they were still static.

A static website is one where your web browser requests some files from a server, and all the web server does is hand over the files.

From that point on, your web browser works on its own. Any event handlers, timers, etc are processed by the web browser.

The job of the server was just to hand over files. Those files are the same, no matter what user asked for them, or what computer they were using.

We often call these static files a frontend.

We can build impressive, useful websites with just a frontend. We can include images, videos, and sounds. We can react to user input. We can change what we show on the page. We can even use things like the Web Storage API to store data so that if you leave the page and come back, your data is still there.

Almost all of this curriculum website, for instance, is a static site. It contains a lot of content. It is interactive (e.g. has quizzes). A few small parts of it are not, e.g. the attendance register in the day plan uses a backend to store data.

But frontends also have limitations.

Limitations of a frontend

Four major limitations of a website which only has a frontend are:

1. We only have access to information we knew when we made the frontend.
2. We can’t share information across computers.
3. We can’t interact with other users.
4. Everything is public.

Let’s talk about each of these.

When we ask a server to serve us a static frontend, it can only give us the files it knows about.

Imagine we wanted to display the current weather in Rome on our website.

How could the server know that information? How could it give it to the user?

We know that our static frontend can only serve the files it knows about. So someone will need to save in a file on the server what the weather currently is.

There are a number of ways we could achieve this, e.g.:

• We could never update our file, and just serve out of date data.
• Someone with access to the server could connect to it and edit the file.
• We could set up some automated process which checks the weather, generates a new file, and uploads the new version to the server.

If we only host a static frontend, it is possible for us to update the information in it. But it is limiting.

Another approach, is that our frontend could know how to request information from a backend. That could be a backend we run, or one that someone else runs.

We have seen this when we have used fetch. fetch allows our static frontend to ask some other server for information.

We might not know whether the server we’re asking for information is static or dynamic:

It could:

• Only serve information it already knows in its own files (i.e. it is a static frontend itself).
• Look up the information from a database which may change.
• Ask yet another server for information.

Even if we use fetch from our frontend, it is still a static frontend. We still serve it to users by giving them the same static files.

If someone else already runs a server we can fetch from, we can probably keep our own website just being a static frontend.

But if we need more dynamic information, and there isn’t a convenient place we can fetch it from, we may need to write/host something we can fetch from ourselves. We call this a backend.

The Web Storage API allows a website to store data so that if you leave a website, and go back to it, it remembers some things.

A frontend has no way of bringing your data with you when you move between different computers.

Imagine you’re using a to do list website on your computer, which only has a frontend. You add an item to the list. We can use the Web Storage API to make sure if you close the tab, and go back to the list, the item is still there.

But if you open the website on your phone, you won’t see that item. Because the Web Storage API only stores the information locally in your web browser - on your computer.

This is one of the limitations of a static frontend - on its own, it can’t share information across devices.

If we want to share information across devices, we’ll need some backend to store that information, and share it.

One computer needs to tell the backend to store it, so that the other computer can ask the backend for the information.

We could write our own backend, or we could use an existing service like Firebase which provides backends for things like authentication and data storage without us needing to run a backend ourselves.

But if we want to share information between devices, we need to use some backend.

Any website that lets you log in will involve a backend.

We know if we want to share information between different computers, we need some backend to store that information so that the other computer can ask for it.

When we have multiple users, we have the same problem.

Imagine we’re building a chat website, where multiple people can visit and talk with each other.

We couldn’t build this if we only had a static frontend. When one user sends a message, other users wouldn’t have any way of knowing it was sent. The message doesn’t go anywhere, it stays in the sender’s browser.

If we want different users to be able to interact in any way, we need a backend they can interact through.

There are lots of examples of interaction, e.g.

• Live chat
• Seeing each other’s posts
• Liking someone else’s picture
• Seeing what someone else wrote in a list

All of these things require storing information from one user somewhere, so that another user can ask for it.

On websites, users don’t interact directly with each other. They interact with a backend.

A frontend is served, completely, to a web browser which then interprets it.

This causes problems if there is information we don’t want to give to the user.

An example is a password.

Imagine if we had a website, and wanted users to enter a password in order to access it.

Somewhere, we’ll need to write some logic which compares the password the user entered with the actual password.

If we only have a frontend, this poses two problems:

1. We need to give the web browser the actual password, in order for it to be able to compare them. And everything we give to the web browser, a user could also see if they wanted (e.g. by monitoring their network traffic).

2. A website can’t trust a web browser. Once a website hands information over to a client, it doesn’t know what the client will do with it.

   For instance if a frontend has three pieces of data, but wants you to tick a box before it shows them to you, the user can probably bypass that agreement by running some custom code in the web inspector console.

   If we need to guarantee the user ticked a box before being shown data, we can’t include the data in our frontend until after the box has been ticked. Which means we need to ask a backend for it.

We know that a frontend is a collection of static files that can be handed to a web browser, and which the web browser will evaluate.

A backend is a program which a frontend can communicate with. The frontend can ask it for (or tell it) information, and the backend can respond.

A backend can be written in any programming language. It doesn’t need to be the same programming language as the frontend is written in (but it can be).

Most backends are long-running. They are programs that keep running for a long time, waiting for a frontend to communicate with them. This is important - if the program wasn’t running, the frontend wouldn’t be able to communicate with it.

Not all websites have a backend. There are lots of websites which are just a frontend. But if we need to overcome any of the limitations we’ve explored, our system will need a backend. Sometimes we need to write the backend ourselves. Other times we can use one someone else has written.

There’s nothing special about a backend. It’s just a program that can be communicated with. You’ve already written plenty of programs.

A backend is just a program, but it gets its input from an HTTP request over the network, instead of from a command line flag or file. And it writes its output in an HTTP response over the network, instead of to stdout or a file.

Here is a small backend written in JavaScript:

import express from "express";

const app = express();
const port = 3000;

const quotes = [
  {
    quote: "Either write something worth reading or do something worth writing.",
    author: "Benjamin Franklin",
  },
  {
    quote: "I should have been more kind.",
    author: "Clive James",
  },
];

function pickRandomQuote() {
  const index = Math.floor(Math.random() * quotes.length);
  return quotes[index];
}

app.get("/", (req, res) => {
  console.error("Received a request for a quote");
  const quote = pickRandomQuote();
  res.send(`"${quote.quote}" -${quote.author}`);
});

app.listen(port, () => {
  console.error(`Quote server listening on port ${port}`);
});

If you save it in a file, make sure there’s a package.json file in the same directory, npm install express, and run the file, it will start a server listening on TCP port 3000.

Express is a library which lets you listen for HTTP requests, and describe how you want to respond to them with callback functions.

Only two things in this file are new to us:

app.listen(port, () => {
  console.error(`Quote server listening on port ${port}`);
});

This code tells Express to listen for incoming HTTP requests on the port specified by the port variable (and also to log to stderr saying it’s doing so).

app.get("/", (req, res) => {
  console.error("Received a request for a quote");
  const quote = pickRandomQuote();
  res.send(`"${quote.quote}" -${quote.author}`);
});

This code is instructing Express how to handle a request. It says “If you get a GET request for the path /, call this callback function”.

What this callback function does is pick a random quote from an array, format it as a string, and send it as the body of an HTTP response. It also logs to stderr when it does this.

Our example backend is stateless 🧶 🧶 Stateless Stateless means it doesn’t keep state. It doesn’t remember things between one request and the next. One request cannot impact another. . If you make several requests to it, it will always do the same thing (even though doesn’t always return exactly the same result!).

If you made a request from a different computer, or from different country, or on a different day, it would keep doing exactly the same thing. If you restarted the server, it would keep doing exactly the same thing.

If our backend allowed users to add new quotes, it would start having state. It would need to remember what quotes had been added. It would be stateful.

import express from "express";

const app = express();
const port = 3000;

const quotes = [
  {
    quote: "Either write something worth reading or do something worth writing.",
    author: "Benjamin Franklin",
  },
  {
    quote: "I should have been more kind.",
    author: "Clive James",
  },
];

function randomQuote() {
  const index = Math.floor(Math.random() * quotes.length);
  return quotes[index];
}

app.get("/", (req, res) => {
  const quote = randomQuote();
  res.send(`"${quote.quote}" -${quote.author}`);
});

app.post("/", (req, res) => {
  const bodyBytes = [];
  req.on("data", chunk => bodyBytes.push(...chunk));
  req.on("end", () => {
    const bodyString = String.fromCharCode(...bodyBytes);
    let body;
    try {
      body = JSON.parse(bodyString);
    } catch (error) {
      console.error(`Failed to parse body ${bodyString} as JSON: ${error}`);
      res.status(400).send("Expected body to be JSON.");
      return;
    }
    if (typeof body != "object" || !("quote" in body) || !("author" in body)) {
      console.error(`Failed to extract quote and author from post body: ${bodyString}`);
      res.status(400).send("Expected body to be a JSON object containing keys quote and author.");
      return;
    }
    quotes.push({
      quote: body.quote,
      author: body.author,
    });
    res.send("ok");
  });
});

app.listen(port, () => {
  console.error(`Quote server listening on port ${port}`);
});

Here we have added a new request handler. If someone makes a POST request to the path /, we try to interpret the body they posted as a JSON object.

If we can find a quote and author in it, we will store it in our list of quotes, and start serving it up to future requests.

If we can’t process the request, we return an error describing what went wrong.

The details of exactly how we understand the request aren’t important. The important thing is that we are taking information from the request, and are then modifying the quotes array.

Because we’re modifying the quotes array, it is now state.

We’ve already explored limitations of frontends.

We know a backend is just a program that runs for a long time.

Lifetime

A major limitation of backends is that “a long time” probably isn’t forever.

Sometimes we change the code of the backend. Or we need to restart the computer it’s running on for an upgrade (computers are real, physical things!). Or its computer loses power and we need to start it again.

When this happens, the program starts again.

Think back to our quote server that allows users to POST new quotes. If we had to stop the server and start it again, we would lose all of the quotes users had saved. They’re just stored in a variable, and a variable only lasts while the program it’s in is running.

Location

Another major limitation of a backend is where the code runs.

A backend’s code runs on whatever server it’s running on.

In contrast, a web frontend’s code runs in the user’s web browser.

Latency

One problem here is latency.

You’ve already seen latency problems when using fetch. We used Promises or async/await to handle a latency problem. And we did things like set placeholder text while waiting for data from a fetch.

Imagine if you had a “dark mode” button on a website, but clicking it required making a request to a backend.

Depending on where the backend and the user are physically located, it may take anywhere between 1ms and 500ms for a request to go between them.

If every time you clicked on something on a web page you needed to talk to the backend, you may need to wait half a second just for the request to travel to the server and for the response to travel back, ignoring how long it takes to actually process the request. This would be unusably slow for many applications.

So sometimes we do work in a frontend to avoid needing to talk to a backend.

For instance, many websites that show you a list of information let you sort or filter the information. We often implement sorting in the frontend, so that we don’t need to wait for a round-trip to a backend. Sometimes this even means we implement some functionality twice - once on the backend and once on the frontend, so that on first page load the backend will do some sorting or filtering, but if you interact with the frontend, additional sorting or filtering may be done there.

Context

Because web frontends run in the user’s own browser, they have easy access to lots of information about the user’s computer. For instance, a user’s browser knows what language to to use, what the user’s time zone is, how big the browser window is, etc.

If our frontend code were instead running in a backend, the browser may need to include all of this information in every request it makes, just in case the backend needs to know it. This has a couple of drawbacks: It makes the requests bigger (which makes them slower, and maybe cost more), and it ends up sharing lots of data with the server that it may not need, which may compromise the user’s privacy.

Imagine you’re trying to buy a sofa. There are a few ways we could do this:

1. We could take the entire architectural plans for our home to a shop, and for each sofa work out where it could fit.
2. We could go to the shop, and for each sofa we may like, go home and see if the sofa would fit into our space.
3. We could measure the space we want to put it in, and then go to a shop where you can compare all of the available sofas against those measurements.

Approach 1 requires transferring a lot of data. Approach 2 requires a lot of traveling back and forth. Approach 3 tries to minimise both of these things.

Pull not push

A backend lives at a well-known address - we know how to connect to it. A user’s web browser does not.

This means that a backend cannot try to open a connection to a user’s web browser. The web browser needs to initiate the request, and then the backend can reply to the request.

Once a web browser opens a request to a backend, there are some ways to keep a two-way communication channel open. But the very first request needs to come from the web browser.

When trying to decide which components we need, and in which component functionality should live, it’s useful to think about the capabilities and limitations of each.

A few guiding principles:

• If you need multiple users or computers to share some information, you need a backend to coordinate this.
• If you need your data to be reliably persisted, you need a database (or other storage component) to store it.
• If you need things to respond very quickly to a user interaction, you probably want to do this in a frontend.

Try to identify which components we would need for each of these applications:

We’re going to take a frontend you’ve already made, and overcome one of its limitations by adding a backend.

You should already have built a quote generator frontend before. The quote generator you already made used a static array of quotes that were hard-coded 🧶 🧶 hard-coded Hard-coding is when we write the exact data in the source code of our programme, rather than fetching it from some data source or generating it. in the frontend.

We will add some extra functionality to our quote generator. Users will be able to add their own quotes, and then other users on other computers will be able to see the quotes that were added.

Because we want to be able to store data across computers and users, we know we will need a backend.

Because we want to be able to change what data we’re storing, we know our backend will need to be stateful.

Because we don’t want to have to learn about databases right now, we’re going to accept the limitation that when we restart our server, we will lose any added quotes.

Communication protocols

Before we get started, we should agree how our frontend and backend are going to talk to each other.

The example stateful backend we looked at before exposed this API 🧶 🧶 API An API - Application Programming Interface - is a description of how one program can interact with another. :

This API is asymmetric:

When you POST information to it, you post structured information - the backend can easily tell which part is the author and which is the quote.

When you GET information from it, the information is less structured - the server has already formatted the author and quote into a string.

Pre-formatting the data may be convenient for the frontend if the backend knows exactly how it will be presented. But it takes away some flexibility from the frontend. If the frontend wanted to show the quote in italics, or the author in bold, this would be hard, because it would need to split the formatted string back up into its constituent parts.

We probably instead want to change our GET endpoint to also return structured information, which the frontend can choose to format how it wants:

We could also design different APIs for our frontend and backend to communicate - the important thing is that they agree on what API we will have.

Now that you have a backend which meets the API specification we designed, let’s make a frontend that can talk to it.

You are free to borrow parts of your previous quote generator website, or make one from scratch.

import cors from "cors";
app.use(cors());

Because our frontend talks to our backend, it has fewer limitations than the previous quote generator you made.

We don’t currently expose the capability to add new quotes in our frontend. But our backend does allow this. A user who knew where our backend was running, and what API it exposes, could use curl to add quotes, or could even build their own frontend for adding quotes.

Websites tend to only be useful if they’re running somewhere that people can access them.

We need to deploy our frontend and our backend on the internet so that people can use them.

And they need to know how to talk to each other. In your frontend you probably hard-coded your fetch to fetch from http://127.0.0.1:3000.

First let’s deploy our backend somewhere. Then when we know its address, we can update our frontend to talk to our deployed backend, and then deploy it too.

TODO(https://github.com/CodeYourFuture/curriculum/issues/1337): Add some pictures of this when we have a well-described backend hosting platform.

Make sure you can use your frontend which was deployed to Netlify. Try adding a quote, and make sure it shows up in your frontend.