Understand Session and Cookie

Most people are familiar with cookies. For example, after you log in to a website for a while, you might be asked to log in again. Or, if you like web crawling, sometimes websites can block your crawlers. These are all related to cookies.

If you understand how the server handles cookies and sessions, you can explain these situations. You may even be able to find ways to bypass some website restrictions. Let me explain step by step.

Introduction to Session and Cookie

Cookies exist because HTTP is a stateless protocol. In other words, the server can't remember you. If there were no cookies, you might have to enter your username and password every time you refresh the page. That's not acceptable. The cookie works like a tag that the server gives to you. When you send requests to the server again, the server can use the cookie to recognize you.

To sum up: A cookie can be seen as a "variable," like name=value, stored in the browser. A session can be understood as a data structure, often a "map" (key-value pairs), stored on the server.

Note: I said "a" cookie is like a variable, but the server can set more than one cookie at a time. So sometimes we say cookies are "a group" of key-value pairs. Both ways of saying it are correct.

Cookies can be set on the server side with the HTTP SetCookie field. For example, here is a simple server written in Go:

func cookie(w http.ResponseWriter, r *http.Request) {
    // set two cookies
	http.SetCookie(w, &http.Cookie{
		Name:       "name1",
		Value:      "value1",
	})

	http.SetCookie(w, &http.Cookie{
		Name:  "name2",
		Value: "value2",
	})
    // write string to the webpage
	fmt.Fprintln(w, "html content")
}

When a browser visits a website, you can use the developer tools to check the details of the HTTP communication. You will see that the server responds with two SetCookie commands:

After this, the browser will include these two cookies in the Cookie field for every request:

The purpose of a cookie is very simple. The server uses it to tag each client (browser), so it can recognize them later. Of course, HTTP allows you to set different parameters for cookies, such as expiration time or making a cookie available only for a certain path.

However, many modern websites have complex functions and need to handle a lot of data, like shopping cart features on e-commerce sites. The amount of information is too big and complex for a simple cookie to handle. Also, cookies are stored in the HTTP header, which takes bandwidth and can use up network resources if you store too much data there.

Session can help solve this problem together with cookies. For example, a cookie might only store a variable like sessionID=xxxx. The browser just sends this cookie to the server. The server then uses this session ID to find the corresponding session, which is a data structure. This session stores all the detailed information for the user, like the shopping cart. The server can then use this data to give the user a customized web page. This makes it easy to track users.

A session is a data structure designed by website developers, so it can store all kinds of data. As long as the client sends a unique session ID in a cookie, the server can find the session and recognize the user.

Since sessions are stored on the server, they do use server resources. So sessions usually have an expiration time. Servers will regularly check for and delete expired sessions. If the user visits again after the session expires, they may need to log in again, and the server will create a new session and send the session ID back as a cookie.

Now that we know how cookies and sessions work, is there any practical benefit? Besides interview preparation, let me tell you a sneaky trick: you can use this to get some services for free.

Some websites give you a free trial the first time you use their service. After that, they ask you to log in and pay. You may notice that the website remembers your computer, and you only get another free trial by changing your computer or browser.

So, how does the website remember you even if you haven’t logged in? The answer is simple: the server sets a cookie on your browser and uses session on the backend to record your state. Each time your browser visits the site, it sends the cookie, and the server checks the session to see if you already used the free trial and blocks you from getting another one.

What if you stop your browser from sending cookies—pretending to be a new visitor every time? You can find where your browser stores its cookies (different browsers store cookies in different files), and delete them. For Firefox and Chrome, there are lots of plugins that let you edit or delete cookies directly. For example, I use a plugin called EditThisCookie for Chrome. Here is their website:

These plugins can read, edit, and delete cookies in your browser for the current web page. Of course, it's okay to get a free trial once or twice, but I don't recommend doing this all the time. If you use it too much, the website might remove the free trial, so consider paying for the service if you use it often.

This is a simple introduction to cookies and sessions. Cookies are part of the HTTP protocol and are not very complicated. Sessions are customizable. Next, let’s look at how to implement session management in code.

How session works

The principle of session is not hard, but implementing it well takes some skill. Usually, there are three main parts: Manager, Provider, and Session. Each is a class or interface.

1. The browser uses HTTP to ask the server for the web resource at the /content path. There is a Handler function for this path. The Handler parses the HTTP header, finds the sessionID in the cookie, and sends this ID to the Manager.

2. The Manager is a session manager. It stores some settings, like how long a session lasts and the name of the cookie. All sessions are kept by the Manager in a Provider. So, the Manager will pass the sid (sessionID) to the Provider to find which session matches this ID.

3. The Provider is like a container, often a hash table. It maps every sid to the correct session. When it gets a sid from the Manager, it finds and returns the Session object for that sid.

4. The Session holds the user’s information. The logic in the Handler function reads this information, creates the HTML page for that user, and sends it to the client.

You might ask, “Why make it so complex? Why not just use a hash table in the Handler to store the mapping between sid and Session?”

This is about smart design. Let’s talk about why we separate things into Manager, Provider, and Session.

Let’s start from the bottom, with Session. Since a session is just a key-value pair, why not use a hash table directly? Why make a new structure?

First, because the Session structure might store more than a hash table, like sid, access count, expire time, or last access time. This helps us build things like LRU and LFU algorithms.

Second, sessions can be stored in different ways. If we use the language’s built-in hash table, the data is kept in memory. If there’s a lot of data, the program might crash, and all session data is lost if the program ends. So, sessions may be stored in other ways, like in Redis or MySQL.

That’s why the Session structure gives us a layer of abstraction. It hides different storage ways and only needs to provide a common interface to handle key-value pairs:

type Session interface {
    // Set a key-value pair
    Set(key, val interface{})
    // Get the value for a key
    Get(key interface{}) interface{}
    // Delete a key
    Delete(key interface{})
}

Now let's talk about why we need to abstract Provider. In the diagram above, Provider is just a hash table that maps sid to Session. But in practice, it can be more complex. Sometimes, we need to delete some sessions. Besides setting a session's lifetime, we can use other strategies, for example, the LRU (Least Recently Used) cache eviction algorithm. In this case, Provider needs to use hash linked list or similar data structures to store sessions.

So, as a container, Provider hides the details of algorithms. It uses suitable data structures and algorithms to organize the mapping between sid and Session. You only need to implement these methods to add, delete, update, and find sessions:

type Provider interface {
    // Create and return a new session
    SessionCreate(sid string) (Session, error)
    // Delete a session
    SessionDestroy(sid string)
    // Find a session
    SessionRead(sid string) (Session, error)
    // Update a session
    SessionUpdate(sid string)
    // Recycle expired sessions using LRU or similar algorithm
    SessionGC(maxLifeTime int64)
}

Finally, let's talk about Manager. Most of the real work is done by Session and Provider. Manager is mainly a collection of parameters, like session lifetime, strategies for cleaning up expired sessions, and available session storage methods. Manager hides the operation details, so you can flexibly configure the session system with it.

In summary, splitting the session mechanism into several parts is mainly to decouple the parts and allow custom features.