System Design was HARD until I Learned these 30 Concepts

If you want to level up from a junior developer to a senior engineer or land a high paying job at a big tech company, you need to learn system design. But where do you start? To master system design, you first need to understand the core concepts and fundamental building blocks that come up when designing real world systems or tackling system design interview questions.

In this video, I will break down the 30 most important system design concepts you need to know. Learning these concepts helped me land high paying offers from multiple big tech companies. And in my 8 years as a software engineer, I've seen them used repeatedly when building and scaling large scale systems.

Let's get started. Almost every web application that you use is built on this simple yet powerful concept called client server architecture. Here is how it works.

On one side, you have a client. This could be a web browser, a mobile app or any other frontend application. And on the other side, you have a server, a machine that runs continuously waiting to handle incoming request.

The client sends a request to store, retrieve or modify data. The server receives the request, processes it, performs the necessary operations, and sends back a response. This sounds simple, right?

But there is a big question. How does the client even know where to find a server? A client doesn't magically know where a server is.

It needs an address to locate and communicate with it. On the internet, computers identify each other using IP addresses, which works like phone numbers for servers. Every publicly deployed server has a unique IP address.

Something like this. When a client wants to interact with a service, it must send request to the correct IP address. But there's a problem.

When we visit a website, we don't type its IP address. We just enter the website name. Right?

Instead of relying on hard to remember IP addresses, we use something much more human friendly, domain names. But we need a way to map a domain name to its corresponding IP address. This is where DNS or domain name system comes in.

It maps easy to remember domain names like algo masteraster. io io to their corresponding IP addresses. When you type algo masteraster.

io into your browser, your computer asks a DNS server for the corresponding IP address. Once the DNS server responds with the IP, your browser uses it to establish a connection with the server and make a request. You can find the IP address of any domain name using the ping command.

When you visit a website, your request doesn't always go directly to the server. Sometimes it passes through a proxy or reverse proxy first. A proxy server acts as a middleman between your device and the internet.

When you request a web page, the proxy forwards your request to the target server, retrieves the response, and sends it back to you. A proxy server hides your IP address, keeping your location and identity private. A reverse proxy works the other way around.

It intercepts the client request and forwards them to the back end server based on predefined rules. Whenever a client communicates with a server, there is always some delay. One of the biggest cause of this delay is physical distance.

For example, if our server is in New York, but a user in India sends a request, the data has to travel halfway across the world and then the response has to make the same long trip back. This roundtrip delay is called latency. High latency can make applications feel slow and unresponsive.

One way to reduce latency is by deploying our service across multiple data centers worldwide. This way, users can connect to the nearest server instead of waiting for data to travel across the globe. Once a connection is made, how do clients and servers actually communicate?

Every time you visit a website, your browser and the server communicate using a set of rules called HTTP. That's why most URLs is start with HTTP or it secure version HTTPS. The client sends a request to the server.

This request includes a header containing details like the request type, browser type, and cookies and sometimes a request body which carries additional data like form inputs. The server processes the request and responds with an HTTP response either returning the requested data or an error message if something goes wrong. HTTP has a major security flaw.

It sends data in plain text. Modern websites use HTTPS. HTTPS encrypts all data using SSL or TLS protocol ensuring that even if someone intercepts the request, they can't read or alter it.

But clients and servers don't directly exchange raw HTTP requests and response. HTTP is just a protocol for transferring data, but it doesn't define how requests should be structured, what format responses should be in, or how different clients should interact with the server. This is where APIs or application programming interfaces come in.

Think of an API as a middleman that allows clients to communicate with servers without worrying about low-level details. A client sends a request to an API. The API hosted on a server processes the request, interacts with databases or other services, and prepares a response.

The API sends back the response in a structured format, usually JSON or XML, which the client understands and can display. There are different API styles to serve different needs. Two of the most popular ones are REST and GraphQL.

Just a quick note to keep this video concise, I'm covering these topics at a high level, but if you want to go deeper and learn these topics in more detail, check out my blog at blog. algammaster. io.

Every week I publish in-depth articles on complex system design topics with clear explanations and real world examples. Make sure to subscribe so that you don't miss my new articles. Among the different API styles, REST is the most widely used.

A REST API follows a set of rules that defines how clients and servers communicate over HTTP in a structured way. REST is stateless. Every request is independent.

Everything is created as a resource. For example, users, orders, products. It uses standard HTTP methods like get to retrieve data, post to create new data, put to update existing data, and delete to remove data.

Rest APIs are great because they are simple, scalable, and easy to cast, but they have limitations, especially when dealing with complex data retrieval. REST endpoints often return more data than needed, leading to inefficient network uses. To address these challenges, GraphQL was introduced in 2015 by Facebook.

Unlike REST, GraphQL lets client ask for exactly what they need. Nothing more, nothing less. With a REST API, if you need a user's profile along with their recent post, you might have to make multiple requests to different endpoints.

With GraphQL, you can combine those requests into one and fetch exactly the data you need in a single query. The server responds with only the requested fields. However, GraphQL also comes with trade-offs.

It requires more processing on the server side, and it isn't as easy to cast as REST. Now when a client makes a request they usually want to store or retrieve data. But this brings up another question where is the actual data stored.

If our application deals with small amounts of data we could store it as a variable or as a file and load it in memory. But modern applications handle massive volumes of data far more than what memory can efficiently handle. That's why we need a dedicated server for storing and managing data.

A database. A database is the backbone of any modern application. It ensures that data is stored, retrieved and managed efficiently while keeping it secure, consistent and durable.

When a client request to store or retrieve data, the server communicates with the database, fetches the required information and returns it to the client. But not all databases are the same. In system design, we typically choose between SQL and NoSQL databases.

SQL databases store data in tables with a strict predefined schema and they follow ACIT properties. Because of these guarantees, SQL databases are ideal for applications that require strong consistency and structured relationships such as banking systems. NoSQL databases on the other hand are designed for high scalability and performance.

They don't require a fixed schema and use different data models including key value stores, document stores, graph databases, and wide column stores which are optimized for large scale distributed data. So, which one should you use? If you need structured relational data with a strong consistency, SQL is the better choice.

If you need high scalability, flexible schema, NoSQL is the better choice. Many modern applications use both SQL and NoSQL together. As our user base grows, so does the number of requests hitting our application servers.

One of the quickest solutions is to upgrade the existing server by adding more CPU, RAM or storage. This approach is called vertical scaling or scaling up which makes a single machine more powerful. But there are some major limitations with this approach.

You can't keep upgrading a server forever. Every machine has a maximum capacity. More powerful servers become exponentially more expensive.

If this one server crashes, the entire system goes down. So, while vertical scaling is a quick fix, it's not a long-term solution for handling high traffic and ensuring system reliability. Let's look at a better approach, one that makes our system more scalable and fall tolerant.

Instead of upgrading a single server, what if we add more servers to share the load? This approach is called horizontal scaling or scaling out where we distribute the workload across multiple machines. More servers is equal to more capacity which means the system can handle increasing traffic more effectively.

If one server goes down, others can take over which improves reliability. But horizontal scaling introduces a new challenge. How do clients know which server to connect to?

This is where a load balancer comes in. A load balancer sits between clients and backend servers acting as a traffic manager that distributes request across multiple servers. If one server crashes, the load balancer automatically redirects traffic to another healthy server.

But how does a load balancer decide which server should handle the next request? It is a load balancing algorithm such as roundroin, least connections and IP hashing. So far we have talked about scaling our application servers.

But as traffic grows, the volume of data also increases. At first we can scale a database vertically by adding more CPU, RAM and storage similar to application servers. But there is a limit of how much a single machine can handle.

So let's explore other database scaling techniques that can help manage large volumes of data efficiently. One of the quickest and most effective ways to speed up database read queries is indexing. Think of it like the index page at the back of a book.

Instead of flipping through every page, you jump directly to the relevant section. A database index works the same way. It's a super efficient lookup table that helps the database quickly locate the required data without scanning the entire table.

An index is stores column values along with pointers to actual data rows in the table. Indexes are typically created on columns that are frequently queried such as primary keys, foreign keys, and columns frequently used in rare conditions. While indexes speed up reads, they slow down rights since the index needs to be updated whenever data changes.

That's why we should only index the most frequently accessed columns. Indexing can significantly improve read performance. But what if even indexing isn't enough and our single database server can't handle the growing number of read request?

That's where our next database scaling technique, replication, comes in. Just like we added more application servers to handle increasing traffic, we can scale our database by creating copies of it across multiple servers. Here is how it works.

We have one primary database also called the primary replica that handles all write operations. We have multiple read replicas that handle read queries. Whenever data is written to primary database, it gets copied to read replicas so that they stay in sync.

Replication improves the read performance since read requests are spread across multiple replicas reducing the load on each one. This also improves availability since if the primary replica fails, a read replica can take over as the new primary. Replication is great for scaling read heavy applications.

But what do we need to scale right operations or store huge amounts of data? Let's say our service became popular. It now has millions of users and our database has grown to terabytes of data.

A single database server will eventually struggle to handle all this data efficiently. Instead of keeping everything in one place, we split the database into smaller, more manageable pieces and distribute them across multiple servers. This technique is called sarding.

Here is how it works. We divide the database into smaller parts called Sards. Each sard contains a subset of the total data.

Data is distributed based on the sarding key. For example, user ID. By distributing data this way, we reduce database load since each SA handles only a portion of queries and speed of read and write performance since queries are distributed across multiple SS instead of hitting a single database.

Singing is also referred to as horizontal partitioning since it splits data by rows. But what if the issue isn't the number of rows but rather the number of columns? In such cases, we use vertical partitioning where we split the database by columns.

Imagine we have a user table that stores profile details, login history, and billing information. As this table grows, queries become slower because the table must scan many columns even when a request only needs a few specific fields. To optimize this, we use vertical partitioning where we split user table into smaller, more focused tables based on users patterns.

This improves query performance since each request only scans relevant columns instead of the entire table. It also reduces unnecessary disk IO making data retrieval quicker. However, no matter how much we optimize the database, retrieving data from disk is always slower than retrieving it from memory.

What if we could store frequently access data in memory? This is called caching. Caching is used to optimize the performance of a system by storing frequently access data in memory instead of repeatedly fetching it from database.

One of the most common caching strategies is the cash aside pattern. Here is how it works. When a user requests the data, the application first checks the C.

If the data is in the cache, it's returned instantly avoiding a database call. If the data is not in the C, the application retrieves it from the database. It stores it in the C for future request and returns it to the user.

Next time the same data is requested, it's served directly from C, making the request much faster. To prevent outdated data from being served, we use time to live value or TTL. Let's look at next database scaling technique.

Most relational database use normalization to store data efficiently by breaking it into separate tables. While this reduces redundancy, it also introduces joins. When retrieving data from multiple tables, the data must combine them using join operations, which can slow down queries as the data set grows.

Denormalization reduces the number of joints by combining related data into a single table, even if it means some data gets duplicated. For example, instead of keeping users and orders in a separate table, we create user orders table that stores user details along with the latest orders. Now, when retrieving a user's order history, we don't need a join operation.

The data is already stored together leading to faster queries and better read performance. Denormalization is often used in read heavy applications where speed is more critical. But the downside is it leads to increased storage and more complex update request.

As we scale our system across multiple servers, databases and data centers, we enter the world of distributed systems. One of the fundamental principles of distributed systems is the cap theorem which states that no distributed system can achieve all three of the following at the same time. Consistency, availability, and partition tolerance.

Since network failures are inevitable, we must choose between consistency plus partition tolerance or availability plus partition tolerance. If you want to learn about cap theorem in more detail, you can check out this article on my blog called Cap theorem explained. Most modern applications don't just store text record.

They also need to handle images, videos, PDFs, and other large files. Traditional databases are not designed to store large unstructured files efficiently. So, what's the solution?

We use blob storage like Amazon S3. Blobs are like individual files like images, videos, or documents. These blobs are stored inside logical containers or buckets in the cloud.

Each file gets a unique URL making it easy to retrieve and serve over the web. There are several advantages with using blob storage like scalability, pay as you go model, automatic replication, easy access. A common use case is to stream audio or video files to user applications in real time.

But streaming the video file directly from blob storage can be slow, especially if the data is stored in a distant location. For example, imagine you are in India trying to watch a YouTube video that's hosted on a server in California. Since the video data has to travel across the world, this could lead to buffering and slow load times.

A content delivery network or CDN solves this problem by delivering content faster to users based on their location. A CDN is a global network of distributed servers that work together to deliver web content like HTML pages, JavaScript files, images, and videos to users based on their geographic location. Since content is served from the closest CDN server, users experience faster load times with minimal buffering.

Let's move to the next system design concept which can help us build realtime applications. Most web applications use HTTP which follows a request response model. The client sends a request.

The server processes the request and sends a response. If the client needs new data, it must send another request. This works fine for static web pages but it's too slow and inefficient for real-time applications like live chat applications, stock market dashboards or online multiplayer games.

With HTTP, the only way to get real-time update is through frequent polling, sending repeated request every few seconds. But polling is inefficient because it increases the server load and waste bandwidth. As most responses are empty when there is no new data, webockets solve this problem by allowing continuous two-way communication between the client and the server over a single persistent connection.

The client initiates a websocket connection with the server. Once established, the connection remains open. The server can push updates to the client at any time without waiting for a request.

The client can also send messages instantly to the server. This enables real-time interactions and eliminates the need for polling. Webockets enables real-time communication between a client and a server.

But what if a server needs to notify another server when an event occurs? For example, when a user makes a payment, the payment gateway needs to notify your application instantly. Instead of constantly pulling an API to check if an event has occurred, web hooks allow a server to send an HTTP request to another server as soon as the event occurs.

Here is how it works. The receiver, for example, your app registers a web hook URL with the provider. When an event occurs, the provider sends a HTTP post request to the web hook URL with event details.

This saves server resources and reduces unnecessary API calls. Traditionally applications were built using a monolithic architecture where all features are inside one large codebase. This setup works fine for small applications but for large scale systems monoliths become hard to manage, scale and deploy.

The solution is to break down your application into smaller independent services called microservices that work together. Each microser handles a single responsibility has its own database and logic so it can scale independently. communicates with other microservices using APIs or message cues.

This way, services can be scared and deployed individually without affecting the entire system. However, when multiple microservices need to communicate, direct API calls aren't always efficient. This is where message cues come in.

Synchronous communication, for example, waiting for immediate responses doesn't scale well. A message Q enables services to communicate asynchronously, allowing requests to be processed without blocking other operations. Here is how it works.

There is a producer which places a message in the queue. The queue temporarily host the message. The consumer retrieves the message and processes it.

Using message cues, we can decouple services and improve the scalability and we can prevent overload on internal services within our system. But how do we prevent overload for the public APIs and services that we deploy? For that we use rate limiting.

Imagine a bot starts making thousands of requests per second to your website. Without restrictions, this could crash your servers by consuming all available resources and degrade performance for legitimate users. Rate limiting restricts the number of requests a client can send within a specific time frame.

Every user or IP address is assigned a request kota, for example, 100 requests per minute. If they exceed this limit, the server blocks additional requests temporarily and returns an error. There are various rate limiting algorithms.

Some of the popular ones are fix window, sliding window, and token bucket. We don't need to implement our own rate limiting system. This can be handled by something called an API gateway.

An API gateway is a centralized service that handles authentication, rate limiting, logging, monitoring, request routing, and much more. Imagine a microservices-based application with multiple services. Instead of exposing each service directly, an API gateway acts as a single entry point for all client request.

It routes the request to the appropriate microser and the response is sent back through the gateway to the client. API gateway simplifies API management and improves the scalability and security. In distributed systems, network failures and service retries are common.

If a user accidentally refreshes a payment page, the system might receive two payment request instead of one. Adam potency ensures that repeated request produced the same result as if the request was made only once. Here is how it works.

Each request is assigned a unique ID. Before processing, the system checks if the request has already been handled. If yes, it ignores the duplicate request.

If no, it processes the request normally. If you enjoyed this video, I think you will love my weekly newsletter where I dive deeper into system design concepts with real world examples. I also share articles on system design interview questions and tips to help you prepare for interviews.

You can subscribe it at blog. algamaster. io.