Complete Backend Software Engineer Mind Map - Everything You Need to Know (2 HOURS!)

everything you need to be aware of as a backend engineer is going to be talked about in this video I put together this insane mind map which is going to talk about every major technology and show you where it fits in in the bigger context now if you want to become a backend engineer or strengthen your backend engineering skills you don't have to know every single technology in this mind map this is about helping you think and speak intelligently about different Technologies and helping you understand how everything is connected if you want to take this

information and turn it into a step-by-step road map I'm going to do that in a followup video so stay tuned for that and we're going to go through this if you want to follow along I have a link to this down below this will help you get the most up to-date version if I do end up adding or changing anything in this list you can also print it if you want to use it as some kind of reference so to start we're going to go into the backend Frameworks and there are a bunch of different

languages here the main takeaway from this is that you can program a backend in pretty much any major programming language certain ones are going to be more popular for web development and I would generally recommend to pick something that has a decent level of popularity because this will basically show that enough people have gone through it and issues have been resolved and you're not working with some super cuttingedge technology that doesn't have a lot of development over time or resources for support and that's going to be something that you can apply for pretty much any

technology I generally like to recommend people get newer technology that's not super dated but not so new that you're working in basically a beta software where you have to worry about breaking changes or running into issues that nobody else has faced before so we'll go through each one of these but it's important to understand that you first have the programming language so for example we have Python and within python you will also pick a framework so a framework is basically just a more extensive library that someone or some team creat created to help you build

applications that way you don't have to build everything completely from scratch so in the example of python three popular ones are D Jango flask and fast API there's probably like a million others the goal isn't to know all of the Frameworks the main thing is to know the most popular Frameworks for each major language and when it comes to choosing a framework it's often the case you'll want to pick one that makes sense based off of what your team is using if you're working in an existing company or software so if you're given an application

written in Jango then yeah use Jango but if you're starting with a brand new project you can think of the targeting of that particular framework often it's the case a framework will Target simple small applications or building large Enterprise applications and it makes sense to pick the one based off of what you're trying to build so in this case flask will give you a really Simple Start you can get an application going in a single python file whereas Django is going to give you a little bit more Str structure and the setup is more complex

but it might be the framework for you if you eventually are going to have a more complex backend then it's probably the case there will exist Frameworks that will Target a particular feature or capability so with fast API it's all about creating very performant apis so that might be something for you as well and if you're watching this thinking wow there's so much stuff here where do I even start it's often the case that you can achieve the results you want with many different languages or Frameworks you don't have to choose exactly the right one

so I recommend following some learning path and sticking to that so you can become familiar with all of these different technology categories so at the bottom of the road map I have an example learning road map and you can follow through with that or you can wait for my upcoming video which will go through that material now if you need a little bit more support I do offer one-on-one mentorship and this is if you want me to meet you where you are in terms of skill and help you bring you to the point of being

a professional backend engineer so I have a link for that application Down Below in addition to the road map so check out those two resources and if you do end up applying for the mentorship I look forward to working with you but let's get back to working on the backend Frameworks let's just go through each one of these in no particular order but I'll grab some of the more popular ones first so we have JavaScript there are a lot of different options here one of the biggest ones is expressjs this will give you a really

easy way of building a backend API next up we have nestjs which is going to achieve similar results but it gives you that larger framework so you could kind of compare this to Python's Django and flask expressjs gives you the Simplicity ease of starting like you would get with flask and then nestjs is going to give you everything you would possibly need kind of like Jango and when you have larger systems like that it often comes with different conventions or opinionated approaches to developing software for some people that's great it gives you exactly how to

do things but for others you might want a little bit more freedom and not to have to kind of fit into a strict system so in that case you might want to go with something like Express instead of nestjs then we have fastify which boasts faster speeds and a better development experience that's ultimately going to be up to your judgment but that's another option as well then on here we have two other ones njs and nextjs these two are kind of interesting because the these are often used to create full stack applications and they just

have a backend component to it so these are kind of different Technologies than Express nextjs and fastify but I wanted to put them in here because they're going to give you a JavaScript backend so let's take a look at nextjs first nextjs is going to be pretty similar nextjs is going to utilize react components and it's going to give you the framework for building a full stack application so both front end and backend with this you're going to get things like static site generation overall it's a really cool technology and has grown a ton in

popularity so if you're building a new application it's definitely worth considering especially if you want to work within JavaScript and you're already using react then it would just make sense to check out nextjs and see if it's right for you nextjs is very similar but it's utilizing the view JavaScript framework so Vue is similar to react a frontend framework but it's a little bit different so if you're working within a view environment you might want to check out njs next up we have C the major framework with this asp.net core you may also hear asp.net

MVC this is a great option but it's within C which is largely tied to the Microsoft environment so in this context there's a good chance you'll be working with Microsoft SQL Server Microsoft Azure if you're in a Microsoft shop then yeah it makes sense to use C otherwise there's a lot of other options on here that aren't coupled to a particular system now that doesn't mean if you use C you have to use Windows or you have to use SQL server or any of these other Microsoft products but it's developed by Microsoft so that's where

you might get the best experience ICP for various things and overall it's really great you'll probably get a very similar experience with Java and spring boot Java being the main programming language in the Oracle ecosystem now if you're going for spring boot within Java there's also cotlin which is often positioned as a potential replacement for Java and Colin also has a spring boot framework as well as this kto R framework I'm actually not sure how to pronounce that so I don't want to assume I haven't personally worked with that but it's probably a good option

if you're working within cotlin PHP is very huge for the web it Powers things like WordPress you can use it for custom backends with code igniter or laravel haven't personally used PHP in this context but I know it's very popular it wouldn't be my personal first choice but there's not anything necessarily wrong with it I would just probably choose JavaScript or python or I've actually worked with rust professionally and I use the axom framework so the interesting about rust while it is newer and might not have the established history that you might get with some

of these other options it is a really good ecosystem and gives you the power of a programming language like C++ but has a better ecosystem for things like API development so you wouldn't typically build an API with something like C++ but with rust you can easily do that actic is another popular one but within every language there's like a 100 Frameworks so I'm just putting a few of the most important ones so I had a pretty good experience developing in Rust definitely did have that feeling of like wow this is newer not nearly enough support

or people who've solved this particular issue yet but if you're already working within a rust environment maybe you're using rust for some lower level programming and you want to build an API then this is awesome so I definitely see rust being a much more popular option going forward as well as languages like go which you can see as a lower level language but often more used in the context of web apps and microservices so go is a great option if you're already in that kind of environment here are two Frameworks with go next up let's

talk about Elixir we have Phoenix and then Ruby we have Ruby on Rails all of these would be great options just depends on which language you end up going with elixir is a functional programming language the people that use Elixir seem to love it it's not super popular and also probably wouldn't be my first choice as a language to use for building a backend I did use it for one particular job and it was kind of a rough experience learning it as it's not super similar to some of the other languages and Frameworks I've used

while a lot of the principles of how to design the app were the same I just felt the learning curve to go to a functional language like elixir was a little bit High but if you're starting from scratch and for some reason you want to use Elixir go for it just keep in mind if it's something where other developers might end up having to work on it not nearly as many people work with languages like elixir rust cotlin and Swift which I really wouldn't recommend for a backend language I think it's too tightly coupled to

the Apple ecosystem you may use Swift for building the actual iPhone application but when it comes to the back end I probably wouldn't choose Swift unless I knew it was only going to be used in that specific iPhone app environment and I didn't need to use it for anything else but if I was just building a backend for a general web app I would not choose Swift personally similar idea with cotlin which I think is probably a little bit more open and a good choice than using Swift but cin is often used for Android app

development so you might end up using cotlin for the Android app but then choose a different language for the backend if you're already developing in cotlin then you could potentially create some stuff in cotland for the backend but if you were just trying to create a general backend and you needed to choose one of these programming languages I probably would choose something that more people have used such as JavaScript or python I generally do concern myself about the popularity of the language and the framework it's not the only thing I use to make a decision

on which one of these to use but it definitely is a factor a great way to get an idea on popularity is using surveys such as the stack Overflow survey so here are some popular Technologies JavaScript being the top of the list here then we have python typescript which is basically JavaScript Java C PHP so if you're working with some language or some framework that's down here yeah there might just not be very much support which is one of the big Reasons I'm a little bit hesitant about Elixir the learning curve is steep and not

a lot of people use Elixir and I only listed some of the major programming languages there's probably web development Frameworks for a lot of these other languages that we didn't even talk about about but this will give you a pretty good starting point and if you are familiar with all of these you should be good to go because you'll understand how other languages and Frameworks fit into this larger mind map now I have a small comment up here that says worth mentioning web assembly I couldn't figure out where to put this so I was like

here looks great web assembly is a technology basically allowing you to use any of the major backend programming languages to do certain things on the front end so if you're familiar with development you probably know that JavaScript is the main programming language for front-end development so if it comes to just learning as little languages as possible you can pick JavaScript because you can do the front end and the backend and you don't have to learn a different programming language for the back end however this could be changing with the adoption of web assembly as it

is allowing us to use other programming languages for things on the front end I think this should have a capital W here but you know everything has different upper cases and lower cases so but web assembly is a target so if you imagine taking a compiled language such as rust and instead of compiling that to run on an x86 machine you could compile it to run on web assembly which is supported in a browser so the browser is acting as an interpreter for web assembly allowing us to use these other languages for frontend work so

it's likely the case you're still going to have JavaScript it's more so that for certain things you could use these backend languages for frontend and things especially if it's very performance heavy if you're trying to make a video editor or a Maps application or something that requires a lot of computation and you want to run that within a browser web assembly might come up to make that more performant it's not a different language it's just a technology that'll utilize these languages you're already familiar with it's just going to help you have more functionality from these

languages this does kind of get into the front end but this is likely something you'll want to be aware of as a backend engineer next up on this list I have ORS and database libraries now I didn't go through every omm and library that exists there's just unlimited for every major programming language there's going to be libraries for working with databases now these could be libraries to work with one particular database or it could be an OM which will work with multiple databases so let's take a look at this list to give you an idea

of some of the most popular ones here now I listed out omm and I'll explain what that means here in a second but you may also find libraries to work with a particular database so if you were working with postgress in a certain language there might be a library to do that for you to make that connection easier to retrieve data edit data and all of that so for example within python you might work with pyop G if you wanted to work with postgress the first four on this list are examples of OMS which we'll

talk about here in a moment then we have a database Library sqlx let's talk about this one first so this is an example that I used with rust and this is a database library that will allow you to execute SQL or SQL to different databases so it's not a library to work with one particular database it will allow you to work with multiple databases but it's not exactly an omm so the difference between this kind of library that can work with multiple databases but is not an omm and then these kinds of libraries which are

OMS is that these are object relational mappers they will execute the SQL for you bring them into objects in your code and then the way you interact with those objects would be methods and properties on those objects so I didn't want to get into all of the details and split out the types of database libraries into different branches as I didn't really feel it to be necessary the main thing you need to understand is that you can basically do one of three different things you can use a library to work with a specific database directly

so an example would be pyop G2 with python to work with the postgress database you may also hear of this as an adapter basically a tool to connect your code to the database next up you might have a library that somebody would write to work with multiple databases and give you some easy commands but it's not an omm the main difference here is it doesn't abstract away the SQL itself you are the one who will still write out the SQL language and we're going to get into databases here in a moment but you're going to

basically write that SQL and then the library will take care of actually executing that on the database and retrieving that data bringing it into memory for your application you might then have to do some manual mapping taking the results of those queries and putting those into objects in your code finally we have OMS which is the highest level of abstraction this is basically when you will write something like users. getet all you're not using the SQL language but you're actually using whatever programming language you're working with so if you're working within the context of a

python application you might make your query in Python using python syntax and that omm will translate that to SQL for you retrieving the data from the database and then bringing that into your code as objects the ORS have the highest level of abstraction and it is often the way to easily get started working with databases but they do have some downsides they often do not have as much control because the library authors give you a specific set of functions you can use you're limited to those functions they may give you an option to execute raw

SQL but that just depends on the library this is usually fine for most applications so when people get started working with an omm is often an easy way to build out applications without having to get super deep into all the database stuff as you get more specialized queries you might not find the specific function you need or behind the scenes that function might not be executing the most optimized query where the OM could be doing something fancy in code that you wouldn't expect if you just executed that SQL directly on a database this section of

the Mind map is really just designed to give you an introduction to database libraries because I wanted to mention the potential concerns around om so you can do some additional research on OMS when you start to build out a project you can decide if it's something you want to use it is a great way to get started but I highly encourage that everybody picks up SQL and has an understanding of the database at the database level and not just from within your application so one of the sections we're going to get to will be databases

and that's an extensive list of of the most popular databases so we'll get to that in a moment but for now just be aware that within your application you're going to need a way to connect to your database that will either be just a small direct library or adapter to work with a particular database or it will be a library for you to execute SQL or it will be an even larger library or framework which will be an OM so the first one on here we have is Entity framework and that is within the C

environment so if you're working with an asp.net core application this can help help you generate your tables and work with data that's another cool thing about ORS they abstract away a lot of stuff making it really easy to create tables and not have to worry about all the uh ddl which you would have to worry about if you were creating the tables yourself next up we have Prisma this is really popular in the JavaScript World however I have noticed that Prisma is growing beyond that to where you can use the Prisma syntax with other om

or libraries so definitely worth looking into I used Prisma within a node or JavaScript environment and I was able to use it to work with a postgress database but it also supported mongodb which is pretty uncommon for omms so the fact that it had some nosql support and I needed to do something with mongod DB this was a really good choice for me next up we have SQL Alchemy and D Jango omm these are both within the python world I just wanted to list two from the same language to show you that it's not always

the case that there's only one om as an option just like back in Frameworks there's often multiple ORS you can choose from so you just got to weigh the options looking at the pros and cons of each maybe try one out see if you like it but you probably won't go wrong with any of the major OMS all right now I want to get into an interesting section which is CMS or content management system and this is going a little bit away from core backend development but is going to introduce a a lot of interesting

things that you should be aware of as a backend engineer CMS is short for Content management system if you're not sure what that is it's a system that helps you manage your content so you can think of basic examples such as WordPress and one of the major benefits of a CMS is that people who are non-technical can use the CMS to create content and manage the website so this is going away from Custom web apps and getting a little bit more into content creation but this is still an extremely popular thing you might need to

do as a backend developer now it's not often the case you're going to be working with these content Management Systems as a backend developer you might for example you might be working in PHP for these three or C with umco but another popular route is the Headless CMS so a headless CMS will give you just the backend so the system for people to go in and create content and organize and manage that content but it doesn't tightly couple that with a front end so within the backend career you might want to be familiar with these

different cms's because say one of these cms's delivers the content as an API you might need to configure that or set it up in a certain way or make sure the connection between that API and the front end is working these are often coupled with a front end and I put here SSG which stands for static site generation so it's often the case that these will be paired with one of these tools which will be just the front end that will then make requests to your back end to retrieve the content to be displayed on

the page if this all sounds like crazy jargon don't worry you don't have to know all of the information when it comes to cms's I just wanted to leave it in here as a section because you should at least be familiar with the idea so you can imagine say you're familiar with WordPress what if we had WordPress but instead of automatically showing that in a WordPress front end we just had a headless WordPress that delivered the content through an API then you could use one of these front-end tools to consume that API and display that

information on a page so we're not going to get into the details of all of these different cms's and static site generators you can do additional research if you want I'm actually not even sure the difference between all of these but I just know that these are some of the most popular ones so we're making pretty good progress in this but we still have a lot of sections to go through so next up we're going to take a look at we'll go databases since we just got done talking about om's this will help us kind

of tie that information together now the key thing with databases is to not be overwhelmed with the number of databases out there I've only listed some of the most popular databases and I have a pretty big list here there are a lot more and it seems like every database has some specialty or some fancy marketing words and it's sometimes challenging to understand what it is even for so my goal is to give you an idea of what each of the major databases is for and how it's different than the other databases that being said the

main way to categorize these is easy either SQL or nosql SQL stands for structured query language and is a programming language but it's more of a standard so all of the SQL databases will use the SQL language but might have variations and things that are different between databases overall though if you learn one you're going to be able to effectively work with all of them things might just be a little bit different in how that database works or some of the data types or things like that the nosql and this is confusing it doesn't mean

that there's no structure it means not only structured query language so some of the databases in the nosql category might still have the option to use structured data but they might have other things as well so we'll get to that in a moment let's first take a look at the structured databases we have all the major ones here that you should be familiar with and just to get started from the beginning we have two down here red shift and snowflake which are a little bit different than the rest up here they're all structured and that

they store things in tables and that's the main thing that makes something a structured database but these two are largely prioritizing analytics so if you have a lot of data and you want to run analytics on that data you you will use an analytical database that's if the core features of these structured databases is not adequate which usually it is and you have some serious analytical needs then you can look into an analytical database or you might hear it as a data warehouse so I didn't do it in this example but if you wanted to

you could split this whole section up into two categories oap which is online analytical processing and then everything else being oltp online transactional processing a transaction being a purchase an interaction that you need to record creating a post basically the creates the updates and the deletes and obviously these databases are all going to be capable of reading data but if your primary thing is just to read data and then calculate numbers off of that data that's where you would go for the analytical types of databases so transactional analytical I'm hesitant to do too much splitting

though because then it just becomes hard to organize the main thing that these things have in common is that they're table based so everything goes into tables with columns there is a whole world that you could explore if you start looking down the path of red shift and snowflake which are two popular data warehouses red shift is part of AWS we're going to talk a lot about AWS here soon and then snowflake is another option out there there's probably a lot more of these analytical databases or data warehouses but the thing that I was trying

to get to is that if you start exploring data analytics and data warehouses there is a whole realm of information and this is going to get you into the world of big data which is going to talk a lot about things like data streaming and Hadoop or Kafka this is something I don't talk a whole lot about in this mind map I do this on purpose because while you might explore some of that stuff as a backend developer there is pretty much an entire different career path or you could create an entire different mind map

for the world of analytics so that's going to bring in things like data engineering ETL and even some analytics like Predictive Analytics so getting into machine learning and AI it's a very interesting world but I just felt it to be kind of out of scope of what I'm trying to accomplish in this mind map which is really about focusing on backend Services databases apis and how all of that stuff works together you most certainly might encounter some of those things so you should be aware that they exist but you might not so you don't have

to go and learn about data warehouses I personally have never needed a data warehouse for anything as all of the standard SQL databases for transactional processing can do a lot of the same stuff it just might not be able to do it at a very large scale but for the majority of use cases probably most of the use cases I'm going to encounter and you're going to encounter will be fine just to use a transactional database so that's specifically if you really need to get into advanced analytics let's now go through the main transactional databases

I will just go from top to bottom here SQL Server it's the Microsoft database similar to C and Microsoft ashure it's just an offering from Microsoft competing with similar things from other providers so SE server is really good it's been around for a while I've used it for Stuff nothing to complain about it's just very tightly coupled to Microsoft same idea with others on this list so Oracle database it's just the Oracle equivalent some database administrators probably freaking out that I just said that Oracle and SQL Server are basically the same thing they're all databases

storing structured data as a developer you're probably not going to notice a major difference switching between these two as get into the details yeah there's probably some really critical differences between them but day-to-day it's not going to impact us as developers unless you really get into more of like database Administration but most of that is out of scope as well plus the SQL between these is going to be pretty similar the differences with syntax and language will really come up if you start working with the procedural languages with databases so a lot of these databases

are going to have their own procedural language you can use to do more custom stuff so if you need to do custom logic you would use plsql for an Oracle database SQL PL is the SQL Server equivalent and there's probably languages for all of these it's not something you're going to probably do a whole lot but it might come up and those are just languages specific to that particular database these languages may come up if you're working with stored procedures and you're not just doing standard SQL so there is a chance it will come up

I generally try to bring that logic into the application layer instead of leaving it in the database but if you're in a company or an environment that's pretty database heavy there's a chance that they'll leave some of that logic at the database layer and you'll just have to familiarize yourself with that procedural language for that database that would be for things like database triggers or procedures where you need to do additional validation or whatever it may be but you're not going to be using the procedural language for day-to-day SQL like creating rows or reading data

whatever you need to do for creating crud applications for creating reading updating and deleting next up we have db2 this is IBM's Flagship database and those are the three major Enterprise databases SQL Server is probably the most common of these three that is used by developers but there is often times you'll be working in Oracle or db2 if you are in an Enterprise setting but it's often the case that Oracle db2 these are pitched to large Enterprises with some pretty intense data needs SQL Server kind of fills the middle ground between small scale developers and

Enterprise and then we have the open- source databases which usually will be the pick for firsttime developers creating their own app so hopefully that gives you a general idea of where these databases fit it's kind of a generaliz so it's probably not always the case but if you just want to know what people think about certain databases these guys are Enterprise SQL Server is kind of Middle Ground developer friendly but can still be considered Enterprise coming from Microsoft and the MySQL and postgress notel Enterprise these are open source the main thing you should know is

that MySQL is owned by Oracle the original MySQL team forked MySQL creating Mariah DB so this is kind of like a my SQL sticking to its roots if that makes any sense basically some concerns about MySQL now being owned by Oracle is that it's going to pull you into that Oracle ecosystem trapping you for eternity and some people don't really like that which looking at this from Oracle perspective it's kind of funny if somebody can make a successful product just advertising that they're not in your ecosystem so that's something to consider when you're looking into

these databases do you want to be tightly coupled to a particular ecosystem doesn't mean you can't ever move but things are just going to work better for example if you're SQL Server if you're in Azure working with C they kind of make an entire ecosystem kind of like apple and all of their products all right I'm kind of ranting you can tell I like databases if you're not sure which database to go with postgress is a great option because it's open source it's free and you can use it for free in a commercial setting but

my thought is if you're just starting out and you're trying to pick a database go with postgress and if you need one of these other databases for some reason you can switch over fairly easily because it's all based off of the SQL standard all right last structured database on this list is SQL light you may also hear people say SQ light I've always said SQL light you may also hear some variations in pronunciation with postgress as the full word has a ql at the end so you might hear postgressql or variations of that I just

call it postgress so sqlite is a bit special because it's an embedded database and what that means is it's delivered with the software now for a backend developer you probably won't use sqlite for the majority of the applications you build and that's because you're building a system you can imagine that system over here and lots of clients connect to the server so you have a single location and that's going to host the database sqlite would be a popular choice if you were delivering software to a bunch of clients and they actually ran it locally so

instead of having a centralized server you actually distributed software and that software needed a database to store some information locally that's where sqlite would shine you can use sqlite for centralized storage but it's not the first choice that being said I have learned sqlite I think it's a great option to learn and practice with it's super easy to use and set up because it doesn't have a dedicated database management system it's really just a file on your computer it's super easy to start working with databases when you're working with SQL light it's even included with

some programming languages so for example if you're working with python and you want to store some data in a database you can just use sqlite immediately without having to go install stuff and get things configured and set up so that's one of the main reasons I like sqlite and it's also good to understand what people mean when they say an embedded database it doesn't always mean embedded systems like people are talking about when they say they're an embedded engineer working on iot devices or something like that it can just be desktop software I like to

think of it more as the database is embedded with the software and it's not at a remote destination acting as a server client setup so I'm pretty sure that covers the core SQL databases I'm sure there are more out there but again you know exactly how they're going to fit into this larger structure next up let's talk about nosql databases and this one can be a little bit overwhelming because these don't all have core similarities like the structured databases a lot of these are just competing products from different companies in nosql databases some of these

are specialized databases for certain use cases that are different than some of the other nosql databases and sometimes there are competitors so a little bit more complicated but we're going to work our way through this list getting a basic idea of each of these now just so it's clear and I think I've made this point already you do not have to get experience with each of these I have not used most of the databases in this list it's really about becoming good at a few of these and using certain ones for certain use cases so

when you have some application and you know you might need to do X Y or Z you can research which database might be the best for that so let's first take a look at mongodb here now this is known as a document database and what a document is is basically key value pairs pretty much like an object in JavaScript so this is going to be pretty similar to Json if you are familiar with that which we'll get to notations here soon it's a little bit different because it contains data types Beyond just Json so you

can look up the bson spec if you wish but you can tell this is describing what looks like a person it has their age and their status and groups the main difference between a nosql database and an SQL database is that there can be differences between individual documents so the next person you put in this collection could have a name an age and then maybe favorite books or something totally different than this document comparing that to a structured database you're going to define the columns which will be general not specific values this is just what

you will store in the table and then every Row in this table can only put those particular columns so that's the major difference if you have variation in structure between documents then it makes sense to consider a nosql database mongodb is one of the more common ones that Developers start learning now I'll skip over index DB for just a sec and talk about document DB which is pretty much the same idea provided by Amazon so AWS document DB it's going to allow you to store documents and what that means practically is that document DB is

a managed database AWS hosts it and you can access that database and pay for it as you use it compare that to mongodb while you can get Cloud offerings you're not require ired to so you can download mongodb and you can run it locally however there are major mongodb providers such as mongodb Atlas which will host mongodb for you so that's always an option as well now let's take a look at index DB not really sure why I put those between the two but no big deal index DB is a database in the browser and

this is really kind of cool because it allows you to store information that the user that is using your page might need to store to temporarily giving them a consistent experience between visits but it's not something that's persisted to the backend this gives you a database structure in the browser and that might be more useful than just using something like local storage now I generally wouldn't recommend this for any critical information because it's local to the client meaning they could disconnect from the server they're not going to be able to give that information to you

and because it's client side they can access that information as well so it's not something you would use for storing critical information or if you needed to make sure that information was persisted indefinitely you would not want to use this because the client could never come back or they could clear all of their browser stuff and then they're not going to have that data any longer couch DB offers some interesting similarities between index DB now I wouldn't say these are a direct comparison but one of the selling points of couch DB that makes people use

it is it will allow people to store data locally and then if they later regain a connection to a server they can then sync their local data with the server so if you need that offline experience but you still want to store data in a centralized location a server then you might want to check out couchdb next up we have neo4j and the best way to describe this besides saying that it's a graph database is to show you some of the examples from their website so here are some of the use cases such as knowledge

graphs accessing deep Dynamic context by connecting data so this really shows how data is related and can lead to some really cool visualizations if that's something you're interested in you can see a basic example here showing how things are connected here's a visual I was able to find without actually going in there and creating a database you can see that it shows the nodes and how they're connected and it has like this cool liquid movement thing but yeah that's an interesting project you might want to check out next up we have Dynamo DB this is

a serverless database provided by Amazon the whole idea of something being serverless is not that it doesn't use a server behind the scenes more so that you as the developer are not responsible for the server it's kind of like a managed database but it's completely managed where you don't even provision a server that someone else manages you don't provision anything you just pay for for the use of the database so it's a key value nosql database giving you singled digigit millisecond performance at any scale and that's obviously one of the selling points because it's serverless

you just pay for what you use if you need to scale it up or scale it down and the idea with serverless is no cold starts version upgrades maintenance Windows patching and no downtime big table is a product from Google and is the inspiration for Cassandra and hbase so this is different than some of the other datab Bas is out there because it specializes in wide data and by wide we mean lots of columns so if for some reason you have some data and you need to store thousands of columns about that data you might

want to look at something like big table this is a fully managed database from Google Cassandra took some of these principles and designed it to be a fault tolerant database so this is something that will be replicated across multiple nodes and can support basically a distributed system using commodity Hardware so you could have thousands of nodes and they don't have to be expensive servers so you can read about some of the use cases or case studies with Cassandra I think Apple at one point said they had 75,000 Cassandra nodes supporting 10 pedabytes of data so

this will basically enable a company to have a distributed system making that possible with commodity Hardware with a higher node count comparing to paying for just a couple really expensive servers for example so this is decentralized in Tech but it's not decentralized in ownership so these would all still be owned by Apple which is just something interesting if you ever do research in like decentralized technology or blockchain this is a little bit different where it's still all the same owner of the data it's really just about decentralizing the data across a substantial number of nodes

hbas is similar in idea to big table in Cassandra being a wide column database however the difference here is that it's all within the Hadoop ecosystem so if you have no idea what that is probably don't worry about it right now but again as I mentioned earlier about analytics this is an entire environment around big data and you might run into hbas if you're working within a Hadoop environment otherwise I would look at some of the other options here next up on this list we have surreal DB which is a newer database that you may

have seen people talking about online or maybe it's something you will run into but the target audience of surreal DB is not entirely clear because it's kind of a database that does everything so that's really their marketing it's like the database that could replace a lot of other databases you might be using so it's the ultimate multimodel database they definitely have good marketing and branding but it's probably up for you to decide if it's a great database or not with full graph database functionality surreal DB enables more advanced querying and Analysis which allows for multi-table

multi-depth document retrieval efficiently in the database without the use of complicated joins next on this list we have redis and this is definitely a good one to know so be aware of it but I would encourage you to try it out and become familiar with it because reddis is an inmemory cache that's what it's primarily known for reddest has a variety of other things it provides provides and can do but the main thing redus is known for is caching of data so you can make your backends faster or use less computation by not having to

retrieve or compute things that you've already computed so in memory cache very valuable for improving the speed of your applications now if you're using reddis purely as an in-memory cache it's going to likely be used in addition to some other database so you'll have some database that's powering your backend application if you wanted to visualize it you could then think of reddis as sitting between your backend application and the request from the client so it will first ask reddis hey has this value been retrieved or calculated before if yes we just give that value back

to the user but if it has not then we'll go process whatever needs to be processed retrieve new information from the database and then give that to the client caching it so if anybody asks for it again we can just get that cached file value now when we say in memory that just means this is not persisted to disk so if the Reddit server is restarted all of the data that has been cached will be reset and this is generally fine resetting a cach or clearing a cach is a pretty typical thing you would need

to do anyways so it's not the end of the world if your cash is cleared you're not storing anything in there that's critical that you couldn't lose now I did mention reddis can do a variety of things so for example you can have it persist data but when people think of reddis they usually are talking about an inmemory cache now let's take a look at mem cach tier this is another inmemory cache so this could be seen as an alternative to redis while it's also pretty popular it's not as popular as reddis and if you

were to choose a new caching system you're probably going to go with redis but if you even go to get hosted reddis for example you might see M cached as an alternative option so there might be some specific scenarios where mem cast shines and I don't want to say to never use that but reddis is going to probably be your default choice now elastic search we're going to keep this real simple it's a nosql database used for searching data so if you have some complex data set and you want to be able to full text

search it you could look into elastic search now let's take a look at Firebase and really there is like a couple of different products within that so there's Firebase fire store which is the default choice if you're going to go with Firebase but you might also hear of the other option which was Firebase realtime database which is still an option to use but pretty much in every scenario you're going to go with Firebase fir store so here is a quick comparison and just scrolling through this you can see that fir store is preferred in every

one of their scenarios so they are pushing to use fir store and to not use the real-time database but without getting into the details of the differences between these two let's first talk about what even is cloud fire store and when you might want to use it it's another Json compatible document database now I'm going to explain one of the reasons why I believe Firebase has become so popular while I'm sure there's lots of other features and capabilities when there's millions of different databases what is the reason Firebase has taken off it's because it's very

useful for what I would typically say are primarily frontend developers because the entire Firebase ecosystem is a good ecosystem to provide an entire backend so when you use Firebase not only do you get that database but you also can get a backend layer which can make it very easy to have a backend to connect your front-end application to so you can almost think of Firebase as a backend as a service you can just use Firebase for all of your backend setup and you just worry about the front end I've seen a lot of content creators

on YouTube basically using Firebase so that they can focus on the front end material not having to build out an entire backend from scratch it's also extremely popular for mobile applications as well so you'll often see Firebase being the backend for mobile app devs now one more important thing to know about Firebase is that it is a managed backend so this is hosted on the cloud by Google it's a Google product so it's going to be something that you have to connect to the cloud you don't download Firebase on your local computer however that's where

a Supa base can come in and that is positioned as an alternative to Firebase that's open source and the magic here is that you can get it hosted or you can self-host it so it's something that you could download put on your own server and use as you please this means there are also multiple options out there when it comes to managed super base hosting most likely though if you're looking for fully managed superbase you'd probably just go with the superbase company themselves who offers hosting or self-hosting it with any of the other major Cloud

providers without having the managed aspect throughout this video we've talked about hosting some but we haven't really gotten into the details of the different types of hosting and what that looks like that's what we're going to talk about next now as always when you're doing a video that covers a billion different Technologies it's kind of challenging not to overgeneralize or potentially not have everything 100% split up into the perfect categories because some of the things we're going to talk about will probably cross over into some of these other categories so if you're a rep from

one of these hosting providers and maybe you offer a little bit of everything here totally understand that and it's also something that you should understand as a developer often these services will provide a variety of different things but what I try to do here is kind of categorize by what these companies are most known for so first let's get the foundation what is hosting basically if you want to run a web server or some application and you don't want to run it from a computer in your house and have people connect to it over the

internet you can host that application you pay money to hosting provider they run all the services and then people can connect to it there we have three General categories I grouped these into so we have the shared hosting companies we have platform as a service and then infrastructure as a service we'll get to those two in a moment let's first talk about these shared hosting providers so when I sign up for shared hosting I may run say a WordPress website and there might be a lot of other people running WordPress websites from the same server

sharing resources so you can think of your application as just one of many applications running on that server all the resources are shared and there's no real separation from the different websites now while there's probably things set up to keep things secure between websites and make sure one website doesn't impact the others that is a potential risk so if you're paying for shared hosting and it's sharing Resources with another website on that same machine let's say something goes wrong with that other website and it starts to consume too many resources or maybe there's a security

issue there's a chance that could impact your website while it's ideal that that doesn't happen or that you know there's some things in place by these hosting providers to make sure that doesn't happen it's not something that would likely be guaranteed and because of that some people avoid shared hosting not only does shared hosting limit your resources but you're sharing the machine resources with other people it's the cheapest option and this will give you things like hosting for a few bucks a month or whatever it might be the way the companies package the hosting changes

so on one website you might pay per website or you might just pay a certain amount and get unlimited shared hosting so then you could set up as many websites as you want as long as you're a paying customer the big ones on this list you might be familiar with God Daddy dream host blue host sight ground these do all basically the same thing and I don't really have a recommendation on which one to go with I've personally used a lot of dream host I've used blue host and I've used GoDaddy all have been pretty

good I've also used hostinger I hosted a Minecraft server from hostinger at one point that was pretty cool 99% of the time you're not going to be using shared hosting as a backend engineer this is really if you just want to launch a simple website and you don't really want to do anything with custom backend coding you might be able to do some WordPress plugins or something but you're likely not going to be running a custom API from shared hosting in fact a lot of the shared hosting is only going to provide a very limited

set of options when it comes to what language and version so you might be able to use PHP and nothing else or there might be a few different options but most of the time it's going to be very limited so that's where the idea of a virtual private server comes in and often it's the case that you'll go from shared hosting and then upgrade to a VPS this is where you get basically a dedicated server but it's virtualized behind the scenes they might be running multiple of these virtual private servers on a single server so

there's some limitations however it's pretty much a dedicated server from your perspective so this is where you can basically have a Linux server you can run whatever you want on it and because of that you can run custom apis or anything that you need from that server now when you're doing this you're kind of in control to manage the inbound and outbound of this and you're likely going to need some kind of web server we'll talk about some of the most popular web servers here in a moment but just letting you know that you're kind

of in complete control of that environment next up we have dedicated servers and this is pretty much what it sounds like they host a server and you pay for it monthly you can do whatever you want with it it's not virtualized so you're not sharing Resources with anybody else you can research why you might want to go with a VPS versus a dedicated server it often comes down just to what you need and costs in this scenario you're basically just paying for the infrastructure so it's kind of infrastructure as a service and we're going to

talk about more infrastructure as a service in a moment whenever something says as a service it basically just means a subscription payment model they're running the computers for you and you're paying a month fee to use them however as a backend developer I don't tend to think of these companies when I'm looking for my backend hosting Solutions yes you you might run into that but most likely you're going to look at one of the major Cloud providers for infrastructure so Amazon web services Google Cloud platform Microsoft Azure those are the major three there are a

few others we'll look at but those are going to give you a lot of more services and you tend to really think about those when you say infrastructure as a service so we'll get into this next category here which is paas this is platform as a service and here are some of the most common ones you're going to run into as always there are a ton out there so this list is not comprehensive but versel netlify are two really popular ones right now and then we have a lot of these other ones from major companies

which will often help you deploy apps onto that particular Cloud so we'll talk about what that means in a sec but for versel and netlify basically you can think of them as abstracting away all the computer hardware and they just give you a really good experience for deploying backend applications and often you can do front-end stuff as well so it's not just specific to backend but that's what we're talking about in this video so if you're looking for a place to deploy an API you can do it on these platforms and they're going to make

the experience as easy as possible that's really what you're paying for when you're paying for a platform so to explain the difference real quick between a platform and infrastructure as service platform makes the experience very easy it's basically a wrapper around infrastructure whereas if you're paying for infrastructure you're in charge of setting everything up yourself which can be very painful it is a valuable skill but you don't get that nice oneclick deploy experience so versel netfi to my knowledge do not run any servers on their own they are going to use the infrastructure companies so

AWS and other Cloud providers and they're just writing software to make it very easy to do that without you having to go into AWS and set up a bunch of stuff because of this if you're doing cost comparisons if you just look in general versell is going to be more expensive than if you hosted on AWS directly that might not be 100% true all of the time especially if one of these platforms are able to provide some special pricing or you know perhaps set up the infrastructure in a way that would save money compared to

how you might set it up if you did it on your own or they might just be reducing prices for customer acquisition you can go to one of these platforms do something cheaper than if you went to one of the cloud providers directly Heroku is another one that's pretty similar in concept to versel netlify I guess I just didn't see it at first in this group because I had it prefixed with Salesforce I wanted to put Salesforce there because I didn't even realize Heroku was owned by Salesforce until recently so Heroku is another platform as

a service that's going to help you deploy to some of the major Cloud providers now let's take a look at this next one here AWS elastic bean stock this is a platform as a service on AWS so it's going to make the experience deploying to various AWS infrastructure a lot easier so while AWS provides the infrastructure they also provide a platform to deploy to it without you having to set up everything on your own similar idea with these other ones so I'm not going to go through each individual one Azure has something that's very similar

for Microsoft cloud and then Google app engine same idea for Google Cloud then we have digital ocean app platform I think they host their own stuff so they use their own infrastructure same idea with Oracle Cloud platform with the Oracle Cloud so let's take a look now at the infrastructure platforms these are the major ones so we have Oracle Cloud infrastructure that's just the full name for Oracle Cloud we have Amazon web services Google Cloud platform Microsoft Azure IBM Cloud Alibaba cloud digital Ocean aami or LOD and there's a ton of other ones out there

as well pretty much any companies that provide infrastructure or Computing Hardware they're going to be in infrastructure as a service which I mentioned briefly but you know some of these that offer dedicated servers could be considered infrastructure as well I just group them separately because I tend to think of those more as easy hosting providers and then I kind of think of this group here as companies where the main thing that they try to sell is infrastructure so when you go to one of these websites they don't advertise hosting a website for $3.99 a month

or whatever they're basically selling a variety of different Services all of those around infrastructure so far so good we've made it through three major sections in this Center section and then we have these two floating over here on the sides but let's go through the next one which The Logical next one is going to be clients and servers since we were just talking about hosting what web server software is actually used well if you're working with the infrastructure directly or you have a dedicated server you're going to be responsible for choosing what web server you

want to use and the web server is what actually handles requests to that computer and makes it possible to run applications from that computer the three major ones I'm familiar with are engine X Apache and IIs there's also caddy on here which I've heard a little bit about when I was just researching different web server software but I haven't used or experienced that directly and as always there's other ones out there I'm sure but these are the big ones you might run into so let's just talk about I real quick this is a web server

that's really popular for asp.net and Microsoft applications so if you're not doing that then I would probably go with one of these two in the middle I have used both of these a bit but I've grown to kind of like engine X really it's just about getting them set up and then probably never touching them again because they work you might have to do a little bit of stuff with like SSL certificates and maybe some configurations for routing or redirection but overall you're not going to be doing a whole lot at this level hopefully it's

just something that you can set up once and it works this is where more differences show up between using infrastructure and using a platform if you're using a platform as a service there's a good chance they abstract away all the web server stuff for you so you don't have to even think about setting up engine X or Apache it's just done the difference is if they just hand you a virtual machine or a dedicated server you're going to have to learn how to set all that stuff up yourself now let's talk about things from the

other end so the client that actually makes a request to the web server so when I say client here I'm specifically referring to the software that makes the request so the browser that's going to be the case majority of the time people are going to make requests to your web application through HTTP or https doing that through a browser so I group these in three different categories but if I expand these it'll basically just be the major browsers you might be familiar with already the first word over here is the engine that powers those web

browsers and then in parenthesis is the JavaScript engine similar ideas and you can do more research on the differences but behind the scenes these four here in This Blink V8 section have similarities because they share the same JavaScript engine the engine is what actually makes it possible to execute JavaScript code within the browser and you're probably going to get a more consistent experience between these four because they all use the same JavaScript engine now just to get sidetracked for a minute if you're studying JavaScript and browsers and engines you might hear the term runtime V8

is not the runtime you can think of the runtime as where the JavaScript code is executed so you could say the runtime is the browser here but if you use node.js or backend JavaScript the runtime is actually node.js and the engine is still V8 so we still have the same core JavaScript processing it's just the location is changed from front end in a Brower browser to backend with no JS so just saying that to be careful not to say V8 runtime it's the V8 engine not to be too picky on words but you might hear

people say one or the other and just wanted to clarify what is correct now as for the words webkit blink and gecko you might hear these as the rendering engine they are what take the HTML CSS and JavaScript and make it functional in the browser by providing different browser apis and different things you can do with Java script so it's a partnership between the JavaScript engine which provides the language and the execution of the JavaScript and the rendering engine which provides the apis and the Dom so while this isn't really designed to be a tutorial

on web browsers it's just important to know that behind the scenes of these common web browsers there are a lot of things going on and the introduction of a web browser has changed a lot of things as we go on web browsers are becoming more powerful and more standardized and readily available on more and more systems and what this can do is give us a standard environment that we can build upon so that we can create apps that run on any systems this is kind of similar to how operating systems abstract away differences in Hardware

one layer higher browsers abstract away differences between computers so you can create an application to work correctly in a browser and we don't have to worry about whether it's being ran on Windows Windows Mac or x86 or Arm based systems we just create it to work in JavaScript with these different JavaScript engines and rendering engines you may also hear of this set of browsers as chromium browsers which is an open source browser project that Chrome was built on so that scratches the surface of web browsers there is a ton more on how web browsers work

they're kind of magical that does get more into the realm of front-end development but certainly not not a waste of time in terms of studying as likely even as a backend engineer your work is going to be tightly coupled to a browser environment especially going forward as more and more apps are moving to browsers the skills that you develop in browsers definitely will not be wasted now let's talk about cdns which kind of sit between servers and browsers so if you make a request from a browser to hit somebody's web server you might go through

a CDN which stands for content delivery Network here are the most popular ones you've probably heard of cloud flare if you've worked in back in development at all and then we have cloudfront which is Amazon's aami CDN Azure CDN Alibaba CDN Google Cloud platform CDN seems like every major cloud provider has some CDN service and even cloudflare does offer some infrastructure as a service we didn't talk about it specifically but you can look up cloudflare workers this is their serverless compute option and then they also have R2 which is their storage but let's not get

back to infrastructure let's focus on cdns here the CDN is a Content delivery Network and its purpose is to do two things the first of those two things being Edge distribution so if you imagine a graph and you have a bunch of different locations where the center might be your web server the CDN is basically going to copy the pages that people request and distribute them to all of these different locations so that when somebody closer to one of these Edge locations makes a request they don't have to go all the way to the main

server they can just request it from an edge location now in general the internet is very fast you can make a request to something on the other side of the world and get a response relatively quickly but if you're building a service and you want to make sure the times for loading are as quickly as possible the less distance the request has to travel because you know there's actual networking for these requests the faster the response to that request is going to be received back by the client so this can drastic Ally improve performance so

that's the first major thing and then the second major thing which is included in this concept is caching so when someone makes a request instead of having to recompute that information they are just getting a cached version of that page which is going to be a lot faster because there doesn't have to be any computation or requests to a database now caching is a more challenging topic when it comes to backend engineering and it's probably something I'd like to make a dedicated video on like all the different layers of caching but just as an example

you can have caching in the browser you can have caching at the CDN level you can have in memory database caching with something like redis so there's a lot of different spots where information can be cached if you're ever in a scenario where you make some change but you're not seeing that change reflected there's a good chance it's being cached to fix that you may have to clear some caches so if you deploy a new change to your API there's a good chance you'll need to go into cloudflare or any of these other cdns clear

the cache so that the new pages and the new requests get sent to the edge notes and then you could also have questions about do you cach the actual API responses or do you just cash frontend pages and these are all things that you can have control over the main thing you need to understand is that any level of caching you'll need to clear that cache if you need new changes to be propagated so that users get the latest data so that's your basic intro to cdns definitely an important topic to be really familiar with

as a backend and engineer now another section that may be interesting to add in here that I did not put in here would be internet service providers isps these are the people you pay for internet access and I say that because isps are responsible for the routing between these different sections so when you make a request from the browser to some server on the other side of the world it's because of your ISP that that request is made possible I didn't want to put that here right now though because I didn't want to get too

much into the physical networking because it is something you can kind of just abstract away from your mind you just know that requests go through the magical internet Black Box you don't really need to know all of those details as a backend engineer but if you are working in a role that might be a little bit lower level working at the network level you may benefit from having a deeper understanding of how the actual internet networking works I obviously decided to leave that out but just wanted to give it a quick mention if it's something

you wanted to spend more time studying that wraps up the section on clients and servers up next we have communication and this is a general category where I wanted to talk about some protocols as well as different notation or basically the language or the way you decide how to write things that you communicate you can imagine calling somebody in real life on a phone and just because you have a means to communicate doesn't mean that you can communicate you still have to have the same language so if I'm speaking English you're speaking another language we're

not going to be able to communicate Translating that back to our example here the phone would be the means of communication but the actual language would be the notation and that would basically just be the exact syntax we use in our code to communicate let's first look at this section here these are the main approaches to communicating you don't necessarily have to have experience using all of these but you should at least be familiar with these now I say this with pretty much every section there's probably a lot more in software engineering there's pretty much

Unlimited information so don't feel like you have to know everything but you should at least be familiar with these main ones so if somebody says you know web hooks you know what that is that's actually probably a good one to get experience with but for example someone says a soap API you don't want to be like what I already like took a shower today like what are you talking about so we're going to give you the basics of what each one of these are but you don't need to go and actually Implement all of these

there's just a few you should really have EXP experience in so let's first talk about rest apis because that's going to be the most popular this stands for representational State transfer which basically is a way to transfer state or data through HTTP or https which is one of the protocols we're going to talk about here in a moment so the way you do this is you associate a particular verb you can kind of think of it as the official term is HTTP methods but you can get data you can post data you can delete data

put data and there are a variety of other HTTP methods this describes your intent when you make a request so when you go to a website such as youtube.com that is a get request you're requesting the HTML JavaScript CSS from the server there's also scenarios where you post data or you can think of it as create data and this would be submitting a form in that situation you're giving information to the server and you associate that with an http method so the server understands how to interpret it when you're building a rest API you're going

to make end points on the server side to accept different request types so the user the end client could receive data with a get request or it could give us data with a post requests this is often going to use Json which is the notation that's basically the syntax for transferring data back and forth and this is probably something you're familiar with if you've done any backend development so rest it's a really good thing to know you should probably prioritize that here first so let's talk about soap real quick the only thing you really need

to know is that this was the approach to building apis before rest and it used XML so rest is the modern replacement and you shouldn't be creating a soap API at any point now that's a general statement there might be scenarios where you need to build a soap API to interact with some Legacy systems or something but really rest is going to be the modern default now let's talk about the flip of this so a rest API the client makes a request with a web hook the server actually announces something so this is really popular

with Integrations so if you want to be notified when something happens often times services will provide web hooks that you can hook up to so that you receive a notification when something happens so that's pretty much the summary of web hooks definitely important if you're building any kind of notification systems or interacting with other services out there on the internet now let's talk about graphql this will serve a similar purpose to something like a rest API but the way it's done is different the idea here is you can make a request to the server let's

say you were trying to retrieve some information but you can adjust that request almost like a structured query language query SQL with it instead being called graphql so the client can adjust that request and retrieve just the information it needs the cool thing here is that on the back end you just set up the graphql endpoint you know set up some rules or restrictions but then the client has control and can request exactly what it needs the control is put into the client's hand relieving some effort on the back end so pretty cool structure especially

if you do enjoy like the database feel of it where you can select exactly what you need to retrieve from the back end so definitely something worth looking into and there's also a lot of services around graphql for generating the appropriate requests so you can set this up in your application so that the clients can go in and adjust their requests with like a web page and it's not always in code so pretty cool thing that you should probably at least know the basics of next up we have a couple things here that say RPC

which stands for remote procedure call which you can think of that idea as invoking a function on another computer Json RPC is a very simple implementation of a communication protocol but a major difference is that it's bir directional so rest is a one-way thing thing where the client makes a request and then the server responds but the server is never going to initiate the communication with Json RPC you can have bidirectional communication where the server initializes some communication this is Handy for real-time communication because the server can say something as soon as it needs to

instead of waiting for a client to make a request grpc is a very similar idea and this is a protocol built by Google and it's going to utilize protuff which is one of the other notations we're going to talk about so protuff are really useful for intera communication so if you have microservices or perhaps you are working with some lower level languages and you want to integrate that with an API you built in another language you might communicate between these apps using protuff so grpc is a communication protocol that utilizes protuff and it's a bidirectional

protocol so the server can communicate to the client directly without waiting for a request web RTC server side events and websockets are all things for real-time app development so you can look up all of the differences about these I would say that web rtc's specialty is with things like video serers side events is when servers need to broadcast events and then web sockets are ideal if you want ongoing instant communication so if you had a video game or instant messaging that's probably where I would go with websockets so when it comes to getting a grip

on all of these rest is the default for API learn that when it comes to realtime stuff learn websockets when it comes to broadcasting announcements web hooks when it comes to intera communication grpc or rest so that gives you a pretty good idea of these different comms protocols now I've said protocols but really you should probably consider these as means of communication or architectures for communication using protocols so HTTP https is a really good example of a protocol that rest uses to communicate over so the protocol is actually more of the technical spec detailing how

that communication actually happens and all of the communication Styles we just talked about are made possible thanks to the protocols so perhaps this would be better just said as apis or Communications whereas the protocols which we're going to talk about now would be some of these here so HTTP https hypertex transfer protocol and hypertex transfer protocol secure these are probably the two most popular you're going to see and these are built with methods get post put delete patch and others TCP is a layer lower so HTTP and https will use TCP so this gets all

into the different levels of networking which you can do additional research on you probably just need to understand the basics to Be an Effective backend developer but obviously knowing more is not going to hurt TCP is used to make network connections another one you'll probably hear is UDP with the primary difference that UDP is not a reliable means of communication you can think of it as one computer broadcasting a message and then anybody who picks up that message takes the message but if they don't get the message the sender does not care it's basically just

broadcasting that information so TCP is a connection based protocol UDP is connectionless if you need to transfer data and you need to make sure it gets there without being destroyed that's where TCP is going to come in now as a developer building apis you don't actually usually have to worry about this but if you start working with lower level systems where there's some network communication then you might have to be aware of messages being transferred via TCP or UDP now another thing I wanted to mention is that devices get IP addresses you will see ipv4

addresses and IPv6 which is designed to ultimately replace ipv4 so if you see something like 1920 0.0.1 that's an example of an ipv4 address that's the loop back address so that refers to your own computer but there's a variety of other number combinations which are going to refer to individual devices IPS are given to the entrance to your network and then devices within that Network can have IP addresses that are local to that Network meaning my device on my network might have the same IP address as your device on your network but because the those

IP addresses are just local to our individual networks those are not a conflict by any means however we should both be given a public IP address from our internet service provider these will need to be unique but that public IP address is for my entire network and does not uniquely identify a single device due to there being a limited number of ipv4 addresses IPv6 is being introduced and is slowly growing adoption where you can now have many more different IP addresses so that's ipv4 and IPv6 next up another protocol you should be familiar with is

SSH this is used to remotely connect to another computer often it's used to connect to a server and the SSH stands for secure shell and it's probably one of the first things you're going to get experience with when you start working with another server where you're not just working on your local environment for building test applications if you have a dedicated server and you need to work on that server you'll connect via SSH you'll be within the terminal so you'll probably want to be comfortable with a terminal based editor if you need to edit any

text files for example so we'll get into some of that as well but that is a pretty important protocol to be familiar with and when I talk about these protocols you don't necessarily need to understand how they're implemented you just need to understand what they are so SSH is to connect to another computer through the terminal FTP FTP secure secure FTP these are all things to transfer files between computers pretty common if you're working with shared hosting and you need to upload some photos or something so you might have heard of filezilla that's a software

that will use these protocols to allow you to transfer files between computers now the next section I have here is on email this will come up if you're working with custom emails for a domain so if I wanted to have info@ calb curry.com you might run into these settings or if you're trying to set up email on your phone even you'll probably run into these SMTP is the protocol for sending email and then IMAP and POP 3 are for incoming email the major difference is that typically with pop 3 the emails will be downloaded and

not kept on the server whereas IMAP will have them downloaded so you can read them locally but they're still up on the server then we have down here messaging wanted to add this in here especially if you're working with something like Rabbid mq messaging software can be used to facilitate communication between microservices so you don't always have to utilize just these things here software exists to help facilitate communication between smaller services and these are the protocols that might be used behind the scenes I think the major difference is this one being for more complex data

mqtt for simple data and stomp for simple text based data so those are something you might at least want to be Familiar of those words and you can research them more if you are in a situation where you might need to use them now again and these are the protocols so you're probably not going to be writing mqtt stuff you're going to be utilizing something that uses these protocols so at this point we have a pretty good understanding of different protocols you might hear about as well as the different paradigms of communication now let's talk

about the notation which is when you receive a message how do you actually parse the text to know what it says so these are the main ones you're going to run into most popular is probably Json for apis XML is still a thing but that was mainly used with soap apis so you probably aren't going to be using that a whole lot for this we talked a little bit about Proto Buffs already when we mentioned grpc now this is independent of grpc but they'll often be used together a protuff is interesting because of a few

different reasons the first is that they are serialized into a binary format this takes up less space and makes communication more efficient the other major benefit is that they have tight information so the fields within a protuff will adhere to particular data types the structure for the data is described in a Proto file you can use the Proto files to generate types so for example classes or structs in various programming languages so if I build a bunch of information and I want to communicate to a new app with protuff I can give you the Proto

definitions and you can generate the correct types for your app even if it's in a different language so that is an overview of Proto Buffs or protocol buffers for the full words now let's talk about yaml and toml you'll probably run into these for configuration files these are two you're going to run into as well as markdown for notation often for documentation so for example a readme file is going to be a MD file working with markdown will allow you to easily describe how that information should be displayed and there are a lot of markdown

renderers to show that information CSV or comma separated Val is a common format if you want to export some data so say you have an Excel spreadsheet you could probably export that to a CSV file and import that into a different program the ones I would probably start off with is Json and probably markdown so you can create a readme for your project you only really have to worry about these others if you're using them for specific use cases so if you're not working with protocol buffers you don't have to worry about learning Proto Buffs

right now there we go we covered the types of communication the protocols and the different notations you should be familiar with so that covers the five core sections of this mind map the next two sections we have the appdev life cycle which basically talks about the most common things you're going to run into when you start building an app from the start all the way to production and then the section over here is going to talk about some of the most important cloud services if you start working with something like AWS so we'll start by

first taking a look at the appdev life cycle and then we'll come back to the cloud services so we'll expand this and we'll go through these one at a time so first taking a look at your local development environment and this is probably basic stuff you're familiar with you're going to have an operating system which is going to be Windows Linux or Unix or Mac now within Linux and Unix we could get into tons of different operating systems we're not going to get into all of that in this lesson it's definitely worth exploring especially since

you'll probably be deploying to a Linux operating system you might want to be familiar at least with some of the main categories Ubuntu red hat and all these other variations but the main categories is Windows Linux or Unix and then Mac which is technically Unix based but it's going to give you a much different experience than developing on Ubuntu for example generally you should be familiar with your main operating system and some form of Linux because you're going to be deploying to Linux servers so you should probably start familiarizing yourself with the Linux commands as

for local development we have editors there's three main categories and I just gave a few examples not all of the different ones out there you'll want to know at least some terminal based editors so Vim neovim Nano especially if you're sshing into servers then there's the text editor or you might probably hear code editor and I use the example of VSS code it's basically like a more involved text editor giving you some capabilities around code but it's not a full IDE idees some examples would be pie charm or Visual Studio there's often an IDE for

every language that people will tend to use these often have a lot of more capabilities or features especially if they're language specific you might get python specific stuff with py charm and visual studi is going to have a lot of features with anything built in the net world so C you're going to get a lot of capabilities there which one do you choose that's ultimately up to you I personally use vs code right now and I've used to use idees I've worked around with terminal based editors you should probably try out everything but you'll probably

gravitate towards one depending on what language you use and what you like I like vs code cuz can do pretty much anything next up we have Source control you may also hear Version Control the biggest one here is Git You may hear of subversion or other source control systems but git is pretty much the one everybody uses now you'll want to use git for all of your code and that'll help you keep things organized but most likely you're going to want to host that somewhere with the biggest popular providers being gitlab GitHub and bitbucket so

GitHub is different than git git is the software GitHub is a hosting provider where you can store your repos so would recommend using those as well next up we'll talk about containerization which we haven't really talked about much yet so we have Docker this is pretty much the only containerization software you need to be familiar with and then this is going to be used to create images now this is something we could get into in a lot of detail we're just going to go pretty high level here basically you can think of Docker kind of

like a very light VM that runs your code in isolation and when you work with Docker there are images and then there are containers images is basically a snapshot of what the container should look like and you can generate containers from that so when you create an image you can keep that locally or you can upload it somewhere here are some of the most popular image hosting websites which some of these are acronyms which hopefully I can remember we have Harbor this one we have Google container registry or Google artifact registry here we have Azure

container registry dockerhub elastic container registry this is Amazon's version then we have GitHub container registry and gitlab container registry there are a million places that you can store these containers often within whatever ecosystem you're working with so if you're hosting your source code on gitlab you might use gitlab's container registry so that is the basics of Docker but it's often the case you'll need to create containers from multiple different images and get them all working together and because of that we have this orchestration section with the most basic example here being Docker compose which is

kind of like the default way of doing this if you're working with Docker you'll probably learn about Docker compose and this will help you run your applications if you need a little bit more there is Docker swarm and kubernetes and then I have this one on here Nomad something I'm not actually really familiar with but I read about that so you might want to look into that in more detail the big one you're probably going to hear about is kubernetes some people will use Docker swarm but I would say in general the preference has grown

to be kubernetes and this will allow you to control your containers and basically orchestrate them which is the entire section we're talking about kubernetes is going to help you deploy a bunch of containers scale as needed fix things if certain containers go bad so it's additional software that you might end up using there are a lot of different kubernetes Services out there so I listed some of the popular ones here Google kubernetes engine elastic kubernetes service Leno kubernetes engine Azure kubernetes service I probably should have spelled these out and then IBM kubernetes service digital ocean

kubernetes you can see there are a lot of hosted kubernetes options now this is the default kubernetes just hosted sometimes companies will build upon this offering additional services or uh basically allow kubernetes to work nicely within a larger ecosystem so this introduces some of the Enterprise extensions so a big one here being open shift from Red Hat so if you are working within an Enterprise Red Hat environment you might use open shift as your kubernetes option and then there are a couple of other ones you may have heard of here which you can do additional

research on but the idea of Enterprise extensions is that it's basically kubernetes with more features and more cost but that can sometimes be worth it as these providers will help you deploy and get things working so that is a crash course of containerization probably something we could have covered in a lot more detail but this will give you basically the essentials the next thing thing is you can start using cicd continuous integration continuous deployment and this will help you deploy things automatically cicd is a series of steps you may also hear it known as a

pipeline and these steps will take your code from a repository such as GitHub run any tests any kind of validation checking the syntax and then ultimately deploy it onto a server now that server could be for a testing environment or it could be your production environment this could be tied based off of which branch you're deploying to or various other configurations so not only is it just taking the source code and getting it onto the production server there are various other things you can do a lot of testing is done in this stage to make

sure your code meets certain expectations so we're specifically talking about automated testing where that could tie into a cicd pipeline you could also first deploy to a testing environment which can then be manually tested by developers stakeholders testers anybody who would need to interact with that application before moving it to a production environment so opening cicd you'll see a bunch of different tools you can use Jenkins is a very popular one as well as GitHub actions gitlab cicd if you're working within the gitlab ecosystem and circle CI now there's also AWS code pipeline Azure devops

and gcp devops these are the major cicd tools for the big cloud providers each one of these is going to have its own way of doing things and potentially different limitations so you might need to do some research on which one one of these you want to use it's often the case if you get a job you'll be given a setup that already has a cicd pipeline if there's a dedicated devops team this is probably where they're working and probably more in the containerization space now you can tie these two things together so if there

are certain steps when it comes to creating images you can often tie that into the cicd pipeline another thing you will want to do when you are deploying software is test the software I don't do any of that because I don't really make software that has bugs but in theory since you guys probably do make bugs you'll probably want to test your software here are some of the most popular testing tools now generally there's probably a testing framework for whatever language you are using I used P test here as an example so if you're working

with a particular language for your backend find out what testing Frameworks exist and you can use that to create tests in your code this can help you make sure you don't have any regressions if you make some changes you can run all your tests confirm that they work and this is a great way to check edge cases so if you for example put the highest value or one above the highest value what happens what should happen What happens if you make a certain request type to the API you should get this type of response making

those things all automated Postman is a great tool for those API tests where I would probably use whatever language is testing framework for testing functions so actually checking the functionality Postman tests I would instead check the end to end so describing what the request should look like and describing what the response should look like for that type of request you can create a collection of these different tests run them all at once making sure your backend is fully functional with a single click of a button now this is additional code that you have to write

so if you make any changes now you have to change not only your code but you have to change the test as well you also might have to debug tests so not only can you have broken code but you can have tests that are broken as well maybe it says the test is passing but it's checking for the wrong thing then you have a false sense of security and more code to maintain congrats that's what it's like to be a developer so keep your tests minimal just testing the core essential things you need to test

so you don't just have a bloated code base and a ton of testing you have to maintain now Swagger I tend to think of this more as a documentation tool it will allow you to see all the end points for your API but you can easily look through the different API possibilities and test them in the browser here's an example of what that might look like so you can go in here and say hey let's get the stores inventory and then you can run this in the browser getting the response confirming that it works the

way you would expect down here we have selenium Cypress and playright three different automation tools you can look into these can be used to test user interfaces as well as apis and do various things and some of these tools can tie into cicd so you can automate some of that testing and making sure things work before doing an automatic deployment so definitely want to look into these different tools here selenium is more of a front-end testing framework but you can still use it to check that your website's working properly which can help you confirm that

your API is working properly but if you want to work with apis directly probably recommend Cypress or probably playwright which is probably the one of these three I would start with and there are a bunch of other API testing tools out there as well so whatever testing Library you choose to use just stick with it learn it really well and start to become comfortable with some of that automation for me I just use Postman I created a collection of all of the different requests that I would want to make to the API wrote some automated

tests and I was able to export all of that into a file so I could keep it in my repo maybe that's like an old school solution they have some Cloud stuff you can use now but just showing one option that you could try if you're going to be changing it all the time you probably not wanting to export and do all that stuff regularly so you can just keep it local or you can use one of their Cloud options if you're working in a team environment probably getting that cloud variation setup would be ideal

now this one's pretty high level but you need to keep track of stuff somewhere thought I would mention some of these as you probably run into them I don't think you really need to go and study these they're pretty basic but here are some of the different locations you might store software issues now obviously GitHub issues that's going to be the most common for actual code issues as well as gitlab issues but when it comes to tasks and organizing the development process notion bana Trello jira all tools to better your life by making things more

difficult so check out all of those if you want to try those out personally I use notion for pretty much everything it's actually pretty handy but if you're working in an agile environment or something companies do have a tendency of overdoing this and making the actual tracking of development more complicated than going and building stuff but that's an issue for another day next up we're going to talk about monitoring these are tools that you can use to keep track of your running applications making sure they are working as you would expect this could be monitoring

memory usage request counts or it could be keeping track of any issues that the end users are experiencing I'm not going to get into each one of these the one I'm familiar with most is cloudwatch which is part of AWS so that will allow you to get an immediate view of your application as well as keep track to make sure no issues are happening with that you might want to tie into some notification system which some of these might have things for that built in but two popular ones that I wanted to mention here is

pager Duty and then Ops Genie so these are things that can text you call you say hey I know it's Friday night and you're hanging out with friends but you just broke the entire website so go fix that so if that sounds fun check out one of these also why you shouldn't Deploy on a Friday by the way but that's pretty much the end to end so let's do a quick review you start with your local development you push your code into Source control you may containerize it you can optionally have a cicd pipeline which

will take the code from Version Control or from the images get it out into production you can have steps in there for automated testing as well as manually testing using various tools and then you can keep track of all your problems in these issue tracking softwares and then monitor make sure things are going as planned and getting notified if not we're almost there we just have one major section to cover now and this is going to be about the big three Cloud providers talking about the most common services the services you should be familiar with

will depend on the type of work you are doing so I'm going to cover the most important ones however if you specialize say you're doing a little bit more stuff in data science you might get a few more services or if you have some other specialty that you're working in these ones are going to really be about if you're deploying apis what kinds of stuff you're going to run into now as a general approach for this I think you should prioritize one cloud provider now yes eventually having the understanding of other Cloud providers is good

as you might be working in a multicloud environment a company might use multiple services so you should be familiar with all of them to some degree but even just knowing the equivalent services will probably get you far enough because you can compare it to what you already know in one of the services so if you don't know which one to choose if you're working within a company just pick the cloud provider that your company uses if you're going to be working in an Azure environment go ahead and learn all of the Azure services if that's

not the case then I would just recommend AWS AWS has the largest market share and that's just the one I'm most familiar with so that's what I'm going to recommend but of course you're not going to go wrong learning any of the others so if you have a preference for Google Microsoft Amazon go ahead and pick your favorite unfortunately though you do not get unlimited free computation with Amazon Prime now when I was first learning these I was finding it myself getting frustrated because it seems like Amazon has some wild names for these and you

have to kind of memorize them this frustrated me at first but as I started to learn them it was actually helpful because the Google Cloud platform and Azure names are more general or generic that they're actually harder to remember for example AWS elastic bean stock such a specific weird name that once you actually have it memorized it's hard to forget compare that to Azure app service it's just kind of General at first that might be helpful because you can kind of guess what that does but when you introduce a bunch of services and they all

have these General names it gets a little confusing that's just been my experience take it for what it is let's just start going through these one at a time and I'm sure as soon as I finish this video I'll think of a few more services I should add so if that's the case I'll update the Mind map so you can go get the most upto-date version a lot of these we've already talked about which is great so AWS cloudwatch it's all part of monitoring so monitoring is used to keep track of what's going on with

your applications then we have gcp Cloud monitoring and as Azure monitor managed databases awsrds gcp cloudsql or SQL Azure database so again as a reminder this would basically be if you wanted to for example host postrest in the cloud RDS is not the database it's just the name for the database hosting for storage we have AWS S3 gcp cloud storage and Azure blob storage kind of like the word blob so if you need to store files for file uploads in your application you'll end up using S3 or the equivalent from other clouds next up we

have compute so if you need to allocate some computing power ec2 gcp compute engine and Azure VMS this is the equivalent to a dedicated server from a hosting provider so if you went and got a dedicated host server from one of these providers that is the closest thing AWS has now we've talked a little bit about manage services and then we also have serverless services so when something is serverless it still has servers behind the scenes it's just that you do not deal with them you just pay as you use it so you do not

provision a certain size of computer or a certain amount of Ram or anything like that so we have AWS Lambda gcp Cloud functions and Azure functions identity this has to do with permissions kind of boring but I wanted to throw that in here because you probably hear it AWS I am gcp I am or Azure ad active directory the I am is identity and access management so this is used to control the access different users have as well as services so Services can be limited by the rules you define here now when it comes to

DNS if you want to buy a domain or control different DNS settings we got AWS Route 53 gcp Cloud DNS and Azure DNS now if you're working on anything with virtual clouds you'll have AWS VPC gcp VPC and Azure vnet now when it comes to CDN services AWS cloudfront gcp Cloud CDN and Azure CDN for cicd we have AWS code pipeline gcp Cloud build or deploy and then Azure devops pipelines now when it comes to certificate management so say you wanted to have SSL or https for one of the services you are running you would

use AWS ACM or gcp certificate manager or Azure key Vault certificates now when it comes to Containers we have awcr elastic container registry gcp artifact registry and Azure container registry going back to one of our previous things on image hosting if you see GCR this is replaced with Google artifact registry but you might see the previous floating around as well next up we have serverless compute which is a little bit different than serverless functions I guess I didn't get into that in much detail but if you have some function that you want to execute you

can use a service like AWS Lambda and you don't have have to worry about the larger environment that that would normally run in contrast this with serverless compute it's basically serverless Computing but not just at the function level you could actually run full apps from this so we have AWS fargate Google Cloud run and Azure container instances kubernetes we have AWS elastic kubernetes service Google kubernetes engine and Azure kubernetes service now we have this IAC which we haven't talked about yet but this is short for infrastructure as code so something you might run into you

might not need to do it but basically this gives you a way to describe the cloud infrastructure you have set up in code instead of going into the console and setting everything up manually this will give you the control and flexibility to do whatever you want and describe exactly what that structure is so you could one check that structure into some Version Control and share it with others if needed and easily reset Services because you have the exact configuration defined in these code files so AWS cloud formation gcp deployment manager and Azure resource manager and

then lastly we have load balancing so when you make a request to a server going back to this example here you might not just hit a single computer you may actually hit a load balancer and the load balancer job is to distribute the requests to multiple servers there's a few major benefits here it will distribute and balance the load across multiple servers so you can scale and it will allow you to replace individual servers without the service going down so say you had a web application and you wanted to scale to more users you're already

capped out what are you going to do one option is to scale vertically and this would be you can visualize growing this way using a larger server with more resources that can handle more users and this is fabulous but that doesn't give you the fault tolerance if that server shuts down for some reason so if you have multiple servers you can load balance them and distribute the load horizontally this is the difference between scaling up and scaling out which one should you do well as you grow you're most likely going to want to have that

fault tolerance provided by a load balancer so you'll definitely want to scale out to some degree and then you can adjust those servers vertically so a little bit of both now I'm sure there are detailed guides out there on the details of scaling web applications but that will give you the basics you want to have good machines that can handle the workload but you also want that fault tolerance and you achieve that with load balancing so we have AWS elastic load balancing Google Cloud load balancing and Azure load balancer and then for other surfaces TBD

but maybe I'll add in some more here later the final thing is the example learning road map and this is what we're going to talk about in the upcoming video so two things be sure to check out this road map down below also I have mentorship if you want basically guidance accountability and skill sharpening to get to the point of being a good backend engineer and getting employed I help people with that so I have a link down to that below so that's the first thing the links the second thing is check out the upcoming

video that's going to take all of this information and tell you the exact things to study if you're new or overwhelmed with all of the different options you don't have to learn all of these things to Be an Effective developer so this road map is going to give you exactly where to start so this is pretty much every technology you should be familiar with if you want to be a successful backend developer I'm positive there are plenty that I missed there's probably some that maybe I should remove from this list these are the questions I

have for you so if you have any things that you would like to add drop it in a comment down below again check out the links to get this resource and to the mentorship and I will see you in the upcoming lesson where we'll talk about the road map see you there