CI/CD with JetBrains TeamCity | TeamCity Tutorial

in this video we're going to talk about a modern and powerful ci cd tool called teamcity first we're going to see some of the cool features and benefits of using teamcity we will also see the architecture of how teamcity actually works and finally i will show you how to setup and configure your first continuous integration pipeline in teamcity in under 15 minutes traditionally when we think about cicd tools like jenkins we think about a tool that needs a lot of administration and configuration so administrators have to set up the tool and configure it with bunch

of plugins so that developers can start using it and start creating build pipelines for their projects in jenkins specifically you have a vast ecosystem of plugins and for any imaginable configuration step in your build pipeline like deploying to a remote server for example using credentials checking out from a git repository executing npm commands etc so for any of these steps you have to find a suitable plugin and configure it and use it for your build now the main advantage of teamcity is that the setup and administration is way easier so if you want to set

up a web hook for automatic build triggering for example you don't need a plugin for that you get that out of the box or you want to use gradle with a specific version for your project no need to configure it you can simply use it in your project or you want to connect to a docker registry or slack account simply add a connection to your project and you can use it in your build configuration and most of the common configuration steps that i mentioned are already built in so you don't need any external plugins or

third-party add-ons to use them now if you don't want to set up and take care of the teamcity installation yourself you can also use the manage solution teamcity cloud to create and run your builds which of course makes getting started with teamcity even easier now as i mentioned creating and configuring builds is very easy using teamcity ui but in complex projects with lots of logic and lots of steps you probably don't want to be configuring everything in the ui plus you may want to easily replicate your build configurations so instead you want to automate and

script that whole process which means all your build configuration is in git repository so you get version control and history of the changes in your configuration so you can maintain your build configurations much easier and make them reusable and you can actually script the builds in teamcity with kotlin which is an open source programming language now compared to yemo format since kotlin is a full featured programming language you can write more powerful logic and use all the tools and libraries that it provides and also it makes it easy for developers themselves to script their ci

cd pipeline configurations now if moving to configuration as code completely is not an option you can do a hybrid in team teamcity meaning make changes to your build configurations in the ui and then commit those changes as code to your code repository so this way you will still have your build configuration as code another cool feature of teamcity is that you can create build chains from your builds by defining dependencies between them that means you can configure which builds depend on which ones in a chain so whenever you run a build teamcity will check all

of its dependent builds and run them first in a correct order and you can also easily pass the artifacts or other outputs from one build to another but in addition to running builds in sequence you can also run them in parallel an example use case for this would be if you want to run the same tests on different platforms or environments like on different operating systems or different cloud platforms etc before building and deploying the application and releasing it and one more great feature i want to mention is that you can run what's called personal

builds directly from your code editor personal builds are like developers own builds which only they can access so this is a very convenient way to trigger builds on a remote team city server from your local development environment this can be really useful if you want to test your changes before committing your code personal builds are available for all major ides like eclipse intellij and visual studio so if you have teamcity plugins for these ides you as a developer can also see build status configuration and all this information in your ide without leaving your local development

environment now let's see how team city works it has a very clean architecture you have a central management component called teamcityserver where you can configure your builds and see your build information all in a nice ui now once you configure the builds they need to run somewhere right for that team city has build agents and you can add and connect any number of build agents to your server these agents can be installed anywhere on an ec2 server on your own on-premise servers it could be a docker container etc and each agent can have different tools

installed on them and for each build teamcity will automatically detect what build agent can execute that build based on the steps configured in that build for example if your build is using npm command to run javascript tests for example you will need a build agent with npm installed on it to be able to run that build if your build creates darker images from your application and pushes them to a docker registry obviously docker will be needed to run these steps right so teamcity will automatically detect a build agent with docker installed on it and execute

the build on that build agent and in the free professional version of teamcity you can connect up to three build agents to your teamcityserver now let's see how teamcity works in action first we will install teamcityserver and then configure a continuous integration pipeline for a simple java gradle project that builds a docker image and pushes it to a private docker hub registry and once we have configured that we will connect build agents to the teamcity server so that we can run our build so let's do it so the first step is to install and set

up a teamcd server so the main component basically where we can create and configure our builds and on the official website of teamcity you can see different installers for different operating systems plus we have an option to install teamcityserver as a docker container with this command right here and as a managed teamcity alternative you can use teamcitycloud so for those who want to avoid setting up and configuring teamcityserver and agents themselves you can use team city cloud instead to get started faster and more easily however in this video i'm going to show you how to

set up teamcity using docker so basically we have this teamcityserver image from jetbrains which has most of the stuff already pre-configured inside so we're just going to start this with simple docker run command and we're going to have teamcityserver available and i'm going to do this on a remote server on digitalocean's droplet server which i just created and the only thing i have done on this server is basically just do ept upgrade and install docker right other than that it's completely fresh new server so let's execute the docker run command and let's also see all

the options that we have to add to this command so i'm gonna do a new line here and we have a couple of volumes that we want to configure for teamcity because all the configuration that we do the builds that we create etc they need to be stored somewhere right and we want them to be persistent so we're configuring volumes for that so i'm going to call this team city data these are named volumes and these are going to be mounted into the data team city server data directory then we have another volume for the

team city logs so let's call that teamcitylocks so know that these are basically just named volumes in docker you can call that whatever you want docker will create this physical location on the host automatically and these are the path inside the team cd server container so these two volumes are needed and then we have the usual port basically that we bind on the host here you see the teamcity server is running on port 8111 and that's the port that we take for host binding and finally we have the image name right so let's do a

new line and we want to run all that in background so detached mode and the name of the image and i'm going to execute this and teamcityserver image will be downloaded the latest image and in a couple of seconds basically we will have the teamcity server available awesome so if we do docker ps we see teamcity is running and available on this port and i have already allowed the firewall rule for that port on my droplet that means that using the public ip address and port 8111 we can now access teamcity and on the initialization

setup step you basically have an option to choose which database you want to use to persist the data for a test version you have this internal database that you can basically very easily get started with so you don't need an external database for it so we're going to choose that and proceed so now that teamcity is initialized and ready we can create our first project and connect it to a git repository so uncreate project and for this project i'm going to use my java gradle application which has a docker file so we're going to build

docker image and push that image to a private docker registry you can find the repository link and all the other relevant links in the description of the video so i'm gonna copy the url of my project and here we're going to need credentials of git repository itself and proceed so the connection to repository was successful here you can set the project name default branch and some other configuration options we're going to leave all that at defaults and proceed and here as you see teamcity detected automatically from the project which build steps we would need to

execute right since it's a gradle project we have the gradle build task and we have docker tasks but to show you how you can actually configure that manually i'm just going to click here our build configuration is going to be simple we're going to do gradle build first because we want to create a jar file from our java application i'm going to scroll down here to gradle and once selected we have options for that specific runner type right so for gradle we have all these options so we can call the step build application or build

jar and here we have the gradle tasks that we want to execute let's say i want to do clean with gradle first and then we do build so these two tasks will be executed you can just separate them like this we have a gradle wrapper in the project so we can choose this and everything else can stay the same and let's do save and this is how build step list basically looks like so we have build application and the next step will be to build docker image right so i'm going to choose docker runner and

as you see for docker you have docker specific configuration let's call this build image the build is selected as default we're going to use docker file as a source and to select the path to file we can actually click on this icon here which will show us the files in our git repository right so this is the same structure as we have here and from here we can basically select the path docker file here is in root so i'm just going to click on it and that's the path to file and right here we need

to specify the name and the tag that docker build will basically use and as you probably already know from my other docker videos when we want to push a docker image to a docker repository we need to tag that image using the repository name right so in my case i'm gonna push that docker image into my private docker hub repository and that's the name of the repository so i'm going to copy that and then we have the tag option and let's say we want to take our application as my app and some version let's say

9.0 and it's also common practice to append a build number as part of the tag name and this build number build count etc values are usually available in a cicd tool using build parameters and in the build steps you have this convenient way of looking up all the build parameters that teamcity provides you with so if you want to append build number here i'm just going to click on it and this will basically append a build number at the end of the tag we can also build multiple images with different tags in this step by

just providing them on the next line so in addition to this specific version we're also going to build an image with just latest tag so every last image that we build is going to be latest as well and we can define it like this and let's save this step as well so we have build application jar file build image and now we can create push image step with docker runner we're going to choose push here and these are the two images that we are pushing and if you notice in every build step you have this

advanced option which basically lets you choose a conditional for when the build step can run right so you can configure whether this build step should run only if the previous ones were successful etc and we're going to save this step as well and this will give us a build in the project that we created with these three steps now docker repository as i told you is a private one right so how does teamcity know how to connect to the docker repository to basically push the images into it right usually we need to do something like

docker login to authenticate with the private dock registry for this type of connections basically from teamcity to other platforms or other servers we have connection configuration on our project so if i go to my java app project on the project settings we have connections option here so all of the connections to this external systems that you need in your project you can basically configure all of them here so in our case we're going to choose docker registry which is docker hub this is the default registry address which is docker hub this is what we're using

and here we are going to need the credentials that's my username and the password and we can test the connection as you see connection was successful so i'm going to save it and now once that's configured in the project we can use it in all the builds of this project for that going back to the build settings in build features we're going to configure something called docker support and add registry connection and we're just going to choose docker hub here save and we're good to go and as you see here this will basically log into

docker registry before the build gets executed right which is exactly what we need because we need the authentication before docker push command gets executed and with this configuration our build is configured and ready to run but to run a build we need an agent so this is teamcityserver and we need agents to actually execute the bills and right now we have zero agents because we haven't connected any to our server so as the next step we're gonna connect an agent to our teamcityserver so how can you deploy and set up team city agents you have

the same options here as for the team city server you can install it directly on your operating system run as a docker container or again if you want to avoid manually setting up and configuring agents you can use team city cloud which will automatically make all the suitable agents available for you i'm going to use the docker image here as well for a fast local setup by executing the following docker command again configuring the volume for configuration data and specifying where the agent should connect to to the tmcd server using server url environment variable so

let's run this and now if we go back to our teamcityserver ui and agents we still have zero agents but we have one agent which is unauthorized so this is the agent that we just deployed but not authorized just click on the authorize and now we have one connected agent so that's basically how you can connect agents to a server as a docker container now as i already mentioned teamcity will automatically detect which agents are actually able to run which builds based on what's configured on them right which tools are installed on them so let's

actually see how that detection looks like so i'm going to go to the build and here we have this compatible agents tab right and it says compatible agents is zero or none and we have incompatible agent so one agent deployed which doesn't meet the criteria for running this build and teamcityserver actually detects that by checking whether docker.server.version is available on that agent and it is not because we deployed it as a docker container but there is no docker available inside it right so basically it's not suitable to run our pipeline because we need to run

docker commands for our build so now let's run and connect an agent which is going to be compatible to run our build and to do that we're going to run another teamcityagent container but we're going to mount the docker from the host inside the container using the following docker commit you can find the detailed explanation of all the command options you see here and how docker is mounted from the host in the video description along with all the relevant links so check out the description for that and let's run it so our second team city

agent is running and now if we go back to our team city ui you see that we have another unauthorized agent so let's authorize that as well and this may need some time but if i refresh this now you see that we have one compatible agent and we have one that doesn't meet the requirement so that means that now we can actually execute our build and to do that we have this run command here with some additional options so for example you can pass in some parameters if you want you can select the agent so

for example if you have multiple compatible agents you can select which one you want to run this on etc we're just going to run the build so here we see the overview of the build the status branch etc and here you can also see the locks of the build so we see that all these steps get executed and we should be able to see our latest tag as well as tag with the specified version and with build number in the docker hub private repository and here you also have a summary of the images that were

pushed to the repository now of course in your projects you want to configure automatic triggering of builds when you push the changes or a pull request is made and also configure which branches should be built etc and you can configure all of that pretty easily with the triggers in the build configuration where you have all these options available for different use cases and for different trigger types and finally as i mentioned with the best practice and the new trend of having configuration as code instead of doing everything in the ui you also have an option

to see all of these and save this configuration as code and in the build configuration we have this view dsl button and if i click inside you get this auto generated build configuration code which basically describes exactly what we did in the ui with the build steps and triggers etc written in kotlin programming language and optionally you could also work with this file which you can use as a basis to basically expand on your configuration and configure the rest of this stuff as code now this is obviously a subset of the things you can do

in teamcity because you have tons of other powerful features in addition but this is to demonstrate how easy it is to get started with teamcity to set it up to connect the agents and basically just configure a simple pipeline for your project now i hope you learned a lot in this video let me know in the comments what you think about teamcity and what's your experience with other cicd tools and with that thank you for watching and see you in the next video