Learn ROS 2: Beginner to Advanced Course (Concepts and Code)

ready to learn Ross 2 and take your robotic skills to the next level in this Ross course I will cover beginner to advance topics not only will I cover the important Ross 2 Concepts but I will also show you step by step how to implement the code we will start off with beginner Ross topics such as how to install Ross 2 executables topics Services parameters and action workspace Co can build creating your own packages publisher and subscribers and finally creating your own la file later we'll go into more advanced topics such as urdf for robot

description Arvis simulation gazebo robot simulation Ross 2 control camera liar and depth camera sensors in gazebo simulation talk about the slam toolbox and end it with some robot navigation so by the end of this Ross 2 course you should have a solid understanding of Ross 2 Concepts and be ready for real world robotics problems all right let's get started [Music] if you're new to my channel I have a website at kevinwood robotics.com where I have a bunch of resources on Robotics and computer vision so check it out and subscribe to learn more so first I'd

like to show you how to set up WSL so that you can use Linux on your Windows PC this step is optional it's for people that don't have a Linux machine or want to run Linux on your Windows PC if you already you have a Linux PC set up you could go ahead and jump to the next section so most of the steps I got will be based off of this website but I just simplified it to the main key steps that I need to use to install it first off we um we go to

the website right we I use that for reference but you don't need to just follow these steps here so you want to open up a Powershell so you want to do run as admin so after you run as admin you're going to open up this window here so what you want to do is run the WSL install and then after you do that restart your PC and then come back to these steps then to see what uh distributions you have you run WSL d-list and online you're going to see these are all the distributions so

we will be using um this one right here you should be able to see um your Ubuntu here okay so that's how you do uh the setup for WSL and then next up you want to install your extension so if you type in WSL go ahead and install your extension here and then you want to open up WSL and vs code like I have here what you want to do is run Ubuntu and then just type code dot so if you do code dot you'll see this uh window here pop up then you should get

up and running like I do so before we can do anything in Ross we need to make sure we have Ross installed so the specific version that we're going with today is the humble version the reason I chose humble is because the release date was from 2022 so it's been around for a while to be pretty robust and the EOL date the end of life date is in May 2027 so we still have about 2 years left so that's pretty good you can see that the next best one if you're looking for an alternative would

be Jazzy which has the end of life date of May 2029 so without further Ado let's jump right into the installation process so you want to first run these commands to set up your local we're using a utf8 and then after that you want to do some software properties and universe repo um install then you want to set up your um Crow install and your Ros key setup here then after you do that you want to Source your configur ation then do a update and upgrade then what I'm doing here this is the main installation

step is a um doing the full desktop install so just run that and then the next step is what you want to do here is if you run this you want to uh Source it so this will allow the Ross two command to work so when you hit Ross two you should see U something pop up here showing that your Ross 2 has been installed properly so in order order for you to use Ross 2 commands you have to do what's called sourcing your Ross 2 so the point of sourcing is so that you let

your computer know where the Ross 2 installation is located and that way you can use the Ross 2 commands if you don't do this step you're going to have to either manually Source rosu every time or you could do what I'm going to about to show you which is setting it up in some of your configuration file so you're going to see this thing that says Ross 2 command not found so so you typically will see this if you forgot to Source uh your setup. bash file so the best way around this is to set

it up automatically so you could put it in your bash SRC file so this will allow your Ross 2 command to work every time you could run it the command Way by running this or for me if you're in vs code and you like to do things the gooey way what you can do is uh go to your uh bash SRC here and you could just add in uh the step opt Ross humble setup. bash so that every time you open a new terminal it will set up um everything for you so when you want

to run a program in Ross you typically run what's called an executable so you could think of an executable as the program that you're running and these programs have been typically built by some source code using culen which are some tools we'll talk about later but for now just treat executables as programs and Ross and and they're located in what's called packages so packages are something that holds a bunch of code and other instructions on how to build it or any other configuration files but for now what you need to know is that executables belong

in certain packages and to run a program you would run the executable so I'm going to show you step by step how to actually use the commands right now so first off we're going to use the turtle Sim so you want to run these two steps to update and install the turtle Sim package so after you do that you could list out your package by doing Ros to package list so you can see these are the lists of the packages available so um if we look here we can see that our Turtle Sim package exists

and we can also see the list um we can see all the executables so you can see Ross 2 uh package executables so here you can see that um these are the things Turtles s offers okay so that's for the executables and we could do um specifically just for the turtle Sim package we could do Ross two package uh executables and just type in the package Turtle Sim and you can hit tab to complete these things so you can see for turtle Sim there's only four so this isolates the list that we're looking at okay

so where where is this Turtle Sim um package so uh what we could do is check out um go CD opt and then we have Ross humble and we hit enter hit LS we can see that it's actually going to be somewhere inside here um but let's open up a code a new terminal here so here I open up the terminal and if I hit contrl p and I type in Turtle Sim you're going to see that it's in here okay so this is executable it's going to be in share a m index you can

see is in this path here so if I scroll up you're going to see aent index resource um resource index packages so that's where you can locate it and it's all under the the share folder here okay so that's that's what we could do if you want to find where some of the uh packages are located it okay so after you do that um maybe we want to run the package so if we use a run command is going to have the package name followed by the executable name so I'm going to do Ross to

run and you can always do Ross 2-h to see the help okay so you Ross to run and then the package name was Turtle Sim hit tab to complete and then we're going to do the turtle turtle Sim node so if we run this we should see the turtle M node running okay so this is our Turtle right here and then the next up we could use our T op key so going to slide this window over a little bit I'm going to close this so just going to rearrange a little bit and you can

see um if I run another node called let's see I'm going to say Ross 2 run Turtle Sim uh the next node is called uh Turtle teleop so this will allow me to move the turtle so let me just slide this over a little bit more so you can see um but you can see if I use the arrow keys and the forward you can see my turtle is moving so the way Ross works is they communicate between what's called nodes so these nodes are typically responsible for sending or receiving data and you could do

this through different ways like top services or actions I'm going to be going over the different types later on but in this video we'll be focusing on some of the commands you could do to use your nodes you type Ross to node DH you'll see the help for the Ross to node and then we could do Ross to node list you can see is one of the main function Ross to node list you can see that if I run this there's no uh nodes running right now right so let's go ahead and run a node

first so we do Ross to run uh Turtle Sim Turtle Sim node so we have our Turtle node up and running here so we have once you do that and we do Ross to node list we will see that Turtle Sim is a node that's running okay so we could do different things to see the info so if we do Ross to node info and we type in the node we could see different things about this node so what this shows is there's a subscribers there's a Publishers there's a service uh servers and then as

well as the clients action servers and action clients and those we will talk about later on in this video but for now you know that U by using a Ros to node info command we could find out more info about a certain node that we're dealing with so previously we looked at nodes now we're going to dive deeper into topics so topics is the thing that connects two noes together so you can see on the left we have a publisher node and on the right we have a subscriber node so the idea is a publisher

node is going to send data to a topic and the subscriber node is going to subscribe to that topic to receive that data so in this video we're going to be covering some of the commands you can use for Ross topics so first first off let's start two of our nodes so we'll have Ross to run uh Turtle Sim and then Turtle Sim node so this is our Turtle Sim node up and running and then we'll go ahead and run another one that's Ross to run Turtle Sim and then Turtle teleop okay so this will

allow us to control it as you can see we're moving our Turtle now what we can do is we could list out the topics so we could do Ross two topic list and this will show all the topics and then if we add a dasht to this we could see the type associated with the topic okay so in this other window we could go ahead and open the rqt graph so you can see this is our uh node and if we refresh uh this is everything that's talking so you can see between Turtle Sim and

turtle teleop turtle um the command velocity is being passed through here's some action that we'll talk about later on but for now we'll focus on this right here so you can see our Command velocity is one of the things listed so if we want to see the specific topic output we could have another terminal and then we could say Ross to topic and then we could do Echo and the topic name so in this case is turtle one and then the command velocity so what this means is as I move the turtle it's going to

Output the topic okay so that's what this does and we could view the info by using the Ross to topic info and we could pass in the uh topic name which is command velocity so we can see that subscription count is two publisher count as one and the type is geometry messages message twist so another way we could see the interface definition is we could run this Ross to interface show and then we have our geometry messages and then as we saw here it's going to be the twist so if I enter this you're going

to see uh what this takes in is a vector 3 linear of uh XYZ all floats and the angular part is also uh floats float 64 for the XYZ okay and we can also uh try publishing the data at a specific rate so it's going to follow the Ross it's going to have the form of Ross 2 topic Pub and then the once just means that it's going to do it once in the exit and we're going to pass in the topic name the message type and the argument which here is going to be the

command so if I copy this and I run this okay so Ross two you can see it it just drew and if I do it again you can see true again okay so that's what this does and notice that it's if we look at the whole command again we have the Ross to topic Pub once the topic name and then the type and then the message so we saw that it's going to be XYZ for the linear and then XYZ for the angular okay and we also publish the data at a specific rate so uh

this right here is top Ross to topic pub with rates at one Hertz and then again we have the topic name the type and then the message or the the goal okay so if I run this you can see it's going to just keep going in a circle because it's at a certain frequency okay so I'm going to go ahead and stop this now what we could do is Ross to uh Echo and we can see the post here here so actually let me just run this again so here let's just go ahead and open

up another one so here I'm going to have a Ross 2 top pick Echo and then the turtle pose you can see that it's outputting the pose as this turtle is moving okay and if we come back here and refresh this you see that we added the pose here okay because now it's outputting and we can also view the topic frequency here so um going to go ahead and stop this if I do Ross 2 topic uh Hertz and the topic name turtle one uh pose well now now the pose you can see that um

it shows us the average rate average rate min max standard deviation and the window okay so this will tell us the frequency so previously we were looking at how publish sure and subscribers would communicate between topics with topics what we notice is that there's a stream of data that's always continuously being published or subscribed with Services it's a little bit different there's no concept of continuously publishing instead what happens is you have a server and client relationship where a client is going to request some data and then the server needs to respond before the client

can receive the data so let's go ahead and take a look at some of the commands for services so let's go ahead and start off by uh running two of our nodes so Ross 2 run Turtle Sim and then Turtle Sim node and then in our another one we're going to run our tele op so we're going to have Ross to run uh Turtle Sim Turtle teleop key okay so as you can see our turtle is moving so here we're GNA do uh we're going to list out their services so we do Ross to Service

uh list you can see these are all the services so under the turtle one we have all of this stuff and then all under teleop Turtle we have all of these things as well so we can see the type associated with by adding a-t so this give us more details um if we need to use that information later on so we also find the service with a specific type so we could do Ross to find and then you could pass in the type so for example if I want to do this one here um STD

and then Sr let me just copy that so you can see rosu Server fine it'll tell us that um clear and reset will have these this type here so you can see if I uh look through this let me just drag this a little bit but you can see that clear has this type uh reset has this type as we expected based off of the output okay and we can also see the interface so we could do uh Ross to interface show and then the service type so we could do turtle Sim SRV and then

let's say uh spawn so you can see for spawn uh this is the output you have a float XY and Theta and then you have the string name which is optional okay so now that we know how to uh the details of that we could look at the calling it so for calling it let's go to a new window and if we do Ross to uh service call and then let's say we want to clear and then we need to pass in the type so STD Service uh mty you can see that it cleared the

drawing that we had here okay and then now if we want to spawn a new turtle we could do Ross 2 service call and then spawn and we have our Turtle you can see that um here let's actually change the location so or if we drag this you should see it you can see our Turtle got spawned over there okay or let's just uh choose this location so you can see it more easily you can see our turtle cost Bond over here okay so for that one you can see the out the function argument is

service call and then your service type and then here is a format of the the specifications for what you want it based off of this interface okay now let's talk about parameters sometimes when you want to input data into a node it's convenient to pass pass parameters into your node especially if you have different configurations for different type of robots you're working with this is very good to have different inputs to your nodes it's kind of like having inputs to a function but in this case our function is a node so let's see some of

the commands we can do to work with parameters in Ross so let's start off by running two nodes so we're going to do Ross 2 run the turtle Sim and then Turtle Sim node and then in our other one we're going to run the tele op so let me move this to the side so we're going to say Ross to run Turtle Sim and then Turtle teleop key so you can see our uh turtle is moving and then next up we're going to list out our parameters using Ross 2 param list so this is the

list of our parameters and you can see the turtle Sim one we have these parameters so we'll play around with the background a little bit so we could actually get the parameter value so to get that we could do Ross 2 param get pass in the node name which is Turtle Sim and then the parameter so background and then background G so you can see it returns 86 we can also do something similar but this time as you set you pass in an extra argument which is a value so Ross 2 Pam set pass in

the uh node name and then the parameter background G and we'll pass in the value of 255 and you can see the background has now changed okay so you could also view all parameters for a node by using the dump function so you could do Ross 2 param dump and then pass in the node name Turtle Sim and you can see these are all the current values that are set for the backgrounds you can also store all of these inside a yamamo file so you could do Ross to Pam dump and we'll pass in the

node name and then the our file is going to be turtles sim. yaml so all of our parameters will go inside this yaml file and if we look inside here we could see these are all of our parameters that we have stored okay so the beauty of this is you could load in your param file so you could do Ross to Pam load and then pass in the node name and then the file which is your turtles sim. yaml and this will load in to to whatever setting you had but since we're already blue you

didn't see a change but another thing you could do is load the pram on Startup so you could actually do Ross to run and you're going to pass in the package name and then the executable name which is Turtle node and then all these additional arguments here so you have um after turt turttle Sim node you're going to see turtle turtle Sim node and then Das d ross D args and you're going to do prams file D- params Das file and then Turtle sim. yaml so this should open up a new window with the light

blue background so now we're going to look at Ross actions this Ross actions uses the same server and client model but with a Twist so the thing that we're adding to it is it's going to have a goal feedback and results so we're going to be looking at the different commands and Ross that we can use with actions first let's uh run two nodes so Ross to run Turtle Sim Turtle Sim node and then in the other one we're going to run our tele op so let me slide this over so we have um Ross

to run Turtle Sim Turtle teleop key so you can see our turtle is moving and then in the teleop window what we're going to do is observe the following so if we press uh G you can see that it's rotating and you can see here it's going to tell us the the status so uh if we press one of the letters and cancel early we're going to see something else happen so so if I press F here it says rotation got cancelled right so I do it again pretty much anytime you press a key and

then you press f it is going to say goal cancelled and then if I try toggling between uh two commands here you're going to see that uh rotation goal receed before a previous goal finish AB boarding previous goal so it's always going to take the latest goal command so that's that's one of the beauties of actions so we want to see the action clent in node info on the bottom so if we do Ross 2 node info and we're going to pass in teleop Turtle so we could see all the info for this node here

and you could see specifically in the actions there is this uh absolute the RO rotate absolutes that we've been monitoring okay and we could do a Ross to action list this will show all the actions and you could see- t for the type associated with it okay and we can also see the action info we could do Ross to action info and then pass in the action name so we have turtle one rotate absolute so this will tell us the action action clients action servers so you can see the client is the teleop and the

server is the turtle Sim so we can see the data type for the action by using the interface so we do Ross to interface show and then pass in Turtle Sim action rotate absolute we could see that it communicates by using uh float 32 of theta okay and then that's desired and you have the angular displacement in radians is also uh it's going to be a Delta okay so we could go ahead and now send action goals following this structure called Ros two action send goal and then you pass in the action name action type

and the goal okay so if I copy this line here and if I run this here you see I have Ross 2 action send goal the turtle one rotate absolute and a turtle Sim action rotate absolute okay so if I go ahead and run this you're going to see that um it moved or actually it was sort somewhere else so if I uh make it go elsewhere like d for example and then I run this again you're going to see a rotate facing up so up until now we have just been using Ross code that

was provided by Ross we haven't really created any of our own packages yet but now we are going to start doing that but before you can do that you have to understand a concept in Ross which is called the Ross workspace so the idea of the Ross workspace is pretty simple it's a way that you organize your folder structure so that you can place your packages this in your Source folder so the typical process will look like this you're going to have a folder which is your workspace so you can think of folder and workspace

as the same thing you want to give it a name that's representative of your workspace in this case because it's a demo we're calling it Ross 2 workspace you can have more than one workspace at the same level if you wanted to but the main idea is that the Ross 2 workspace is going to hold the different files that we want to build and run that contains our source file and install files so the main part that contains our packages is going to be under the source folder so this Source folder is going to be

a folder that you need to create after you've created your workspace folder so you create rosty workspace folder and inside you create the source folder once you do that you're going to set up a package which we're going to go over in a later video but the main idea is that when you create a workspace after you create the source folder and you do some building with culen which we'll talk about the folders here the build install and log folders these folders here are going to be generated and inside of these folders are going to

contain executables and other information that allow you to run your executable so now we're going to talk about the culen build tool so culen is a command line tool that you can use to build your packages so previously we've been using using only code that was already pre-made but now we may want to make changes to the code or even write our own code and build our executable files from our source files so that's where culen comes in and just as a reminder we have what's called packages so the packages will typically be inside the

source folder that's under your workspace so here is the Gazebo tutorial this is a name of a package I'll be covering gazebo later on but you'll see that inside of the Gazebo tutorial package you're going to have more folders so you may have a config folder include folder a launch folder a meshes folder but the main part that all packages have is it's going to have a source folder which is going to be your source code and you're going to have the cmakelist.txt which is going to help with some of the build process and you

have the package.xml which will support the build process as well okay so now let's go ahead and take a look at some of the commands we can use with culin to build our packages so first off you want to run this command here to install your culin I've already done that and then you can make your workspace and navigate to your workspace so go ahead and do that I already made my workspace so I will just navigate to my workspace and then what you want to do is you want to add some source code to

your folder so you're going to get clone from here some Ross tutorials so I've already done the step but just do that and then what you want to do next is set up the culen tab completion so you could do Echo and then insert this autocomplete command here and then this will get your culen TAP completion up and running so here we're going to do the build uh step so what we're going to do is run culen build Sim link install so here is it allows us to install and the Sim link what that does

is it's going to prevent copying the files and just make a Sim link to your files okay so let's go ahead and build our uh package so we do culen build and then D- Sim okay so I just hit Tab and the auto complete is working so this will go ahead and build it okay so the build took 21 seconds and then after it's done building what we want to do is do LS and we can see the contents of this folder so you can see that we have a build install and lo folder the

next step is to Source your folder so we want to run this command uh Source install local setup. bash and this will allow us to run some of the packages that we just uh built so here we do Ross to run Turtle Sim and then tur Turtle Sim node so this will be our Turtle Sim node up and running and you may think that this might be the same one as the original Turtle Sim so let's do some modification so you can see that it's actually different okay so let's go ahead and find our um

file that has the name of our Turtle Sim so inside our Turtle frame you could hit contrl p and search for the turtle frame file and then after you do that you could come in here and we could just add in my name Kevin so what this will do is if we go back to our read okay so here we edited that and then we just want to read do our build step so I'm just going to go ahead and uh copy this again so I'm going to rebuild it and then Source it and after

it's done building we could go ahead and run our Turtle Sim node again so now that I'm have this up and running you notice that the name has changed to Kevin so that proves that we are editing our local Turtle Sim so we just saw how to build packages with culen now we're going to talk about how to create your own package so this is good if you have your own custom code that you want to run in Ross and that's when you will want to create your own package and build it using the steps

we just described earlier so now let's go ahead and take a look at the commands that you need to use to create your own package in Ross so the prere is you want to install Ken and create your workspace which we did in our previous video so do that if you haven't done that already and then to create our package what we're going to do is navigate to our source so CD source so we're in Ross to workspace source and then you want to run the create package command here so uh this command is Ross

to package create-- build type A me cake and then the node name the name of your uh node and then the name of your package so if I go ahead and run this it's going to create my package and we could go ahead and check out the folder that it made so if I go to sour my package you see that um inside my source it created my node. CPP so if you look here this is just um my CPP file that that says hello world essentially so you could see this and then you have

my cmake list these are all the build instructions and then the package.xml is all the meta information like your email um version and whatnot so that's the essential um basics of of your package okay so here we're going to go ahead and run um we we just built our we just created our package and now we want to build it so here we want to CD uh Ross workspace and then now we're going to do culen build- das package uh packages select and then do my package so this will build our package so this is

a short package so it's going to be be pretty fast and then now we're going to do Source install our local. setup. bash and then I can go ahead and do Ross to run my package and then my node so you can see it says hello world okay so that's the um program running and we got everything up and running so that's good so in the earlier part of this video we discussed how to use publisher and subscribers now we're interested in creating and building our own publisher and subscriber package using culen in this specific

part of the video we're going to be showing a C++ version the one right after this will be a python version so let's go ahead and see how to create it in VSS code but first off um we're going to be using cmake in here so if you're using VSS code you want to install this cmake extension and first off we want to create the package we want to run the command to create the package so you want to go to your Source folder in your workspace and then run this command here Ros to package

create build type aend cmake CPP Pub sub that's going to be the name of my package so I hit enter that will create my package and then you can see here it creates a folder and this is a folder that's my package which has the include the source the cake and the package uh. XML file okay so we're going to take a look at the source code that we'll be using for this tutorial today so um there's going to be two main functions that we're playing with so the first one is this right here this

is the uh publisher function it's going to be the publisher member function. CPP so you could get it in my repo but these here are the includes that you need to include so this is for the timing and the memory stuff uh strings this is the Ross related thing so this is the basic inclues that you need for a typical publisher uh function and here we have some resources on C++ classes and inheritance if you need some uh refresher but here what we're doing is we're inhering uh the node class to make our minimal publisher

class and then inside of here of our Constructor we're um initializing these values here and we're going to be creating a publisher here here and this publisher is actually a shared pointer so I have some videos on shared pointers as well you could go check that out it's going to be under the smart pointers but you can see here in the private we have defined these um smart pointers here uh shared pointer specifically for the timer and publisher so when we uh create this publisher we're saving it to this publisher sharp pointer and similarly here

we create a wall timer and we're storing it in the timer share pointer so this right here is going to be um it's going to be doing a call back you pass in the Callback function so our callback function here is the timer callback and what this will do is it's going to spit out the message Holo world and then publishing uh the message here that it's going to be updating and you see it's based on this data. C string but here the Holo World here it has a count count Plus+ so we have a

variable here that we have and we're just incrementing at every iteration okay so that's what this does and then here is the main Loop a lot of these functions will follow a similar structure so here we'll do some initialization and then the spin so this will create an instance of the minimal publisher and enter Loop and keeps a note alive with the callbacks up and running and after you're done we do a shutdown and that's the gist of it here's a subscriber member function. CPP so this one once you publish someone has to subscribe to

it so here again we do inheritance and then we have our Constructor to create our minimal subscriber and inside the subscriber we create a subscription this right here is a Shar pointer as you can see and this will do a topic and then this will have our call back so in our call back all it does is respond to what it's getting back so so it'll say I heard message. dat and whatever the message is you will get that response back and in the main function we have a Knit Spin and shutdown which is similar

to our publisher structure that we just talked about okay so what we want to do is get those two functions we want to extract those two functions and we want to put it in our source folder so again these are some resources you guys could check it out but what I'm going to do is just copy those two files that we just one over and then go right into here to our source file in our CPP Pub sub in our source uh we're going to paste that in there so those two functions are in there

now and then the next step is we want to update our package.xml and our cmak list.txt files so if we take a look at that we see this is our cmake file here so what we see is that the cake file the general structure is it's going to make make the project have some cmake minimum requirement and then the part that we're actually adding is this chunk of code right here so when you create a cmake list by default um when it's created during the package generation it's going to look something like this and then

the cmake list that we're going over is what we're adding onto it so the things that we're adding on to it is we're going to add some lines to find our packages that we're using and then for the publisher what we want to do is add an executable so it's going to be called the talker and then it's going to tie it to our uh publisher member function. CPP and then this will do the dependency linking for the talker and the things that it'll use similarly for the subscriber we'll do the similar structure and then

the installs here will um set up our targets for running and then this right here is for some copyright and Licensing that we won't really care about out and then this is the final step to make the package and then for the package.xml the part that we're adding to the uh originally generated file is this part here that depend on the RCL CPP and the STD messages okay so I'm going to go ahead and copy those two files and I'm just going to delete these two and we're going to update with our new file here

so if I paste it these are our updated files that we'll be using so if we go back to our um read me we're going to do now that we have updated our files um you could run this step here to check for missing dependencies this one I would say is optional um but you want to First go back a folder and then run this so this will do any dependency check so the final step is you want to build it using culen so I'm going to run the culen build package select and then the

name of the package this is CPP Pub sub so if I go ahead and run this this will build my package and it'll take a few seconds to build but once it's done building we could start using the package okay so when we do after some building we would create another terminal and then we would CD to Ross uh 2ws the next the important important step is to Source it so that we could find the talker file and then we could run Ross to run CPP pups sub talker so if I go ahead and run

these commands what this will do is start publishing and then in my other terminal I could use the listener so here in my bottom terminal I could CD source and then run the listener and you can see that it's getting the feedback of what is hearing okay so now we're going to see how we can create our own publisher and subscriber Ross package in Python this time it's going to be pretty similar but there's going to be a few differences so pay attention to the differences if you're trying to implement this in p Pyon we're

going to create a package from our Ross to workspace Source folder so we want to CD into that and then run the Ross to package creates command to create our package called P pup sub so if I go ahead and run this it's going to create my package and you can see in my folder here we have a folder called pip pup sub and by default it's going to have a pip pup sub folder with a nitpy function here and then a resource folder a test folder a package.xml a setup. CFG and then a setup.py

okay so we're going to look into some of the source files that we'll need so we have a publisher member function here and this is what we'll do the publishing so I have some resources if you need some refresher on python classes and inheritance here but here what we have is we have a minimal publisher that inherits from a node and we have a Constructor here so here in the Constructor we initialize with the name of the publisher called minimal publisher and then we have a self. publisher uh variable that we're storing as a member

variable after we create the publisher with our string the topic and 10 so we pass that in save it and then we have a time period of 0.5 and then the timer we're going to create a timer and then store that timer and then we have a counter variable I that starts at zero and we have our callback function here that's going to get our message as a string data is going to be hello world and then it's going to spit out the number that we're um retrieving and then here we have the self out

publisher publish is going to be publishing the message and then here is going to be what is going to Output what is publishing and inside of a main function we have the similar structure as we mentioned in our C++ video but here we have a nit and then we instantiate the minimal publisher and then spin will'll get it up and running and then when you're done it'll do some destroy and shut down okay so here here's a sub uh here's a subscriber um we have the subscriber function here or the subscriber file so here what

we see is the minimal subscriber we're going to inherit the node again we create a Constructor with the name of our subscriber and then we have we we're going to create a subscription with string topic and then a call back so here our call back is going to be I heard some message and then inside of our main function we're going to call the NIT call our minimal subscriber function or class we're going to create an instance of it and then spin destroy and then shut down okay so that's the general structure of the publisher

and subscriber functions so next up is we want to update some of our files specifically we have our package.xml and our setup.py so let's take a look at what changes we made so here is our package.xml file as we see by default it'll generate these files but what we're going to do is add these two lines here the RCL pi and then the STD messages so that we can use some of the Ross functionalities and then in the setup file what we're going to do is add these two files here so um or these two

lines here so inside the entry point we have a talker so this will set up our pup sub and then our publisher member function and then then our subscriber member function and this will allow us to use those functions that we created so back to my readme um let's go ahead and uh make those updates so we saw the things that we have now we want to move it over so we're going to copy these two files here which is our source files I'm going to copy it we're going to our piip Pub um folder

here and then we're going to paste it in here so this is those our functions and then we have our package and setup.py we're going to copy those U we're going to go ahead and just remove uh those two files since we're changing it and then we'll go ahead and update that so if we go back to here we're going to paste it we have our new updated package and setup files so now that we have our updated files what we're going to do is uh we're going to do the Ross this right here is

to check for missing dependencies which I would say is optional um but you want to CD back to your previous folder and then run this so everything is up to dat and then we're going to build with culen from Ross 2 workspace so I'm going to copy this culen build package select Pi PB sub this is going to build our package and after the package is done building what we want to do is create another terminal and then run the talker from our Ros to workspace so before you do that you need to Source it

so that the talker command will show up so if I go ahead and run this this will start my talker and then in the other terminal we want to start our listener so to start our listener we're going to run the similar command but now with the listener so here we have our listener and you can see that it says I heard hello world and the number that it's publishing so now I want to talk about launch files so up until now we have been writing our commands in the terminal when we want to execute

something so for example if we're launching a publisher or subscriber we would do it in the terminal and write the commands there sometimes if you have a lot of things you want to start for your program it could be pretty tedious to write it line by line in different terminals so that's where launch Files come in launch file allows you to automate this in a python script so that you can call the different commands just by writing it inside of your generate launch description function that you can see here so each launch file is going

to be different Case by case you may have a different one for different packages and you could customize this based on the specific nodes that you want to run so this is just a example of a launch file which I'll be using later in this video but here you can see that um I'm running a launch file here so you could have a launch file that calls a launch file so this is what this is doing you could create a robot description again it's a topic we'll talk about later on but here you can see

that you could launch a robot State publisher node by using these commands here so you would call node set the package executable name and if there's any parameters you could pass parameters inside of your node as well so here you can see we have a joint State publisher node so there's a bunch of noes that you could set up here you could also call the execute process this allows you to run a Ross terminal command directly from your launch file and after that you're going to see all the way at the bottom the final step

is to turn the launch description with all of the function calls that we just went over earlier so to actually run the launch file what you would do is run the command Ross to launch followed by the package name and the launch file name so in this specific case that we were looking at it would be ross2 launch gazebo tutorial and gazebo do launch. so you're going to notice that the typical structure of launch file names is going to be something. launch dopy so just go ahead and follow that convention okay okay so now I

want to talk about urdf files so when you have a robot whether it's in simulation or in real life you need a way to tell your program how this robot is described so we do this by the urdf and we're going to see some of the conventions and how to actually create one from scratch so let's get started I will go over what is an urdf file the general structure go through the general structure of a link talk about the visual types of geometries used in urdf file files the origin rpy and XYZ material property

Collision inertial a link example talk about types of joints parent and child link origin and axis Dynamics limits a joint example and finally we'll go over a robot example so by the end of this video we will see how we could control R2D2 in Arvis by using the sliders on the right so what is an urdf file so urdf stands for Unified robotics description format and it's axml file used to describe the visual physics and connections of a robot with joints and links so here is a general structure of an urdf file so we typically

will have an XML version on the top we'll have a robot name and then it's going to be broken up into two sections typically so you have a link and a joint so inside of the link we'll have a link name and have the visual collision and inertial properties and inside of the joint we're going to have ACC Parent Link Child Link origin and so on so these this is the main structure that we typically will see and if we take a look at a generic joint it typically looks like this where we have a

joint where has which has a frame and then each of these other components like the visual collision and inertial will have a frame as well so now let's dive into the links in an urdf so a link is used to describe a segment of a robot so you could Define a visual collision and nural properties and the geometry and origin properties are used in the visual and collision properties to describe the shape and location of the link so you could take a look at the general structure of a link so here we've expanded into our

previous structure for the link uh but when you describe a link you could go into details for the different parts some parts are optional and some are required depending on what you have so you can see inside the visual we have a geometry which will have a geometry specific code and then we'll have an origin that's described by rpy and XYZ and then we have a material that will have some uh color that we Define and inside of the Collision is similar from the visual in that we will still have a geometry and then we'll

also have an origin and then in inertial we're going to have something new which here is the mass in kilogram as well as the inertia so the visual part of the link is used to describe the geometry of the link and usually this can be an acurate representation of the robot or just a simplified version depending on the goal that you have so for acurate models you could probably use a 3D modeling software like solid works and you could export the STL file to get the geometry so there's different types of geometries used in urdf

files so typically you're going to see things like boxes spheres cylinder and meshes so for box sphere and cylinder you define it um using the geometry specific code as you can see here so box you use XYZ sphere you use a radius R And the cylinder you have a radius and a length and then for meshes you're going to have a file path to your STL or Dae file and then you have a scale depending on how you want to scale your part so the origin which controls some of the frame locations as well as

the um orientation and position of your parts defined by rpy and XYZ can be quite confusing so let's go ahead and take a look at what that is so the verion is used to define the rotation uh using the RO pitch yaw as well as the translation using the XYZ relative to the Link's coordinate frame so the order of rotation for rpy is zyx so first rotate about X and then about the Y and then finally the Z so the rotation operation is applied first and then the translation so for the visual and collision part

uh Visual and collision the origin describes how the geometry is transformed relative to the links frame so for inertial the origin describes the location of the center of mass relative to the links frame so for the origin typically is defined with the syntax rpy equals rpy and then XYZ equals XYZ and we will see the axis colors usually defined by red green and blue which corresponds to X Y and Z so for the material prop property is used to define the color as well as the transparency of your color using the rbga and it's going

to be normalized from 0 to one so the Syntax for that is you have a material name equals color name you have a color rbga equals to rgba so some common colors you might see would be for example red would be 1 01 green would be 0 1 01 and blue is 011 yellow is 1 1 01 and then finally cyan is 0 111 now if you want to define the Collision the Collision property can be used to describe a bounding shape around the link for Collision detection so usually people use simplified shape for this

for faster calculation when detecting for collision between other Collision bounding shapes so the Syntax for that is you have a collision tag and then inside of here you have a geometry which you will have a geometry specific code and then to define the origin you could set the rpy and XY the inertial tag is used to describe the inertia and the mass of the link so to do that you have the general structure here which you will have mass this mass is in kilograms and then the nura you have the iix uh ixy ixz the

i y y iyz and then izz so let's go ahead and take a look at a simple link example so this simple link example here we have to find the material for blue and then we have a link name called Bas link and inside of here we Define our visual so inside of our visual we have a geometry which we're using a cylinder and we have our origin so our origin we're not moving and then we Define our material here as blue and our Collision we're also defining some geometry that's very similar to the actual

visual part and also the origin we just keep it the same and then here for the ntial we just set as some Mass values and some inertia Matrix so let's go ahead and run this launch uh file which will allow us to visualize our simple link urdf file so you can see here this is just a simple link and you can see that everything is up and running which we expect which is a blue cylinder so the general structure of a joint in a urd file is it's going to have a parent link and a

Child Link so inside of here we have an example of a joint so we have a joint name that has the name of our joint and then the type of joint which we'll talk about later and then here is the parent and child link which we will have a name for it and then we're going to have an origin an axis and limit so then we're going to have a Dynamics for it as well so the limit is going to depend on the type of joint which we'll talk about later and but after you have

everything for this you'll have a complete joint that can connect your links together so the types of joints that you have there's a different few want that you can use so you have a fixed which will have no motion between links you have a continuous which will have a rotation about the axis and there's no limits you have a revolute which is rotation about axis with limits in radians you have a prismatic which has the translation about an axis you have a floating one which will have six degrees of freedom so three translation and three

rotation and finally you have a planer which will have a motion on a plane with two translations and one rotation so the parent and child link here is you're going to have a parent link with the name of your parent and a Child Link with your uh name of your child so this is usually like the order that you define your links so you're going to have everything is going to be relative to something so when you start expanding more links you have to keep track of what comes first and what comes later so for

the origin and axis in terms of joints the idea of that is a little bit different from links so for joints what we're actually doing is changing the frame location so the origin describes the location of the child's frame relative to the parent frame and the axis describes the axis of rotation for the joint so the Syntax for that is rpy equal rpy and XYZ equals XYZ for the Dynamics the Dynamics will include the damping which is expressed in Newtons uh seconds per meter and then the friction which will be the static friction expressed in

Newtons so the Syntax for that will will be damping equals the damping coefficient and friction equals the friction coefficient so for the limits limits are for revolute and Prismatic joints only the lower and upper fields are in radians or meters respectively and then the effort is the max effort in newton meters and velocity similarly is going to be the max velocity but in radians per second and the syntax for that is going to be like this so limit lower upper effort and velocity now let's go ahead and take a look at a joint example so

here we we have a simple joint urdf so let's go ahead and open that file so here we have a simple joint so inside of our simple joint um what we have to find here is we're going to build on top of what we had so here we're using a blue and then here we have a base link and our link we have a new link here called a right leg so here we're going to be rotating 90° about the Y AIS and then shift down by3 m so that's what this is for the origin

and we're going to make this one white and then here we have a joint so for our joint we have a base to right leg and then we're going to make it a continuous type of joint and then here we Define the axis to be about the Y AIS and then we have a parent link which is our base link and then our child link is going to be our right leg and here we're going to be Shifting the origin of the Child Link by this amount so that it goes to this side so we

could go ahead and run this now so we come here and run this uh launch file and this will allow us to view our joint and you can see here we have a leg on the side and then this here will allow us to rotate the leg about the y axis which is the green axis that we see now finally we could go ahead and take a look at a full robot example so we'll be making a robot with 16 links and 15 joints So Below is a table that shows the 16 links so you

can see here we have cylinders boxes and then um we split it up into the right and left side we also have a gripper as well and a head and a main box and you can see the joints we're connecting all the different parts together so for the joints we have the different connections and we specify the different joint types so you can see some are fixed some are going to be continuous some will be pratic and some will be revolute so we could go ahead and run this example here by using our launch file

so this will generate our robot example and allow us to move the different positions with our joint State Publishers so you can see if I rotate this around you can see this is our full robot and you can see that uh all the frames show the different links uh and joints and it goes to the positions as we wanted to and you can see here this is our control center which will allow us to move the different things so this one here is moving the gripper so this gript right here is actually using a STL

file so if you take a look at the file for this the robot example you can see this is our full example here um but the main thing that is different in here is that we have a mesh file name which we talked about earlier that will Define our actual gripper so you can see here you can play around with the different controls and see how that works so now I want to talk about zacro files so we just finished talking about urdf files but what are zacro files so zacro files you can think of

are files that allow you to create functions that you can call inside your urdf file so the main idea behind this is you don't want to have a lot of repeated code so this will help simplify things and make your urdf much more readable so let's go ahead and see how this looks like in vs code I will start off by going over the zacro command go over the robot State publisher method the urdf launch file and then talk about the zacro property zacro macro show you how to use a urdf using zacros in the

same file and then urdf using zachos in separate files so to use the zacro command the way to use it is have the zacro uh command here followed by the macro do zacro and greater than model. urdf so what this will do is take your zacro file and convert it to a urdf file using the zacro function function the next way to use zacro is using the robot State publisher so you could use this inside of your launch file so for this you need to have the path to your urdf and then inside of your

launch file you could have a launch ross. actions. node we're going to use the package robot State publisher and then the parameters that we'll be passing is a robot description and we're going to be running the zacro command inside of the launch file to convert our zacro file into a urdf file the Third Way is using the urdf launch package inside of your zacro file so inside of here what we're going to do is call our fine package share the urdf launch and then inside of here we're going to have a urdf package with the

description of your robot and followed by the model name of your um file so let's talk about zacro properties so zacro properties are useful when the same property shows up at multiple places and this letes you change the value of all instances at one location instead of having to modify it all the places individually so the way to declare a property is to have the zacro colon property and then name equals property name and then the value equals some value that you set so to actually use a property you would put it somewhere inside of

your code where you say variable equals and then um using the money symbol the dollar symbol and Then followed by these brackets curly braces and the property name so the zacro macro so here macros are used to replace several lines of XML code with the option of having input parameters so this is useful when several lines of code are repeated inside of your files so to declare a macro what you're going to do is say zacro colon macro and name equals macro name and parameters equals the list of your parameters and then you have some

logic in the middle and Then followed by the close zacro macro and to actually use this macro you would say zacro macro name and parameter equals the value of the parameter that you want to set so urdf using zacros in the same file we could take a look at this example file here so we can see for our um number four here so here we actually could declare our zacros inside of our files so if we take a look at this file we could see that in here we have to find our zacro properties up

on top and then for example this is a zacro macro so in this case we're using this for a chunk of code for our inertial properties and you can see if we want to actually use it then we could find you know the location that we're using this uh default inertia here so you can see for example you could just call zacro default inertial and then pass in the mass parameter so that's how it looks like and then if we go ahead and run this we could see what is happening just to verify that it

works so I could run this uh launch command here um our launch file and you can see that it renders our robot all working so next up we could use urdf using zacros in separate files so we can go ahead and take a look at the general structure so we're going to have a separate zacro file that has this structure here with the XML version your robot and you have to include this tag here for the zacro and then you might have some property and some macros inside of your zacro file and then using your

zacro file so uh notice that the separate file is going to be called fil name. zacro and then inside of your main file that's using the zacro file is going to be called something. urdf do zacro and inside of here to actually use the file you're going to say zacro include and then the file name which is fil name. zacro so in this example we're going to actually use um our robot Parts as the zacro file that we're moving all of our macros in and then our robot example zacro external urdf the number 51 that's

going to be using it so you can see here we've used all of our um macros here inside so you can see we have different macros we have a default inertia macro we have a wheel macro we have a lake macro and then we have a gripper macro so the reason we split up these macros is because we have a left and right for the these different parts and there's no need to repeat our code so if we look at our example here you can see that uh our example has been simplified by a whole

lot because we use the property of symmetry and you can see here we have included our file name so just to take a look at a few you know we have our default inertia here and we have our uh different gripper zacro and so on so this really simplifies our code so we could go ahead and run this just to verify that our new example is working so if I run our launch file you should see our robot show up and you can see it behaves exactly the same so we know that's by separating our

file into these zacros nothing has broken now let's talk about Aris so Vis is a visualization tool in Ross that allows you to visualize the motion of your robot it also has a lot of additional features that's great for camera integration you could view the frames there's like Point Cloud features as well so we'll dive into how to use Arvis in Ross I will show you how to simulate your robot in Arvis using the robot State publisher and our own State publisher I'm going to go over the R2-D2 Ross package go through the rvis configuration

talk about the urdf file that I'm using review the launch file go over the stay publisher source code update our setup.py build our package and finally run the R2D2 example in Arvis so by the end of this video we're going to see this robot here dancing around in [Music] circles so I went ahead and created the Ross two package using the Ross 2 package create command and we're going to add the dependency rpy next up we're going to talk about the rvis configuration that we're using so the point of this configuration file is that when

you set up your rvis environment and parameters or settings plugins you want it to remember what you set it to so instead of having to set it each time you're going to have the settings stored somewhere so that when you start up Arvis you could just use it from those settings so the specific things we care about is the fix frame setting that we're using as well as the robot model and we want the topic to be coming from the robot description and we also have the TF plugin set up so we could see the

frames so here is a file that has all the configurations it's called R2D2 Arvis so when we actually call the command later you're going to see we're going to be using this file directly and some of this stuff will make more sense when we start up Arvis so you'll see more details later on so the R2D2 urdf file that we're using is pretty simple but if you're new to urdf files I have a video on it so you could check it out but the basic structure of our urdf file is we have couple of links

the axis leg one leg two body head rod and box and then we're connecting them with joints L one connect l two connect tilt Cil Periscope and box connect so we're going to link it between the parent and child links so that we can create the model that we showed earlier in the beginning of the video now let's take a look at our launch file so our launch file is going to start up our robot State publisher and our state publisher so we're going to call it demo. launch. and you can see right here we

have different settings so first off we're reading in our urdf file here and then once we read it we're going to pass it in later so here we're going to store it as the robot description so we have two nodes the first node is the robot State publisher and this we pass in our robot description and then the second one is our state publisher here that we're using which we will create our own and go through later on so these two combinations will allow us to actually animate the robot in Arvis so you can see

here this is our state publisher node that we created so we've inherited from the node class we call called our node the state publisher and inside of here we do some things in terms of setting up the creating the publisher here as you can see and we're getting the joint States we set up a broadcaster a node name so these are all just some initial settings that we do and then later on we have a degree and loop rate that we're setting to and we have the robot states which we've defined here for uh the

different angles that we're moving the different joints and then here on we have a frame ID called the Odom for odometry and then we have the axis and then here we have our main Loop that does the most of the simulation so here we have a while loop that's going on we get some clock time we get the joint names that we're interested in and then here we update the positions you can see with the X and Y values as well as the Z so XYZ and we have some rotation for roll pitch jaw that

we go ahead and update so after we update everything we want to publish the um New States and then we send the transforms and then here we actually incremented by doing uh different plus and subtraction based on what we want the robot to be doing and then finally we place a little sleep at the end and here we have a Oiler to querian as you can see which we use here and it's defined here just doing the querian oiler to querian conversion okay and inside our main function we're calling the state publisher node and that

will kick off everything and make it up and running to get ready to build we have to do some modifications to our setup.py file so up here we've added these Imports because we need these modules later on and we've added these two things so that we can find our launch file and urdf files and finally to use our state publisher we've added this line inside our entry point so to actually get our program up and running you want to First build it which I've already done and then once you do that you want to open

up two terminals one terminal to launch the launch file and then the other terminal to start up Arvest 2 so you want to CD to the directory Source it and go ahead and run it so I'm going to run one side for my launch file and then my other side for my Arvis and you can see my robot here is up and running so previously we were talking about some of the settings you want to make sure your um Arvis configuration has so to actually see what those things that we were talking about you can

see here is a fix frame so you want to make sure that's set correctly and you have the robot model which we've added in and you see the robot description here is robot description for the description topic and these TF transform is what allows us to see the transformation so now this is where things get more exciting we're pretty much ready to do some more advanced simulation now so with all the knowledge that we've learned up to now we're ready to create our own simulation using gazebo and Ross so in this specific video we're going

to be using a tabot robot and we'll be integrating in gazebo so we're also using Ross 2 control as well so so the main idea is this we have Ross 2 control which is responsible for the motor command so the Ross 2 control can be used to control position velocity or Torque commands of the robot and gazebo is going to be the simulation environment that has realistic Behavior such as physics so you can have the actual weights of the robot to have a realistic torque estimation and there's additional features you could add on such as

friction to get even more realistic things but for now we're going to take a simplified case and just try to simulate the tabot in gazebo with a basic gravity setting I'm going to start off by talking about how to install the Gazebo and Ross to control packages go through the package structure talk about the corn frames that you need for your urdf file go through the changes that you need to make for your cmake list and package.xml file go over how to display your robots in gazebo and the different things you need for your launch

file talk about the types of Ross 2 controllers go through the Gazebo and rosu control add-ons that you need to put inside your urdf file go through the control yaml file that you need to set up your control parameters and finally simulate your robot with the joint publisher so by by the end of this video we're going to see the tabot doing some air curls to get everything set up we need to install our gazebo and Ross to control packages so here we have the different ones that we need so just run these commands here

now let's go ahead and create our package so I made a package using the Ross two package create command and this is a general folder structure that we have so there's a couple of new things that we've added we added a new config folder we added a new launch file folder there's a new mesh folder um a urdf folder so these files are also new ones including the joint controller we have the Gazebo launch. we have our STL files in here we have our joint publisher that we've added as well as our zacro file for

our Tesla bot and then the two main changes we made are inside our cake list and package.xml which we'll go over later on now let's take a look at our coordinate frames that we need for our urdf file if you're new to urdf files I have a urdf tutorial in my channel so go check it out so here is a list of all of the joints that we're using so we have the connection from the world to the head head to the body body to the shoulder shoulder to the upper arm uh upper arm to

the lower arm and then finally the lower arm to the hand so these are all the lists of the connections between the parent and child types and then we have the type of joints where we have some fixed ones and some revolute ones so all of the revolute joints you can see here are about the z-axis so you can see here is a summary of the joints definition that I've made for the testot and here is the testot zacro file that that we have right here and you can see that here it just a bunch

of links and Joints to put together the robots and all of the STL files are gathered from the meshes folder okay now let's take a look at the changes that we've made to your cic list.txt file so the different changes that we make are the three things one is to find the package other is to create the executables for a joint publisher and finally to copy the files to the desired directories so let's take a look in here at the actual changes so you can see these lines here is to include the packages that we

need here is to create the executable for the joint publisher and then finally these steps here is to copy the files over now for the package.xml file the changes that we need to make are to add the dependencies for our RCL CPP as well as our trajectory messages so you can see that we've added these two lines over here now for our launch file we need to add some things to display our robots as well as other actions so we have a launch gazebo which will start up our gazebo we have a robot State publisher

which will publish the state of our robot for visualization we have a spawn robot component which will display the robot in the Gazebo we have a joint State broadcaster which will publish the state of the robot for control and finally The Joint trajectory controller that will receive receive and control the robot joints so let's take a look at our actual launch file here so you can see here we have our includes up on top and then here we're actually utilizing the Gazebo Ross uh package and then their launch file gazebo launch. and then here inside

we have a robot description so this will parse our urdf file the tesab bot. urdf do zacro file and store inside robot description which will we then pass into here for robot a publisher so that will get the information of our robots and finally this will spawn our robot by using the robot description for our topic and then the next two steps deal with the Ross to control so one we're both loading controllers but one is for the joint State broadcaster and then the other one is for the joint trajectory controller so we just go

ahead and run the five uh actions and then we'll get everything up and running so coming back to here we can see that now to actually get everything up and running what we want to do is build source and run the launch file so couple things to note is that sometimes if you see it not working properly it could be because you didn't close your gazebo um so you could actually Force start it by running this command down here so let's go ahead and open up our terminal and we could just run the command I've

already sourced it but you can see if I run the um command here gazebo launch. py we could see our robot spawn inside gazebo so it'll take a minute for it to start up but you can see right here we have our robot inside gazebo and you could pan it around you could use your mouse wheel to spin around like I'm doing here you could zoom in zoom out with the key and you can also pan by dragging okay so this is the robot model that we're using and later on we're going to see us

animating it so to actually move our robots we have to have a little bit of understanding of the different types of Ross 2 controllers so here is just to name a few but the main thing you want to note is once you've decided the type of Ros 2 controller so you could control the effort like the torque or current you could control the position the velocity there are special type of controllers that are meant for wield Vehicles called the diff Drive controller and these are just some of them out there there's more things like PID

control admittance control and so on but here the main thing you want to notice is that when you choose your controller you want to make sure you know what message type that they're using because later on when we create our publisher we need to send the messages in the correct format so in our case we're dealing with the joint trajectory controller so here it uses a joint trajectory. message so this is the type of message that we're using so later on we'll take a look at how to actually format it properly so if you want

some references you could check out the out there inside the common interfaces so to actually get it working you have to set up the Zeebo and Ross to control in your urdf file there's a certain section that you have to add inside of it so I'll be going over that uh but depending on the different type of controller you're using you could choose different combinations of State interface and command interface so the state interface allows you to read data and the command one allows you to send data so the different types of interfaces you're dealing

with would include position velocity and effort so here is the general structure of your urdf file so you can see here all of the stuff on top is going to include your links and joints and then the key part that we're adding on to our urdf file would be the Ross to control components so here we have uh this part the plugin that we've added and then inside of here you could have the list of your joints so here we have joint one and then you could sew on to list your joints two three and

four however many you end up having and inside of each joint we uh specify the command interface as well as the state interface which we show in a little bit and then finally you have the Gazebo component here which will include our joints controller yaml file which has the parameters that we're using so if we go ahead and open up our file here for our urdf file you could take a look at what's inside so here inside of our our Tesla bot zacro file if I scroll all the way down we can take a look

at our Ros 2 control here so specifically in this example we have two joints that we're controlling one is the arm upper to lower for the right and then the one for the left and you could set some parameters for the Min and Max angle for the joint and then our Command interfaces is going to be position State interface is also position and then uh we have two more here for velocity and effort and we also set an initial value to be zero here okay and then for our gazebo plugin here this part is going

to look exactly the same as we showed earlier now for our Ros to control yaml file that has our parameter so the general structure of this is you have one section that has the controller manager and then here you have some update rates the name of your controller um in this section as well as your joint State broadcaster and then you have another section that will have the name of your controller and the parameters you're using such as the joint names as well as the interface names command interfaces and state interfaces so here is a

concrete example of the one we're using so we've set an update rate of 100 and here's the name of our controller here and this is the type of controller and then the two joints that we're using it's a left and right upper arm to lower arm and then finally we have our Command and state interfaces specified here now to simulate our robot and gazebo we need to use the joint publisher so for the joint publisher the first thing we need to understand is the message types that we talked about earlier so in this case we're

using the joint trajectory message so here is the actual definition of the message so you can see that the joint trajectory message uses the joint trajectory Point um so if we take a look at that the joint trajectory points includes position velocity acceleration effort as well as the duration so so we could take a look at our joint publisher file now so here we created a node called Joint publisher and then inside of here we created the name uh inside of here joint joint trajectory controller joint trajectory so inside of here we have the main

thing that does all the work is the Callback so our callback function we start off by making a message with the correct type so notice that this right here has been included so when we make the build the files are made available to us cuz we've specified the dependencies uh but here you can see that we have the joint names inside our message so it's as a vector of just a bunch of names that we push back and then here we have a point that we've declared so the point is going to be a struct

that contains the different information so we have a point. positions and we push back to positions as like a vector and then we have the start time which we've defined and then we want to push back the message uh push back the point inside the message and then finally we print out our positions in this um command here so here we have our usual in our main function the Knit Spin and shutdown okay so this is the main logic that actually controls the joints so now to actually run it we would have the previous terminal

that we had up and running so we want to have the Gazebo launch up and running and then we're going to have another one that does the joint publish so if I go ahead and run that one so this is going to be the terminal that actually causes the robot to move so you can see now my robot is moving and you can see the arms is moving according to the positions that I'm feeding it so we just finished looking at how to simulate our tabot using gazebo and rosu control but what you're going to

notice is that a lot of times when you're simulating things you want to have a live plot of your data you may want to visualize the joint trajectories or even the torque values as a function of time luckily they have a tool called plot juggler so I'm going to show you the steps and the commands you can use to set up your plots so the agenda for today is we'll talk about how to install plot juggler run plot juggler plot the signal save the plot layout and load the plot layout so by the end of

this video we'll see how the joint trajectories for the elbows will be plotted out in real time so you can see this is what we'll see by the end of this [Music] video so to install plot juggler all you need to do is run these two commands one is a plot juggler and the other one is plot juggler - Ross to run plot jugler you could run it from the terminal using Ross to run plot jugler plot juggler or you could also put it inside your launch file here using the execute process which I've done

so in one terminal we could go ahead and start it up by running this command which which we introduced in the previous video and then the other command to start up our joint publisher so I'm going to go ahead and run that so here I've already Source it going to run my launch file so this is going to start up plot jugler and they're known for uh you can see this is our plot juggler here and then in my second terminal I'm going to go ahead and run the joint publisher so this is publishing the

joints you can see my robot here is now moving doing the curls but now we need to actually set up some of the things for the plot okay to set up the plot you see here under streaming there's a Ros to topic subscriber so you have to click the start button and then here you could choose the topic name that you're interested in for me is going to be this one here so I'm going to hit okay and then once it loads you could expand this expand it again and you're going to see under points

it's going to be the points that we're looking at so if I pull this down you could see more of the window and you can see here is the positions so you can go ahead and click on this and then drag it over and this will plot the joints for one of them and if you hover over this by the X icon you can see here the vertical or horizontal split I'm going to split it up and down and then now I could drag my second signal here to see both joints at the same time

and you can also drag it in the same plot if you want to see two plot together so if I drag position one you can see there's two overlay together okay so you may be interested in Saving this layout that we've created so to do that you could click these icons here one is load and then the other icon is save so you want to click the up arrow and then there's going to be a window that pops up so this is the window and you could go ahead and save it to the name that

you want here I'm calling a two joint layout and it's saved as a XML file okay so now if you actually want to use the layout that you saved what you want to do now is use the other button so this is a load button and you want to click the layout that you saved and you hit open and after you hit open you're going to see a window that pops up start the previously used streaming plugin and you're going to hit yes and you're going to confirm the topic that you want and once it's

done loading you see that your two plots are now showing so this is great if you want to reuse your plot layouts without having to redo it every time okay so now I'm going to show you how to integrate a camera in your gazebo simulation so this is great if you have any sort of image processing or computer vision tasks that you want to simulate so let's go ahead and jump into vs code and see how to set up everything I'll be going over the camera model talk about the Ross camera sensor in the urdf

file go over how to view camera and images in gazebo and View Camera images in Arvis by the end of this video we will see how we could simulate the tabot POV in the simulation that we can see in this video right here now let's talk about the camera model so here is the camera pinhole model at the very bottom here we have what's called the camera Center so the camera Center has a coordinate frame attached to it we have the x axis the Y AIS and the Z axis so the x axis here is

pointing towards the object that it's looking at so here we have an object here this is in the world space and then it gets projected to the image plane in a 2d view so this right here the green plane is called the image plane and this plane is defined by the width and the height and we also have What's called the field of view so this field of view is here is going to be the angle that describes how much of the view that is looking at and then also we'll have different concepts later that

we'll be using which is a clipping plane we have a near and far clipping plane which will describe um the region of image that we're actually looking at and extracting in our images now let's take a look at the Ross camera sensor urdf file so the general structure of the part that you need to add to your urdf file is this region here on the left so this is a general structure and what you have is the camera reference here so this is going to be the name of your link and in our case we're

calling that head so that's what you can see right here we have this head and then here we have the sensor name camera and type so that's going to be the same and then we have a posst XYZ RX r y RZ so we'll explain how we get this value here and then visualize we set to True update rate is some rate that you want and then here we have the field of view in radians so here we're using one and then we have the format R8 G8 B8 so this is going to be the

format for the RGB and then you could change the different ones but most likely you'll be using this one to begin with and then we also have the width in pixels and the height in pixels so here we're using 640 and 480 and then here we also have to find the near and far plane in meters so we've defined it here as 0.05 and one for the far so here we have the plug-in name here it's going to be camera controller and the file name is going to be the lib gazebo Ross camera. so and

lastly we have the frame name which is going to be the name of your joint and here we're using world to head so that's all you need to add in your urdf file to actually get everything up and running so now let's go ahead and take a look at how we derive the actual Corner frames so if we take a look here we have what's called our world frame so our world frame we have the XYZ axis so the first thing that we're going to do is look at our camera pose this is our initial

pose so you can see here we have um our xaxis right here so this x-axis is the initial pose of our camera and then later we're going to be modifying it so if we scroll down we see the first transform that we do is going to be this right here so here in our transform what we have is we have a rotation about the Z axis which will allow the camera to be viewing in the same direction as the robot and then we have a z translation upwards so that's this part this is the translation

and this was the rotation so we have the translation that will bring it up to the eye level so that's the main point of that okay so if we scroll down one more time we have our camera posst two so what we're doing here is doing an additional transformation so this transformation here is a rotation about the Y AIS of the camera and this will allow us to view down so that we can see the hand so that's what this component here is so 0.7 so now what we want to do is go ahead and

view the camera and emission gazebo so what you want to do is run these steps here in one terminal so we'll pretty much start the Gazebo do launch file that we went over previously in our tesab bot simulation and then we're going to run in the second terminal or join publisher that will make the robot actually move its arm so let's go ahead and see that in action so I went ahead and ran that and you can see that here we have the arm robot arm moving up and down doing the bicep curls and you

can see it's POV here of seeing its own arms moving in this image plane down near the floor so now we're going to go ahead and view the camera images in Arvis so what you need to do first is make sure you have the joint State publisher in your launch file otherwise some of the joints won't show up the fixed joints don't need it just only the revolutes so you want to add this section here into your launch file which is the joint State publisher node so have that in there and then it'll run the

node to actually publish the joints over to Arvis so in the third terminal what you want is to start up Arvis you could do the configuration if you have a configuration set up already but I'll show you how to set up everything manually so to do that you just need to run the Arvest to command and then here we also have the pseudo to install the joint State publisher if you haven't already so we're going to go ahead and run Arvest 2 here so if we start up Arvest 2 um you could see it start

starts up blank and then what you want to do is add the different things here so you can see um we could add in our robot here our robot won't be moving it's just going to be the model because we're not actually um we're not actually connecting the simulation here but if you wanted to see the robot you go ahead and choose the description topic and choose robot description here and you can see our robot and if you want to see the image you could go ahead and add in the image here so here if

we scroll down you could see um here we have the image and inside of here you want to make sure you've selected the right uh topic here so here you could choose in topic there's a drop down it's going to be camera image raw and if we select that you can see down here is going to be our image you can see the image the same scene that we saw in gazebo we just finished talking about how to set up cameras now we're going to talk about how to set up Lars Lars are great because

they could give you a dense Point cloud of your scene so it's perfect for 3D reconstruction like for autonomous vehicles or even for manipulation of objects so here we're going to go over the steps I need to go through in vs C to get that up and ready to go we'll be talking about what is a lar and how does a liar work types of Lars gazebo lar sensor plugin for your urdf file the lar gazebo simulation and the lar Arvis simulation by the end of this video we will see how we can generate liar

in gazebo on the left and the same thing in Arvis on the right so what is a lar and how does a lar work so lar stands for light detection and ranging and it works by sending light and determining how long it takes to reflect back to surface So based on the time and speed of the laser we can estimate the point in space so here here's a simplified model of what a lar does so let's say we know three parameters let's say x is a distance from the Light Art to the point T is

the time to send and receive the light after reflecting and V is a speed of light so if we assume that this is X and we assume that to go back and forth is going to be 2x then we could approximate this as V * T since T is the total time if we want to for the distance X then we could do V * T / 2 so this is for a single point you could imagine if you were to expand this to two-dimensional or three-dimensional you would just have points in more than one

direction now let's take a look at the types of Lars you may encounter so we have the 1D lar the 2D lar and 3D lar so these are some of the common types you could see but the main difference between these three type of Lars is the direction of the Rays that it's shooting so for 1D you could just imagine you have something that points in Only One Direction in 2D you will have a liart that's pointing in all directions in a single plane as you can see here and then in 3D you just imagine

if you have something and it's pointing in all three directions like a sphere so this could be good if you're trying to detect a whole um environment and you're trying to reconstruct the entire scene then you may want to use 3D or you could be smart with your 2D and kind of move up and down and from there you could generate a 3D scene as well okay now let's go ahead and take a look at the Gazebo lar sensor plugin that you need inside your urdf file in order to see the lar in gazebo so

first off we have What's called the um gazebo reference and this is going to be the name of the link of your reference so in our case for here in our previous videos we're using head as our reference link and then Ray is going to be Ray and then the name of your liar you could just call it something in our case we're calling it liar sensor so the poe is going to be XYZ RX r y RZ so this is going to be some post that you choose for us we're going to just choose

zeros for everything um but if in your case you could update that to something that makes sense for you you could also choose a link that's already attached to the same lar or you could just um trans transform it relative to some link that you have so next up you have the visualized Boolean here so to see the lar you want to put true for that and here we have the frequency in hertz for that we're going to do 10 and then here we have samples which is going to be the number of points here

we're using 720 the resolution we're using 1 and then the resolution is going to be the scan resolution so if you want to have more Spark you could uh change that value or play around with whatever makes sense for you and here we have the Min and Max angles in radians here we're doing a full circle so we're using negative and positive Pi next up we have the Min distance and max distance in meters here we're setting it to be 0.1 and 30 and then we have resolution in meters so the resolution in meters we're

going to set it to 01 and then finally we have our plugin so the plugin here is going to be the name of your plugin so here I'm just calling it lar and the file name that you're actually going to be using is going to be called lip gazebo rosr sensor. Soo so this is going to be the same um and you could choose different ones depending on which lar sensor you're using but here for the simulation we'll be using this one now let's go ahead and take a look at how to see our lar

in our gazebo simulation so I went ahead and already built it and then I'm going to go ahead and run my gazebo launch file from our tabot simulation so this is going to pull up our tesab Bots with the lidar uh scanners as you can see here all the light rays shooting radially outwards so you can see if I zoom out you can see it's a giant Circle um but if I go back in you're going to notice that right now all the rays are still uniform because it doesn't see any objects so if I

go ahead and drop a sphere in here and then if if I go ahead and scale it so I could hit this button here to scale and then I'm going to make it a little bit taller so you can see that it's going to start blocking some of the Rays so here when later when we get into Arvis we're going to see the same contour line that's being intersected by the Rays okay now let's go ahead and see how we can see our lar simulation in Arvis so by default the lar will output points using

the topic lar out where lar was the name of our plugin so by default Arvis we use the point Cloud 2 but if you want to use laser scan you can go ahead and modify the plugin in our urdf file like this so the main thing is you can have an output type specified here so let's go ahead and see the lar in Arvis we could just go ahead and start up Arvis here so type in Arvest 2 and this will launch another window that will show us everything and if you want to see your

robots so you could have like a relative Direction you can go ahead and choose your uh robot model here and then we want to come down here and then inside of here is the description topic and you want to choose the robot description and this will show us our robot and then if you want to see a point Cloud you can come here by topic and you can see here is our Point Cloud 2 and you can see it shows up the Contour and you can see the topic here by default is slider out so

if I go ahead and move this you can see that this is my Contour as we expect it from our gazebo simulation another very important sensor that I want to talk about is the depth camera so depth camera is great because it can help you do things like depth estimation but more importantly it could handle things like 3D reconstruction if you're trying to reconstruct your sing from a depth map so here I'm going to show you step by step in vs code how to set that up in this video we're going to see how we

can use the Gazebo simulation of our Tesla boss to look at a sphere using a camera and then we're going to see it in Arvis to visualize the point cloud and then also see the dep map as you can see on the bottom that be going over what is a depth camera talk about the Ross depth camera gazebo sensor setup for your urdf file talk about the depth camera frame orientation which can be pretty confusing and talk about the Gazebo death camera simulation and finally show you how to set up everything in Arvest to see

your death map and the point Cloud so what is a dep camera a depth camera uses one or two cameras so here we have a right and left camera and usually there's something that we call the IR projector so what this does is it's going to project some IR dots and sometimes if it just uses One camera it's can be enough to detect the location of those points and since you have two points of view you can figure out the disparity between the left and right image and then from there you could determine the depth

using some similar triangles and from the depth we can finally determine the point Cloud for the scene so here is an example of a Intel real sense depth camera the d435 so here you can see this is a general structure of the camera and some important parameters would be like the base length you need to know the distance between your cameras and then you have what's the focal length because when you have a camera model you have um some focal length that will um project your scene to your image plane so that's the general idea

you could have like an object here and this object is projected in your image plane so I have a video where I talk more in depth about U understanding depth maps with open CV so you could go ahead and check out this video for more information so now let's take a look how we could set up our death camera gazebo sensor for our urdf file so previously we used our head to camera transform in the gazebo section but now we're creating a new joint for it and we'll explain the details of that later on so

we're changing this post which took care of some of the transforms and now we're setting the post to be all zeros and then we're going to be creating two links so we have one link here called the camera link and then another link here called the camera frame link so we're going to have two joints the these joints will connect the different pieces so we have the head and then the camera link so we're making a joint to conect connected to and then here we're doing the transform that we did previously here so we're moving

this transform that we did previously to this and then we're creating a second joint called camera link to camera frame link so this will do the transform between these two links and then this is going to be the transformation that we're going to be applying and in the later section I'll talk about the details of what that actually entails so if we take a look here um the gazebo urdf file for the depth sensor is going to be very similar to the camera sensor so I talked all about it in this video my camera gazebo

and Arvis video so if you want more information on how we set up the camera you could check out that video but here you can see this is the part that we need to add inside our gazebo file so you can see here we have the camera link and then we have the camera and here's the main part that we changed is now the depth type so this will enable our depth images when we're in Arvis so I updated some things like the horizontal field of view um and then I also played with the clipping

because I want to see a broader range and then here I just renamed some of my plugins and the key part you need to add here is actually the frame name so this will associate your um camera frame to the right link which will take care of some of the transforms that we did previously so let's take a look at the camera camera frame orientation so below shows the transform between the camera link to the camera frame link so initially the camera link has the x-axis pointing in the direction of the camera so you can

see here is the xaxis and it's pointing in the direction of the camera but we want to find our camera frame link so this will describe the camera frame which is how typically the camera is oriented um when you're actually going to dis the point Cloud so we're going to do a series of transforms so the first one is a rotation about the X in the negative pi2 Direction and then that's our first transform and that's going to rotate our Z axis so now our Z is pointing to um in this direction and Y is

pointing down and notice that we're doing about the original x-axis and then we're doing a second rotation about the z-axis in the negative direction of High halves so you can see now our x- axis is pointing to the right so this is our new x-axis location but the key point is to have the z-axis pointing in the direction that the x- axis was pointing in so this is what we're expecting the camera link frame to be oriented so if we scroll down you can see that here is um the view of the expected resultant frame

so you can see that the the final orientation is the Z and that's the expected orientation of our camera frame and you can see X is pointing to the right and then our Y is pointing down so if we don't do these transforms our Point cloud is going to be all messed up and it's going to be like upside down and the sphere will be this way so if you do all this correctly then you should see your sphere in the right orientation somewhere on the floor so now let's take a look at our gazebo

simulation so we're going to set up our scene so that when we're in Arvis we could actually see something in Arvis so go ahead and go to your workspace build it uh source and then run the launch file that we talked about in our tabot simulation in our gazebo tutorial video so after you run that you're going to see the tabot simulation here inside gazebo and you could just throw in a sphere like this and that will set up our scene to view everything in Arvest when we actually set up everything okay now let's take

a look how we can set everything up in Arvest to see the depth map and point Cloud simulation so in the terminal you just want to run your Arvest 2 command and then here you could set up everything so initially there's nothing and then when you hit add you're going to see a Arvis window pop up and what you could do is double click this for the point cloud and the point Cloud will show up and if you want to see the robots then you want to add in the robot here so if I scroll

down you can see the robot model and if you double click that uh what you want to do is choose the right topic which is the robot description and you can see our robot is now showing up so this is our Point Cloud that um we're expecting the orientation is all correct because we set up the frames correctly so now we could go ahead and add um image here so you could uh you could also view it from here so under depth image raw you can see image here and you double click that you can

see this is our depth depth map image here so this is pretty nice if you want to use that for something later on but we could also see in our gazebo simulation if I pull that up um you could also play around with changing the location and seeing the different things move around so here you can see I have the ball and if I drag it and you can see the death map here and the point Cloud updates which is what we expect so it's nice to see everything moving uh dynamically all right in this

video we're going to show you how to simulate a mobile robot using gazebo and rosu control specifically we're going to be using the Cyber truck and we're having our tabot ride the Cyber truck because the tabot can't walk yet so it's going to be riding along telling it what to do so if you haven't seen my testot simulation go ahead and check out this video right here but in this video we'll be covering these topics here so we'll be talking about how to do the Gazebo and R to control installation the package structure the cmake

file changes the coordinate frames the urdf file the Gazebo and rosu control launch file the gazon rosu control setup for the urdf file the yamamo file for the diff Drive controller and the command publisher to send the commands to wheel robot and finally go over some visualization of the data with plot juggler so to get gazebo and rosu control setup make sure you go ahead and run these commands to install everything that you need need so here's a package structure for our gazebo simulation so if you haven't made your package already go ahead and run

this command here to make your package but the structure for folders is right here as you can see below so we're going to have a new folder called config and inside of here we're going to have our yaml file we're going to have a new folder for our launch file and then make our own launch file here we're going to have a new folder for our meshes to contain our STL files we're going to have a new file called the diff Drive publisher CPP and we're going to have a urdf folder that contains our zacro

file make some changes to our cmake list and then just leave our package.xml alone okay now let's take a look at the things we need to modify in our cmakelist.txt so up here we have our fine packages we have one for our men cmake our geometry messages and our rcpp and next up we want to include our directories we have our launch urdf config M meshes and we want to add our executable here so we have one for our diff Drive publisher which will be sending our commands and then we have our Target dependencies set

up here to use these two things and finally we have our install targets which we including our diff Drive publisher and have our destination here next up let's take a look at our coordinate frames for our wheel mobile robot so here we have four wheels you have the first one the second one the third one and the fourth one so these four wheels we have our coordinate frames I'm drawing at XZ the Y AIS you can figure out by the right hand rule but all of these we have shifted here so you can see these

dots here is the actual location I'm just drawing the frames on the outside for better visibility and you can see our Tesla bot here is right at the center of our cyber truck and we have some key Dimensions here are 1.46 which is a separation of our wheels and then this value here 4175 is the radius of our wheel so if we scroll down a little bit more you can see this is the top view of our or the side view of our uh robot and our Tesa bot the truck and the tabot so you

can see some of the key Dimensions here that we care about is this Dimension here the 1.5 so this is the distance between the center and the wheel in the X Direction here and then we have another one a very small distance here which is a um Z the Z shift of the wheel relative to the center and then we have another Di mention here that's going to be the distance from the robots to the base of the the body of the car so these will be some key Dimensions that we'll be using later on

in our urdf file okay so now let's take a look at our urdf file for our wield mobile robots so here are the lists of links that we're going to be going over we have the body window the front left wheel front right back left and back right one thing to make note is that you need to make sure you have your Collision tag otherwise the wheel is just going to sink through the ground and your robot's going to disappear and here are the lists of joints that we're having so everything is going to be

connected to the body so we have to make that joint connection to allow it to um rotate later on and one thing to notes is that the direction of the axis needs to be flipped because previously when we were looking at our Wheels we have one wheel we have the z-axis pointing outwards so because they're pointing outwards we want it to rotate in the same direction so you have to flip the direction for the axis so if you want to more have a more in-depth review of the urdf file I have a video right here

so you can go ahead and check it out but let's go ahead and take a deeper dive in our cybertruck urdf do zacro file here so here is the urdf file as you can see here so we're going to take a look at that up here on top what you can see here is we have our body and for the nura um the nural tags I'm not really caring about that right now you could fill it out to be more accurate later later on but the key part right here is we're adding a mesh for

our body so we have a body. STL file here and then we're going to just leave the scale to be one: one because I've already set up the scale correctly and if we scroll down a little bit you see we have our window so the window is just something I created so that you can see the windows in different colors and again it has its own window. STL file and then next up we have the front wheel so for our front left wheel what we have is um we just have some filler here for the

nerell the nerell tag um but the main important part is going to be the Collision I made the Collision tag to be very similar to the actual uh size of the wheel and then again I have a wheel STL file this wheel STL file will be the same for all of the wheels so I'm going to repeat the same thing for the front right and we have the back left again it's repeated and then um you have our back left here back right so all of these will be pretty much the same the main thing

that's going to be changing is going to be the actual joints so if we take a look at the joints the different things that were actually modifying here is uh these values here so if we take a look at this you can see the key part is going to be the origin so if you notice here uh some of these values will be different so you can see that this one is going to be positive and this one is negative here because if you look at our coordinates Sy some previously and the y1 is in

the positive this one is in the positive y this one is in the negative Y and then here we have the same thing um now we're shifting it to the back so here we havex with positive Y and then we havex with um negative y so these are the combinations to put the wheels in the right direction and then we have some rotation here we have a negative pi2 a positive pi2 negative pi2 and positive Pi Hales so these will allow the direction of the axis to be pointing with the z-axis outwards like we saw

in the coordinate frames earlier and then finally we have our continuous tag so this will just allow it to be rotating um so that's one of the key parts and finally we have our axis here so notice that we have in the positive Z and then negative Z positive Z and negative Z so this will allow the directions to be in the right uh direction as we talked about earlier and then finally each one we have a parent and child so that's going to tie the body to the correct wheel okay now let's take a

look at our launch file that we need so the main things that we have we have a gazebo launch a robot State publisher node a robot spawn node a joint State publisher node a load joint State broadcaster a load joint trajectory controller and finally a run plot juggler okay so here you can see that these are all the Imports that we need to have everything up and running and the first thing that we have here is you can see we have our our gazebo launch so this will start up our gazebo do launch. ply file

so this is the core thing that we're calling and we're calling it from our package here called gazebo Ross and this will start up our gazebo and you can play around with World files here if you want to and then right here we have our robot description so this will read in our cybertruck dodf do zacro file and then it's going to use the zacro command here and pretty much extract everything and convert it to urdf file and then we have a robot St publisher here so this will get some of the joint informations that

we'll need for our simulation and we scroll down a little bit more we have some things for the Arvis so you can see right here we have our joint State publisher node so this will be the things that will show the things in Arvis when we actually try to render it and then we have a robot spawn node so this will actually um get some of our robot description information we have a low joint State broadcaster so this is some of the things for Ros to control to make some of the commands available and then

finally we have these last two things so here we have our load diff drive base controller and this will get some of our commands for moving the wheels and we have our run uh plot juggler so this will help visualize some of the things so tying it all together we have our launch description here so we're going to just call all the things that we've created up on top and then return it as a launch description so now if we run this Ross 2 launch wield robot gazebo launch. pfile we're going to see the rendering

of our robot show up here so you can see that this is our cyber truck here all nicely rendered you can see we have the robots up on top of the car as we expected and you can see that is a nice uh color that we've set and we'll go over some of that later on but right now everything is not moving yet we could just see the simulation of it but after we start sending the commands to everything you could see the wheels moving and you can see the nice on the wheels um it's

black right now so you can't see all the details but you could play around with the lighting and the colors if you want to see more details uh later on but you can see it's a pretty nice rendering here in gazebo okay so now let's talk about some of the tags that I need to add in our zacro file so that we could actually use the Ros 2 control to move the robot so here we have the Ros 2 control uh tag here and what we want to have is a gazebo system and type is

system so in here we're going to use our Hardware so we have our gazebo Ross to control gazebo system and then we want to set up the joints that we're actually going to be using to be moving so this is going to be a rear rear wheel drived uh cyber truck so we're going to have one joint name here we have the body body to back left wheel we're going to be commanding it in velocity and we're going to be using our state interface to be monitoring the position and velocity and then similarly we're going

to be doing this for for the right wheel and if we scroll down uh we're going to see that we have our uh gazebo tags here so these are some of the things I need to pay attention to so here we have our gazebo tag here so this will use our gazebo plugin it's going to be called lib gazebo Ros to.so and the name is going to be called gazebo rosu control and here we're getting the parameters from our uh diff Drive controller. yo file which we'll be going over later on and this is going

to be located in our config file and then here we have our gazebo references so some things to note is that once you set this up you can set the colors and then for each of the wheels you get to set um also the colors of the wheels but also two main things you want to make note of is that we're actually setting the MU values to be zero so we want the front wheels to be uh just passively rotating since they're not being driven so these will be the key things you want to add

to make sure that it's not interfering so make sure you have those friction parameters set and we have our mu1 and mu2 so mu1 is the main Direction and mu2 is the perpendicular Direction okay so that's what the mu1 and twos are for and if you scroll down a little bit more you see that we have uh the same thing for the back and back left and right wheel except those are driven so we're just not going to set a Mew value for that and then some of the Tesla bot things um I went over

previously so I won't be going over that go ahead and check out that video if you want to see more detail okay now let's take a look at our diff Drive controller yamamo file here so this yammo file will allow us to set some of our configurations for our robot control so the main things we want to make note of is we want to set our left wheel names right wheel names the wheel Separation The Wheel radius and the base frame ID so right now we're only using uh two wheels but if you want to

use uh four wheels you can set it as a list here so you can see here we only have a list of one for each and some things to make note of is that you want to set right here the joint State broadcaster and then we have our diff Drive based controller here so these are the controllers that we're going to be using and then we have our wheel separation here our wheel per side is going to be one here and we have our wheel radius to be 4175 which we know from our STL file

and we scroll down a little bit more some of the key things to know is that we have our Odom uh frame ID is called odam and our base frame ID is called body okay now let's take a look at our Drive commands publisher for a wheel mobile robot so we will be controlling our mobile robot with the twist Dam message through the diff Drive based controller command velocity topic with our own diff Drive publisher so we have a series of different message types that we'll be utilizing so let's take a look at that so

you can see here we have a Twist stamped message so here we have a Twist that's going to be a type twist and then from there we're going to be using from The Twist message so inside the twist message we have a vector 3 for both linear and angular and then what a vector 3 is is going to be defined in the vector 3. message which is going to be floats 64 of XYZ so essentially you're going to have um XYZ for the linear and XYZ for the angular but one thing to note is that

because our robot is in a plane what you're only going to have is going to be a linear X and then the angular Z to control the left and right because in practice a car you imagine you have a car here you could only go forward and then you could turn um turn right or turn left and that's going to be controlled by the angular Z so really even though a Twist uh twist stamp has six values we're going to be using two of the six so now let's take a look at our diff Drive

publisher file so here we have our file that's going to be sending our Command so here we have some some of our includes and we've created our class here called diff Drive publisher and inside of here we're going to make our node and then once we make our node we're going to create our publisher here and then we have our uh topic here set up with our certain frequency and then we have our call back here um using our timer create wall timer and then finally we have our private here called publish command so this

publish command here is the main thing that we're going to be using we're going to be sending a linear dox at 0.1 and the angular doz at 0.1 as well so this will allow it to the robot to start rotating while going straight and then finally we have some other private variables that we've declared here and then we're going to call our uh typical init spin shut down and then get everything up and running okay so in one terminal we had our gazebo launch file running and then in our other terminal we're going to run

our diff Drive publisher here so this will actually make our robot start moving and if you can see here if I open this you can see our robot is rotating as we expect so if you zoom into Wheels you can see the wheels are turning and this one's turning forward and the other one is also turning forward but you can see this one is turning a little bit slower as expected since we're actually having a rotate so you would expect one wheel to be turning faster than the other wheel so from the top view you

can see that it's starting to uh curve making an arc and this is exactly what we expect the motion to be and you could play around with the different speeds to see if it could go faster you could also make it negative To Go reverse you could change the direction of the angular Z to make it turn left or right based on whatever you want so you could play with that and see what suits your needs but you can see right now that everything is up and running um you could also try adjusting the Mew

variable and then you'll see that the front wheels will have issues because of the friction U but you can see right now it's just not rotating it's just sliding but that's okay uh visually uh you can see that because it's still performing like we wanted to then it's pretty good now you may be interested in visualizing the data and plot juggler so I have a video that I um posted recently here so you can go ahead and check it out for some of the details but what we're going to do is just show you how

to visualize some of the data that we've already set up so if you go to layout I've already saved a layout and again I went over this in that video but once you upload the layout you get to see that um we have some of our topics that we've chosen here the Odom and the joint States I'm going to hit okay and you can see that our plus will be showing so here you can see these are our wheel veloc ities you can see that um the green is the the called body to back left

and the Orange is the body to back right and you can see that there's a difference in terms of the magnitude so the orange one is higher and the the green one is lower which is as we expect so that just means that one wheel is turning faster than the other and this right here is a XY plot that I've also generated so this is showing you the trajectory of the mobile robot in this case our cyber truck as is moving along in a circle so this might be useful if you want to do some

analysis on the trajectory in this video we're going to be using our cybertruck and the slam toolbox in Ross to create a map of our environment by driving the robot around we'll be talking about what is slam how to install the slam toolbox how to set up gazebo world how to run slam toolbox and set up the parameters how to set up Arvis settings how to run the Arvis from the config file how to handle this special error that you might be be seeing called No map received and how to create the map with slam

toolbox how to save the map and finally how to load the map all right so what is slam slam stands for simultaneous localization and mapping so that's the acronym for slam and the main question that we want to answer is where am I so to figure this out you could break it up into two parts first we want to find the map of the environment that's the mapping part and the second part is location of the robot relative to the map so that is consider localization so with simultaneous localization and mapping we're actually doing both

of these things at the same time so there are three main frames that we want to consider in Ross when we're dealing with slam and the first one we want to look at is Odom so with the Odom frame is a fixed frame of the robot starting location and the main properties of that is that it's continuous and smooth but one of the main issues is that it can drift so drifting means it might go somewhere over time and it's not any longer accurate no longer accurate okay and then here we have the map map

is a fixed frame with a robot starting location and it's the map's origin so some properties of that is is discreet but the thing is is that it's relatively accurate so that's why you want to rely on this map and the base is the location of the robot so in the perfect world we can map the environment using our our odometry which is our wheel encoder and our lar Point Cloud so as the robot moves we'll collect the point cloud and the Odom and map frames would be coincident so if we look at this image

here this is a perfect scenario okay so if you imagine the robot is moving around so here it moves to like position one and it gathers this uh Point cloud data it moves to position two and then it gathers this point cloud data here and then it moves to position three and gathers this point cloud data here so you can see that the point cloud data perfectly aligns with the wall so here it's inside a rectangular space and it's going to be perfectly capturing the data so in the perfect world The Odom and map frame

is going to be coincident and it's going to be at the location where the robot started but in practice you know drifting happen so what we're actually going to be seeing is something like this okay so in practice we're going to be seeing drifting so what that means is you know you have your map frame and the robot moves to some position but maybe as it moved there the orientation kind of got lost so instead of capturing instead of having the point cloud map exactly where the wall should be it's maybe slightly at an angle

and again it might move to another position and you can see that it's not perfectly aligned with the wall and it moves to another position and you can see it's off again so this is the cause of drifting and what ends up happening is that let's say by the time the robot returns home the Odom frame is actually going to be in some position that's no longer accurate okay so you can see that it's been drifted away from the map frame and it's going to be causing some error so this is why we have slam

slam is used to correct for these errors and uncertainties and by using the lar and odometry and computer vision optimization methods we can more accurately estimate the pose of the robot so the whole slam pipeline you could think about it as uh these steps here so you have sensor data as the input and then you could think of it as breaking it up into two parts you have the front end and back end and the output is going to be your map okay so the front end is going to be things like feature extraction um

you have things like the different fast sift orb and serve I have videos on that you can go ahead and check it out you have the feature matching part which I also talked about in some of my open CV videos and then here you have the determined transform of the camera or the robot because typically the camera is going to be on top of the robot and next up we have the back end so the back end is the key part is the optimization so you gather all of these things so previously for the front

end we're talking about consecutive frames so between like each segment something happens but the back end looks at everything all together and tries to minimize the error of everything so there's methods like the extended common filter particle filters Le squares the graph post graph optimization typically they like to put it in graphs for um data structure because it's more efficient so once you have that the idea is you want to break up the map into discrete cells and the slam algorithm could detect determine the probability of the cell you know being either occupied or empty

so occupied is going to be black and empty is going to be white so here is an example of the map that we'll be creating later and you can see the black spots um as you can see black was occupied right so black is usually going to be some obstacle so in this case our obstacle is our wall so black is like the estimation of where the wall could be and all the white spaces is the parts that is just free space okay okay so how do you install the slam toolbox so to install the

slam toolbox is pretty simple you want to run these two commands the pseudo app get update and pseudo apps install Ross and your drro plus a slam toolbox then everything should be up and ready okay so now we're going to talk about how to set up the lar for slam toolbox so we will discuss how to add the lar in your zacro file so first we need to add the lar link and transform it so that the lar is in the correct orientation otherwise the object will appear to be on the left when it should

be in front so here's the part you need to add to your zacro file we have a link name called lar link and we're going to be combining the head and lar link together with a joint called called head to Liar link okay and for our rotation we're going to be adding a pi Hales here and later we need to add something else for the sensor message so you want to make sure that the message type is the correct type and that type is called sensor message and it's going to be laser scan so you

want to add this part two to your zacro file for your plug-in so previously we've had this but the part you need to add is this part the output type is going to be the sensor message laser scan because the slam toolbox is expecting this type specifically okay so now we're going to talk about how to set up gazebo world for slam toolbox but before slam toolbox can work properly in Arvis we need to make sure that the Gazebo world has objects in it otherwise it won't work the lar needs to detect objects and pass

that data to slam toolbox so here is a part that we want to add inside of our launch file so this is how you could start up gazebo this is how we started up previously but the main things that we've added is this part here which is a launch argument so inside of our launch argument uh we have a config and we've created a world file called walls. world and another thing that we've added is the use sim time is true which we're adding for everything else inside of our launch file so that's the main

thing we've uh updated and to actually see the world um it's called we're going to be going through the steps where we CD Cen build and then source and then run our launch file so I'm GNA go ahead and do that so here I've already built it I'm just going to run it and this is going to go through the process of everything so I've already got Arvest set up we'll go into that later but here you can see our world so this is our world that we've set up and you can see the walls

okay okay so now we're going to talk about how to run slam toolbox and set up the parameters so first we need a copy of our parameter file which could be found from the following path here and then there's a couple of things to note there's two terms that's being use is online and asynchronous so online just means it's running in real time and asynchronous just means that it's going to process the latest data and then everything's going to be running in parallel but some of the data might be skipped and we need to make

sure that the parameters in the parameter file which is called mapper prams online async doyl is set up correctly so the main parameters that we're going to be touching are these part the Odom frame which we're calling Odom map frame map uh base frame which I'm calling body here and then our scan topic which is a default name that the laser lar that we're using which is lar out so some applications may require a robot footprint frame which we're not using but you could go ahead and look that up if you need it okay so

couple ways of uh to run your slam toolbox to run it from the terminal you go ahead and run this command here Ros to launch launch slam toolbox online async launch. and you want to pass in your parameter file one thing you want to note is it's called slam params file and just specify the location of the parameter file and say use sim time is true and you could do the same thing using it from the launch file using this uh these few lines of code here and then when we start Arvis later we should

be able to see if our slam is working properly okay so now we're going to talk about how to set up your Arvis settings so here in uh the global options what we want to do is make sure the fix frame is set to the map frame and then the next up we want to look at the laser scan so for the laser scan setting you want to make sure the topic is slide or out here and next up you see the map so the map we have map here for the topic make sure that's

set correctly and then we scroll down a little bit uh we have the robot model here and you want to make sure you have the robot description and then lastly we have our transforms which you could turn on and off as needed and make sure you set all of these in order because it's going to matter okay so how do you actually start Arvis from the config file for slam toolbox so in Arvis we've went ahead and saved our Arvis file into this path here and then if you want to run Arvis from the launch

file you could go ahead and add this part to your launch file okay so how do you handle the no map received error from slam toolbox you may see the following error here no map received and at the transform it says could not transform from blank to Odon so the list of things you want to make sure is that all of these things are happening in order because uh especially if you're not using a launch file then you may do these steps out of sequence so something uh some part of the pipeline might be expecting

data and it's not seeing it so first off make sure gazebo is running and then there's objects in gazebo once you have all that you could run your slam boox toolbox so this will actually start up the map topic and um later on when you actually open up Arvis then you could finally set the fix frame to map map should show up only after the slam toolbox is showing and then the laser scan should only be able to pick up stuff once there's objects in gazebo and then finally you can set up the maps to

start visualizing the map okay so hopefully if you follow all these steps uh in the right order you shouldn't see any errors that says no map received okay so how do you create the map with slam toolbox so to create the map we need to drive the robot around and to do that we're going to be using the teleop but before that you need to make sure that the diff Drive controller. yo file has the used stamp velocity here set to false so go ahead and update that file and then now we can run the

teleop to move the robot around uh we're going to be using the Ross to run teleop twist keyboard and then we're going to be using the command velocity called the div Drive based controller command velocity unstamped okay so it's only going to work with unstamped so make sure that you have that set up correctly so now that we have that set up we go ahead and open up our terminal and we're going to have one terminal that's going to run our launch file and then in our other terminal we're going to and run the tele

op so this tele op will let us use our keyboard here so once I run that I'm going to pull this over to another window here and then here we're going to set up the Gazebo and Arvis side by side so this will allow us to compare it and see what's happening so now we're just going to go ahead and drive the robot around so here you can see that initially it's facing like the X Direction so we're just going to rotate this so it's facing in a similar Direction okay so now that you can

see both uh both views we could go ahead and start driving it around okay so now that we finished driving around you can see that it created a pretty good map is pretty Square you can see there's some parts that's not quite perfect um the over the overall orientation is a little bit rotated but I would say relative to where the robot is uh if you look here it's actually pretty accurate in terms of the relative location so if we take a look at the actual frames we could come here to TF and we could

actually show it but here you can see that if we zoom in um you can see that here is the here is the Odom frame so you can see Odom drifted quite a bit I think it seems to be more sensitive to uh when you're actually rotating and then here is the map frame which is uh the center of our map but in terms of the overall map shape and the relative location of the robot relative to the map that's generated I think it did a pretty good job you can see that there's some parts

here that's uh not not quite good like on the right but um I think overall it's I think it does the job okay so how do you actually save this map that you want and then reuse it later so what you want to do is come here to panel and then add new panel then here you're going to see the slam toolbox plugin and then you hit okay and then here down here you could actually just uh write the name of your map so you could uh write map as your name then you could hit

the save map and then serialize map and then once you do that um it's going to go ahead and save it to your file uh folder in your root Ross 2 so if we take a look here in our file structure you're going to see that uh it should have created these files here called map. dat map. uh PGM map. pocra and map. yaml so the main thing to note about these files is that uh there's different functions the PGM is a cell occupancy data the yo is a grid resolution and origin location the stuff

like that the map. data is a data from mapping and the post graph is for some of the optimization stuff so for me I'm going to be putting into the config folder uh but for you you could put it wherever you want okay so now that we've saved our map how do you load the map from the slam toolbox so first thing you got to do is update your maper prams online in AC sync. yo file so you want to make sure you have these two lines one is a file path and then another one

is a map start at Doc and set to true and after that it's pretty simple all you need to do is rebuild and then run your launch file again so if we go ahead and run this you should see that our Arvis will now have the map automatic when we start it up so I'm going to minimize the Gazebo and you can see here that our map is already up and ready okay so now you could use this for localization with like the nav toolbox and things like that in this Ross 2 navigation tutorial I

will show you how to use nav 2 to navigate to your mobile robot in Arvis so if you haven't seen this video already go ahead and check it out the slam toolbox simulation tutorial we will be building on top of that so it might be useful for you to check it out but in this video our main focus will be the nav 2 toolbox so we will go over how to install the nav 2 toolbox go over how to remap the command velocity for nav 2 go over how to start the mobile robot simulation and

view the slam map talk about how to start nav 2 go over how to set up the cost map in Arvis and finally go over how to navigate your mobile robot with nav 2 in Ross okay so to install nav 2 it's pretty simple you want to run this command here PSE sudo app install Ross and your Dr version I'm using humble but here you have two main things you have the navigation 2 and then the nav to bring up so once you run this command you should be up and ready to go okay so

now I want to talk a little bit about remapping so when you use nav 2 by default everything is set up to work with command velocity so one of the things to note is that uh if you have a robot set up sometimes by default you might be using a different command velocity so in our case in our previous slam tutorial video we were actually using the diff cont command velocity unstamped so in order for nav 2 to work you actually need to remap it so I go over step by step how to do that

in more detail in uh one of my documents here the how to remap command velocity for nav 2md it's on my website so you could go ahead and check it out I'm going to link it in my comments below but the main idea is you want to go into the launch file and then edit that so I'll I'll lay out the details there so go ahead and check it out so now we're going to talk about how to start our mobile robot simulation and view the slam map so go ahead and CD into your folder

source and then we want to run this command here the Ros to launch wheel robot gazebo launch. py so this will start up our Arvis and gazebo so I've already sourced it and everything so I'm just going to run the command to start it and once you do that you're going to see your gazebo show up so this we covered in our slam tutorial on how to set that up but our main focus will be Arvis here so here you can see we can view our simulation in Arvis we've created our map previously in our

slam tutorial so um you could try to play around with creating different Maps if you don't like how your map looks like um so it's going to be a little bit of trial and error so here you can see that all of our map has been loaded and then our next step will be to uh set up some of the cost map run our nav 2 and then we could actually start navigating it but this will be the foundation to get everything started so make sure you can do this properly so how do you start

nav 2 so what you want to do is um we're going to CD and then Source inside another terminal and then we're going to be running our Ross to launch uh the nav to bring up Navigation launch. piy and we want to use um the simulation time to set to true because we're using a simulation and we want the other stuff to be in sync the other notes that we have set up so if you go ahead and run this this will start up the navigation and this will allow us to proceed in our next

step later which will allow us to set up the cost map but to actually see the cost map you need to make sure that we run this launch file so what this does here is it'll tie some of the commands together to our actual robot which we'll see later on when we're actually trying to navigate it so right now we see our typical map but what we're going to go over now is how to see the cost map in Arvis so to actually see your cost map in Arvis you want to click the add button

um in the panel under your displays here and there's a couple ways to get there but usually I like to go to the buy topic Tab and then you're going to see here the global cost map cost map and then the map so once you see the map you want to click okay so you can see that there's these regions that pop up so um the gray regions are what it expects to be obstacles and then the white is not obstacles so you can see has a general um outline of the square that we previously

created which represents the walls and if you want to change the color scheme what you can do is come down here and then click on cost map this will give you a color view which is a little bit easier to see so that's just a optional setting that you can play with the final step is to navigate your mobile robots using nav 2 so now that we have set up everything we have our map visible we set up our cost map and we also launched our nav 2 with the nav 2 bring up this will

actually be connecting the command velocity to the actual robots and you if you've done your remapping properly which I've laid out in my website then your robot should be actually connected cuz otherwise if the command velocity is not mapped properly then your robot will not move so you want to go up here and click on the 2D gold pose and then once you do that you could click a starting point and then you could drag the arrow to where you want it to go and once you drag it to the direction that you want to

go you release your mouse and then your robot will start navigating so what this is actually doing is it's sending um pretty much the commands the commands to navigate the robot so you have like some direction and some angle and sometimes the robot freaks out as you can see here so um you could look into details of why that's happening but you can see as it keeps moving it eventually uh kind of steadies steadies a little bit in terms of the direction and it finally settles to a final Direction so this a little bit glitchy

probably cu the map is not perfect but you could play around with that and see if generating a new map might might help so hopefully you learned a lot in this rosu course if you want more material on robotics AI machine learning controls or computer vision make sure to check out my YouTube channel I have a bunch of playlists and videos on various topics that I think will be very useful for you so go ahead and check it out make sure to subscribe and I'll see you later