Open Univesp l Grisha Liebel

[Music] e for right correct are you ready gisha I think so yeah okay so first of all thank you so much for being here with us okay and I know that you understand a little bit of Portuguese you speak Spanish and everything so now it's up to you so we are eager over here to see what you have to show us thank you so much okay let's see what I'll show you then okay you're welcome uh as you see I'm I'm just having a wide screen I'll draw a little bit and I'll just talk about

three different topics that's what we discussed beforehand and these are topics that uh are primarily in education but I'll I'll I'm doing research on them I'm using them in my classroom so it's really a mix of different things and it's uh essentially three different things I'll be talking about and some of them are a bit connected a lot of them are not really uh the first one is on the topic of Project based learning and you see my handwriting is not fantastic it's a lot worse than my English uh so how about using projects in

the classroom in different courses the other topic is about teaching in a team so in contrast to the classical lectures where you have one person uh giving courses giving lectures with multiple people and and how you can do that in a in a good way uh and we have both used this in the classroom in the past but we've also done some research on the topic and then finally I'll talk a bit about work that I'm actually mainly at the moment doing on neurodiversity so dealing with students that have different uh cognitive function and how

to include them maybe how to better use their strength and so on uh on the last topic I think I have to say that as I said this is ongoing work so there's a good chance that might be some in the room that are actually better at that than I am so it's really just an an Insight on uh what I am doing and please interrupt me whenever you feel like if there are any questions it's not I'm not only here to talk all the time so go ahead if you have any questions so Project

based learning uh the first topic is something that we have been using uh I have personally been using for the for over 10 years now uh it's become very very popular in in Europe in the US in different universities but I also have to say that it's not like everyone uses it everywhere that's definitely not the case so we have a lot of these traditional lectures where uh 300 students 400 students and there's just someone lecturing but what we very often uh want to do is we want to have uh groups of students in many

cases that somehow work together uh they work together on one problem and the reason why we want to do that is that is pretty much the reality in whatever you do afterwards there are very little very few professions left where you are alone uh and that's why we want to expose students to the real world and we put them in different groups and there are of course there are lots of different ways methods of doing this uh I'll just since we don't have you know I could speak 90 minutes about just this but just to

to talk a bit about what how I usually approach this um so usually I start with either I start with a very good idea I have so I might have an idea for a project I'm teaching a class for example on software projects and I have an idea this is what I want to do I think this would be a really nice project to do in a group or I start from what we call the Learning outcomes so thinking about what do I have to teach the students and how can I put that into the

project so usually I come from one of these two sides and try to figure out what exactly is it that I try to teach the students what should they be doing in the in the project um and based on on these different things based on the idea I'm having based on the learning I try to design what exactly the students should be doing and more much more important what the students should not be doing uh and for for those of you here or the ones watching this uh I think the biggest problem that that I

see when when teachers do Project based learning learning for the first time is that they go too far into the realistic corner so they say we do something very realistic and you throw all the difficult things at them at the same time and the students are just very confused uh so usually you have to have a good idea you have to have an idea what students want to learn and then you be need to be very very careful how you control everything else so just as an example I'm teaching classes on software engineering and I'm

teaching the students for example different methods to do that I teach them things like how to estimate how how they can plan ahead how long things will take and so on what I don't want to teach them in my class is a lot about programming because there are other classes for example where they learn programming where they maybe learn very detailed things uh that's not part of my class so my project usually is very restrictive on how much time they should spend programming what other other teachers what I've seen what others do is very often

that they say let's do a real software project and then all the students do is programming that's all they focus on but it's actually not at all what I want them to learn so I think you know if if I summarize in five minutes if you want to do Project based learning what to look at is what should the students learn but in particular much more important what should they not learn what do you need to take away so that the things don't go too far in One Direction uh so I think this is really

the the balance and then the the other thing is a lot of us teachers Educators we like our traditional lectures because we have a lot of control over what is happening we can design exactly what's happening we can design all the assignments uh in Project based learning you should expect that you have some kind of iterations you should expect some kind of chaos all right the pen is stopping you should expect that the first round things go horribly wrong and you'll have to do it again and you'll have changes so what I often do uh

I do this because my class is on on software and this is how software is developed but usually I try to do some kind of iterative work so the students work for two weeks they deliver something then they work for two weeks and so on uh and the great thing about this framework is well it fits for the software context but also it allows the students to make mistakes they can learn from it they can do things again it's not the classical idea where you do one assignment now you go to the next topic forget

all about the first one but there should be some kind of repetition uh and as a teacher this means you also have a chance to fix things you realize the first two weeks something is horribly wrong and then you fix it in the next round or you make it slightly better uh but in in terms of mindset you need to go a bit away from the idea that everything will be great the first time it will not and that's okay that's also part of here the the real world right things are not always ideal um

just to sum this up of course this is a very quick run through but we have we have done this in a lot of different courses we have done this also a lot with uh with companies for example that they come in and support us and give us problems or talk to the students so it's a very flexible way uh it's also something that has been done internationally between universities that students work with students from other countries because again many projects we have at the moment in software but also in construction in other areas are

international so it's part of the real experience to actually talk to different people and uh have different backgrounds yes a question question in terms of time yeah how long does it take when you start from the beginning up to the end a new project for me to design or for the students to do the whole thing when when they let's let's for you to design okay but then when they start because we have your preparation and then you have the class let's say like this the project itself I would say it depends on how experienced

you are in doing that if you do it the first time it will take you a long a long time to prepare MH uh I think if I do a project in a topic that I'm very familiar with I can easily set that up in two weeks okay so that's not I don't think the preparations are much longer than for having assignments or an exam it's similar thing okay uh and then for students the idea is usually that it should be a prolonged time because again that's more realistic okay The Other Extreme is putting people

in a room for an exam for 40 minutes or so so it it should be prolonged in my case I usually do 10 weeks okay because the our term is 12 weeks okay so there's a little bit of time in the beginning there's a bit in the end and then there's 10 weeks of project work okay yeah but I I think you can do this very different uh I know that for example in the UK they have these Capstone projects that run an entire year okay so you know you can do this probably from four

weeks so you can even do very many it's about around 20 weeks right it's uh yeah that's what we do that's why I asked you because we can do something shorter or longer yeah and uh how many hours per week uh study do they in our case the students take four courses in parallel that's fulltime St okay so they are somewhere to 10 to 12 hours per course per week okay and then in my class there are some hours for lecturing or things like that so I would expect they spend somewhere between 5 to eight

hours a week on the project okay 5 to eight hours a week if they spend the time they should okay thank you always the question um any any more questions on the project based part uh I think there are a lot of there a lot of really good okay a question here in Brazil do you understand I understood that yes okay just go if you don't you ask me okay for so um it's this is a mix also in Germany so for example I'm from Germany originally we have well this is now 15 years ago

I don't know how it's looking now but 15 years ago pretty much all of my courses were tradition I was sitting in the lecture hall I was listening and then I was doing an assignment at home uh and I've seen similar things for example in Spain and and in Portugal I think it's uh one thing is actually just doing it I think you would you'd be surprised how how adaptable a lot of the the young students are uh and the other thing is well maybe calling it scaffolding and this is something in in the univers

context that might be tricky that they might need a lot of support in the beginning to get going that it's just you know a little bit of push there is a teacher or there's someone that says oh have you worked on this uh maybe working with with deliverables or things they have to do that instead of just saying well now you're free go ahead you have certain things that they have to hand in or present or so um and this is of course a problem if you have lots and lots of students how exactly to

do that but I think that those kind of things you need to work with and it's very similar if you if you think about industrial projects uh you also in in a lot of cases it's not just like okay in three years you delivered a building but there are things in between right and that's maybe the way for some people to to make companies work I think it's similar with students but uh it's certainly not easy when you have very large courses I can see that okay let's go to the other side of the question

but uh with the majority of these students prefer this kind of of teaching you know do you think they they like this how they generally I think most students like it yes uh my own view since most of my education was in Germany and then I came to Sweden where we did this model pretty much all of the time I think a mix is good as I said in the beginning it's not like you have to do this in every single course I think there are courses I'm not 100% sure how I would do this

in a class on calculus for example so there are cases where tradition additional lecturing might make a lot of sense or maybe a few projects there are cases like software development where it's really important to have these kind of experiences so I think that the easy answer is always it depends I think you need a mix of these and students are different the majority likes these I would say yeah how do you evaluate the students like they have to inter deliver something or yeah we do uh let's use a different slide for that I usually

use rubric based evaluation so that they're different kind of criteria they have to fulfill and usually this is quite a mix some of these have levels that talk about I don't know deliver something deliver for example a plan for your project uh deliver I don't know set up some set up some kind of software write tests for your software again this is now in the software context uh and then some of these checks are just have they done it or not is it there is it not there very basic others can be a bit more

qualitative so for example uh well they have delivered tests for the software and you know they're covering this much of the code or so so you can have some assessments that may be more quantitative or more precise and others are more past fail and usually for each of these iterations that I have shown you for each of these uh little blocks here I I have one of these rubrics that said this is what you have to deliver and this is roughly how I will grade you uh my experience is that less is more so again

a a typical reaction when you come from classical lecturing is that you try to be overly precise that you go okay I have to grade them exactly on zero to 100% and how exactly do that uh what I usually do is I try to be very open uh and for me it's not the grading on the project is not the most important thing there should be some kind of grading but it's not I don't care that it's very precise it's more about making sure they go through it and they have the experience um rather than

very being very very precise in the grading uh and again that's why you need to have a mix some courses maybe you need to be more precise in my course I have the project but I also have an exam at the end so I can check them on some other things but the project you know I try to let go a little bit and say I make sure everyone works but I don't make sure everything has to be perfect or graded yeah uh I have a question um how many students do you usually deal with

um I have done these kind of courses from 20 to 250 students 200 yep 250 I have courses with 300 uh tell us a little bit about it how is to do this with 200 students I'm curious yes uh so in the courses with 200 students I have a few Tas that help me a few assistants but we talking five it's not 20 or so it's it's five people uh what I do is I have a lot of these kind of things I have a lot of checkpoints that just is it there is it not

there yes no very easy to grade very quick so I try to reduce the effort there I also try since this is a class on on software development we use a lot of electronic tools where I can see if a student has delivered something or not um so that in case there are problems in the group for example the group complains that someone is not working I can actually check um the other thing I'm using which I haven't discussed before is peer evaluation so the students in the group they need to assess whether the others

have delivered things and whether it was okay whether they have have talk to each other and so on uh so that the effort is not all on on us for for evaluating um and then in person we actually only have one meeting per week where where my Tas meet the different groups and it's they they play the role they play a bit the manager or customer role they come in now and then and say okay how's the progress what do you do for next week and so on uh so that's that's how we deal with

200 students uh I mean these classes are really really nice when you have 20 students and you can do it all yourself but it is possible to scale this uh to some extent I think you need to you need to let go more of the grading and make sure there are things that are very easy to grade or even automatic um that's how you can do that with so many students one more question uh how many people for each group um five to nine oh that's great thank you Lely it's it's almost like ours yeah

yeah so yes very similar very similar yeah yeah it's it is an uh it is a fairly International way of doing things I think I've seen very similar classes yeah I have a question uh the questions were about the students but I have a question about the instructors yes because when the instructor uh starts working with peer instruction or uh project project based learning or other Active Learning techniques flip the classroom I I believe it's difficult for the instructor to change the way he teaches because the workload is is great and the traditional lecture is

in his head so he he must change the way so do we have an suggestion how to change the mind of the instructor to use this kind of techniques it's uh it's very good that you asked this because that's my next topic great so I'll just go there are several ways but one of them that I personally uh have worked with as a PhD student actually not as a professor so I'm a bit on the other side is team teaching as a way of changing things uh so I'll just jump over there if you still

have the question we can talk about it again later but I think it's definitely one way of of addressing uh things so what we have done in certain classes and I I'll talk more about that is to teach in the classroom so we're standing in the classroom why can I not draw here with two people okay the legs are disappearing again that's fine um and we have two people talking together at the same time sometimes across each other uh why do we do this the reason we do this is we have certain classes for example

I had classes on on software modeling where we draw diagrams uh we for example design our systems how they are connected and things like that and this is a typical example of a topic where there is usually not a lot of feedback when you program the computer tells you if it works or not in some other cases if you may be you know if if you have an engineering class where you can do simulation the computer tells you whether it's okay or not in math you sometimes have that in a lot of topics like this

one you draw something you can always draw something it's not a problem so how do I know whether it's good or not good is a very difficult discussion and a lot of people say this is actually more like art it's it's an art of designing software it's not a it's not a science uh and this kind of model of having multiple people that actually discuss what is going on in the class is a bit closer to what you have in the past in in the apprenticeships so for example if you learn to paint you do

that by going into a workshop and looking at what the the master is doing and then you do something and the master is commenting on okay this is maybe not the way of doing it uh and this is exactly what we have here we have one teacher that is basically the the the professor or the the more experienced person and we have someone who is a bit more like The Apprentice uh and the professor starts pretty much doing a lecture just okay we do this this is the topic for today and The Apprentice pretty much

plays the student that has no idea what's going on and starts asking asking question starts going okay what did you do now why did you draw a line here does this make sense and they start having a discussion it's not necessarily the the professor always telling them ah this is how it works this is how it wrong but they start having really a discussion how you could do it differently and so on um and the reason we do this is for a lot of different ways first The Apprentice down here is pretty much playing the

the role of the students and a lot of times he or she is actually asking questions that a lot of the students have in their heads and maybe they don't want to ask or you know they think they're stupid and everyone knows it so a lot of the questions that that person asks are actually the things that maybe should be discussed um the professor of course makes mistakes as well so maybe that's when the other person notices uh and also it shows through the discussion you you find a way to discuss Alternatives because other wi

especially when it's when it's an art when it's not very exact uh the students might get the impression that however the professor does it it's exactly the one way to do it there's no other option so by discussing this you get a bit of a broader picture you get Alternatives and you actually verbalize what's going on in the head because if I'm doing math on the board there's a lot going on in my head that maybe I'm not telling I'm not saying and maybe it's even automatic so by having a discussion in the classroom Things

become clearer um this is this helps a lot with understanding in the class and some students really like it some absolutely hate it because they really like the idea that there's one way of solving things and why do we have to have a discussion uh but it also helps and this is now where we get into how to change the uh the mindset maybe the discussions usually continue outside of the class as well between these two people uh so what we did in our setup is that we had the professor who is responsible for the

course and the we had a PhD student who was helping who was like a TA in the course and usually when we were done in class we kept discussing that well maybe we should be explaining this in a different way maybe we can do this kind of assignment and uh our view on this was that it helps a lot with this kind of being stuck in one way of teaching uh it's not only about the way of teaching whether we do lectures or projects it's also what I think most of us teachers have seen when

we teach a course three four times it becomes very hard to change anything uh and that's you know if if this person here the The Apprentice if they are a little bit either very open or they're a bit more experienced they start questioning things and say he should we really be doing this and that way is that a good idea maybe we should change it uh so this has originally this has kind of happened accidentally we we just we started doing this with two people that were just discussing they like to discuss a lot uh

but our experience is really that this helps a lot with developing courses over time uh and this might also be a good way for for example developing project courses to actually put for example one person uh in the room who is more the the expert on the topic and maybe someone else who is a bit more experienced with projects and you start discussing things uh I think the important thing about this topic is that there is this discussion going on uh there are there is one setup of doing team teaching where where it means one

person is doing one lecture and the other person is doing another lecture uh you often for example see that in medicine you have a medicine course and one person is doing the lecture on Cardiology the other one is doing it on the nervous system is different areas uh that's good for other reasons but this setup that that we're using is really on teaching everything together and discussing the course together so that the quality becomes better overall um so that's an approach we have used again this is nothing that works in every single class yeah it

might work for some topics uh and I don't know whether it answers your question but I think it's one way that can actually help changing People a bit let me microphone question here uh a provocative question yeah sometimes uh let's start with computer science we change from the traditional way of develop systems to scr and we don't have any scr leader we don't have it's just working in a team yes uh when uh we listen some critics to to the to the school uh the the professor used to be the owner of the class yeah

H don't you think this is kind of trying to uh broke this ownership to work more in a in a group we are going to something more like a scr than that that with one person controlling everything it is uh yeah it's an interesting question and I think we earlier discussed a bit uh hierarchy and and kind of being formal what we observed or we haven't tried this with a lot of different people what I think our theory is that you need to have someone who is in charge the professor who is very open to

criticism you cannot have this you know this traditional lecture who is like I am in charge and the way I do it is right so of course you but that I think that's true for every kind of change you need someone who is open enough that there can be someone like a young PhD student that says well I think this is wrong we should maybe do this differently uh so it I think it requires that if you're really uncomfortable with that as a professor then maybe you should not try that it will not be good

yeah absolutely and of course just like with with different trends for example in in software development I think it's a balance you might realize that at some point you have gone too far and maybe you need to have some more control what we often said is that you know the professor needs to be open to criticism but also the professor needs to have kind of a script an idea how the lecture will go so that it doesn't just become a strange discussion and nothing else happens but there needs to be some kind of plan okay

this is what we'll do this is the examples we will discuss and then you can you can improvise a bit yeah [Music] thank [Music] for [Music] okay there I need a bit of translation I think okay it's a long one no we just talking about about how you evaluate students and then it the main question is how can you do if first if you present the rubrics in first and if you have an any kind of agreement with the students and uh if you don't have lots of resources we do have lot like a classroom

with 40 students for example how do you deal with that yeah uh so one one thing is these rubrics are open so we always publish them to the students before so they know what they what they are evaluated on and of course that always comes with the risk that they just optimize for exactly what we grade on I think that's what you have to live with uh that's why we very often we just grade on what we want them to do we don't maybe care so much about all the details of how well they do

it uh regarding the you know scaling or how do we deal with a lot of students I think I would try to at least automate part of these things so as I said earlier for example we use systems where we see every single contribution uh and I think one way would be to to actually automate how much contributions do there need to be and those kind of things that we actually don't have to manually do anything but it could be graded in an automatic way again it's never perfect it's not a great way of grading

but that's the the tradeoff of having lots of students uh so I I think I would really try to to automate a large part of the grading and then maybe really have a few things that you really care about but those you can really grade manually you can manage uh but I haven't tried it so I would have to see maybe we can have an experience together yeah that's the beginning okay then I just very quickly jump to the last topic I think and that's fine because that's ongoing works so I can just talk a

bit about what we have done um so this is a slightly different angle because it's not about how we teach it's about how we C uh support certain types of students and uh that's in our case neurod Divergent students so students that have uh conditions like autism spectrum disorder ADHD uh attention deficit hypers activity disorder or dyslexia for example so issues with uh usually reading reading comprehension uh and what I'm doing here is probably not great for them by the way but there we have just started a lot of work in in how we actually

help with our courses and we have done some initial work that is very simple because the main idea is how can we do it without a lot of resources we don't have a lot of support for for these kind of people so it's not like I have five people that can help me with with different things so it's up to me and what I have done uh in in an initial round is first of all that we have changed the the look the presentation the style of our slides of our assignment texts and material that

that we offer uh and we have done that following a lot of different guidelines that exist for these conditions and it's usually things like you know having a different background white is not fantastic fantastic uh having different fonts different font size more space and and things like that uh also when it comes to our assignments for example we have really tried to carefully go through them a lot of times and make sure we have them in the right wording it's very clear what you have to do and improve the clarity basically because for example uh

that's something people with ADHD very often struggle with you give them a lot of different instructions and they're just lost where to start uh this is difficult because it very much clashes with this idea here that you have very open problems uh so how we cannot be very specific because we want to leave it very open uh and this is something we we try to work with but we don't have a great solution for that right now but we have mainly worked on this level and so far this has been really successful so the students

at at rikic University have been quite positive about that uh we have if in terms of numbers we have about 10% of students that have one of these conditions so it's quite a large number even though the one does not want to stay on here uh whatever 10% um so they were quite positive and now we are we're working on the next rounds of this so we we're trying to look at can we make our organization better how to find materials how to maybe guide people through the different courses uh so it's it's really at

an initial stage but the focus is very much on how do we do this in a simple way because again we we have talked for the different topics we have talked about how do we convince people to do things and of course you can either have a department or a group that really focuses on having this support but that's not what we have so we have to convince individual teachers this is how you can make your class better more accessible more inclusive uh and this is roughly where we are at the moment um but we're

looking at different ways how to improve presentation organization and and topics like that but it's not you know it's not like we are building amazing tools at the moment it's very basic support and it's a lot of it is also just about awareness making actually making these people more seen that they exist and that the other students also acknowledge that uh and that the teachers understand that and that's pretty much the level where where we are so that's definitely a topic where we can still learn a lot and I think you at unas also doing

a lot already in in that direction which uh we are not very good at so uh ongoing work any questions on this or on anything I've been talking about or yeah lots of questions lots of questions um how about evaluation in these cases if if you if you perform perform the same evaluation considering neurodiversity yeah uh also i' would like to know I would like to ask you to tell us a case yeah a difficult case so evaluation first maybe um currently according to our University rules uh these students the only official support they get

is that they get longer time for the exams so there's kind of a guaranteed uh time I think is 25% usually that you have more time that's currently what the university does then on the lecturer side we have a lot of flexibility what we can do uh and I think I would say at the moment we actually don't do a lot so we there is not much difference in how we evaluate partially because we're only starting to learn about this uh and also because it's very difficult of course depend ending on the condition and the

the detailed severity of the condition it you know there there's a huge difference in in how they behave how they perform certain tasks uh I mean some cases maybe if I do software testing someone with autism might be better so should I evaluate them harder or not so I think we're only just learning these things but we're having a lot of discussions in the context of for example the projects uh and other kind of skills that that we know are often difficult for for some of these students so for example a lot of people with

Autism have issues in group work because of communication social skills uh and there we are we are having really a discussion about should we change the setup for them or not because we know they have difficulties but we also know they'll have to deal with it in the industry later so it's a it's a tricky discussion one I can tell you one case not from us but from another University in Iceland uh because I've recently talked to a lot of the different study counselors in the universities what they do when they have project courses with

autistic students is they still have to do the project but they usually get a smaller setup so instead of being in a group of five six people they get paired up with one other student and then they just work together with them uh and their experience is that this works extremely well and usually these two form really really strong connections uh and that's usually one autistic person and one non-autistic person and that seems to work so it's it's a good example maybe of of some kind of Middle Way um but it's something we we keep

discussing concrete cases uh I think in the recent years it's quite difficult because of covid so I don't have very direct interaction but you know I can tell you about a couple of of difficulties that come up maybe uh one very interesting case uh that we looked at well we we looked at what what we did the kind of changes in the slides and so on but we also asked what else can we do we just asked the students and we had one student that said uh Co is is really nice because I can be

at home I can focus on the lectures I can it's silent all is nice another student said Co is really difficult for me because when I'm at home my brain tells me I have to relax uh so I think it's two examples of that that show how difficult this is and that know that we will we will probably not find overall solutions that solve all of this but it's more about trying to understand what we need to offer or how flexible we need to be uh so there's no there's no easy way thank you thank

you more questions okay I need so he's asking about if you evaluate or not the social show skills when you have a group you have the deliveries and then you have the project itself but how do they do if they are participating the mostly behavior and yeah you evaluate not only the project the result no we don't it's soft skills I'm so sorry yeah so there is a soft skills are difficult we so so social skills okay yeah no but but even even that I mean even social skills right because you know some people are

maybe better than others in in communicating is that something we should grade them on or not I think what we try to do uh so quick answer is we don't evaluate them on the the longer answer is we try to support them in in developing that so for instance I mentioned earlier that we do this kind of peer evaluation that the students the students grade each other and they say this student has participated and we don't really we we don't really use this to grade we don't say well everyone says you are not working so

you fail but when we see that something is going on we see the group says there is a person that does not participate we actually sit down with the group and we discuss these issues uh and that's also there was one question earlier of how how we deal with 200 students I think that's the other situation where we actually go into the class and we sit down with the groups and we we do these these kind of things in Sweden where I was before we had in parallel to a lot of the project courses there

was a a mandatory course where where they were teaching and and learning a lot about social and soft skills uh so that's another way I think we can do a lot better still in supporting this I'm not sure grading or evaluating is the best way but I think there should be more support at least uh and that's that's also where we have a long way to go okay that's perfect more questions do you think this could be a trend not to focus on the content but focus in soft skills and yes so I have I

I haven't mentioned this here but currently I am uh I'm evaluating one of our Bachelor programs and as a part of that I'm I'm interviewing a lot of students and I'm interviewing a lot of companies in Iceland and the trend I see in Iceland and I don't know how that is here but it's I know that it's Sim similar in in for example in Sweden is the companies are looking for people that know the basic skills and they know some of you know the principles a bit of the theory and they are really good at

working in teams I very rarely hear companies that say we want them to know all these technical different things all the details no they want the basics they want people that are flexible and easy to integrate in the work life so I think it is getting more and more important also of course now we we are discussing a lot things like chat GPT and so on I think the the very technical skills will get less important other things will get a lot more important so I definitely see see a trend there uh again I Believe

In the Mix I don't think we should stop completely evaluating some of the more technical deep skills but currently we're maybe too much on the technical side and we need to get a bit more to the non-technical social team side yeah yeah lovely perfect questions no so uh I'll just go in Portuguese for a while sure for for [Music] for for foree SK [Laughter] e IND for for for for and our teacher let's say like that so thank you so much it was really lovely and precious uh the moment we could interact was not a

lecture because we interacted so um it's a pleasure to have you here we hope you have enjoyed your stay at Univesity thank you so much thank you thank [Applause] [Music] you [Music] oh