Evolution of software architecture with the co-creator of UML (Grady Booch)

the entire history of software engineering is one of rising levels of abstraction so what we're seeing here is the rise of another level of abstractions which gives us all these extraordinarily powerful Frameworks from which I can build systems and which as I alluded to the architectural decisions that were front and center for us back then are now embodied in these so now becomes a decision what cloud service do I use what messaging system do I use what platform do I use that's the decision which has a lot of economic decisions and not just software kinds

of decisions associated with it so I think the role of the architect in effect has changed because now I'm dealing with systemic problems not just software problems themselves grdb is a trailblazer in software engineering he built his first computer 57 years ago at just the age of 12 and is known for his decades long work in advancing the field of software engineering and software architecture he is the co-author ofl and has originated a term and practice of object-oriented anal is and design he's an IBM fellow an ACM fellow and has been awarded several other prestigious

awards for his work in software architecture he's the author of six books and more than 100 technical papers on software engineering in this conversation we cover the first two Golden Ages of a software engineering how uml was created and why gr disagrees with how it has evolved since version 1.0 how the practice of software architecture has changed over time Grady's views on large language models interesting stories like how Grady was offered to be micros Micosoft Chief Architect but said no to Bill Gates and a lot more if you enjoy the show please subscribe to the

podcast on any podcast platform and on YouTube it's safe to say I'm talking with a living legend in the field of software engineering so welcome to the podcast emphasis on living I'm not done yet yes absolutely so to kickoff you're a chief scientist at IBM that's a pretty fancy title what does it mean and what do you do I'm curious to know well there was a time that my business card said I was a free radical but upper management didn't like that so I had to find something a little bit more tame um actually the

more important titleposition is that of fellow there are I think 68 of us still active no 89 of us still active and this has been out of 350 or thereabouts fellows throughout the history of IBM so we're a fairly Rare Breed uh I was made a fellow upon the acquisition ition of our company rational software back in 2003 and the great thing about being a fellow is it's it's rather like having tenure meaning we trust you done good things we want you to continue doing things let's give you the degrees of freedom to do that

and so as a fellow and with my focus upon first software engineering and then later upon AI I am given a lot of degrees of freedom to pursue what I think makes a lot of sense to trade to as Alan Kay would say invent the future so in my journey from starting at 2003 I first stayed with the rational division but then quickly moved over to research because IBM's bureaucracy realized I was a person who worried about the next five to 10 years not the next quarter and indeed my very early work in research was

looking at finding ways to automate the discovery of patterns within Legacy software systems this is something we were doing pre neural network days and it was was interesting trying to see if we could discern the design patterns from the gang of foreign elsewhere it never went anywhere because it was a hard problem and so I then began to in the architecture sense of things I worked with a lot of customers and actually had been doing that for decades where I'd be parachuted in the customer would say come help me Mr Wizard uh with this particular

architectural problem and the exciting thing about it for me is that for many decades I was engaged in projects across every conceivable domain and I'll pause there to say that roughly around the turn of 2010ish is when I began to be drawn back into the space of AI So when you say you worked on Legacy systems what does a legacy system mean to you well the moment you write a line of code it becomes a legacy system until you throw it away so all code to some degree is a legacy system Facebook is a legacy

domain Google is Legacy uh Heavens even uh open AI has a legacy problem because reality is that as I often say old code Never Dies you have to kill it once you have built something that's useful then it's going to live on and so you're unless you have fully disposable code there is some body of code that you have there that represents something in mble something that has a cost something that has some degree of technical debt to it it may be very very small if you look at many of the classic organizations in the

big Financial space these are groups who have been working with code bases since literally the 60s I had one engagement with the Internal Revenue Service in the United States because they've been trying to modernize their systems since the 1960s now let's go back in time what was happening in the 60s well you had the of the population the increase of Social Security and the like and more and more organizations who were creating lots of paperwork and so the banks and the government realized by the late to Mid 60s there was simply so much going on

that you couldn't do it by hand and why did banks used to close at 3 p.m. because they needed the time for the humans to reconcile accounts the IRS was very much like that and so there was during the 60s this period ative automation of of human processes and so most of that was written in IBM 360 Assembly Language Zoom back to today there is still code in the IRS system written in IBM 360 Assembly Language running on emulators upon emulators upon emulators but that creates really difficult problems because some of that code embodies business

rules that are within the Assembly Language itself so how do you change that and the answer is you're faced with a real human problem of how do I transm morgy old code in cobal Assembly Language so that it can work on modern technology there's only so much you can do via emulation and then would you consider that the government you know makes new business rules every year how do you keep up with that that's a real and present Legacy problem Facebook has the same thing although their code doesn't date back that far Google has the

same problem as well open AI will soon have that problem this episode is brought to you by work OS if you're building a SAS app at some point your customers will start asking for Enterprise features like s authenication skim provisioning and fine grain authorization that's where workos comes in making it fast and painless ad Enterprise features to your app their apis are easy to understand and you can ship quickly and get back to building under features workco also provides a free user management solution called alkit for up to 1 million monthly AC users it's a

drop in replacement for odd zero and comes standard with useful features like domain verification rulebase Access Control bot protection and MFA it's powered by radic components which means zero compromises in design you get Limitless customizations as well as modular templates designed for quick Integrations today hundreds of fast growing stups are powered by work OS including ones you probably know like cursor versel and perplexity check it out at work.com to learn more that is work.com this episode is brought to you by Savala it's a true Heroku alternative where you can deploy applications manage databases and host

static Siz for free saala is a platform designed for teams with his preview and pipeline features developers can collaborate on any stack while being assured of the security of their workloads from staging to production Savala holds all the major security certifications companies are typically looking for their application hosting offers automatic G integration Docker image deployments hybrid for optimal cost savings vertical and horizontal Auto scaling TCP proxy support and optional private network connections for your databases their free static site hosting is perfect for landing pages documentation sites and more it also includes preview deployments for easier

iterations and seamless teamwork sella features an easy to use interface unlimited seats no hidden tricks and transparent usage based pricing with Enterprise level cloud for dos protection for workloads of any size sign up and deploy today go to seal.com that is Savala with ael.com when you say you are parachuted to help a bunch of different types of companies can you give a sense of what types of companies you worked over the years the decades to help with their architecture their legacy code their Tech du every conceivable domain truly is I've had the opportunity to work

with obviously the financials I've done a lot of work in the defense sector um in fact to go way back in time complex systems were really not started in the commercial realm but they really began in the world of of Defense the phrase I also use here is that all of modern Computing was woven on a loom of Sorrow what we see in modern Computing was born from World War II and the Cold War particularly a system called sage the semi-automatic ground environment that came about during the 50s and indeed was operational till the 1980s

this was a system built in response of the Soviet threat of them taking uh bombers over the Arctic and coming into the United States before we had satellites and pervasive radar and that was a system that was that pre that precipitated the creation of what we call the Sofer crisis it was what triggered the creation of the NATO conference later in that decade in which a group of folks came together from around the world saying you know how do we attend to this problem and it was really at the peak of what I'd call the

first golden age of software engineering so we have defense systems I worked with a lot of realtime systems everything from pacemakers to uh uh to uh uh to Subway systems to gosh what else uh CT scans and the like uh truly you name a domain and I probably spent some time in it the James web Space Telescope currently uses the uml in its design that's pretty freaking cool if you think about it jumping Way Forward you've now been working with a lot of companies been involved in a lot of projects and also influen a lot

of software architecture the broader field how would you describe the field evolving over the decades you were clearly part of some key techniques invented and becoming commonplace what was this like so I alluded to a phase that called the first golden age of software engineering this is the realm of the time of functional or not functional but algorithmic languages such as Fortran and Cobalt uh APL lisp and the like although lisp was sort of a a multimodal kind of language but the dominant way that we decompose systems was through algorithms and so you saw the

rise of structured analysis and design techniques which made a whole lot of sense at that time this is where you had the jordens and DeMarcos and constantines and the like because the presenting problem for software systems was they were generally not distributed they were largely monoliths and how could we build larger and larger systems that were sustainable and economically interesting over time well the Golden Age of that first golden age of software engineering began to change as we started to see the rise of distributed systems and that Rise Again happened not in the commercial world

but happened in the defense world uh the arpanet was you know funded by the government funded by DARPA I was as I I mentioned to you earlier I got my first email address in 1979 when there weren't that many email addresses around in fact one small story there when we had the arpanet in the air I was teaching at the Air Force Academy at the time we had a little uh mimeographed document that listed the email address of everybody in the world I think at that time there were a few thousand people so you could

we knew who everybody who everyone was at the time was pretty cool to go back so the first distributed systems were happening in that domain indeed back to Vandenberg I worked on a system called the Telemetry integrated processing system which was a a close network of some 32 mini computers Min computers were being' be a thing and so the problems of how do I deal with taking a larger system and breaking it up into multiple distributed Parts was beginning to emerge as a problem hadn't reached the commercial sector yet so we saw we the industry

began to see that there were limitations to what one could do with algorithmic decomposition and so there were these pressures all around to try to attend to the next kinds of software software that was distributed software that was real time software that was multilingual software that worked on a variety of computers and I had to deal with all the normal aspects of distributed systems which is they're going to fail at various times and I've got communication issues and the like so it led to a realization that we needed to think about software in very different

ways the other thing that was happening is in research you saw the rise of languages such as simula and small talk which we're looking at the world in fundamentally different lenses so here we are again in the late '70s and again I'm what a 20-some I was asked by one of my former teachers at the Air Force Academy say Grady would you go help the Department of Defense figure out how to use this new programming language called Ada so we can apply it to Modern software engineering techniques now why was the government worried about this

by that time software was a real problem for the Department of Defense actually for all of the federal government because there were several thousand languages in use that exploded because Fortran and Cobalt were useful for some things but not for all things and so there was a decision made to build one language to rule them all and that was the Ada programming language Ada was far ahead of its time it was a language that was influenced by simulant small talk and others but used the ideas of abstract data types from uh from lisov and galgan

and others it used the ideas of information hiding from David parnis all ideas that were very new at the time but frankly are part of the atmosphere in which we breathe right now and so as an industry we really didn't understand the methodologies to make that work thus was born the m b method I was here I was in 79 till about 81 I was going back and forth across the United States helping helping the federal government and helping contractors try to apply this new language in new ways and this was the beginnings of the

second golden age of s Engineering in which it was not so much the complexity of the algorithms but it became a systems engineering problem systems that were dealing with distributed systems that were very new at the time and that was the essence of the bch method it was the things I learned about helping organizations architect systems with these new kinds of domains and new kinds of languages could you explain what the B method is I understand it has to do with objectoriented programming but coming from you as the person who invented it it will be

nice to explain what it is and why it was important well let's go back to Plato talking about going way back I wasn't around then but I've read about him a little bit there's this wonderful Treatise he wrote a dialogue in which there's a debate about how one should best look at the world should I look at it as a as atoms or should I look at it as processes well the first golden age of software engineering was more focused upon the processes the algorithms but there's a parallel way of looking at the world and

that's looking at it through the atoms if you will the classes and objects within them so yes I was influenced by by uh abstract data type Theory by by Plato by a lot of other interesting philosophical things that were coming together at the time of looking at the world in fundamentally different ways so the B method was really trying to codify that how could we decompose systems not based upon algorithms but how do we decompose it based upon classes and objects and again that's where I was influenced by by lisof and parnis and and dyra

and horror and the like uh names that are probably unfamiliar to students these days but they were they were representing the theoretical underpinnings of the first and second generation the B method was basically saying hey here's a new way of thinking about the world and and so it said look not at algorithms but look at combining data and and uh and processes algorithms together in one thing thereby classes now we did some things right we did some things wrong I think the things we did right was classes make a lot of sense in terms of

abstraction what we did wrong is we overemphasized the notion of inheritance inheritance was all about let's you save code because we can you know build generalizations and the like that proved to not make a lot of sense because we ended up doing lots of desparate desperate kinds of abstractions that's okay fast forward to today and people say well what difference does it make and the answer is it's part of the very atmosphere in which you breathe and so you don't even think about it you look at you know redus and you may build things upon

it but you know if you look at it you're really dealing with a set of abstractions that redus offers you and those abstractions are class-based so they're baked into the way of thinking of those kinds of systems so in short the bch method was let's look at the world not through algorithms but instead through objects and classes the last thing I'll mention is one of the things the bch system the bch method hinted at that really did not catch full form into the uml was looking at systems through multiple points of view now we'll come

back to that in a bit when I talk about felik kushan so just so I understand because myself and most of us listening will have started our careers long after the B method was invented for us classes variables inheritance that's pretty common pretty everyday right things but as I understand it wasn't like that back then right so could you talk us through what the environment the technology was like so so that what what made the huge method so new interesting or or Innovative well let me go even further back to the 1950s and show you

a parallel story there was a time in the growth of algorithmic programming languages where the idea of a sub routine was considered controversial why because doing a function call added at least two or three more instructions which was computationally EXP ensive so even function calls and decomposing something into sub routines was viewed as an architectural aberation and people opposed it because it was inefficient well obviously we think well that was stupid because we need it for our management of complexity the same thing I think was true back in the days of early object orientation I

mean people were doing object orientation in algorithmic languages because you'd have these things in cobal called you know Common data areas people would devise here's all this common data and as a matter of practice but not language you would say this data is used in this way by these algorithms and vice versa in fact going back to that project I mentioned to you at at vandenbberg Air Force base with those 32 computers on the side of every one of those computers every day we'd see a print out of here is the common data pool and

so it was the abstractions right in your face face CU they changed and so people were trying to do algorithmic or objectoriented decompositions but the languages didn't support it there was a need to do that kind of thing and until the languages came into play there was no way to bring those ideas together efficiently and so yes the bch method was very much a reaction to the forces upon building software intensive systems to look at classes trying to apply it with modern languages and building a methodology around it today we take it for granted because

our languages make it easy for us to do this it's just fascinating to think back how revolutionary it was and compare to just how commonplace it is and also to think about how things that we invent today and that are revolutionary in 20 years people will be like oh yeah that's common place right exactly one thing that you're known for and you also mention you're associated with it ISL but can you share how this was created what was the goal of it back then who were involved and what was the need that it was solving

at the time right so in 1982 two of my classmates from the Air Force Academy Paul Ley and Mike delin uh Paul had been a roommate of mine at the Air Force Academy he was an economics major Mike Devin was a computer science major Mike and I had a few classes together the first time I ever met Mike was in was in an unarmed combat course by the way so not your typical thing you get at colleges but hey you know I I was trying to be a warrior that's where I first met Mike and

I think he beat the stuffing out of me if I'm not mistaken in a pugil stick competition but Mike and here I was at vanberg Air Force Base Mike and Paul were at U at Stan up in the Bay Area they were working at the satellite control facility and I engaged with them because they had one of the first largest Ada projects that was going on so I went up and helped consult with that project the two of them also went to Stanford and I think I swear there's something in the water at Stanford because

they then connected with art rock and Hamrick conquist the two premier Venture capitalists at the time uh art rock and H Quist were the key Founders to Apple and they also contributed to the funding of what became rational software so in ' 82 Mike and Paul got together with me and said let's start a company and we we did it was this company called rational rational machines Incorporated whose intent was to build a software development environment for this new coming Ada programming language we saw that to be an opportunity uh in which we could make

piles of money and we built Hardware at first because this was the time when you know many computers were were becoming affordable you had sun coming into play here but none of them were powerful enough to do the kinds of things what we were doing so Mike designed a system I helped build the methodology around using the system it was a system called the r1000 and that was the dominant Ada system used around the world for Ada at the time well around I don't know would have been the early Jump Ahead a decade now here

we are the mid 90s and uh I was getting a little tired of doing that kind of stuff and branching out in other places and I find I found that there was traction that I was getting from the bch method into the commercial sector I was giving a bunch of lectures at the time and at one time uh there was a a a gentleman in the audience who asked a really insightful question and afterwards he and I met up a guy by the name of BNA strrip and it turns out BNA was working on a

thing called C with classes which was the predecessor to C++ the two of us got together we hit it off we found that we were doing very similar things together and in fact it led to the two of us doing a lecture series around the United States where I got to know him quite well and this was around the time he wrote his first book on C++ if you look at the first edition you'll see here references a lot of my ideas and that's when my book on objectoriented design came out and I reference his

his work a lot too so oh the the B method and C++ kind of grew up together well I thought this was interesting and I remember a particularly important meeting I had with Mike and Paul around the time uh it was at the Red Carpet club of United Airlines in in Denver and they met with me and said hey Grady we're thinking of moving the company in the direction of embedded systems and I said to them well good for you I think that's a stupid idea because you're missing the commercial sector uh go off and

have fun I'm going to do different things that gave them pause and I think they realized wait a minute maybe there is something here in the commercial space we were finding that we were having challenges continuing to grow the business in just the defense sector which is what led them to that and so they then made the decision hey let's take the bo method and make it real and thus was begun the beginnings of a system called Rose rational object oriented software engineering our first tool the first prototype by the way I wrote wrote in

small talk it was a wonderful system I wish IID kept the the source code for that around but I remember making changes to it just literal minutes before we first the F did the first demo and so that's where we sort of broke out from the defense sector into the commercial sector and it was a big hit because that was the time when I think lots of others were recognizing object orientation was a good light way of looking at the world and C++ actually supported that well that led to two things not just Commercial Success

on our part we began to make lots of money and we began acquiring other companies and we started filling out the software engineering life cycle can you just tell us what rational software did this commercial thing that was a hit yep so rational soft the rose rational objectoriented software engineering was a personal productivity tool if you will that ran on an IBM PC it ALS oo ran on I think we eventually moved it to a number of other devices um it ran under windows in the first one that basically allowed you to you know draw

uml diagram not uml but boot diagrams so that you could then reason about and think about your design we did a little bit of code generation but really it was just a design tool to help organizations think about their designs and you know people use it quite well to document and specify and build their system now we started making lots of money off of it and so we started acquiring companies we bought a requirements company uh Ed Jordan came to me and said go look at these folks we did we bought a small company out

of Cambridge uh called Pure Atria which was led by a gentleman by the name of Reed Hastings Reed came to us and we bought his company and he said and we're talking about the founder of Netflix right yes yes so we bought his yeah Reed Reed realized he's a lousy CEO and so he took his money hang out around for a few years figuring out what he was going to do and actually that was a lot of the seed money that helped form Netflix it is a small world in that regard so at that time

at the peak of what we were doing by the late 1990s IBM rational was sort of dominating the space of software engineering because we had tools across every part of the development life cycle and this is where the ideas of incremental and IR sof development came into play long before today what we call continuous integration and continuous deployment we were already doing that with a rational machine in our tools we had pioneered those ideas uh because we had built uh incremental compilation tools and the like so here we were in the 90s and we had

a whole set of tool sets around this but because this was clearly gring gaining Traction in the marketplace we weren't the only ones and we were seeing the rise of hundreds if not a few thousand compan companies they were beginning to try to do objector kinds of things and this was the beginnings of the second golden age of software engineering where you had Pete code and and uh Constantine back again and Jordan again and Martin uh we had uh Evar yakas and and Jim Ramba and so it was a very vibrant time where organizations were

trying to say gosh we've got these great tools we've got the arpanet we've got personal computers how do we build software for it so the presenting problem was in software what is the best way to Design Systems using this very very robust very powerful set of tools we have at our hands and so rational being in a very interesting space we said you know we're kind of dominating the market let's keep going here so we hired Jim Rumba and the task Jim and I had was to combine his methodology omt object management technique with mine

we were sort of the two leading ones at the time and then we bought Evar aquin's company because both Jim and I were using what was the idea of use cases again talk about something that's part of the atmosphere use cases are just something you think about but they were new at the time in the 90s there were an idea invented by Evar in his work primarily at Ericson and so we were working together with Evar on building software for base base stations for the burgeoning uh cellular telephone networks which were a thing back at

the time as well too so here we have the three of us who were brought together by rational and our task was let's unify our methods now you could never have found three two very three very different people I'm pretty amazed that we didn't end up with one of us in the hospital and one of us in jail we were so so very different and I won't go into further detail on that except to say except to say that I'm very proud of what we created and from that was Bor in the uml so we

decided you know this is not just ours we need to make it something that the whole world could use so we made the decision to release it into the object Management Group and that's was born uml 1.o I sort of drove most of that working with Jim andar I wrote the primary document for it obviously it was the three of us working together I don't want to you know I want to give them complete credit believe me uh but after uml 1.0 I would was emotionally exhausted and wanted to go off and do new things

so I kind of walked away from it at the time I want to pause and mention one other person who was important here to actually two other people the first was Philipe krushen so we realized that our work was so big we couldn't just do the methodology and the notation so Jim Evar and I worked on the notation that was the uml and and uh Philipe worked primarily with some of iar's people on the methodology and th was born the rational unified process notice the emphasis upon unified Philip brought to the table the very important

idea that we' begun to see hints at with the B method and that is looking at the world through multiple points of view Philipe has this idea of the four plus1 view model which he grew from his work with building the Canadian Air Traffic Control System again a very complex distributed system those ideas were eventually made manifest in an i e IEC ioc standard 4220 on architectural description which basically says if you're looking at an architecture you have to look at it from multiple points of view use cases logical view process view implementation View and

deployment View and that's a very important and profound piece the other thing that came into play another person that came into play was um was Walker Roy now Walker's an interesting guy you talk about a small world his father win Royce who had the pleasure were working with when he was at lock at the time windroy was the gentleman who wrote the paper on waterfall life cycles and his son was basically working on you know spiral models in the B method when was misunderstood because he was not endorsing waterfall methods he said that's a stupid

idea in fact look at paris's paper a rational design process why and how to fake it it's from that the rational software name came out of by the way um which says at one level it looks like waterfall inside no it's what we today we would call Agile so here we are here we are what the late 90s early 2000 uml 21.0 was in the bag and that was the life of Grady at the time hey developers we've all been there it's 3:00 a.m. and your phone blares jolting you awake another alert you scramble to

troubleshoot but the complexity of your microservice environment makes it nearly impossible to pinpoint the problem quickly that's why chronosphere is on a mission to help you take back control with differential diagnosis a newly distributed tracing feature that takes the guesswork out of troubleshooting with just one click DDX automatically analyzes all spans siai Dimensions related to a service pinpointing the most likely cause of the issue don't let troubleshooting drag you into the early hours of the morning just DD exit and resolve issues faster see why chronosphere was named a leader in the 2024 gner quadron for

observability platforms at chronosphere doio pragmatic that is chronosphere doio SL pragmatic and do I understand correctly that the goal of uml was to describe a system so when I think back to college withl we have the different boxes of different classes we have the arrows between them they describe relationships depending on the the type of the arrow is and you when you look at this this whole diagram you get a sense of the the structure of the software you're building yes so if you look at the very first line of the uml 1.o standard I

believe it says something to the effect the uml is a visual language intended to reason about visualize specify and document the artifacts of a software intensive system it says nothing about it being a programming language in fact I voraciously pushed back against that it was a language meant to think about and reason about a system to think about the world in object-oriented ways particularly in ways that you looked at it through multiple points of view devops today by the way is simply an amalgamation of deployment and implementation views but we didn't call it back that

way then we looked at it from those kinds of views and that's really where uml 1.0 was it was meant to be how do I think about these things how do I reason about them and I always intended for you know you'd write a uml diagram and you'd throw most of them away now unfortunately many people didn't do that and in the move from uml 1.0 to 2.0 there was a faction of individuals and companies who said no we want to make the uml very precise we want to turn it into a programming language and

that was I think a profound mistake I never intended the uml to be a programming language but the net result result of that was to make the uml much more complex much larger and the emphasis then was upon not using it to reason but to generate code and to reverse engineering now the reverse engineering I can get that makes a lot of sense but turning into a programming language was a mistake and I think that began the decline of the uml because people were using it the wrong ways and it's Peak the uml probably had

a 20 to 30% you know penetration in the marketplace which is pretty cool if you think about it and so I'm proud of looking at the systems in which it was used but most of all I'm proud of the fact that it helped people think of building software in different ways so when you say that a PE it had 20 to 30% usage does this mean that about 20 to 30% of commercial developers were using it at the time and what what time was this around what time yeah here we're talking around 2000 plus or

minus a few years uh remember that I Microsoft was big in the midst of this as well too that they actually worked with us to take you our Rose product and make it a part of visual studio so we had a team that was working up in Seattle to make that happen and it was a major selling point for Microsoft at the time because it actually helped their customers build more complex software so so yeah we're talking around 2000 or so but what what H what else happened in that time frame the answer is the

internet was so the arpanet had moved over to the internet we were beginning to see companies in the late 1990s move on to the internet which was great but there was a challenge there there was first how do I even build systems for distributed work in in uh in the web and how do I make money off of it what does those systems look like and so that's why Microsoft was interested because we were helping their customers move from the p to distributed systems but on the other hand there was also a lot of hype

that you know the the internet's going to improve your sex life it's going to do all these kinds of things it was just totally overinvestment in that space so a little after the millenum we saw the we saw a backlash and there was this great downturn in the uh in the marketplace where people had built things and realized they weren't necessarily economically uh economically sustainable so now we are here in 2003 IBM and Microsoft were still using our tools heavily because they were important for their customers IBM and Microsoft both bid for us and I

think ibm1 at some some bid of 2.7 million billion dollars to buy to buy uh to buy rational and so we went over to to IBM and became part of it and that made sense because at that time there were 3,500 of us in 14 countries we had reached almost a billion in revenues which was pretty extraordinary for a company around that time but it was time to be absorbed now one other story I'll tell before I I move before I pause again uh here I was IBM acquired us uh they made me a fellow

immediately which had never happened before it usually takes years of being part of IBM and a couple months after I got a phone call said hey Grady it's Bill visit me so I I went up and flew up to Bill Bill Gates Right Bill Gates yeah i' I'd done some things to the bill before and so bill was at the time still CEO of Microsoft he said hey Bill took me into his office we had like a 30 minute meeting scheduled we ran for two hours much to the annoyance of his staff and he sat

down and said Grady it's not public yet but I'm going to be moving out of my role at Microsoft because I would to you know do other things and Grady you know I've got two roles I'm CEO and I'm Chief Architect of Microsoft I'd like to give you that job of Chief Architect for Enterprise and so I said bill that's very interesting and so I said give me a little time and uh I went around and met all of his you know his his main Reports most importantly Balmer never met me and that was a

red flag and it was around the time too I realized that Microsoft was a particularly nasty company you had the office group and you had the you had the windows group that just couldn't stand one another so I eventually came back to Bill through his his uh his hiring folks and said Bill I'm flattered but you know you have a profoundly dysfunctional company and I'm not the one to fix it so Bill thank you but no thank you I think I use something to the effective it would only end in tears for both of us

if I accepted so let's move on so I stay at IBM and it was the good decision to make it would have been a bad decision for me to go and there's this cartoon about Microsoft um created by uh by a software engineer cartoonist Manu cornate about the organizations of Microsoft the two organizations and they're holding guns against one another yeah yeah that's where it was they needed somebody to knock it I'm I'm a lover not a fighter and I was not the guy to break things up wow what a story so I started uml

in college and to this date it's part of several college curriculums but interesting enough in the industry I've just not really seen it used for for anything at least the companies that I worked at and the startups and scale-ups and and large companies I work with I kind of see a resistance to using it when it's brought up claiming that is too formal uh because we do use architecture right we use boxes and arrows and and we we diet diagram I'm curious to know you know we talked about howl was used by 20 30% of

the industry at some point but what what happened in your view so this leads us to you know contemporary architecture that I've got a Shelf full of books on architecture both in the you know older ones and new ones and if you look at what a lot of people speak of as software architecture today I think it's reasonable and and and sound and there's good stuff there but in many of the kinds of systems these Architects talk about the architectural decisions have largely been made for you I'm going to build a system that requires message

passing well let's go find you know uh uh rabbit mq or m whatever or redus or or or whatever I need the architectural decisions have been made for you so a lot of the activities of contemporary Architects is simply taking very large Frameworks and components and weaving them together which is a very Noble and wonderful thing to do they also represent systems like meta and D is particular they have grown their architecture and system over a few decades now and the stuff they're building on top of it is largely evolving and building upon those apis

which does not require the deeper kinds of architectural thinking so I use this let me give you an image here of of three AIS system along one axis you have um uh you have levels of of ceremony if I'm a startup then it's just my other people's money and no one else's and heck I can write disposable software and if I fail I'll just go find another venture capitalist of course I'm not going to worry about any kinds of degree of ceremony because just Build It Go hire some brilliant people and make it happen that's

wonderful on the other hand if let's say I'm doing something like U I don't know building the Next Generation uh intercontinental ballistic missile system which uses about I don't know about half a billion half a trillion dollars you bet you're going to use more ceremony because you have to have degrees of accountability so that's one axis the next axis is that of of uh of risk so if I build a system and says oh if I fail you know you know so and so's not going to find their grinder match big deal on the other

hand if I fail and somebody dies that's a problem and you're going to use a more disciplined architecture the third axis is that of complexity if I build a system that people have done again and again then heck I don't need anything he heavans this is where prompt engineering comes into play I go build an apus by building prompts cuz we built these things I don't need no stinking uml for that thing on the other hand if I'm building something that I've never built before let's say I'm building not an llm but I'm building a

constellation of llms work together and I want to weave them together with non- neural systems then you begin to think about architecture and that's what the uml comes into play there's a sweet spot for a tremendous amount of software development going into place that doesn't need the uml and does not need any kind of thing like that but on the other hand you go a little further out in that three dimensions and yes there are people all over the place using uml I mentioned the James web Space Telescope I still work of financial companies who

are doing that where the risk the complexity uh the ceremony is sufficiently high that it Demands a bit more formalism so do I understand it correctly that you're saying that software architecture has changed from the 90s and 2000s when systems were new architectures were new software architecture was still a lot more novel and if we look at Venture funded startups and big Tech today what we see is one they're just not as risky if they fail big deal and then two is we can use a lot more software that's out there and been architected and

you know we can for example use reddis as a cache and it is there it works we don't need to think too much about it and then the third is that there's a lot of startups who just don't need ceremony basically they don't need audits they don't need formality they don't they they can just go like all right let's just do whatever we want to it we don't need that kind of auditability do I understand that these are the changes that are are in play with software architecture and is this why like some of these

formal methods are just not as popular with startups and scale UPS yes in fact I think it goes to the root of the economics of software development let's go back to the first age of software uh first golden age the machines were far more expensive than the humans and so it required one to do some thinking before I even got to the machine because machine time was very very expensive for me and so yes uh algorithmic decomposition structured analysis design techniques made a lot of sense because we needed that kind of optimization if you move

to contemporary times cop reputational resources are like water to a fish they're they're available to anybody I've got on my desk behind me I've got a a a my own personal cloud of four Nvidia single board computers which has more processing power that existed in the world in the 1970s and that's pretty amazing that's a few thousand dollar there and so the economics have changed vastly such that that you don't need to think about it so much because heck it becomes disposable AI I think is also changing that because it allows me to build things

where I don't even have to think about design heck I don't even have to think about software I just prompted for those things to build something and one offs and once I'm done I throw it away so in that sense it comes back to the economics but there will continue to remain a class of software that's new unique breaking new ground that still requires that kind of architectural thinking and it's interesting because just recently I read how Amazon is using formal methods for awss 3 they're they're publishing a blog post detailing uh like how they're

doing it and they're doing this to catch those really really edge cases that only happen in one in a billion one in a one in a trillion but at their scale this is a regular reoccurring event and I found it fascinating how some of these methods are making their way back yeah well let me let me set aside formal methods in a moment because it turns out that's a different topic for which I have some experience and opinions but go back to Amazon you go to their websites and they have a whole language around architecture

Microsoft does as well too around Azure it's the way I describe an architecture in Amazon it's those you know their blocky gu diagrams it says hey I'm going to build this kind of thing and here's my particular notation for it and furthermore here there are some examples so even Amazon and Microsoft have recognized that architecture plays a role but there are enough times people have done these kinds of things it says oh you want to build this kind of system then you want to use these services in fact here are some examples for it you

can find on our website so without them really acknowledging it Amazon and Microsoft view architecture is still important but there are enough patterns that one doesn't have to go through the process of rediscovering those because I can build those things and this is a representation of the I think the the maturation of our business we move from algorithms whichever one can use to design patterns to now architectural patterns which Amazon and Microsoft have codified themselves so let me switch over now to formal methods formal methods have always been a thing however formal methods in my

experience have been a niche part of every software intensive systems because formal methods only go so far in what domains they can cover and so you'll see as what your example described Microsoft began using formal methods in U in uh their drivers to validate the uh uh the the correctness of their of their things hard driver their Hardware exactly yeah and that was an important move forward I I've been with projects that use things like you know I've got this system in which people might die let's run a formal analysis upon it the thing is

though that those formal methods don't deal with real world things because they don't deal with space and time they deal with functionality so I have always found formal methods to be of use but only for parts of a system and never is drivers of the architecture itself speaking of software architecture these is the role software architect is not really popular anymore at least at the likes of startups and and big Tech instead we do have Architects but they're inclusing called staff engineer principal engineer distinguished engineer the architecture is and they do still do architecture but

there's a different focus on it now you were there when software architecture was created when the first software Architects were created as a role can you tell us how you've seen this role be created and then evolved throughout the decades so two things influenced my understanding of the space first is I didn't really call myself an architect but I helped people design the systems they were building I was heavily influenced by a dear friend of mine Mary Shaw uh she's a professor at Carnegie melon I think she won the national met of Honor under Obama

if I'm not mistaken and she wrote this really profound book called software architecture in which she began the first exposition of architectural patterns Mary's just Del lightful human being and that's when I began to understand the formalizations of what architecture could be and the other thing that influenced me was architecture from other domains particularly civil engineering and the like and there's in the defense world there's also you know ship Building architecture and airplane architecture and the like so the term architecture is one that's very well respected outside the software engineering World ignore the title for

a moment and let's go back to First principles what is it at all and you've probably heard me say this all architecture is design but not all design is architecture architecture represents the set of significant design decisions that shape the form and function of a system where significant is measured by cost of change so software architecture and architect has this this horrible emotional baggage around it so think of it as it's all about making decisions what are the decisions that shape my system as an architect that's what I'm doing as the project manager or whatever

I'm also about making decisions but one subtle difference is that it's no longer just the decisions about the shape of the software but it's the shape of the system itself where it embodies itself in the physical world where other systems and humans themselves as well that's what it is it's all about decision process and have you seen this Ro change did you see a golden age of companies where they were employing software archtics that were empowering them to CH out design I'm seeing a bit less of this I'm curious are you seeing something similar is

this just a bubble that we're seeing just because you seems you're embedded in a lot of companies um there's another uh sound bite I'll give you which is that the entire history of software engineering is one of rising levels of abstraction so what we're seeing here is the rise of another level of abstractions which gives us all these extraordinarily powerful Frameworks from which I can build systems and which as I alluded to the architectural decisions that were front and center for us back then are now embodied in these so now becomes a decision what cloud

service do I use what messaging system do I use what platform do I use that's the decision which has a lot of economic decisions and not just not just uh uh not just software kinds of decisions associated with it so I think the role of the architect in effect has changed because now I'm dealing with systemic problems not just software problems themselves and have you heard about the role called Solutions architect I think it was created maybe 10 years ago yeah and it's about people who are doing Cloud architecture it's it's fascinating it's is a

ro specific to the cloud and as you said they they make economic decisions what services do I use do I use AWS gcp if I use AWS do I use ec2 or do I use another service yeah and that's why it's a system systemic issue you generally if you're a startup you're going to hire somebody who's done that before who knows where the skeletons are buried who knows who knows you know what the costs of these things are and so that you'll hire those kind of folks because they'll they'll accelerate you because they've made those

decisions and they sort of know in the shape of what you're building now what decisions next make sense and they are systemic decisions because they have economic and long-term Associated consequences to them one thing what comes up with software architecture is migrations I do notice that most starge companies are being hurt by long running migrations and these migrations usually happen thanks to software architecture changing for example from going from monolith to microservices or changing Technologies for example going from Noe to go or upgrading major version and Frameworks for example one version of angular to the

other one how do you think software architecture and migrations are connected and why do you think software migrations are just so darn hard and they don't seem to be going away they are migrations will will plag us until the heat death of the cosmos uh I believe because there's always you're always building economically viable software but then the technology is changing out from under you that compels you to consider migrating uh you know consider the migration from monoliths from the 60s 7s 80s till all of a sudden economically we had many computers and now distributed

systems you're still not going to use an iPhone input put everything up in a Mainframe but those changes compel you to consider an architecture that better balances where the processing takes place if all of a sudden I can begin to do Edge inferencing on my devices that's going to be something that's going to change architectures as well so there always these changes both in the hardware as well as societal changes if you will that impact the structure of my systems that that are the forces that impel me compel me to do this migration uh but

why is it hard well another sound bite I'll give you is that um the code is the truth but the code is not the whole truth there is so much that is outside of the code that represents design decisions they're rationale uh what I why I chose this versus why I didn't choose that subtle things as to why did I name things this way way because of its impact that is is long understood long misunderstood and so while the code may be the truth the problem with moving migrations is that you lose there's a there's

a loss of information and it's difficult to to try to recreate those design decisions just from the code the people who wrote that initial code you're trying to move they may have they they probably cashed out or they've died or some combination of and so you you don't you don't really know why those decisions were made and so you're working a little bit in the dark yeah you're you're mentioning people have died and I don't think we usually think about that but I guess when you're thinking about systems that are 40 50 years old that

is the reality of some of them is what will the Lennox Colonel look like when lonus eventually retires so how will it how will it drift because he has provided a firm and much needed hand upon the conceptual Integrity of that system and that's what the the chief decider makes he or she is the one that provides that conceptual integrity and when that person is gone then you naturally see drift and that's inevitable it's it's the uh it's entropy software all software exhibits some degrees of entropy without that without adding that kind of uh Force

to it one thing that is very relevant today in terms of Technologies and architecture is AI uh this technology is here it's revolutionary it is disruptive now you've been in the industry for closer to 50 years if you look back how do llms and AI compare to the industry to pass Innovations and events because for for many of us in the industry those of us who've been for for 20 30 years or so llms do seem like the biggest change of software but when you look back have you seen something that was comparable in terms

of impact or the pace of change that's a great question yeah um the first I think was just the realization that I could build distributed systems as opposed to putting all my processing on one that was that was a a change that rattled everything in the way we built systems so at the point in time that all of a sudden I could have a network of many computers and then eventually that carried on to microcomputers and devices at the edge like phones themselves but that transition in the growth of Min computers was seismic that I

don't think people understood the full implications of until much later because it required us to rethink the way that we put systems together so we saw there the the rise of a great degree of uncertainty because we didn't know how to how to build these systems I have messaging across them do I use RPC do I use something else what what do I use to communicate do I use shared memory and so there was this period of exploration that we didn't know about until it finally we realized oh these are the common ways that work

and can you remind me around what time this was was this around the 80s was it around the 90s there was the late 70s uh because here you had well let's go back way in history um there was the thing called gosh what's the name of it I'm I'm drawing a bit of a blank here but we first here we are in the late 60s early 70s in which mainframes were were you know roaming the Earth but even then it was a realization that these machines were sometimes being underutilized and so there was born the

formation of the first time sharing systems the next thing that happened around the same time were systems such as Whirlwind which came out of the Lincoln Laboratories which was beginning to take machines and touch them to the real world so all of a sudden now you had real-time Computing so you had this Confluence of two very interesting things you had large machines in which we were beginning to develop uring kinds of operating systems and then machines off to the side that were touching the real world these came together in really the rise of many computers

particular digital Equipment Corporation and the like where miniaturization which frankly came up came about because of the investment the department of fense was making in semiconductors making it economically interesting to have a computer that could sit on your desktop that one or two people could use full-time so it took kind of the ideas of the embodiment of of uh things like Whirlwind and then the the distributed systems we were people building now all of a sudden I could take these things and I could start building them together and that's what was happening in the mid

to late 70s that we saw the rise of those distributed systems there was one last thing there was the rise of client server systems where you saw dumb terminals that's what IBM was doing at the time the green screens talking to mainframes but the rise of those new machines meant I could move some of the processing off to the edge as opposed to leaving it in the monolith so those are the forces that led us to rethinking about systems so do I understand correctly that programmers at the time they were used to working on these

large computers and distributed computing came in absolutely and it just completely changed the Dynamics so youngsters came in they started to embrace this distributed computing but existing programmers they kind of stick to what they knew what was efficient on the main frame was was that what it was like it was and then when you had things like TCP pip and HTML and all those things around it it all of a sudden we now had a vocabulary we had now had mechanisms that we could bind these things together so I would not say that llms are

as pervasively as important as the rise of of distributed systems but there's a parallel to it the second thing I'd say that is sort of similar to that is the rise of gpus which came from the gaming industry because there they were solving a very different problem how do I deal with how do I deal with more photorealistic things in my games and we realized it's all Matrix multiplications and so Nvidia began to move into that Marketplace and we had the rise of gpus which dominated that space and it wasn't until Andrew ning came and

said wait a minute those gpus used for gamings use the same kind of mathematics as do our deep learning kind of things and poof all of a sudden we had this we had this Perfect Storm of lots of data powerful hardware and the rise of interesting algorithms uh back propagation and the like so is a perfect storm so yes this is an exciting time there's no doubt that that large language models are interesting but we must be careful and we'll go into that in a moment so let's go into that I'd love to hear your

candid thoughts about llms their applicability The Innovation and the tradeoffs that they come with well so to set the stage before people worry that I'm just pontificating about things I know nothing about um you I alluded to at the very beginning that and here I was 12 14 whatever I was interested in what was burgeoning becoming AI at the time that is always kind of stuck with me and in what would have been um I I've always you know pursued an interest in that space and it wasn't until IBM drew me back to it around

the time of Watson that I began to make it a full-time thing so David fuchi who led the development of Watson uh had called me and said hey Grady uh I'm gonna give this lecture I can't do it would you do it for me and I said David happy to do so but only on the condition that I can choose the topic and the topic I chose is what is the architecture of Watson Jeopardy and as it turns out nobody had documented it so being an expert in the architecture space I sat down with David's

team for several months and documented the ASB built architecture of Watson Jeopardy it's not a neural network system it turns out to be a pipeline architecture that through a pipeline brought together a number of statistical systems AI at the time was all about predictive statistical methods not neural networks and uh and methods in knowledge engineering and so I documented the architecture that caught the eye of IBM management and it's for the first time I really documented an AI architecture for those of us who don't know this Wason Jeopardy this was the IBM computer that played

the game Jeopardy and won if I remember it won yeah we we built all we beat all the humans in that space and at the time this was on in mainstream media in the Press everywhere and this was shown as an example of an AI system that can outperform a human on a very human task which is this popular show Jeopardy yeah right and it was part of the the the trajectory IBM had been on in that space because before that we had deep blue which beat the the leading uh chess player at the time

Gary kaspar Gary cospro yeah yeah so so we did that yeah that we did that through brute brute forth methods Brute Force methods then Watson Jeopardy came along beating humid in natural language processing at the time IBM had already been doing things in that space with a uh with a system again statistical systems we eventually sold it to to nuance and the like but we were we were pretty dominant in the space of natural language processing and IBM W Watson Jeopardy was the peak of that so IBM asked me hey Grady this is cool stuff

we're going to we're going to commercialize this would you please you know help us do a study as to what IBM should do with this so I led a study for about a year in cognitive systems what should IBM do now this was interesting because I I studied what Watson was doing I looked at what was happening in the marketplace here we were what 2010ish or thereabouts a little bit later it was that decade and I made it very clear to IBM management this is pretty cool but be careful because there are things we know

it can't do and I was very clear to our management that be careful about hyping it well remember in my meeting with uh with Jenny when I was doing the briefing of about the the project Jenny was the CEO at the time she asked me a question one of the VPS stepped in and step sort of started talking over me and said well yada yada and I politely said to him well thank you but I think you're wrong and I went on to explain to him now being if I were not a fellow I'm sure

I would have been fired on the spot but as a fellow I get to say things like that now they didn't they did not in invite me to join the Watson group so I was happy with that so I worked off on to the side but you know it's sort of an I Told You So that yeah there were things that we can't do and by God we can't do them so I was working in the underground for several years and I'll get to the answer of your question in a moment but it explains why

I can say something meaningful so I was kind of an outsider I was kind of doing my own thing inside IBM and then we had a team down in Austin who had been approached by Hilton Hotels who said we'd like to build a a robotic concierge for our hotels and so we chose uh some robots from Alder Baron a company out of south of France they built these three- foot tall robots called pepper and smaller ones called Nao uh and they built a robotic system was kind of cool it was a question and answering system

based on Watson technology but there was something missing out of it and the team came to me and said Grady would you please please help us out so I did I helped improve the architecture we actually installed it in a few Hilton Hotels but as it turns out just down the road from that team was the Johnson Space Center and so we began to be involved with them and I was thought this is all great because I can back to get back to my space space routes so they had a system called the robena 2

which was a humanoid robot that at the time was on the International Space Station a beautiful piece of engineering and NASA was exploring the idea of human robotic interactions for the space for the the mission of Mars so the group of us sat down and said what would be an interesting design problem for us to explore that would Propel us to look at hard problems because we want to do hard things and we realized it was the mission of Mars because they had two interesting use cases the first is because of the speed of light

issues uh you couldn't rely upon Miss control on the ground you had to take it with you this was the how problem we needed to build a how minus the kill all the humans which use use case which for for some reason NASA didn't want that use case I don't know it's their choice the second is at the time NASA wanted to put robots on the surface of Mars to help astronauts you know build their build their uh scientific experiments and build their habitats and the like so I remember one afternoon I'd sat down and

said I know how to architect this this would have been around 2014 and I built a neuros symbolic architecture we called self and it used the ideas of Marvin Minsky Society of Mind together with Rodney Brooks's notion of subsumption architectures together with hofer's ideas of strange loops and those three came together forming this self architecture which we then built and so we experimented with nas for a while to build some software behind the robonaut 2 do things like hey robot here is a here is a a scientific process go do this or go go clean

these filters which is a common station keeping activity so we were actually building the software that allowed humans to interact with it and also serve as question answering things that led us to Soul machines a company out of New Zealand they were an offshoot of the work that John Cameron had done in the movie movie uh the U uh shoot what's the what's the one he's done he's done a Trilogy of them now uh that he had this great technology where you put cameras on the AR on the artist and it would measure their muscle

movements they took that they built a neural model of of human musculature and they had the great hardware for it but they had no software so we helped them in that regard we then also worked with a company called um Woodside an oil and gas company out of Australia because they a problem similar to to NASA they were Building Systems for oil rigs which is a very dangerous environment and so they wanted to build Cooperative robots just like NASA was doing so here we were with this system called self that was frankly a neuros symbolic

system it had neural pieces but also symbolic pieces based upon those three architectural cisions it was pretty cool at its peak I had 35 people working on this but then we recognized IBM was uh across six Laboratories I should mention around the world which made it possible for me to live and work out of Maui Hawaii which is where I still am but then IBM recognized IBM's Watson was bleeding cash so they fired everybody and again being a fellow they didn't fire me so I continued in that space but that was my exposure to to

uh the first set of of new kinds of architectures for about six years now I have been working with a set of neuroscientists to try to understand the architecture of the organ organic brain and so rather than you know software kinds of things I've been studying things like cortical columns and and the loops that the ex in the hypothalamus so I've been looking at architecture from the lens of a software architect into these kinds of organic systems so that now leads us finally back to your question what do I think about large language models the

answer is they're pretty freaking cool however they are by the very nature of their architecture unreliable narrators that's what I say politely if I'm going to be impolite I will say they allow us to build that global scale generators because they are clearly they are stochastic parrots they do not reason they do not understand but they do produce some very coherent results because they allow us to navigate a latent space that has been made very complex through training it through the Corpus of the internet so that I think is back to the perfect storm we

have large language models based upon data and the algorithms and the hardware that allow us to build these coherent kinds of things but again we have to be careful Gary Marcus and I have been voracious and consistent critics of Sam and others who are saying we're just on the cusp of AI AGI well I think that diminishes the Elegance of what AGI actually is within our humans it diminishes what the beauty of what our human intelligence is and frankly we're not going to get there by scaling we've already seen we're we're hitting we're hitting limitations

upon it Elon whom I've gotten into as well on the internet a number of times on his his particular views of the world on AGI he's been promising full self-driving for years now we're not going to get there because those kinds of models are simply they're the wrong architecture if I've got to build a nuclear power plant to build my systems I'm probably doing the wrong architecture and so yes I think there are valid use cases for large language models but we have to be careful because they are indeed dangerous yan yan blocked me on

Twitter because at the very beginning of their work galactic I called him out on it and said you Yan this is great stuff but do you realize the implications and he simply dismissed them and I kept calling him out he got tired of me doing so and eventually blocked me but this is another I told you so we are seeing the the joys and the dangers of large language models so just to double click on this it sounds you're saying that large language models by themselves will not get us to AGI but do I understand

that if we combine it with other tools like neural Nets or neural symbiotic systems we can actually get closer to these intelligence systems and if we take this example that you just said a robot on Mars that is able to clear filters and follow instructions what do you think we would need to get to that type of intelligence right and I even add to this if you look at cultures such as Japan where you have an aging population a shrinking population the US is very much in this place you have tremendous need for Elder Care

and so the robotic use case and not just the full humanoid robotic use case comes into play there are clear and present needs for these kinds of things as well too so yes there there are lots of great use cases so uh I've always believed this to be a systems engineering problem which is why if you go back to the architecture of self that's why I've been focused upon neuros symbolic systems now let me get a little philosophical for the moment you I'm pretty sure you're sentient I can't guarantee it but I'm just guessing that

you are and you're not some amazing large language model and why do I think it's a fair it's a fair guess and why do I believe that it's because I have a theory of mind about you that through multimodal ways and my interactions with you I have a theory that says you're sentient you're someone who has its own agency who has your own feeling and behaviors and needs and the like and that's cool you're not you're not a robot and so the human mind has evolved over Millennium to develop that through an architecture that yes

it's neural in nature but on the one hand neur artificial neurons are a shadow not a shadow they're an echo of a whisper of What organic neurons are they are a small small piece of it furthermore if you look at the architecture of large language models which as you observed are relatively simple there's a simple layering to them which ignores the Exquisite complexity of the human mind we have in our cerebral cortex uh cortical columns we have tens of millions of them they're in humans about seven layers deep reptiles have them as well too they're

a bit smaller those are the ones that appear to be the place our homonculus work where we build the predictive models of the world world but we also have places where emotions and and and other decision- making take place among these entangled architectures associated with the thalmus and the like and furthermore we have other things going on with it appears to be our our hormonal system that appear to be other ways of message passing within our within our Neal networks so in that same regard large language models are interesting but they also are a whisper

of a shadow of I think what reality is and so where the next level is and Gary and I have observed there we're reaching diminishing returns on what scale can do the open AI folks and Elon believe scale will continue for a long long time Gary and I are saying no we're reaching a limit there you need to think about it in other ways that brings us back to those architectures the last thing I'll mention is at the very beginning you talked about me being involved with embod cognition that brings me back to my roots

again what is embodiment it's Building Systems that are in and of the world that that respond to the world and act in the world large language models are largely unimodel they work through text maybe static images woven together through video and the like but they're very sensory sparse our minds and our intelligence have grown because they've been embodied in the world in that regard I think there are many kinds of intelligence and exist but our human intelligence grew because of our embodiment and it's going to require some fairly complex architecture to get that that's why

I've been studying human architectures for six years because I don't know enough about them let's see if that influences the way I build softare architectures and as closing you previously tweeted something really interesting around this and I'll quote this you tweeted we need a standard way of visualizing the architecture as well as the activity of llm and to generalize any artificial neural network netw sort of a uml for AI now this was a year back have you seen anything happen here and why do you think llm architecture is so important uh in fact I have

seen both thought I have both thoughts and I have seen movement so internally I've been helping out with providing a little bit of architectural adult adult supervision in our in our large language model work and I've been trying to figure out what's the way to visualize the kinds of things we're doing turns out it's still veryl like that these are now boxes that are systems under themselves and they're message passing systems along the way so I'm trying to figure out the best ways to do that as you may know just this week um Alpha fold

released its source code and weights publicly so I'm I'm jumping on that and I'm going to use that as a basis of can I describe the architecture of alpha fold 3 using some uml delivera so stay tuned I think there's work to be done in that regard and what advice would you have for software Engineers who are just starting out the software industry there's plenty of recent graduat who find themselves in a Chile job market the there's a threat of what feels like AI tools changing How We Do software engineering you've gone through a lot

of cycles of innovation and software engineering what advice do you have for those starting out to set them up for a successful software engineering career indeed when co-pilot and chat GPD came out I received a flood of messages from folks saying my gosh have I chosen the wrong career because you're not going to need developers at all you will always need people who make informed decisions no matter what the language is software engineering again is one of rising levels of abstraction it just that our tools have changed I learned to program in Assembly Language uh

people today are going to be learning in programming with languages that are at a much higher level of abstraction so the ad i' give such folks is twofold uh don't worry don't be afraid there's always going to be some really cool work for you to do there and in fact I would say it's quite the opposite this is an exciting time because there is so much opportunity so many Cool Tools so much computational resources that in many ways you were limited only by your imagination and so what I encourage people to do is to first

learn as much as you can second don't get stuck in just one domain don't you need to become an expert in some space but the world of computing is vast and so find some space that nobody's in right now and go make a name for yourself there because there are lots of those kinds of places there for you and the third thing i' advise is go have some fun I mean my gosh the toys we have at our disposal they're amazing and wonderful F and cheap there's so much a single person can do at so

low expense to go change the world I I I'd like to also close by saying I'm not done yet either I'm having a tremendous amount of fun and in addition to the things I told you in in studying the human brain and working with large language models there are two projects that I'm engaged in that I've been working with for a long time now uh the first is I've been trying to write a book on architecture I'm glad I did not write it because I know so much more now than I did before and this

is a different architecture book rather than saying here's how you do it I've been I've been working with a number of companies to document their ASB built architectures so I'd like to put Alpha fold in it Photoshop is in it what's the architecture of Photoshop what's the architecture of a climate uh monitoring system what's the architecture of Wikipedia so I'm trying to document the asilt architecture of people of AR of systems that people use today that they may never have thought about the idea being is that there are many different architectural Styles let me expose

you to them because you may have studied only one particular one the world is a vast one the second is I this is the software architecture's guide book right uh yeah the software architecture handbook which my handbook which my long long-suffering editor has been has been uh been patient with me for a literally a decade now and it's something I hope to finish before I die U the second book is I was on the Board of Trustees for the Computer History Museum for about 10 years and we' hired a new CEO he came to me

and said he had been at PBS we had a conversation and said hey uh John why don't you do a documentary like Carl Sean's Cosmos he paused and said well Grady why don't you be our Carl and I said I'm no Sean but that's an intriguing idea so I've been on a journey to do a documentary and writing a book about um Computing The Human Experience which looks at the history of computing and what it means to be human so I'm looking at what is computational thinking look like uh what do it how has Computing

change the individual Society Nations how has it change science and the art and religion and and such and ultimately it asked the question in the present of computing what does it mean to be human so that's the larger project I'm working on right now which I hope to finish up a alsoo before I die and let's wrap up with some rapid questions so I'm just going to ask and then just shoot you don't need to think too much about it what was the first programming language that you used for Tran for Tran what project did

you commit code most recently and what language did you use in it most recently uh my own language my own project self python python what do you do to recharge from doing software inuring related work I live in Maui I wake up that's enough yep enviable place and also answer sir what are two books you would recommend for those who would like to understand more about software architecture Mary Shaw's book software architecture I'd read that one other books kind of pale in comparison but I'd start there well thank you very much Grady for being on

the show this was really interesting and fascinating it was a pleasure thank you for having me thanks to Grady for this great conversation you can find ways to contact Grady in the show notes below if you enjoyed this podcast please subscribe on your favorite podcasting platform and on YouTube for some additional takeaways from our conversation please see the show notes thank you and see you in the next one