Moore's Law is Dead — Welcome to Light Speed Computers

Mo's law might be dead for decades we've relied on the idea that the number of transistors on a microchip doubles every 2 years leading to exponential growth in computing power this principle has fueled the digital Revolution enabling everything from smartphones to the internet but we're reaching the limits of silicon based Electronics transistors are now approaching the size of individual atoms and we can't go any smaller without running into Quantum and thermodynamic uncertainty this means that you can't shrink them anymore the world has decided that path towards making computers better is making them bigger and all

the while our Ambitions for artificial intelligence are skyrocketing the last seven years has been an incredible time for AI the compute has gotten so fast that in many ways the networking is now the bottleneck AGI isn't just a concept in sci-fi anymore it's a tangible goal but achieving AGI will require vast amounts of computational power and data potentially far beyond what our current technology can handle so is that it is that the end of computation in Moors law or maybe the next step is beyond the atom and towards the speed of light using light some

believe the answer lies in photonics a popular subject in computational literature but never before has it seen mainstream commercial usage until now we've built photonics at a scale that no one's ever seen before something that just wasn't possible over the past decades and at the Forefront of this revolution is a company called light matter [Music] [Music] light has been revolutionizing how we send information since the 1800s when people started sending signals using flashing lights and mirrors but the real breakthrough came in the 1960s with the invention of fiber optic cable thin strands of glass that

can trap light and use it to send data this transformed how we communicate today most of the internet now runs on these vast network of undersea cables connecting us to the internet while as a species we've gotten really good at sending information using light actually Computing with it or using light to do math has been much harder but scientists have dreamed about Optical computers since the 1980s because light is so much more fast and efficient than electricity but controlling light for computation is incredibly tricky light wants to spread out and Scatter and getting it to

make sharp turns or stay in tiny spaces goes against its very nature it's taken until now for Science and companies like light matter to start making technology like photonics viable photonics is the science and technology of detecting generating and controlling photons or light by contrast photonics uses Optical fibers or wave guides on a chip this means data can be transmitted faster over longer distances with less energy lost in light matter is pioneering this field by integrating photonic components directly into computer chips enabling the new method of chip communication that could lead to AGI to understand

better how this technology works and the limitations of today's computational methods we filmed with CEO and co-founder of light matter Nick Harris in light matters paloalto lab we're in paloalto and we are working on what will be the future of how computer chips and the supercomputers that power artificial intelligence are going to be built I'm Nick Harris and I'm the co-founder CEO of light matter at light matter what we're doing is we are enabling these AGI supercomputers that are being built toward realizing these machines that are able to think at the same caliber as the

best humans on the planet so what is Agi AGI or artificial general intelligence is a bit of a Hot Topic lately and the general theory is that it's an AI that could be capable of understanding learning and performing any intellectual task at a human level or maybe even better however recent acceleration and the AI boom we're living through has made us take it a lot more seriously what would it take to make it real and what would it take to compute it named after IBM researcher Robert Denard for ades this was the magic rule of

computing as we make transistors smaller they would use less power and run faster it's why your phone today is more powerful than a room-sized computer from the 1970s but around 2006 this free lunch of better performance ended we hit fundamental limits of physics and suddenly making chips better got a lot harder and a lot more expensive this is why companies like light matter are so exciting they're trying to reinvent Computing itself to keep scaling as we've known it going so my name is Sim Janowski I'm the new CFO here at life matter I was previously

at Nvidia for 7 years heading up investor relations and strategic Finance amazing time to be there life matter is a pretty amazing company it was tough to leave Nvidia I have to say it took a lot of sleepless nights and a lot of deliberation and thinking about it it came down to in terms of my decision to come here is the combination of the team and the technology how did you first get interested in computers so I think for me interest in Computing Maybe started with gaming and then it was figuring out how to write

game programs and how to create websites and then how to build computers and then once you're building the computer the question is how do these things work and what exactly is going on inside the machine I went to work at a big Semiconductor Company studying transistors which are the device that ultimately Powers all modern computers and what I learned there is that there were some fundamental challenges associated with continuing to make computers better faster more efficient what I discovered while I was figuring out how I could contribute to Computing in general was that light was

probably going to be a very big part of the story and so I went to graduate school at MIT and studied uh fonics using silicon the same platform that all computer ships are built on today and I specifically studied Quantum information Theory and Quantum Computing what I learned along the way was intricately how light works and what the future might look like uh with light compared to electrons light is pretty ideal for communication in transmission of information it enables faster and more reliable interconnects over long distances because it's literally traveling the speed of light also

light generates less heat and consumes less energy making it sustainable for large scale systems like these AI supercomputers we're talking about today while electrons dominate the field of computing today companies like light matter well are trying to make light matter when people think about light they think about the colors they think about light emitted by the Sun but it turns out that everything that has a temperature is Shining Light and light is actually fundamental in uh how we build Electronics today but people don't quite realize it it turns out that the signals that radio send

and many forms of communication are actually already light and the problem is that people haven't quite realized that light's going to be going into computer chips to make them faster and more efficient and we're working on that [Music] future light matter was the first ever company founded to commercialize photonics technology and now they're ready to talk more about their two products passage and advise so this is passage this was built in 2019 it's a 200 mm by 200 mm chip so what's included here is an array of 48 Max maximum size chips that are stitched

together with Optical wave guides going across the entire platform and by bringing wave guides close to each other and building different geometric shapes we're able to manipulate light encode information in light and ultimately enable these gpus and switches to build these data center scale supercomputers there are Optical fibers the same Glass Technology that's used for connecting the world is attached to the side of our onic chip built on Silicon ultimately passage looks a lot like a electronic chip except that we've got light moving around the die and if you look at it through an ear

infrared camera you'll actually be able to see what's going inside the chip and what we're able to do is by programming the chip we can set up how all 48 of the tiles sitting on top of Passage are connected and the way that this works is you take a customer GPU or a switch or CPU and you integrate it on top of Passage and now those chips can communicate using light so we realized that there were some fundamental challenges both with how chips do computation and how they transmit information between each other especially in the

context of these large AI workloads that use teraflops and peda flops of computation operations per second and what we realized was that uh by using silic and fonics and using light we would actually be able to Drive the Energy Efficiency Drive the speed and continue the road map for computing and here is what's coming soon so this is the next Generation passage and it's able to communicate at over 100 terabits per second so in this chip we're able to attach 256 fibers uh the largest optical fiber count in the world and the highest bandwidth chip

in the world think of this as a super highway for data passing between different chips it uses like to transmit information between processors enabling unprecedented bandwidth and ultra low latency communication this opens up a new frontier of speed and interconnect ability that current GPU clusters lack today additionally as we try to push more data through more interconnects around the chip we run into What's called the shoreline problem where you're literally running out of space to plug electrical wires into the chip photonic Sid steps this entire issue by using optical wave guides within the chip itself

this allows for data to be transmitted through the entire chip massively increasing the potential bandwidth this is actually the flow for assembling one of these Enis chips so youve got two 25 billion transistor die and those diee get attached to the substrate and then you flip that over you attach the four fonic cores and then you attach another substrate that allows you to communicate with the socket electrically to deliver power and signals and that turns into this chip here which goes on the card in electronic chips data transfer is limited by the electrical resistance of

the materials in the capacitance of the circuits which causes delays and energy loss and visce is its own all-in-one photonic Computing chip it combines traditional Electronics with photonics to perform computations using white this isn't just a small upgrade it's a completely different way of thinking about computation imagine data flowing through the computer at literally the speed of light using different colors to process multiple things at once while traditional computers are stuck waiting for electrical signals to bounce back and forth like a very fast ping-pong game light moves through these chips without any delay and here's

what's really clever they can stack what light matter calls virtual processors on top of one another almost like creating multiple computers in the same physical space they're doing all this while using far less power than regular chips which is amazing because energy is one of the biggest costs in Computing today especially for AI this isn't just making computers a little better it's reimagining how they work with light as overall computing performance is increasing today's bottleneck is not about the chips or the speed of these chips it's about the networking or the interconnects between these chips

that's the bottleneck we're seeing today networking is the communication in data exchange between multiple processors or chips or integrated circuits within a Computing system networking is critical for enabling efficient performance in modern multi-chip systems the last seven years has been an incredible time for AI and a lot of what I saw is the unbelievable increase in the pace of computing a thousand times increase in the performance of a single chip which was amazing even more amazing in that it came after mors law had essentially ended but a lot of those fces are starting to hit

certain walls and most of that has to do with the connectivity so the ability to connect thousands hundreds of thousands and millions of those chips to each other to be able to communicate at the speed of light and that's really what's going to be needed for the next 10 20 plus years in Computing and so the next challenge was really the one of how do you interconnect them at scale transistors are so small at this point that they're approaching the size of the electron and this means that you can't shrink them anymore so you've got

to figure out new way for chips to get better and the world has decided that the first path towards making computers better is making them bigger the reason it's troubling that transistors were hitting bottlenecks and fundamental limits is that in almost every field these things are all tied to progress in Computing our ability to simulate the world perform calculations connect in real time so if progress wasn't going to continue at the same rate it always had been there would be a Slowdown in the future coming and I wanted to make sure that that wouldn't happen

and try to figure out how you could augment computers sort of like the bionic man and add light to these systems my name's uh Steve Clinger I'm VP of product with light matter I've been at the company for about 2 and a half years tell me more about how the chips themselves and their speed isn't necessarily in the bottleneck that but that it's more more than networking between them there's I guess a few relevant Trends here you know one is pretty well known that the size and complexity of the models over the last few years

has increased dramatically in these large language models to train something of that scale you need a huge number of compute units but the challenge is they actually all need to be able to communicate at very high speed with very low latency or delay amongst one another in the traditional design those compute units the points at which they're trying to escape all of this data they're Bound by the shoreline of the chip so the electrical Shoreline of the chip is the fundamental limiter you just run out of space to put the high-speed signals that are sending

all this data and the only thing we could try to do is make those signals run faster at a certain point you just can't fit more of them what we're doing is we're allowing bandwidth to escape anywhere in the area of the chip get a dramatic scaling in the amount of bandwidth that can be simultaneously communicated outside of these chips also they can simultaneously communicate with more similar chips today the biggest supercomputers in the world are around 100,000 gpus and that's doubling every year the challenge with today's supercomputers is that the life of a GPU

in these systems is one where they're sitting there waiting for data from memory twiddling their thumbs waiting for data from the neighboring GPU and one of the really big challenges for the computer scientists and machine learning scientists who train AI models on these supercomputers is figuring out how to leverage the fact that it's not really a single supercomputer it's a bunch of little supercomputers that are tied together with very low bandwidth interconnect and at light matter what we're doing is we're enabling these supercomputers to act as a single massive chip so today if you look

at how gpus are built they sit on top of a silicon substrate and you've got a GPU and the memory elements integrated on top of that silicon substrate what we're doing is replacing that silicon substrate with a new silicon substrate that enables those chips to send data using light and so what we've managed to do is figure out a way to slipstream Next Generation technology into the existing [Music] ecosystem to see this Innovation up close light matter has built their own Min fabrication facilities and uses Advanced microscopy to develop and inspect their chips having these

capabilities inhouse allow them to rapidly prototype and iterate on their designs R Tesh Jane senior VP at light matter gives us a tour of their clean room facility I am a senior vice president here at light matter I lead the engineering and operations team my team essentially focused on building products that are end to endend all the way from Silicon engineering down to system level and rack level validation so this is our clean room facility this is an endtoend capability where we're going from a silicon that's showing up on a wafer all the way out

to a packaged unit that has fibers attached that can be deployed into a system for uh you know verification and validation at the system level what you're looking at here is essentially are you know the fiber attach process where we're dropping the uh FAU onto the VR bank and using that Shining Light we're evaluating whether or not there is active alignment of those fibers in the vrps and then once that's done once the active alignment is achieved we're able to dispense epoxy to ensure that we can seal the fibers uh in the wi groups here

we're looking at the wave guides that lead to the input of the chip so this is how you get light into light matters products so here you can see the wave guides the optical fibers sit in these grooves they're actually like a V that's etched into the Silicon so inside of each one of these uh fibers we have 16 colors of light and these are all around a wavelength of 1300 NM in in wavelength and the light is coupled into the wave guide that's built into the Silicon the wave guide is a few hundred nanometers

wide and about 100 nanom tall and inside that tiny little device you've got 16 colors of light that are traveling together the reason light was interesting is that it relies on a fundamentally different set of physics computers today are built on electronics and in electronics you have this particle the elect R and it's governed by a specific set of properties there are things like resistance inductance and capacitance and these describe the power dissipation so how much energy the chips are using how fast the chip itself can switch and you compare electrons to light what you

start to realize is that light would be very interesting it doesn't have a concept of resistance or inductance or capacitance light is operating in the hundreds of terahertz regime electronics are operating in the gigahertz regime some other interesting properties are parallel communication that's able to happen in a way that you can't achieve with electrical signals silicon photonics uh has been developed over the past 20 years around the year 2000 the technology really started to uh reach some level of maturity where they started to think about what systems of fonic components might look like now the

challenge 20 years ago was that there were no foundaries or commercial partners that would be able to take these things and scale them and there was a bigger problem electronics and electrical wires were still the main medium for doing this and those wires were able to continue making progress at a sufficient rate for people to not look at what the future might look like and at light matter what we've done is we've built photonics at a scale that no one's ever seen before millions of optical components on a chip something that just wasn't possible over

the past decades I love chips every time I get to see one of of the chips that we're building it it blows my mind from the work that I did in graduate school with my co-founder Darius uh to today the advancements we've made are huge and it's just pure joy seeing the things that we built developing this kind of Technology isn't easy in fact it's never been done before it requires significant investment cuttingedge research and the perseverance to overcome technical challenges but the potential rewards are immense the death of Moore's Law doesn't Mark the end

of technological or economic progress instead it marks the beginning of a new age of computing only limited by the speed of light [Music]