New Breakthrough: New Light-Based Computer Takes Over!

over the past few years we witnessed an incredible  AI Revolution which has been driven by AI chips in fact the demand for computing power has never  been higher meanwhile the scaling of classical computer chips has slowed so what's next while  Graphine chips Probabilistic Computers and Quantum Computers are still in the making light based  computers are already arrived in this episode I will break down a new light-based computer  chip which is on its way to data center right now and I can't be more excited about this let  me shed some light on it Photonic Computers have been in the making for decades it all started  60 years ago with the development of optical fiber for communication and over time we got  excellent at sending information with light now if it works so well why not to use light for  computing in fact researchers have been long working on building light-based computers  by now you've likely heard this idea that light-based computers are faster than digital  computers because light is traveling way faster than electrons well it's true and not true at  the same time let's take any conventional chip NVIDIA GPU for example during computation there  is an electron that travels through a copper wire and this wire acts as a conductor and this is  how it always works in fact the problem here is not the speed of electron but the medium itself  the wire one light travels at 300,000 km/ second in this case we are talking about mm/ second and  again here it's not a problem because wire is a conductor so it's full of electrons so here we can  reach speeds way faster than mm/ second now you see we can't simply say that photons are faster  than electrons it's way more complicated than this in reality the real reason why digital computers  are slower than light-based computers because in digital computers we need to switch from zero to  one from one and zero and this switching requires us to charge and discharge a capacitor and this  takes time and this is where the real slowdown is coming from I explained this concept much more  in details in my previous episode on Reversible Computing a great episode make sure to subscribe  to the channel right now and watch it right after this video so by now we understood that the  real slowdown is coming from this switching from charging and discharging a capacitor  which is slow so that's where the light-based chips save the day because nothing like this is  happening in the photonic world in photonics we compute data without stopping it basically we  are computing as a data is flying by and this computation on the fly happening in the range of  femtoseconds which is one quadrillion of a second so it's very fast the main feature of light is not  light as itself but the main feature of light is that you can realize an Analog Computer and this  is the difference it's not so much the light part when it comes to the math it's more the analog  nature of light that you can natively exploit that's also why we call it Native Computing and  the main advantage here is that you can carry out complicated mathematical functions without  digitalization and that's very interesting in fact if we want to perform a simple summation on a  digital chip to add up two numbers we need roughly 200 transistors those tiny devices all the digital  computer chips are built off so then when we want to do a square root of this number we need another  7,000 transistors and then when we want to do a Fourier transform on this we you need roughly 1  million transistors so you see the more complex function you want to implement on a digital chip  the more devices the more transistors the more chip area it will take what's so interesting  when we want to implement a Fourier transform with light we can do it on a single optical  device so you get much higher computational density and you might be wondering how is this  even possible you know people who are wearing glasses if you are wearing glasses you are wearing  every day a Fourier transformator on your nose and it performs this function using no energy at  all once you understand this you can use the same principle to implement such complex operations on  a light-based chip using special photonic elements just think about it we can replace 1 million  devices with just one optical device device and it's passive so it means light just passing  through it allowing you to do complex math without spending any energy at all and the same applies  for multiply operation where on a digital chip we need roughly 1,500 transistors on a photonic  chip we can do it with just one device so we get much higher computational density that's the  reason why the interest in light-based chips is growing at light speed in practice it took many  decades since this concept of computing with light emerged till the time when we figured out how to  actually use it for computing purposes one of the main challenges is that light is really hard to  control it tends to spread out and scatter and it has taken industry really long time but Q. ANT  has finally built a fully functional commercial light-based computer their new computer chip  is called NPU (Native Processing Unit) and it's powered by light rather than electricity  we are already shipping first service to high performance computer centers and we've decided on  that the first processor generations are coming on the standard interface of of the CMOS world  mainly PCI Express and what we actually deliver to the customer are fully equipped servers  which are compatible with x86 structures so in the end you get a server module you plug in the  ethernet cable you plug in the plug power plug and the system operates what's even more interesting  their breakthrough technology relies on a special material they're using so called lithium niobate  essentially they deposit a thin layer of lithium niobate on top of silicon dioxide which sits on  top of silicon and this particular material is Q. ANT proprietary technology which is fundamental  for the success of their computer chip in several ways first of all it's the only material which  allowed them to build all the required optical components in the chip in one material and this  is fundamental for avoiding losses losses of light because losses of light results in the drop of  accuracy in computations so we want to avoid it at any costs what are the fundamental features  of lithium niobate well the first thing is that the modulators so basically whenever you want to  interact with the light we can realize modulators that can operate in the gigahertz regime so very  fast we can realize these modulators that no light is lost in the modulators and the last thing is  the switching so in the end at the technological granularity level what you're doing you're  changing the refractive index of the material this can be done only using a voltage and I know  this sounds super technical but it's elementary because when you only need to change a voltage  there is no electricity on the photonic part of your processor meaning there is no heat there is  no heat dissipation leing again to a very clean signal and we already talked about that clean  signals are fundamental to reach for instance an 8-bit precision so this is why lithium niobate  is not just another material it's basically the fundamental source of success for building  a Photonic Analog Computer in fact the Q.

ANT chip is the first photonic chip which is able  to achieve the accuracy of 8-bit precision now to be honest what striked me the most about Q. ANT  is that they're having their own fab so they are manufacturing their own chips and basically they  own the entire pipeline from design to technology then they manufacture the wafers dice them package  them write software stack for them that's a lot of work this is very untypical situation for a  startup especially owning manufacturing because this is very assets heavy a question is how this  upstart start up is managing it all and the most important why why do they need this fab light  chips the structure of the light chips are per physical definition so by the laws of physics  are pretty large you can't realize a photonic circuit with a 50 nanometer width because then the  light would not be guided so in that sense what we have is we have access to a CMOS foundry an old  CMOS foundry from the 90s and we repurposed it with strategic investments of a few tools to  serve for the production of our own photonic chips so in that sense yes it's not cheap but in  comparison to what you need to invest in the CMOS world it's easy and and this is a big advantage  to the future as well because think about there are a lot of outdated CMOS foundries in the  world which could be repurposed to build high performing chips for the AI next generation AI  Supercomputers but using mature technology from the 90s I mean this on its own is a production  paradigm shift this is indeed a paradigm shift very interesting example of turning so to say  obstacle into opportunity and seeing all the investments governments are making into the  photonic technology and into the photonic fabs and also seeing all tech giants like NVIDIA  TSMC AMD going all in this fab's might have bright future let me know your thoughts in the  comments now before we discuss how this new light- based chip works what it's capable of and how it  compares to the state of the art GPUs for example have you ever wondered how much of your personal  private data is floating around online your name address even information about your family members  unfortunately it all gets out there thanks to the data brokers that spread this information online  and this exposes you to risks of data breaches and of course personal security you've probably heard  about cases where databases containing information about millions of users are sold online and sadly  this is happening more and more frequently that's where Incogni the sponsor of today's episode comes  in Incogni helps you remove your personal data from databases used by data brokers I used Incogni  to remove my personal data from those databases and it's surprisingly easy you create an account  give them permission to act on your your behalf and they send data protection law compliant  requests to these companies forcing them to remove your information from their databases and  you can even track the progress of these removals day by day on your dashboard as someone who values  privacy I highly recommend that you try out and Incogni and put an end to annoying spam emails  and calls use my code INTECH at the link below to get 60% % off an annual plan thank you Incogni  for sponsoring this episode back to the bright new world now it's time to discuss applications  and how it compares to state of the art GPUs and here honestly I spent quite some time looking into  specs trying to make apple to apple comparison but it's really challenging one thing is clear that  this Analog Photonic approach offers way better efficiency just think about it there is no wires  so no resistance no heat generation so these chips require much less power to operate especially at  high frequencies and here we are talking roughly about 30 times better efficiency compared to the  conventional digital chips now with respect to scalability I think the fundamental question  is can we compete with a GPU cluster because this is in the end what our basically what our  direct competitor in the present data center is and to give you a bit outline to the future so  in two years from now we're going to have Native Processing Units so processors coming on a PCI  Express card that have the same performance than a graphic card in two years on the AI relevant  functions but on the same side we anticipate that these systems have a 30x smaller power consumption  than a graphic card in the future now what does this mean if you look on a server today you can  bring eight graphic cards into one server rack and then you're at the edge of what's being reasonable  in terms of power consumption we can bring much more cards into the same space and by that  increasing the computational density in the server and since we still have energy budget left we can  bring much more servers into a server rack and by this increasing so this is the forecast of today  and I might be wrong and it's even better tomorrow but the forecast says that if we equip one of  those servers and we plug the same electricity in as they plugin today already we can exceed  the computational density in the server rack by a factor of 10 what's even more interesting  according to Q. ANT their chip is built for both inference and training of AI models and this is  very interesting you know typically we distinguish between two different kind of workloads a more  simple inference when we have already pre-trained model we apply new inputs to it and we ask to  recognize an object an image for example to recognize a fox and on the hardware level this  typically reflects into performing many multiply accumulate operations in parallel and we've  decided on that we are fully concentrating on the AI Inference and the AI Training so the layout of  our chips is always that an a chip can basically serve both purposes so we can run AI Inferences  which in the end is fundamentally saying similar to a vector matrix multiplication and on the AI  Training we are basically going a different route because we can in contrast to what training or how  training is established using a CMOS equivalent GPU architecture but the chip layout is always  the same and this is very interesting because in order to do training we need to constantly update  the model weights we need to constantly adjust it to improve its ability to make better more  accurate predictions and to do this in photonics might be really challenging as we discussed in the  beginning of the video in photonics this concept of capacitance or storing in intermediate results  does not exist so nothing like in The von Neumann Architecture where we have this local memory in  photonics no storage available in fact it works entirely different here the longer we can make  the light to propagate through the chip without stopping it the more we can benefit from the  properties of light let's say you want to train a neural network first you encode your weight into  the phase of light and as the light propagates through the chip you modify it along the way one  by one and at the output you get the final value and only then you convert it back to digital and  then save it to memory for that the Q.