Open transcript
So this is one of Google's latest models, and it's weird. It's called Gemini Diffusion, which you're seeing is happening in real time. It's not sped up, and here I create seven different apps in just about 30 seconds. Now, this model is not going to be quite as powerful as something like Gemini 2.5 Pro, but as you can see, it's really fast. But what I think is even more interesting is how it creates this text and this code. All right, so this is an early preview for Gemini Diffusion. So it's still very early in development, but it's interesting in part because of how different it is from anything we've seen before. Notice the speed. It spits out 1,300 tokens in just over a second. The highest I've seen, I think, is something like 1,600, although it can probably go higher. So that's all of the Harry Potter books written in about 22 minutes. So if we want to create a xylophone here, we'll click on that. And 1.5 seconds later, the xylophone is complete. Now I can have a soundtrack as I talk. Oh, we try a penguin astronaut, whatever that is. So these are like the pre-built, built-in sort of prompts that Google suggests. Suggested prompts, I guess, is a better way of calling it. Write a story about a penguin named Percy who dreams of being an astronaut. Make it very long, filled with unexpected twists. So here are 3.5 seconds, about 2,600 tokens. Tokens are, of course, either full words or sometimes certain like bigger words are broken down into multiple tokens. Punctuations and all that stuff also costs a token. So on average, you can think of as 100 tokens is like 75 words. If we click on this fireflies, it says to create a simulation of fireflies that move within a confined box. So here we can change kind of how many fireflies we have within the box. And they're attracted to the cursor, interestingly. That's, I mean, it's pretty cool. So again, this is not going to be anywhere as complex, as interesting, as powerful as Gemini 2.5. Don't expect some crazy next generation software produced out of this thing quite yet. But really fast, let's take a second to understand what diffusion models are. So autoregressive means that it spits out some text and it predicts the very next word that is going to come out. And that prediction is based on all the previous words. So the thing that comes before it is the context for what comes next. Where are we? And then going is the prediction. That's autoregressive. And it also goes in a loop each time. So this is actually a great sort of a little illustration for it. So if it says once upon, LN predicts A. And then it starts over. Once upon A, it predicts time. Once upon a time, it predicts the next word, there. What's the next word? Can you predict the next word? Once upon a time, there. And you feel like was is probably the most likely word, right? Or maybe lived or something like that. Once upon a time, there lived a princess. So this is what we call sequential token prediction, right? We predict the very next one, one after the other in sequence. And this can be slower because it has to do everything in sequence. And also as this sort of gets longer and longer, as you can imagine, it gets harder and harder to kind of keep all of that within context. So this is kind of why we might run into context window issues if we have too long of inputs or outputs. And it can't go back, right? So once we get once upon a time there, right? And if you can't, you know, if you need to go back, you can't kind of backspace. You're sort of limited. You can't correct errors after you've predicted a few words. Diffusion models. And this new model is a diffusion model. It works a little bit differently. And that we train it by giving it images of various objects. And keep in mind, most of the diffusion models have been image models. Now we're beginning to see text diffusion models. But I think it's easier to understand how the images work. So we sort of make this image noisy. We add a random static to it until it's kind of meaningless. It's just a bunch of random dots on a page, right? So here you can see this dog going and getting more and more noisier until you can't see anything. There's no data here. And this is what the model is trained on. And then we sort of run it in reverse, right? So we take a random image with just random noise on. We're like, okay, turn that into a dog. And it kind of runs it backwards, right? By slowly filling in the pieces where a dog might be. And kind of doing multiple, multiple turns through that kind of denoising process until finally we have an image of a dog. There's a great video online that kind of talks about the math behind it. But one of the things that the presenter kind of brought up was this idea that Michelangelo actually said this. Every block of stone has a statue inside it. And it is the task of the sculptor to discover it. So you can think of the diffusion models as doing that. So it takes this noisy image and slowly denoises it until it's the actual image. So what this means is instead of a sequential next token prediction, we have kind of this thing where it's sort of the whole thing kind of happens at once. And now it's iterative, meaning it does this, then this, then this. So it kind of like walks through several stages of that denoising process. But it's working on the entire thing, on the entire image at one time. And this kind of means that can be a lot faster because it's kind of like parallel processing versus doing everything in sequence one at a time. It's working on the entire scope of the image at once. And it can kind of maintain a global coherency easier, right? Because instead of having to remember, you know, that whole chain that it was doing prior to that, it's looking at the entire kind of scope. And as you can imagine, if you're drawing it in iterations, you can kind of correct certain mistakes or you can start with the broad stroke, so to speak, and then kind of zoom in on the finer details. One of the most fascinating papers to read for me was called Beyond Surface Statistics. It's about how these diffusion models figure out what to do and how to draw certain objects. How they acquired those skills is quite mind bending, I would say. We'll get to that in just a second. But really fast, this is the Gemini Diffusion, the experimental text diffusion model. Join the waitlist if you want to get on it. It took me about a day since I signed up yesterday. I got it today. So your results may vary. But they're saying large language models are the foundation of generative AI today. We're using a technique called diffusion to explore a new kind of language model that gives users greater control, creativity, and speed in text generation. So this is kind of their illustration of how this diffusion model works to sort of, you know, make the text to correct the text. You know, in effect, this is slowed down so we can see what it's doing because, as you saw, this thing produces results incredibly fast. And so the kind of the advantages and what we're hoping to see with these models moving forward, I mean, we're seeing the rapid response to speed already. It promises to have more coherent text, right? It generates entire blocks of tokens at once, meaning it responds more coherently to users prompt and auto regressive models. And iterative refinement. So it corrects errors during generation for more consistent outputs. Here are the benchmarks. So as you can see, it's very similar to Gemini 2.0 Flashlight, right? So it's seemingly very, very similar in results to that model, which is a small model. It's one of the little bit older models. But this diffusion model seems to be very comparable to it. So here's an example of some stuff it can do. So it will tell you repeatedly that it can't create images. It can. It does it kind of in HTML. So it's coding them up. So as you can see here, I asked it to create a round dragon creature that's animated. So there it is. As you can see here, it's kind of hovering up and down. And here's how we can make additions to it. We're going to say add a fire around it. I can't help with this request. This is a very early thing. So you got to give it a break. But as you can see, OK, so it added some flames. They're kind of floating around this dragon creature. So as you can see, it takes 2.1 seconds. Like it's lightning fast. So if you wanted to create some simple video game animations for a video game you're creating, coding it up in HTML in this case, but I'm sure it can do other languages. I mean, it might be so much faster than drawing it or even generating it in some other AI generation software. And then you can make different additions to it. And let's say we wanted to have an attack animation. We're going to say now create an attack animation for it. And OK, so it's giving me a lot of gripe now. It was so much better when I was starting out. Now it's refusing a lot more requests. But again, this is like early testing beta, all that stuff. We're going to say create a sleeping animation for it. Is that less violent? Is that the issue here? OK, so it's kind of closing its eyes to sleep. That's pretty cool. Now make it wake up in alarm animation. Let's see what it does there. But as you can see, yeah, I mean, that's pretty good, right? Kind of. Oh, it did it once. Let's make it loop. It only did it once, but I'm sure it can be kind of put on a loop. So it's like waking up in alarm. So you can see here, super easy to create the quick bytes of code very, very rapidly. That took 1.1 seconds. We're going to say make a forest with moving bunnies on it. So again, it goes with HTML, creates a little forest. Are these bunnies moving? Let's see. By the way, you can see the code up here if you wanted to. Most of these things it spits out, they're usually a few hundred lines. So 199. That's usually in general, like kind of what you expect it to spit out. But it's not moving. Let's start a new context. So we asked it for the same thing. In two seconds, it comes up with this. So now something's moving. Okay, I can see the little bunnies moving kind of strangely. They're more like floating bunnies. But you can see how rapidly it makes all that stuff, makes all the changes. Here we're making it make a planet tic-tac-toe game, a 4x4 tic-tac-toe game. To use a Saturn and Earth emojis for the players. So I will be Saturn. That is Earth's move. I'm going to go here. Is it too obvious what I'm trying to do? Oh, I guess not. Okay, I win. Well, the AI needs a little bit of work. But the tic-tac-toe AI, not the diffusion AI. Because this is looking pretty good. I mean, kind of nailed everything. We're going to say make a snake in the matrix style. Have it shake when I eat fruit. And away it goes. And let's see how many seconds. 3.7 seconds. That's pretty good. Let's see what's happening here. There we go. For some reason, it's not eating the fruit. So I told her that the fruit is on a different grid than the player is. It's still not doing anything. Can't eat the fruit. Fix it. The one thing is, even when it makes mistakes, it's kind of okay. Because of how quickly it's able to fix it. How quickly it, you know, changes the code. But for some reason, we're having an issue here. And in terms of how quickly you can write, I'm going to click toast. It says, you know, explain the merits of toast. And translate into like 10 other languages. As you can see here, it's translating into a bunch of different languages. 1,000 tokens per second. So almost 8,000 tokens at 7.5 seconds in a whole bunch of different languages. If I take this, I'm going to say translate this into 20 other languages. Actually, I'm going to say like 40. Let's see how far we can go with this. And I'll just paste that in there. Let's see how quickly it's going to do that. And there it goes. I can't even read that. I can't even read what languages are being presented. I just, it's so fast. Yeah, it goes by so fast that I can't even understand what's being written. Meaning I can't even see like if a language is familiar, like this is Lithuanian. It says it in English, right? The name of like Irish, right? But I can't even read this as it like flies past. So as you can see here, it's generated 16,000 tokens. Unfortunately, we've crashed the service. So I do apologize about that. So a lot of people, when I posted this online, they were, you know, asking like, how good is it at coding? You know, don't expect anything next generation, right? The large language models are currently the reigning king. There's nothing quite as good as what we have with, you know, Gemini 2.5 Pro with Claude 3.5, 3.7 with OpenAI's, you know, 03 or 4 Mini, etc. There's nothing close to that. And of course, Grok, we're supposed to get 3.5 soon. So I'm definitely going to test that out when we have it. And apparently Grok 4 dropping later this year. Now, I thought we were going to see Grok 3.5. It was supposed to come out last week or two weeks ago. And you can see the website changing. They were adding stuff. So you saw some sort of a forward momentum, but it just did not come together. But I mean, this version of Grok is not a bad coder. So if there's big leaps with 3.5 and 4, certainly that's going to be very exciting. We're going to have another very strong contender in the running for the best coding model. A lot of people complain if I forget to mention that Claude is there. Or some people get upset if I mention that Grok exists because we all have our favorites, etc. But I think the point is the more awesome models that are out there competing, the better it is for us. You know, we get better models for cheaper. We get competition like we win. But really fast, there's this paper beyond surface statistics. It's out of hardwood and it's about diffusion models. But there's a very similar paper in some ways that's about large language models. That's called Othello GPT. And there's a very interesting piece in all of them that kind of gives us a glimpse into how these things work. Why are they smart? Why are they able to do these things? And as they say here, you know, these diffusion models, they have an impressive ability to produce realistic images. But how they work is kind of mysterious to us still. And as an example, I say, you know, when we train them purely on images without explicit depth information, right? So we give them 2D images. We don't tell them like the depth on there. So if you think about it, if I show you a picture of a house, you know, it's a flat image. It's pixels on a flat plane. Now, you, you've seen a house before. So you can kind of figure out how it would look in 3D space. You can kind of think about what happens if you go on this side of it, on this side, if you look at from above, etc. That's only because you've seen depth out there in the real 3D world. That actual image has none of that information. It's just pixels on a flat plane. So we train these models on these 2D images. And somehow they're able to put together coherent pictures of 3D scenes. They kind of understand how light works, reflections, how shadows work, stuff like that. They understand, they seem to understand depth and 3D-ness of objects. How? And so a lot of people kind of say, well, it's just sort of these like surface statistics, right? So they kind of understand some correlation. When we say dog, they kind of understand how certain pixels, kind of the statistical distribution of those pixels, if we want to see a dog. You know, we've heard that explanation before. It never really made sense to me. Not if you saw enough of what these things do. This idea that it's just statistics, it doesn't make sense. There seems to be something else that's happening that we don't fully understand. And maybe it is statistics in the end, but how, you know? As they say here, it's a mystery how, let's say, they take the words car in the street and turn that into a picture of a car in the street. Do they memorize superficial correlations between pixel values and words? Or are they learning something deeper, such as an underlying model of objects such as cars, roads, and how they are typically positioned? This seems to be kind of more of what's happening, right? And so they trained one of these models. Again, no 3D depth information, just 2D images. And they used a probe to kind of look into the inner workings of this model to kind of see how it's doing it. What is it doing to create those images? And so we give the prompt AMG301933 minor two-seat, right? So a specific car, right? This is kind of what the car looks like. And so the fusion model, right, it starts to decode. So the first ones you can see here, like it's just noise. Or maybe here it's beginning to do something to decode it. Step 1, step 2, step 3, step 4, step 5. So by step 5, I mean, you can kind of see the outlines of the car. You can kind of see it taking shape here, right? So this is normal. This is what we know. This is what we can see. But with this probe, they were able to kind of look into how it was kind of representing the things on the screen. And interestingly, what they found is like early in this process of it denoising the image, of it creating the image, there was depth from internal representation. So it kind of had an idea of where the foreground is, where the background is, what's further away from sort of the camera angle, what's closer. Even though, keep in mind, it never has any data to support that. I kind of just figured out those sort of concepts. So this red means it's closer to the camera. Blue means it's farther away into the distance. And if you look at this image, you can tell, like, what's the closest thing to you? It's probably this wheel. What's the farthest thing away from you? It's whatever back there, right? The tree line or whatever that is, right? And you can see from the very first sort of step, even though you can't really see what it's doing, we can't tell what this is going to be. We know that this thing is going to be closer to us. And this blue is in the background. It's farther away. And as it continues, you can see it kind of taking shape. Here's another where you can kind of judge the depth from the image. And here, the salient object. So it kind of understands what the main object is, right? So when you tell it, you know, this car, it knows that the car goes here. From very early on, it sort of, like, knows that this is where the main thing goes. And everything around it is, like, you know, decoration, like some grass or whatever. So there was an interesting interview between Andrew Ng and Jeffrey Hinton. And at some point, Andrew Ng is saying, do AI models understand the world? He continues, we think they do. And he talks about two, I think, a fellow GPT and maybe even this Beyond Surface Statistics paper as support for what he's saying. He's saying these models might understand the world in a way, if we define it, understanding as having a mental model that is able to predict what happens in the world. Right? So if this model, this AI model, developed some sort of an internal model in its brain, so to speak, about how the 3D world works, even though it's never seen it, it's only seen 2D images. But it kind of went, well, I think this is kind of how it works. This is how shadows work. This is how light works. This is, you know, what a car is. It develops some sort of a mental model about how to produce those images. And that's likely how large language models work and how AlphaFold and all those other ones work. They create some mental model to be able to predict these answers. And this is where, you know, if we're talking about whether or not these models understand, it really comes down to, like, how do you define understanding? If we define it as having a mental model that predicts the world's outcomes, then it certainly seems like understanding. Maybe not the same as human understanding, but some sort of understanding of the world. Because, again, it only saw 2D images. And by seeing enough 2D images, it begins to understand about depth. For it, that's just kind of this abstract concept that it's never encountered before. It just created to sort of understand how to create images that look good to us, that appear to us like they're, you know, of the 3D world, of the real world out there. So check out Gemini Diffusion just to see how fast that is. Don't expect amazing stuff. But do understand this is a completely different approach. The fact that this thing can generate code is kind of wild, right? Because, again, when we write something, how do we write it? We write it, you know, a letter at a time. We predict what word comes next and we keep writing it. Can you imagine, like, taking a paper and just start shading it in? And over time, this passage sort of just reveals itself. Like, you generate the entire passage instead of predicting it word by word by word. That seems kind of wild. The fact that it can do that is just something interesting to think about. And I think if this is a promising avenue and Google continues to develop it, and this is a way that maybe over some time period can approach how good large language models are now, I mean, it sounds like that would be incredible because, again, it's fast. It has a long-term coherence and it kind of has this iterative process of fixing mistakes as it's creating things step by step. It can kind of, if it sees something going wrong, it can kind of fix it on the fly, so to speak. So definitely very exciting. And hopefully this is a brand new avenue of AI progress. But let me know what you think. If you made it this far, my name is Wes Rob. Thank you so much for watching and I'll see you next time.
