Open transcript
So there's some big AI news today, but first, here's something out of China where they're teaching a robot how to dance. All this is real. None of this is AI generated video. I really enjoyed watching this until this one point at which I stopped enjoying it. See if you can spot the moment that, yep, yep, yep, there it is. They gave it an axe. Why? Why did they give it an axe to produce this nightmare fuel? There it is. Just imagine this thing going for you, coming for you with the two axes and it's chop, chop, chop, chop, chop. That's our reality, friends. That's the future. I mean, I can't help but be impressed, but why do they have to demonstrate it with the two axes and it running back and forth and just chopping things up? That's that's a little wild. I mean, phenomenal, phenomenal on one hand and terrifying on the other. In other news, if you weren't aware, China's doing really well with robotics and AI, like really well. And one thing that I've been noticing for a while now is a lot of it is open source. A lot of the leading Chinese AI models, some of the leading AI robotic companies in China are open sourcing their research, their code, everything that they produce. So it's available for everyone to use for free. Or if they, for example, provide the hosting kind of the, they run the model for you. It's a lot, a lot cheaper than the Western counterparts. So here's Balaji who has in the past predicted some things pretty well. I feel like, so that's him right here. So he's a Stanford, Coinbase, A16Z. So somebody who tracks tech pretty well, somebody who is able to kind of fairly accurately predict where things are going. Here's his take on China and open source. And by the way, this seems to be becoming kind of the more sort of mainstream view, accepted view, maybe. So he's expecting a complete blitz of Chinese open source AI models for everything from computer vision to robotics, to image generation. And we've been already seeing kind of a steady stream of it coming out of China. The release of the deep seek models, you know, wiped in a day, something like 1 trillion off of the global markets. There's a lot coming out of China that's open source and very remarkable. So he's inferring this from public statements, but the apparent goal is to take the profit out of AI software since they make money on AI enabled hardware. Basically they want to do to US tech what they already did to US manufacturing, namely copy it, optimize it, scale it, and then wreck the Western original with low prices. He's saying, I don't know if they'll succeed, but here's the logic. First, China noticed that deep seek release temporarily knocked 1 trillion off of US tech market caps. So when that got released, a lot of people got spooked all over the world. They started pulling money out, NVIDIA specifically, but many other companies as well. Second, China's core competency is exporting physical widgets more than it is software. So for example, you know, the, the axe murdering robot is the only thing that comes to mind, but I'm sure there are others. It's just the only thing I can think of is that axe murdering robot. And I know it didn't actually murder anybody, but it's just, that's what I'm seeing it doing in my mind. But the point is China exports atoms, not bits, right? So physical stuff instead of software stuff. Most of the things that US is leading on in terms of AI is the quote unquote software, neural nets, all that stuff. Third, China's other core competency is exporting things at such massive scale that all foreign producers are bankrupted and they win the market. See what they're doing to German and Japanese cars, for example. Fourth, China is well aware that it lacks global prestige as it's historically been seen as a copycat with a deep seek becoming number one in AI is now something they actually consider possibly achievable and a matter of a national pride. Fifth, deep seek has gone viral in China and its open source nature means that everyone can rapidly integrate it down to the level of local officials and obscure companies. And they are doing so and posting the results for praise on WeChat. WeChat is massive and most people here in the US really haven't seen it or used it, but it's a big chunk of a social network. So here's the instant messengers of the world. So by active monthly users. So most people are probably familiar with WhatsApp, Facebook messenger, Telegram. But as you can see, there's four other ones that are quite big that a lot of us here in the United States, at least are not going to be familiar with. And while deep seek was obscure before recent events, it's now a household name. And the founder met with Xi, but also number two in China, Li Qing, they also have unlimited resources now. So certainly similar to how the US state is supporting AI infrastructure and companies and progress, certainly it's safe to assume it's pretty much guaranteed that the Chinese state is also supporting their various AI efforts in much the same way, or perhaps even more. So if you kind of think about that, China thinks it has an opportunity to hit US tech companies, boost its prestige, help its internal economy, and take the margins out of AI software globally, at least at the model level. The other big thing I would add is if you think about over the next few decades, the AI sort of infrastructure all over the world will be built on someone's sort of foundational tech. It could be the US, it could be open source, it could be China. And a lot of companies like Google, OpenAI, Anthropic, they did kind of mention this, like all this stuff globally will be built on the leaders of AI, whoever that is, right? So if it's China, if they're open source technology, everyone in the world will be building on top of that. If it's US, then everybody will be building on top of that. And this is something that the US is thinking about, this is something that they want to maintain a lead in. But the point is, the US frontier labs, the AI companies will not be able to make money if the alternative is free open source software, or extremely cheap open source software. It's near impossible to compete, not just because of the pricing. It's also because of the fact that you can run open source software on your local machine, you can fine tune it, quantize it, you can turn it into other models that you can use for your own special use cases. And you can kind of get rid of any sort of censorship that you want. So for example, the DeepSeq models were doing some pro-China censorship, right? So certain things that the Chinese government didn't want that model talking about, it would not talk about. By the way, somebody turned some of those events into a Ghibli style illustrations, just FYI. But very quickly, after that model got released, Perplexity changed it into kind of a US facing version that would show that information and talk about those subjects. So basically, you don't want to have a company that's competing against an open source version of whatever you're doing, especially if it's as good, even if it's not quite as good, just the fact that it's open source, it's a big deal. It's hard to compete with. So if China is able to undermine the US tech companies, they will instead make their money by selling inexpensive AI-enabled hardware of increasing quality, from smart homes and self-driving cars to consumer drones and robot dogs. By the way, this is Balaji. This is sort of his opinion that we're reading here. I feel like, you know, I agree with most of it. All of this, the reasoning, everything else, it makes a lot of sense to me. But just keep in mind that these are his words, not mine. And he's saying basically China is trying to do to AI what they always do, study, copy, optimize, and then bankrupt everyone with low prices and enormous scale. I don't know if they'll succeed at the app layer, but it could be hard for closed source AI model developers to recoup their high fixed costs associated with training state-of-the-art models when great open source models are available. Last, I agree it's surprising that the country of the great firewall is suddenly the country of open source AI. But it is consistent in a different way, which is that China is focused on doing whatever it takes to win, even to the point of copying partially abandoned Western values like open source, which seemed like the hardest thing to adopt. At that point, they did build censorship into the released deep CKI models, but in a manner that's easily circumvented outside of China. So yes, exactly. That's what I was talking about. I believe it was perplexity that rebuilt it very quickly after the release to take out that censorship. So you might conclude they don't really care what non-Chinese people are seeing outside of China in other languages, as long as this doesn't interfere with China's internal affairs. Anyways, this is an area I've been watching and my reluctant conclusion is that China is getting better at software faster than the West is getting better at hardware. By the way, recently, DeepSeq updated the V30324 March 24th. It's a major boost in reasoning performance. Keep in mind that this isn't a reasoning model. This is their sort of a base model. It has stronger front end development skills, smarter tool use capabilities. So the left light blue line is the original DeepSeq V3. The shaded deep blue line is sort of the new DeepSeq. So as you can see in each of these categories, it's either a small boost or quite a large boost in its abilities. And if you noticed this deep gray line, that's the GPT 4.5. So as you noticed, it got a lot closer to it. It's better in some of these use cases, and some is just a little bit behind. But overall, looking at these charts, you'd say that it's probably on average a little bit better. Again, everybody kind of uses their own tricks to display that their model is the best. They show the sort of benchmarks on which they're better than the competition. So benchmarks tend to be kind of a good kind of a jumping off point to just kind of see where it roughly is, but you don't want to just rely on those. However, there's a lot of people online that are testing it out and are saying that it's impressive. People are saying it might not be better than Claude 3.7 Sonnet coding, but it's better than most other models. DeepSeq V3 is not just a small update. It's a world of difference in terms of front end design. It might be better than what the R1 is capable of. So the DeepSeq reasoning model. We have some interesting breakthroughs out of China. This research paper, Jarvis VLA. It's basically a vision language action model that plays Minecraft because why not? And they have their own GitHub repository. They have it on Huggy Face. So some of the authors are out of Peking University. And as you can see here, this Jarvis VLA vision language action model running on Quen 2, an open source model, which is just 7 billion parameters in this case. So you can see here, it just completely smokes all the other approaches that were used. It's better at doing various in-game actions in Minecraft. It's better at mining blocks, killing entities, crafting items, smelting items. So as you can see here, it's just really, really good. A lot better. I think it beats it out in every category as far as I can tell. I might do a full deep dive on this. This was pretty interesting, but they found a new approach to train these vision models that seems to work really well for these tasks. Normally how they're trained is you're using sort of expert footage, right? So Minecraft players, as they're doing tasks, those training pairs are collected. So it's like, here's how you mine a tree, right? And then you show player mining a tree, chopping a tree. What do you do in Minecraft? You punch a tree. That's what it is. You punch a tree until it turns into, you know, wood. So it's like, here's a player punching a tree. That's how you get wood. If you haven't played Minecraft, I swear that's how it works, at least in the beginning. And that's normally what these models are trained on. So normally the training pipeline is they take a pre-trained vision language model, right? So something that's like, that's able to see and also has language, like an LLM with vision. And we train them via imitation learning on large scale trajectory data. So the point is they're kind of like doing this post-training with examples, right? Here, they have a bit of a different approach in that they start by just doing text-based world knowledge. So they'll say like, what is a boat? And this vision language model says, well, it's this drivable vehicle entity used primary for fast transport of players and passenger mobs over bodies of water. Bamboo rafts look different, but function identically to other boats. So notice how this text, it sort of gives you everything you need to know about those objects, specifically in the game of Minecraft, right? This is not necessarily in the real world, right? And it really condenses and compacts all the info, right? So it's a drivable vehicle entity, right? So your player is able to get in there and drive it. It's for fast transport, meaning it's going to move faster on water than the player would. What can transport players and passenger mobs over bodies of water, right? You see how it like really compacts what that thing does in the game. And then how is, if there's different versions of it, they're saying, well, bamboo rafts will look different, but function identically, right? So that really compacts what that item is in game, what it does, right? And then another question is, can stone pickaxe be used to mine diamond blocks, right? And there's this long answer saying, no, this is how, what you need, a diamond block, whatever. Point being, this training really helps it sort of in words, understand how the game world works. Then we train it to kind of visually be able to describe what it's seeing. So for example, you know, caption this image here, so you can see kind of this green landscape. So it kind of describes what it sees, but it's also sort of interpreting in terms of the game world, what's important about that environment. This peaceful environment is ideal for resource gathering, building a base, or exploring hidden caves beneath the surface. So it gets like, this is where you might want to start the base, right? Not near lava or on some island or whatever. So it's, it's able to describe images as they relate to playing the game of Minecraft. Then we do visual grounding, right? So in this sort of thing, where's the sheep, there's the sheep point to the recipe book. There's the recipe book, point all the cows, all the cows, et cetera. Then we do the trajectory gameplay. So this is where you have sort of a clip with the commands that you use to make that thing happen, right? So how do you attack? You do mouse click or whatever, how you make a recipe. This is how you make a recipe. So what is the point of this whole thing? They're saying that their experiments demonstrate that post training on non-trajectory tasks lead to a significant 40% improvement over the best agent baseline on a diverse set of atomic tasks. So basically atomic, like as in small tasks, like chopping down a tree, et cetera, they're saying when we just show them how players would do that thing, it gets it. Okay. It figures out how to do it. But when we use this approach of first starting with words and then pictures, and then, you know, this is how players would do the thing, that approach leads to dramatically better results. And they explain it as in part, because these models begin to kind of generalize a little bit better. So instead of just mimicking what the player's doing, they start to sort of understand why they're doing certain things, how certain things sort of generalize to other similar things, et cetera. And we have open source, the code models and database to foster further research. Manus AI is another thing that uses some of the sort of the open source infrastructure and the promise to open source a lot of their stuff at some point in the future. And this thing is extremely impressive. Yes, it uses Anthropics models to run the thing that it's doing, but the way that they managed to put everything together is very, very impressive. So they're using a Linux, an open source system that's free. They're using a virtual machine on which that Linux, that Ubuntu distribution is running. They used Anthropics, you know, API to have those models do the work for Manus. So you might look at it and say, oh, it's just a wrapper. It's just this, it's just that. But the point is, it's good. It does what it does really, really well. Here's OpenAI's bold revenue forecasts. So as you can see here, like the green is going to be other products. A lot of that they're believing is going to be AI agents. So they have in the works a $20,000 a month PhD level AI agent, a $10,000 a month sort of developer, software developer agent. And I think it was a $2,000 a month, kind of like a remote worker, knowledge worker, sort of AI agent. So OpenAI is projecting their annual revenues to increase 100 fold by 2029, which would be great for OpenAI. It would be great for the other AI companies. It would be great for NVIDIA because a lot of that sort of revenue would flow back into buying more NVIDIA chips, right? So of course, all the other AI companies will kind of see a similar increase in revenue. Unless, of course, for every single one of these products, there is a open source version that's available. That's again, if it's as good, that'd be terrific. But you know, with open source, it doesn't even have to be like 100% as good, right? That trade-off is going to be different for different use cases for different companies. But I mean, if something's 90% as good, 80% as good, but either very, very inexpensive, you know, free if you're running it locally or very, very inexpensive if you're kind of running it in the cloud. I mean, at some point, it becomes very hard to justify paying these proprietary models, kind of like the retail cost, if there are other open source options available. Now, at this point, you've probably seen a bunch of these videos by Unitree Robotics. One of the more sort of exciting things that Unitree is doing a lot of really cool stuff. It beats the recently, well, not recently, I think it was last year, sometime the fastest sort of sprint that our BiPIL Robotic did. Pretty sure it still holds that record or maybe someone else beat it. I'm not 100% sure, but it's a very impressive robot. It is getting, that's, I mean, that looks pretty agile. Oh no, I was, I paused it to move on, but I'm just looking at what's about to happen. I mean, I want to see what happens. You want to see what happens. I mean, you know, by the look on this guy's face, you know what's about to happen, but okay, okay, let's see what happens. There it is. You knew it, but it didn't fall. That's, that's, that's impressive. Wow. I could not do that for sure. That's pretty good. And obviously that wasn't sort of motion captured, right? Cause the robot had to adjust to, you know, like the physics and the momentum and the force in real time. So it's not like somebody could have like scripted that movement. It had to figure out, you know, use its sensors and actually figure out how to balance. As it's slipping, notice that it, it, it, its feet are slipping on this concrete and it has to like adjust for that. That's pretty impressive. Here's it doing a flip. Now don't get me wrong. So a lot of this, some of this is probably going to be using some sort of a mimicry of human motions, whether that's like video capture that it learns from, or it's some sort of a, you know, motion capture that it learns from. But there's also a lot of demonstrations where it's pretty obvious that it has to be kind of reacting to the real world. Like if it gets pushed, it has to, you know, sense its environments and react autonomously to that. You know, we've seen stuff like this, where basically these robots, they're trained in a simulation, something like Nvidia's Isaac gym, right? So they're trained in there to learn the skills and then it's taking out of the simulation and they're able to do it in the real world. And it tends to be very effective, very robust, but Unitree does have some of the stuff open sourced, right? So it has the Unitree RL gym. So the reinforcement learning gym for their various robots and stuff like that. So they're providing a lot of the stuff for developers that want to build on the platform. They're providing an open source. And there are many, many, many more examples of where, you know, a lot of these Chinese products, they do have an open source components. As Balaji is saying, he's expecting a blitz of open source projects out of China. And again, if he's correct, and it seems like he might be, I think the reasoning behind what he's saying makes a lot of sense, right? If China is able to just kind of like destroy the cost of software and just sell the hardware, who would be able to compete with that really on the global stage, right? Because US currently, I would say has better software, more users, more companies developing, you know, various AI, various frontier models and various software around that, as well as non-AI software. There's definitely a leadership in that space. But in terms of, you know, the production, other than AI chips like NVIDIA, other than that sort of use case, the production of stuff, I mean, China is able to outcompete. If we need to build the most drones, China can build drones. If we need to build the most robots, China can build robots, right? So by kind of crashing the price of software, by creating a massive supply of good open source software and AI, etc., they're able to sort of have that attack vector, if you will. As I replied to Balaji here, it's like the Art of War AI edition, right? So Sun Tzu, the Art of War, an ancient Chinese military in general that sort of wrote up some rules for how to fight wars. So this is sort of like the new chapter, the AI edition of that book. And I think it 100% could be an effective tactic, and it's kind of brilliant. Now, obviously, it's a little bit concerning that this is happening sort of on a more, like if there was just competition between countries just to be competitive, to have a tech lead, competition is good, innovation is good, open source software and everything else being available to more people, more developers, that's, of course, good. I'm very excited about that, especially with robotics, the fact that, you know, if somebody builds a whole robotics sort of ecosystem that developers are able to build all sorts of stuff with, that seems like a very positive thing for the world. If everybody can build robots for their own use cases, right? If, let's say China makes cheap sort of robots that everybody else is able to train for their own use cases anywhere in the world, they can refine them. Like that seems just absolutely incredible. But the whole thing that there's sort of this competition between the US and China that could potentially result in something that's a little bit more militarily involved, obviously, that's very concerning. In other words, that there are some risks here of this kind of like getting out of control, et cetera. And, you know, the next generation warfare with AI and drones and robots, that does seem very, very scary. One thing that this really kind of brings back to mind is Google's, we have no moat sort of leaked memo. So semi-analysis posted this, this is May 2023. So this was a while ago, keep this in mind, this is early 2023. And so this is a leaked memo out of Google. So at the time, of course, OpenAI was making great strides, right? They had their chat GPT moment. And this is somebody, we think it's a researcher at Google writing this memo. So this isn't the views of Google necessarily, or even their employees. This is just one person. But to me, it seems like this person kind of really had a glimpse into the future. And here's what they're writing. They're saying, well, we're looking over our shoulders at OpenAI quite a bit. Who's going to cross the next milestone? What's the next move going to be? But the uncomfortable truth is we aren't positioned to win this arms race and neither is OpenAI. While we've been squabbling, a third faction has been quietly eating our lunch. I'm talking, of course, about open source. Plainly put, they are lapping us and they go on to give examples from that time that kind of prove this point. But since then, we've had a lot more examples kind of proving this point as well. This line jumps out. So the open source community, they're doing things with a hundred dollars in a 13 billion parameter model that we struggle with, with 10 million and 540 billion parameter models. Right. So kind of we've seen that in the deep seek moment, right? There's a parallel there. He's saying while our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open source models are faster, more customizable, more private and pound for pound more capable. This has profound implications. We have no secret sauce. So this is a great read because again, this is somebody that kind of like saw the future and just wrote about it, you know, however many years before where we are now. And based on that, that means that the conclusions that they draw from it might still be good for us to understand right now. And what they're saying, and this is my interpretation of it. I encourage you to read this for yourself. I'll leave a link down below, but they're saying owning the ecosystem, letting open source work for us. They mention that. So a lot of this back then started with the leak out of meta slash Facebook. We don't know if it was an actual leak or it was kind of like a, they leaked it on purpose, but they leaked their models and became this global viral hit. All of a sudden the global ecosystem, like everybody, all the developers all over the globe started to work on this thing, started to improve it, right? As this person is saying, because the leaked model was theirs as in meta's, they have effectively garnered an entire planet's worth of free labor. Since most open source innovation is happening at the top of their architecture, there's nothing stopping them from directly incorporating it into their products, right? So basically the entire planet went to work improving meta's systems, their product, their, their models and meta went, Oh, cool. Thanks. And they were just able to start using it. That's a massive, just free innovation, improvement and labor going to meta. And the value of owning the ecosystem cannot be overstated with Google that successfully with Chrome and Android by owning the platform where this innovation happens, Google cements itself as a thought leader and direction center, earning the ability to shape the narrative on ideas that are larger than itself, right? Again, so this is kind of the point and this is the parallel to what's happening here, right? So if we, as in sort of the U S if we try to hold on to the models and keep them secret and proprietary and China goes a sort of a scorched earth and starts just open sourcing everything, eventually there's going to be open source vision models and frontier models and coding models and, you know, robot training, RL gyms, whatever, whatever you can think of, there's going to be an open source version of it. And all the people that are interested in building and working on this stuff will be working in that ecosystem. They will be contributing to the deep seek ecosystem and the Jarvis VLA, whatever that is, that ecosystem, right? To the unit tree robotics and all of the brainpower and labor and innovation will be happening within that ecosystem. So they will have more users, more data, more developers, more innovation, et cetera. And it's really hard to overtake that sort of snowball once it gets rolling. And so this person saying like, the more we control the models or try to, the more attractive we make open source alternatives. And Google should establish itself as the leader in the open source community, taking the lead and cooperating with everybody, right? And that probably means taking some uncomfortable steps, maybe relinquishing some of the control over the models. We cannot hope to both drive innovation and control it. So if you agree with what he's saying at that place in time for this particular problem, then logically you can translate this to a national level. How would the U.S. compete? Well, we would have to really focus on creating our own open source ecosystem. We would have to push for a lot of more models to be open source, to start building that ecosystem and attracting the global developers, all the talent, all the users, et cetera, to sort of publish more research and publish more open source projects so that all of this is happening in this ecosystem instead of someone else's. But let me know what you think. Do you think China, this is their plan to sort of use the open source software and AI almost to kind of undermine American companies? Is Balaji, is he right about how this is going to play out, right? They want to sell the hardware. So they're going to like tank the software by basically giving it away for free and trying to match it to the quality and abilities of the sort of the U.S. software, software slash AI, right? And what are the alternatives for U.S.? Is it to just ban all the Chinese models and everything else? Well, again, that might backfire because then the rest of the world might choose to adopt their cheap open source effective models, right? Instead of the expensive, right, sort of protected proprietary U.S. models, right? So we might keep the control within the U.S. but lose on the global stage, so to speak. Or do you not even like worry about these things? I mean, a lot of the people watching this, they're not from U.S. or China. You guys are from all over the world. So let me know what perspectives you have that are not one way or the other. I understand that maybe my view on this is a little bit more U.S. centric. I've heard a lot of people out of Europe, the EU, out of South America, a number of people out of several comments that I've read that are from Africa, that are kind of like saying how the U.S. and Chinese sort of competition, how that affects them over there, which is a very unique perspective. So definitely let me know what you think about this. Are you excited about open source in general? But, you know, are you worried about the fact that maybe this is being used on a more kind of aggressive, maybe like military level for nations to kind of undermine each other? Let me know what you think. This is going to be a bigger and bigger conversation. I feel like moving forward. If you made this part, thank you so much for watching. My name is Wes Roth and I'll see you next time.
