Transcription

AI. I've 1v1'd AI twice before and I've won both times quite easily, but today I got reached out to by these students from Stanford University that said they built a GeoGuessr AI for their class that I will not be able to beat. Let's see if it's finally time if an AI can beat a pro GeoGuessr at GeoGuessr or if I'm losing my job. Lion, we're back. It's time to face the AI again. Quickly take me through your AI that some of my audience might have seen, some of them might have not seen. Where are we staying with your AI right now? So my AI since the last match with you, I would say also improved a bit. I switched from learning it completely from scratch to something that's called ResNet, a pre-learned model. Probably not the best AI today. So you're saying it's not the best AI. I guess this is a good transition to Stanford. Okay, so I kind of forgot to just introduce these guys, but just for reference here on what I'm going up against, this is Michael. He's currently getting his master's in computer science at Stanford. This is Lucas, or Sean Mendes, and he's getting his master's studies in computer science at Stanford as well, and he also spent his last year in AI research focusing on large language models and the robustness of computer vision models. And then this is Silas, who is a PhD student in AI at Stanford. And then this is me, a man that likes to go maps. He's alluding that your guy's AI is pretty good. Where does it stand right now? What's its average score? How does it look? Yeah, so just to walk you through the accuracy, we're currently guessing 92% of countries correctly, and we have a median kilometer error of 44 kilometers, which translates to an average score of 4525. All right, so basically what you're saying is I have absolutely no chance of winning today, unless I play the best game of my life. I don't see how this is... Okay, so what makes it so good, though? How is it so accurate? We're basing it on the CLIP model, which is a large foundation model that was trained on billions of images, and then we're also fine-tuning that and adding on top of that a bunch of other tricks like meta-learning and refinement. Another thing I think that worked really well for us, we kind of split up the world into a lot of small cells, and we're really smart about the way we pick these cells. They both respect political and natural boundaries, and then we also refine our guesses within these cells. And yeah, it worked really well, actually. So something that's super interesting about AI is that compared to the ResNet model that Leon mentioned, we're not only using images, but also text. So instead of only training the model on images of the world, we can also find new information about a given part of the world, like average temperature, typical climate, and then add it as text to the model. And it turns out that if the model learns both images and text, it becomes much more accurate at geolocalization. It turns out to be better at geoguessing. Oh, dude, I'm so screwed. I'm so screwed. Okay, I think we just get into this. The tables have turned to where it's, can the AI beat me versus can I win the game? How long have you guys been building this product? Was this something that you guys built after my video, or has this been like, you guys have been working on this for a while? I think we worked on it for like two months or something, like last fall. After your video, actually, yes. Okay. Was this like a school project or was it just like a passion project on the side? Yeah, we had a mini class. Hopefully, did you guys get a grade on it? Yeah, we did. It was a great class. Let's get into the game and let's see what happens. All right, it's game one against the AI. Guys, I honestly don't think I'll be able to win the game. If it's guessing the way they said it's guessing, there's like a very, very small chance that I'm going to be able to pull something off here. But I'm going to try my hardest. Round one, it looks like we're going to be Malaysia. Do we go mainland or do we go Borneo Island is the question. The AI is probably going to know what part of Malaysia this is, assuming. I'm going to go somewhere over here, like just center Borneo. It could be mainland. I don't see stickers or anything on the pole, but we'll see. And of course the freaking AI, dude. I might be happy if I can pick up points. That's like where I stand right now. It's Poland, not very hilly. I'm just going to go central. Let's go. This is massive. This is massive. Okay, 94 points. Poland is, is you can't reach against Poland. So that makes sense. Round three looks like Mexico. It's not Mexico. It's Brazil, but we're in Brazil. Don't want to guess. It's super dry. I do. I go north guys. I'm actually tryharding. So bear with me. Is it going to be like somewhere up here? I'm going somewhere north. Dude. I'm not, I listen guys. Life goes on. Round four. Okay. Does it know U S is a question. So far pigeon is the, has been, it's worse than the best AI South Hampton Barnard school. I mean, it's definitely gonna be like the South or like somewhere East. No real clues for me here. I have to get a guess off. It's guessing first to like one alpha moon. I'm going to pre Northeast. Like these rock walls feel like Northeast, as I was going to say, around five guys, it's not over yet. Okay. I could still win here. All right. So this is just Italy Po Valley. Maybe. I will say I did win a round. That's a win in my book. Has the AI seen any of these photos before? I know it hasn't seen any of these photos. So we, we train it on around more or less a million images from 250 K locations, but the probability that overlaps is super, super low. So basically hasn't seen any of the images. So it's like not training on like GeoGeo specific maps. It's training on actual like Google maps, street view, like there's 5 billion 3D locations in the world, but we train only on 200,000. So it's basically virtually impossible that has ever seen any of these photos. That's humiliating even more. All right. Let's do a couple more games and let's see what happens. I'm going to try and pick up a game. I'm not very confident, but we'll see. All right. So we're gonna go in the game too. I think my strategy to win a game, I think I shouldn't try and make it to like late rounds. If I can get like a good multi-off, then I'm sitting pretty, you know, then again, it doesn't really make any mistakes, but I feel like if it gets like a rural Europe or something, or like a Dominican Republic or like something that's like, or if I keep making guesses like that, give me some Taiwan action, bro. Okay. Round two. Okay. Something like this is good for me too, but it presumes to be a Canada. Is that French? Is that bait ever? I mean, I have to go New Brunswick here. I mean, I'm reading. I low-key thought this is PEI with the red soil, but I'm going to assume it's over here. Okay. I don't know if the AI is reading, but pigeon is a great guest there. Um, I'm gonna, I'm gonna have to ask him about that. Okay. Round three. We're gonna be back in Canada, but this is a weird Canada. I have to go somewhere in the same region. I feel like that'd be crazy to get two back-to-back, but it makes sense. Dude, that makes even more sense, bro. You get the single yellow there. Okay. But somehow the AI can understand the difference between rural Canada with a single yellow and not. Okay. We're NA guessing, but I feel like I can get Florida here. Like this should be like a South Carolina, Florida. I mean, Florida was my first instinct, but I feel like it's going to be somewhere South Carolina, Georgia region over here at the sand. Dude, dude, I feel like, is this ever the 27 or 23? It's definitely up here somewhere North South. I'm going up here. Okay. It was a 23. No dude. If I could just find it. All right. Around six from a five X do my NA guessing guys. Do not judge my NA guessing. Okay. I'm sorry. I got to lock in here. Thailand is not easy to region guess. You have palms. It's going to be somewhat coastal Shirley, maybe South of Bangkok. I also think it could be like East, like somewhere over here, but I also think it could be like down here somewhere. I'm going to, I'm going to make a risky guess and get stuff here, but it could be over there. It was over there, dude. I'm not gonna lie, dude. I feel like it played well. My NA guessing, not that good. I have a couple of questions guys. This is like the post-game interview. It's like, I feel like I'm an NBA and you guys dropped like 30 points in the fourth. A single yellow road line like that is extremely, extremely common in like a Canada, but it's still somehow seems to guess against the odds. And it gets to the right country when it's the U S when it's like 1% chance of the time, is it just guessing off trees? Is it putting the, even the road lines or things like that in the calculation? Like what's I'm trying to figure out this thought process here. If we even know this thought process, it's basically very similar to how you approaches, right? It's a lot of different methods. It's just that this model picks up a lot of methods that we wouldn't pick up on. It takes the image and finds a representation of the image. And this representation contains a lot of information. We did like one visualization of it. And for example, in Taiwan, it really looked at these phone poles. So it does look at things that also he was looking at, which is interesting. I have another question that in the first round of that game, we had a sign that said New Brunswick on it. I was like, okay, well, that's easy. I know it's gonna be New Brunswick. Does any of your guys's AI able to read that and recognize where New Brunswick is? Or do you guys even know if it like, it takes actual like information of street signs or anything like that into account? It might actually not be that good at reading text, but it definitely takes into account the structure and like the way the street lines look like what colors do they have? What shape do they have? Pretty similar for mine. So there's no Googling going on here. Not yet. Yeah. Let's keep it that way. Okay. All right. Let's do a couple more. I think I can win the game. I think if I can get some good guesses and stay late round and get like a Poland or something where it's like, I don't think it's possible to region guess Poland or like a small country like Lithuania, where if you guys can region guess Lithuania, then it's like, just take the win. But I think I can get a win. I'm going to try. My spirits are not down. I'm honestly just more so just like impressed by the AI to where I can't even get mad at losing. Cause it's just like what an advancement of technology. You just have to appreciate it. Like I said, they built this in like two months, which is like insanely incredible. And also like, I haven't made any bad guesses, you know, it's not like I'm sitting here like blundering. I've made a couple silly mistakes here and there, but um, I probably should look at this round and region guess Italy here. Sardinia. And it fricking gets Sardinia, bro. Dude. Like what do you want from me, dude? Okay. Turkey. Should I just start instant sending and like throw it off? I'm going to try it. Put some pressure on it. Okay. Put some pressure on it. See, it hasn't even guessed yet. It's it doesn't know where to guess. Okay. Not the best guess from your boy, but that's okay. Is this the arch panhandle? This is the arch panhandle, bro. One of these cities up here. Oh, okay. Every single AI is getting the arch panhandle around four, 1.5 X Kenya. It's getting Nairobi. Let's go. Let's go pitch with 86, five, two X. Okay. Is this Andorra? It could be Spain, but I mean, I just have to get into right here. One of those were like, I could pick a point in it. Maybe not. Not sure why I said maybe not, but you know, we're picking up points, 10 points. It's nothing to scoff over around six, 2.5 X. We have a Canada sign there. It feels more like Manny or Sask. North beach estates. This is not good. Do I go like Ontario here? Is it ever like near Thunder Bay? I don't know. I'm guessing Manitoba. I'm not very good at Canada. It was fricking Alberta, dude. It fricking... I don't think the AI has made a bad guess yet. Would you say the average kilometers it was away from a guess was? So the average is 44 kilometers, but a lot of guesses were within five kilometers and extremely rare that you get one like completely off, like on a different continent. But if you get it completely off, it would mostly be an example where a human would make the same mistake. Okay. Let's, let's go again. We'll see two more. Let's do two more. Yeah, that's true. I'm waiting for, for at least you guys below 5,000 points to be honest, but we'll see. I don't even know what game this is anymore. Let's just, let's just start it. I'm already humiliated guys. I also stayed up to 1am to do this. Okay. Cause like it's Stanford and I think they're actually, they had to like run this on Stanford servers or something. So they're like at school right now, which is kind of funny. I've had this round before I've had this round. I don't remember where it was. I thought it was somewhere in Montreal though. There's somewhere near over here. Okay. A little bit of a Cheeto, but listen, okay. I think we can take round two Ghana lock in. Do I just go Accra here? I think I have to just go Accra cause there's palms and we're near the coastal. We're not near the coastal, but capital hedging does not work. You they're still closer, bro. The map is warped around three. Why are we keeping so much in a Mexico? I guess somewhere over here. Nice. That's massive. Oh, and they went Brazil. Well, dude, AI are so dumb. I haven't, I haven't blundered a mistake like that. Oh, holy AI. You're so dumb. Get, get the game, your actual dog water. I'm about to, can you guys let's go. Why do you imagine going Brazil and Mexico? That's like try hemisphere guessing. Oh my God, dude. Embarrassing. Okay. All right. Neck and neck. Do you, if they didn't have fricking traverse, bro, this is going to be orange. Somewhere West-ish. I think it's probably over here. Maybe never Cordova. This is kind of risky guess, but guys I'm locked in. Let's go. We had one up there. I can do this. I can do this. I can do this. I got to lock in. Dude. I could have picked up so many points there. If I went more West, like I thought that was a little scared. It's going to be Colo. It should pick this up. We're gonna be West of the Andes. I hope this is East of the Andes when we said it's gen four though. So it should be freaking go baby. Let's go. Okay. Lock in, lock in. I see one good guess here around seven. Dude, I can not blunder this. I looked over my discourse screen and they look like they're quivering in their boots. I don't know what this is. Is this Spain or Italy, dude? Is this Spain or Italy? It has to be Italy, right? Or is it Spain? I'm going to go Spain here. If this is Italy, I'm so sad. Dude, what an embarrassing way to come out. Yeah. We actually, we actually saw that the first image that we were taking was still of the countdown. All the rounds until the last one, we were always taking, we're taking four images, just looking around. The first image was always of the countdown of the next round. My points were a little asterisked is what you're saying. Thanks for bringing me down a little bit. I was looking a little too happy on my end. Oh man. Is it fixed? Because I was also guessing on only the countdown and I forgot to refresh on my end as well. So now that we're both refreshed, we can, we can, we can see. Dude, on that last round, I was like, if this is Italy, I'm going to be so sad. I was like, it's Spain or Italy, it's Spain or Italy. And then you 5k'd Italy. Like it's, it's so crazy. All right. I'm going to mute. Good luck. All right. So they pretty much just said, f**k you. You were only excited because our AI was bugged. So that was kind of f**ked up, but listen, I'll take the couple of points I got. Okay. What are these rock walls? Okay. So just give me the UK. If they can UK region guess, props to that. Okay. Well, you know, in hindsight, Scotland does make sense there, but life goes on. I got to lock in, dude. We haven't gotten like any like South Africa or anything like that. Like where I feel like I can make some decent guesses here. We're in Mexico. I want to guess here. I mean like pretty North. It's so dry, bro. Like I have to go somewhere like up here. It was Mexicali, dude. This is just, this is going to be North Turkey, surely. Like what else would this be? Problem is how far East I want to go. Like somewhere over here, where is there like tea nearby? Probably not. I'm going to hedge between the two. It's definitely going to be somewhere on the coast here. Like it has to be. Oh dude. What the f**k? I'm actually like, there's part of me that's like embracing it. And there's part of me that's just like, I hate this fricking AI, dude. Okay. Just Brazil. Just Brazil. I can pick up big points in Brazil. It's a big country. Is there area codes or anything that can help me? I don't think so. I'm going to be guessing somewhat East-ish a bit. Like right here. Yeah. Yeah. Yeah. Okay. I just need to get a rule around that it doesn't understand where it is. Like Switzerland. Does it understand low cam is the question here. It doesn't know Swiss privacy laws. If a lake to the South with mountains. I mean, that's a fine guess. It could be this one. Pretty big city. I'm going to go over here. Wait, what? I thought it was like to the South and it just 5k'd bro. Dude, it just 5k'd. I don't even want to win anymore, but I can't win, but all right. GGs. Like that Switzerland round was a 5k. We were like, theoretically, like it got the city, right? Like on a round like that where you can see like there's water and you can see, does it know like cardinal directions? Like, okay, we know there's water North. So we should be looking for water North. Does it like line that up almost perfectly? Like the distance and everything that we're using it during training, but not, uh, not doing the predictions or while we're playing the game and not using the heading. So, okay. But it is trained on like knowing what's North versus like West. Yeah, me dude. Okay. Let's do one more game. I think we see a trend here, but let me just see if I can get at least one good game. Again. I think I might, I think I forgot refresh. So I was still guessing off the countdown for the past couple of games. Um, we'll do one more. We'll do one more for the last one. Can traverse come to my team? Yeah, sure. Yeah. Come on over. Let's come help me out, bro. You got to take down these Sanford lads. All right. Let's run it. All right. In honor of KD, if you can't beat them, join them, right? Germany. Okay. This is not a Berlin sign. Do I know what that sign is though? Do I know what that sign is? Or you can also find this unique Sarah fought on signs here to the thin white border. I think it's best to guess somewhere in this, like, uh, all these different cities here. My teammate went in Stockholm. Okay. That's okay. It's Munich. Dude. There's not even coverage down there. Is there, and there might be, I actually should know that's Munich street signs. Yeah. I did a whole video on Munich street signs. I'm so washed. All right. Sweden, South Sweden. Actually it could be Finland, but I like Sweden. Okay. My teammates hedging with me. My teammates hedging with me. Let's go. I like this. I like this hedge. It was Finland. Okay. Shout to traverse saving it, bro. It shot the AI, bro. Bring it home. I around three years. Just Finland. This time. I know it's Finland guys. I'm going to go somewhere. Like I'm going to go more North than this guy to be honest. Could not be good. Okay. It was more North. The AI dude did get around 4.5X. This one is going to be like North. My AI teammate also thinks so. Nice. Okay. This is good. This is good. 43 kilometers, 340 points. Let's go. We win these bro. Lock in, lock in AI. We have French road signs here. Dude. Part of me wants to go course here. Like I'm not even gonna lie. It's a D177 on course. If my teammate guesses mainland, my teammates going Italy, then I have to safe hedge it. Cause I don't think I should go court. I'm going course. Okay. But what if it was guys, but what if it was, but what if. Eswa. I'm going to guess here. I don't know if my AI guesses with me. Okay. He does. Capital dude. Again, it's closer. Around seven. What is this? It's Denmark. It is Denmark. Where'd my teammate go? Netherlands. It's not Netherlands, but that's a fine guess. I think I'm just going to hedge. Belgium. Are those Belgian plates? Okay. He's keeping us in it. Around eight. Greece. Go more North. That could be an Island to be honest. I went Sardinia. Dude. It literally just got the neighborhood, right? Dude. It almost gets the road, bro. Guys. It hasn't even seen these locations yet. It's like just North Switzerland. Obviously I'd like my teammates guess here. My AI teammates guess. I'm going to go just East. Just a hedge. That was more West, dude. Okay. We're still in it. We're still in it. We just need one good guess. We need them to blunder. This should just be Pretoria. Like Joburg. Somewhere over here. Thank you. Traversed. Thank you. AI. You saved me. All right. Brazil. Wasn't it up here or was this the Brazilia round? How do we just get repeat? If this is a Brazilia round. I forgot where it was. I forgot where it was and I blundered it, dude. If I could remember if I can repeat. Oh my God. Jesus guys. Also great teamwork. You know, we had some back and forth. Not enough. Not enough. But you know, we played well together. Okay. We hedged. We made good guesses for each other at times. All right. I have one more idea I would like to run by you guys. I spent about like a month of my life playing Cambodia only. I would love to challenge the AI to region guess Cambodia. Let's do it. Okay. On the Cambodia only map. What's interesting about AI is it doesn't know we're going to be playing a Cambodia only map. That's the human advantage already. Cause like, I know it's only going to be Cambodia. It could think it's like Thailand or something on an off chance. I'm already kind of going in with an advantage. And Cambodia is like very, very hard to region guess. So like, I think I could pinpoint something. Traversed, you can stay with me though. Cause I'm not that confident. Oh, and then we can do Laos. Cause I feel like Laos, like I will not miss the city in Laos ever. So I feel like I could, after this, we should do Laos. All right. This isn't this is the map. I spent a good portion of December learning Cambodia. I don't really remember much. I'm kind of not that good at it all. Cambodia is like very hard. I could put a lot more time and probably do a lot better, but let's just see if I can win something here. I mean, round one should be Phnom Penh, like South Phnom Penh. Like I said, guys, this is a very hard, my teammates going to Thailand. Okay. This is going to be a safe guess. I think I'm going to go somewhat South of Phnom Penh. Could be North-ish. Okay. It did get Cambodia at least. Okay. Round two. This feels more East, but it could be West for my life. I might go somewhere like over here. Actually, it could be North too. Dude, I hate Cambodia, bro. I'm going to go somewhere like... Bro. I'm not even making good guesses, bro. I just said I was like, play this. Okay. I think I know this road. This is the West road, right? Somewhere over here. I don't know where this is. Guys, I don't even know Cambodia, dude. I don't even like it like that. It's not even fun. Yeah. I don't even like Cambodia, guys. It's okay. I don't even like Cambodia. Okay. Let's do Laos. Let's do Laos. Let's do Laos. Let's do Laos. I'm going to lock in. Good luck. All right. If I f*** up Laos, bro, if I can't win Laos, I'm just taking the L, dude. Because it doesn't even know we're only guessing Laos. Like this is like the most cherry-picked way for me to ever win. And if I don't win, then that's just a skill issue. Like there's no way it even goes Vietnam here because there's no way it sees the yellow plate. I just don't believe it. Okay. It did. It did. It did. That's okay. 27. There's no way I can, I, this is Luang Prabang. No, it's not. This is, this is Vietnam. This is Luang Prabang. Guys, I literally sat there in Luang Prabang and walked every single street, sat there and walked. So I would know where this is. If I ever got this on Street View. That's how dedicated I am to Laos. Okay, guys. It's somewhere over here. Why did I guess near the airport, bro? My God, I'm throwing, bro. I didn't even know it was that far East. This is Pakse. Unless it's Savannakhet. I might want to go to Savannakhet here. I'm going to Savannakhet. If it's Pakse, I'm sad. Let's go. It went South. Let's go. It went Thailand. This is good. This is good. Okay. I mean, this should just be Luang Prabang. I'm trying to make sure this isn't Savannakhet. It looks weird. Mountains South. Dude, it could be like a weird Savannakhet, but it shouldn't be. It should be up here. I keep guessing not where there's coverage, like by the airport, but that's fine. This is the bridge in Pakse. I know this bridge. This is bridge. Okay. Come on. Just go Thailand, you loser. Just go Thailand. I know you want to go Thailand. You're not beating me, bro. I'm not going Savannakhet on Pakse. I'm not. This is this road in Luang Prabang. I literally walked this road. Was it this one? It was this one. It's where the night market is. Oh my God, dude. Bro, they're laughing on the other screen, bro. Why are they laughing, bro? That's what they laugh about. This is going to get South Phanthian though. Losers. Oh, it wasn't even South Phanthian. Dude. Oh my goodness. 1200 points. Okay. This is Pakse, right? Yeah. This is surely Pakse. If this is Savannakhet and they go Savannakhet, it doesn't seem to know Savannakhet, dude. It doesn't even know that it's playing a Laos map. It's making these guesses as if it has the whole world it could be. This isn't Vang Vieng. I don't know what this is. It was Luang Prabang. Oh my God. I could have won that, dude. I was like, those mountains didn't look like Vang Vieng at all. I got to lock in. Is this Pakse or Savannakhet? I don't know what the rural areas look like. I wish my teammate would have taken one, dude. Do I go Pakse or Savannakhet here? Oh Lord, please help me. Okay. Dude, just give me like a Vang Vieng or something easy. I mean, Vang Vieng is good. Is this near the train station? North? I think so. Like, I'm literally cheating to win pretty much, but he's playing a map that you really can't up. Let's go. Do I feel good about that? Not really. That was just like such like a, like an underhand pitch, you know, that that's what it took. Okay. Well, I'll be sincere. I don't feel good about that, but I'll still take it. Okay. Traverse, we did it. We got to win. That's a dub. Would you guys say like, you have the best like GeoGuessr AI? Like humbly, I think we're pretty sure that we have the best in the world. Okay. I like it. I like it. After this video's out, you're gonna have people like, I challenge you. What's next? Like, are you guys still working on this? Does the AI get better? I think we're kind of at a point where we don't see any immediate improvements anymore, but we're possibly trying to publish a little paper. All right. I'm a big fan of AI and just like this in general, like I was probably the most fun I've had playing GeoGuessr in like a while. What I like most about the AI is its potential to help GeoGuessr players learn. If there's ever a world where like players can like look at what the AI used to guess, to like reverse engineer for us to make our own guesses, like that is like the evolution of GeoGuessr, I really feel like. Like, I didn't know like this was a pattern for it to be this region or something like that. Is that possible? Or like, what's the possibility there? Yeah, it's definitely possible, but it takes a lot of effort. It's very manual work, but for a given photo, we could reconstruct what the AI model was focusing on. And usually it's very easy then from there to guess what the meta is. We can actually send you some of these pictures to display in the video if you want to. Pausing the video real quick because I literally just looked at this visualization. I realized that the freaking AI was using the smudges on the camera to help make an educated guess here. At the very top here, I'll circle it. It's literally using the smudge on the camera, which is like very, very common in that region of Canada to make a guess. So if it's picking up things like that, like shout out to the AI, shout out to my fellow meta enjoyers out there. If it's picking up on things like that, this thing is, it's going to be unbeatable. All right, I took the L, but you know, it was a fun L to take. I know the last comments were like, you know, eventually one day AI will get there. It's just a matter of time, you know, less than one year later, here we are. So congrats. But thank you guys for allowing me to play against your AI. It's a privilege. Maybe, you know, we get a couple of more GXR Pros next time. And it's like, instead of one GXR Pro versus two AIs, it's five GXR Pros versus one AI, something like that. But we'll see. We'll see if you guys can get 10,000 likes and we'll get together. That'd be pretty fun. I can see the comms are in like pretty fucking funny, dude. 2,000 likes. Okay. 2,000 likes. But yeah. Thank you guys so much. Thanks. All right, guys, that was fun. I'm currently working on a new retirement plan. My GXR career is obviously over now. So if you guys have any like other ideas for like what I could do now that I have no job, but if you guys enjoyed this, subscribe or don't. I'll see you guys next time. Goodbye.