YouTuber Paralogical downloaded data from over 5 billion chess games to find the rarest move in chess. Slight spoiler: there are many possible moves that weren’t played in any of the games analyzed. The data and analysis programs used are available on Github:
This is a lil’ code to analyze chess .pgn files, with the goal of finding the “rarest” move in chess.
That is, the rarest move notation (standard algebraic notation) given a large number of input games (e.g. every rated game from lichess) in pgn format.
However, since there are many moves that never happen, this is moreso counting and categorizing moves of various types rather than finding one specific rare move.
If you want a game, I say one game, five dollars, five minutes. So we play a five-minute game for five dollars. If you said you don’t want no clock, I might say I give you one game, $10, because without the clock, it’s longer. You’re wasting time.
Some people say $5 to the winner. That means, we play each other and whoever wins gets the $5. That’s tricky, because I don’t know how strong you are. You might beat me and I lose $5. I’ve wasted time AND I’ve lost money! So I’m one of those people who don’t say $5 to the winner.
I’ll give you a lesson, a half hour for $20. I have some children that come just to see me once a week and I give them a lesson — $20 for a half hour. And there’s a lot of NYU students that come by, we give them a discount for being students. One hour for 40 bucks.
Marcel A. offered this advice that applies to nearly any situation:
The one thing I tell my students is that when you get to a confrontation of any type, you have to remain calm. When you remain calm, you can see the board a lot clearer. You can see the person you’re playing or arguing with a lot more clearly, for who and what they are. So you don’t even have to entertain that shit. You understand?
Nathaniel W. shares what he’s learned about people:
They timid, they’re not willing to take a chance. See this? [He moves a pawn forward one space.] That means sometimes people don’t want to be hurt. They have a fear of losing.
And E.G.G.S. offers perhaps the wisest advice of all:
I’m stuck right now. I can’t give any life advice.
Playing chess helped to alleviate the suffering of Jews and allowed them a few brief moments of relief from the hunger, the cold and the fear, temporarily easing their loneliness and sense of isolation.
Watch as David Howell sets up several historical chess positions and quizzes world champion Magnus Carlsen on them. Spoiler alert: he knows them all. The one he gets in just four moves after opening is just…otherworldly.
I was tested similarly in my world championship days, and yes, most Grandmasters can identify thousands of games and positions. It’s not trivial, but, as Magnus said, it’s our job! But his serious interest in the past (human!) games is also a competitive advantage.
Like many of you, I watched and loved The Queen’s Gambit on Netflix (trailer). Part of the reason it’s so compelling is the care the show’s creators took in accurately portraying the chess players, games, and tournaments. Over the past few weeks, I’ve been reading articles by and interviews with chess experts on what the show got right and wrong. I’ll link to some of them below, but this interview with Garry Kasparov, one of the best chess players ever and an advisor to the show, is particularly worth reading. Here he talks about choosing a real game to be played as the final match in the series (spoilers):
Most of the games, it was not difficult, but the biggest challenge was the last game, because the last game is just, it’s a full game. And the problem is that the last game had to be played by the Queen’s Gambit. Of course I could pick up games from other openings, but it would be very much against the spirit of the book. How did I find a good game that will be played for 40 or so moves adjourned in a complicated situation? And then you have this very important element of Benny and his team calling from New York. It means the position had to be complicated. I found a few games and picked up one: Patrick Wolff against Vassily Ivanchuk, Biel Interzonal, 1993. Wolff sent me a note a couple of days after the show was released: “I recognize the game.” It was quite an obscure game. He said, “Garry, how on earth did you find it?” I said, “I had certain parameters, with the gambit, the number of pieces left, so basically, I ended up with 700 games.” It’s not perfect, because it’s not exactly as complicated as I want it, but it fit the book description: game adjourned, complicated position. And even with all the ruckus, he’s pushing the rook. The rook is trapped in the center. I preserved most of the game description and I think it helped, because that’s a climax, and the climax is something that people always recall.
In this charming short film, The Magic of Chess, some young competitors at the National Elementary Chess Championship explain what they find so intriguing about the game. From an Atlantic piece on the film:
The children interviewed in the film are articulate and wise beyond their years. “When I asked the kids questions like, ‘What has chess taught you?,’ I was surprised, given their limited life experience, that they could formulate a response beyond the obvious mechanics of the game,” Schweitzer Bell told me.
Chess “teaches you how to make a plan,” one child says in the film.
“When you lose, you learn from your mistakes,” says another.
I was curious about the tournament, so I looked up the results and the standings are so full of Asian, Indian, Latino, and other non-European language names that my browser offered to translate the page for me. Another thing that jumped out at me was that most of the top competitors in the K-6 competition were 6th graders, except for 2nd grader Aditya Arutla finishing in 8th place. Wow!
Chess is as physically demanding as many other sports due to stress and because the human brain uses a ton of energy. Many of today’s top chess players train and eat like pro tennis or soccer players.
In 2004, winner Rustam Kasimdzhanov walked away from the six-game world championship having lost 17 pounds. In October 2018, Polar, a U.S.-based company that tracks heart rates, monitored chess players during a tournament and found that 21-year-old Russian grandmaster Mikhail Antipov had burned 560 calories in two hours of sitting and playing chess — or roughly what Roger Federer would burn in an hour of singles tennis.
Robert Sapolsky, who studies stress in primates at Stanford University, says a chess player can burn up to 6,000 calories a day while playing in a tournament, three times what an average person consumes in a day. Based on breathing rates (which triple during competition), blood pressure (which elevates) and muscle contractions before, during and after major tournaments, Sapolsky suggests that grandmasters’ stress responses to chess are on par with what elite athletes experience.
“Grandmasters sustain elevated blood pressure for hours in the range found in competitive marathon runners,” Sapolsky says.
From experimental game developer Pippin Barr, several variations on the game of chess that makes the game more interesting (or at least weirder). In “Clone” mode, every time you move a piece, a copy of that piece is made. In “Chance” mode, selecting a piece causes the piece to change randomly to another type of piece (e.g. from a pawn to a rook) that you can then move. In “Gravity” mode, pieces fall to the bottom of the board unless they’re blocked by other pieces. In “Quantum” mode, a new piece is spawned in each possible new position of a selected piece.
From directors Molly Brass and Stephen Tyler, this is a really lovely & poignant short film about Chess Forum and its owner, Imad Khachan, a Palestinian refugee who came to America to get a PhD in American literature and ended up as the owner/operator of a classic NYC establishment.
Anybody who doesn’t speak any language or different languages, they can sit here and play chess. You can still hold a meaningful conversation without saying a word.
After an ownership agreement between Khachan and a former business mentor fell to pieces, Khachan opened the Chess Forum directly across the street from his former partner’s shop, The Chess Shop. His move triggered what, in New York chess circles, is still known as the Civil War on Thompson Street.
“Sometimes attack is the best defense,” Khachan said of his decision.
His move tore New York’s tight-knit chess community in two. A ceasefire eventually settled in, with each shop courting its own customers and suppliers. His business rival closed in 2012, but the feud taught Khachan a lesson as strong as any he learned on the board.
“Like any chess game it’s the thinking ahead that keeps you one step ahead of the guy who’s shooting after you and not hitting you,” he said. “You have to keep moving.”
Our analysis provides new evidence that even simple turn-based games contain a great deal of interaction richness and subtlety, and that the different levels of communication should be considered by designers as a real and legitimate vehicle for social interaction.
In this video from the New Yorker, chess granmaster Garry Kasparov talks through his four most memorable chess games: two against Anatoly Karpov, one against Viswanathan Anand, and the final game in his rematch against Deep Blue, in which he gets wrong-footed by a move that the computer didn’t know how to make. Even if you’re not a huge fan of chess, it’s instructive to hear Kasparov talk about the importance of what’s happening not on the board — things like body language and confidence.
With just four hours of practice playing against itself and no study of outside material, AlphaZero (an upgraded version of Alpha Go, the AI program that Google built for playing Go) beat the silicon pants off of the world’s strongest chess program yesterday. This is massively and scarily impressive.
AlphaZero won the closed-door, 100-game match with 28 wins, 72 draws, and zero losses.
Oh, and it took AlphaZero only four hours to “learn” chess. Sorry humans, you had a good run.
That’s right — the programmers of AlphaZero, housed within the DeepMind division of Google, had it use a type of “machine learning,” specifically reinforcement learning. Put more plainly, AlphaZero was not “taught” the game in the traditional sense. That means no opening book, no endgame tables, and apparently no complicated algorithms dissecting minute differences between center pawns and side pawns.
This would be akin to a robot being given access to thousands of metal bits and parts, but no knowledge of a combustion engine, then it experiments numerous times with every combination possible until it builds a Ferrari. That’s all in less time that it takes to watch the “Lord of the Rings” trilogy. The program had four hours to play itself many, many times, thereby becoming its own teacher.
Grandmaster Peter Heine Nelson likened the experience of watching AlphaZero play to aliens:
After reading the paper but especially seeing the games I thought, well, I always wondered how it would be if a superior species landed on earth and showed us how they play chess. I feel now I know.
Unpredictable machines. Machines that act more like the weather than Newtonian gravity. That’s going to take some getting used to.
Albert Silver has a good overview of AlphaZero’s history and what Google has accomplished. To many chess experts, it seemed as though AlphaZero was playing more like a human than a machine:
If Karpov had been a chess engine, he might have been called AlphaZero. There is a relentless positional boa constrictor approach that is simply unheard of. Modern chess engines are focused on activity, and have special safeguards to avoid blocked positions as they have no understanding of them and often find themselves in a dead end before they realize it. AlphaZero has no such prejudices or issues, and seems to thrive on snuffing out the opponent’s play. It is singularly impressive, and what is astonishing is how it is able to also find tactics that the engines seem blind to.
So, where does Google take AlphaZero from here? In a post which includes the phrase “Skynet Goes Live”, Tyler Cowen ventures a guess:
I’ve long said that Google’s final fate will be to evolve into a hedge fund.
Why goof around with search & display advertising when directly gaming the world’s financial market could be so much more lucrative?
Garry Kasparov, who is one of the top chess players ever, said that his 1999 match against Veselin Topalov was the greatest game of chess he ever played. In this video, MatoJelic goes through the game, move by move. Even if you only have a passing interest in chess, I’d recommend watching…it gets really interesting after the first 10-12 moves (which are presented without explanation) and listening to someone who is passionate about a topic is often worth it.
Magnus Carlsen and Sergey Karjakin are competing in the FIDE World Chess Championship Match in NYC and are currently tied going into the final match. By all accounts, it’s been a tense competition. But watching chess being played in real time is perhaps only for die-hard fans. Here’s video of Karjakin thinking about a move for 25 minutes:
And here’s Carlsen thinking about Karjakin thinking about the same move for 25 minutes:
Really Bad Chess is an iOS game by Zach Gage that randomizes the distribution of pieces when the board is set up, so that you might start a game with 4 queens, 3 knights, and only 2 pawns in the back row. The result is that you get a completely new strategic game each time, but you still play with the familiar tactical rules of chess. What a great idea…I can’t tell if people who really love chess will love or hate this.
Knightmare Chess is played with cards that change the default rules of chess. The cards might change how a piece moves, move opponent’s pieces, create special squares on the board or otherwise alter the game.
It employs the same board and pieces as standard chess; however, the starting position of the pieces on the players’ home ranks is randomized. The random setup renders the prospect of obtaining an advantage through the memorization of opening lines impracticable, compelling players to rely on their talent and creativity.
Using behind-the-scenes footage shot over the past decade, Magnus is a feature-length documentary about reigning world chess champion Magnus Carlsen.
From a young age Magnus Carlsen had aspirations of becoming a champion chess player. While many players seek out an intensely rigid environment to hone their skills, Magnus’ brilliance shines brightest when surrounded by his loving and supportive family. Through an extensive amount of archival footage and home movies, director Benjamin Ree reveals this young man’s unusual and rapid trajectory to the pinnacle of the chess world. This film allows the audience to not only peek inside this isolated community but also witness the maturation of a modern genius.
Over the years, however, I’ve started to wonder whether Netflix’s big decisions are truly as data driven as they are purported to be. The company does have more audience data than nearly anyone else (with the possible exception of YouTube), so it has a reason to emphasize its comparative advantage. But, when I was reporting a story, a couple of years ago, about Netflix’s embrace of fandom over mass culture, I began to sense that their biggest bets always seemed ultimately driven by faith in a particular cult creator, like David Fincher (“House of Cards”), Jenji Leslie Kohan (“Orange is the New Black”), Ricky Gervais (“Derek”), John Fusco (“Marco Polo”), or Mitchell Hurwitz (“Arrested Development”). And, while Netflix does not release its viewership numbers, some of the company’s programming, like “Marco Polo,” hasn’t seemed to generate the same audience excitement as, say, “House of Cards.” In short, I do think that there is a sophisticated algorithm at work here — but I think his name is Ted Sarandos.
I presented Sarandos with this theory at a Sundance panel called “How I Learned to Stop Worrying and Trust the Algorithm,” moderated by Jason Hirschhorn, formerly of MySpace. Sarandos, very agreeably, wobbled a bit. “It is important to know which data to ignore,” he conceded, before saying, at the end, “In practice, its probably a seventy-thirty mix.” But which is the seventy and which is the thirty? “Seventy is the data, and thirty is judgment,” he told me later. Then he paused, and said, “But the thirty needs to be on top, if that makes sense.”
Some of you will know that Average is Over contains an extensive discussion of “freestyle chess,” where humans can use any and all tools available — most of all computers and computer programs — to play the best chess game possible. The book also notes that “man plus computer” is a stronger player than “computer alone,” at least provided the human knows what he is doing. You will find a similar claim from Brynjolfsson and McAfee.
Computer chess expert Kenneth W. Regan has compiled extensive data on this question, and you will see that a striking percentage of the best or most accurate chess games of all time have been played by man-machine pairs. Ken’s explanations are a bit dense for those who don’t already know chess, computer chess, Freestyle and its lingo, but yes that is what he finds, click on the links in his link for confirmation. In this list for instance the Freestyle teams do very very well.
I wonder what the human/cyborg split is at Buzzfeed or Facebook? Or at food companies like McDonald’s or Kraft? Or at Goldman Sachs?
In terms of comparison, Magnus Carlsen, the world’s current #1 and owner of the highest ranking ever, is 2-1-4 at the same tournament. Which is pretty typical; the best players draw a lot. Over his career, Carlsen has drawn almost 50% of the time and Caruana about 40%.
The modern times of chess have a new king, king Fabiano Caruana. One has to look back to 1968 where in Wijk Aan Zee the legendary Korchnoi started with 8,0/8. The times now are so different and the competition so fierce that already Fabiano’s success can be proclaimed as the most memorable streak in the history of chess.
Along the way, Caruana has beaten Carlsen (#1), Levon Aronian (#2), Maxime Vachier-Lagrave (#9) twice, Hikaru Nakamura (#7), and Veselin Topalov (#6) twice. If you look at the unofficial live chess ratings, you’ll see he has moved into the #2 position in the world, jumping a whopping 34.1 points in rating. He also owns the fourth highest rating in history, behind Carlsen, Kasparov, and Aronian. Caruana plays Carlsen again today, starting from the more advantageous white position. (via @tylercowen)
Update: In his eighth match, Caruana drew against Carlsen but clinched first place overall with two matches remaining.
Update:Seth Stevenson has an article in Slate about what went down at the Sinquefield Cup, saying “one of the most amazing feats in chess history just happened, and no one noticed”.
Sometime earlier this year, Scott Kildall and Brian Sera used archival photos of the hard-to-find set, turned them into 3D models of the chess pieces, and made a pattern for 3D printing your own set:
Something tells me Duchamp would have loved this whole thing.
Update: Welllllll, Duchamp may have loved this, but his estate definitely did not. Duchamp’s estate sent Kildall and Sera a cease and desist letter, forcing them to remove the 3D models from Thingiverse. Which, the irony! So, Kildall and Sera, riffing on Duchamp’s mustachioed Mona Lisa, have created a set of six 3D-printed chess pieces with mustaches modeled on the Duchamp set. Fantastic.
It’s a well-known fact that White has a small advantage at the beginning of the game. To maintain this advantage, White should press their advantage to take over the middle of the board as quickly as possible. The most popular first White moves from 1850-2014 are shown below. Note that all of these are fairly aggressive openings that build toward control of the middle of the board.
In 1850, White openings were fairly homogeneous: Most chess experts played King’s Pawn. Chess players didn’t begin to explore variants of the King’s Pawn in earnest until the 1890s, when Queen’s Pawn (moving a Pawn to d4) started to replace King’s Pawn in some player’s repertoires. The 1920s saw another burst of innovation with the rising popularity of the Zukertort Opening (moving the Knight to f3) and the English Opening (moving a Pawn to c4), which completed the set of staple first-turn openings that are really ever used nowadays.
“Tell me about your game,” Spiegel said. Sebastian flopped into the chair and handed her his notepad, where he’d recorded all the moves for both players in the game.
Sebastian explained that the other guy was simply better. “He had good skills,” he said. “Good strategies.”
And this is the point where many of us would simply say something along the lines of “did you do your best?,” in which case the likely response is “Yes.” Everyone is at least let off the hook. The teacher for ensuring students try their best, the student for having lost to someone better. Spiegel did not take this approach.
The most critical missing piece, Randolph explained as we sat in his office last fall, is character — those essential traits of mind and habit that were drilled into him at boarding school in England and that also have deep roots in American history. “Whether it’s the pioneer in the Conestoga wagon or someone coming here in the 1920s from southern Italy, there was this idea in America that if you worked hard and you showed real grit, that you could be successful,” he said. “Strangely, we’ve now forgotten that. People who have an easy time of things, who get 800s on their SAT’s, I worry that those people get feedback that everything they’re doing is great. And I think as a result, we are actually setting them up for long-term failure. When that person suddenly has to face up to a difficult moment, then I think they’re screwed, to be honest. I don’t think they’ve grown the capacities to be able to handle that.”
The best chess player in history, 23-year-old Norwegian Magnus Carlsen, has released an iOS app where you can play simulated games against Carlsen at various stages of his career, from age 5 up to the present. The Telegraph has the details.
Anyone who wants to find out more about his playing style can do so with Mr Carlsen’s new app, which allows users to play him at the different levels he has achieved since the age of five.
The app is built on hundreds of thousands of different positions from Mr Carlsen’s games, be they classical, rapid or blitz, to determine what moves he would make at those ages.
The aim is to promote chess among as many people as possible to make the sport more popular and accessible.
“The good thing is that you can play me at any age. At age five, anyone has a chance to beat me,” Mr Carlsen said.
So what is it like for Mr Carlsen to play against his younger self?
“He is really tricky,” the champion said. “Even Magnus at 11 years old was a very gifted tactician. A while ago I played as a test Magnus [aged] 14. I outplayed him at some point positionally. And just boom, boom, he tricked me tactically.
“But he makes mistakes as well, so I just have to be patient.”
I played chess with my dad growing up, but actually learning how to play (studying openings, notation, etc.) seemed daunting and at cross purposes with what I liked about the game. So I stopped playing sometime in college and never really picked it up again. But I’ve maintained a non-playing interest in the game and have even been playing a little bit recently again, teaching my son how to play. The other day I ran across mate-in-one puzzles (iOS app), which seem more my speed.
The tl;dr of this training plan is, play a lot, analyze your games, and primarily study tactics. Your knowledge of openings, endgame, middlegame, etc. will come from analyzing your games and going over grandmaster games. Only study one of those specific topics if it is clear you are specifically losing because of that topic.
The Chess World Championship is currently underway in Chennai, India. Through nine games, challenger and world #1 Magnus Carlsen is leading reigning world champ and world #8 Viswanathan Anand by the score of 6-3. It seems as though Carlsen’s nettlesomeness is contributing to his good fortunes.
Second, Carlsen is demonstrating one of his most feared qualities, namely his “nettlesomeness,” to use a term coined for this purpose by Ken Regan. Using computer analysis, you can measure which players do the most to cause their opponents to make mistakes. Carlsen has the highest nettlesomeness score by this metric, because his creative moves pressure the other player and open up a lot of room for mistakes. In contrast, a player such as Kramnik plays a high percentage of very accurate moves, and of course he is very strong, but those moves are in some way calmer and they are less likely to induce mistakes in response.
Or perhaps Carlsen’s just inspired by the lovely chess set they’re using? Either way, he needs just one draw in the remaining three games to win the Championship. Or putting it another way, Anand has to win all three of the remaining games to retain the title.
White moves first, but then Black gets to move twice. Then White gets to move three times in a row, then Black four times in a row, then White five times in a row, and so on, with continuing escalation as the game proceeds.
I’m overwhelmed to be in the same room as these men. I played through their games (well, except the younger ones) as a teenager, developing a love or dislike of their styles, and scratching my head at their depth. The skill gap in chess is remarkable: these Grandmasters would demolish someone who would easily beat someone who would wipe me off the board. Amid my admiration, I feel a vertiginous impulse: I could leap out of my seat, scatter the pieces, and make history as the world’s first chess streaker. The temptation soon fades.
Each player aligns the pieces, although the boards are already laid out in pristine formation. It’s a curious habit I recognise from my own experience. It helps to get your hands on the tools of your trade, to feel they’re yours.
As chess increased in popularity across Europe in the 1800s, the proliferation in the variety of chess sets caused confusion amongst competitors, especially those hailing from different countries. The English typically used Barleycorn sets:
English saw a different design for every chess club: St. George sets with their appearance of stacked disks, Dublin sets with more rounded middles, and Northern Uprights with columns instead, as well as elaborate, easily tipped Barleycorn sets. Germany had delicate Selenus sets, beautiful beyond belief, but fragile, tippable, and problematic for play. To tell which piece is which on some of these sets one must count the stacked crown. France saw elegant Regence style sets with some of the most confusing signatures in history. As in the English sets, queen’s were represented by orbs. The king’s floral crown closely resembles the modern Staunton signature for the queen. Knights were always taller than bishops the old French sets. Bishops were represented as fools, not clergymen, and therefore lacked the signature miter. What was worse, the knights in these sets were sometimes simple turned designs, not the recognizable horse’s head. This lead to common confusion as to which minor piece was which. The confusion of antique French knights and bishops is still a common problem today.
Then in the 1849, Nathaniel Cook designed and John Jaques began to sell a set that eventually came to be called the Staunton chess set:
Howard Staunton was regarded as the top chess player of his era and organized the first international chess tournament in 1851. Staunton endorsed the set and it soon became the standard in chess competitions and, later, the official standard of the World Chess Federation. The most recent iteration of the official Staunton set is Daniel Weil’s design for World Chess:
If you’re interested in learning more, Jimmy Stamp has a nice piece about the design of the original Staunton set and Weil’s update at Smithsonian magazine.
Pentagram’s Daniel Weil has designed a new chess set that is currently being used at the World Chess Championship Candidates Tournament in London. The set is beautifully iconic and simple.
Grantland has a story about a chess cheating scandal with an interesting section on the the history of chess-playing computers.
The Virginia scandal involved the opposite ruse, in which a machine surreptitiously called the shots for a player. The chess engines this scheme centered on are relatively new: Computers only surpassed humans at the chessboard during young Smiley’s lifetime. Scientists had an easier time designing digital brains that could produce atom bombs or navigate lunar landings than they did fashioning a machine that could play chess worth a darn. Plainly, until relatively recently, chess was too complicated for computers. An analysis of chess’s complicatedness in Wired determined that the number of possible positions in an average 40-move game is 10 to the 128th power, a sum “vastly larger than the number of atoms in the known universe.”
In 1966, MIT brainiacs entered MAC HACK VI, a computer program they’d devised, into the Massachusetts Amateur Chess Championship, making it the first computer program ever to enter a tournament. It drew just one match and lost four.
[…]
But by 2007, a chess engine called Rybka was routinely shutting out grandmasters even when spotting the humans a pawn and taking black, thereby letting humans go first, the more statistically desirable position. Computers have gotten noticeably better since then; humans haven’t.
The man-machine war in chess is no longer contested: “Computers are better than us,” says USCF president Ruth Haring.
Stay Connected