lol someone on Reddit posted about how “engine correlation” on ChessBase’s Let’s Check feature works and it’s dumber than I could possibly have imagined. A move is deemed to have engine correlation if it matches the suggestion of any engine, and those engine suggestions are crowdsourced on ChessBase from all the engines users have ever run on the position. And critically, it ALWAYS uses suggestions from ALL the engines on file, you don’t get to choose. The Redditor spells out the obvious implications here:
This is a key point that few people seem to grasp. For the Let’s Check feature in ChessBase, the pool of engines used is literally just based on crowdsourced compute from all engine analyses ever run on a particular game. What this means is, the more attention a game gets, the higher its engine correlation is going to be by virtue of more available analyses/engines to match. Furthermore, a game’s engine correlation score can increase over time as more and more crowdsourced analysis is added to that game, but can never decrease.
Food for thought: which player in recent times has come under intense scrutiny, and consequently, has had their games intensely analyzed like no other by a wide gamut of engines?
In other words, amateur sleuths have run a million engines on Hans games and now the fact that a million engines have been run on those games constitutes evidence that Hans cheated. Here’s a list of engines run on a single Hans game:
Stockfish 11
Stockfish 10
Fritz 11
Stockfish 13
Komodo 14
Stockfish 12
Fritz 16
Stockfish 15
Fritz 16 w32
Stockfish 14.1
Stockfish 7
Komodo 10.2
Fritz 11 SE
Deep Fritz 14
Fat Fritz 2
Deep Fritz 13
EngineUnknown=0? <<<< wtf is this… all of these correlations > were end game with no other engine showing best move.
Komodo 12 64 bit
Houdini 5.01
Fritz 17
Stockfish 170121
Houdini 6.03
Deep Hiarcs 15.0
Deep Fritz 14
“EngineUnknown=0” lol, OK. Stockfish SEVEN? WTF is a “Deep Hiarcs”?
In total over the games analysed by Yosha, the evaluations of more than 150 different engines were used.
In conclusion, these “proof” videos are all garbage and you should bump your skepticism of them up by several orders of magnitude.
I’m not bad at this myself, I’ve caught a lot of cheats on chess.com. Some were obvious, but some were more subtle.
It’s MUCH harder to do this though when the cheat is a grandmaster themselves, and Hans is clearly playing at GM level unassisted. Another GM will know when moves are too suspicious to play.
one thing with this vid from hikaru’s vid talking about that video, Caruana doesn’t know his 2014 run was only in the 80’s when he was all 100 oh I’ve done that
lol the plot has thickened on this “engine correlation” thing, grab your popcorn…
So the spreadsheet Yosha uses in her video she credits to “gambit-man”, an individual who requests to stay anonymous but who, like Varys, desires merely to serve the realm. He provided the Let’s Check analysis data showing that Hans had an unusually high engine correlation. But remember how any ChessBase user can use any engine and it will then be included as a potential correlation? Can you see where this is going yet?
It seems to me that gambit-man could easily have submitted doctored data specifically to Chessbase to incriminate Hans. Just upload however many engine analyses are needed, until Hans shows as 100%.
For example, look at Black’s move 20…a5 in Ostrovskiy v. Riemann 2020 ChessBase GameIt shows that the only engine who thought 20…a5 is the best move was “Fritz 16 w32/gambit-man”. Not Fritz 17 or Stockfish or anything else!
Another example: Black’s moves 18…Bb7 and 25…a5 in Duque v. Niemann 2021 ChessBase Game. Again, “Fritz 16 w32/gambit-man” is the only engine that says Hans played the best move for those two moves. Considering the game is theory up to move 13 and only 28 moves total, 28-13=15, and 13/15=86.6%, gambit-man’s data makes this game jump from a normal-looking 86.6% game to “evidence of cheating.” Hmm.
Yeah I am following this and it’s dumb. However, it’s not clear to me that Let’s Check is calculating engine correlation against every user eval that’s been done for each position. If that was the case, then I’d expect Magnus would already have tons of 100% games. I was looking at it this morning and seems to report three engine suggestions per move, so I’m thinking maybe it has some selection criteria (something like search_depth * engine_strength) and then calculates engine correlation from just those?
That makes sense. On one of the Reddit posts someone said this:
Chessbase tries to pick the newest engine with the greatest depth. How it prioritizes this is a mystery to me. But that is why you see the discrepancy. Deep Fritz x14 may have gone to 17 levels on one computer, but 35 on another. In the Chessbase software you can see the top three engines and their depth.
I would speculate that the way Chessbase chooses the “newest engine” is by compile time of the engine assembly, assuming that is accessible via the UCI interface. That’s probably how that gambit-man dude managed to get his custom compile of Fritz 16 in the list of engines used.
Edit: I looked at the UCI protocol and this information is inaccessible, so I guess Chessbase just uses whichever engine(s) most recently analyzed the position.
I think I read something about getting replaced by someone running to a higher search depth on same engine, so these custom compiles we’re seeing might be to avoid that. We need to see what a stock Fritz 16 spits out on these boards. Dunno if I have enough popcorn to stay awake for that.
Apparently this gambit-man dude posts on the chesscom cheating forum and is renowned for constantly seeing ghosts. I doubt he like custom-wrote an engine to get the correct answer but I wouldn’t be surprised if he was like “hmm, this game is close to 100%, what if I used THIS engine on this move?” and didn’t see any problem with doing this.
