Puzzles that are much harder than their rating

I just played a puzzlestorm and encountered this "1000-Rating" puzzle in like the 5th puzzle of the storm: https://lichess.org/training/D9enb

Suprisingly i could not solve puzzle in like 2-3 mins allthough my puzzle rating is about 2400 and i "lost" the puzzle storm at the 5th puzzle, which should not happen, since the early puzzles are usually very easy to solve

When i looked at the solution Qd5+ Be5 Qxe5+, i was suprised that the engine even played Be5 as a reply to Qd5+ which just hangs a piece for nothing. Relplies like Ke7 (mate in 10) or Kg5 (mate in 4) would be more natural since the game goes on and of course the white player should calulated the mate in 10 or mate in 4 if it even works. But thats too hard for a 1000-rating puzzle. Instead the puzzle ends after Qd5+ Be5 Qxe5+!

So how can this puzzle have only 1000-rating despite having two replies to calculate that are very difficult? Is it that most players give the only possible check and "win the puzzle with luck"? Is it that case that the puzzle gets harder for "stronger" players, since they just dont make a move and hope for the best and do actually try to calculte?

This is all speculation, but I think it's reasonable. A lot of low-rated puzzlers don't calculate to the end of the puzzle, they just use intuition derived from solving a lot of other puzzles. This means that they will solve puzzles that have an obvious forcing move. So some puzzles that take a lot of calculation are artificially low-rated.

In my own case, I didn't read your solution but attempted the puzzle on my own. I couldn't find the solution, but eventually "took a guess" and got it right.

I also have this strange feeling about some puzzles.

Sometimes in puzzle series, you have these strangly rated puzzles in the 1000/1400 range with unsound moves or needing some calculation (like choosing to push a pawn to promote or to defend agaisnt the opposite past pawn, or some attraction or deviation moves ranked 1000), and then, you have a lot of very easy mate in one and very recognizable patterns puzzles ranked 1500 to 1900.

I noticed puzzles lost some ratings points when solved. So it can explained : It has been solved so many times it lost a lot of rating points. But something should be done to rectify this. Everything's patterned should be ranked lower than some non patterned needing calculation.

this 2 moves puzzle being rated 1000 seems fine

@Mr2000 said ^

So how can this puzzle have only 1000-rating despite having two replies to calculate that are very difficult? Is it that most players give the only possible check and "win the puzzle with luck"?

Yes, they are lucky guess puzzles. If there's a check and it happens to be the correct move, then they get it right. ...especially when stockfish countermoves are ridiculous from human point of view.

There's also that puzzle ratings are from non-timed settings, but in puzzle storm you would need to find the solution in couple of seconds...

i dont understand how they work

@zenmodeon2022 said ^

this 2 moves puzzle being rated 1000 seems fine

The point is, that it shouldn't be a 2-move puzzle, since no human would just hang a piece in the first reply. The human would rather try to keep the game going and try to run with the king. It would be interessting if there was a way to either remove "underrated-luck-answer" puzzle from puzzle storms or make the engine not hang a piece as first move.

@Furo951 said ^

I also have this strange feeling about some puzzles.

Sometimes in puzzle series, you have these strangly rated puzzles in the 1000/1400 range with unsound moves or needing some calculation (like choosing to push a pawn to promote or to defend agaisnt the opposite past pawn, or some attraction or deviation moves ranked 1000), and then, you have a lot of very easy mate in one and very recognizable...

I think your explanation makes sense. Those puzzles lose rating points from "lucky guesses" like "i give a check and see what happens". Unfortunately there is nothing to learn from those puzzles, since when the puzzle is won, most players dont understand why the puzzle is won. I wish there was away to removes those puzzles from lichess or at least from puzzle storms, since they distort the results.

What i also noticed, that in puzzle racer, there are many players with low blitz/rapid rating that constantly score very high, which they "shouldnt" since they "arent that good". The analogous explanation is, that they "just guess" or "play intuitively" and dont really calculate.

the puzzle rating will change with time depending on more or less players getting it right. if a problem went all the way down to 1000 I guess it is correctly suited for that rating range. With time it will fluctuate up if it deserves (and probably does). it's all part of the fun failing unexpectedly at some random 1000 problem

@Mr2000 said ^

What i also noticed, that in puzzle racer, there are many players with low blitz/rapid rating that constantly score very high, which they "shouldnt" since they "arent that good". The analogous explanation is, that they "just guess" or "play intuitively" and dont really calculate.

It seems curisous to me too. I'm not sure that's the explanation. To get good scores in puzzle racer, you must make long streak withthouth failing, if not solving everything in a raw (or there is another way i didn't know). "feeling" and "intuition" are often memory reminiscence and pattern recognition, and are part of tactical tricks to solve puzzles and find tactics in games.

So perhaps :
-Their ratings are not actual. They don't play, play often, play serious, or they play in bad conditions, but they work seriously on tactics.
-They are very fast but still fail when it become more complexe.
-They are tactical "beast" but have some lacks in positionnal play and opening theory.