@Graque said in #19:
> Interesting article, thanks for sharing!
>
> It seems one easy thing though that the authors forgot to include is some comparative ranking (either using Kendall Tau or something else) of how the participant's ability to solve the puzzles lines up with their Elo rankings. In other words, although participants' judgments weren't very similar to the Elo rankings, how about participants' performance on the puzzles?
>
> Without this, it seems hard decide between:
>
> 1. People aren't good at estimating the difficulty of puzzles
>
> 2. The easy/medium/difficult categorization wasn't correct for this particular group (because of statistical significance, difference between Chess Tempo conditions, etc.)
Yes, I think @Maouna is making similar points and I agree these would be important to consider. They do have a separate comparison of Kendall's Tau to players' ability to solve these specific puzzles, but not an estimate of overall puzzle ELO. This relationship is at least moderately positive, but it also does not reach significance, which could be a power issue for sure. Thanks for reading!