Your network blocks the Lichess assets!

lichess.org
Donate

Help us create better chess puzzles!

Hello, I'm a future software dev and a chess player for quite a while now( 5-7 years), I've been thinking quite a lot about how can chess puzzles be improved, I'm here to share my ideas, I hope this is the right place to do it, I think the main problem is evaluating whether a player understands the solution or not such that he would only succeed in the puzzle if his thoughts would allow him to get the best of the position no matter what the other player plays( almost), for example a lot of times in puzzles the solution is to sacrifice a piece, and there are two choices for the opponent, either to take the piece or not, if the opponent takes a piece he gets mated in 10 or sth, if he doesn't he loses a piece after a more basic continuation, so as a player you try to find the mate after the piece sacrifice, you may find it and you may not, but if you're only prompted to reply on the opponent not taking the piece you sacrificed, well you could have gotten the mate wrong and you would still get the puzzle right, so you can't know if a player got the puzzle right by asking him for just one line. I think having an algorithm that would determine the critical lines and the order in which they should be shown would take puzzles into the next level.

Well, all of this is just the idea, I think in practice it could be pretty hard, the critical part I think is having an algorithm that determines the critical lines, one idea could be to try to evaluate how hard moves are in a given line for the player to keep the advantage, and trying to ask player on the lines that are the hardest to find, for example a line where the opponent backs off and he's down a piece would be easy to reply against, but a line where the player has to come up with a line to defend a bunch of mating threats while he's up a piece would probably be trickier. Another critical part would be to know the right order of showing the replies to the player, because sometimes seeing the opponent's ideas from one line could very much help in figuring out another line, it's here where I'm not sure if this whole idea could even work, I feel like sometimes maybe there is no best order of lines, but I think showing multiple lines to the player would still greatly help the player better understand the position without having to analyze the position which is very tedious IMO.

Hello, I'm a future software dev and a chess player for quite a while now( 5-7 years), I've been thinking quite a lot about how can chess puzzles be improved, I'm here to share my ideas, I hope this is the right place to do it, I think the main problem is evaluating whether a player understands the solution or not such that he would only succeed in the puzzle if his thoughts would allow him to get the best of the position no matter what the other player plays( almost), for example a lot of times in puzzles the solution is to sacrifice a piece, and there are two choices for the opponent, either to take the piece or not, if the opponent takes a piece he gets mated in 10 or sth, if he doesn't he loses a piece after a more basic continuation, so as a player you try to find the mate after the piece sacrifice, you may find it and you may not, but if you're only prompted to reply on the opponent not taking the piece you sacrificed, well you could have gotten the mate wrong and you would still get the puzzle right, so you can't know if a player got the puzzle right by asking him for just one line. I think having an algorithm that would determine the critical lines and the order in which they should be shown would take puzzles into the next level. Well, all of this is just the idea, I think in practice it could be pretty hard, the critical part I think is having an algorithm that determines the critical lines, one idea could be to try to evaluate how hard moves are in a given line for the player to keep the advantage, and trying to ask player on the lines that are the hardest to find, for example a line where the opponent backs off and he's down a piece would be easy to reply against, but a line where the player has to come up with a line to defend a bunch of mating threats while he's up a piece would probably be trickier. Another critical part would be to know the right order of showing the replies to the player, because sometimes seeing the opponent's ideas from one line could very much help in figuring out another line, it's here where I'm not sure if this whole idea could even work, I feel like sometimes maybe there is no best order of lines, but I think showing multiple lines to the player would still greatly help the player better understand the position without having to analyze the position which is very tedious IMO.

@mediusonly

Find the best mouvv , at the last time..:

https://lichess.org/V9P3ekIYhXMH

AnD get out thisS Arabiean DrawmaT!?

@mediusonly Find the best mouvv , at the last time..: https://lichess.org/V9P3ekIYhXMH AnD get out thisS Arabiean DrawmaT!?

@revolution_rnt said in #15:

I wish it had dark mode
Pretty sure it does. If not, you could always use a darkmode browser extension.

@revolution_rnt said in #15: > I wish it had dark mode Pretty sure it does. If not, you could always use a darkmode browser extension.

@Fluffydragonfes said in #24:

Pretty sure it does. If not, you could always use a darkmode browser extension.

all features not necessary to the project are not implemented. But styling in general should work. however not the onces from lichess. Maybe stylus scripts for Lichess might play with board looks...

there is no profile . For now, only the puzzle random offer, and the dashboard-history work to my experience.

I wish they would tell us more about the project. But it seems the competition context has some information blackout about it.
I wonder what that means for transparency after the competition. Chess things before Lichess did not have a strong tradition about that (my lasting impression, maybe mistaken from some gaps). But I wish them the best. And hope for the best on that aspect.

@Fluffydragonfes said in #24: > Pretty sure it does. If not, you could always use a darkmode browser extension. all features not necessary to the project are not implemented. But styling in general should work. however not the onces from lichess. Maybe stylus scripts for Lichess might play with board looks... there is no profile . For now, only the puzzle random offer, and the dashboard-history work to my experience. I wish they would tell us more about the project. But it seems the competition context has some information blackout about it. I wonder what that means for transparency after the competition. Chess things before Lichess did not have a strong tradition about that (my lasting impression, maybe mistaken from some gaps). But I wish them the best. And hope for the best on that aspect.

why is it so important to you, for us to create an account with the same username?

why is it so important to you, for us to create an account with the same username?

It is very important to create an account with the same username so the rating puzzle can be calculated using the glicko's lichess puzzle rating of the user.

It is very important to create an account with the same username so the rating puzzle can be calculated using the glicko's lichess puzzle rating of the user.

@StudyKeyser said in #13:

About the ML competition where we have to guess the puzzle's rating, what is stopping me from scraping the lichess puzzles database and submit the real ratings ? I guess if one does that he won't be eligible for prizes

These puzzles can't be found in lichess db. That's why we need human evaluators to solve them :)

@dleli said in #5:

is this just about predicting the puzzle "rating"?
As a chess person, I find this problem very well addressed by all the tactics trainers around: lichess, chess.com, chesstempo. I very seldom find myself given with a problem and then, after solving it, thinking that the difficulty rating was crazy far from my perception. Of course this may change for player stronger than me, but this is how it feels to me weak player (and problems solver).

This problem has not, in fact, been addressed by the tactics trainers. What all these websites rely on is having a large number of people solve the puzzles in order to be able to rate them.

But suppose you missed a tactic in your own game and you want to see how difficult it was to solve? The research that we are promoting will help solve that :)

Your research idea of studying human perception of puzzles is very interesting and I encourage you to pursue it! For now, we settled for a problem that is well-defined and one for which we have ample training data (3 million puzzles courtesy of lichess!)

@nikonoel said in #6:

Couldn't you implement a login via lichess, using its API? Don't quote me on this but I think this should be helpful?

Unfortunately we are mostly scientists, not web developers, so we don't have the skills necessary to build that in our free time.

@affstein said in #7:

I would have preferred if you had disclosed who precisely "we" is. Based on the first paragraph, I thought you were lichess developers and I have the impression, that this misdirection is intentional.

I am sorry that you felt misled, nowhere does it say that we are in any way affiliated with lichess (although lichess staff did help us with a number of things for which we are very grateful! Also we really hope that the research will contribute to open source chess in the future). Pro tip: official lichess blogs are actually published by the lichess user: https://lichess.org/@/Lichess

Your accusation is hurtful, but I understand how you could come to such a conclusion. For my part, I used "we", because it's the opening paragraph where we try to get people's attention and it's shorter than "a group of Polish Machine Learning researchers and chess enthusiasts".

@dboing said in #16:

how many puzzle tried do you need us to try minimally. And is there some notion of time or duration of puzzle done in a row.

would 30 be enough? How many total puzzle IDs in your set? by the way, curious.

It would be nice to have some feedback about the puzzles we did. so I can stop when you have enough. I want to help, but I am selfish too... On lichess I don't do puzzles just for the rating. There is more stuff to do with them. It is as much about me doing them as it is about the positions themselves, my encountering them.

Maybe the dasborad and history of puzzles is function. I wlil look.. I just did not see the usual underboard limited history, so guessing no history visible to user.

Thank you for signing up and helping us with the data! The more the better, there are over 2000 puzzles on the website, I don't think anyone will solve them all in this time :) Maybe treat it like a duolingo lesson, solving 2-3 puzzles daily? It would help us a lot and not take a lot of time, I promise. Unfortunately, we had to turn off access to many pages (like e.g. user profile) as it contains information about user's puzzle rating, which in turn can be used to infer individual puzzle's rating, and that could compromise the integrity of the competition. Thank you for your understanding!

@dboing said in #19:

been clicking on many links on the front page. a few are not 404. I am glad that the "source code" link leads somewhere

github.com/FrugoFruit90/lilaPuzzles

I looked at the repository, but I don't think the readme file is having any changes about possible modificatinos and purpose of that fork. Maybe authors could give us some hints, about the extent of open access to what they can spare for us given the competition informatino black out (???). Or we can be patient. I think open source code is limited when it comes to such data analysis research projects. So the curious might be left hungry. That would be me.

This blog was actually meant to explain the idea of the competition and the lichess fork :sweat_smile:
You can find more detailed, scientific information on the competition's website and forum: https://knowledgepit.ai/predicting-chess-puzzle-difficulty/

@StudyKeyser said in #13: > About the ML competition where we have to guess the puzzle's rating, what is stopping me from scraping the lichess puzzles database and submit the real ratings ? I guess if one does that he won't be eligible for prizes These puzzles can't be found in lichess db. That's why we need human evaluators to solve them :) @dleli said in #5: > is this just about predicting the puzzle "rating"? > As a chess person, I find this problem very well addressed by all the tactics trainers around: lichess, chess.com, chesstempo. I very seldom find myself given with a problem and then, after solving it, thinking that the difficulty rating was crazy far from my perception. Of course this may change for player stronger than me, but this is how it feels to me weak player (and problems solver). This problem has not, in fact, been addressed by the tactics trainers. What all these websites rely on is having a large number of people solve the puzzles in order to be able to rate them. But suppose you missed a tactic in your own game and you want to see how difficult it was to solve? The research that we are promoting will help solve that :) Your research idea of studying human perception of puzzles is very interesting and I encourage you to pursue it! For now, we settled for a problem that is well-defined and one for which we have ample training data (3 million puzzles courtesy of lichess!) @nikonoel said in #6: > Couldn't you implement a login via lichess, using its API? Don't quote me on this but I think this should be helpful? Unfortunately we are mostly scientists, not web developers, so we don't have the skills necessary to build that in our free time. @affstein said in #7: > I would have preferred if you had disclosed who precisely "we" is. Based on the first paragraph, I thought you were lichess developers and I have the impression, that this misdirection is intentional. I am sorry that you felt misled, nowhere does it say that we are in any way affiliated with lichess (although lichess staff did help us with a number of things for which we are very grateful! Also we really hope that the research will contribute to open source chess in the future). Pro tip: official lichess blogs are actually published by the lichess user: https://lichess.org/@/Lichess Your accusation is hurtful, but I understand how you could come to such a conclusion. For my part, I used "we", because it's the opening paragraph where we try to get people's attention and it's shorter than "a group of Polish Machine Learning researchers and chess enthusiasts". @dboing said in #16: > how many puzzle tried do you need us to try minimally. And is there some notion of time or duration of puzzle done in a row. > > would 30 be enough? How many total puzzle IDs in your set? by the way, curious. > > It would be nice to have some feedback about the puzzles we did. so I can stop when you have enough. I want to help, but I am selfish too... On lichess I don't do puzzles just for the rating. There is more stuff to do with them. It is as much about me doing them as it is about the positions themselves, my encountering them. > > Maybe the dasborad and history of puzzles is function. I wlil look.. I just did not see the usual underboard limited history, so guessing no history visible to user. Thank you for signing up and helping us with the data! The more the better, there are over 2000 puzzles on the website, I don't think anyone will solve them all in this time :) Maybe treat it like a duolingo lesson, solving 2-3 puzzles daily? It would help us a lot and not take a lot of time, I promise. Unfortunately, we had to turn off access to many pages (like e.g. user profile) as it contains information about user's puzzle rating, which in turn can be used to infer individual puzzle's rating, and that could compromise the integrity of the competition. Thank you for your understanding! @dboing said in #19: > been clicking on many links on the front page. a few are not 404. I am glad that the "source code" link leads somewhere > > github.com/FrugoFruit90/lilaPuzzles > > I looked at the repository, but I don't think the readme file is having any changes about possible modificatinos and purpose of that fork. Maybe authors could give us some hints, about the extent of open access to what they can spare for us given the competition informatino black out (???). Or we can be patient. I think open source code is limited when it comes to such data analysis research projects. So the curious might be left hungry. That would be me. This blog was actually meant to explain the idea of the competition and the lichess fork :sweat_smile: You can find more detailed, scientific information on the competition's website and forum: https://knowledgepit.ai/predicting-chess-puzzle-difficulty/

how is the project going? will you keep us informed of how your database and puzzle pool get experienced. It might bring the attention back to that experiment. And it would be nice to hear anyway about it from time to time.

Whether you won or not. So that your efforts and ours might have some closure. Or are you continuing the experiment for its own sake. etc..

how is the project going? will you keep us informed of how your database and puzzle pool get experienced. It might bring the attention back to that experiment. And it would be nice to hear anyway about it from time to time. Whether you won or not. So that your efforts and ours might have some closure. Or are you continuing the experiment for its own sake. etc..

@dboing said in #29:

how is the project going? will you keep us informed of how your database and puzzle pool get experienced. It might bring the attention back to that experiment. And it would be nice to hear anyway about it from time to time.

Whether you won or not. So that your efforts and ours might have some closure. Or are you continuing the experiment for its own sake. etc..

The competition has finished, the winners were found.
We invited the top teams to submit their papers explaining their approach etc. We will also write our own paper explaining the competition, summarizing different approaches etc.
In December the conference will be held in Washington D.C., USA.

I think around November we will get all the papers and I will write a follow-up blog post to present the winners and invite to the second edition of the competition.
Cheers!

@dboing said in #29: > how is the project going? will you keep us informed of how your database and puzzle pool get experienced. It might bring the attention back to that experiment. And it would be nice to hear anyway about it from time to time. > > Whether you won or not. So that your efforts and ours might have some closure. Or are you continuing the experiment for its own sake. etc.. The competition has finished, the winners were found. We invited the top teams to submit their papers explaining their approach etc. We will also write our own paper explaining the competition, summarizing different approaches etc. In December the conference will be held in Washington D.C., USA. I think around November we will get all the papers and I will write a follow-up blog post to present the winners and invite to the second edition of the competition. Cheers!