Is this AI analysis of a 5 game losing streak correct?

Clownboots

The question is not that difficult and there is no need to over analyse the data. What should the real number be given the set of numbers provided? Do you agree with the AI number of 1.32%, using the parameters given?

mkubecek

As I tried to explain: under the assumptions taken and with the definition of "probability" used, the calculation is probably correct. (I didn't bother to check.) Whether those assumptions and definition are reasonable and whether the result is actually useful for any purpose, that's a completely different question.

glbert

edited

@Clownboots said in #11:

What should the real number be given the set of numbers provided?

you ignore the "adjustments" and just multiply the given winning chances. interestingly, the winning chances given by the ai for individual games look approximately correct. anyway, i calculated around 0.6%, but one should probably round that to 1%. there's several assumptions* in there, so the estimate is not precise enough to give three digits of the probability.

Do you agree with the AI number of 1.32%, using the parameters given?

i would round that down to 1%, calling it 1.32% is absolutely ridiculous. and then i would agree with the ballpark of the number.

also, you would have gotten an answer to this question way faster if you did not start with a heap of ai garbage. don't do it. people are developing terrible habits there.

assumptions: draw rate between players of that level, their elo ratings being fixed (and remotely correct, which is unlikely to be true for all of the six players), independence of games

@Clownboots said in #11: > What should the real number be given the set of numbers provided? you ignore the "adjustments" and just multiply the given winning chances. interestingly, the winning chances given by the ai for individual games look approximately correct. anyway, i calculated around 0.6%, but one should probably round that to 1%. there's several assumptions* in there, so the estimate is not precise enough to give three digits of the probability. > Do you agree with the AI number of 1.32%, using the parameters given? i would round that down to 1%, calling it 1.32% is absolutely ridiculous. and then i would agree with the ballpark of the number. also, you would have gotten an answer to this question way faster if you did not start with a heap of ai garbage. don't do it. people are developing terrible habits there. * assumptions: draw rate between players of that level, their elo ratings being fixed (and remotely correct, which is unlikely to be true for all of the six players), independence of games

glbert

edited

talking about the 1.32 again btw, i hope people can see how ridiculous that number is. the ai is feeding you an over-precise estimate based on the known phenomenon of people trusting overprecise numbers more, because "surely a number that precise couldn't be just made up". the main goal of the ai is not to give you the actual truth, but to convince you that it's giving you the truth. those two goals are quite different.

basically, when you are given a number that is this precise in a context like this, you already know it's bullshit from the start. it would be dumb even for the winning chance of one player, much less for the combined chance of 5 total wins by 5 different players.

when calculating winning chances for even a single player, i would say even rounding to 1% is quite optimistic. in most cases rounding to multiples of 5 would probably be the best you could do without pretending to know more than you actually do.

talking about the 1.32 again btw, i hope people can see how ridiculous that number is. the ai is feeding you an over-precise estimate based on the known phenomenon of people trusting overprecise numbers more, because "surely a number that precise couldn't be just made up". the main goal of the ai is not to give you the actual truth, but to convince you that it's giving you the truth. those two goals are quite different. basically, when you are given a number that is this precise in a context like this, you already know it's bullshit from the start. it would be dumb even for the winning chance of one player, much less for the combined chance of 5 total wins by 5 different players. when calculating winning chances for even a single player, i would say even rounding to 1% is quite optimistic. in most cases rounding to multiples of 5 would probably be the best you could do without pretending to know more than you actually do.

discoooooord

is there anyone in this thread who hasn't lost 5 games or more in a row? I smell more cheating paranoia

glbert

edited

oh, what discoooooord said reminded me of something i forgot about: the result of these calculations also depends on where you got the players from. just multiplying winning chances is only reasonable if you did not play those players yet.
however, if you picked a random losing streak and asked "wow, what were the chances of that happening?" the calculations would be different. or rather, you posed an incorrect question.

oh, what discoooooord said reminded me of something i forgot about: the result of these calculations also depends on where you got the players from. just multiplying winning chances is only reasonable if you did not play those players yet. however, if you picked a random losing streak and asked "wow, what were the chances of that happening?" the calculations would be different. or rather, you posed an incorrect question.

Panagrellus edited

Like others above, I think adding adjustments for accuracy muddies the water here, but the basic principle (multiplying the individual probabilities) looks correct.

For example, getting five sixes in row with a regular dice has probability:
1/6 * 1/6 * 1/6 * 1/6 * 1/6 , which is 0.013 %. So far so correct.

BUT what you can't do is take a long series of game results (say, hundreds) and focus on a sub-series of 5 lost games within this larger series, and then claim that this 5-game-streak, which was arbitrarily selected from the total series, was highly unlikely. This would be a classic case of cherry picking.

Within a large series, it is likely that you find all kinds of sub-series - chance produces patterns. If your question is whether something really unusual was going on after losing 5 times in a row (out of a much larger series of your entire online game experience), you'd need to ask the question:

"How likely is it that within a series of (say) 5000 games played on Lichess, and considering each opponent's elo, that there will be at least one series of 5 or more lost games."

==
Additionally, this type of frequentist stats assumes independence of events. But as we all know, there is something called "tilt" , where lost games lead to more lost games due to psychological reasons. So the assumptions are not really valid for chess games played by fragile human minds.

Like others above, I think adding adjustments for accuracy muddies the water here, but the basic principle (multiplying the individual probabilities) looks correct. For example, getting five sixes in row with a regular dice has probability: 1/6 * 1/6 * 1/6 * 1/6 * 1/6 , which is 0.013 %. So far so correct. BUT what you can't do is take a long series of game results (say, hundreds) and focus on a sub-series of 5 lost games within this larger series, and then claim that this 5-game-streak, which was arbitrarily selected from the total series, was highly unlikely. This would be a classic case of cherry picking. Within a large series, it is likely that you find all kinds of sub-series - chance produces patterns. If your question is whether something really unusual was going on after losing 5 times in a row (out of a much larger series of your entire online game experience), you'd need to ask the question: "How likely is it that within a series of (say) 5000 games played on Lichess, and considering each opponent's elo, that there will be at least one series of 5 or more lost games." == Additionally, this type of frequentist stats assumes independence of events. But as we all know, there is something called "tilt" , where lost games lead to more lost games due to psychological reasons. So the assumptions are not really valid for chess games played by fragile human minds.

Clownboots

@glbert said in #13:

you ignore the "adjustments" and just multiply the given winning chances. interestingly, the winning chances given by the ai for individual games look approximately correct. anyway, i calculated around 0.6%, but one should probably round that to 1%. there's several assumptions* in there, so the estimate is not precise enough to give three digits of the probability.

Thanks for your reply. Three points. 1- I have no real problem with expected score the AI answers that normally recognising that winning and losing streaks are normal, 1% error is not that significant.

2- I am interested in the expected accuracy of players move selection. Here I think the AI does not understand the question at all.
3- It is my opinion that players can improve more objectively in strength if they accept that their ratings are reasonably accurate and that they will play at a range of strengths for individual games, with the caveat that after a large enough population sample they will have an average accuracy number. Therefore I would think a streak of having five opponents play at over 90% is probably more of an outlier statistically for that rating group. And I still remain curious as to how to measure such samples and how far they differ in standard deviation. The same would be true if I had a winning streak of 5 games against players with an accuracy rating of 55%. Obviously the population of games played by my opponent would be a significant factor and I wonder how this relates to rating.

@glbert said in #13: > you ignore the "adjustments" and just multiply the given winning chances. interestingly, the winning chances given by the ai for individual games look approximately correct. anyway, i calculated around 0.6%, but one should probably round that to 1%. there's several assumptions* in there, so the estimate is not precise enough to give three digits of the probability. Thanks for your reply. Three points. 1- I have no real problem with expected score the AI answers that normally recognising that winning and losing streaks are normal, 1% error is not that significant. 2- I am interested in the expected accuracy of players move selection. Here I think the AI does not understand the question at all. 3- It is my opinion that players can improve more objectively in strength if they accept that their ratings are reasonably accurate and that they will play at a range of strengths for individual games, with the caveat that after a large enough population sample they will have an average accuracy number. Therefore I would think a streak of having five opponents play at over 90% is probably more of an outlier statistically for that rating group. And I still remain curious as to how to measure such samples and how far they differ in standard deviation. The same would be true if I had a winning streak of 5 games against players with an accuracy rating of 55%. Obviously the population of games played by my opponent would be a significant factor and I wonder how this relates to rating.

Clownboots

@Panagrellus said in #17:

"How likely is it that within a series of (say) 5000 games played on Lichess, and considering each opponent's elo, that there will be at least one series of 5 or more lost games."

Thanks for your reply.
I am not concerned with the losses. I simply want to be able to measure my opponents and my own accuracy of moves across sessions and openings so as to place performance in perspective. For example if I play a certain opening like the Ruy Lopez I may be interested in what strength my opponent played the Black pieces. Sure it can be measured by the players ELO, but there is still a qualitative difference in the games if my opponent played at 70% compared to 80%. With this information I can group populations and understand how much my own results vary against certain scenarios.

@Panagrellus said in #17: > "How likely is it that within a series of (say) 5000 games played on Lichess, and considering each opponent's elo, that there will be at least one series of 5 or more lost games." Thanks for your reply. I am not concerned with the losses. I simply want to be able to measure my opponents and my own accuracy of moves across sessions and openings so as to place performance in perspective. For example if I play a certain opening like the Ruy Lopez I may be interested in what strength my opponent played the Black pieces. Sure it can be measured by the players ELO, but there is still a qualitative difference in the games if my opponent played at 70% compared to 80%. With this information I can group populations and understand how much my own results vary against certain scenarios.

mkubecek

It doesn't really work like that. Even the same player having 95% accuracy in one game and 85% in another doesn't necessarily mean they actually played better chess in the former. If you are interested, I posted some examples showing how misleading accuracy can be in https://lichess.org/forum/community-blog-discussions/ublog-pCxNOHqU#5 (comments 5 and 6).

Sure, there is "the other chess site" which, reportedly, shows you a "game rating performance", a made up number supposedly expressing "the rating level you played as in the game". But that doesn't have any practical sense, its only purpose is to make people feel better and pay more.

It doesn't really work like that. Even the same player having 95% accuracy in one game and 85% in another doesn't necessarily mean they actually played better chess in the former. If you are interested, I posted some examples showing how misleading accuracy can be in https://lichess.org/forum/community-blog-discussions/ublog-pCxNOHqU#5 (comments 5 and 6). Sure, there is "the other chess site" which, reportedly, shows you a "game rating performance", a made up number supposedly expressing "the rating level you played as in the game". But that doesn't have any practical sense, its only purpose is to make people feel better and pay more.

Your network blocks the Lichess assets!