The big danger in evaluating point systems and pairing systems is that common sense isn’t always right.

In fact, in comment #8 I said:

“I was a bit surprised with the internal data that showed that the error rate was much smaller in the middle of the finishers vs. the top and bottom”

After I got the actual simulation data, it became clear that the middle actually has the best resolution, while the top and bottom have the worst. Same for BAP and non-BAP. Optimal pairing algorithms are very complicated beasts and the smallest change in pairing preference changes the average rank difference. This cuts both ways as it means with the proper evaluation function of different pairings, it is possible to significantly improve the result. It it also possible to significantly hurt things too.

Anyway, my simulation results show that BAP vs. non-BAP changes things about as much as giving the higher rated player white instead of black, when both players have no color imbalance, eg. very little. What this means is that I will be able to easily create a BAP pairing program that achieves BETTER metrics than the current swiss. I already have it so it gives everybody equal colors without losing any resolution. While it won’t be possible to get double round robin accuracy with log2 rounds, I can at least optimize it to be as good as it can get.

BAP does not hurt tournament outcome accuracy because higher rated players do better, regardless of the point system.

Clint

Not sure why you think that BAP randomizes the results in the middle, or that upsets don’t happen.

I never said upsets don’t happen.

As for the first part, I first got that impression from your comment (#6 above, Jan 3):

In your example, you are comparing two players in the middle of the field and in a swiss with around log base 2 rounds, there is very little resolution of the players in the middle.

You say yourself there is little resolution in the middle, hence my “randomizes” characterization.

I have come up with a tournament format that is independent of BAP, but takes advantage of BAP and have described it on gmslugfest.com. The plan is to have a $25,000 first prize, which can be split N-ways in case of an N-way tie. The top 4 players with 3 rounds to go play a big money quad. The rest of the players are also all playing in quads for a $1500 prize.

I now have a tournament simulation infrastructure so I can test a variety of point systems and pairing systems to see what their statistical behavior is. I can also see what the worst case scenarios are for BAP and non-BAP. So far, the BIGGEST factor that determines who wins the tournament is playing strength. Basically, the best chess players win tournaments, regardless of what scoring or pairing system is used. That being said, my interest is in minimizing any negative factors and maximizing the positive factors.

Clint

Maybe you have a better way of comparing pairing systems than my compare the average rank of each player?

Outliers interest me far more than averages. In part because I learned long ago that outliers tell you more about your dataset than “normal” points do.

But in this case it’s of practical concern as well. As an organizer, I have to field the complaints of the players who play in my events. And I’ve never met a player yet who was satisfied with “in your particular instance this isn’t correct, but on average this is correct.” They’re only concerned with their specific case. Especially where money is concerned.

You’re also playing to a different audience than I. Only a few players who come to my events care about first prize, mainly because it’s a rare event that has more than 6-8 capable of winning it. The gulf between highest and lowest rated player nearly always over 1000 rating points. When I award upset prizes they are routinely won by players with 700+ point upsets, something that would rarely be able to happen in your slugfests.

I have to be concerned with prizes other than first, or I lose my players. I have to be able to explain what I’m doing to them and make them understand it, and right now I wouldn’t relish the idea of explaining to them I’m using a scoring method intended to reduce draws to 25% (especially since the draw rate in the last two events I organized was already below that ratio) but which may randomize the results in the middle of the table a little. It’s a price few of my players would be willing to pay.

Mathematically, the purpose of a tournament is to determine the ranking of all the players that are in it. Chesswise, a double-round robin eliminates all the big variables that any other tournament format has, eg. different opponents, colors, etc. Given a pairing system that has significantly fewer rounds than a DRR, random chance enters into the picture, so I ran iterations until the error rate converged. With an average of 50 players, in about 100 tournaments, the error rates converged to pretty close to the values that I got at thousands of iterations, so 100 tournaments seemed like a sufficient number to eliminate the errors.

I randomly generate a set of players of specified elo with a random number of rounds and then at the end of the tournament, the finishing rank of the players are compared with the actual rank. In some cases two players could have very similar ratings, or even exactly the same, so some amount of “error” is expected. Anyway, I don’t have all the kinks worked out yet, but initial results using thousands of different tournament configurations shows a RR (not DRR) having an average error of around 0.3 ranks. Swiss using standard pairings ends up averaging around 3 ranks, with “basketball” swiss around 2 ranks. Since there are an average of 50 players, the error rate is 4% to 10%. “basketball” swiss is where the top player in each point group plays the bottom player as opposed to the player in the middle. Just this change cut the error rate in half, so this seems like a very good thing to do.

Now, when I used BAP to do the pairings, there was no statistically significant difference with the normal midpoint swiss pairing, but there was a bit over 0.5 error rate more when using the basketball method. The results are preliminary and not 100% confirmed, but so far:

RR 0.3
Swiss ~3
BAP Swiss ~3
Basketball Swiss ~2
BAP Basketball Swiss ~2.5

I was a bit surprised with the internal data that showed that the error rate was much smaller in the middle of the finishers vs. the top and bottom.

I hope you an agree that looking at one specific tournament, or even one specific case is not really meaningful when the statistical variation is literally millions of possibilities. Maybe you have a better way of comparing pairing systems than my compare the average rank of each player? If so, I would be glad to implement that.

With a statistically verifiable metric, we can objectively compare different point systems, pairing systems and see which one comes closest to the DRR ideal with minimum number of rounds.

With basketball swiss, around 3% of the tournaments ended up with a winner that wouldn’t have won based on old fashioned points. But that means 97% of the time they are one and the same and that explains why we haven’t seen it in practice yet. I want to thank you for inspiring me to finally write the BAP pairing program! I still haven’t implemented the strict color balancing that I use when I run BAP tournaments, so it is quite possible that the BAP numbers will change. Now that I have a statistical verification framework, I can optimize the best pairing system possible and my feeling is that since BAP provides more information about the result, it will be possible to create a more accurate pairing system using BAP than the old point system.

Clint

BAP makes chess a bit like tennis.

I don’t see the analogy at all. I actually designed a tennis-like chess tournament once: Play proceeds at 1 round per day, a round consists of best of three matches, match consists of best of 12 games. Because of the duration, time control had to be rapid, used blitz. Never had the time to test it, though.

What is the unfair reward that the player who scored 6 BAP with 2 black wins vs. 6 BAP with 3 white wins, when neither win any prizes and presumably the 3 white win player does better rating-wise?

The points on the scoreboard, therefore the standing in the tournament. There are plenty of players who don’t play for prizes or rating points (Heck, if that were the reason I played, I would have quit long ago). If you insist on prizes, they could easily be in the running for a class prize.

But since that annoys you, let’s use a different example: Player A has 2 black wins and 2 white wins (one white draw) going in to the last round. Player B was perfect with three black wins and two white wins. A gets Black and wins. A and B finish in a tie, despite the facts that A beat B (no big deal there) and also that A finished unbeaten, giving up a single draw while B lost a game! By any rational judgement I can understand, A had a better tournament than B, played better, produced a better score in terms of game results (both were perfect with Black while both only failed to win one game with white, which A drew while B lost) and yet finished tied with B.

In your own Slugfest, the scoring system would have cost GM Akobian money by dropping him from what would otherwise have been a certain check via a 3-way tie for 2-4 and instead dropping him into a 4-way tie for 4-7. Most events offer a 2nd prize, though not all, but very few small weekend swisses offer a 4th place prize. If this system is applied to bigger events, then little things like 2nd place ties may start to matter to the players.

Maybe this is all doom and gloom, I don’t know. I’ll keep watching for further info, though. Meanwhile, I’ll keep experimenting on my own with formats and prize structures.

By “most deserving”, I mean the player that would have won using the old point system is expected to also win based on BAP points. While it is not guaranteed to be this way, all the events so far has come out this way.

In your example, you are comparing two players in the middle of the field and in a swiss with around log base 2 rounds, there is very little resolution of the players in the middle. Neither of the players you mention would be in the running for prize money, so while one player would have bragging rights about having scored the same number of BAP points, this does not seem to be to be a flaw with BAP. If you are saying that a point system needs to rank all players in a tournament that is not a round robin,then I doubt there exists such a point system. The current point system doesn’t do it and neither does BAP.

Why does BAP have to do things that the current point system doesn’t do? It makes all players play every game as hard as they can. The player that would have won using the normal system has been winning the BAP tournaments. White does not throw away draws recklessly and ends up with approximately the same overall performance if the old system is used. Current rating system is preserved. Cheating that currently exists does not seem to get worse. One reason is that a lot more players are in the running for the prizes in the last round and that has more than compensated for whatever increased temptations BAP creates for cheating.

BAP makes chess a bit like tennis. If you lose the first game in tennis, it is either expected or really bad and you can’t fairly assess where you are until you play the second game. Other than prize money, BAP only affects pairings during the tournament and is therefore not something that I consider a “reward”. the prize money has gone to the players that would have won normally. Remember that the games are rated using normal methods.

http://www.slugfest7.com/public/140.cfm has some math that will hopefully appeal to your math side. If the draw rate is 20%, then BAP is color neutral with a 1.5 white win to black win ratio. I hope we can agree that any point system that is less biased than the current one is actually more fair and at the GM level I would be surprised if the white win ratio isn’t around 1.5 to 1.

What is the unfair reward that the player who scored 6 BAP with 2 black wins vs. 6 BAP with 3 white wins, when neither win any prizes and presumably the 3 white win player does better rating-wise?

Clint

“The whole cheating issue is something that BAP does not deal with, my assumption is that collusion between players happens when dishonest players play each other. Cheaters will cheat, honest people won’t.”

Yes…and no. There’s a grey area that many players feel comfortable moving in. Is it cheating to sit down during rounds 3&4 of a weekend 5-rounder and do “income negotiation?” Lots of players don’t feel that way, it appears. There’s an old saying “If the opportunity for murder always came along with the motive, who among us would escape hanging?”

I think the vast majority of players are basically honest, yes. But even the honest succumb to temptation, and I see this as putting another temptation before them. So far I’ve seen crosstables for small round-robin affairs, but no swisses. No large population groups, only small test ones. I think the results are interesting, yes, but not interesting enough for me to venture in just yet. I’ll be keeping an eye on things, though. Count on it.

As for the rest:

“Who cares what the white vs. black ratio of BAP points was if the prizes go to the most deserving players.”

Your rhetoric gets away with you here. Are you really meaning to say that a player who is more comfortable playing black is, by definition, “more deserving” than a player who is more comfortable with White?

What I continue to be skeptical about is the fairness of rewarding people out of proportion to their achievement. If both players play perfectly only one gets rewarded. In a six-round event, a player who wins one black game and two whites will beat a player who wins all three whites. Before you respond that the first player did something more difficult than the second in winning with Black, note that the first player *also* did something weaker than the second, in that he failed to win one of his games with White; surely failing to hold an advantage is a worse fault than failing to overcome a disadvantage. A player who wins 2/3 with Black is equal to a player who wins all three Whites, even though that first player failed to even hold a draw in a single game with white?

I’m a math geek, I need numbers. My theoretical analysis doesn’t make me optimistic about your system, but I’m well aware theory doesn’t always match practice, so I’ll be watching future events. Maybe I’ll come around to your POV, but not at the moment.

This whole BAP will increase cheating is only a theoretical problem, that already exists, so I am not sure why it is even something against BAP. Hopefully, we can agree that a good test of a point system is its effect on honest players and that we don’t have to construct point systems based on minimizing the incentives for cheating. In any case, when there isn’t a big prize fund, cheating isn’t an issue and if the prize fund is big enough, the tax liability the winner has will complicate any prize sharing arrangements.

People have speculated that due to the point system, the white players would go “crazy” and start tossing games for no reason. The evidence so far doesn’t show that at all, in fact, your analysis shows that the ratio of white wins, draws and black wins is virtually unchanged under BAP! Clearly, white is simply playing the way white should. Loss of rating points and not wanting to lose overrides any irrational desire to lose a game that they can draw as white. Would you toss a drawn game as white because it doesn’t affect your BAP score? You lose rating points, you lose to your opponent and your opponent gets 2 more points making your chances of prize money significantly less. There are a lot of reasons for white to fight for a draw instead of being silly.

This means that BAP does not change the overall results, but it has eliminated the grandmaster draw. Since BAP is not used for ratings, whether white or black ended up with more or less BAP points seems to be moot. Who cares what the white vs. black ratio of BAP points was if the prizes go to the most deserving players.

Faster controls are certainly one way to reduce the draw rate by increasing blunders. Personally, I like to play as close to a blunder-free game as possible, so I am biased against the speed chess solutions. The good players can just play to draw the normal time control and play the decisive games at the faster and faster time controls. Do we really want to have the world championship of chess decided by a game of speed chess? Tournament chess is not speed chess and deciding tournament games by speed chess is probably the best option if BAP didn’t exist.

I am not sure what you continue to be skeptical about as BAP has achieved all of its primary goals and even some secondary goals. While current results do show black scoring more BAP points than white, it has not changed who wins the prize money. For chess to be a sport, there needs to be a winner and a loser for every game and BAP achieves this primary goal, since a draw is a small victory for black.

You really need to see a BAP tournament in person to feel the difference it creates. Players look forward to their black games. There are no agreed draws, until it is getting close to insufficient material and clearly drawn. In fact, it makes all the draws the fighting draws that people say are an important part of chess. I agree and with BAP one quarter of the games are fighting draws and three quarters are decisive games.

Why not tiptoe to the BAP side of things by holding a small BAP event so you can see firsthand what its like. Pairings are much fairer to the players. I can do the pairings for your event and provide the prize fund.

Clint

P.S. With a small prize fund, there is no cheating issue, so you can see the full effect of BAP with a modest winner take all prize. By having only one prize, it minimizes the desire to split the prize as the more people who share the first place prize, the smaller amount each person gets.