Website founded by
Milan Velimirović
in 2006
1:06 UTC
| |
MatPlus.Net  Forum Competitions Difficulties with points estimation in solving competitions |
| |
|
|
|
|
You can only view this page!
| | Page: [Previous] [Next] 1 2 | | (1) Posted by Georgy Evseev [Monday, May 18, 2009 11:08]; edited by Georgy Evseev [09-05-21] | Difficulties with points estimation in solving competitions
This is initial message. It only _lists_ many practical situations from solving competitions. These situations have been seen many times (and will occur again in future). Currently, the judge makes the decision in each case separately, and different judges make different decisions in similar cases. This often leads to resentment, arguments, protests and conflicts. My idea is first to document such situations and then to find in discussion more or less universally acceptable decisions and in future make them part of rules. I hope to describe the situations themselves in following posts.
This topic only covers questions concerning points for solving (distribution and awarding).
Here is the list. Hopefully, I'll be able to edit it, if something is to be added.
1. Halfpoints, quarterpoints and so on.
2. Ambiguity and miswritings.
3. White play repetition.
4. Change of threat continuation.
5. Multiple threats.
6. Multiple threats (false) or threat error.
7. Zugzwang (threat is shown by solver).
8. False zugzwang (no threat is shown by solver).
9. Branching in study.
10. Study, lengthening of play (as can be intended by author).
11. Study, lengthening of play (unnecessary).
12. Study, the final.
See post 6
| | | (2) Posted by Frank Richter [Monday, May 18, 2009 12:23] |
I remember a special case in a draw study:
A black move gave white a chance to win (difficult to see), but most of the solvers wrote only the draw variation.
| | | (3) Posted by Jacques Rotenberg [Monday, May 18, 2009 12:35] |
As far as I know, there are no negative points for the writing of a bad move.
The results sheet is generally prepared with the distribution of points as decided before by the judges.
These points are given when the expected move is written on the expected place.
About the mistakes in writing, the judges are generally very severe, and all the time the correct move correctly written is not readable, the corresponding points are not given.
| | | (4) Posted by Axel Gilbert [Monday, May 18, 2009 19:03] |
I'll add a case that may occur in three or more-movers :
Wrong key but (some) correct variations, that show partial understanding of the problem's intention.
Always a zero-pointer, I think, but is it really worse than giving the right key and missing thematic variations ?
| | | (5) Posted by Steven Dowd [Monday, May 18, 2009 22:51] |
Axel makes a good point.
I've never formally competed, but watching the quick solve here some time back and based on my experience solving from magazines, I find that the key is often easy to find although I often could not explain the thematics of the problem (especially #2) if my life depended on it. I sometimes wonder if all the "tries" listed on some twoers are only tries if you understand the thematics of the solution, thus the only ones who would be misled by the tries are those specifically looking for thematic intent.
| | | (6) Posted by Georgy Evseev [Tuesday, May 19, 2009 10:29] |
1. Halfpoints, quarterpoints and so on
Current status (and some history).
Many of you should know that initially any problem in competition counted only a single point for the solver. If "at least a half of full solution" was found, then the solver was to receive 0,5 points. The switch to the 5-points system was intended to resolve two difficulties. It allowed more variety in marks (so, the winner is less often determined by time) and also was expected to get rid of fractions in marks.
I've checked Lubomir Siran's book. Yes, for 5 or 6 years after the rules were changed all solvers were receiving the marks in full points only. And after that the fractions stroke back. Almost all of the following competitions (and I am speaking about WCSCs) had at least halfpoints in marks. In 5 or 6 cases we also had quarterpoints and there was at least one case in which thirds (together with halves, so sixths - in practice) were used.
The reason is quite simple: there is nothing said in rules about the fractions of points. So, every director makes his decision as he sees fit. Nothing prevents him from using tenths or even smaller fractions.
My opinion.
As the use of halfpoints has become almost universal, there is no sense in returning to full points. But it should be explicitly stated in the rules that halfpoint is the minimal unit that can be used during evaluation of solution.
2. Ambiguity and miswritings.
Current status.
The following paragraph covers this situation in current rules.
"9.7. If a move is written incorrectly, unclearly or ambiguously, this variation or single solution is regarded as incorrect. If, however, the Director (or the jury, pt 13.5) is absolutely sure that the correct move was intended, this variation or single solution must be regarded as correct."
It is quite ambiguous itself: the second sentence contradicts the first. According to it the judges' decision is final and the first strict provision becomes only recommendation.
There is different approach from judges to this situation. Some are very strict and forgive nothing, others tend to give points to the solver if his record can be interpreted correctly. Sometimes even some self-made "rules" are used, like for example: if the judge thinks that there is a miswriting then he checks if the move written is really available. If yes - then no points for solver. If no - then this considered easily correctable writing mistake and solver receives points for this variation.
My opinion.
I was always a supporter of strict rules. But there is a difference between miswriting (a move is written incorrectly) and ambiguity (the record does not distinguish which of two rooks or knights (rarely also pawns, queens or bishops) make the move). The former is mistake and the latter is only inaccuracy and may be sometimes pardoned.
Trying to conserve the wording used, I suggest the following form.
"If a move is written incorrectly or unclearly, this variation or single solution is regarded as incorrect. If a move is written ambiguously, this variation or single solution is regarded as incorrect, if all candidate moves are legal and nothing in the record allows to distinguish between them. Presence of signs "capture", "en passant", "check" can be used for resolving ambiguity, absence of such signs or presence/absence of sign "mate" cannot."
Example.
White: Kd3, Sc3, Se3, Be1;
Black: (Qg3), Ra7c1, Bh6;
#3
Key is written as 1. Sd5 (1. Scd5 intended)
a. If black Qg3 is present, then 1. Sed5 is illegal and ambiguity is resolved.
b. 1... Ra3+ 2. smth - shows that line have been cleared, ambiguity is resolved. If no check is written, ambiguity is not resolved.
c. 1... Rc5 2. smth - shows that line have been cleared, ambiguity is resolved.
d. 1... Bxe3 2. smth - capture shows the presence of piece on e3, ambiguity is resolved. If no capture is written, ambiguity is not resolved.
e. 1... def 2. Bc3 - shows that square have been cleared, ambiguity is resolved.
f. 1... def 2. Sg4 - knight remained on e3, ambiguity is resolved.
g. 1... def 2. Sd1 - ambiguity is not resolved.
(to be continued)
| | | (7) Posted by Gilles Regniers [Tuesday, May 19, 2009 11:36] |
I remember a direct problem in a solving competition with three variations after three different Black promotions on a1.
I wrote, h8:Q (and h8:B, h8:S) in my solution. Clearly, Black cannot promote on the 8th rank, so my solution was considered correct. Is this a wrong decision according to you? Should I have gotten zero points? Because it was clear that I had found the solution of the problem, which is really what a solving competition is about, I think.
| | | (8) Posted by Michael McDowell [Tuesday, May 19, 2009 12:43] |
The director must always have leeway to make common sense decisions in certain cases. One year in the WCSC a S#2 had a key where a rook moved to a square to which the other rook could also move. However, all the black defences came from moves of a piece which was directly unpinned by the key piece, so if anyone wrote a full solution it was perfectly clear which rook they had moved! A large number of solvers did not specify which rook, but the director allowed their solutions to stand (though he said afterwards that he wanted to avoid a riot - as if solvers would behave like that....).
I recall a h#6 where a black queen selfblocked by moving 1.Q(h2)-c2 then 2.Qg6. I accidentally wrote 1.Qb2 then 2.Qg6 and despite the fact that the rest of the solution was correct and my intention obvious I was given zero points on the grounds that 1.Qb2 was a legal move! That I felt was a case where the director was a little harsh.
| | | (9) Posted by Georgy Evseev [Tuesday, May 19, 2009 13:54]; edited by Georgy Evseev [09-05-19] |
@Gilles Regniers & Michael McDowell
My idea is to remove dependencies on the director/judge's decision as far as possible, because it is the ground for conflicts. Judge's decisions often seem harsh when they are not in your favour. And when the solver competes in serious tournaments, the judge has the right to expect him avoiding silly mistakes and to punish him for such mistakes.
So in your examples with miswritings, I think, yes, the variations should not be counted. I'll also never forget my personal example of similar blunder. It was a h#4. I do not remember an exact position (found - P0552394 in PDB or very similar - G.E.), but it was something like this:
White: Kb1, Bc1 Pb2d2 and the rook somethere at the 8th rank
Black: Kf4, Pb3d3d4 and some pieces blocking and able to block flights around the king.
In the solution white sacrifice the rook on c2 (king moves to the corner), while black finish blocking the flights around their king (and, of course capture the rook dxc). The mating move is 4... d3#, and as you can see it is the only move available to white at this stage! But I had somehow written 4... e3#. The result - 0 points for this problem. It was hurtful, but I am no one to blame but myself.
The second example of Michael well fits into my suggestion. If black play with unpinned piece, then it was unpinned. This is enough to resolve ambiguity and the judge would have to make the same decision.
| | | (10) Posted by Neal Turner [Tuesday, May 19, 2009 17:30] |
This is a very interesting thread which I'm sure will attract a lot of attention.
I've been running solving competitions here in Finland for many years and have also assisted quite often in the WCSC and have found that many of the problems cited by Georgy can be avoided.
For Tournament Directors selecting problems for solving, a high priority should be to 'avoid trouble'!
This can be done by avoiding choosing problems which are likely to cause problems:
- Those where the key move can be written ambiguously or possible ambiguous moves in helpmates. I would never choose the example that Georgy gives.
- Those with virtual threats, multiple threats, or other duals. The problems should be as 'clean' as possible, which unfortunately excludes many excellent examples and makes the search for suitable candidates that much harder.
- Choose problems which ease the marking scheme and work: a #3 waiter with 15 variations is no fun for solvers or markers.
In fact when I'm searching for problems I tend to scan the solutions rather than the diagrams, as it's the quality of the solution which will qualify a problem for inclusion.
Studies of course are the most difficult to handle.
The first thing I ask myself is: Can I understand the moves in the solution?. Very often it's not at all obvious why some (especially black) move is being made. I'm not a very good solver so I reckon that if I can understand a move then so will the solvers - which is good! If there's even one move I don't understand I don't choose it.
Solvers are told that only the 'main line' is required. This means that those with branching solutions, however good, must be excluded. Also the main line must be 'strong' - something 'nice' must be happening pretty early so the solver knows he is on the right track. Analytical/theoretical studies are quite unsuitable.
I also avoid studies in which it's black who's making the clever moves. This is because I believe that points should only be awarded for white's moves, and also because these types are very difficult to solve.
Domination studies pose a particular problem in where to end the marking. I used to think the last mark should be given for actually capturing the piece, but after much hassle I've given up on that and would now give full marks for the move which traps or forks the piece.
In studies where sequences are repeated by the author just to demonstrate the draw, I think it's reasonable if the repeated sequences are ignored (or just given in parentheses).
It's no secret that the biggest problem in solving competitions is the solvers!
When they're not making silly writing errors, they're taking short-cuts and generally trying show how clever they are at giving headaches to the judge.
I tend to be lenient with regards to writing errors, usually plain common sense will show the way.
In general I ask myself: Has this guy solved the problem? If the answer is 'yes' then I would give him the points.
However I can understand if someone asks: Why should the careless solver get the same reward as the careful one? The answer is of course that he shouldn't! The remedy I propose, which is not possible under the current rules, is to give him the points for solving the problem then knock off say a half-point for his carelessness. In full knowledge that this would lead to long queues outside the judge's room when the results went up - did I say avoid trouble!?
Ok, maybe I've written too much. But I would be interested to hear the opinions of other solving judges - and anybody else!
| | | (11) Posted by Steven Dowd [Tuesday, May 19, 2009 17:46] |
A simple question:
I know there are such things as mathematics solving competitions. Whether or not you agree with Hardy's proclaimation that chess problems are simply applied mathematics, there is a similarity (every stereotype has a grain of truth) and surely math solving competitions would have similar problems.
Has anyone explored how this is handled? I know we have several mathematicians here and as problemists who must have competed in such things and maybe even supervise them.
It seems a reasonable thing to look at to determine our own rules. I wonder if they have stricter guidelines on what is acceptable and what is not, more standardization (maybe a form?), and so on.
And I am not a judge, have not even ever had the chance to enter a solving competition (except the practical (OTB play) ones that used to be in every chess magazine before computers), so my opinion should be taken with the grain of proverbial salt.
| | | (12) Posted by Georgy Evseev [Wednesday, May 20, 2009 10:53] |
@Neal Turner
Thanks for your great post. I agree with you that a good work of Director can resolve many difficulties before they have arisen. But, unfortunately, there are almost no provision in rules, concerning Director's work before the tournament. There are only two specific paragraphs:
3.3. The Director is responsible for the selection of the problems to be solved. He makes the diagram copies for the tournament. It is his job to ensure that the solvers do not know the problems.
9.4. The Director must determine the distribution of points for a solution (i.e. for different variations, moves or single solutions) before the tournament starts. For a helpmate(s) with more single solutions the distribution of points should be indicated on the solving sheet.
As everyone can see, there are no limitations or even suggestions for points distribution. For example, in a moremover with many variations Director is allowed by rules to give all 5 points for the key only. Of course, this is silly, but still allowed. I think, some guidelines should be provided for Directors, but it should be theme of another topic. I have some thoughts (like "official solution", considered like a pattern for comparison).
Another good idea is the idea of fixed penalty for some mistakes or errors. I was thinking along the similar lines. I would agree that in 90% of ambiguity cases the solver really intends the correct move and so something like "0,5 points penalty per error per variation unless stated otherwise by Director beforehand" seems adequate. I am going to suggest similar penalty in some other cases.
(There was an interesting non-standard case of ambiguity at Rhodes WCSC in helpmate twomover - see http://www.milanvel.net/mp/snapshot/rescbody.php?px=1238581031&fid=xshowm&tid=245). 1. dxe - ambiguous!, should be written 1. dxe5. There was no discussion in Rhodes concerning this, so I consider this ambiguity was ignored by judges - as queen moves later and so it was not captured.)
I'll discuss study difficulties later, but you have correctly pointed at some of them.
@Steven Dowd
Thank you very much, your idea seemed interesting, so I made a fast Google search. It looks like in mathematics competitions everything is much simpler, as every task has a simple answer (number or expression) which can be shown in some standard form. They do not have such things as partial solution or a solution with some errors in it. Still, I'll not say there is nothing to find in mathematics competitions rules.
@Everyone & Administrator
I still hope to write comments on all cases I have listed in my first post. I can continue adding separate posts with small fragments or add everything to initial post, so that main contents will be together (in this case I'll also move the first fragment into it). Which way is better?
| | | (13) Posted by Hauke Reddmann [Wednesday, May 20, 2009 15:24] |
@Georgy:
The statement "They do not have such things as partial solution or a solution with some errors in it" concerning math competitions is is not necessarily correct.
I follow the yearly Putnam Exam in Am. Math Monthly, and they do have, I assure you.
Hauke
| | | (14) Posted by Georgy Evseev [Wednesday, May 20, 2009 16:21] |
@Hauke Reddmann
Thanks. Do your know, how the work of participants is evaluated at Putnam Exam? Anyway, I have found the data (but not the rules for evaluation) for several last years. It is quite interesting: any task can give up to 10 points to a participant and nobody had ever received mark in the range 3-7. That is, there are two groups: correct or almost correct solutions and incorrect solutions (maybe with some good ideas). This does not count as partial solutions.
| | | (15) Posted by Steven Dowd [Wednesday, May 20, 2009 20:39] |
Georgy, there are also other forms of academic competitions (physics being one)that might bear looking at.
I also think the services of a good psychometrician - the people who design standardized tests - might be useful. One thing you learn in education is how to ask the question correctly and to place the answering mechanism in a format which should elicit the correct answer only from those who really know the answer,(and distract those who don't to the incorrect answer) and psychometricians are specialists in this. I saw a psychometrician with no experience in an area pass an old standardized test for certification simply because he knew what the question was looking for by power of deduction from the verbage used.
I am not saying hire a consultant, there may well even be a person with chess experience who would do this willingly for no charge - and then of course there are students, who may be looking for a master's thesis project that could provide us with recommendations.
Again, these are just ideas, I claim no special expertise in solving competitions. I have enough trouble solving my own problems..... :) And they may be impractical or unusable suggestions, but suggestions they are.
| | | (16) Posted by Hauke Reddmann [Sunday, May 24, 2009 14:20] |
Putnam: I have no idea (only a summation is given in the
journal) but if you ever wrote a math test in your childhood -
probably like that :-)
E.g. I used to get loads of half points for knowing all the
formulae but not being able to add right :-)
Hauke
| | | (17) Posted by Georgy Evseev [Monday, May 25, 2009 09:59] |
Sorry for delay with my next post, I simply do not want to suggest something unfinished.
For now, let me throw the following question into discussion.
In a threemover there is a black knight on b1 (the only black knight on the board). Any move of it defends against the existing threat. There is additional defensive effect in move 1...Sd2, which requires another answer from white.
So, in director's solution these variations are shown in the following way:
1...S~ 2. A (1 point)
1...Sd2! 2. B (1 point)
Two solvers have missed this subtlety and produced the following records:
First
1...S~ 2. A (no mention of 1...Sd2 anywhere)
Second
1...Sa3/Sc3/Sd2 2. A
The key is correct in both cases. How many points should be awarded them for these variations?
The answer is obvious, until one tries to motivate his decision...
| | | (18) Posted by Jacques Rotenberg [Monday, May 25, 2009 12:15]; edited by Jacques Rotenberg [09-05-25] |
The answer is evident and motivated : both first and second : 1 point.
Because one variation is correctly written.
The second variation or not written, or written false, brings 0
| | | (19) Posted by Miodrag Mladenović [Monday, May 25, 2009 12:32] |
@Jackques (18)
I completely agree with Jackques. According to the rules it’s 1 point in both cases. However I think that rules are wrong. According to the rules someone can write all first moves in twomover and get 5 points for sure. Theoretically someone could write all possible combination of first and second moves in #3 and according to the rules 5 points should be given. In my opinion there should be a penalty points for incorrectly written variations. For example, -0.5 for each incorrect variation. Of course no one should be given less than zero points per problem. However penalty points would prevent solvers to write all possible combinations of moves.
If I remember good at one of the former Yugoslav national championship we had a case where solver wrote all possible moves in #2. And I think that he was given 5 points for solution. Personally I would never do something like that myself but I think rules should be improved to prevent situations like this one.
| | | (20) Posted by Jacques Rotenberg [Monday, May 25, 2009 12:53]; edited by Jacques Rotenberg [09-05-25] |
@ Miodrag
To introduce penalties or negative point is very .... negative !
In fact most difficult to deal with.
An easy solution for the case given :
When two moves are given at a same place (for example the solver writes 1.A and also 1.B), the judge has to read it as the wrong one, and not to give the point.
This is not a penalty : the right move was not written, instead of it, was written a choice of moves.
And this is not only formal : in fact the solver wrote himself that he did not solve the problem !
So the rule, on this point at least, remains excellent!
| |
Read more... | Page: [Previous] [Next] 1 2
MatPlus.Net Forum Competitions Difficulties with points estimation in solving competitions |
|
|
|
ISC 2024
