...
To streamline the review process and to prevent overloading reviewers since 0.9.1 the tiebreaker concept was added. When a user is a tiebreaker for a paper they will only be asked to review this when the other reviewers cannot agree. This means that the list of papers a reviewer should review is shorter (but can become longer over time when there are more tiebreaks required).
When a tiebreak becomes necessary the tiebreaker automatically receives an email asking their review for the particular paper. You can configure this behaviour by changing the value of application.ini/core.observer.tiebreaker to 0.
Whether or not a tiebreak is required is decided with the following algorithm.
| Gliffy Diagram | ||||||||
|---|---|---|---|---|---|---|---|---|
|
LOD = Level Of Difference and is calculated with the formula below, and has a value between zero and one. LOD is higher if two people of equal experience disagree than if two people of disparate experience disagree. For fixed values of experience, LOD increases as the disagreement in ratings increases.
...