Proposal for Ranking Guidelines

EPIC Rating System Review

Last revised on: August 22, 2006

Last revised by: Eric von der Heyden

Summary: The EPIC Council has proposed an improvement to the plugin rating system. The goals and the proposed action items are outlined below.

The proposed process for soliciting feedback and making these changes is as follows: Feedback will be solicited initially from a subset of Eclipse Community through the Phoenix mailing list and the Phoenix bug tracker. Feedback will also be solicited from the Eclipse Community as a whole through a yet-to-be-determined process. Once the feedback has been gathered and the course of action decided by the Council, the changes will be entered into the bug tracker.

Goals for improving the rating system:

• Decrease fraudulent rating
• Increase transparency
• Increase the perceived credibility of the rating system
• Increase participation.

Proposed action items as of August 1:

A working group from the Council made the following suggestions for initial improvements to the rating system:

1. Stop proxy voting. Proxy votes would cease to count towards total but the occurrence of an attempt would be visible on the plug-in rating page.

2. Publish information on how the rating system works and how it protects against fraud.

3. Require an email address to rank – different from a full registration.

4. Remove “outside” rating thus removing a possibility of a rating to default to 10. Instead provide and promote the option for web sites to link to a plugins rating page on EPIC

Other courses of action:

1. Require registration to vote. This would replace 1-3. The working group decided not to propose this action as a first step as it might overly reduce participation. However, this is still a viable alternative.

2. Suggestions were made by Eric Clayberg at Instantiations. Here are several to consider:

• Increase the minimum number of votes to a higher fixed number of 50 or a number relative to the plugin with the most votes (such as 1-2% of 4000)
• Restrict the amount of low votes from a single IP address within a certain time frame.
• Increase the number of top-rated plugins displayed to more than the current 10.
• Weighted voting – a registered vote would count for more.
• Ability to respond to voter’s comments