Topic: Suggestions for changes to the Awards system

As I mentioned in the 2012 Awards thread, I receive feedback from both Ash and Robert on how to potentially change the Awards system for 2013.

In a nutshell, this is what was discussed:

Once nominations are concluded, we will hold a private vote with all community members invited - this will be similar to how elections are run. Likely we would have a ranking system, so that you would rank those who are nominated in order from who you most think deserves the award to least deserving. A weighted point system is then applied to give us a final tally.

Based on the results of that vote, we will pick up to 3 candidates for a "final" vote. For instance, if two people had 30% of the vote, and everyone else was lower than 10%, we'd likely only take those two for final consideration. If there were 8 people all within a few percentage points, we'd take the top 3 of those 8. I'd likely also reserve the right to extend this to 4 in cases where the voting was very very close among the 3rd and 4th candidates.

After we have these "finalists", there were two ideas presented:
1. We have another community-wide vote, with only the finalists on the ballot.
Pros: The community has the final say in who receives the awards, and no time is spent in deliberation at all.
Cons: Those who weren't "finalists" will see that they were not finalists. Voting results could potentially be based on popularity, not on the award descriptions. The community would have to take time to vote twice.

2. A committee of 3-5 is formed from those people who are NOT finalists, and potentially from an outside source (external friends of Phoenix who are not members of the community). This avoids potential bias, as the past 2 years we've had committee members that are also nominated for awards - somewhat unavoidable when 3/4 of the community is nominated! They will deliberate on who is most deserving of those awards and make the final decision. This could potentially also involve multiple committees if the finalists for various categories are vastly different.
Pros: No one knows if they were a finalist or not, thus avoiding potentially hurt feelings. Committee gives a better chance the award is based on merits related to the award, and not general popularity (I believe it limits it, since the committee must describe WHY they are choosing a particular person).
Cons: Community doesn't get "final" final say. Time must be spent in deliberation (which I personally believe is a Pro).

Thoughts or comments?

One other idea I have been thinking about, to potentially limit mass nominations (*cough* Ash *cough*), is to only allow each member a single nomination in each category, or perhaps 2 or 3. Thoughts on that?

-Jason

http://www.phoenix-rp.com/img/pips/4.png http://oi60.tinypic.com/5otabo.jpg

Thumbs up +1 Thumbs down

Re: Suggestions for changes to the Awards system

It's worth mentioning that I intend to keep the Friend of Phoenix award as one per year, but would offer special awards in the event of a retirement of a highly-respected player or (God forbid) the death of one of our members.

Re: Suggestions for changes to the Awards system

I don't believe there's any problem with people being nominated and finding out they didn't make the final round. I don't know what levels of excitement or anxiety other people experience about our annual awards, but with two rounds of voting I think that in the first round people won't be nearly as excited or anxious, and that the second round is when levels start to rise because those few made it as far as they did.

Plus, in a way, such a system builds in its own "special mentions" because the people voted into the first round but were not voted for the annual award have definitely received a significant level of acknowledgement of a job well-done. So we can actually dish out some congratulations to people who passed the first round and but didn't win the 2nd round and the perks of deliberation lost, most notably "special recognition", has a fair replacement I feel.

It may lose some of the special touch that some kind words and specific mention of noteworthy things but with enough people deciding you are deserving of the final round that should amount to something.

Actually something we could do with two rounds of voting is allow each voter to make specific comments as to why they believe that person deserves to be in the 2nd round of voting and then the top comments from those can be used to create a message of special recognition for each of the finalists (I'd probably recommend making these special messages lead up to the announcement of the actual winner), sort of like how Silent lead up to the Friend of Phoenix award.

~Robert

http://i41.tinypic.com/2q2e0ig.png

Thumbs up +1 Thumbs down

Re: Suggestions for changes to the Awards system

I think having a committee is good, as it removed the "popularity contest" element which, whilst important, is ultimately not the best way to go about things. A public vote on nominees to guide the Award Officer in choosing their committee, and then to give said committee a starting point, seems the best plan to me. I'd also suggest that, whilst those chosen to be the Award Officer are doubtless going to be excellent individuals, perhaps taking that position should be considered a disqualification from site-level awards for that year, just to avoid the potential of allegations of corruption. Better still, have a previous winner take the post.

Re: Suggestions for changes to the Awards system

Agreed. The best process, in my opinion is:

1. (Mass!) Nominations. I say "mass" a little tongue in cheek because I know I do tend to nominate widely, but I do think it right to give praise where it's due and to recognise with the encouragement of a nomination that a member brings something noteworthy to a sim or the site more broadly.

2. A member-wide vote on those nominations to streamline the nominees to finalists and to determine who can be on the committee. The finalists would be announced along with "special mentions" as appropriate, partly simply for information and partly to build excitement as the final awards approach. Possibly finalists could be invited to the committee, but would have to recuse themselves from winning any award that year.

3. The Awarding Committee deliberate on the remaining finalists and a spokesperson announces the winner, with a lead up of special mentions about the runners up.

Ash

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Silent Hunter wrote:

It's worth mentioning that I intend to keep the Friend of Phoenix award as one per year, but would offer special awards in the event of a retirement of a highly-respected player or (God forbid) the death of one of our members.

Nice idea Silent.

Deborah

Thumbs up +1 Thumbs down

Re: Suggestions for changes to the Awards system

Why not a nomination into a instant runoff selection?

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Could you explain what you mean exactly?

http://i41.tinypic.com/2q2e0ig.png

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Robert, are you asking Osprey for further clarification? I wasn't sure.  tongue

http://www.phoenix-rp.com/img/pips/4.png http://oi60.tinypic.com/5otabo.jpg

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Sorry about all the double-posting, my mouse is broken and is treating a single click as a double click. I didn't realize I was doing it until a while after I had been posting double posts.  sad

~Robert

http://i41.tinypic.com/2q2e0ig.png

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

RLongtin wrote:

Could you explain what you mean exactly?

Sure. You have each voter to select their first, second, and third preference. You can set it up where a nominee must get a 50%+1 agreement in order for someone to be selected as the winner.  With every pass you can remove those that don't receive a minimum of votes. Eventually a winner will be found. If someone's pick survives their preference carries, but if their pick is dropped, their vote changes to their second, then third pick.

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

That's the same thing as "elimination" which I suggested at one point someplace regarding our award system (I thought I suggested it here but now I'm finding I don't know where I mentioned it, probably in private e-mail with Jason and Ash when we were discussing the 2012 awards) except that you're crossing it with Ranked-Pairs.

I'd say that notion has some merit, though I might like to do some testing to figure out how it would work out numerically.  smile

Also were you suggesting that the awards committee are the ones who do the voting, or did you mean for it to be up to the general membership?

~Robert

http://i41.tinypic.com/2q2e0ig.png

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Ah, I did mention it here http://www.phoenix-rp.com/viewtopic.php … 928#p45928 but I wasn't crossing it with Ranked-Pairs, simply comparing it to Ranked-Pairs.

~Robert

http://i41.tinypic.com/2q2e0ig.png

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

General Membership. It would create a better polling of opinions. After all, the fans select for the MLB All-Star Game, so why not for our awards?

Otherwise I would say anyone voting on who receives a award should be a still-active member with a minimum of 1 year of membership here.

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Fair point on the voters needing to have been here for a year. I think going for ranked-pairs elimination might be overkill but we'll have a more refined list...as mentioned, I'll need to play with the numbers and see what that actually looks like, how many rounds of voting will be necessary for "n" candidates, etc.

~Robert

http://i41.tinypic.com/2q2e0ig.png

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Well, the original reasons I didn't leave it up to the general membership were 2 fold:
1. It's one more thing the EVC has to take on to get people out to vote. And it generally comes *before* the elections and thus may make people exasperated.
2. A reasonable chance exists that it becomes a popularity contest, rather than a serious discussion on the merits of each individual (or sim).

As for who votes, I was considering for next year making the committee much larger than 3 (perhaps 6 or 7), to make it easier for those nominated in categories to abstain.

One thing we could potentially do is make it like the Oscars, so only certain people vote, but I don't like that idea as I feel it could become clique-y and elitist. But maybe others do like that idea or have better ideas off of that?

-Jason

http://www.phoenix-rp.com/img/pips/4.png http://oi60.tinypic.com/5otabo.jpg

Thumbs up +1 Thumbs down

Re: Suggestions for changes to the Awards system

This line of thinking would bring the concept to paralysis. Why do we even have yearly awards for that matter, if it ends up being a point of consternation for those that organize it?

Leave it to me to bring up the buzzkill card, huh?

Last edited by Osprey (2013-07-26 01:41:47)

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

Because it was an idea that carried over from the old country, and people generally liked the idea. It's the same reason we have Sim Awards - people feel good when their efforts are recognized, and healthy competition raises the bar for all of us. Also the reason why we note the "most active" sim each month.

http://www.phoenix-rp.com/img/pips/4.png http://oi60.tinypic.com/5otabo.jpg

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

A "popularity contest" is implying that someone will win based on how much they are liked by others in general as opposed to the desired outcome which is to have the winner be someone whose talents specific to the award they were nominated for are praised above all others for the concerned year.

So the issue is that we're worried people will utilize information outside of this to evaluate a person's performance? When I was going to school to be a teacher, this same sort of issue is raised when it comes to grading and the answer always came down to creating a good rubric and at times using tricks like not looking at the name.

This is how we remove bias in the teaching profession anyhow but no reason it can't work here.  smile

~Robert

http://i41.tinypic.com/2q2e0ig.png

Thumbs up Thumbs down

Re: Suggestions for changes to the Awards system

An award rubric. I like that idea!

http://www.phoenix-rp.com/img/pips/4.png http://oi60.tinypic.com/5otabo.jpg

Thumbs up Thumbs down