The ultimate episode ranking thread

[Isis]

...Note that, being a top-10 thread, it gives an advantage to love-it-or-hate-it episodes (which appear on some top-10 lists) over episodes that are generally considered average (and therefore appear on few if any top-10 lists). Is this the kind of thing you meant by “off”?...

OK, captrek, here's another way to express my point, which I would say at this point is a minor one. Of episodes that do not appear in a rater's ranking, the more the rater likes the episode, the shorter shrift it gets in the poll. To give an example, say I list my top ten episodes in a "Top Ten" thread. Say "Errand of Mercy" is my #11 and "And the Children Shall Lead" is my #79, but because these are not "Top Ten" episodes for me, they don't appear in the ranking I provided. Say you incorporate those results in this thread. I'm saying the "Errand of Mercy" gets short shrift. It's my #11, but in your poll, it's handled the same way as my #79. Does this shed any light on things?

...Note also that the lower half of the graph remains tangled.

Although this can be taken with a "grain of salt" because we're still not talking about tons of people, there seem to be some consistencies in people's preferences. In particular, the first seven in your latest diagram have always come out pretty high. (This isn't meant to be a scientific statement, just an overall general "take" on things.)

 

[CorporalCaptain]

It makes sense to me, Isis.

But the conclusion one gets from the point is that in order to get an accurate ballot, the voter contributing it must answer 79 question.

If you don't provide a number for every episode, the ballot will suffer from this problem.

 

[Captrek]

...Note that, being a top-10 thread, it gives an advantage to love-it-or-hate-it episodes (which appear on some top-10 lists) over episodes that are generally considered average (and therefore appear on few if any top-10 lists). Is this the kind of thing you meant by “off”?...

OK, captrek, here's another way to express my point, which I would say at this point is a minor one. Of episodes that do not appear in a rater's ranking, the more the rater likes the episode, the shorter shrift it gets in the poll. To give an example, say I list my top ten episodes in a "Top Ten" thread. Say "Errand of Mercy" is my #11 and "And the Children Shall Lead" is my #79, but because these are not "Top Ten" episodes for me, they don't appear in the ranking I provided. Say you incorporate those results in this thread. I'm saying the "Errand of Mercy" gets short shrift. It's my #11, but in your poll, it's handled the same way as my #79. Does this shed any light on things?

Yes and no.

The poll summarizes expressed preferences, not actual preferences. (Unfortunately, we don’t have Kor’s mind sifter.) If your ballot only lists your top 10, then your favorites of the remainder get “short shrift” in the same sense that your favorites get short shrift if you don’t cast a ballot at all.

The Schulze Method handles this incomplete information pretty well. Your ballot in this example doesn’t say that EOM and ATCSL are “the same,” and doesn’t push EOM down and ATCSL up toward mediocre. Your ballot simply expresses no opinion at all about EOM vs. ATCSL.

This “ultimate” ranking has failed to live up to that title. It is based on very incomplete information. Dozens of posters have contributed no information about their preferences beyond their top 10, and millions of Star Trek fans have contributed no information at all about their preferences. The graph is a synthesis of the information that is available on these ballots, and shouldn’t be represented or read as anything more than that.

 

[Isis]

It makes sense to me, Isis.

But the conclusion one gets from the point is that in order to get an accurate ballot, the voter contributing it must answer 79 question.

If you don't provide a number for every episode, the ballot will suffer from this problem.

Corporal, I would just make one change to what you said; I'd change the beginning of your second sentence to read, "But the conclusion one gets from the point is that in order to get the most accurate ballot..." Otherwise, I absolutely agree with you.

 

[CorporalCaptain]

Quite.

 

[Isis]

...Your ballot simply expresses no opinion at all about EOM vs. ATCSL...

True, captrek, but in my example, I have an opinion on EOM vs. ATCSL. What I was saying is that the ballot doesn't capture that preference. However, as you said in an earlier post:

...Using arbitrarily limited information about the preferences of some posters is...less deficient than using no information about the preferences of those same posters...

I absolutely agree. The more information provided, the more reflective the results will be of actual episode preferences. I absolutely agree that using preferences from Top Ten threads is the way to go for those people who didn't respond directly to this thread.

...This “ultimate” ranking has failed to live up to that title. It is based on very incomplete information...

As the saying goes, "it is what it is." However, I think a few of us got something out of it (I certainly did), so captrek, your thread was definitely a worthwhile contribution to this board.

 

[CorporalCaptain]

captrek, your thread was definitely a worthwhile contribution to this board.

It certainly was!

 

[Captrek]

Thanks for the positive comments, and the interest and support you two have shown throughout the thread.

I just wish we had been able to attract wider participation. If we do this again in the future, is there anything we can do differently to get more participation?

 

[CorporalCaptain]

You're welcome, captrek, it was my pleasure actually.

I don't know why we're the only cool people on the whole boa--Nah, just kidding.

I'm still a little wet behind the ears on the BBS, but I have noticed something. Those rank-your-top-10-episodes threads are always being made. It was fun for me at first, once, but I'm done with those.

So, point #1 is that I wonder just how many people actually even looked into this thread. They may have just assumed it was one of those.

And, point #2 is, for those who did, were they predominantly lazy noobs? If they were, I bet they said, "screw this," after glazing over on the second sentence of the OP, and just went back to tossing peanuts elsewhere. God forbid something that isn't a cookie cutter of everything elsewhere on the web, to them, in my estimation.

Me, I was intrigued by the thread title enough to check it out, was actually interested in the problem enough to read the OP, and had my grey cells stimulated enough to participate.

So, after all that, in answer to your question, I don't know.