Tabulated review threads sorted by average score

Discussion in 'Trek Literature' started by Sho, Jan 13, 2012.

  1. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    I have trouble sleeping tonight, so I got back up and banged together a little toy program that goes over a text file full of TrekBBS review thread URLs and computes a HTML file that tabulates them sorted by average score.

    The average score is determined as follows: Each vote is given a weight from 1 for "Poor" to 5 for "Outstanding". Then the sum of all votes is divided by the number of votes and rounded to two places.

    Here's the output: http://www.eikehein.com/stuff/trekbbs/review_threads.html

    This is static, i.e. it'll be outdated as soon as any new vote comes in past the time I generated the page (see below regarding future plans).

    Here's the source code, under GPL v3: http://paste.kde.org/186488/03186132/

    And the URL file used as input, put together manually via a search for "review thread": http://paste.kde.org/185954/64256221/

    It's written in Python 3 and requires the lxml library. I ran it on Linux, but it should run equally well on Windows and other operating systems, with an invocation à la "python3 threads.py threads.txt > threads.html". Beware that this was written hurriedly while suffering sleep deprivation, so it may eat babies or attract Borg cubes.

    Obviously there are tons of things that could be improved, starting with the scoring algorithm: The Children of Kings comes out on top with a single "Outstanding" vote at present (edit: the output has since been regenerated to reflect additional votes), because the number of voters is not taken into account (luckily this is the only thread with a sample size that's too small to be useful). It would also be nice to output a little fancier HTML, with sortable table columns. And the review thread discovery could be automated, obviating the need to prepare an input file.

    At least it does already take care to reject threads that don't follow the "standard poll format", so the data should be reliable :).

    Stay tuned, I'll probably revisit this and make it a proper, live website at some point - or, of course, you can grab the source and beat me to it if you want!

    Thanks to Thrawn who inspired me to do this with this post.
     
    Last edited: Jan 14, 2012
  2. DorkBoy [TM]

    DorkBoy [TM] Captain Captain

    Joined:
    Sep 28, 2001
    This is very cool.

    Kinda reminds me of the Spoiler-free Opinion Summary site, which is a site for grading episodes of Trek that I've always loved. If I remember right, it started off as grading threads on the usenet trek boards, back in the early 90s, and eventually he set up a website and automated it.

    I've often wished there was an SOS for Trek books. Your script could become that. :)
     
  3. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Thanks :). Yeah, I was thinking, given the ever-popular "So which ones did you love the most?" thread format, it would be fun to actually pull together what hard, numeric data we have about the forum participants' preferences.

    Edit: I really like the little vote distribution bar graphs on SOS, another one for the todo :).
     
    Last edited: Jan 13, 2012
  4. Thrawn

    Thrawn Rear Admiral Rear Admiral

    Joined:
    Jun 15, 2008
    Location:
    Washington, DC
    Hah! This is awesome. Always knew my anal-retentiveness in making these review threads would pay off someday! :)
     
  5. Stevil2001

    Stevil2001 Vice Admiral Admiral

    Joined:
    Dec 7, 2001
    Location:
    2010
    I used to run a website called "The Star Trek Novel Rankings," actually, which attempted to do that very sort of thing! It is long gone, sadly.

    I'm glad to see my love of The Children of Kings validated. ;)
     
  6. Stevil2001

    Stevil2001 Vice Admiral Admiral

    Joined:
    Dec 7, 2001
    Location:
    2010
  7. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    That's nifty, maybe if I do wind up taking it in this direction I could seed it with your dataset? :)

    My original ambition was just to track the voting already going on here, though, as a service to the forum. I think being able to compare just the threads here has its own value / fun factor socially, so I definitely want to retain having a site just for that :). I hope to find some time to make a first auto-updating and more respectably looking+handling version later this weekend or next week. Keep making those review threads!

    My worry regarding a more general voting site is that it would necessarily double as a book directory, and thus duplicate that part of existing sites. Perhaps it would be more sensible to plug a rating component into a place like Memory Beta. OTOH, maybe rating and associated charting / report generating features are big enough to stand alone and make the redundancy a minor offense - the greater UI focus that affords would be a big plus I should think.

    Oh well, high-flying plans and so little time :). I'll focus on the TrekBBS version for the moment, but keep reusability in mind for later.

    Re The Children of Kings, best way to fix that is for everyone to head over there now and vote :p.

    Edit: I've taken a few moments to add a datestamp below the table, fix a problem with polls lacking any votes turned up by the new The Rings of Time poll, fix a typo in one of the column headers and reflect a second vote for Children. Output and source code link updated.
     
    Last edited: Jan 14, 2012
  8. Therin of Andor

    Therin of Andor Admiral Admiral

    Joined:
    Jun 30, 2004
    Location:
    New Therin Park, Andor (via Australia)
    Yeah, I miss it, too. Always good when you see your quirky faves are others' faves.
     
  9. GHS

    GHS Lieutenant Commander Red Shirt

    Joined:
    Sep 9, 2001
    Location:
    Washington
    Wow, this just became my favorite thread ever!

    Not only do we get Sho's cool collection of all the recent book polls, but we get another peek at Mollmann's site (and, to a Wayback Machine novice, it seems extremely reluctant to divulge more than a peek).

    Mollmann's site was the greatest thing in the history of mankind while it was around. As one who paid for many a good - as well as many a bad - Trek book, it was fun to see the consensus on the comparative level of talent possesed by the authors (which - in the case of the ones who proclaimed themselves experts of their craft - was doubly amusing). The site didn't get updated that often, though (was it updated shortly before its demise, Steve, or did it die off with a ton of votes never getting entered)?

    Kind of ironic now to see Steve himself now getting smacked around in Sho's ratings (actually, he comes out in the middle of the pack, but somewhere Pamela Sargent must be taking perverse pleasure, nonetheless)
     
    Last edited: Jan 15, 2012
  10. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Well, another way to look at it it is that with 2.68 being the lowest score and the range going from 1-5, people seem to generally be relatively happy about their TrekLit :).

    Though it's interesting to note that many of the poll options may be relative rather than absolute, i.e. "Above Average" might not satisfy the question "Is this a good book?" but only "Is this a good TrekLit book?" depending on the reference frame people adopt while voting. So technically you probably can't really derive a general level of satisfaction with TrekLit, except that "Poor" is, well, poor in an absolute sense, so general dissatisfaction would be visible. Then again, it's probably moot - people who generally dislike TrekLit are unlikely to vote here. So TrekLit-relative poll options make good sense.
     
  11. Kertrats47

    Kertrats47 Commodore Commodore

    Joined:
    May 3, 2010
    Location:
    Alberta, Canada
    I kind of like GoodRead's 5 star system.

    One star = I didn't like it.
    Two stars = It was OK.
    Three stars = I liked it.
    Four stars = I really liked it.
    Five stars = It was amazing.

    But that's neither here nor there, I suppose. I like the system here as well, and it wouldn't really do to change the ranking mid-stream.
     
  12. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Yeah, we're kind of stuck with it now if we want to be able to compare existing and future threads fairly. But I think it's ok anyway :).

    Settling on a rating system for the possible general voting site is an open question, though.

    Reminds me of something I read recently: http://blog.steepster.com/post/226679106/better-rating-system
     
  13. Stevil2001

    Stevil2001 Vice Admiral Admiral

    Joined:
    Dec 7, 2001
    Location:
    2010
    To be honest, I'd very nearly forgotten I'd ever ran before this thread appeared. It looks like some of the series will display, and some will not. Who knows. The domain was not mine, but a friend's-- and I lost touch with him many years ago now.

    Well, thanks. If only more people had been so enthusiastic, eh? I don't think it ever got the critical mass of votes it needed to be really useful, but I suspect had I stuck with it, that may have come someday. Oh well.

    I updated it monthly for a while, if I remember right, but then began to lose interest. I don't know if there were tons of unentered votes, but I doubt it.

    The irony did not go unnoticed!
     
  14. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Alrighty, I finally found some time to make this a little nicer! Let's go over the changes:


    • The location has changed: http://www.eikehein.com/stuff/trekbbs/review_threads/ (I've set up a redirect from the old file since I can edit the original post no longer).
    • Probably the most important bit is that the listing will now be automatically updated twice daily to pick up new threads and votes.
    • I've made it look a little nicer, although the colors could still use some tweaking.
    • It's now possible to sort by every column, in ascending or descending order, by clicking on the table headers.
    • Threads with less than four votes are no longer included in the main listing, but rather listed separately.
    • I've added an intro / explanation blurb.
    • I've changed the license of the source code to AGPL v3 and set up a proper repo and viewer: http://www.eikehein.com/repositories/view/index.php?p=trekbbs-pollranking&a=summary (the old warning still applies, though: this is relatively hurriedly written, scrounging scraps of time from other activities).
    Please note that while this page is still very simple, it does make use of some relatively recent web technology, including WOFF fonts / @font-face and CSS border radius. Do tell me if it looks bad on your end (here's how it's supposed to look: screenshot) so I can try to improve things, but upgrading your browser might also be in order :).

    So ... you like?

    Edit: Didn't have time yet to add little vote distribution graphs, but that's coming too!
     
    Last edited: Jan 17, 2012
  15. Stevil2001

    Stevil2001 Vice Admiral Admiral

    Joined:
    Dec 7, 2001
    Location:
    2010
    It looks quite nice!
     
  16. Defcon

    Defcon Rear Admiral Rear Admiral

    Joined:
    May 9, 2003
    Location:
    Germany
    I like it, and as someone who always dutifully sent his rankings to Steve for his site it's nice to see something similar returning. :)

    I wonder if it would be feasible to have "Classic review threads", so that over time more and more of the backlog would be included in the list, too. Obviously not every book at once, but maybe one thread per series every week or something like that.
     
  17. JD

    JD Admiral Admiral

    Joined:
    Jul 22, 2004
    Location:
    Arizona, USA
    Very nice. I think I'll be visiting that fairly regularly.
     
  18. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Thanks guys :).

    On the topic of adding more threads to it, there's a few notes to make, and bullet number two needs your input:


    • The data mining component of this currently filters the thread list for those having "review thread" in their titles before loading them up to look for poll data, to avoid having to load all of them (which would be a lot, especially on the first run to assemble the initial thread cache: it did look at all 140 pages worth of thread list in that case - on subsequent runs it only asks for a day's worth of activities of course, which is the shortest queryable timeframe). If you know of any threads featuring standard polls missed by this, please tell me and I'll add them!

    • There are various review threads that don't have a poll. Grafting polls onto them is possible, but inconvenient: According to Rosalind she can't add polls to existing threads directly, so to add one to the new The Rings of Time thread I had to open a second, poll-equipped thread, which she then merged into the older one. Now I'm willing to do this work for other threads, but it means a new reply (due to the merge) pulling them back to the front, which might be annoying, especially if done all at once. Maybe one per week? How do you all feel about this? And the mods would need to support it too, of course.

    • I also like Defcon's idea of occasionally opening new threads for old books, perhaps as a sort of community re-read initiative inbetween new releases.
     
  19. Rosalind

    Rosalind TrekLit's Dr Rose Mod Admiral

    Joined:
    Mar 30, 2005
    Location:
    Sydney, Australia
    <mod hat>
    I don't mind, as long as it's not done all at once
    </mod hat>

    <research scientist hat>
    if you add polls to old review threads now:
    -- you won't have the same people who posted in those threads voting, so you don't get a good representation of the reviews.
    -- what people vote a year (for example) after they've read a book would statistically be different from what they would vote straight after they've read a book. so the ratings would mean different things.

    so, I would discourage adding polls to old threads, but rather have new polls for them, and have the ratings in a different table.
    </research scientist hat>
     
  20. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Hmm. Those are good points. OTOH, my page is already liable to alter the dynamic of older polls: If I keep making posts while having the link to it in my signature, new people are invariably going to find the polls and vote in them. So we're going to see poll participation by non-posters anyway, and might as well extend that to "new old" threads, no? :)

    Of course this does raise the issue whether my page is a problem because it interrupts the synergy between thread posts and poll votes. But I'd say lets not be afraid of a little change :). Plus there never was a mechanic that required posting to poll - and I know that I, as a relatively new arrival, intended to vote in the older polls once I've read those books even prior to making the page.

    Plus, there's something that might offset the problem specifically related to poll grafts: Thread subscriptions. The folks who posted in those threads would get notified of the new reply yielded by the merge, I think, and might thus vote after all. Or we could add a second reply specifically to provoke that.

    If all else fails, I think the forum population is relatively stable, so the intersection between folks watching the traffic back then and watching the traffic now is pretty large. If they see those threads pulled back up and check out the new post, they're going to vote. Especially if we include something in the posts to encourage them, pointing out this consideration.

    So all things considered I'd prefer the poll graft option myself :).
     
    Last edited: Jan 18, 2012