Tabulated review threads sorted by average score

Discussion in 'Trek Literature' started by Sho, Jan 13, 2012.

  1. Thrawn

    Thrawn Rear Admiral Rear Admiral

    Joined:
    Jun 15, 2008
    Location:
    Washington, DC
    Yeah, I think poll grafting probably doesn't make any sense. You won't get many people voting, for one. I think it might be better just to skip those.
     
  2. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    The page might change that by making old threads a lot more visible, though :). Like the four new votes The Children of Kings got ... though admittedly there was also this thread behind that.
     
  3. Kertrats47

    Kertrats47 Commodore Commodore

    Joined:
    May 3, 2010
    Location:
    Alberta, Canada
    I kind of like that idea, too... maybe a "book club" style posting, a past Trek classic that can be voted on every week. Only one per week is a good idea, though, to limit the clutter. As someone who is slowly making his way through the Trek catalog and blogging the attempt, I approve!

    Yeah, I have to admit I went back and rated The Children of Kings because of this thread. :)
     
  4. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Alrighty, as promised earlier I went and added little vote distribution graphs to the "Voters" column, to better visualize what exactly goes into a score :).

    For example, a book in the middle of the pack might be there because voters predominantly picked "Average", or it might be a polarizing specimen that garnered both "Outstanding" and "Poor" votes. Whichever is the case is apparent at a glance now. The Children of Kings is a good example of the latter at this time.

    Edit: Forgot to mention: On the topic of the poll grafts, I did a little more brainstorming with Rosalind and think that it should actually be possible to add polls to existing threads without the cumbersome merge maneuver after all. However, we might still want to reply to them after adding a poll to encourage votes by previous posters, to offset the effect Rosalind mentioned, and thus still requiring throttling to once per week. That said, I'm holding off on this for now and waiting for more feedback first.
     
    Last edited: Jan 18, 2012
  5. Defcon

    Defcon Rear Admiral Rear Admiral

    Joined:
    May 9, 2003
    Location:
    Germany
  6. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Whoa. Thanks for pointing this out, I had indeed no idea.

    I've taken a quick peek at XenForo's own installation of XenForo to determine how hard it will be to adapt the data miner: http://xenforo.com/community/

    First, let's take a quick look at how the data miner currently operates - you can probably skip this if you don't care for technobabble, but then what are you doing in a Star Trek forum? :) Anyway, here we go:

    1. It visits the forum's thread list page, using variables in the URL it loads to convey its desired view options, which are to sort the threads by their creation time from new to old, and to limit the listing to a specific timeframe. On the first run that timeframe was actually the limitless "since the beginning", and on subsequent runs it asks for only the last 24 hours, which is the lowest queryable timeframe and sufficient to catch new threads when running twice daily.
    2. With the thread list page loaded up, it walks over it (while following pagination links if there are any, i.e. loading further pages as needed) and notes down the URLs of any threads which contain a configurable trigger phrase, currently "review thread" (the character string search is done case-insensitively).
    3. Then it reads in the contents of its thread cache, a file containing the URLs of threads gathered in previous runs.
    4. Having this set of previously gathered thread URLs from the cache and a set of any new thread URLs gathered from the thread list page, it builds both the union of the two and the difference of them, i.e. sets containing all now-known thread URLs and of thread URLs not yet written to the cache. The latter is then added to the cache to serve the next run, and the former is used in the next step.
    5. For each URL in the just-computed set of all known thread URLs it visits them, mining any poll data from the thread page.
    6. Then it loops over the assembled thread data again, to eliminate those threads which do not follow the expected poll format, to separate them into two bins for threads meeting the minimum vote threshold and those which do not, to sort them (which the new, fancier page can actually do client-side using JavaScript as the ability to dynamically resort by clicking the column headers demonstrates, but just in case you don't have JavaScript enabled in your browser the static HTML served up to it still presorts the table rows by average score) and to generate the report page.
    In effect, this gives it the following properties:

    • It picks up on new threads and their votes.
    • It picks up on new votes in old threads.
    • It picks up on new polls in old threads (i.e. it's already ready for grafts).
    Now, back to XenForo. Looks like it allows me:

    • ... to list threads ordered by thread creation time by loading the right URL. However, it doesn't appear to allow me to limit the timeframe of the query. That means I'll have to adjust the thread gathering logic slightly to make it "walk over threads, possibly following pagination links, until hitting a thread that's older than the time of the last run" (or some fixed cut-off time delta vs. the current time). Fortunately it includes the creation dates right in the listing, so that's easy to compute. There's a bit of a curveball in the form of sticky threads with old dates leading the list, but I can easily skip over those of course.
    • ... to view poll results without needing to be logged in. It includes nearly the same data in its display of them as vBulletin does: Poll option texts, votes per option - what's missing is the number of total votes, but that's of course easily built by adding together the per-option counts.
    Bottom line: The thread gatherer will need a bit of work, and I'll have to adapt the parsing code to slightly different HTML, but it shouldn't take me longer than writing this post did ;).

    So, no reason to hold off on poll grafts. A more pressing question is whether XenForo allows for adding/grafting polls, which I haven't looked into yet.

    Edit: After not finding a direct way to add a poll to a test thread I made in their forum. I found this question-and-answer thread: http://xenforo.com/community/threads/add-poll-to-an-posted-thread.21206/

    So, grafting is possible in XenForo, but only using the cumbersome merge maneuver, which we may actually be able to avoid in vBulletin after all. Dunno if it would be a good idea to add all the polls in one swoop while still on vBulletin, and then just throttle doing the "hey, did you see the new poll" replies (if we want to do those at all, etc etc). Then again I guess we already had come to terms with the merge requirement earlier.

    Yay! :)
     
    Last edited: Jan 18, 2012
  7. CaptainDonovin

    CaptainDonovin Fleet Captain Fleet Captain

    Joined:
    Jul 8, 2006
    Location:
    Labrador City. woof
    I like it, interresting to see the vote totals per book thread.

    Guess now I'll have to actually remember to vote in these things now.
     
    Last edited: Jan 19, 2012
  8. Therin of Andor

    Therin of Andor Admiral Admiral

    Joined:
    Jun 30, 2004
    Location:
    New Therin Park, Andor (via Australia)
    How weird that it still only has five votes. Did no one else but five of us have an opinion on this novel?
     
    Last edited: Jan 21, 2012
  9. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    One of those I haven't read yet *lowers gaze in shame*.
     
  10. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    Please excuse this double post, but a quick question: When I saw Stefan open that thread about Double Helix' Quarantine just now, I was sorely tempted to encourage him to use his powers as thread creator to add a poll, since it appears to be the first thread about specifically just that volume. However, I don't want to outstay my welcome and annoy everybody with this poll business in random threads until they're sick of it. Go or no go, thoughts?
     
  11. Skywalker

    Skywalker Admiral Admiral

    Joined:
    Feb 24, 2005
    If we're going to eventually do "classic review threads" for past novels, I don't think there's any need to really pester Stefan about it right now, since we'll get to that book eventually. Assuming the classic reviews would go in order of release date, that is.
     
  12. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    *nod*, makes sense.
     
  13. Skywalker

    Skywalker Admiral Admiral

    Joined:
    Feb 24, 2005
    That does bring up another question, though. If we're going to do classic review threads, how far back should we go? If we go way back, then at one book a week, it could take a very long time indeed for us to get through them all. Maybe it would be better to start with the beginning of the DS9 Relaunch/post-NEM era and work our way forward from there.
     
  14. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    I agree that completing the backlog of more recent titles first would probably be more useful to more readers, also in terms of the ranking table. Many are just now trying to catch up with the new continuities, myself included.

    But that might be boring to those all caught up, so maybe we should alternate?
     
  15. Thrawn

    Thrawn Rear Admiral Rear Admiral

    Joined:
    Jun 15, 2008
    Location:
    Washington, DC
    No, I like the idea of starting in, oh, 2000 or 2001 or around there, when the DS9 relaunch was getting going. Maybe choose a few earlier novels to do here and there, like Lives Of Dax, Vendetta, and the early New Frontiers.
     
  16. Defcon

    Defcon Rear Admiral Rear Admiral

    Joined:
    May 9, 2003
    Location:
    Germany
    Yeah, I guess it makes sense to start with the various relaunches and probably Vanguard and New Frontier thrown in for good measure, as those are probably the books most people have read and which are still (relatively) fresh in people's memories.
     
  17. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    I'm about to start Harbinger, could open the thread after :).
     
  18. Therin of Andor

    Therin of Andor Admiral Admiral

    Joined:
    Jun 30, 2004
    Location:
    New Therin Park, Andor (via Australia)
    So have still only five people actually read "Children of Kings" and bothered to vote?
     
  19. Sho

    Sho Fleet Captain Fleet Captain

    Joined:
    Sep 8, 2006
    Location:
    Berlin, Germany
    "All systems operating within normal parameters" - really just five so far :).

    Maybe it really was a low seller, considering there was just one vote at the start of this ...
     
  20. Therin of Andor

    Therin of Andor Admiral Admiral

    Joined:
    Jun 30, 2004
    Location:
    New Therin Park, Andor (via Australia)
    I was a little behind getting to "Children of Kings", but I thought I recalled the usual number of people discussing the book when it came out.

    Yep. Five pages of discussion, but no poll on that thread:
    http://www.trekbbs.com/showthread.php?t=119686