How does the Joystick Pad works?

[andreas.balzer]

Hello,
I'm a software engineer and working on a way to select and move objects in a 2D environment on a Touchscreen with very high accuracy.

In "Star Trek Voyager - Technical Guide Version 1.0" by Rick Sternback and Michael Okuda, page 16 (see 1) I came across the "Joystick Pad" that is used to navigate and possibily launch torpedos/fire phasers on board Voyager and the Enterprise in Star Trek Nemesis (see 2, time stamp 1:01 and following).

See 3 for a redrawn version of the control.

I am wondering how this (fictional) user interface control is supposed to work. I understand the horizontal and vertical scrollbars are used to do x and y translation (see 4, page 38). What functionality do the other buttons serve?

1. What is the logic behind the control?
2. Is there any reason why in the Nemesis scene Worf activates the left side of the control and suddenly the right side changes color to gray?
3. Are there any thoughts behind lcars color coding in the show other than switching to red for a code red?

I would appreciate any response.

Best Regards,
Andreas

1: http://de.scribd.com/doc/17602871/Star-Trek-Voyager-Technical-Manual-40-Pages-1994
2: http://www.youtube.com/watch?v=DHUSpAtemfg
3: http://lcars47databank.wikia.com/wiki/File:LCARS_ProlixusWeapon_Sys.png
4: http://de.scribd.com/doc/17602666/Star-Trek-TNG-Technical-Manual-182-Pages

 

[Timo]

It's probably clear from the start that the designers had no "actual" functionality in mind when drawing these buttons. Even the ones with meaningful labels on them were probably labeled more or less at random, albeit with buzzwords that as such make sense.

However, it would be easy to argue that the crossed-scrollbars control mechanism places certain requirements on a keypad like this. The mechanism completely ties down one hand in a position that is difficult to maintain: if constant steering is required, at least one finger must be precisely positioned in one location and moved as required. Other fingers will find it difficult to achieve anything simultaneously - unless key functions are associated with buttons within easy reach of the crossed-scrollbars section. And that basically means in the four sectors falling in between the bars.

There's no point in having labels on those buttons, since the user's hand will be covering them and preventing him from reading the labels, so for once the use of meaningless strings of numbers in the graphics makes sense! Most probably, the sector buttons would be user-definable shortcuts to functions whose actions are visible outside the area covered by the hand - say, the keys numbered 15 and 78 would "expand" to the cluster of function buttons visible to the upper left, and these in turn could be used to redefine the function of the keys 15 and 78. Any other sector buttons would similarly expand to other function clusters as needed.

The big "FIRE" and "ABORT" buttons seem stupidly placed, as a right-handed person would be somewhat blocking his left hand access to them if simultaneously operating the cross-scrollbars. Perhaps they are mere indicator lights, and the trigger is actually slaved to that tiny square segment of the upper vertical scrollbar?

Anything to the left of the column that is connected by a curve to the "ABORT" square is probably unrelated to weapons firing, as Worf's duties include other stuff as well. The next column to the left has various directories and ship's departments listed, and is probably used to remotely summon these resources (so that Worf himself merely sends requests and gets answers by people or expert programs at the other end, and no fine control interface is required). The next column probably has a target sight on top, status indicators relating thereto next, and a way to reconfigure the entire display at the bottom.

Constant reconfiguring is probably the very point of interfaces like this. The layout is so distressingly non-functional and non-optimal because it represents the current user's current reconfiguring of the necessary elements for the task at hands, with irrelevant elements left as is. A few moments later, Worf might call up a completely different view, or his replacement officer would choose her preferred layout, or a communications interface would pop up to allow Worf to answer an incoming hail.

So,

1) No logic to it other than placing quick-select, reconfigurable buttons in the sectors between the scrollbars, for the needs of the day.
2) Worf has chosen to redefine keys 15 and 78 (his options including AUX BANK, PROBE SELECT and others), so his previous work redefining keys to the upper right (selecting between torpedo warhead type, apparently) is now forgotten and fades to the background.
3) Nope. But row upon row of identical-looking buttons probably needs a scheme of (randomly) alternating colors so that two adjacent keys aren't confused with each other.

Timo Saloniemi

 

[Maurice]

In short: it's scenic dressing intended to imply functionality but not functionally designed.

 

[F. King Daniel]

It always looked like the kind of digital D-pad found on 80's and 90's game console controllers to me.

Which is, of course, an even more ridiculous way to fly a giant spaceship than Riker's joystick from Insurrection!

 

[Mr. Adventure]

Which is, of course, an even more ridiculous way to fly a giant spaceship than Riker's joystick from Insurrection!

Nothing beats that.

 

[Timo]

I rather fancied the grey hemispheres that were seen on the shuttlecraft control panels early in the show. The characters used them as flight controls, placing their hand over a hemisphere much like they'd grip a joystick.

A three-dimensional surface that's still as sensitive as the flat control panels and possibly moves like a trackball would offers a lot of potential for controlling a spacecraft with one hand...

Inset version, "Coming of Age":

http://tng.trekcore.com/hd/albums/1x19/comingofage_hd_213.jpg

Without the covering hand, "Unnatural Selection":

http://tng.trekcore.com/hd/albums/2x07/unnaturalselection_hd_238.jpg

Still there, but Wesley knows how to fly hands off, "Samaritan Snare":

http://tng.trekcore.com/gallery/albums/s2/2x17/samaritansnare025.jpg

Later on, the inset was covered up and an all-flat panel began featuring the classic "cursor keys" graphics.

Timo Saloniemi

 

[QuinnTV]

I rather fancied the grey hemispheres that were seen on the shuttlecraft control panels early in the show. The characters used them as flight controls, placing their hand over a hemisphere much like they'd grip a joystick.

Yes! I liked those and thought it a very interesting idea. I also went for these same HD caps when Trekcore put them up.

 

[Herkimer Jitty]

It always looked like the kind of digital D-pad found on 80's and 90's game console controllers to me.

Which is, of course, an even more ridiculous way to fly a giant spaceship than Riker's joystick from Insurrection!

Yes, it is rather silly. You'd need at least two joysticks to effectively fly a ship. One for rotation, and another for translation.

 

[Timo]

Fundamentally, though, two sticks isn't better than one mainly because sticks are stupid in a 3D environment. They are meant for 2D, say, forward motion where two types of rotation solve all of your problems, or crane handling where two types of translation suffice, because they can only be tilted in two cardinal directions and that's it. Two joysticks only provide two translations and two rotations, leaving one of each wanting.

Twisting would add the third dimension, though. But you can only twist a joystick along one axis without confusing the motion with one of the tilting ones. Just ditch the stick and have a ball - like the TNG shuttles. That one can be rotated every which way, and shoved forward/aft, left/right and up/down, all with one hand, and still leaving the fingers of that hand to do other tasks.

Not that the Trek shuttles would really need six degrees of freedom, considering how often they fly like aeroplanes or sometimes helicopters...

Timo Saloniemi

 

[B.J.]

I use a Space Explorer from 3DConnexion to manipulate 3D models at work. It has a full 6 degrees of freedom, all contained within one puck-shaped controller. (Personally, I liked it better when the products were called Spaceballs! )

 

[Johnny]

Fundamentally, though, two sticks isn't better than one mainly because sticks are stupid in a 3D environment. They are meant for 2D, say, forward motion where two types of rotation solve all of your problems, or crane handling where two types of translation suffice, because they can only be tilted in two cardinal directions and that's it. Two joysticks only provide two translations and two rotations, leaving one of each wanting.

Twisting would add the third dimension, though. But you can only twist a joystick along one axis without confusing the motion with one of the tilting ones. Just ditch the stick and have a ball - like the TNG shuttles. That one can be rotated every which way, and shoved forward/aft, left/right and up/down, all with one hand, and still leaving the fingers of that hand to do other tasks.

Not that the Trek shuttles would really need six degrees of freedom, considering how often they fly like aeroplanes or sometimes helicopters...

Timo Saloniemi

I've never noticed those circular balls on the shuttle controls before. Interesting, but not particularly cinematic. I always preferred the option of a recessed joysticks in the consoles.

Also, why would two joysticks not suffice? The thrust and direction would be separate, no? So, one to control amount of thrust and it's direction and one to control the angle on any axis?

J.

 

[Timo]

A classic joystick can only control two axes, while a spacecraft operates on six, that's the big problem. Having one work as a "multi-throttle" will give you two axes of translation - say, fore-aft and up-down but not left-right. So you will have to cover left-right with the other stick by rolling like an aeroplane and then yawing, and so you can't do left-right at a standstill, only in forward (or aft) motion. Clumsy in tight spots, such as shuttlebays...

For three axes, you need a z-axis-twisting joystick, i.e. something like those balls. But why not add translation there, too, and do away with the second stick? Twist for angle, push/pull for thrust.

Timo Saloniemi

 

[Albertese]

A classic joystick can only control two axes, while a spacecraft operates on six, that's the big problem. Having one work as a "multi-throttle" will give you two axes of translation - say, fore-aft and up-down but not left-right. So you will have to cover left-right with the other stick by rolling like an aeroplane and then yawing, and so you can't do left-right at a standstill, only in forward (or aft) motion. Clumsy in tight spots, such as shuttlebays...

For three axes, you need a z-axis-twisting joystick, i.e. something like those balls. But why not add translation there, too, and do away with the second stick? Twist for angle, push/pull for thrust.

Timo Saloniemi

Six? I count three. One of which is not strictly necessary to get around in in three dimensions. If you can pitch and roll, which are already three dimensional maneuvers in a real sense, you can get along fine. Now, I agree that yaw is a pretty useful function.

I remember back in the day playing the old TIE Fighter combat simulator came on my old PC as a teenager... I think I almost never actually made use of the yaw function, and I could maneuver around just fine! It's only occasionally really needed. Pitch and roll can get you around just fine and I figure the "plus sign" shaped "D-pad" (to use the video game parlance) would be all you'd mostly need to get around. I'm sure a simple "shift key" would be all you'd need to make one of the bars be used for yaw. Or even maybe you touch one of the quadrant areas of the circle and pull that around in circles to indicate a yaw motion. I'm sure that would work just fine.

--Alex

 

[Timo]

Six? I count three. One of which is not strictly necessary to get around in in three dimensions. If you can pitch and roll, which are already three dimensional maneuvers in a real sense, you can get along fine. Now, I agree that yaw is a pretty useful function.

Getting out of the shuttlebay already calls for an additional translation - if you can't move straight up and down, you are going to scrape nasty grooves in the floor of the bay every time you attempt takeoff or landing with a craft lacking wheels!

The same with the helicopter-style landings a shuttle is often asked to perform. Under the pull of gravity, you could slide sideways or back/forth by tilting and applying upward thrust, the way a chopper does it, but that won't work if you want to dock in zero gee - you need full three axes of translation in addition to full control over three angles of rotation.

Timo Saloniemi

 

[Johnny]

Six? I count three. One of which is not strictly necessary to get around in in three dimensions. If you can pitch and roll, which are already three dimensional maneuvers in a real sense, you can get along fine. Now, I agree that yaw is a pretty useful function.

Getting out of the shuttlebay already calls for an additional translation - if you can't move straight up and down, you are going to scrape nasty grooves in the floor of the bay every time you attempt takeoff or landing with a craft lacking wheels!

The same with the helicopter-style landings a shuttle is often asked to perform. Under the pull of gravity, you could slide sideways or back/forth by tilting and applying upward thrust, the way a chopper does it, but that won't work if you want to dock in zero gee - you need full three axes of translation in addition to full control over three angles of rotation.

Timo Saloniemi

Hold on...

Stick 1 (Ship Direction):
L&R - Roll
F&B - Point Down and Up
Twist - Flat Turns

Stick 2 (Thrust Direction)
L&R - Straffing
F&B - Forward and backward thrust.
Twist (or that additional button mentioned earlier, plus F&B) - Flat up and Down movement.

I'm not sure why you'd need any more?

I'll admit for something more precises (and in less pressured and tense situations) it would be worth using the panels, but if you want to knock your ship out of the way real quick, grab the joy-stick!

 

[Timo]

My worry was that normal joysticks don't twist, so not even two of them would do much good. (The additional worry was that aircraft-like motion, easily covered by just a single stick where the throttle could be a thumbwheel or whatever, doesn't work for shuttlecraft. But that's a somewhat different issue.)

OTOH, if a twisting function is to be built in, why not go the full mile and make the movements intuitive? That is, twisting for turns only (pushing forward or left on the pivot counts as twisting, too), and sliding for translations (it's easy to yank a stick laterally fwd/aft or left/right, but a ball is better for up/down yanking). And that could be done with two controllers, or then with just one. Which is why I love the TNG balls, because they look like they could do it all with one hand.

Timo Saloniemi

 

[Johnny]

My worry was that normal joysticks don't twist, so not even two of them would do much good. (The additional worry was that aircraft-like motion, easily covered by just a single stick where the throttle could be a thumbwheel or whatever, doesn't work for shuttlecraft. But that's a somewhat different issue.)

OTOH, if a twisting function is to be built in, why not go the full mile and make the movements intuitive? That is, twisting for turns only (pushing forward or left on the pivot counts as twisting, too), and sliding for translations (it's easy to yank a stick laterally fwd/aft or left/right, but a ball is better for up/down yanking). And that could be done with two controllers, or then with just one. Which is why I love the TNG balls, because they look like they could do it all with one hand.

Timo Saloniemi

I'm not sure what joy-sticks you've had, but even some of the simplest sticks nowadays have a yaw facility, even things like force-feedback are farely common too.

I don't really understand how a ball that can only control three axis, is any better than a joystick that can only control three axis...? Am I missing how a ball can control direction and thrust at the same time?
If it's that thrust is controlled through the console then surely that brings this all down to preference of control between ball and stick?

IMO, the ball doesn't really give you any feel for where you're at, if there's any visual representation on the ball of where you're at then you're going to have to take your eyes off whatever you're trying to manauver around, almost the complete removal of tactile feedback doesn't seem like a wise choice especially if it's sensitive work. (That's why I have a problem with touch screen, but that's another thread!)

I used to have a roller-ball mouse when I younger, the one when the ball was on top of it, and I tell you it was a pain in the bum! All you got was cramp in your thumb from trying to move it too quickly. I got rid of it after a year and I've never gone back.

Also, with a stick you can hang on to it if your chair goes out from under you, you can see what it's doing from a distance, when it points up it's off, and you can hold on when the gravity fails!

 

[B.J.]

As I mentioned upthread, there are devices *today* that have all 6 degrees of freedom built into a one-handed controller (see 3dConnexion). It's quite intuitive, I use one of their products for work every day. Just twist or push/pull harder or softer to control the rate of movement.

However, I was wondering if anyone knew what actual spacecraft, like the Space Shuttle or Soyuz, use for their control systems? I've been able to find pictures of their controls, but nothing that explains how they're used yet.

 

[Timo]

I don't really understand how a ball that can only control three axis, is any better than a joystick that can only control three axis...?

Well, it isn't. But a ball can be more intuitively made to control the spacecraft on the necessary six degrees of freedom, IMHO.

Imagine gripping that ball, or a rock, or a potato. It's a firm grip, and your fingers probably sink into the ball (and may reach "hands on stick" buttons for further functions), but that doesn't make it different from a joystick yet. What does is the next step, where you imagine the ball is in fact the shuttle. (Heck, Type 7 even looks quite a bit like that!)

Push the ball forward without twisting and the shuttle moves forward. Twist the ball forward and the shuttle's nose tilts down. Now, you could build a joystick that does these same things (that is, accepts "push" as a valid command rather than just "twist"), and you could cover the sideways moves the same way. But with a grip of a ball, you can also lift the control so that the shuttle goes straight up, which is not all that comfortable with the vertical shaft of a joystick. Plus you get a bit of extra leverage for twisting around the vertical axis.

The big thing there is that the ball (if soft on the surface) fits all hands. In order to twist or lift a joystick, you have to customize it for your own hand size before you get comfort and leverage.

As for hanging on to the controls, well, that works for the steering wheels of cars or boats, as those only cover one degree of freedom and are immune to stresses in other directions. Hanging on to a joystick or joyball for dear life will always send unwanted control input to your vehicle...

However, I was wondering if anyone knew what actual spacecraft, like the Space Shuttle or Soyuz, use for their control systems? I've been able to find pictures of their controls, but nothing that explains how they're used yet.

Good question. The shuttle controls for seated pilot and commander are obviously classic and trivial - a joystick for wing trailing edge control surfaces (pitch and roll) and pedals for rudder (yaw), and of course there's no throttle because the shuttle is a glider. But supposedly reentry is also initiated when seated, so the joysticks must have a secondary mode for orienting the vehicle with RCS burns before the OMS burn.

Dockings and the like are probably done with the aft console controls. There's a classic joystick there for the manipulator arm (frequently updated), and two sticks that look like automobile shifting sticks, supposedly giving two degrees of freedom each (looking much like the original eighties versions even in latest pics). No doubt there are dozens of modes for using those sticks to control the RCS thrusters in sequence, depending on what is to be achieved - and dozens more from all sorts of software modernizations and bypasses that depend on plugging in a laptop that's a hundred times more powerful than the vehicle's own computers.

But I guess an astronaut has no real control over individual nozzles, save perhaps for fiddling with the fuses.

Timo Saloniemi

 

[Albertese]

Timo, do you have a screencap of this TNG-style ball you are describing? I don't recall any such device that would fit your description, but it's been a while since I actually watched TNG so I'm fully prepared to accept that I just missed something along the way or forgot about it.

Also, I was thinking about the D-pad being used just for rotational movements, I had always assumed that throttle controls, for all three axes of flight, would be handled elsewhere... some of those other buttons they're always tapping on whenever they do something.

--Alex