Is it Equinox
, by Dennis Muren and stop-motion animators Jim Danforth & Dave Allen? I believe it originally was a student film and later expanded, as was Dark Star
(can't get link to Wikipedia to work)
I seem to remember another animated monster film called Vortex
, with "Dallas" actor Jim Davis, but can't find anything resembling it. Hulu used to have it. It's from around that same time I think, but they moved it to Hulu Plus last I looked.