As to the engineering spaces...you could probably put those on either deck, although a shipwright might be inclined to have as short a distance as possible for travel between the engine "room" and the cockpit/bridge.
I was thinking more along the lines that it would make sense for the lounge (which is presumably also the mess area) to be adjacent to the crew quarters.
And oh, speaking of Serenity
, I realize there'd have to be a medical bay too. Not sure where that would go. Maybe close to the labs and engine compartment in case of accidents. And next to the transporter stage, in case someone gets hurt on a dig and needs to be beamed up and tended to quickly. (A small civilian ship might not have the fancy "Beam directly to sickbay" setup that a Starfleet vessel would have.)
(Need to allow for reactant tankage, storage for crew-consumables, spaces for RCS thrusters, and so forth as well...and you said this baby's expected to be away from the core systems - and anywhere else that's considered "known" - for extended stretches. Up to a year or two? I have got to re-read Buried Age myself. Soon.)
The expedition takes just under one year, round trip.