The reason why it's space-time originates from the fact that in Special Relativity space and time are seen to be inextricably linked through the Minkowski metric with diagonal [-1,1,1,1], which is the signature of flat space-time. The -1 encodes the facts that the invariance of the speed of light and that there are no preferred reference frames result an apparent trade off between the measurements of space and time of a body moving relative to an observer (time stretches and space shrinks).
In General Relativity, the presence of a spherically-symmetric mass in space-time results in other metrics such as the Schwarzchild, Kerr, Kerr-Newman, Reissner–Nordström, and so on, depending on whether you also factor in the rotation and the charge of the body. For the Universe, the Friedmann–Lemaître–Robertson–Walker metric is usally invoked, although its assumptions of homogeneity and spatial isotropy have recently been shown possibly to be incorrect over distances as large as 4Gly. (http://www.space.com/19220-universe-...iscovered.html
You can think of a metric as a mathematical object that encodes the gravitational effect of the body on space-time and thereby on other bodies. Dropping the time component produces predictions for such things as the deflection of EM radiation by a massive body such as the Sun and the precession of the perihelion of Mercury that do not agree with observation. Adding the time component into the mix gives answers that are in good agreement.
The real problem lies in explaining why you can usually only go in one direction along the time axis (apart from rather outlandish solutions of the field equations) but either way along a spatial axis (unless you've fallen inside the event horizon of a black hole).