I admit this is a subjective interpretation of the series on my part, but I think the sterility of the Matrix goes beyond a simple green filter (and it's not as though the audience would have been confused about which scenes took place where without said filter). Look at how little plant life we see in the Matrix, and how grimy so much of the city is. Consider that, in the big POV shot of Neo flying to the highway battle, we see pretty much nothing but factories, industrial areas, and the financial district downtown in the distance. Look at the Woman in the Red Dress construct program, which fools Neo into thinking is the actual Matrix he's spent his whole life in, and consider how little clothing and color variation there is, even by the standards of a business-centric downtown. One can argue that despite all these glaring stylistic flourishes, the Matrix is meant to be just like our real world, and just because we don't happen to see the Matrix Hawaii or Mardi Gras or what have you doesn't mean it's not there, or that it doesn't happen. AFAIK, there's nothing in the movies themselves that invalidates either interpretation. (Whether one wants to consider elements from The Matrix Online or The Animatrix is a secondary question.) Personally, I prefer to think that the "present day" of the Matrix program is only a pale, almost entirely urban shadow of the real world. In any case, however, Edit_XYZ's point about the potential for instant execution and/or possible memory deletion strikes me as solid.