To be exact, NX in modern Starfleet doesn't quite denote experimental ship as much as it denotes prototypes for ships intended to see series production in the future. Or at least none of the NX-registered designs remained unique, single-ship ones - even some further examples of the
Prometheus can be glimpsed in the fleet of VOY "Endgame". And of course when we get closer looks at series-produced versions of ships that had the NX rego, those versions have NCC registries (save for the second
Defiant), and sometimes even the original prototype converts to NCC (the
Excelsior).
It's a bit like the difference between modern X and Y aircraft designation prefixes in the USAF. Previously, X, Y and Z there denoted early attempts which would be refined into production models later on; nowadays, X is reserved exclusively for weird experiments that have no Air Force applications and will never be series-produced, while Y means prototype. NX in Starfleet is the equivalent of Y (although it is used in connection with the ship's
registry, not with a numerical identifier of the ship
design such as in YF-22 or YB-49 or whatnot).
Actual experimental designs in Trek, such as the soliton waverider or the testbeds for the Warp Five Project, have seldom been given recognizable let alone systematic registries.
Remember the conversation between Forrest and Archer in the inspection pod. Forrest asks if Archer is OK with having the military (i.e. the MACOs) on board, and Archer replies "I don't have a problem with non-Starfleet personnel."
Since that's such an outlier, and since Starfleet personnel does not really
differ from MACO personnel in any respect save the uniform style (both carry heavy personal weaponry for coordinated shipboard and shorebound fighting), I'm eager to see that as mere interservices rivalry: "military" vs. "navy". It's just a bit confusing that "military" today also encompasses "navy" - it did
not do so just a century or so ago, and fighting forces included both militaries and navies.
Though one has to wonder just how useful the hull numbers are on Federation Starships. Why have the NCC as part of the hull number at all?
Supposedly, NCC and NX are not the only prefix letters possible. Onscreen we get NAR a couple of times, for vessels of "semi-Starfleet" status, and then there are plenty of graphics indicating that other letter combinations are in use for various civilian ships.
That there is any paint on Starfleet hulls at all may be purely a matter of tradition, with no practical relevance whatsoever. But Federation records supposedly divide ships according to their letter prefixes, so that there can exist NCC-123, NAR-123 and NSP-123 simultaneously, these being different vessels. In that sense, if Starfleet bothers to paint anything on the hull, it better paint the letters in addition to the numbers.
Timo Saloniemi