Romantic and not too touristy:
-
High Line, a disused el line converted to a park gives a unique, even for NYC, juxtaposition of the urban and nature.
-Walk the bridges. I'd recommend Brooklyn (though it can be a little touristy) or Williamsburg bridge. The Williamsburg bridge is less touristy, though with less history, but has great views of Manhattan and Brooklyn and some really wonderful graffiti along the way.
-Want to see the Statue of Liberty? Skip the expensive tourist boat and take the Staten Island Ferry. You won't have to pay, and you get really close.
-Explore the boroughs. Try the Park Slope neighborhood of Brooklyn for awesome eats and shopping. Other great shopping can be found in the East Village and Alphabet City, I'd recommend wandering the side streets between 3rd Ave and Avenue B, 6th through 11th streets have great boutiques and unique finds.
-Eats? For the real Village Experience try the Yaffa Cafe (St Marks btwn 1st and A). The decor looks like a drag queen exploded inside and all the waiters and waitresses are sexy and rude. Another great place for restaurants is the West village south of Washington Sq. Park. You can find all the best hole in the wall cuisine. And speaking of pizza (speaking of food in NY is always speaking of pizza), people like to say Ray's is New York's best. They're lying. Any number of pizzerias in Manhattan, Brooklyn, and Queens will get you a better slice. Avoid Little Italy for food. The restaurants there aren't that great and it's not worth all the tourists. Black Betty's is one of my fave brooklyn spots for food, amazing hummus!
-Parties. If you're looking for a place to dance, again, don't forget the other boroughs. You'll find a lot of cheaper, cooler, more easy going venues in Queens and Brooklyn.
-Nerd? Check out Forbidden Planet, a great comic book store on 13th and Broadway, conveniently for us nerdy types it is directly across from the Strand, the second best bookstore after Powell's in Portland, Oregon.
-Arty? There is just as much incredible art on the streets of NYC as in the museums. Check around neighborhoods of Brooklyn and Queens to see the vibrant street art of New York.
Finally, the best way to experience NYC is just walk it. Anywhere you go you'll be close to food, drugstores, anything you need, so just get up and go!