After the main structure (where the Bed Bath & Beyond is now) burned, a man by the name of Joseph Hart converted the carriage house (the current location of the museum) into a "country resort." The "day hotel" was popular among New Yorkers who wanted to escape the hustle and bustle of the city, which at that time extended only as far north as 14th Street. In those days, one could take a stagecoach or steamboat up and spend the day at the hotel sipping lemonade in the ladies parlor or drinking and playing cards in the gentlemen’s tavern (see photo of reproduction card tables and bar at right--note the area enclosed by "bars" to secure the liquor and cash).
The hotel is set up with card tables mid-game, a dining table set for lunch, and upper halls with Greek revival furniture lining the walls as they would have been during a party. The garden is small but lovely as well, and if you go up the stairs on the west side, you can see part of the original shale Olde Post Road that connected NYC with Boston.
If you go, expect a very personalized guided tour by a young, very affable, and well-informed docent. It is well worth the $8 admission fee. Also check their website for concerts, free events, and site rental opportunities (Mount Vernon Hotel Museum & Garden). This really is a nice little historic gem in NYC.