It's never really stated how far Bajor or Bajoran territory was from the Federation.
In the early seasons there were several references to the area being considered "the frontier" by Starfleet characters. That seems to indicate that Bajor was fairly far from Federation terrority but close enough to seek membership.
In later seasons, these references were dropped and the plots often had Starfleet and established Federation planets like Earth within a relative short travel distance.
Both could work. Bajor might have been just outside Federation terrority, like Mexico City is to the United States. "The frontier" could then be just an expression for anything outside the Federation.