It's similar to how the US has bases in places like Okinawa and Diego Garcia. In theory the Japanese and UK governments could tell them to leave. In practise they couldn't force the US military out of their bases.
If it's just a matter of might, sure, the Bajorans could not independently dislodge the federation from the base. On the other hand, states have diplomatically have the right to refuse to launch missions from their territory, even when a foreign country operates that base. As the senior Bajoran official at DS9, Kira would be the one to have immediate say over what missions are launched from DS9, regardless of who would command the Defiant.