Hard disagree and I say that as a huge fan of The Dark Knight.
Batman's rogue gallery is extensive enough that there's no reason a run of films can't create engaging stories without the Joker appearing once.
You're thinking as a comicbook fan. Most movie-watching people know Batman from the movies and the cartoons, NOT the comics. So yeah, Joker is as much part of Batman for the average viewer as the Baterang and Batmobile is. Rogue's Gallery means two things for them; f*%& and all. So yeah, when a new Batman movie is made, they'll think 'Joker when?'.
Do I agree with you? Yes. Does the average movie watcher do? Nope.