I think that's the wrong impression. Jackson does what he thinks is necessary to tell the story. He does not go "Oh, I need to fill 3 hours with random crap." He goes "Oh, I have so much random crap, I need more than 2 hours to tell all of it." He always said he wanted to include the backstory that isn't part of the Hobbit novel, but part of the Lord of the Rings universe/mythology. And that needs more time.
The story of the Hobbit itself could be told in 2-3 hours in one film.