The truth is, without Hogan and his crew, Schultz would likely have long been dead. He was a sweet, gentle man who couldn't truly harm a fly, and he was more likely to overlook mistakes than punish for them.
Perhaps Klink also knew this, and thus insisted on keeping Schultz around so as to insulate him from the full horrors of Nazism? (And with the expectation that, in return, Klink could work for Schultz's toy factory after the war.) Kind of like a protective parent or big brother would.
I'm also curious as to how Klink functioned in the active duty military. Isn't he a veteran of World War I?