We have to defer to the person who knows him intimately, even if we wonder about their judgment.
Why? If A says that B is an asshole, this may well be an objective fact if A isn't B's embittered lover, but is highly unlikely to be objectively true in the opposite case. Insults are an extremely unreliable type of evaluation...
Had, say, Brown lived to tell the tale, we might have trusted his judgement a bit better, as passions (in a 1960s TV show) wouldn't run as high between these two males unless we were given hints of a love triangle or some such.
There's a reason why self-evaluation isn't a popular business practice, and why outside consulting services are valued!