I mean holo-wife could not do the telepathy deal either.
But okay, how about if when he goes through Pon Farr he obviously has NO idea what is going on, goes psychotic and kills a bunch of elves. That would be what leads to his banishment, deep into the program.