It's the most obvious choice, but you can't argue that it has got to be the worst: being assimilated by the Borg and forced to kill thousands of your fellow Starfleet colleagues. Remember, when asked if he remembered anything right after being disconnected from the Borg, Picard says "everything." So, he has all of those horrible memories where he had Borg vessels destroying starships and killing thousands, with his will completely subdued. It was probably the most nightmarish case of forced schizophrenia one could possible experience.
In "First Contact", it is clearly shown how much that experience scarred him. And in various points during the TNG series, he is reminded of what he did as a Borg. How much can you rationalize it? That feeling of utter hopelessness and how easily he could have remained a Borg for the rest of his life.