FWIW, the dialogue establishes the Ligonians as "closely humanoid", meaning they are less human than, say, Zephram Cochrane of Alpha Centauri. Which sort of rules out both the "known Earth colony" and "lost Earth colony or abducted Earthlings" theories.
Also, Picard seems to think that the culture of the Ligonians closely approximates Song China. Song is credited with introducing things like paper money, gunpowder weaponry and a standing navy, the last of which also gave the dynasty remarkable exploration capacities. It is also the dynasty during with meritocracy reached its peak in China. Doesn't really sound much like the Ligonians - they are farther along in their technology, they don't yet seem to be exploring space with or without a standing navy, and the sort-of-matrilinear economy doesn't ring a bell, either.