With polygamy, the odds are not as bad as that, but eventually the species will still disappear.
However, the dialogue in "Elogium" never establishes flat out that the Ocampa would only be fertile once in their lives. The closest it comes to that is the phrase quoted above, and as said, it can be interpreted as meaning that the Ocampa females can only start
producing children at this specific time; nothing goes against them continuing it thereafter.
I agree that the Caretaker probably did tamper with Ocampa family planning policy, perhaps even promoting a policy leading to extinction. But the elogium thing could work out fine as is nevertheless.