^So could you compensate by using a bigger mold to cast the silver bullets?
Yup. You'd only have to calculate the shrinkage and create a mold the correct amount larger. That only leaves the question about how hard the silver is in terms of fitting into the barrel. Lead (and copper jackets) are soft enough that the bullet squeezes itself into the rifling and seals the barrel, so all the expanding gas goes into pushing it. If a bullet is too hard with no give, it could jam in the barrel and cause the barrel to rupture.