Families of abelian varieties with many isogenous fibres