Mapping class groups of highly connected manifolds, II