Extremal hyperbolic manifolds and the Selberg trace formula