On Duflo's conjecture concerning branching problems