Twisted Milnor torsion for finite group actions