They're not too loose, they seem to float just fine, and the pads stay nice and close to the rotor. The wheel bearing nuts were a little loose which lessened the noise after reassembly but didn't get all the noise down.
I've noticed that the top of the caliper where the floating part touches the housing is making the noise. I'm thinking maybe the wedge on the bottom with the metal strip may not be tight enough. Weird problem.