You have to think about what going on even when you arent hearing anything. There is always a base line tone (threshold). When you get a target that base line is increased. Depending on the circuitry, you can have a large increase, a tiny increase, or it can try to make all the increases large. If you turn the threshold up really high, then its going to take a much stronger signal increase to "break through" the background and be heard. If you turn the threshold down too much, you can have weak signal be completely missed because they didn't raise it enough to become audible. That can also change based on detector, settings, and especially your own hearing acuity). This is why traditionally, people are always instructed to set a threshold to "barely audible" but of course, in our current age of digital processing, its unclear just how much the old standard still applies.
Personally, I like to still keep a barely audible threshold. I can hear it drop out over iron rejects without having the constant loud tone of all-metal.