It's the luck of the draw, mostly. Even if you hunt bigger, more urban homes, where your likelihood of finding silver seems greater (and may in fact BE greater), you still might get nothing.
I've hunted some rural sites out in the middle of nowhere where no trace of a house remained, and gotten silver. Sometimes OLD silver. Then again, I've struck out many times, too. Such is the nature of these places. You never know- some of these places are truly remote and it is hard to imagine anything ever being there. Usually only one or two good finds result, if I get anything at all.
Even if you do everything right, there is still gonna be that element of plain old luck, working for you or against you.
Remember, Lady Luck favors the persistent. But not necessarily the stubborn. There's a difference between "persistent" and "stubborn". If you are consistently getting nothing, then try, try again, by all means. But learn from your mistakes, too. Maybe you didn't even make any mistakes, but constantly thinking and strategizing and fine-tuning your hunting technique will help you anyway. The persistent man doesn't let some bad luck discourage him, while the stubborn man hunts the same old nonproducing sites again and again. Both are better than the pessimistic man who throws in the towel and quits, though. A persistent man who learns has the greatest odds of success. A stubborn man too can sometimes make a good find or two, by being stubborn, but not as much as if he added a little thinking to go with his bullheadedness. The quitter gets nothing. I would rather be persistent than stubborn, and rather be stubborn than a quitter. (I've worn all three hats in my time, though).
Ask yourself where you would be lying if you were an old silver coin. Go to places where such coins were obviously carried, and spent, and lost. And try, try again. Even on the best of sites you may not be successful at first. But then again, sometimes those apparently stingy sites can conceal one or two good coins (sometimes REALLY good coins), and it takes a stubborn (or better yet, persistent) soul to find them.
The law of averages says you'll get the goodies eventually, if you go to the right places, often enough. And if you're smart enough to figure out where the right places are, and keep hitting them, Lady Luck will smile upon you... eventually.
She likes to take her sweet time about it, though.