If I can see it on Google Earth, your holes are too deep or you are using a GPX!
But seriously, this is just about the worst thing detectorists can do because just one bad press release can trigger a cascade effect for the rest of us all around the world. I'm in Australia and unfortunately, many private leaseholders (who were once fine with Detector use on their property) have begun to reject requests to detect on their property. Forgetting to fill in a hole may result in injury to livestock etc and it's such a shame that just a few bad apples have caused so much trouble for the rest of us.
I saw a guy dig holes up the side of a hill with his detector and he wasn't filling them in. So my Prospecting partner walked up to one of the holes and whooped and hollered and pretended to find something amazing in one hole before moving onto the next and going through the same routine again. The hole-digger wandered over to see what my parter had found and he brought out his lucky nugget that he sometimes carries. You should have seen the anger on this guy's face when it dawned on him that he might have ignored a 4 ounce nugget. He kept trying to re-balance his detector and, just as he walked over the crest of the hill, my friend would whoop and carry on again at another of the digger's holes. The digger half-jokingly suggested that we "go dig our own holes" and this was the trigger to my partner who let fly with a tirade of verbal beatdown on the digger for not filling in his holes. Now bear in mind that the shallow holes we dig are absolutely nothing compared to the craters and holes that the Wombats leave behind. But a great example of how long a hole can remain is easily observed near old diggings. 160 years of erosion has left the holes by the early prospectors pretty much the same way as when they were first abandoned.
The hole-digger eventually got annoyed that his detector wasn't working properly and left. We filled in his holes and left a handful of lead shot in one - along a handwritten note on plastic with a waterproof marker to remind the jerk when he returned that he should fill in his holes in future or face our wrath.
We have shovels: I'm tempted to bury one of these useless morons someday and leave them there.