There are tons of models to choose from, but having been involved with Geocaching many of the needed qualities are similar.
A. Waterproof or water resistant.
B. Good battery life with a back light that can be turned ON/OFF. Does it use standard batteries?
C. Excellent sensitivity under the tree canopy.
D. Depending on your eyesight, is the display clear & large enough for you to see?
E. Does it come with Topo maps, if not can they be purchased? If so how much extra cost?
F. Does it come with computer software that will let you upload & download maps?
G. For Treasure Hunting a touchscreen is out of the question, sand and dirt will scratch it. You need buttons, and if you are going to be wearing gloves check out how big the buttons are and their placement.
H. Does it look like it can be dropped and survive? Because at some point you will drop it!
I. Look at an online manual to see how many button presses to mark a location. Also look to see how text is entered if you are going to label the locations with a long description.
J. Warranty?
As far as accuracy, remember that each time you start the unit you want it to lock on to as many sats as possible before beginning to walk with it. About 20 minutes would be ideal but not always practical. Therefore I get out of my vehicle, turn the GPS on, set it on the roof of my car while I get the rest of my gear together. The clearer the view of the sky the better! If you locate a site you want to mark as accurately as possible, find a clear view of the sky, and set the unit down to let it soak for 20 minutes, then mark the location. 10ft/3m is about as close as is normally expected out of the better handhelds. With surveying units, several kilobucks, you can do better as they can use some fixed earth sites to further triangulate your position and correct satellite signal jitter.
These are just a few considerations. I would look at the Geocaching web sites to see what people think of the latest crop of units as this is the primary tool of that hobby.
HH
BarnacleBill