Did you find this near an old house site or historical area? If so it could be pre-CW and a relic of the conflict.
Horstmann also produced buttons for Southern states prior to the Civil War, so there's a good chance this could have been a battlefield pickup by a Union soldier as a war souvenir, and then lost after returning home. Relic hunters in my area will find an occasion Confederate item in the middle of a Union camp, and are generally considered to be battlefield souvenirs lost by the Union soldier while encamped. Here's an excellent reference link to a site by one of the first relic hunters in my area who now has his collection, personally dug and purchased, in a museum. http://relicman.com/
This site shows several variations of "Horstmann". Suggest you check out his collection as well as email him your pics for confirmation. I've used him on more than one occasion for clarification of relics. BTW, if anyone is ever in Winchester, VA I highly recommend stopping by to check out the museum. You'll be impressed with the displays of relics on display. Mr. Ridgeway and his father were some of the first relic hunters in this area and were fortunate to dig on several virgin sites before the popularity grew. I had the privilege of meeting both gentlemen years ago and was allowed to view their huge dug collection prior to creation of the museum; the stories they told were amazing of the things that used to be commonly found but now are getting scarce (10+ US plates in one day from a field hospital is just one of the stores that comes to mind). Apologies for the long post and going off-topic.
Congrats on your button and HH.