Having had this sort of experience before, I had resigned myself to it being a nasty non-native invasive, since it was both pretty and alive, a sure sign of a plant that is Up To No Good. (Cynic? Me? Never!)
I had sort of filed it mentally as a "go look at some point," but got distracted. Then, while looking on an unrelated gardening site, I tripped over a thread asking to ID the very same plant. To my delight, it's actually a Carolina native--the American strawberry-bush!
It's listed as attracting wild turkeys, wood thrushes, eastern bluebirds, yellow rumped warblers, and northern mockingbirds...and, of course, deer. While I'm not exactly sanguine about that last, it's still a delightful thing to have in the yard, and I delighted to have it. (Dude. If wild turkeys showed up in the backyard...well, it's a good thing Kevin knows CPR, because I might expire of pure glee.)
Pictures, and more info about the strawberry-bush, or "hearts-a-bursting" can be found here!