The distinctive bark of a young Kentucky coffeetree.
The branches look dead in winter, bare of twigs, the buds hidden away. The genus name translates as “naked branch.”
The high top of this older male tree looked amazingly shrubby.
A nearby female was festooned with seed pods.
The bark of a mature specimen.
Of trees and their memories: there’s a lecture on dendrochronology on Sunday afternoon at the Torrey Botanical Society meeting. Which reminds me that I wrote a short piece for JSTOR on dendrochronology’s origins.
If you go to the lecture, keep an ear out for Ravens! A pair was working on a nest last week on the very library building the lecture hall is in, but since then they seem to have transferred their allegiance to a nest across the road at Fordham U.
Backyard and Beyond 
I haven’t found any of those in my Ozark forest, though they are native to the region. (Note: I haven’t examined every tree on the 80 acres.)
This from a U of KY site is interesting: “Although widely distributed, this tree is a rare forest tree and occurs in scattered populations.” http://www.uky.edu/hort/Kentucky-Coffeetree
I wonder why this is? Some claim that there are no more megafauna to eat the thick, heavy pods and thus transport the seeds. Course it’s been 10,000 years….
I’d read somewhere that the trees are still “recolonizing” North America after being pushed out during the last Ice Age. As you said, though, they’ve had a lot of time to get the job done. Maybe that’s part of the explanation though.