I agree.
The Outermost House is a beautiful book. I had known of it for a few years, but just read it last week because my wife and I moved to the Cape in January.
Another interesting book that a friend loaned me is,
These Fragile Outposts. It is "a geological look at Cape Cod, Martha's Vineyard, and Nantucket." It was written in 1964 by Barbara Blau Chamberlain and is out of print, but used copies are not hard to find online--I paid $10 for one because my friend wanted his lovely first edition hard cover back
These Fragile Outposts gives a very readable account of how the Cape and Islands were formed, outlines the different periods of glaciation, and explains where all those rocks and cliffs and dunes came from. And I learned the history behind that ubiquitous Cape Cod blue paint--pretty cool.
Threaded Mode