twice to Nancy Mygatt, for four years in the '70s to actress Audra Lindley, and since 2001 to actress Noreen Nash. He was the father of three children, including actor-director James Whitmore Jr.