Kirk Douglas stars as Larry Coe, a gifted architect who, unhappily married, falls in love with his beautiful neighbor Maggie (Kim Novak), whose marriage is also on the rocks. The two meet secretly while Larry is at work building a "dream house" for the eccentric writer and playboy Roger Altar (Ernie Kovacs). These clandestine trysts are known only to one other person - their mutual friend Felix Anders (Walter Matthau). But when Larry is offered a tremendous career opportunity in Hawaii, he is suddenly town between his home, his career and his love for Maggie. And when Felix starts making passes at Larry's wife (Barbara Rush), the foundations of his entire life starts to crumble.