Upon its release in November 2005, Neil Diamond’s 12 Songs album enjoyed the biggest opening sales week—not to mention some of the best reviews—of his four-decade recording career. Diamond was elated. He had stripped his music to the bone, recording new songs acoustically with the help of producer Rick Rubin, and that bold decision was being warmly embraced by the public and the press.

Days later, the CD was removed from stores.

That’s because the 12 Songs disc, along with 19 others manufactured by the Sony BMG conglomerate, had been embedded with spyware. The resulting outrage forced the company to pull the offending CDs from stores, making the music Diamond had spent the last two years honing and perfecting unavailable for weeks. “I was very disappointed,” recalls the celebrated singer and songwriter, who sank into a depression after discovering what had happened. “It interfered with the attention that was being paid to the music.”

12 Songs nonetheless went gold, and the company made things right with Diamond (signed to Columbia Records, now a Sony BMG division, for the last 35 years) by re-releasing the album the following year in an expanded edition. And ironically, the whole affair laid the groundwork for Diamond to continue his artistic roll. “My mindset was to get back to writing as quickly as possible,” he recalls. “So I licked my wounds for a couple of weeks, then jumped right back into it and began writing a new album.”

Songwriting was again a sanctuary for Diamond, just as it was when he first put pen to paper as an intense, sensitive teenager at Abraham Lincoln High School in Brooklyn. Scarcely had he learned his first chords on the guitar his parents gave him as a 16th birthday present than the songs began to pour out of him.

It was a very different talent, however, that landed Diamond a scholarship to New York University: fencing. (“I haven’t fenced in years,” he reports today. “I fenced at an alumni meet years ago, but my fencing days are over.”) Still, it wasn’t long before he was spending free hours shopping his songs at the Manhattan music-publishing businesses known collectively as Tin Pan Alley. By the spring of 1962 he had dropped out of college, six months shy of graduation (the school granted him an honorary doctorate in 1995), to seriously pursue songwriting.

The great husband-and-wife songwriting team of Jeff Barry and Ellie Greenwich (the Ronettes’ “Be My Baby,” the Crystals’ “Da Doo Ron Ron”) heard artist potential in the young man and helped him land a record deal. His major radio breakthrough came in 1966 with “Cherry, Cherry,” the same year his song “I’m a Believer” became a chart-topper for the Monkees. “I’ve been working full-time and making a living at it ever since,” he notes with typical understatement.

Diamond spent the remainder of the ’60s building on his solo success with hits like “Girl, You’ll Be a Woman Soon,” “Kentucky Woman” and “Sweet Caroline,” and proving his mettle as a dynamic live performer. His onstage prowess was documented on the 1972 live double album Hot August Night, which sold 2 million copies and further cemented his stardom. Even as he continued his pop success, Diamond stretched his artistry by composing the Grammy-winning soundtrack for the 1973 film Jonathan Livingston Seagull and collaborating with Band mastermind Robbie Robertson on 1976’s Beautiful Noise.

In 1978 he teamed with Barbra Streisand—with whom he had unknowingly sung in the same high-school choir—for the duet “You Don’t Bring Me Flowers,” which became the biggest radio hit of his career. His movie debut, 1980’s remake of the landmark Al Jolson film The Jazz Singer, met with a critical and commercial drubbing. The soundtrack, however, remains his top-selling album, having spun off the hits “Love on the Rocks,” “Hello Again” and the enduring anthem “America.”

Diamond spent much of the 1980s struggling to contemporize his sound in the face of a changing musical landscape on albums like 1986’s tellingly titled Headed for the Future. During the following decade he explored the music of other writers with efforts like Up on the Roof: Songs From the Brill Building and The Movie Album: As Time Goes By. And all the while, his concerts continued to attract countless self-proclaimed “Diamond Heads”—the trade magazine Amusement Business named him the top solo draw of the 1990s.

But there was a personal price to be paid for his monumental achievements. Diamond’s two marriages both ended in divorce, the first in 1969 and the second in 1995. “I’ve always been hard to live with, and I still am,” he admitted soon after his second split, which cost him a reported $150 million. “I’m a moody person. Melancholic.” He remains close to his four children, two from each union.

That second breakup helped inspire the songs on 2001’s Three Chord Opera, his first fully self-penned effort since 1974’s Serenade and the album that marked Diamond’s public recommitment to songwriting. He was soon courted by longtime fan Rubin, the star producer who had similarly sought out veteran artists like Johnny Cash and Donovan for fruitful collaborations. Rubin encouraged him to reconnect with his roots as a songwriter and—crucially—to accompany himself on guitar in the studio, which Diamond hadn’t done for decades. The superstar who had sold 48 million albums in the U.S. and still commanded sellout arena crowds worldwide had now re-identified himself as a singer-songwriter first and foremost.

Now comes Home Before Dark, his second teaming with Rubin. “This album has some of my best work,” Diamond declares. “I’m very, very excited to have people hear it and enjoy it.” Like much of his work, Home was recorded at ArchAngel, the West Hollywood recording studio Diamond has owned for more than three decades and the place he calls “my musical home.”

We caught up with Diamond, now 67, on a day off from tour rehearsals. “We’ve been so productive that we can spare it,” he figures. He took the opportunity to cast a detailed look back at the early days of his songwriting career, the evolution of his artistry and what the future might hold.

For more, get the latest Issue of Performing Songwriter, ISSUE No. 109