In the course of a long, hard life that followed, Mr. Carter, a sharecropper's son, forgot about that day, the song and the man who captured it on tape, Alan Lomax.
Until about two weeks ago, when two people visited him in his Chicago apartment to give him some amazing news - and a $20,000 check.
Last week, he boarded an airplane for the first time, flying to Los Angeles, where he was celebrated for contributing to the Grammy Awards victory of the "O Brother, Where Art Thou?" soundtrack, which won album of the year honors.
The album's producers used the version of "Po Lazarus" Mr. Carter recorded in prison for Mr. Lomax, a musical archivist and music writer who was traveling the American South with his tape recorder.
For more than a year, as "O Brother" climbed the charts and sold millions of copies, its producers searched for Mr. Carter, 76, not only to acknowledge his contribution but to pay him thousands of dollars in royalties.
How they found him is an unusual story in an industry rampant with tales of swindled royalties, corruption and stolen song credits.
Searching through the archives of the Mississippi penal system, Social Security files, property records and other public records and various databases, the record's producer, T-Bone Burnett; the Lomax archives; and an investigative journalist for a Florida newspaper found Mr. Carter in Chicago with his wife, Rosie Lee Carter, a longtime minister of the Holy Temple Church of God.
Less than two weeks ago, Mr. Lomax's daughter, Anna Lomax Chairetakis, who runs the archives, and Don Fleming, director of licensing, visited the Carters and presented a platinum album of the soundtrack and a royalty check for $20,000 - the first installment of what may become several hundred thousand dollars in payments. Mr. Carter, a former shipping clerk, told them he had never heard of the album or the film.
Mr. Fleming mentioned that the album was outselling the latest CD's of Michael Jackson and Mariah Carey. "I told him, `You beat both of them out,' " Mr. Fleming said. "He got a real kick out of that. He left the room to roll a cigarette and when he came back, he said, `You tell Michael that I'll slow down so that he can catch up with me.' "
Mr. Burnett said he first heard the work-gang song about five years ago when he was listening to music in the Lomax archives in New York City. "It just made a deep impression," he said. "It was such a beautiful version, a soulful version of a great song."
Later, when Mr. Burnett began working on the album, the song was placed first on the soundtrack of the film, which was made by Joel and Ethan Coen. The film is a sort of Depression-era fable loosely based on "The Odyssey." The soundtrack, on the Lost Highway label of the Universal Music Group, features a powerful cross section of Southern roots music by performers like Ralph Stanley, Alison Krauss and Union Station, and Emmylou Harris. The soundtrack won four other Grammy Awards in addition to album of the year.
Mr. Carter will earn royalties for being the lead performer on the Lomax recording used on the album. Because "Po Lazarus" is in the public domain, he will also earn songwriter royalties, which go to the performer once the copyright expires. Mr. Burnett said Mr. Carter's royalties could run "well into the six- figure range."
The album has sold five million copies and was the No. 2 country music album on the Billboard chart this week, after Alan Jackson's "Drive." The Grammy triumph is expected to push sales far higher.
Mr. Carter said over the phone that he was not sure what he would do with the money. His daughter Elizabeth Scott, a real estate broker, said she and her two sisters were relatively well-off. Their children include a Chicago police officer.
Ms. Scott said that although her parents were comfortable in a large apartment of a building they own in the Austin area of Chicago, they might use their windfall to find an apartment more appropriate to their needs. Mr. Carter often uses a wheelchair.
Mr. Carter's early life was not easy. He left home at 13, and ended up in the Mississippi prison system four times, the Lomax organization found. Two convictions were for stealing, another was for a parole violation for possessing a gun and a fourth was on a weapons offense. Mr. Carter said he remembered being in prison only once.
The Lomax organization said Mr. Lomax recorded the song at Camp B at the Mississippi State Penitentiary in Lambert. Mr. Lomax, who is 87 and lives in central Florida, has explored and promoted folk music around the world for more than six decades. In the 1950's he focused on the American South, and issued 16 albums on the Atlantic and Prestige International labels. In his Southern trips, Mr. Lomax, using a tape recorder, often sought out the music of the African-American South.
Mr. Fleming, the Lomax licensing director, said the assumption early on was that Mr. Carter might be dead. He began searching for Mr. Carter by checking Social Security death records. Then, using the Freedom of Information Act, Mr. Fleming began looking through the files of the parole board in Mississippi and found evidence that Mr. Carter had moved to Chicago around 1967.
At the same time, Chris Grier, a reporter for The Sarasota Herald-Tribune who was working on a project about Mr. Lomax's life, came in contact with Mr. Fleming. Using the newspaper's databases, Mr. Grier came up with a list of James Carters in the Chicago area. Because the name was common, Mr. Grier said, he began concentrating on spouses. He tracked down Mrs. Carter's name on property records.
"She owned a storefront church," Mr. Grier said. When he met Mrs. Carter, Mr. Grier said, she told him her husband's birth date and said he had spent years in Mississippi. Further investigation convinced Mr. Grier that this was the James Carter he was looking for. The Lomax group, after meeting Mr. Carter, confirmed that he was the man they were seeking.
In the years after prison, Mr. Carter worked at a number of jobs, including as a shipping clerk, but seemed unable to focus on any one occupation.
Mr. Carter was initially reluctant to fly to Los Angeles for the Grammy ceremonies, but Ms. Scott told him it was "the chance of a lifetime." He flew with his wife, Ms. Scott and his other daughters, Hattie Tucker and Corie Macklin.
He said over the phone on Friday that he was flattered by all the attention, but that he just wanted to get on with his life. He said he barely recalled "Po Lazarus."
"I sang that a long time back."