Oskar Groening, who is physically frail, was sentenced to four years in prison in 2015 for his role in the murder of 300,000 people at the Nazi death camp Auschwitz.
However, he had not started serving his sentence due to a legal argument about his health.
Prosecutors said in August that a medical examination showed Groening was fit to start serving his prison sentence, though Groening's lawyer disputed that.
On Wednesday a court in the northern German town of Celle said: "The higher regional court thinks, based on expert opinion, that the convicted man is able to serve his term despite his advanced age.."
It said enforcing Groening's sentence would not breach his fundamental rights and added that special needs related to his age could be catered for in prison.
In a 2015 court battle seen as one of the last major Holocaust trials, prosecutors said although Groening did not kill anyone himself while working at Auschwitz, in Nazi-occupied Poland, he helped support the regime responsible for mass murder by sorting bank notes seized from trainloads of arriving Jews.
Groening, who admitted he was morally guilty, said he was an enthusiastic Nazi when he was sent to work at Auschwitz in 1942, at the age of 21.
Want to enjoy 'Zen' reading - with no ads and just the article? Subscribe todaySubscribe now