A Reuters journalist on the Golan said Austrian troops had already moved from the Quneitra crossing point to a United Nations base inside the Israeli-held part of the heights on Tuesday.
"The first 60 to 80 soldiers will land in Vienna tomorrow afternoon, so you can already see the withdrawal on site," Austrian Defense Ministry spokesman Andreas Strobl told Reuters in Vienna.
The Austrians have patrolled the buffer zone between Israel and Syria as part of the United Nations Disengagement Observer Force, known as UNDOF, since it was set up in 1974.
The Vienna government said last week it would pull out after worsening fighting between Syrian rebels and government forces sent its soldiers running for cover.
Two soldiers were wounded last week after Syrian rebels captured a border post, then were driven out by government troops.
Foreign Minister Michael Spindelegger said Austria would now negotiate with the United Nations about an orderly handover to the next contingent, "if there is one," but reserved the right to stick to its timetable for a full exit within four weeks.
UN spokesman Martin Nesirky told reporters in New York that 67 Austrian troops would leave the Golan Heights on Wednesday as part of a pre-planned troop rotation and would not be replaced by further Austrian troops.
"The Department of Peacekeeping Operations is in continuing discussions with the government of Austria about the timetable for withdrawing the remaining 310 troops in the observer force to allow for a smooth transition with replacements," he said.
Nesirky said talks continued with other countries about possible replacements for the Austrian troops. He said about 170 Fijian troops would deploy later this month to replace Croatian troops, who have already withdrawn from UNDOF.
Russia has offered to replace Austria in the Golan Heights, which were captured by Israel in the 1967 Middle East War, but the United Nations turned down the offer because the agreement with Israel and Syria precludes permanent members of the UN Security Council from taking part.
Chancellor Werner Faymann defended neutral Austria's decision to withdraw from Golan, where its roughly 380 soldiers make up the biggest contingent in the 1,000-strong force.
"We never could have and would never have wanted to take on a military mission to mediate or intervene between the opposition rebels and governmental troops," he told reporters after the government's weekly cabinet meeting.
"We took over a different mandate, which was appropriate for a neutral country."
He denied that Austria, which also has peacekeeping troops in hot spots including Lebanon, Kosovo and Bosnia, would suffer in international stature from the move.
Want to enjoy 'Zen' reading - with no ads and just the article? Subscribe todaySubscribe now