Somali government forces backed by African Union troops moved cautiously on Sunday into areas of the famine-struck capital abandoned the day before by Shebab rebels, a senior military official said.

A small number of gunmen from the Al-Qaeda affiliated Shebab militia were still in war-torn Mogadishu after their surprise pullout by the bulk of their forces on Saturday morning, with skirmishes breaking out.

"Government troops and the African Union troops moved into several positions, including Mogadishu stadium," said Yusuf Dhegobadan, a senior government army officer.

"We are still maintaining cautious advancement into the stronghold of the Shebab fighters", he told reporters, speaking at the city's stadium, which until Saturday had been a rebel stronghold.

AU-backed government troops have been battling Shebab rebels in Mogadishu to secure aid delivery routes for victims of the drought threatening more than 12 million people in Somalia and other east African countries.

Until Saturday morning, government and AU troops controlled just over half of Mogadishu, including the airport and port, while the Shebab controlled the city's northeast.

"There are few of the Shebab trying to benefit from the advancement of our troops, carrying out desperate attacks," Dhegobadan added.

"But we are progressing cautiously, and we will eliminate them very soon."

Government officials celebrated the hardline rebel pullout, but a Shebab spokesman said Saturday that the withdrawal was merely "a change of military tactics."

Some civilians warily visited homes in former rebel-held areas they claimed they had not been able to access for several years, ever since Shebab took control of large parts of the city.

"We are very happy to come and see our houses for the first time (in) nearly three years," said Mohamed Farah, who visited his house near the stadium.

"It is too early to return, but we are planning to start repairing the building, because most of the houses have either collapsed or been damaged in the fighting."

Residents are still fearful of attacks.

"No one can say what is going to happen next, because the Islamist fighters are still active in some parts of Mogadishu," said Abdi Said, a resident in the city's Wardhigley neighbourhood.

"They are capable of carrying out attacks any time," he added.

The Shebab had for two years been promising to topple the Western-backed government in Mogadishu, but always fell short of smashing its last defences despite a deadly and costly offensive.

The UN has officially declared famine for the first time this century in Somalia, including in Mogadishu and in four southern Somali regions, and warned that the famine could spread.

Much of southern Somalia -- including the majority of regions declared to be in famine by the UN -- is controlled by Shebab rebels, who continue to ban several key aid agencies from operating.

Drought-hit Somalia is "the most severe humanitarian crisis in the world today and Africa's worst food security crisis since Somalia's 1991-92 famine," the UN has warned.