Al-Shabaab gunmen killed and injured more than 39 in Nairobi

Sources claim there have been at least four plots to attack affluent targets such as the Westgate mall. Photograph: Kabir Dhanji/EPA

It is only seven years since Ethiopian forces swept into Somalia with the political and military backing of the US to topple the Islamic Courts Union, an Islamist movement that had taken control of much of south and central Somalia after years of disastrous feuding between warlords. Ethiopia's vastly superior forces routed the youth militias loyal to the courts with hundreds killed or driven from the cities.

However, the Ethiopian intervention was the cue for the emergence of what had been the unheralded youth wing of the courts movement, "the shabaab" – meaning "youth" in Somali. These young fighters regrouped and took the war to the Ethiopians, who wearied of the guerrilla conflict and withdrew.

In their absence another force under the command of the African Union – made up of troops from Uganda, Burundi, Djibouti and latterly Sierra Leone, as well as Kenyans in the south – attempted to hold al-Shabaab, or Harakat al-Shabaab al-Mujahideen, to give the group its full name, at bay.

By 2010 the Islamic extremists held sway over much of south and central Somalia and appeared set to take the capital, Mogadishu, itself, where an appalling urban war was being fought. The future of the weak internationally backed transitional government appeared bleak until the arrival in 2010 of a devastating drought and famine that eroded support for the movement after it opposed foreign aid.

A year later al-Shabaab surprised many observers by withdrawing from Mogadishu in what it called a "tactical retreat" into the southern hinterlands of Somalia. Then, it lost its economic lifeline in the southern port city of Kismayo when Kenyan forces, fighting alongside a former warlord Ahmed Madobe, overran the city last year.

Since then there have been many predictions of the collapse of the movement but it has proved adept at managing the divisions between Somalia's fractious clans and disrupting attempts to form an effective government in Mogadishu in a series of terror attacks.

Many of those attacks follow a similar formula to that in Nairobi, with gunmen following in the wake of car bombs or grenades to inflict the maximum number of casualties. In the past fortnight there was an attack on a popular restaurant in the Somali capital, the owner of which has suffered repeated assaults on his businesses.

The Islamists have long been split between Somali nationalists, who see their jihad in local terms, and foreign fighters who see the conflict in the Horn of Africa as part of a global struggle. The international jihadists showed their influence when they conducted bombings in the Ugandan capital, Kampala, on the night of the 2010 World Cup, killing more than 70.

A spate of grenade attacks in Kenya followed the country's decision in late 2011 to intervene in the war to the north of its borders. Most of the casualties, until Saturday, suffered in the remote frontier towns of Kenya's north-eastern province and in the poorer immigrant neighbourhoods of Nairobi. However, there have been at least four plots to attack affluent targets such as the Westgate mall, thwarted by intelligence agencies, sources told the Observer.

There were fears on Saturday night that anger over the assault would spill over into attacks on the large Somali minority in Nairobi. The New York-based monitor Human Rights Watch reported earlier this year that Kenyan police and security services had carried out widespread abuses of Somali refugees under the cover of responding to terrorist threats. Kenya hosts nearly 750,000 Somali refugees, many of whom live in the complex of camps at Dadaab just inside the country's border with