In February, Umarov claimed responsibility for the bombing of a passenger train travelling between Moscow and St Petersburg in November, warning that "the war is coming to their cities" and that "the zone of military operations will be extended to the territory of Russia".
Russian officials have blamed Muslim separatist fighters from the North Caucasus for the Moscow attacks, the deadliest in Russia's capital for six years.
Neave Barker, Al Jazeera's correspondent in Moscow, said: "Doku Umarov is perhaps the most high profile on the Kremlin's wanted list, the most high profile of people calling themselves warlords operating out of the troubled region as a whole.
"He is believed to have led the siege at Beslan in 2004, he's also wanted for a string of other attacks as well."
In Wednesday's statement, Umarov said the blasts in Moscow were an act of revenge for the killing of civilians by Russia's security forces.
He said: "I promise you that the war will come to your streets and you will feel it in your lives, feel it on your own skin."
Speaking to Al Jazeera, Shamsuddin Botokayov, a representative of the "Caucasus Emirate", said: "Our emir confirms this operation and it was because of what the Russians did last month and killed innocent people.
"The Russians do the same operation in Chechnya, the people in Russia only watch these operations on TV and do nothing about it. But we live it as a reality in Chechnya.
"That's why we want to take the war to the streets and homes of Russia so their people can taste what our people is going through.
"Our emir says he want the Russian people to wake up and to tell Putin and his government to leave the Caucasus and go back home.
"The emir says that this attack will not be the last, the coming will even be bigger."
Earlier on Wednesday, at least 12 people were killed and another 18 injured in two bomb blasts in the North Caucasus region.
The first blast occurred in the town of Kizlyar in the southern province of Dagestan on Wednesday morning, when a car bomb near a school was detonated, killing two police officers.
Rashid Nurgaliyev, Russia's interior minister, said the bomb went off as police tried to stop the suspicious-looking car.
|At least 12 people were killed in two bomb blasts in Dagestan [Reuters]
He said as police and residents gathered at the scene of the blast a suicide bomber, wearing a police uniform, approached and detonated a set of explosives.
A high proportion of those killed were policemen, authorities said, including the town's police chief.
Barker said there are fears that the bombings in Moscow and Dagestan are pointing to an escalation in violence in Russia.
"The parallels are quite striking between the attacks in Dagestan and Moscow," he said.
"This morning the federal security service said the two female suicide bombers that blew themselves up in Moscow may have been acting as part of a group of up to 30 more suicide bombers.
"The security service believes they could have been recruited and trained by a man called Said Buryatskiy, or Alexander Tikhomirov, who was a well-known Chechen warlord.
"He was killed in a special operation by Russian forces at the beginning of March.
"Officials have said that both of these attacks may be a revenge for the killing of one of their principal commanders in the field."
The North Caucasus has been the site of two wars in Chechnya and hundreds of violent attacks since the 1991 collapse of the Soviet Union.
Violence has spread from Chechnya to the neighbouring regions of Dagestan and Ingushetia.
Speaking to Al Jazeera, Magomed Mutsolgov, a human rights worker in Ingushetia, said: "There will be some tighter security here, more police checks, even rough violent checks, not just because of what happened in Dagestan but also in Moscow.
"And across all the north Caucusus there will be more raids, and lots of checks of suspicious people.
"There will be various lists – relatives of people who died, relatives of militia, relatives of victims.
"I'm sure there will be more violations, persecutions and discomfort for people. To put it mildly. I don't doubt this will happen".
Vladimir Putin, Russia's prime minister who led Moscow into a war against Chechen separatists in 1999, said on Tuesday that those behind the Moscow bombings must be scraped "from the bottom of the sewers" and exposed.