In a statement said to have left his inner circle incredulous, the Pope said he no longer had the mental and physical strength to run the church, which has an estimated 1.2 billion followers – including 850,000 in Scotland – and would leave on February 28 after just less than eight years.
The Vatican said it hoped to have a successor in place in five weeks' time. A new Pope could be elected by Palm Sunday on March 24 and be ready to take over by Easter, one week later.
Spokesman Father Federico Lombardi said even Pope Benedict's closest aides did not know what he was planning, adding that the pontiff did not intend to influence the decision of the cardinals who will enter a secret conclave to elect a successor.
The Vatican also ruled out any specific illness or depression and said the decision was made in the last few months "without outside pressure".
Georg Ratzinger, 89, who is a priest, said he had been "very surprised" to learn of his elder brother's resignation but said he had been considering it for some time.
The last Pope to step aside was Pope Gregory XII, who resigned in 1415 amid a schism within the Church.
Several pontiffs, including Benedict's predecessor John Paul II, did not step down because of the confusion and division that could be caused by having an incumbent Pope and his predecessor living at the same time.
The Vatican has said Benedict did not fear a possible schism, adding that once he stands down he would first go to the papal summer residence south of Rome and then to a cloistered convent inside the Vatican walls.
Benedict's time at the Vatican has been beset by child sexual abuse crises, the scandal over the leaking of his private papers by his personal butler and turbulence with the Islamic and Jewish faiths. The first German Pope for 1000 years and the second non-Italian in a row, his official inquiry into abuse in Ireland led to the resignation of several bishops.
Benedict, who in 2010 made the first papal visit to Scotland in 28 years, also stepped up the Church's opposition to gay marriage and underscored its resistance to a female priesthood and to embryonic stem cell research.
In a statement, the pontiff said: "In today's world, subject to so many rapid changes and shaken by questions of deep relevance for the life of faith, in order to steer the ship of Saint Peter and proclaim the Gospel, both strength of mind and body are necessary, strength which in the last few months has deteriorated in me to the extent that I have had to recognise my incapacity to adequately fulfil the ministry entrusted to me."
Up to 120 cardinals will now gather to choose a successor. Voting rules allow for just one method of selection – by two-thirds majority.
Leaders of the Catholic Church in Scotland expressed shock and sadness at the resignation.
Cardinal Keith O'Brien, Scotland's most senior Catholic, said: "Like many people throughout the world, I was shocked and saddened to hear of the decision by Pope Benedict XVI to resign. I know that his decision will have been considered most carefully and that it has come after much prayer and reflection."
Philip Tartaglia, Archbishop of Glasgow and the former Bishop of Paisley, said: "We've all been surprised and shocked, it was very unexpected. I think we're all really trying to get our head round it – both people like myself and people from the Catholic community.
"It's unsettling for a start if a Pope dies or this (the resignation), because it's unprecedented, or almost unprecedented; it makes you a bit unsettled.On a personal level, the Pope has been quite influential on my life – he made me Bishop of Paisley and he made me Archbishop of Glasgow. I've drawn a lot on his teaching and on his style in my own preaching and teaching, and for that reason I'll miss him."
First Minister Alex Salmond said: "We should respect the decision of His Holiness to pass on his ministry in a selfless gesture, on health grounds, in the best interests of the Church.
"Like many Scots, I remember with great fondness the resounding success of Pope Benedict's visit to Scotland in 2010 and the papal Mass celebrated at Bellahouston Park. I wish him a very peaceful retirement."