LJUBLJANA, Yugoslavia, Monday, July 1— The Prime Minister of Yugoslavia and the leader of the breakaway republic of Slovenia announced late Sunday night that the Yugoslav Army had been ordered to withdraw to its barracks in an effort to ease the country away from the edge of civil war.

Early this morning, the Government in Belgrade took another important step toward restoring order when it installed a new head of the federal presidency, ending a six-week impasse that weakened the federal Government's authority to deal with separatist challenges from Slovenia and Croatia.

"We can announce following these talks that the withdrawal of the units to their barracks will begin tomorrow," said the Slovenian Prime Minister, Lojze Peterle. Army's Ultimatum

The order, if carried out, would in effect leave Slovenia with its independence. But there is no assurance that the army will comply with the order. In a statement on television on Sunday night, the federal Prime Minister, Ante Markovic, said the army had acted on its own on Saturday when it delivered an ultimatum demanding that Slovenia abandon control of its borders.

On Sunday, the Slovenian government rejected the ultimatum, daring the federal army to take the "decisive military action" that it had threatened and spurring Mr. Markovic to travel to Ljubljana for talks with the Slovenian President, Milan Kucan.

The negotiations gained momentum with the arrival of a delegation from the European Community, which rushed back to Yugoslavia after seeing the near collapse of the cease-fire that they had arranged on Friday.

The community team, including the foreign ministers of Luxembourg, Italy and the Netherlands, threatened to suspend up to $1 billion in community aid scheduled for Yugoslavia if a cease-fire was not immediately enforced.

The Dutch Foreign Minister, Hans Van den Broek, a member of the European Community team, said that the essential question was, "Can we maintain the cease-fire and can we prevent further loss of life and suffering?"

He said the next step would be a meeting of the Conference on Security and Cooperation in Europe scheduled for Wednesday in Prague, A preparatory group is expected to meet in Vienna today,

Slovenian leaders have attributed the national army's restraint after the ultimatum expired at 9 A.M. Sunday to widespread international condemnation of the army's action along with threats of sanctions.

But residents of Ljubljana (pronounced LYUHB-lee-ah-nah) took no chances and headed for their cellars when air raid sirens went off in the Slovenian capital shortly after the deadline.

Slovenia and Croatia, Yugoslavia's two richest, most Western-minded republics, declared themselves independent of this multi-ethnic federation of six republics on Tuesday. The federal Government has called both declarations illegal, but it has yet to take action against Croatia, whose moves toward de facto independence have not gone nearly so far as Slovenia's.

It was unclear whether the installation of a Croatian this morning as head of the collective presidency, which is rotated among the country's six republics, would end Croatia's slide toward independence. The constitutionally mandated transfer of presidential authority to the Croatian representative, Stipe Mesic (pronounced STEE-pay MEH-sich), had been blocked by pro-union republics, led by Serbia.

Vasil Tupurkovski, the Macedonian delegate to the presidency, said that the "most delicate part of the negotiations" was "the guarantees for implementation."

He said the European Community foreign ministers had "provided guarantees, including guarantees that Croatia and Slovenia would suspend implementation of their proclamation of independence."

Mr. Tupurkovski said that, for the collective presidency, this "will practically amount to a rescinding of our approval of the proclamation of the President." Scores Killed or Wounded

The Yugoslav Army's four-day campaign to take over border crossings into Slovenia met unexpectedly stiff resistance from the republic's "territorial defense" force. Scores of people were killed or wounded in fighting before Friday's cease-fire took effect.

The army ultimatum's key point demanded that the status of Slovenia's frontiers with Austria, Italy and Hungary return to federal control.

The Slovenian government is adamant that the disputed borders are now Slovenian and that returning them to Yugoslav control would gut the republic's declaration of independence.

"The Slovenian Assembly most strongly condemns the brutal aggression of the Yugoslav Army and federal organs on the Republic of Slovenia's territory and resolutely rejects the aggressors' ultimatums," said a Slovenian Parliament statement released after an all-night session. Insisting on Independence

The Slovenian President has maintained that in the meeting with European Community mediators on Friday he never agreed to suspend Slovenia's declaration of independence. His republic, which has for months advocated a gradual break from Yugoslavia, would only suspend for three months further moves to assume functions performed by the Yugoslav Government in Slovenian territory, he said.

"The ministers never asked that the documents themselves be suspended," he said. "We cannot accept that our decisions themselves will be suspended."

Slovenian officials said that violence had diminished since Saturday morning, but that one Slovenian soldier was wounded in a shooting incident Sunday.

"In general it's peaceful," said Jelko Kacin, Slovenia's Minister of Information. "There are some points, mainly along the borders, where there has been some scattered shooting. But these are mostly incidents where someone is taking potshots." Attack by Gunmen in Car

At least three people, including a police officer, died in shooting incidents in Ljubljana late Saturday night and early Sunday. Policemen on the capital's main street at about 2 A.M. said gunmen in a car had opened fire on bystanders near the parliament building.

Uniformed militiamen crouched behind parked cars and building pillars and stopped and frisked anyone who wandered onto the silent streets.

Barricades jammed street crossings in Ljubljana, and road checkpoints and blockades slowed travel throughout the republic. Businesses in some areas have announced that today will be a normal workday, but the Slovenian Defense Minister asked that plants near army barracks not open.

By all accounts, Yugoslav Army units are surrounded by Slovenian territorial defense forces behind barricades throughout the breakaway republic and are often without regular supplies of food and water, and oftencut off from their headquarters and from access to medical assistance.

"We are practically surrounded by the territorial defense," Col. Jovan Miskov, second in command of Yugoslavia's main Ljubljana barracks, said at a news conference on Sunday.

Photos: Slovenian troops as they arrested three men yesterday at the train station in Ljubljana, the republic's capital. (Reuters) (pg. A1); Although a cease-fire was reportedly being observed by both Slovenian and Yugoslav forces, air raid sirens sounded in Ljubljana yesterday, sending residents, some carrying blankets and portable radios, scuffling to shelters. High-flying jet planes could be heard over Ljubljana on Saturday night. (Agence France-Presse) (pg. A6) Map of Yugoslavia highlighting Ljubljana (pg. A6)