New Yorkers & Co.; Developer vs. Himself Over Coliseum Project

Mortimer B. Zuckerman is at odds with himself.

The pugnacious side of him wants to sue the pants off Salomon Inc. for pulling out of the deal with his Boston Properties Inc. to erect a gigantic office building on Columbus Circle.

The calmer Mort Zuckerman - the personality he wears unfailingly in public - clings to the hope that, somehow, he can emerge from this extravagant gamble as a wounded but wealthier puppy, beloved by fellow citizens, celebrated for his stick-to-it-iveness.

''We are not interested in continuous confrontation,'' Mr. Zuckerman said last week after agreeing to settle his differences with the city and Salomon. ''We want to build, and we want what we build to be well received.'' Deal Collapsed Last Month

The deal - to construct a skyscraper on the site of the New York Coliseum - collapsed last month when a state judge in Manhattan ruled that the city had violated its own zoning rules when it auctioned off the 4.5-acre site in 1985. Salomon, reeling from losses in the stock market turmoil of October, had already backed out of its agreement to arrange financing and to make the tower its world headquarters.

On New Year's Eve, Mr. Zuckerman and the city signed a new agreement to proceed. He would pay the city less for the site, the two square blocks beneath the coliseum - about $357 million rather than $455 million - and build a structure with 52 acres of floor space rather than 63.

Salomon would pick up most of the horrendous costs the project had incurred in the two years since Mr. Zuckerman submitted the winning bid against 14 other developers: the bills for the thousands of tons of granite already quarried for the exterior of the building and for the miles of steel that would have supported the twin-peaked structure. Biggest of all, Salomon would pay the architect and engineers the $27 million in fees already billed and the lawyers the additional millions, not yet calculated, for their work on the lawsuit against the project, which they lost. $10 Million Guarantee to City For his part, Mr. Zuckerman agreed that Boston Properties would pay the city a big guarantee - perhaps $10 million - even if the project were eventually blocked by a municipal board or agency.

''The city needed to show that it had scalped the developers,'' Mr. Zuckerman said. So he offered his thinning head of hair. At least that is the image he is cultivating.

''He projects a sense of vulnerability,'' said Kent L. Barwick, president of the Municipal Arts Society, the civic group that filed the suit that killed the project. ''But don't be fooled. It is really part of a very successful negotiating technique.''

It is a technique he has used often. To win the bidding, for example, he first raised his offer to $455 million from $353 million. That left him alongside one other bidder at more than $100 million above the pack. The Salomon Carrot

Not satisfied, he enticed Salomon to promise to hire an additional 3,500 workers by the time the new headquarters was finished, an idea particularly appealing to Mayor Koch. ''Creating jobs for New Yorkers is what we are here for,'' the Mayor said last year. When the deal went before a city review board, Koch administration officials said repeatedly that Salomon was threatening to leave town if the contract were not approved as written. (In the two years since the deal was signed, much has changed for Salomon. Even before the October market collapse, Salomon had said it would lay off 12 percent of its work force and re-evaluate its need for space in Manhattan. The 3,500 new jobs have evaporated, and at least 800 existing positions are also gone.) After the city had approved the project, the community boards with jurisdiction over the site wanted Boston Properties to scale back the building by about 10 percent. ''Mr. Zuckerman and his partner, Ed Linde, were ready to go along,'' said Joseph Rose, chairman of Board Five. ''The city refused.'' Approval Trouble Expected

Now, because of the lawsuit, the Koch administration will have trouble winning approval for a building nearly 20 percent smaller than originally designed. Mr. Rose and his fellow board members say the city is stubborn. He calls Mr. Zuckerman and Mr. Linde ''responsible and civic minded.'' He describes the Koch administration as ''greedy.''

When prominent New Yorkers last summer protested the size and shape of the original design, Mr. Zuckerman was ready to do anything to appease them. ''I was trying to leave town July 29 for my first vacation in four years,'' Mr. Zuckerman recalled. ''Suddenly I found myself in a fight with Jackie Onassis.''

Mrs. Onassis was only one of hundreds of neighbors of Central Park who signed petitions against the project, but in Mr. Zuckerman's mind she personified the opposition. Without telling Moshe Safdie, his first architect, he started looking for a replacement, someone who could design into the landscape rather than dominate the skyline. He chose David Childs, an architect at Skidmore, Owings & Merrill with whom he had often worked. 'Cautiously Pessimistic'

This time, Mr. Zuckerman hopes the process will be different. ''I'm cautiously pessimistic,'' he said. ''I'm sure there will be more lawsuits.''

As Mr. Zuckerman tells it, the turmoil over the Columbus Circle project is just how he likes to do business. He has given a little ground but left himself in a stronger position. ''A good deal is one that is fair to all parties,'' he said. ''You never want to give your opponent the feeling that the negotiations have been a complete loss. Let him win a point.''

On the other hand, he is not above limiting the other side to just that, a single point. He often plays softball with a team of media celebrities in Sag Harbor near his Long Island beach house. He is the pitcher. The game he remembers best involved a team from Connecticut. ''We won 19-1,'' he said with a grin. ''I remember every pitch.'' Wealth of $200 Million Is Safe

No matter how it proceeds, or whether it proceeds at all, the Columbus Circle project does not threaten Mr. Zuckerman's personal wealth of at least $200 million. It will be isolated from the 80 other developments Boston Properties owns and manages along the East Coast and in San Francisco.

Mr. Zuckerman is perfectly willing to begin construction without a tenant, as he did with his only other New York venture, 599 Lexington Avenue, which is between 52d and 53d Streets. He is confident the Coliseum site and design will entice someone before the building is completed, in 1991 at the earliest.

The project could, however, threaten the aura of success he has constructed as carefully as his buildings. No Zuckerman project has ever lost money, he said. Bull Market Helped

But then his career in real estate has paralleled a very strong bull market in properties. Shortly after he established Boston Properties in 1970, the Northeast market began to rise. After 1976, Federal tax policy made real estate the most favored investment in America. By the time the national market began to cool in 1983, he had shifted to New York, the strongest city in the nation for commercial real estate.

Mr. Zuckerman's strong desire for public acceptance has often led him to change his coloring to fit his circumstances, a trait enhanced by the diversity of his experience. He was born a Canadian but has taken on United States citizenship. He holds law degrees from both McGill University in Montreal and Harvard University, though he has never practiced, and he spent seven years teaching urban planning at Harvard.

He used part of his fortune from real estate to become a magazine publisher, first buying the Atlantic magazine and then U.S. News & World Report. He added a career in journalism by appointing himself editor-in-chief of U.S. News and writing a column for it. Atlantic Suit Lost

When he bought the Atlantic, he beamed at acquiring one of Boston's cultural treasures, a magazine identified with the city's intellectual life for more than a century. But when he discovered that its financial state was poorer than he expected, he stopped making payments to the former owners.

They sued. He countersued. Last summer, he lost.

He was forced to pay the former stockholders the rest of the purchase price, and he must pay the former editor, 65-year-old Robert J. Manning, $750 a month for life. Be that as it may, Mr. Zuckerman still owns the magazine.

Now he has a tenuous hold on a corner of New York's physical treasure, Central Park, and he is willing to risk at least $10 million to hang onto it. ''A lot of the criticism so far has been unfair,'' he said, exhibiting more than a little hurt at the scorn heaped on the now-discarded design. ''The site is an eyesore now,'' he said. ''But it is the most important location in New York, and we want to build there.''