Merchants in Jamaica, Queens, lost a bid yesterday to have the Metropolitan Transportation Authority reconsider bus route changes that they fear will harm their businesses.

The merchants group, led by the Jamaica Chamber of Commerce, filed suit last month against the M.T.A., arguing that rerouting some bus lines that intersect Jamaica Avenue, the neighborhood's main commercial thoroughfare, would harm retailing. They said the Authority should have formally reviewed the environmental impact before approving the plans.

The M.T.A. has said the changes are needed to carry passengers to three new subway stations opening next month on Archer Avenue as part of a long-delayed extension of the J line.

The subway extension, which is to open Dec. 11, will connect the J line and the E and F lines at the Van Wyck Boulevard station. Officials hope the extension will relieve at least some of the overcrowding on the E and F, both of which end in Jamaica. Bureaucratic Juggling Mandated

Since the Archer Avenue plans have been formally adopted, however, yesterday's ruling required a certain amount of bureaucratic juggling.

First, the board had to officially reopen the issue of bus rerouting. Then members put a resolution in the record saying the changes ''would not result in any large or important environmental impacts'' and the board's decision not to require a formal assessment had therefore been right. The plans, unchanged, were then reaffirmed and closed.

''It's an attempt to justify, after the fact, a decision made two months ago,'' said an attorney for the merchants' group, Howard S. Weiss. ''They're trying to cover their tracks.''

The opening of the Archer Avenue extension, which will be the first new subway section in decades, will change many bus routes. Among those are the Q3A, Q4, Q4A, Q5, Q42 and Q85, which now all intersect Jamaica Avenue, but will instead run along Archer, parallel with Jamaica.

Mr. Weiss stressed that the merchants are not opposed to the subway extension itself, but only to those route changes that may divert people from the Jamaica shopping district. Attorneys for the M.T.A. must still respond by Nov. 22 to the group's lawsuit in State Supreme Court in Queens.