Marchers carried an enormous banner that read "Dilma out and prison for Lula," a reference to former president Luiz Inacio Lula da Silva, who was charged Friday with obstruction of justice in an ongoing corruption probe.
"We want our country back and for these people to go," said Vilma Moniz Portella, a lawyer carrying a small inflatable doll of the judge leading the probe into bribes and kickbacks at state oil giant Petrobras.
The scandal has shaken Brazil’s political and business establishment, casting a shadow over the August 5-21 Olympic Games, the first ever held in a South American country.
Marchers said the turnout was smaller in Rio than at past protests calling for Rousseff’s impeachment that have attracted hundreds of thousands of people.
Rousseff’s impeachment trial for alleged violations of budgeting rules is set to begin August 29. A two-thirds vote by the Senate would remove her from office.
Suspended from office May 12, she was replaced on an interim basis by Vice President Michel Temer. If Rousseff is ousted, he would finish out her mandate, which runs until the end of 2018.
In Brasilia, about 3,000 protesters dressed in the green and yellow colors of the Brazilian flag took part in a demonstration against Rousseff outside the Congress, according to police estimates.
Similar anti-Rousseff protests were called in other cities, including Sao Paulo, Recife, Salvador and Belo Horizonte.
Rousseff’s supporters, meanwhile, turned out to rally against Temer in several cities.
Many heeded the call of a leftist umbrella group called the "Fearless People’s Front" and demonstrated in cities such as Sao Paulo where thousands clad in red chanted "Temer, get out!"