Mr Sarkozy said the protests at the government proposals would only promote tension, misunderstandings and anger.
Around 5,000 mainly Muslim marchers took part in a demonstration in Paris, which was fewer than expected.
There were also rallies elsewhere in France and Europe, the Middle East and in Indian-administered Kashmir.
President Jacques Chirac announced a ban on overtly religious symbols in schools last month after an official report into state secularism.
Many of France’s five million Muslims see it as an attack on their religious and human rights.
“When I came here, they told me France was the land of human rights. I found out it’s the opposite,” said 30-year-old Algerian-born Kawtar Fawzy at the Paris protest.
Apart from the Islamic headscarf, the ban – scheduled to be enacted before the next academic year in France – would also affect the Jewish skullcap, big crucifixes and Sikh turbans.
The government proposed the new law as a measure to safeguard France’s secular tradition.
“It is only through dialogue, the path of compromise and mutual respect that each person can find his place in the republic,” said Mr Sarkozy in response to the marches.
Mainstream Muslim groups had distanced themselves from the action, advocating instead continued dialogue with the government.
The demonstrations in Paris and other French cities were organised by a small group, the Party of French Muslims (PMF), which is regarded by many in France as a radical Islamist organisation, the BBC’s Alan Little reports from Paris.
An estimated 2,400 opponents of the ban rallied in London, where there was also a small counter-demonstration.
Outside the capital cities, including Brussels where about 1,000 protesters appeared, the largest demonstrations were held in the French regions. There were small rallies, too, in the Middle East.
An estimated 3,500 marched in Lille, 1,800 in Marseille, 1,500 in Mulhouse and hundreds in other French cities, police and organisers said.