Six symphonic orchestras are cutting ties to famed conductor Charles Dutoit after multiple women have come forward with allegations of sexual assault.

At least five women are now accusing the conductor of assault taking place as early as 1985.

Four women, including musicians and singers, have told the Associated Press about the alleged assaults, many of which took place during the 25 years Dutoit was leading the Montreal Symphony Orchestra, a post he held until 2002, but the most recent accusation dates to 2010.

The women said that the assaults took place in multiple places including his dressing room and an elevator.

One woman said that Dutoit forced her to put her hands down his pants and that he thrust his tongue into her mouth.

Montreal Gazette journalist Arthur Kaptainis wrote Friday that he and many other people heard frequent rumours about Dutoit's alleged behaviour but chose not to say anything, either because they admired his work or because their livelihoods depended on him.

An Ottawas-based music journalist told CTV News Friday that Dutoit once touched her inappropriately while she was interviewing him in his hotel room.

Natasha Gauthier was writing a profile of Dutoit for L'Actualité magazine more than 20 years ago.

"He leans in and he's trying to touch my knee and he's trying to grab at my hand, he wants me to hold his hand, he wants to hold my fingers and my hand while he's talking to me. And so this is not respectful behaviour, this is not profesional behaviour," said Gauthier.

When Gauthier questioned musicians about this Dutoit stopped participating in the profile -- although the article was written anyway.

Now several symphonies are no longer working with Dutoit and his home symphony, the Royal Philharmonic in London, said Dutoit temporarily on leave.

The Royal Philharmonic also said it wants to give Dutoit time to get representation and tell his side of the story.

No formal complaints have been made to police.