MONTREAL -- He was the well-to-do construction boss who rubbed elbows with a Mafia don, and counted stacks of cash in the back room of an Italian cafe that was a noted Cosa Nostra hangout.

But Nicolo Milioto says, to him, old Nicolo Rizzuto was just a pal. Known to the rest of the world as the now-deceased patriarch of the Rizzuto crime family, Milioto remembers him as a guy to share espresso, play cards and count cash with.

Two days into his long-awaited testimony at Quebec's corruption inquiry, the 64-year-old is disputing characterizations of him as some shadowy figure linking political parties, the construction industry and the Mob.

Milioto has been described as a key hub in a wheel of corruption -- the person who connected the Mafia, a major municipal political party and certain parts of the construction industry.

But the businessman, who apparently dubbed himself "Mr. Sidewalk" according to a previous inquiry witness, steadfastly denied ever collecting money on behalf of the Mob.

Milioto said he isn't a member of the Mafia. If he was seen on police video delivering money to the Rizzutos, he said, it's because he was only doing it as a friendly service.

"It's been 45 years that I've lived in Canada and it's been 45 years that I worked 60 to 70 hours a week," said Milioto, who has just retired.

"I raised five children and put five children through school ... and I'm not a member of organized crime. I was the owner of a construction company."

Milioto said he'd never even heard of the Mafia. He was unaware of people in Canada paying the so-called "pizzo" -- protection money.

But he did admit to being aware of the "omerta" principle, the Mafia's traditional code of silence and order not to speak to authorities.

His stated awareness of omerta prompted a wry response from the inquiry judge, France Charbonneau: "So that, you know," she retorted dryly.

The series of vague answers twice prompted Charbonneau to remind the witness that he risked being found in contempt of court.

The inquiry watched several videos of Milioto dropping off and, in some cases, counting cash in the back room of the now-shuttered Cafe Consenza in east-end Montreal.

Milioto was caught on police sureveillance a whopping 236 times at the club over a two-year span while the RCMP investigated the Rizzuto clan. There was no audio to accompany the video because construction meetings fell outside the scope of the RCMP investigation, which at the time was focused on drugs.

But Milioto offered little help in revealing what was discussed as the cash was counted. He offered up a steady diet of "I don't remember" and "I don't know."

Milioto continued to insist that the money he brought to the Rizzutos either came from fellow construction boss Lino Zambito, who has admitted to corruption, or from fundraising for community events.

He also said he borrowed money from Rizzuto on one occasion, despite the fact that he owned a multimillion-dollar business.

He said the loan was interest-free. He originally said that it was for his daughter's wedding or for his house but, since neither event matched the timing of the cash exchanges, he said he couldn't remember what the money was for.

"He (Rizzuto) gave me money. For what? I don't know, I don't remember what I did with that money," Milioto said.

Earlier Tuesday, the retired construction boss said he didn't even know if the Mafia exists, or what it is, and he couldn't define the Cosa Nostra if he tried. He told the corruption inquiry that he's only ever read about the Mob in Canadian media sources.

"The Mafia, is it someone who kills? Someone who steals? Someone who sells drugs? I don't know," Milioto replied.

The tension was palpable between the inquiry counsel and Milioto for a second day.

At one point, Milioto appeared to threaten lawyer Sonia LeBel as she questioned him.

"If you respect me, I'll respect you," Milioto said.

"If you mistreat me, I can mistreat you the same way."

Asked to clarify, he denied having made any threat.

Charbonneau grew tired of Milioto's answers.

She angrily advised Milioto to have a discussion with his lawyer about contempt of court and perjury during a break, before returning to the stand. In the afternoon, she reminded him that vague answers could also be construed as contempt.

Milioto said he read about Rizzuto's Mafia ties in the papers but knew nothing more about it. This despite the elder Rizzuto's arrest 2006 and his son's incarceration in the United States that only ended late last year.

"I'm not aware and I don't know," Milioto said, testifying in French. "To me, he was a family man and a good person."

Milioto said he didn't know if the family's involvement in organized crime was simply an invention of the media.

"What is the Mafia? It's difficult to define," Milioto said.