Diack is provisionally suspended by the IOC while he is investigated by French police over allegations he took bribes to cover up positive drugs tests when in charge of the IAAF.
He had already resigned as president of the International Athletics Foundation, the charitable arm of the IAAF, which the 82-year-old headed for 16 years.
On Monday, the IAAF was implicated in a World Anti-Doping Agency report that accused Russia of widespread doping.
Diack's son Papa Massata, advisor Habib Cisse and the former IAAF anti-doping chief Gabriel Dolle are also being investigated by French police.
The French financial prosecutor said in a statement last week: "Diack is suspected of receiving money in exchange for deferring sanctions for several Russian athletes who were found guilty of doping in 2011, ahead of the Olympic Games.
"Diack and Cisse were arrested on Sunday and released on Tuesday, after being interrogated by police officers and judges.
"They were presented to a judge on Tuesday who informed them that they had been put under a formal investigation. The investigation also continues into whether other persons were involved in suspected corruption."
The IOC said on Tuesday: "The executive board decided this afternoon to confirm the proposal of the IOC ethics commission to provisionally suspend Mr. Lamine Diack, the former president of IAAF, from his honorary membership of the IOC."
The statement went on to urge the IAAF to take action against Olympic athletes accused of doping in Wada's report.
Diack ended his 16-year reign as IAAF president in August, when Briton Coe, a double Olympic 1500m winner, was elected as his replacement.