The button appeared beside Gmail's normal send button and allowed users to shut down an email thread by sending a gif of a Minion dropping a microphone.
However, a flurry of complaints about the button appeared on Google's forums.
The firm has since withdrawn the feature and apologised.
"It looks like we pranked ourselves this year," Google said in a statement.
"Due to a bug, the MicDrop feature inadvertently caused more headaches than laughs. We're truly sorry."
A mic drop is a popular meme in which someone makes a conclusive statement or rebuttal before dropping a microphone and walking off.
Google was not immediately able to clarify details of the bug.
Google added that users who were still able to see the feature could switch it off by reloading or restarting Gmail.
The button was not enabled on the Gmail accounts of enterprise business customers.
It's hard to know what to trust on April Fool's Day as there are are many stories that seem rather strange but are in fact genuine.
The BBC has compiled a round-up of some of this year's odder stories that are apparently true.