with the gloves off - Phrases/Idioms
The American Heritage® Dictionary of Idioms by Christine Ammer. Copyright © 2003, 1997 by The Christine Ammer 1992 Trust. Published by Houghton Mifflin Harcourt Publishing Company. All rights reserved.
With or ready to dispense rough treatment, as in Prepared to oppose the council, the mayor marched into the meeting with the gloves off. This idiom alludes to old-style boxing, when gloves were not used. [Early 1800s]