Let-s-not-and-say-we-did meaning

(idiomatic, colloquial, usually sarcastic, dismissive) Indicates that the speaker does not agree with a proposed action of a group, and does not wish to participate.