I've heard Robert Mankoff, the stippling-obsessed cartoon editor, say the same thing and it always struck me as odd. I believe the way he described it was that the comics should cause a slight curl of the mouth and no audible reaction.
You are not trying to submit the funniest caption; you are trying to win The New Yorker's caption contest.Humor and victory are different matters entirely. To understand what makes the perfect caption, you must start with the readership. Paging through The New Yorker is a lonesome withdrawal, not a group activity. The reader is isolated and introspective, probably on the train commuting to work. He suffers from urban ennui. He does not make eye contact. Laughing out loud is, in this context, an unseemly act sure to draw unwanted attention. To avoid this, your caption should elicit, at best, a mild chuckle. The first filter for your caption should be: Is it too funny? Will it make anyone laugh out loud? If so, throw it out and work on a less funny one.
I always thought cartoons were supposed to be funny and never cared much for those found in The New Yorker. After reading the mag for about two years I've warmed up to them a bit. But maybe, as Patrick suggests, I'm one of the people pretending to get it "in the hope of receiving some sort of osmotic transfer of IQ if they hold the magazine tight enough."