Yep, I pretty much agree. I see threads as conversations. In face-to-face conversations it's natural to tangent to related subjects during the course of interaction. Oftentimes a conversation will end up a long way from where it started. This is normal. It's natural for people to inject humor here and there. But when a conversation starts going far from the subject and those involved still have things to say about the original subject, someone will say something and the conversation centers back on the topic. In real-life conversations, smart-asses tend to shut up because there are significant potential consequences if they persist in disrupting the conversation. In e-conversations, the risk to a "spammer" is much more limited, hence the need for moderators/administrators, who can actually deliver undesirable consequences.
I get as annoyed by those who don't seem to get this and try to insist that threads stay rigidly on-topic as I do by spammers. My brain's filtering function tends to scan, dismiss, and forget a spam post. I get annoyed when a pattern develops, it isn't humorous to me or the others involved, and the perpetrator doesn't quit when asked--report button time at that point.
Lep pretty much nails it--if it's pointless, unappreciated by the majority involved, and subsequently judged by staff to be spam, it is.