Doesn't appear to be related to conversations. It's actually really kind of amusing in a sad way
Sending a reply does reportNotSpam on the message being replied to.
reportNotSpam sets a LastReportedSpam user preference field.
Updating the UserPreferences causes Sieve rules to be rebuilt.
And there's the cost! Rebuilding rules can be quite expensive.
...
As an immediate fix, the interface will (in a few minutes once we roll out the change) delay sending the notSpam until after it's sent the message. So the slow rebuild will still be happening, but it will be happening asynchronously, so you're not blocked waiting on the send.
Later, we'll fix things so setting this particular preference doesn't rebuild rules. Setting other user spam preferences DOES require rebuilding rules, which is why this slipped under the radar.
Thanks for all your help everyone. I'm about to go reply to all the tickets
Cheers,
Bron.