There was a discussion in IRC similar to this one, involving spamming guild war on 1 single guild. If you think they are harassing you, contact a gm (fastest way is VIA IRC), and place your complaint there.
I'm sure you've done this aleady, but make it clear to the offenders that what they are doing is unwanted. If they do it multiple times after telling them it is unwanted, it becomes harassment and the ToS state that harassing players is against the rules. In that case the GM's should be able to warn the offenders. However, if they only do it once or twice it will simply be an annoying, but valid use, of the item.
However, with all that said, I'm unsure that the gm's will run to your rescue when there is a simple way around the issue (removing the buff). Previously it was not possible to remove the buff, and I'm quite certain that it was modified so players can remove it for reasons such as this.