I think you can remove your level 60 one and then when done with the pet exp charms, go back to the NPC and it should let you get another level 60 one
I done that, since I usually keep an alt or two with the flower amulet to make gels for guildies. But you lose whatever progress you made on growing that flower, if you are actually working on L60 elite.
A flaw, sure, but pet XP charms are so valuable and so rarely obtainable, that even in that case, it is worth it to lose the flower, IMO. Or, if your item shop bank page is emptyish, stash the charms there till flower is done.