Afaik, dot based classes in runes of magic fall short of being competitive in instances. Dots don't scale as well as other damage types during burns (when you have potions and short duration damage buffs/debuffs active) , which means classes with a strong burst will generate a significant lead in terms of damage done.
Most bosses are just bursted down using burn classes like scouts, warrior/scout or champ/mage. Dot classes are only really useful in siege or "small scale pve" (questing, solo'ing instances etc.).
Some sueggestions for dot-based classes to try out....wl/r has many magical based dots while providing support skills to self/group members. Still not really competitive in instances though as wl/m is a far better support and as mentioned above, dot classes don't deal significant damage during burns.