I don't think you can get to Yrvandis if you're not a Dwarf, but don't quote me on that.
Getting to Elf Island has become very easy though. At level 10+ go find the Mirror World NPCs at the bottom of the stairs between Central and the Administrative District of Varanas, chat up the one that offers info about Mirror World and a quest marker will pop over another NPC there. (sorry, can't get into the game atm so can't check NPC names) The quest includes a teleport to Elf Island. When you use it, just remember to unlock the Snoop at the Haven of Departure so you can go back later.
You can also just find someone who's willing to port you over.
Side note: If you're a gatherer make sure you don't pop lvl 20 in any gathering skill until after you've done the opening quests through to questing for the gathering instructors in both zones (Pioneers in Howling Mtns and the ones in the Valley of Prep) or you'll get locked out of the quests. If it actually is possible to get to Yrvandis it probably applies to those gathering instructors as well. Same for the short side quest chain between Woody Woods and the herb gathering instructor in Pioneers - get it done before you pop lvl 20 gathering as well.
I'm told that also applies to taking Alchemy to lvl 20 prior to completing the quest *Ami is not food* offered by the alchemy instructor in Logar - get the quest done first or you won't be able to do it.