Appearances can be deceiving. I'm sure there's a lot of activity on certain servers, and little on others. But the best way I've found to determine active population size for a given RoM server is to create a toon on it, then monitor World and perhaps one of the Trade channels. It's not definitive, but active World and Trade channels point towards an active server. Use shift-click on toon names in those channels to find out more about them--classes, levels, and guild membership, if any.
NB: This time of the year, many "younger" players (younger and, mainly, much younger than myself) are in the midst of finals or approaching finals in school. So there will be a dropoff in activity for those people. I'd say that early and mid-evening would be the best time to do your checking.
Also, and in accordance with how I started my reply, don't place too much weight on the populations of lower-level zones. Most of our players are mid-level and high-level, with few of them starting new toons and leveling them up.
And while it might be time-consuming, creating and playing new toons on several servers can be useful in helping determine which server you eventually want to concentrate on and invest in.