imo you should wait until you upgrade your castle before you get all 6 buff towers, only the tp/xp and drop rate are even worth it anyway, until they rescale the towers that shouldve happened chapters ago

but if you place all your towers inside castle before upgrade, youll then realize you have to tear them down later for more important buildings etc, and be wasting a ton of resources