My current set up is:
(Maxed) Potions
(Maxed) Cooking
(Maxed) Attack
(Maxed) Defense
With the Potions, I found there is no reason to raise her affinity past 96, because it won't unlock new pots unless I am mistaken. I do NOT use my cooking housemaid for food any longer because I no longer require the food it provides; instead I use her as a secondary Pot maid. So I always use up both of their fatigues for as many pots as possible.
And then I have attack housemaid for Sardo/Grafu, and Defense for siege/the rest of the time essentially.
If I were to give myself two more, I would probably do one more Pot housemaid and/or one more Attack Housemaid. Another pot one would be good to get what I need, with twelve chances to get good stuff. The attack would be in case I want crit over pattack (or vice versa) in case I want the percentage HP buff + crit rate (used in siege as well.). I'd definitely get one more Defense housemaid as I often get PDef over HP.