I completely understand the problem. Recently my arch has sunken alittle and my foot has pushed out and caused some gaping at the mouth area on various pairs of factory shoes. It is not evident on the handmade ones, only the machine constructed pairs. The handmades can have more arch support built in and it holds the foot better in place so there is less rolling of the foot, but not everyone can wear them, especially flat footed humans.
Yes, my AE loafer and quarter brogues are the most severe, evident even when standing still (college bargain acquisitions that are still occasionally in use). The handmade shoes from Hungary suffer much less from this, although they still bow out a little when I walk. This is the first pair that is truly skin tight. The shoemaker suggested this last and I'm in love with it. Another pair is being made, black full brogue.