There are two sides of the coin on this. First, compact flagship has been proven to sell well. Doesnt matter if you like Apple or not, the iPhone SE is proven to sell in quite a large quantity to the point where they had to built a factory dedicated for the Indian market. It is also highly regarded to be as good as as its bigger brother the 6S, despite its inherent drawbacks (old design, no 3D Touch, first-gen Touch ID, etc.). Ideally all smartphones should have 3 variants, small, medium, large, that is (in common terms) Mini/Compact, regular, and Plus/XL.
On the other side of the coin, due to Android’s backwardness and ridiculously wasteful resource use, there is simply no physical space to fit everything, especially the battery in such a small device. Companies like Sony simply made the device thicker, which is fine by me, but doesnt resonate well in the market. There were actually quite a bunch of compact sized Windows Phone flagships like the Nokia 800, HTC 8X, or Samsung Ativ Odyssey, none of which had an Android version. Until we could somehow manage to get rid of Android monopoly in the smartphone world, I have simply lost faith in non-Apple compact smartphones.
tldr: You can blame Android for the lack of small phones on the market.