Obviously it all depends on what type of terrain you're skating, what style of skating you're doing, etc.
Midwest is crusty, so I don't have the luxery of a smaller wheel.
My favorite are 56mm, but I don't skate tech at all. That is basically my limit for size. It lets me skate in a nice sweet spot of 55-54mm with a bit wider of a surface too. Overall the wheel has a longer lifespan so its more economical. Especially for Bones wheels (which have thin riding patches compared to Spitfire), the V6 56mm are decently wider than the V6 54mm... The V6 54mm are slightly too thin for me.
I would say 54mm is a really healthy size for all types of skating and all terrain you'll find in the street. Enough size to get over things (crust, cracks, etc.) and maintain momentum, but they aren't monster truck wheels like 56mm+. They are a good starting point to wear down from to get the most lifespan out of the wheel.
I honestly feel like wheel WIDTH is something that isn't talked about enough. I feel it really has a huge effect on everything from turning to the general feel of the wheels. I feel like when I'm skating thin wheels that my board is on its tip-toes. However, Spitfire Radial Full were just a bit too big for me. Really felt like monster truck wheels in a way that was a bit too heavy and cumbersome.
I feel like the Bones V6 56mm are essentially my ideal wheel... though Spitfire Conical Full 54mm are nice too with a good balance of width. I think OG Classics in 56mm or 58mm would probably be awesome too, but I haven't gotten to ride those yet.