Because California is extremely large and controls most of the warm-weather coastal territory on the West Coast, people have been willing to put up with a lot of bad policies for the opportunity to live there. Competitive pressure on the state government would be much greater if there were three or four states occupying California's present territory instead of one.
The theory and some of the practical issues involved in this are discussed in the widely-unread Unchecked and Unbalanced. There, I argue against the notion that there are scale economies at work in large governmental units. If that were true, then places like Denmark would be failed states. And Switzerland would be a hopelessly failed state. Each Swiss canton is about the size of a county in the state of Maryland, and yet the cantons have more autonomy than U.S. states. Below the canton level, there is local government.
Basically, governmental units in Switzerland, including the central government, serve populations that are an order of magnitude smaller than governmental units in the United States. We could carve a state like California into a hundred statelets and the result could still function as well as Switzerland.
I think it would be even better if location and government were separated to the maximum extent possible. I would like to see governments operate as franchises. I want to be able to select a different franchise from my neighbor. But that model does not have a working example like Switzerland that you can point to.
The problem with competitive government, or radical federalism, is not that it would suffer from diseconomies of scale. The main problem is that the incumbent governments will fight tooth and nail to stop it from happening.