> Is the number of LCD panel manufacturers limited by some universal constant

It's limited by the ability to forecast the future as well as the costs (in money and time) to build a new panels factory and ramp up its production.

> or why Apple's competitors just don't build their own factories?

Because most of them have no core competency in manufacturing, because manufacturing is expensive to setup and evolve, because manufacturing is hard and because manufacturing does not mean you'll make money (one part of Apple's come-back strategy was to get rid of all its manufacturing capacity, a plan executed by Tim Cook)

Building and maintaining factories is really expensive, and a major distraction (they need to be continually updated to stay relevant). It's only worthwhile if you can guarantee you'll sell most/all of the factories' production potentials. AMD recently spun off its fabrication side for just this reason.