if you can just increase pressure to improve both efficiency and light quality why have they not done so to compete with other lighting sources? was it impractical for lighting manufacturers or dangerious or both?
If you boost the pressure/arc load, the efficacy saturates. The reason is, the broadening starts to extend into invisible parts of the spectrum, so the energy becomes lost again. So the maximum attainable efficacy is about 60..70lm/W (for really huge power and UHP lamps). The light quality rises to nearly 100 already with the UHP projector bulbs. But these are designed not that much for the overall efficacy, but for the point brightness, required for good system efficacy of the overall projector system.
The problem with huge power levels needed to reach the efficacy are the losses from the support materials absorbing the light (the arc becomes very small, what means in fact hidden within the electrode assembly).
Plus the high pressures and loads mean so high thermal load, the known materials can not stand it for too long, so the lifetime is rather limited (few 100's hours for a typical projector bulb).