It varies depending on the light type. LED HID retrofits usually need bypassing, but ballast compatible lamps exist. In fact, Home depot sells an LED filament lamp that runs straight off an H39 ballast for yard blasters. Bypass lamps are better IMO because of less failure points and components wasting energy. Same goes with fluorescent, although ballast compatible fluorescents are fine from my experience.
Most cases I've seen for HID fixtures, the ballast is left in place and simply bypassed electrically. One example is with this Spartan branded floodlight that came off a Deli during an exterior remodel and LED changeover. The floodlight was originally 250 MH, but had an LED Corncob lamp made by Light Efficient Design( which unsurprisingly was not as bright). Much to my benefit, the 250w metal halide ballast was still in there and functional, I just had to do some rewiring.
With fluorescent, I've retrofitted one of my basement recessed 4X F40T12 fluorescent fixtures with Feit ballast compatible LED tubes because I was fed up with the temperamental Rapid start ballasts. Most plug and play LED retrofit tubes are electronic ballast only, but these Feit lamps are compatible with magnetic (haven't tried with preheat yet)I got them from Home depot.

