As far as I have been aware, I have known that lamp current is often difficult to measure on electronic ballasts for fluorescent and HID lamps. Why is this so?

"Difficult" is a relative term. It is true, there are a few complications:
- HF ballasts operate on frequencies outside of the frequency range of common meters. Normal meters end up at 400Hz, but that assumes a 400Hz sinewave. That is valid for both analo, as well as digital meters, old and new (dirct AC measuring movements, passive diode rectifiers accompanying analog depression meter movement, even active rectifiers in latest analog, as well as most digital meters), direct current measurement as well as clamp meters. Most often the main issue are parasitics around the key components, like inductance of shunt resistors or limited bandwith and slew rate of the active rectifier opamps. A square wave present on most HID ballasts contain significant higher frequency components, which when attenuated by the limited meter bandwidth, lead to a wrong readout.
- The lamp circuit has potential, often with high frequency components, on each end, so a common (grounded) oscilloscope sensing voltage across a small resistor can not be used.
- With fluorescents you need to separate filament heating curent from the arc current. So you need something able to separate these.

All these problems could be pretty well solved by an oscilloscope with a clamp on current probe, but first the oscilloscope itself is not one of the cheapest instruments and second the clamp on current probes are one of the most expensive accesory for any oscilloscope, with a price tag often comparable to the price tag of the oscilloscope itself. So things not that much affordable for a hobbyist...

because most HF electronic ballasts operate outside the frequency range of most multimeters can handle, its really pretty easy to measure a HF ballast otherwise with the right tools

and Oscilloscope and a suitably specced high voltage differential probe and current clamp probe is what I use https://www.lighting-gallery.net/gallery/displayimage.php?pos=-211622


but a True RMS multimeter would have no problems measuring output of say a low frequency Electronic HID Ballast for example (just connect it up after the lamp strikes!)

Yeah, I think the main barrier around measuring lamp current on HF electronic ballasts for hobbyists is the cost of the equipment needed to measure lamp current accurately. As mentioned, a oscilloscope with the proper current probe + HV differential probe costs quite a bit more than your standard multimeter and clamp meter designed for line frequency AC.

Plus with the oscilloscope there is another pitfall, mainly affecting HF ballasts: HF ballasts are designed as HF inverter stages supplied from a DC intermediate supply. And that intermediate supply is just a rectified mains (with or without PFC, that does not make much difference) with bare minimum filtering, so there is a significant 100/120Hz ripple on it. This ripple then AM modulates the HF output of the ballast, so in odrer to asses the values accurately, you have to integrate those effects over the full ripple cycle (10/8.3 ms). So when you want to acquire the HF waveform accurately enough and at the same time average out the 100/120 Hz ripple AM modulation, the oscilloscope has to first sample at high enough frequency (at least 1MHz) and either evaluate a record with sufficient number of the ripple periods (at least 10 or so), or allow you to specify a window in which the value (rms,...) is extracted. The first requires quite a long waveform memory (at least 120k samples), the second the advanced "measurement" capability, either of which is available only in the higher end oscilloscope models.
You may get away from the need for such high end oscilloscope when you do partial measurements and then process the data. It requires you to do pretty substantial waveform processing by yourself (either manually or in Excell or so) on multiple measurements (acquire a single HF period looks like, then acquire the ripple AM envelope shape and then "recreate" how the original signal may have looked like and do the processing on that).

And another note: Just the "rms" value is not good enough, you always have to extract the real lamp power. So get both the voltage, as well as the current waveform data together at once.

And for the current probe: If you know the signals you are measuring, you may make a dedicated current probe for it, but very likely you won't be able to calibrate it completely, you have to suffice with calibrating some components and trust the rest:
If you know the fluore3scent HF ballast does not feed any significant DC component (has a capacitor in series with the lamp,...) you may make a current transformer (this you have to trust, so be careful with the number of turns, it does not have to be exactly as I write, but you have to know the numbers) by winding about 2x 10 turn primary (by some plastic insulated wire; wind this with both wires at the same time) and a 100 turns (some thin magnet wire) secondary over a ferrite toroid core, connect the primary in series with the lamp on its cold end (primary in phase so with both corresponding ends towards the lamp if you want to measure the arc current, or out of phase when measuring the filament current), connect the secondary to an accurate 10Ohm resistor (this you may measure pretty well) and sense the voltage across that resistor by the oscilloscope. In case of the arc current, you get 1V/A (Rshunt * Nprimary/Nsecondary; as the arc current is somehow split among both wires, but as each part flows through its primary, the result sums up, representing the correct arc current), for the filament current you get 2V/A (Rshunt * 2 * Nprimary/Nsecondary; as the filament current passes in phase through both primaries, so their effect sums up). You may reduce the number of turns, but be careful about the low frequency limit of your transformer dictated by the Lsecondary/Rshunt time constant. For correct amplitude it has to be  longer than 1/Fballast, if you want a correct phase (e.g. to eliminate reactive currents from the power calculation), it should be more than 10/Fballast.