Before lED headlamps became more popular, it used to be you’d see HID headlamps on certain vehicles. Usually more luxury/performance brands like BMW, etc. And for American cars, would see HID on the top trims like Limited, etc models.

What the reasoning for these being on more luxurious cars, while reserving halogen headlamps for lower trims? And even the ones with HID seem to only use it for the low beams and not on high beams or fog lamps as far as I know.

Was it for appearance reasons, or do these headlamps perform better?

They DO perform better. Up to 3X light output at lower consumption compared to standard halogens. Also of course, whitish light makes the owner to stand out from the crowd of mass produced clunkers.

As can be expected with metal halides, car HID lamps don't like frequent on-off switching too, that's why xenon low-beam halogen high configurations exist.
Higher end versions have mechanically shifted optics.