That reminds me why it is a bad idea to even think about dimmable LED on my bedroom dimmer. When the light is on the dial is illuminated on the dimmer, when the light is off both the dial and an additional guide light strip (extra part, but I put in in just in case the dial didn't illuminate when off) and those LED circuits would cause this type of leakage. Better get some 53 watt or 72 watt GLS halogens for the light level upgrade instead.
With the LED's it depend on the ballast design. The series resistance types (ACRiche,...) are perfectly OK, they would just dimly glow.
Same behavior with LED's rated as dimmable: They would either glow dimly, or stay OFF, the ballast is (well, with good quality product) especially designed for such leakages, as these are common with practically all triac dimmers (the EMC filter)...