IMERC Fact Sheet - Mercury Use in Lighting seems to be a relevant source for amounts of mercury in various types of lamps (it contains a well-arranged table of values).
And in the appendix of this pdf you can find those amounts by wattage.
These statistics are from 2004 so I believe that at least fluorescents now have even lower mercury contents than it was 10 years ago.

Note 1: I'm for ecology but I think there are much worse problems than a mercury from EOLed CFLs and fluorescents, for example the excessive car use. I hope that a fair ratio of people in our country left their dead CFLs in dedicated boxes.
Note 2: I was a naughty child. We used to crush EOLed T12s from offices nearby (they were left in bundles next to bins - yeah, it was some 30 years ago in the socialistic Czechoslovakia). We destroyed many of them not caring that we are breathing mercury vapours. I cannot remember a least problem with anyone - no headache, no vomiting...

As long as it doesn't get inside your body somehow, I personally don't think contact with mercury is that dangerous. I could be totally wrong, though. Dad once told me that as a kid he got a hold of a handful of it and would play with it in his hands. In retrospect he very well could have accidentally made that up in his head (he tended to do that occasionally), but if it's true, it certainly didn't hurt him.

I don't know what about touching and playing with liquid mercury, but swallowing a sufficient amount of liquid mercury or inhalation a sufficient amount of mercury vapour, can cause mercury poisoning.
A large pile of mercury can cause land pollution and if it permeable to the ground water, it can poison them as well.
The amount of mercury in fluorescent and HID lamps is sufficiently small, that a breakage of a single lamp isn't dangerous, but a breakage or disposing a large amount of mercury contained lamps can be dangerous.

Most dangerous is inhalation of the mercury gas, in that form is enters the body most easily.
Swallowing a liquid is not that poisonous, probably as it stays in rather compact balls with rather small surface.
Mercury compounds are a different story, but it depend mainly in how they are soluble in water and/or fats. Those readily soluble are realy strong toxins...
But it is toxic in any form anyway...

As Medved mentions, inhalation is a danger when lamp breakage is nearby or next to you. @merc, do you or have any children with physical or neurological abnormalities?

Instead of mercury, researchers are actively looking at Indium for a UV source in fluorescent lamps. But in the future these lamps might all be inductive. Otherwise the internal electrodes are quickly attacked by the Indium halides.
http://oansuche.open-access.net/oansearch/metadata?oid=293481 (pdf at the bottom)

There are several problems with indium over mercury:
1. Indium have higher vaporising temperature than mercury as it is solid and not liquid, meaning that the lamp will take a while to run-up (much like low pressure sodium lamp). By contrast, xenon (Which was used in some special purpose CCFL as an alternative to mercury, is a gas already.
2. Indium iodide/bromide in MH lamps, like all halides, is corrosive toward the quartz. I don't know if this is the case also with glass at low pressure.
3. In indium iodide lamps, indium produces its blue and violet visible lines, much more efficiently than mercury does with its visible lines, and indium lines boardens and self-reserved at much lower pressures than mercury (I've seen area floodlights with high kelvin indium lamps, which their indium blue line had a self-reserved profile, but still didn't captured their spectra).

No matter how much we like mercury lamps, HID or fluorescent, the use of mercury in lamps will be banned, just like leaded gasoline or smoking in public places was banned. Given time, all these problems dor123 mentions will all be sorted out (more related to HID lamps than the fluorescent discussion above). Personally, I believe the environment comes first and I'll embrace anything to replace mercury in lamps that ALWAYS end up in the dump at best, or in merc's lungs as a kid!

@lights*plus: "@merc, do you or have any children with physical or neurological abnormalities?"
I don't know about any harm (if it isn't something hidden).
Fortunately, we did our mischief in the open air - aside from the bin shed (or in it - I can't remember - but it was decently ventilated anyways).
Crushing 10 pcs. of T12s could be hundred(s) of milligrams of mercury vapour (according to the table above) and we were a couple of paces away - afraid rather from glass fragments.

I didn't want to say that mercury is completely harmless. If I accidentally crushed a fluorescent today, I'd open a window and left the room for a few minutes.
Recently, my wife crushed a 40W circline when tidying up and she didn't take any precautions because I wasn't at home and didn't warn her. She was in the room, sweeping the smithereens and she is OK.
And what about the cardboard boxes for EOLed CFLs that are in our supermarkets? There is no warning if that box accidentally fells of and CFLs are shattered that the supermarket should be evacuated.

As a conclusion: You are right - better to get rid of a mercury at all but it's not as devilish as some might think.

Proper disposal at EOL is better than getting rid of 'poisonous' things altogether, as is using proper lamp design so that they last bloody ages in service.  It's like all this RoHS nonsense, where industrial control systems, military, avionics, core telecommunications backbone and medical stuff is exempt because the RoHS rubbish is unreliable and unreliability in those applications gets people killed.  Just look at all the problems with lead free solder (whiskering, all joints looking bad, brittle fracture, galvanic corrosion and separation due to incompatible metals, health effects due to much nastier and more aggressive fluxes, inferior wetting, higher temperatures cooking boards and parts, et cetera et ad infinitum) and RoHS compliant chip epoxies, which are more hygroscopic and thus have a higher moisture sensitivity level.

Hopefully all this rubbish will be dispensed with in the building pushbacks.  Those cage filament reproduction lamps becoming a staple in advertizing and decoration in the past 2 years or so is a good sign.

@indium instead of mercury:
What attack the quartz are not the halides, but mainly the (atomic) sodium. The halogens in the halides in fact allowed to use the sodium inside a quartz arctube by just attracting the sodium more than the quartz does.
 But the halides are unusable in a low pressure lamp - they require rather high temperature to get vaporized...

And I doubt anyone would spend any money on developing such thing like an induction system, when there is no indication it will outperform the present LED systems.

And for a specialty sources (like the photo-lithography processes), the mercury will be still available to use...

Problems of substitutes for mercury, as discussed by Max and others, indicates both the possibility of abandoning traditional fluorescent lamps altogether and going to electrodeless systems, or just simply investing into semiconductor technology. We have many ways to produce copious amount of light today (what happened to microwave excited sulfur lamps?). I'd like to live long enough, 30 years or more from now, to see what system will win out. My hunch is electronically controlled semiconductor lighting will win for all indoor home-office applications and for street lighting. It's already happening. Oh, and to prevent light-induced cancer (such as breast, via the suppression of melatonin production) a certain part of the blue spectrum will HAVE to be eliminated (electronically, or with filter). See you there!

If anything would win, it will have to be cheap and reasonable light quality.
And this "cheap" does not limit only to cheap to buy, but include energy and maintenance costs.
Because applications differ, each use has different figures associated with each of the components (for a battery lantern the costlier component is the battery, so the energy, while with some warning light the costlier part is the eventual light malfunctioning), something different for a decoration used once for short time (so the purchase/disposal cost become the dominant part).
And the "reasonable quality" is different for a street lighting and something different for home lighting.

Indeed, it seems the LED's are flexible enough to be optimized for each of the task to perform well, in some places their performance is highly questionable.

There is an international treaty aimed to ban the use of mercury from use:

http://en.m.wikipedia.org/wiki/Minamata_Convention_on_Mercury

More bad news for conventional lighting technologies.

If this treaty would be implemented with the aim to really reduce the mercury permission as much as feasible in a given economic environment and after real deep analysis of all consequences of certain rule making, there would not be things like bulb bans. Not because the CFLS do contain mercury and incandescents not (there the incandescents are way worse than the CFL's in an average use, just because the extra energy needed for the incandescent means way more mercury emission than is inside that CFL). But the energy itself would get quite significant tax on it, so it would become quite expensive.
And that will be the most efficient way to make sure people rethink it's consumption and invest on places where it will bring the most energy savings.

But what we are seeing with all the bans is just a public relations bribe: Big campaign, how the move to energy efficient lighting will save so huge amount of energy when summed from millions of users, the figures look astronomic for many people, so they swallow that bait very quickly. So politicians just shake their hands in front of cameras, how their bills protect the environment, win in next elections guaranteed.

But energy taxes are always detrimental to society and are especially injurious to those who can least afford it.  While so much hand waving is done concerning lighting, there are things like motors and heating where a certain workload needs a certain amount of electricity, and it is things like this where these drives to increase energy costs are killing people in the literal sense of the word.

It is only through access to cheap energy that people have the option of thinking about things like the environment.  Where most income goes toward energy (as it is in the third world for things like kerosene and candles for light and food for manual labor), any increase in energy costs is disastrous.  Not to mention the vast increases in per capita economic productivity that cheap energy allows.