Didn't read all of the thread yet, thought I would just post about the method that seemed to work for me. Have not utilized it in several years now, though, and nowadays I rarely dream lucid.
This method was prescribed in some site I have saved somewhere, Ill see if I find it later.
One should make the bedroom completely pitch-black-dark, like real good blinds in the windows, thick and black, so it really is so dark you cannot see at all in there. Also, set an alarm clock to go off sometime in the wee hours, a few hours from the moment you go to sleep (some 3-4 hrs, so you might likely be in REM sleep when it goes off). Also, every night, make the time a little different or soon you will wake yourself without the alarm at the same time, and it defeats the purpose of it.
Then when going to bed, lay awake in your bed for at least half an hour, preferably closer to an hour, and stare in the blackness with your eyes open. Start trying to visualize things, either objects if you can, say, a flower, a football, a cat, a tree, and try to visualize them so strongly that you actually see them floating in the dark. If you can't do objects, just do abstract, like colours, clouds ... I am not good at objects, but I quickly start to see floating masses of changing colors and mists.
While doing this, also try to pay very close attention to the border between "sleep" and "awake" as you drift closer to it. Try to pay attention to falling asleep, how it feels, and try to keep a thread of the awake attention as you slip over.
When the alarm goes off at night, set the clock for your actual getup time, and repeat the staring in the dark and visualizing. Doing this, it took me just a couple of weeks to start getting lucid dreams. The important part is the visualizing and paying attention to the transition to dream state, in the total darkness. Eventually, this will blend the border in your consciousness too, until you suddenly notice you retain part of usual awareness within the dream state.
Other things one might attempt are taking some valerian root, a B vitamin complex, and melatonin when you go to bed. This enhances dreams and deepens REM sleep phase, making dreams more vivid. Also, smoking / chewing - the best option / vaporizing Salvia divinorum before sleep helps, combined with the above practice. Salvia specifically because it helps or forces your mind to shut up and experience in wordless understanding, which aids in this.
Here was an account of one powerful instance of a real lucid "dream" that was actually real, that happened some years ago -viewtopic.php?p=133870#p133870
Couple of years back I had a very vivid dream about my grandmother. I was doing my summer studies, several hundred kilometers away from where my grandparents lived. I hadnt spoken to them in maybe couple of weeks at the time.
In the dream I was in their bedroom, it was dark and my grandpa was praying on his knees in the background (theyre very religious), and my grandma was near to me - telling me that shes afraid that shes gonna have a heart attack and die any time now - that she is having arrhythmia. I took her hands and repeatedly told her that its okay, its not your time to die yet, dont worry, youre not dying tonight.
All the time I had the strangest feeling that this is NOT a dream, that this is absolutely real. Also the room was exactly like I knew it to be, and dark, like at night. When I woke up I instantly "knew" that this was no ordinary dream, but was true. I explained the dream to my roommate (so as to have a way to verify to myself Im not making things up), and then called my grandmother. First thing she does is she tells me that shes been awake all night, having arrhythmia and fearing shes going to die. And that grandpa had been praying all night for her. I proceeded matter of factly to tell her about my dream, and just as matter-of-factly she thanked me for helping her.
This was the first time I had irrefutable proof that the world and our being is much stranger than fiction. The whole experience was real, down to details, and theres no way I could have imagined any of it.