I only use the metronome to record my voice to avoid picking up sounds from other instruments that you hear while recording. Till now the voice has been the only instrument I ever recorded with a microphone, for bass and guitar I always have plugged into my sound card, since I don't own amplifiers yet.
As I'm a one man band, I have never recorded drums and instead I used drum samples to base most of my songs on. I always take care the drum samples I use (existing beats) have a fine beat to play to (in the right tempo/bpm) and so I don't need a metronome to play to. Those beats (I import them on an audio track in my recording programm Logic and I can give those samples the speed I want) give me lots of energy, despite the fact they go on and on without a change. The only variation I made so far was to have some fills here and there by
1. other samples (audio files), for example fills
2. midi (having attached sounds like kick/snare/crash to keys on a keyboard device).
That has worked for me so far. As long as the feel to the drums are good, however they can still improve, it's good while you're in the process of finishing your songs. The 'worry' is when you want to have your songs recorded for real, when every track must have it's right melody, its right rhythm and so on. Like you, I haven't reached that stage yet, but I will in a few weeks. Anticipating on that I'll be working with a real drummer who knows what a good rhythm needs: play a bit behind the beat, a bit ahead of the beat, or right on the beat when warranted, and a sense of which is best at any given moment. He will play drums on a keyboard based on the drum samples I chose to play my songs to. This may seem funny to some, but the sounds of the different drum parts are not to be distinguished from real drums.
Hope this helps.