To Moderate or not to Moderate

I've had my blog for about 4 years now and this question has always puzzled me. Should I moderate my comments or not? In the beginning, when I first started blogging, I set my comments to only allow registered users to post. This kept spamming and unsavory characters away, but it also made it hard for some of my readers to leave comments because they weren't registered users. So, about a year ago, I changed the setting to allow any and every one out there to leave comments. It worked well for awhile, but now I'm seeing that I'm getting spammers leaving comments again.

It's just frustrating. I don't want to limit those wanting to leave comments, but I'm sick and tired of getting on here and seeing things like "here's that website where I make all this money. check it out". What do you guys think? Is there anyway that you know of to moderate comments without excluding some people from leaving them in the first place? Let me know.