Personally, I'm not a fan of people who sing about murdering their ex-wife and having their little girl help them dump her body in the river. Also, the song about wanting to go shoot more kids at Columbine... not ok. But that's just me... Other than that, his lyrics ARE incredible and his songs ARE definitely gripping.
Heh, you really need to listen to his last two albums, then. Not that they are better... but pretty much all of his albums have stuff like that. He doesn't even pretend to have respect for other people in some songs, in others he goes on about how he really cares for them. Eminem kind of writes about whatever is going on in his life... the last two albums have been crazy pictures of the tumult that he's in the midst, of, fighting addictions and desires, relationships, wanting to be better but feeling powerless to be better, et cetera.
Listening to all of his albums chronologically will reveal that his different albums reflect different phases of his life... ones soon after major hurts are hurt and angry. When he feels like he needs to defend himself, his albums are aggressive and careless, when he wants to improve his songs are more apologetic and hopeful.
I guess I like him because he expresses what he feels, and effectively. It might not be tasteful, but it's straight-forward.