The rhythm, beat, lyrics and the originality.
If I don't like the lyrics or vocals. I will just remove the vocals and turn the song into an Instrumental one.

I kinda love bands who uses very interesting instruments on their music especially the violin and the trumpet.