When we feel empathy, we typically give power to another person or institution. We identify with — and internalize — their point of view. In extreme cases, this can lead to the kind of loss of self we associate with Stockholm Syndrome, where a hostage over-identifies with his or her captor. Empathy makes us vulnerable to subordinating ourselves to others, especially to those in power.
Of course the lack of empathy hurts those without power the most because when you do have power those under you are forced to empathize with you or be crushed.