But I am in a bad mood a lot and sometimes I hit/shove/threaten him first
This may very well be in response to years of his abuse. But there is also the possibility of "mutual abuse" where each partner tries to control the other one. If you recognize the profile of the batterer described above in yourself, get help!
No matter what he says you did, he has no right to hurt you