It can be hurtful to hear children say such things.
My advice is as follows:
1) Find out what "love" means to your son. He may not be able to tell you directly, but you should be able to tease out his meaning. For instance, he may be saying this because he doesn't feel you spend enough time with him or that the time you do spend with him is hurried and distracted. He may say these words when you don't give him something he wants or do what he wants you to do. Remember he is defining love from a four-year old's perspective. Still, do not make the mistake of discounting his feelings ... he is trying to tell you that he needs something from you that he is not getting.
2) Spend time with him, one-on-one, giving him your full attention. Do something outdoors that does not involve television. Focus on your sense of connection with your son.
3) Read my book Keepers of the Children which will teach you how to create a close relationship with your son, how to connect and engage him, how to parent responsively and more.
If doing the above does not seem to help, I am available over the phone as a parenting coach.