You shouldn't just spend time with your family just because you are there and its a good time of spending time with them. It's about putting them first before anything and enjoying your moments with them.
I think it means that no matter how much time you spend with your family it will never make you feel better but worse and if you focus on things that you like it can improve your mind.