Mind and matter are often confused on account of Czechs having just one word for both. In English, it is different.
Mind is a verb meaning to have a problem with something. If you say, „I mind you arriving late“ it means that you are not happy about it. It is connected with a personal pronoun – I, you, he/she, we or they. It is not generally used with ‘it’.
Matter, on the other hand, is almost always connected with ‘it’. The verb ‘matter’ means that something is important. So you might say, „Your decision matters to me.“ Or „It doesn’t matter if you come late.“
If you use ‘matter’ with a personal pronoun it sounds emotional. If you say to your girlfriend, „You matter to me“ it expresses that she is very important in your life.
Let’s put the two together:
A: Does it matter if I come later?
B: No, I don’t mind at all. Come when you want.