Beside the point means not related to what actually matters. That is beside the point.
It is blunter than neither here nor there and can sound impatient. It says the remark is true and irrelevant at the same time.
1phrasebeside the pointNot related to what actually matters.
“Whether he meant it is beside the point now.”
beside the point is a phrase.
Practised in Idioms