Al-Lu'lu Wal-Marjan 850 — The Book of Hajj (Pilgrimage to Makka)

Narrated Ibn ‘Abbâs رضي الله عنهما that he heard the Prophet صلى الله عليه وسلم saying, “It is not permissible for a man to be alone with a woman, and no lady should travel except with a Mahram (i.e. her husband or a person whom she cannot marry in any case forever; e.g. her father, brother, etc.).” Then a man got up and said, “O Allâh’s Messenger! I have been enlisted in the army for such and such Ghazwa and my wife is proceeding for Hajj ." Allâh's Messenger ﷺ said, "Go, and perform the Hajj with your wife." ( Sahîh Al-Bukhârî, Hadîth No. 250, Vol. 4 )