Even if it's because he's inexperienced with women, it still makes no sense. He doesn't touch his phone after work? Days off, either? You don't work with him, so about 100% of your relationship would be during his non-working time, but you don't get to spend a minute of that talking to him? You shouldn't commit any more time to him if he can't commit any of his personal time to you.