Crazy unlikely things can happen, but very rarely. Indirect contact with precum has a tiny, TINY chance. I've been down this road before when I was younger and not on the pill, I understand why you are worried. Just know that it's incredibly unlikely. If you're on the pill and taking it correctly, it is basically impossible. If his hands didn't have any on it then you are perfectly fine. I think from what you described, you are fine.