That's a fair point. I choose to believe that if I'm calm and gentle then as far as the spider (or really any creature that I hold) will not see my hand as a threat but rather as a piece of landscape to traverse. Remember how when you were young the world seemed so large, not in a metaphorical sense but in a literal sense. The chairs were bigger, the bench was higher, your parents hands were huge. Imagine that from the sense of an insect. Not only would everything human be incomprehensibly gigantic, it would be indiscernible from an ordinary non-human object. As far as I choose to believe, the spiders and all insects have no concept of humans as anything more than gigantic sometime moving objects therefore they would have no reason to feel threatened except for cases of 'Holy shit, this thing is trying to squish me. I need to defend myself'.

Of course if you have something on your hands that react chemically with the spider in a way acid reacts to our skin, then it will be a different story. Although in this case it is still only what threatens the insect.