It doesn't hit its mark, that brutal strike, because the man he lashes out at is not the boy whom he contracted with. Dorian lets go, but only for the moment. Only because he can't hunt down Gilgamesh with his full resources when Hermione is near. Dorian lets go and releases Gilgamesh to flee, but he makes it clear:

I do not reject you. I accept and embrace you. Always. Rejection? Dorian has faced rejection. And he has changed its mind, over and over and over. I will call for you. And if you don't come, then I will come for you. My King of Kings, don't think you can shake me off so easily. I am not the man you chose because I surrender so easily as that.

And yet when Dorian looks at Hermione, he is almost himself again. The Dorian that Hermione knows. "Don't put it on yourself, Hermione. Frankly, I can't hold for him something he'll never give me, and no one should have to abide by what he demands." Except Dorian. With a brightened smile, he takes her hand. "Take me inside before I swoon? Or people won't believe I'm not a Victorian anymore."