Psalm 2:11 says for us to 'rejoice with trembling'.
Fear does not rob us of our joy for two reasons. One is that it drives us to Christ where there is safety. The other is that even when we get there the part of fear that Christ relieves is the hope-destroying part. But he leaves another part—the part we want to feel forever. There is an awe or wonder or trembling in the presence of grandeur that we want to feel as long as we are sure it will not destroy us. This trembling does not compete with joy; it is part of joy. People go to terrifying movies because they know the monster cannot get into the theater. They want to be scared as long as they are safe. For some reason it feels good. This is an echo of the truth that they were made for God. There is something profoundly satisfying about being “frightened” when we cannot be hurt. It is the best when the trembling comes from the grandeur of holiness.
--John Piper