The fear of going downhill is the same as the fear of anything. Whatever has caused your fear is, by now, almost irrelevant. You've told yourself so many times that you are scared of going downhill, that your brain now defaults to inducing the sensation of fear whenever prompted.
The memory of fear (or indeed any memory) is like a muscle, and you have obviously worked hard on yours and created a rather strong one! The good news is that it can also atrophy like a muscle, all you need to do is stop using it (or at least use it less). The best way to do this is make a conscious effort to replace the fear with a positive thought, set yourself an achievable goal relating to downhill; somthing like 'by the end of the year i will hammer my favourite hill as fast as my mate'. Then work on this; start off slow, just ride the hill. But make an effort not to think bad things are gonna happen, be positive. Just think how good it will feel to finish it, remind yourself that people do this all the time, remember that you are an experienced rider and have the right equipment to minimise the risk etc.
Not saying it's easy, hell i punctured a lung after a downhill crash! But, i didn't die and now i have an anecdote to tell people when they say 'isn't mountain biking dangerous?' Going fast downhill is where the adrenaline is at, and everyone likes that