Experience and training is the answer. The way anyone learns to deal with stressful situations is to have to deal with them on a regular basis. After the first, second, third, and more times that a police officer, fire fighter, EMT, public speaker, doctor, lawyer, etc. has to deal with stressful situations, it seems to them that they will never get used to the dealing with them. However, their ability to deal with them will gradually get better and may even get to be routine—but the fear and anxiety never goes away. If it does, you will become complacent and probably do or not do something that may cause you or someone else to die. Fear is what keeps most people out of trouble. For ordinary people, facing down a threatening person will not occur very often, so they do not get much experience at it, but training for it, such as in the martial arts, will help.
In your case, it appears that a fight did not occur. That may have been because of your martial arts training. Even though you may have felt afraid, your demeanor was probably displaying confidence and the ability to handle the situation. Unless he was hardened criminal, the other guy was probably having to deal with the same bodily reactions that your were. He was counting you cowering in the face of his threats. When you did not, he just kept the verbal threats going for a while and then left. He probably sensed that you would be able to defeat him and it was not in his best interest to pursue the confrontation any further.