Yishai Schlissel, an ultra-Orthodox Jew, emerged behind marchers, drew a knife and begun screaming and stabbing.
The suspect has been identified as the same man who stabbed three people at the parade in 2005, being sentenced to 12 years in prison. He was released just three weeks ago.
Yasmin Yusupov, 20, told the BBC she and her friends were marching when she "started seeing a lot of people running."
"We didn't realise what happened but I was pulling the friends who were with me to run away. We just banged on doors of people's houses. We tried asking them to get us inside.
"When we came back I saw a few bodies on the floor right in front of me. Everyone had blood on their hands because they were trying to help," she added.
Hundreds of police had been stationed along the parade route to ensure such an incident did not occur. Jerusalem police spokesman Asi Ahroni told Reuters there was a "massive presence" of police at the parade but "unfortunately the man managed to pull out a knife and attack".
Israeli Prime Minister Benjamin Netanyah described the attack as "a most serious incident" while President Reuven Rivlin condemned the stabbing as a "terrible hate crime."
Israel has a fairly accepting approach to homosexuals, the Jewish state repealed a ban on consensual same-sex sexual acts in 1988.
However, elements of the ultra-Orthodox community remain hostile. In 2009 two were killed and 15 wounded when a gunman opened fire at a centre for young gays.