Ocalan was sentenced to death in a high-profile trial in 1999, but his sentence was commuted to a life sentence, which he is currently serving in the prison island of Imrali.
In an 11-to-6 ruling, the European judges considered that the Turkish state security court judging Ocalan was neither independent nor impartial because of the presence of a military judge on the panel.
The Strasbourg-based court considered that Ocalan "had been convicted by a court which did not meet the Convention requirements of independence and impartiality."
Thus "a retrial or a reopening of the case, if requested, represented in principle an appropriate way of redressing the violation."
Although the ruling is not binding, Turkey is a candidate to join the European Union, which urged Ankara Thursday to implement the court's decision.
A senior Turkish official said Ankara would take the necessary steps to address the flaws found by the European court.
"The Turkish Republic is a state based on the rule of law and will undertake the procedures that the law requires," Dengir Mir Mehmet Firat, deputy chairman of the Turkey's ruling Justice and Development Party (AKP), told reporters.
He stressed that the objections of the European court pertained only to procedural flaws and did not contest the essence of the verdict, which condemned Ocalan to death for treason and separatism.
"The retrial, if there is going to be one, will be only procedural," Firat said.
A lawyer for Ocalan, Marc Muller, said he was satisfied with the ruling and would consult with his client on whether to ask for a retrial.
"He will probably agree. There should be a new trial," Muller told reporters. "If Turkey wants to join the European family it has to respect the minimum rules that apply to all human beings."
Turkey's main pro-Kurdish party, the Democratic People's Party (DEHAP), welcomed the ruling, saying it might encourage Kurdish insurgents to announce a new ceasefire with Ankara.
"We believe Mr. Ocalan's retrial under universal legal norms will offer a new opportunity to discuss and resolve the Kurdish problem," DEHAP chairman Tuncer Bakirhan told a news conference.
"We believe that a retrial will mark the beginning of a new process towards lasting peace," he said.
The decision of the European court's high chamber upheld a 2003 ruling of the court and condemned Turkey for violating three articles of the European Convention on Human Rights.
The jugdes ruled unanimously that Ocalan's right to legal assistance was violated, along with his right to "adequate time and facilities for the preparation of defence," the court's registrar said in a statement.
The ruling also condemned Turkey for not bringing the Kurdish leader promptly before a judge following his arrest in February 1999.
"The Grand Chamber found that the overall effect of those difficulties taken as a whole had so restricted the rights of the defence that the principle of a fair trial ... had been contravened," the court said.
His death sentence following an unfair trial also amounted to "inhuman treatment," the court ruled.
But ruling in favor of Turkey, the judges said that Ocalan's arrest in Kenya and transfer to Turkey did not amount to ill-treatment, neither did his solitary confinement on Imrali.
Ocalan, 56, has been the sole inmate on Imrali since his conviction as leader of the outlawed Kurdistan Workers' Party (PKK) which has led a bloody armed campaign for self-rule in southeastern Turkey.
Ocalan's sentence was commuted on October 3, 2002, after Turkey scrapped the death penalty in a bid to ease its entry into the European Union.
A possible retrial for Ocalan, considered public enemy number one after leading a separatist campaign which killed 37,000 people, would be seen as a test of Turkey's resolve to embrace European standards as it prepares for membership talks with the EU on October 3.
An EU spokesman said the union expected Turkey to respect a ruling.
"Turkey is a member of the Council of Europe so it is a duty to implement all the decisions of the court," he told reporters.