Lima
Declaration 

Durban
Commitment

ANNEXES

I am appending as Annex I a further description and explanation of my proposed checklist. The remaining annexes contain examples of others work. As should be abundantly evident from the prior discussion, the following is only a brief selection of what might be included. The first three examples constitute prior attempts at developing judicial checklists. Only one includes scores (most never got that far), and as indicated in the accompanying note, the scoring methodology is its weakest point. Nonetheless, I think it (the Blackton report card) is the best example of the genre. The fourth and fifth are draft checklists currently under development by a World Bank member and several external colleagues, and ABA/CEELI.

I have also included two judicial inventories and referenced the numerous works resulting from the FIU/ILANDU collaboration. USAID's list of indicators of judicial performance is too long to include. It is available through that agency's Global Center for Democracy and Governance. Neither the inventories nor USAID's indicators are intended as models for Transparency's proposal. However, they deserve reading if only as a reminder of the variety of topics that require coverage in a true assessment of performance.

ANNEX I: Further Explanation of Proposed Checklist Elements

Although I will not go into this in the detail that may have been expected, I am offering a slightly longer explanation and self critique of the proposed checklist offered in this paper. The obvious points for criticism are the following:

• It is still too long and too detailed
• It is not clear why the specific categories were chosen and what their linkages are to the desired results or behaviors
• The list of criteria seem somewhat arbitrarily chosen; it is not evident why these rather than others were included
• The standards against which the criteria will be evaluated are not obvious; some seem to imply a universal reference point, others a relative one, but words like "adequate" or 'sufficient" leave a lot of room for subjectivity
• There remains the problem of how this list will be used for countries with very different legal traditions or at different stages of development. Given resource and other constraints, could Liberia's score, even after a serious reform, ever be "good"?

I propose to address the criticisms and offer the further explanation simultaneously, starting with points two and three - the choice of categories and criteria. The entire list is based on one fundamental assumption, that a well performing judiciary has the same basic requirements as any organization - it needs to select the right human resources, manage them for performance, have adequate administrative systems and resources, and organize its fundamental business and operating procedures in a manner commensurate with achieving the outcome it desires. (I've added the legal profession as part of the human resource base, although giving it a separate category.) While one can take issue with each of the elements, either in general or as applied to the judiciary, or demand empirical verification, if they don't hold a good deal of the underpinnings of Western civilization also is called into question. Hence, although we could organize research to determine whether a selection system is important to the composite "good performance" (efficacy, efficiency, etc), this seems less useful than trying to determine what kind of selection system is most appropriate for choosing judges or judicial administrators.

The application of these general categories to the two other dimensions (accountability/transparency and independence) is really, as noted, an extended definition. It also follows the logic that if these are the key components of a well functioning organization, then they are also the areas where transparency and independence are most critical.

If one accepts my first arguments, than the burden of the explanation falls on the evaluation criteria. These do have an empirical base, but it rests on collective observations, the product of my and others experience with a variety of real systems in both developing and developed countries. As my own experience skews the list, this is where others comments are most important. I have selected the criteria with a problem-orientation. This is inevitable, I believe, because we have a better understanding of where a selection system or internal administrative processes may fail than what good ones should look like. This also accounts for the length, and that flaw will only be aggravated as more pens are put to the task. Possibly once a definitive working group has accumulated a much longer list, they can consolidate criteria or weed out some that seem less important. After doing so, they may want to add an "open" criterion within each major category, to allow consideration of unusual bad (or good) practices with a major influence on single systems. However, it is well to remember that the point is to provide comparable information on the most important influences on performance, not to cover every base.

This still begs the question of how the criteria will be applied - and what kinds of standards or points of reference will be used, and how differences in legal traditions and the developmental stages of individual countries will be considered. Obviously if both the United States and Senegal audit judicial budgets, the former is likely to do a more thorough job than the latter. Should the scores reflect that, or should each be evaluated against its own national standards and needs? Similarly, one would hardly want to hold Senegal to the standards of equipment and staffing for US or French courts, or to automatically give the latter two systems high grades because they can afford more, but possibly not "adequate" courtroom furnishings and personnel. Fortunately, not all the criteria pose this problem - the existence of a code of ethics and its enforcement or of set criteria for selection are probably less dependent on developmental variations. This may be still more true of those in the transparency and independence dimensions.

Blackton's use of an external reference point - a system assumed to set the high performance mark for all others - doesn't really resolve the issue. It might help on a regional basis. However, intra- regional differences can also be extreme. On at least some criteria, it would not be fair to judge Haiti or Honduras against the mark set by Costa Rica, Chile, or Uruguay. Still, the overall situation tends to support the wisdom of regionally-based grading. Not only is it fairer and less vulnerable to protests of cultural insensitivity; it also is more likely to produce useful results. Knowing how Haiti or Honduras or even Costa Rica stacks up against the United States isn't going to tell me much I don't already know. Measuring the first two against Costa Rica is likely to be more useful, as regards the less favored country and the problems still confronted by the regional standard bearer. This should not mean that the highest ranking country gets the highest possible score in all categories. There will still be a sort of ideal, if not universal standard; for each region it should take into account what is possible given the average level of development, resources, and other environmental constraints. This would of course have to be explained when the checklist is presented.

ANNEX II: JUDICIAL REPORT CARD (John Blackton, Amideast)

Note: as this is the only example for which I have a recorded score, I asked the author, John Blackton (Blackton@AMIDEAST.ORG) for an explanation for how it was done. His answer merits citing in detail:

"You are, I believe, quite right in your observation that the categories are reasonably good, but the metrics are not transparent. Taxonomy is relatively easy. Rigorous and valid measurement within that taxonomy is hard work - both intellectually and in terms of field effort. My quantification would not stand up under examination by any serious 'methodology maven.' Since I couldn't find any scales or indices from which to copy, I used an entirely intuitive process. I chose the 'report card' 1.0-4.0 model because academic grading is also more art than science and because I didn't want to convey the idea that this instrument had any pretense of statistical validity. I then decided that I would take a few areas where I know something [about] the performance in a range of developing countries….I chose two performance categories where I was reasonably familiar with other countries' performance (e.g. I used Singapore as my 4.0 standard in Judicial administration and then ranked… against this benchmark). This gave me two categories for which I had a grade with an external referent (subjective but still a referent). For the remainder of the categories I graded them against the two benchmarks since I didn't have a [good idea]…. where other countries stood……."

ANNEX III: Elements for Evaluating and Identifying Problems in Judicial Systems

1. the legal framework of the country and the role of judges within this framework
2. positions of judges in society and the perception of the system of administration of justice by the community
3. the integrity of the justice system
4. the administration of the judicial system
5. the economic cost of justice in the country
6. access to justice
7. the availability of legal information
8. legal education and training
9. the actual functioning of legal procedures
10. physical facilities of courts
11. the impact of court decision on society; and
12. alternative dispute resolution mechanisms

Source: Ibrahim Shihata, "The World Bank" in Edmundo Jarquin and Fernando Carrillo, editors, Justice Delayed: Judicial Reform in Latin America. Inter-American Development Bank, 1998, p. 120.

ANNEX IV: Checklist for Institutional Evaluation of Judiciary

[This is a working draft supplied by Richard Messick of the World Bank. The initial note, directed to the co-authors, has been left in as a good illustration of how these exercises proceed. To my own mind, this is more of an inventory than a checklist, and while the authors intend it for general application, I'm not sure how one would compare the results. I am also not convinced they have successfully tied their initial problem identification to the judiciary, especially as regards nonsurvey data. There are many extra-judicial explanations for a low incidence of contract-intensive money -- no banks, lack of familiarity with banking practices, various disincentives to putting money there. For example, Ecuador's recent introduction of a 1 percent tax on all financial transactions led to an immediate decrease in bank deposits -- 25 percent or more -- with no change in the judicial system ]

"I have reshuffled the indicators as we discussed, putting all those based on opinion, either the opinion of the public generally or the opinion of more select groups (lawyers, judges, business people, or whatever) in Part 1. As originally written, Part II had three main sections, 'Courts,' 'Private Attorneys,' and 'Prosecutors,' and 'Courts' had four subparts. Per Antoine's suggestion, I have put the sections on lawyers and prosecutors under one of the subparts for Courts, "Competence of Personnel." We should probably also move some of the material on prosecutors, particularly the portions on independence, to other subparts. Also, there are two quantitative indicators from Part 1, homicides per 100,000 and the prison measures that Santos is to supply, that do not fit under any of the four divisions in Part 2. Perhaps we should rename the third one, now called simply 'Efficiency,' 'Efficiency and Effectiveness'?"

Assessing the Performance of the Judicial System
Using Opinion and Quantitative Data

Opinions About the Performance of the Judicial System

A. Deter Wrongful Conduct

Public Opinion Measures --

1. Percent of population fearful of crime
2. Percent of population expressing confidence in state's ability to protect them from crime
3. Percent of population willing to report crime to authorities

   Opinions of Key Informants --

4. Degree to which crime and theft are obstacle to conducting business

II. Facilitate Voluntary Exchange

Opinions of Key Informants --

1. Predictability of judicial decisions
2. Bank/finance company use of secured credit

III. Resolve Private Disputes

Public Opinion Measures --

1. Percent of population that has submitted private dispute to court system in past 5 years
2. Percent of above expressing satisfaction with how dispute handled
3. Percent of above reporting dispute resolved timely
4. Percent of above reporting dispute resolved at reasonable cost
5. Citizen perception of courts' ability to resolve private dispute impartially
6. Citizen perception of courts' ability to resolve private dispute timely
7. Citizen perception of courts' ability to resolve private dispute at reasonable cost
8. Percent of population that has submitted dispute to some form of alternative dispute resolution mechanism within the past five years. [labor or family say?]
9. Percent of those expressing satisfaction with ADR

IV. Redress Abuses of Power

Public Opinion Measures --

1. Percent of population that has brought a case against a government entity (including police) in a court or tribunal [better definition needed?] in past 5 years
2. Percent of above expressing satisfaction with how case handled
3. Percent reporting case resolved timely
4. Percent reporting case resolved at reasonable cost
5. Citizen perception of courts' ability to resolve dispute with government impartially
6. Citizen perception of courts' ability to resolve dispute with government timely
7. Citizen perception of courts' ability to resolve dispute with government at reasonable cost
8. Parliamentarians' perceptions of courts effectiveness in redressing executive abuses of power [split between government and non-government party members]
9. Government administrators' perceptions of courts effectiveness in redressing executive abuses of power
10. Human rights activists' perceptions of courts effectiveness in redressing executive abuses of power

Quantitative Measures of the Performance of a Judicial System

I) Courts

Information would be gathered on four dimensions. While some of the quantitative data would call for significant collection effort, in many cases a close approximation can be had by asking either a sample of significant users of the system (banks, government agencies, etc.) or informed observers (lawyers, court personnel, media).

A.- Independence and Acountability:

1) Indicadores de independencia política (independence of other braches of governement):

1.1.- El gobierno de la Justicia: ¿a quien corresponde? (¿Ministerio de Justicia? ¿Consejo de la Magistratura o similar? Funciones respectivas).

1.2.- Selección de jueces: ¿cómo se realiza? (Quien fija las pruebas, quien nombra al tribunal examinador).

1.3.- El nombramiento y promoción de los jueces: [removal] ¿a quien corresponde?

1.4.- El presupuesto de la Justicia: ¿quién lo determina? ¿quién lo administra?

1.5.- Demandas contra la policía, el ejército o las fuerzas de seguridad:

1.5 a.- ¿quién y cuando puede presentarlas?
1.5 b.- ¿quién las examina y resuelve?
1.5 c.- ¿con qué resultado?

1.6.- ¿Qué imagen de independencia de la Justicia respecto del Gobierno prevalece entre...
1.6 a: la ciudadanía
1.6 b: los profesionales jurídicos
1.6 c: [internal?]

1.7.- ¿Existen informaciones cualificadas y creíbles (reportajes de prensa, informaciones de expertos, etc...) acerca de quiebras de la independencia judicial por presiones políticas? ¿Qué casos, con qué frecuencia, con qué consecuencias?

1.8.- Los jueces y la política: ¿pueden militar en partidos políticos? ¿Pueden presentarse a elecciones políticas: en qué condiciones y con qué limitaciones? 1.9.- La configuración del Ministerio Público:

1.9 a.- ¿Forma parte del sistema judicial o pertenece a la esfera gubernamental?
1.9 b.-Selección, nombramiento y ascenso de sus componentes.

2) Indicadores de independencia respecto de grupos sociales y económicos:

2.1.- Estabilidad y amovibilidad de los jueces:
2.1 a.- ¿Todos los jueces son profesionales (ie., permanentes)?
2.1 b.- ¿Cuál es la mecánica de los traslados y ascensos?

2.2.- Asociacionismo/sindicalismo judicial:¿existe? ¿Con qué características? ¿Con qué grado de afiliación?

2.3.- Casos de corrupción (soborno/cohecho/prevaricación): número de casos oficialmente reconocidos en los últimos 5 años.

2.4.- Quien denuncia, investiga y sanciona casos de corrupción judicial

2.5.- Imagen predominante de los jueces, en cuanto a grado de corrupción/honestidad (actual, en comparación con cinco años antes) entre:
2.5 a.- la población general
2.5 b.- los usuarios de la justicia
2.5 c.-informantes cualificados (profesionales jurídicos, periodistas)

2.6.- Jubilación de los jueces: edad de jubilación (¿es posible jubilación anticipada? ¿quién la decide? ¿en qué condiciones?; situación económica de la jubilación.

2.7.- El prestigio social de los jueces ("auctoritas"): [by level]
2.7 a: en opinión de los propios jueces
2.7 b: según la ciudadanía
2.7 c: según informantes cualificados.

2.8.-Grado de confianza que, en conjunto, los jueces inspiran a la ciudadanía.

3) How does the compensation of judges and other court personnel compare with similarly situated individuals in the private sectors?

Sub-questions: What is the annual salary of a judge of a first instance court and how does this amount compare with either the minimum wage, the average wage, or the average salary for lawyers in the country? What is the annual salary of the lowest level clerk in a first instance court? How does this amount compare with the minimum wage or the average wage?

RICK's Note: Some of the questions in this section belong in Part I, opinion data.

3.- ¿Ante quién y como rinden cuentas los Jueces? (Acountability?

3.1 ¿Ante quién rinde cuentas de su actuación el sistema judicial ("el poder judicial") en su conjunto?
3.2 ¿Deben rendir cuentas de su gestión (asuntos recibidos, asuntos resueltos, etc..) los distintos tribunales? ¿Ante quien?
3.3 ¿Existen estadísticas públicas sobre la actividad de los tribunales? ¿Quién las publica? ¿con que´grado de detalle y fiabilidad?
3.4 ¿Quién y como juzga los delitos cometidos por jueces?
3.5 ¿Quién y como juzga los casos civiles en que una de las partes es un juez?

B. Competence of Personnel

a) Court Personnel

1. How are judges selected, promoted and disciplined?

   Sub-questions: What type of training is required of individuals before they become tenured judges? Is in-service training required before promotion? What methods are there for disciplining judges? How many judges have been penalized under these procedures in the past 5 years? How many have resigned to avoid disciplinary action in the past 5 years?

2. Is there a civil service-like system for hiring and promoting court personnel?

b) Private Attorneys/Notaries

1. Is the market for legal services competitive?

   Sub-questions: How many lawyers in active private practice are there per 100,000? Notaries? Are there restrictions on entry into the legal or notarial profession? What are the criteria? Where an examination is required, what percentage of applicants pass the exam on the first try? Subsequent tries? Are legal fees regulated by law? By a professional association? Are lawyers and notaries permitted to advertise?

2. How are lawyers and notaries held accountable for their performance?

   Sub-questions: Are there ethical rules governing the practice of law? Who sets these rules? Who enforces them? How many lawyer/notaries have been disciplined in the past 12 months for violating these rules? Do courts have the power to hold lawyers in contempt for failure to appear or failing to file a pleading? Is this power regularly exercised?

c) Public Prosecutors

1. What is the number of individuals with power to charge per 100,000? Number of individuals responsible for prosecuting cases per 100,000?

2. What checks exist on decisions made by these individuals? [move to independence and accountability section?]

   Sub-questions: Is the decision to charge recognized as discretionary? Where it is, are there guidelines for how this discretion is to be exercised? Are these guidelines public? Is there a system of hierarchical responsibility for those with the power to charge? Who supervises the department/agency that employs these individuals? How is this official appointed and to whom is this official accountable? Where prosecution and investigation separate, same questions for prosecutors.

3. Are those with the power to charge/prosecute able to exercise independent judgement? [move to independence and accountability section?]

   Sub-questions: What is the annual salary of an entry level prosecutor/charger and how does this amount compare with either the minimum wage, the average wage, or the average salary for lawyers in the country? Are prosecutors/chargers tenured? For how long? What percent of prosecutors/chargers are serving under an exception to the normal tenure rules? Can prosecutors/chargers be transferred without their consent? Is their compensation, including pension, fixed while they are in office? How are prosecutors/chargers selected and promoted?

4. Do lawyers, judges, and executive branch personnel believe the decision to charge/prosecute is more or less influenced by politics than it was five years ago? [move to opinion part]

C. Efficiency [and Effectiveness?]

1) What percentage of court's time is spent on non-court matters?

2) Number of Civil [define precisely] Cases Filed per year/Number of Cases Disposed per year [sometimes called the Cappelletti-Clark index]

3) Reversal rates/Predictability: What percentage of cases are appealed from first to second instance courts? What percentage of those appealed are affirmed? What percentage of cases are attacked collaterally? What percentage of these are affirmed? Where the rules provide an abbreviated procedure for bringing certain types of claims (amparo, special writ, etc.), what percentage of such cases are rejected on procedural grounds? Where a final judgement has been entered invalidating the action of an executive agency, what is the scope of the court's ruling? Does it invalidate the action in all instances? Or only with respect to the party bringing the action?

4) Who uses the courts and for what purposes?

Sub-questions: What are the five most common type of non-criminal cases heard by first instance courts? What percentage of the total docket are accounted for by these cases? Who are the most frequent users of the non-criminal first instance courts? For what kinds of cases? What is the most common outcome?

5) Are the private and social costs of litigation aligned?

6) Are the courts able to enforce debt contracts at a reasonable cost?

Sub-questions: What is the average length of time it takes a large bank to recover a debt from a medium sized, solvent, enterprise that has no substantive defense? How many lawyer hours are required? Is interest available from the time the debt arose? From entry of judgement in the first instance court? At what rate?

7) Is the highest court/courts hearing appeals in civil matters able to set policy and rationalize the law?

Sub-questions: How many cases did the highest court hear last year? How many judges does it have? Does the court have any discretion in deciding what cases to accept?

8) Are the courts able to complement the work of the police and the prosecution?

Sub-questions: What is the average time from detention to judgement in a case involving a serious crime (homicide, rape, serious assault)? What percentage of the total prisoner population awaits trial? What percentage has not appeared before any court?

D. Access

1) How many judges sit for court outside the national capital per 100,000? For federal states, same question per state or provincial capitals.

2) Are citizens able to determine what their rights and duties are under the law?

Sub-questions: Are statute laws and secondary legislation/administrative norms regularly published? Are copies of these laws distributed to courts outside the capital? Have the criminal laws been codified? Those affecting the family (inheritance, child custody, divorce)? Have booklets, information brochures, or other material explaining the law in simplified terms available? Is there an information office/kiosk or other means for informing citizens about the law in first instance court houses in rural areas?

3) In those areas where large numbers of residents do not speak the national language, are court interpreters available?

4) Are methods of alternative dispute resolution available?

Sub-questions: Is there a law permitting disputes to be resolved using alternative dispute resolution methods? Is it considered adequate by lawyers and litigants? Are there informal means (panchayats, community councils, jueces de paz) for settling family matters, land disputes, and other cases involving the poor? Are they biased against identifiable groups (women, the landless, etc.)?

5) Does the state furnish indigent criminal defendants with a lawyer?

ANNEX V: USAID Checklist for Eastern Europe and the Former Soviet Union (Matt Mosner, USAID/ENI)

[I do not know whether the list was ever applied; it comes closer to a checklist than the foregoing example, but the individual questions are extremely broad. There is no indication of whether a quantitative scoring system was intended]

1. The Courts are independent and coequal with other branches of government

1. The judicial branch is independently funded, organized and administered, and has complete control over its own budget;
2. Judicial selection and promotion is based on merit;
3. judicial salaries are on a par with those in the private sector for practicing lawyers;
4. judges, lawyers and the general public have open, unimpeded access to all official, codified, written versions of the law, as they enter into force. As a corollary, the process of law and rule-making by either the court, the legislature, government or the executive is fair, open, transparent, and accessible to all, with opportunity for public access, comment and participation;
5. judges are regularly trained or given access to new judicial practices and procedures and new and/or evolving laws;
6. judges, court officers, lawyers, legislators, and other government officials are governed by written codes of ethics, which in practice govern their behavior, and which codes are enforced by active investigative and enforcement authorities.

2. A constitution and the relevant civil, commercial and criminal laws have the force of law, and support democratic processes and market reforms

1. A new constitution is in place that supports the separation of powers and democratic governance;
2. Democratically elected federal and regional legislatures have been elected that have the power to enact legislation that is responsive to democratic society. These legislatures are responsible to their electorate, conduct business openly, utilize public hearings, base decisions on a written record that is accessible to the general population, etc.;
3. the legislature is the sole authority to enact legislation (instead of the executive acting by decree) and has passed new laws supporting market economy in areas such as civil, commercial, and criminal law.

3. A viable legal profession exists that helps organize the training lf law students, the ongoing training of practicing professionals, and that can regulate its own ethical and professional conduct.

1. Legal, educational establishments are in place, open to all, funded by a mixture of private and public money, which offer students training in all areas of the law relevant to supporting a market economy;
2. The legal profession provides its members with ongoing training and access to information about evolving law and governs itself by a code of ethics and professional responsibility that works in practice to enforce those rules and keep the public trust in legal institutions;
3. Law enforcement officers, prosecutors, and investigators are governed by the same code of ethics, as the general bar, and are subject to the same disciplinary rules and regulations and sanctions as other legal professionals and general citizenry.

4. Laws are applied and enforced consistent with the written language of those laws and the expressed intent of the legislatures that drafted and passed them.

1. The legislative history of any law is based on an official written record that is accessible to the general public and kept as an official, enforceable government record;
2. Judges and other empowered government officials enforce the law as it is written, and are sanctioned under force of law, for their failure to do so;
3. Modernized court and trial procedures, including case management and court statistical and tracking systems are in place
4. Court decisions are based only on the written law accessible to the general public, and those decisions are themselves reflected in written decisions, after a written record of proceedings is taken that provides the basis for appeal, and public access to all aspects of the proceedings;
5. Criminal proceedings are prosecuted only in accordance with international standards of human rights, the relevant national constitution, and all other local laws;
6. Criminal trials are also governed by rules of evidence and rules of procedured governing prosecutorial behavior, requiring the state to provide defendants with notice of the charges against him, the right to counsel, access to the state's evidence, the right to bail and pre-trial release, etc;

5. The general population is aware of the laws, accepts their applicability, acts in accordance with them.

1. The legal system successfully operates to prevent/punish violations of the law by public officials, criminals, etc;
2. Citizens accept the importance and value of equal rights for all;
3. Citizens expect the timely and impartial application of all laws;
4. Citizens have a general understanding of their legal rights, know how to exercise them, and are familiar with the general elements;
5. To the extent that violations of the law and crimes take place, the public interest groups, specific groups NGO's and the media each have the right and the ability to voice and effect change concerning the legal system.

ANNEX VI: ABA/CEELI Checklist on Judicial Independence

[This is a work in progress, and along with Blackton's report card, comes closest to what the working group appears to want. It does include a scoring system, suggesting that countries will receive an overall ranking. Although the topic is judicial independence, the contents address a much wider range of issues]

9/20/99 Draft
A METHODOLOGY FOR MEASURING JUDICIAL INDEPENDENCE*
© ABA/CEELI 1999

INTRODUCTION

The following is a survey, developed by the American Bar Association's Central and East European Law Initiative (CEELI), designed to quantify the independence of the judiciary in any given country. An independent judiciary is widely perceived as an essential component of democracy, and in an era where the countries of Central Europe and the former Soviet Union are undergoing a difficult and lengthy transition to democracy, CEELI determined that it was appropriate to develop a methodology for measuring judicial independence in order to help CEELI, its funders, and the emerging democracies themselves better target judicial reform programs.

0 Defining Judicial Independence

CEELI developed this survey well aware of the obstacles to quantitatively measuring judicial independence. First, there is no simple, universally accepted definition of "judicial independence." "[J]udicial independence may be one of the least understood concepts in the fields of political science and law. . . . This is especially true in reference to judicial independence during the democratization process; its manifestations, limitations, and meaning, as well as fostering greater autonomy among a transitional country's judges, have not been thoroughly explored." Larkins, "Judicial Independence and Democratization: A Theoretical and Conceptual Analysis," 44 Am. J. Comp. L. 605, 607 (1996); see also Tacha, "Independence of the Judiciary for the Third Century," 46 Mercer L. Rev. 645, 645 (1995) ("Although the general principle of judicial independence enjoys broad support, its definition is elusive.").

Nevertheless, some definitions of judicial independence have been reached. Most American commentators note that judicial independence has both institutional and individual components. "In its institutional form, it is a corollary to the principle of separation of powers. That is, the judiciary is a vital branch of government with constitutionally delegated powers, and . . . must be free to act and interact with the other two branches. . . . In its individualized sense, judicial independence means simply that a life-tenured federal judge is free to decide cases in a wholly impartial manner." Tacha, supra, at 645 - 46; see also Abrahamson, "Remarks Before the American Bar Association Commission on Separation of Powers and Judicial Independence," 12 St. John's J. Legal Comment, 69, 70 (1996) (describing institutional and individual components of judicial independence); Fiss, "The Limits of Judicial Independence," 25 U. Miami Inter-Am. L. Rev. 57, 58 - 59 (1993) (noting that judicial independence requires both independence from the parties in litigation, individual autonomy, and political insularity). Larkins builds on this concept to state that "[j]udicial independence refers to the existence of judges who are not manipulated for political gain, who are impartial toward the parties of a dispute, and who form a judicial branch which has the power as an institution to regulate the legality of government behavior, enact 'neutral' justice, and determine significant constitutional and legal values." Larkins, supra, at 611; see also Cohen, "The Chinese Communist Party and 'Judicial Independence:' 1949 - 1959," 82 Harv. L. Rev. 967, 973 (1969) (at a minimum, judicial independence means "that political organs will not interfere with the application of these legal sources (constitutions, statutes, regulations, rules of decisions, and other sources of authority) to the facts of particular cases. . . . [I]t should also mean that political organs will not inflict deprivation upon honest judges who make undesired decisions, nor reward those who make favored decisions."); Plank, "The Essential Elements of Judicial Independence and the Experience of Pre-Soviet Russia," 5 Wm. & Mary Bill Rts. J. 1, 8 (1996) (the essential elements of judicial independence are "(1) guarantee of a fixed tenure, subject to a limited process of removal or discipline for misconduct or disability; (2) fixed and adequate compensation; (3) sufficiently high minimum qualifications in education and experience; and (4) limited judicial immunity.").

1 Quantifying Judicial Independence

Second, the concept of an independent judiciary inherently tends towards the qualitative and cannot be measured simply by counting the number of judges or courtrooms in a country. It is difficult to find and interpret "evidence of impartiality, insularity, and the scope of a judiciary's authority as an institution." Larkins, supra, at 611 et seq. Larkins cites the following faults in prior efforts to measure judicial independence: "(1) the reliance on formal indicators of judicial independence which do not match reality, (2) the dearth of appropriate information on the courts which is common to comparative judicial studies, (3) the difficulties inherent in interpreting the significance of judicial outcomes, or (4) the arbitrary nature of assigning a numerical score to some attributes of judicial independence." Id. at 615. Larkins goes on to specifically criticize a 1975 study by David S. Clark which sought to numerically measure the autonomy of Latin American Supreme Courts. In developing his "judicial effectiveness score," Clark included such indicators as tenure guarantees, method of removal, method of appointment, and salary guarantees. Clark, "Judicial Protection of the Constitution in Latin America," 2 Hastings Const. L. Q. 405 - 442 (1975). "The problem, though, is that these formal indicators of judicial independence often did not conform to reality. For example, although Argentine justices had tenure guarantees, the Supreme Court had already been purged at least five times since the 1940s. By including these factors, Clark overstated . . . the independence of some countries' courts, placing such dependent courts as Brazil's ahead of Costa Rica's, the country which is almost universally seen as having the most independent judicial branch in Latin America." Larkins, supra, at 615.

Third, reliance on subjective rather than objective criteria may be equally susceptible to criticism. E.g., Larkins, supra, at 618 (critiquing methodology which consisted of polling 84 social scientists regarding Latin American courts as little more than hearsay). Moreover, one cannot necessarily obtain reliable information by interviewing judges: "[j]udges are not likely to admit that they came to a certain conclusion because they were pressured by a certain actor; instead, they are apt to hide their lack of autonomy." Larkins, supra, at 616.

2 CEELI's Methodology

CEELI sought to address these criticisms by including both subjective and objective criteria, and by basing the criteria for an independent judiciary on some fundamental international norms such as those set out in the United Nations Basic Principles on the Independence of the Judiciary; Council of Europe Recommendation R(94)12 "On the Independence, Efficiency, and Role of Judges;" and Council of Europe, the European Charter on the Statute for Judges. Reference was also made to a Concept Paper on Judicial Independence prepared by CEELI and criteria used by the International Association of Judges in evaluating membership applications.

Following these norms, CEELI compiled a series of 33 statements setting forth the characteristics of an independent judiciary. Each statement or series of statements is followed by a brief commentary citing the basis for the statement and discussing the importance of those criteria and why it was included. A particular effort was made not to give higher regard to American as opposed to European concepts of judicial independence. The categories considered include selection and appointment procedures, education and training, budget and salary allocations, safeguards from improper outside influences, jurisdiction and judicial powers, transparency of process, ethics, work conditions, assignment of cases, and support for the judiciary by non-governmental organizations.

The methodology for scoring is as follows. Each statement is be allocated a value ranging from 5 to 1. Where the statement most corresponds to the reality in any given country, the country is given a score of 5 for that statement (thus, if all of the statements were true in the country being polled, that country would have a very independent judiciary). On the other hand, if the statement is not at all representative of the conditions in that country, it is given a 1. Conditions that fall between these extremes are to be given an appropriate intermediate "score." Based upon its total numerical score, the country will then fall into one of four categories: "very independent" (highest numerical score, 134 - 165 range); "somewhat independent" (100 - 133 range); "somewhat controlled" (67 - 99 range); and "controlled judiciary" (lowest numerical score, 33 - 66 range). Cf. Cohen, supra, at 972 (suggesting that the degree of judicial independence exists on a continuum from "a completely unfettered judiciary to one that is completely subservient.").

In the interest of simplicity and despite the potential for criticism, we have decided against giving different questions different values or weights; instead, all questions have been given equal weight. This problem has been partially addressed by including more statements in the more important categories; for example, there are eight statements relating to potential improper influences and only two statements relating to education and training.

Some of the subjective criteria would best be ascertained through public opinion polls or through extensive interviews of lawyers and court personnel. Aware of cost and time constraints, however, we decided to address these issues so that they could generally be answered by questioning some judges, journalists, and outside observers knowledgeable concerning the judicial system. Overall, the survey is intended to be implemented by one or more individuals who are generally familiar with the country and region and who would gather the objective information and conduct the interviews necessary to reach a scoring decision as to the more subjective criteria. The final scoring report would also briefly explain how the "scoring" decisions for each statement were reached.

One of the purposes of the survey is to help CEELI and its primary funder, the United States Agency for International Development (USAID), as well as other organizations working in this area, such as the World Bank, determine the efficacy of their judicial reform programs and help to target future assistance. Many of the issues raised (such as judicial salaries and improper outside influences), of course, cannot be directly and effectively addressed by outside providers of assistance. And we recognize as well that those areas of judicial independence that can be addressed by outsiders, such as judicial training, may not be the most important. Having the most exquisitely educated cadre of judges in the world is no guarantee of an independent judiciary; and yet, an independent judiciary does need to be well trained. Moreover, the nexus between outside assistance and the country's judiciary may be tenuous, at best: building a truly independent judiciary requires real political will and dedication on the part of the reforming country. Nevertheless, some sort of objective, quantifiable measurement is necessary to better focus reform efforts, and to provide a means of comparison between different countries.

The real test of the utility of the system will be in its application. CEELI is currently planning to test the survey against four judiciaries: two in western democracies and two in countries in transition. The survey may be revised depending on the perceived accuracy of these "trial runs" or on any practical difficulties of implementation.

Finally, it should be noted that although this survey has been developed by ABA/CEELI, it has not been officially sanctioned by the Board of Governors of the American Bar Association, and the ABA does not intend, through this mechanism, to officially rank the judiciaries of the various states in the United States or of other countries. Rather, as discussed above, CEELI prepared this survey because the agencies that fund programs aimed at supporting judicial independence require some quantifiable measure of the impact of their programs and some tool for refining their implementation.

ABA/CEELI JUDICIAL REFORM SURVEY

1 Selection and Appointment

1. Judges are appointed based solely on objective criteria such as passage of an exam, performance in law school, other training, experience, professionalism, and reputation in the legal community.

2. Ethnic and religious minorities, as well as both genders, are represented amongst the pool of nominees and in the judiciary generally.

COMMENTARY

UN Principle No. 10 provides, "Any method of judicial selection shall safeguard against judicial appointments for improper motives. In the selection of judges, there shall be no discrimination against a person on the grounds of race, color, sex, religion, political or other opinion, national or social origin, property, birth, or status . . ."

Minority and gender representation. European Charter Principle 2.1 prohibits any judicial "candidate being ruled out by reason only of their sex, or ethnic or social origin, or by reason of their philosophical and political opinions or religious convictions." This consideration is important because, although judges are charged with resolving the disputes that society brings before them (and therefore, in a certain sense, must be "above" society), they must also fairly represent society. In addition, minority representation in the judiciary will be key to protecting minority rights and implementing the rule of law in many countries, such as the former Yugoslavia, where minority repression remains a cause of conflict and violence. Some statistics in this area can, however, be misleading. In the former Soviet bloc, for example, women often represented up to 70 % of the judiciary. This was not an indicia of the progressiveness of the region's judiciary, but rather of the low level of esteem with which the judiciary was regarded: being a judge was considered almost a part time job which a woman could do and still have time to take care of her family. Few women, however, filled leadership positions, such as presidents of courts. In addition, men dominated the procuracy, again indicating where the real power in the legal system resided.

2 Education and Training

3. Judges have formal university level training in the law and, in addition, before taking the bench are required (without cost to the judges) to take a series of courses concerning the substantive and procedural areas of the law that they will be working in, as well as concerning the role of the judge in society and cultural sensitivity.

4. Judges must undergo, on a regular basis and without cost to them, professionally prepared and judge-taught continuing legal education courses, the subject matters of which are generally determined by the judges themselves and which inform them of changes and development in the law.

0 COMMENTARY

A truly independent judiciary, in order to apply the law fairly, must know what the law is. As UN Principle No. 10 provides, "Persons selected for judicial office shall be individuals of integrity and ability with appropriate training or qualifications in the law." Judges must also be aware of how their colleagues are handling issues and cases that they frequently are addressing. This means that new judges must be educated concerning the law, and receive additional training concerning issues of specific relevance to judges after appointment or designation. European Charter Principle 2.3 mandates that judicial appointees receive "appropriate training at the expense of the state." In addition, continuing legal education programs are vital if judges are to understand changes in laws, and if the laws are to be applied with any consistency. In many remote regions of Eastern Europe, for example, the judges do not receive copies of the laws and are relying on outdated materials. In order to be able to do their jobs properly, judges need to be systematically informed of changes to legislation. Finally, is important for judges to have control over what they are studying in such CLE courses; the government should not dictate what the judges are learning about. See Plank, supra, at 31 - 31 (noting that judges must possess "sufficient education and knowledge to discern the law and to articulate bases for a decision" and that "an educational requirement allows the judges to command the respect of litigants and society.").

3 Budget

5. The following percentage of the country's or state's national budget is allocated to the judiciary (not including Ministry of Justice and police costs, for example, but including judicial and administrative staff salaries, court education and training, court maintenance, costs for jurors, lay assessors, etc.):

1. 0.04% or less
2. 0.05% - 0.19%
3. 0.20% - 0.34%
4. 0.35% - 0.49%
5. 0.50% or more

6. The judiciary has a meaningful opportunity to influence the amount of money allocated to it by the legislative and/or executive branches and, once funds are allocated to the judiciary, the judiciary has control over its own budget and how such funds are expended.

COMMENTARY

Adequate funding. A judiciary that is well-educated and fairly appointed will serve little use if it is not provided with sufficient operating funds, and the discretion to use those funds as it sees fit. As UN Principle No. 7 provides, "It is the duty of each Member State to provide adequate resources to enable the judiciary to properly perform its functions;" see also European Charter Principle 1.6 ("The State has the duty of ensuring that judges have the means necessary to accomplish their tasks properly, and in particular to deal with cases within a reasonable period."). The UN Principles do not define what is meant by the term "adequate," and obviously, the relative amounts dedicated to the judiciaries of various states will vary widely depending upon those states' relative wealth. Accordingly, we have chosen to look at percentages of national budgets allocated to the judiciary. The percentages allocated in countries that are generally considered to have independent judiciaries are surprisingly small. In 1995, for example, costs for the judiciary consisted of less than a fifth of one percent of the total U.S. federal budget. Tacha, supra, at 649.

The role of the ministry of justice may be important here. In the United States, it is seen as key to judicial independence that the judiciary administers itself. But the Department of Justice in fact did administer the U.S. court system until 1939, when Congress passed the Administrative Office Act, which established the Administrative Office of United States Courts. Moreover, even today in many modern European countries such as Germany, and France, the courts are administered through the ministries of justice. Even in the United States, the judiciary is beholden to a certain extent to the executive branch, which nominates its members and enforces its decisions, and to the legislative branch, which confirms the appointments and provides the overall funding. Accordingly, the issue is not whether the judiciary is funded through the ministry of justice, but rather whether the executive branch uses that funding methodology as a means of controlling or curtailing the activities of the judiciary. And transfer of budgetary control from the ministry to the judiciary itself is no guarantee of judicial independence: Russia recently made such a transfer, but that system is so beset by other problems that its independence is at least questionable.

4 Salary

7. The salary of a senior level judge is generally comparable to the salary of:

1. A government-employed bus driver or sanitation worker;
2. A government-employed clerk or police officer;
3. A public prosecutor;
4. A cabinet-level minister or member of parliament;
5. The president of the country.

8. Salary levels in the judiciary are formally linked to the salary levels of other government officials, so that, for example, Parliament may not vote a pay raise for itself or for executive officials without also raising the pay for the judiciary by the same rate.

9. Judicial salaries generally may not be decreased, unless as a part of government-wide budgetary cuts.

0 COMMENTARY

Adequacy of judicial salaries. A country will not be able to attract capable lawyers to its judiciary unless it sufficiently compensates them. Salaries also need to be sufficient in order to counter the dangers of bribery. See European Charter Principle 6.1 ("[j]udges . . . are entitled to remuneration, the level of which is fixed so as to shield them from pressures aimed at influencing their decisions . . . . "). Obviously, in the United States and many other countries lawyers can make much more money in private practice than in government service. But judicial salaries must be such that they are at least comparable to the salaries of other high-level government employees. In other words, it would be inequitable, and the people would see it as such, if the nation's Chief Justice were paid less than half of what the President is paid. Similarly, it sends the wrong message if prosecutors are paid more than judges. In many countries in Eastern Europe, judges have been paid less than policemen or bus drivers. This is unacceptable, even in a civil law country where judges may take the bench at a very young age, directly from university. Without examining the actual salaries paid (which could range broadly depending on the economies in each country), we decided that the best way to measure the relative worth of the judiciary is to compare the salaries of senior level judges (for example, members of the Supreme Court, Constitutional Court, or other high level appellate courts) with the salaries of other government employees.

Salary guarantees. UN Principle No. 11 states, "The term of office of judges, their independence, security, adequate remuneration, conditions of service, pensions and the age of retirement shall be adequately secured by law." Art. III, section 4 of the United States Constitution provides that compensation for judges cannot be diminished while in office. See also Plank, supra, at 29 - 31 (describing importance of fixed and adequate compensation and noting that "[a]ttracting able judges will help sustain a reasonably high respect for the judiciary.").

5 Safeguards from Improper Governmental and Non-Governmental Influences

10. Senior level judges are appointed for the following terms:

1. Senior level judges may be removed, without cause, at any time;
2. 1 - 5 years;
3. 5 - 10 years;
4. 10 - 20 years;
5. Life.

11. Judges may be removed from office or otherwise punished only for specified official misconduct, and through a transparent process, governed by objective criteria.

12. Judicial decisions are made without outside political pressure.

13. Judicial decisions are made without improper influences by litigants or other interested parties.

14. Judges have immunity for actions taken in their official capacity.

15. Judicial decisions may be reversed only through judicial appellate process.

16. Judges are free from threats such as assault and assassination.

17. Judges are advanced through the judicial system on the basis of objective criteria such as rate of reversal by higher court, numbers of cases handled, etc.

0 COMMENTARY

Improper influence. These issues are difficult to address because attempts to improperly influence the judicial process, whether by the government or by individuals, are inherently secretive and almost always involve criminal activity. Judges in most countries, accordingly, are very unwilling to discuss the problem. The UN Basic Principles, however, recognize the importance of this issue. Principle No. 2 provides, "The judiciary shall decide matters before them [sic] impartially, on the basis of facts and in accordance with the law, without any restrictions, improper influences, inducements, pressures, threats or interferences, direct or indirect, from any quarter or for any reason." UN Principle No. 4 goes on to state, "There shall not be any inappropriate or unwarranted interference with the judicial process, nor shall judicial decisions by the courts be subject to revision. This principle is without prejudice to judicial review or to mitigation or commutation by competent authorities of sentences imposed by the judiciary, in accordance with the law."

We recognize that some of our questions -- such as "judicial decisions are made without outside political pressure" and "judicial decisions are made without improper influences by litigants or other interested parties" -- call for subjective judgments. There simply is no way for those questions to be answered in a fully objective fashion; rather, they will call for discussions with judges, lawyers, and journalists. Responses to such questions will require, however, concrete examples from the press concerning reports of improper judicial interference.

6 Jurisdiction and Judicial Powers

18. The judicial branch, or some part of it, has the power to determine the constitutionality of legislation and official acts, and such decisions are enforced.

19. The judiciary has exclusive jurisdiction over all cases concerning civil rights and liberties.

20. The judiciary works under uniform rules of evidence and procedure that facilitate the conduct of orderly trials and hearings.

21. Judges have meaningful subpoena, contempt, and enforcement powers.

0 COMMENTARY

Judicial jurisdiction. The power of the judiciary would indeed be hollow if it did not have the power, ultimately, to say "what the law is." See Marbury v. Madison, 5 U.S. 137, 177 (1803). UN Principle No. 3 provides: "The judiciary shall have jurisdiction over all issues of a judicial nature and shall have exclusive authority to decide whether an issue submitted for its decision is within its competence as defined by law." UN Principle No. 5 additionally states, "Everyone shall have the right to be tried by ordinary courts or tribunals using established legal procedures. Tribunals that do not use the duly established procedures of the legal process shall not be created to displace the jurisdiction belonging to the ordinary courts or judicial tribunals."

Judicial role in criminal justice. Finally, the courts must be seen as being at the apex of the justice system. As such, the judiciary must be able to control (and declare unconstitutional) certain acts of the state. The prosecution and police must be controlled by a neutral judiciary, not the other way around.

7 Transparency

22. Courtroom proceedings are open to, and can accommodate, the public and the media.

23. Judicial decisions are published and open to academic and public scrutiny.

24. A transcript or some other reliable record of courtroom proceedings is maintained and is available to the public.

25. Courthouses are centrally located and easy to find, and provide a respectable environment for the dispensation of justice.

0 COMMENTARY

The issue here relates largely to the public's perception of the judiciary. We sought to develop questions that would reflect the public trust and understanding of the judiciary, but that could be answered without engaging in the costly process of polling. We added the question concerning the location and accessibility of courthouses because too often, under the communist regimes of Eastern Europe, citizens did not know where the courthouse was until they were arrested. Even today, some trial courts, for example, are hard to find and not centrally located or easily identified. Again, this gives an impression that justice is not an open process. Similarly, we included the question concerning the appearance of the courtroom or the courthouse because of the impression that is conveyed to the public. In Romania, for example, the main courthouse in Bucharest is old, dilapidated, and in general disrepair. In contrast, the Office of the General Prosecutor is new, marble columned, and clean. It is clear where the State is allocating its resources and where the real power is.

8 Ethics

26. A code that governs the ethical conduct of judges (that generally covers issues such as conflicts of interest, ex parte communications, inappropriate political activity, etc.) is in place and enforced (through a process that respects due process).

27. Before taking office and throughout their tenure, judges are required to receive training concerning judicial ethics.

28. A meaningful process exists under which other judges, lawyers, and the public may register complaints concerning judicial conduct.

0 COMMENTARY

A truly independent judiciary must function within clear ethical guidelines. As UN Principle No 19 states, moreover, "All disciplinary, suspension or removal proceedings shall be determined in accordance with established standards of judicial conduct."

Finally, while we recognize that the term "meaningful" when used to describe the judicial conduct process is subjective, the mere existence of a judicial conduct process, without regard to its quality, does not sufficiently respond to the question posed.

9 Caseloads and Work Conditions

29. The caseload of each judge is reasonable.

30. Each judge has the basic tools necessary to do his or her job, e.g., sufficient office space, adequate support staff, word processing equipment, a law library (whether physical or online), etc.

0 COMMENTARY

Caseload. Many judges in both the United States and Eastern Europe complain that they cannot properly perform their jobs because of their heavy workloads. Indeed, the dispensation of justice should not take the form of an assembly line, where processing the cases becomes more important than ensuring fair results. Although, the issue of caseload is important, it is also difficult to measure. Rather than try to determine the number of cases that is appropriate for each judge to handle (which could vary widely in civil law and common law countries), we determined to simply apply a subjective test of reasonableness, and to rely on interviews with judges and lawyers to obtain a score.

Working conditions. The question of appropriate workspace is clearly subjective. Nevertheless, it is not appropriate to have three supreme court justices working in one small office, as was the case in Romania in 1994. The question of support staff is also somewhat subjective; American judges, for example, have (and, because of the burden of research and writing inherent to the common law system, require) much larger staffs than their European colleagues; such also may reflect more the litigiousness of Americans than anything else. On the other hand, some courts have absolutely no support staff (no clerks to assist with filing and no secretarial staff), and this clearly impedes their efforts to dispense justice. The lack of any word processing equipment would present a similar impediment. See Tacha, supra, at 648 ("In order for a judge to handle her caseload and maximize productivity, she implicitly must possess adequate staff, equipment, and physical facilities to carry out her responsibilities. Independent judicial action requires an appropriate level of support which allows a judge to carry out the judicial function without relying on other entities, depending on someone else's assessment of the judge's needs, or giving any thought in the case-deciding role to tangential factors that might influence the speed of deliberation or the outcome.").

10 Assignment of Cases

31. Judges are assigned to cases by an objective method, such as by lottery, or according to their specific areas of expertise, and this assignment process is administered by the judiciary, not the ministry of justice.

32. Once assigned to a case, a judge may be removed only for good cause, such as because of a conflict of interest or an unduly heavy workload.

0 COMMENTARY

Too often, cases are in essence pre-decided by assigning them to judges who are more "controllable" or "predictable." Absent special reasons such as expertise or workload, judges should be assigned to cases according to a blind, random method. UN Principle No. 14 notes, "The assignment of cases to judges within the court to which they belong is an internal matter of judicial administration."

11 Support by Non-Governmental Organizations

33. An association dedicated to protecting the interests of the judiciary exists and is active.

0 COMMENTARY

UN Principle No. 8 states that "members of the judiciary are like other citizens entitled to freedom of expression, belief, association and assembly." UN Principle No. 9 provides, "Judges shall be free to form and join associations of judges or other organizations to represent their interests, to promote their professional training and to protect their judicial independence." Similarly, European Charter Principle 1.7 states that "[p]rofessional organizations, set up by judges, and to which all judges may freely adhere, contribute notably to the defence of those rights which are conferred on them by statute, in particular in relation to authorities and bodies which are involved in decisions regarding them."

The existence of such an organization may be worth a point on the scale, but the real question is whether it is active and effective. Additional points should be provided where an association is engaged in lobbying, training, or other activities supportive of an independent and professional judiciary.