The Access to Justice Index for Federal Administrative Bodies:
Canadian Agricultural Review Tribunal

March 2023 (Updated June 2023)

  The Access to Justice Index for Federal Administrative Bodies PDF

Table of Contents

• Introduction
• Results
  • Category 1: Access to the Administrative Body
    • 1a. Physical Access
    • 1b. Access through Technology
  • Category 2: Processes
    • 2a. Procedural Justice
    • 2b. Representation
    • 2c. Interpersonal Aspects
    • 2d. Informational Aspects
  • Category 3: Costs
    • 3a. Service Charges
    • 3b. Intangible Costs
  • Category 4: Outcomes
    • 4a. Distributive Justice
    • 4b. Functionality
    • 4c. Transparency of Outcomes
  • Composite Index Score

Introduction

The Research and Statistics Division of the Department of Justice Canada initiated the Access to Justice Index for Federal Administrative Bodies project with the goal of developing and piloting a quantitative measurement of access to justice in the context of Canadian administrative law. The resulting Index is an important resource for administrative bodies, wherein results describe how well they are doing to ensure access to justice for their users/clients/parties. The Index is a self-assessment tool intended to highlight the power of data-driven policy and programming and provide clear, concrete areas in which administrative bodies can improve.

The project is an adaptation of the Access to Civil Justice Index, launched in 2014 by the US National Center for Access to Justice, which measures access to justice in the civil context in all 50 states. The categories were developed after reviewing the immense body of literature on understanding the quality and cost of access to justice, particularly from The Hague Institute for Innovation of Law.^{Footnote 1}

This Canadian project aims to address a knowledge gap around access to justice issues in the federal administrative context. It aims to do so in a manner that respects the mandates and the independence of the participating administrative bodies, as well as the strategic outcomes of the Department of Justice Canada (DOJ) - a fair, relevant and accessible justice system and supporting the federal government with high-quality legal services - while contributing to the dialogue on access to justice.^{Footnote 2} The Index itself was developed by a working group comprised of officials from the DOJ, four federal administrative bodies, and two legal academics.

Results for the Canadian Agricultural Review Tribunal (CART) are presented in the following pages by category.

Results

Category 1: Access to the Administrative Body

Category 1 includes both Physical Access and Access through Technology.

1a. Physical Access

Questions in this sub-category focus on the ability of parties to access the offices of the administrative body in person. This is particularly important where there are oral processes. Question 2 asks, “Are oral processes (e.g. hearing, mediation) held in locations as close as possible to the parties?” A positive response to this question is worth 10 points, the highest value given because of the importance to the parties themselves. Another question worth 10 points is whether the rooms used for oral processes accommodate persons with a disability (Question 3). Where there are oral processes, it is assumed that parties want to be present, rather than joining by video conference, although it is important to have that as an option.

The 14 questions and answers for this sub-category can be found in Table 1a below with an Index score of 77. Index scores for Category 1 can be found in Figure 1 at the end of this section.

Access to CART is open and there were five questions where the tribunal lost points. These were:

• Q1 – “Does the administrative body (or contracted service provider) have at least one office open to the public?”
• Q5 – “Does the administrative body have reception staff to assist visitors?”
• Q6 – “Can a party speak with a representative of the administrative body outside of regular business hours (e.g. 8h00 – 17h00) across Canada?”
• Q7b – Are parties informed that they can watch a live or simulated oral process?”
• Q11 – “Does the office of the administrative body provide access to childcare or child-friendly spaces for children of parties or witnesses?”

1b. Access through Technology

The 16 questions asked in this sub-category are presented below in Table 1b with the CART responses. In this sub-category, CART scored 54. CART lost points on the following seven questions:

• Q17 – “Can the administrative body’s website be understood by users with various literacy levels?”
• Q19 – “Is the administrative body’s website accessible to persons with different learning styles, such as the inclusion of visual and audio presentations of information?”
• Q20a – “Does the administrative body make use of web-diagnostic tools, such as Google Analytics or other software?”
• Q21b – “If the administrative body solicits feedback from website users, does it respond to issues raised?”
• Q23a – “Does the administrative body use online forms?”
• Q23b “Does the administrative body use “smart” forms?”

Overall, when both sub-categories of Category 1 are combined, the score for Category 1: Access to the Administrative Body (CART) is 63. Figure 1 below illustrates the Index scores for Category 1.

Figure 1: Access to the Administrative Body

Category 2: Processes

The “Processes” category is divided into four sub-categories: Procedural Justice, Representation, Interpersonal and Informational.

2a. Procedural Justice

Procedural Justice includes 20 indicators on access to both formal and informal processes, which will vary by organization. In the specific case of CART, a party has many choices from which to choose a specific process. In responding to the questions, CART was asked to think of scenarios with each of the different processes in order to determine whether to respond in the affirmative or in the negative.

All the questions are worth five points except for one which is worth 10 points: “Does the administrative body provide interpretation services in languages other than French or English?” This is extremely important where one or more of the parties may not have French or English as a first language or may require other means of communication due to a disability. The CART has indicated that interpretation services in languages other than French or English are not provided.

Almost all questions in this sub-category were answered in the negative. For example, there are two questions worth one point each on internal service standards. Question 33a asks whether CART has documented internal service standards and Question 33b asks whether these standards are monitored. The weight for these questions is less than for the questions on external service standards (Questions 33c, 33d, 34) because it is the external service standards that are important from the parties perspective. CART indicated that it does not have any formal external service standards.

CART lost points for not providing a choice among a variety of processes:

• Q26 – “Can parties choose among a variety of processes as their case goes through the system?”

ART indicated that training and assessment are not conducted with members and staff:

• 24a – Do members receive training on, or are they assessed on prior experience with, active adjudication?
• 24b – Do staff receive training on, or are they assessed on experience with, active adjudication?
• 25a – Do members receive training on, or are they assessed on experience with, objectivity and bias?
• 25b – Do staff receive training on, or are they assessed on experience with, objectivity and bias?
• 26 – Can parties choose among a variety of processes as their case goes through the system?

CART also lost points for the following questions which focus on evaluation and targets:

• Q30a – “In the last five years, has the administrative body evaluated how satisfied parties are with its processes?”
• Q30b – “Has the administrative body responded to issues identified in evaluations of user satisfaction?”
• Q31 – “Does the administrative body have performance indicators relevant to access to justice?”

2b. Representation

The questions in this sub-category were largely informed by the Council of Canadian Administrative Tribunals’ self-represented parties questionnaire circulated to its members in 2015. With eight questions in this sub-category, CART scored 49.

Question 35 is worth 10 points and was answered in the positive: “Does the administrative body provide information to parties who represent themselves (e.g. checklists and other public legal education and information materials, on process, FAQs, and other topics specific to a self-represented party)? This question received the heaviest weight possible, as this provision of information, basic as it might seem, is imperative to those who do not have legal representation.

Question 36 asks whether the administrative body monitors trends in self-representation. CART does not monitor self-representation. Questions 37a – 37d ask whether members or staff receive training or are assessed on previous experience with self-represented parties. CART responded “no” to the following:

• Q37b – “Do staff receive training on how to work with, or are they assessed on experience with regard to, self-represented parties?”
• Q37c – “Does the administrative body monitor members’ engagement with self-represented parties and submissions?”
• Q37d – “Does the administrative body monitor staff engagement with self-represented parties and submissions? (e.g. calls with clients could be randomly monitored)”

CART answered “yes” to question 38 – “Does the administrative body support parties who lack the capacity to self-represent? (e.g. designated staff provide additional information by telephone or in person, provide additional assistance with documents and processes; referrals to outside service providers).”

2c. Interpersonal Aspects

Some of the elements of this third sub-category include whether members and staff receiving training on treating clients with respect (Questions 40a and 40b) and whether CART has a Code of Values and Ethics (Question 41). There are two additional questions (42a and 42b) that ask whether members and staff receive training on the duty to accommodate. There are a total of five questions in this sub-category which are presented below in Table 2c.

CART responded no to the following questions:

• Q40a – “Do members receive training on, or are they assessed on experience with regard to, treating parties, staff and other members with respect?”
• Q42a – “Do members receive training or are they assessed on experience with regard to the duty to accommodate?”
• Q42b – “Do staff receive training or are they assessed on experience with regard to the duty to accommodate?”

2d. Informational Aspects

In this final sub-category, the theme is information (developing, disseminating, etc.) which is an extremely important category. The definition for Public Legal Education and Information (PLEI) in this pilot project is drawn from the Access to Justice Service Agreements which are contracts between the three territories and the Department of Justice:

“an activity that seeks in a systematic way to provide people with the opportunity to obtain information about the law and the justice system in a form that is timely and appropriate to their needs.”

There are 20 questions in this sub-category and CART received a score of 38.

Questions include whether written and oral information is regularly updated (Question 44), whether CART conducts regular outreach activities (Question 45), whether there is information about the CART on relevant websites (Question 46a) and in hard copy in the waiting rooms of relevant organizations (Question 46b).

Question 51 has an asterisk because it is the second of only two questions that receive points where there is a negative response. CART lost five points for a “yes” in response to the question, “Has the administrative body had any substantiated claims with the Office of the Commissioner of Official Languages for the refusal to provide services in English or French filed against it in the past 12 months?”

CART lost points for communicating to participants on their level. These “no” responses include:

• Q45 – “Does the administrative body conduct regular outreach activities? (e.g. on an on-going basis activities such as newsletters, stakeholder meetings, speaking events, etc.)”
• Q46a – “Is there information about the administrative body on the website of relevant stakeholders and information and services agencies?”
• Q46b – “Is there information about the administrative body in the waiting rooms of relevant stakeholders and services agencies?”
• Q48 – “Is there a glossary of terms on the administrative body’s website?”
• Q49 – “Is accessible/plain language used in written resources?”
• Q50a – “Is accessible/plain language used in oral communication with parties?”
• 50b – “Does the administrative body monitor communication materials for accessible language?Q52 – “Are written resources available in languages other than French and English (e.g. Braille, Spanish and Chinese)?”
• Q52 – “Are written resources available in languages other than French and English (e.g. Braille, Spanish and Chinese)?”
• Q53 – “Does the administrative body provide information using visual aids?”
• Q54c – “Does the administrative body provide paper copies of additional resources?”
• Q54d – “Are referral lists kept current?”

Overall, for Category 2, the Index score for CART is 27 and is illustrated in Figure 2 below.

Figure 2. Processes

Category 3: Costs

This category addresses the costs to parties in order to access the processes of the administrative body. Almost all of these costs are in the form of service charges and, as such, are assumed as barriers to accessing the CART specifically for complainants in vulnerable circumstances. The Tilburg research identifies other cost areas (opportunity cost compensation and several intangible costs); these have not been included in the Index.

3a. Service Charges

For service charges, there would be an actual exchange of money in return for services, such as receiving hard copies of PLEI publications, or administrative fees to cover the cost of filing documents in a case. There are 11 questions in this sub-category and CART received an Index score of 42.

Having to pay for access to services represents a clear and concrete barrier to parties with low incomes. Three questions in this sub-category receive the highest weight of 10 points: - Questions 59a, 60a and 61. CART responded “yes” to questions 59a and 60a and “no” to question 61.

For questions 59b, “If there are fees for filing documents, is there a sliding scale or waiver of fees for parties who meet low-income criteria?” and 60b, “If there are fees for the use of rooms, is there a sliding scale or waiver of fees for parties who meet low-income criteria?” CART commented that filings are free.

The first question in this sub-category, Question 56 reminds the reader that especially for CART, it is important to remember that proceedings include any dispute-resolution process, such as mediation sessions. Many newcomers may be unable to provide testimony in English or French. Interpretation may also be required as an accommodation when a party has a disability that impedes clear or direct communication in one of Canada’s official languages.

Question 57 states: “Does the administrative body pay for the translation of key documents (e.g. letters to parties, decisions) into languages other than French and English?” The CART answered “no” which may be problematic for participants whose first or daily-used languages are other than English or French.

Question 64, “Does the administrative body allocate a portion of its budget to the information needs of parties, such as surveys, testing of materials, website revisions, etc.),” was also answered in the negative which speaks to usability issues for participants.

3b. Intangible Costs

Intangible costs include those costs such as pain and suffering or other emotional and psychological effects of the process of filing or responding to a complaint with CART. What is being measured here is whether CART recognizes these potentially negative intangible costs and is acting to address them for staff, members and parties.

The third question (Q67) asks, “Does CART provide assistance or support related to mental health to parties or referrals to relevant assistance or support?” The purpose of providing support is to help offset any adverse effects parties may experience in utilizing the complaints process. Long wait times, complicated paths, or bouncing back and forth between procedural steps can aggravate stress, frustration, and desperation among parties to a complaint. Complainants in particular may become exhausted, agreeing to settlement terms that may not be in their best interest. The psychological impact of the process could also affect a complainant’s ability to self-represent.

While support for the mental well-being of parties is unlikely, the Index is intended to be aspirational. Such support was evident in some of the practical applications of the Tilburg research. CART answered “no” to this question, which was weighted five points, while the first two were weighted only a point each. CART received a score of 29.

Overall, the total Index score for CART for Category 3: Costs is 40 and is illustrated in Figure 3 below.

Figure 3: Costs

Category 4: Outcomes

This final category of “Outcomes,” plays an important role in parties’ overall sense of fairness and justice in the resolution of a dispute.

4a. Distributive Justice

This first sub-category includes questions that are, in many instances, inherent in the analysis that members will undertake while making a decision on the merits, and on the remedy, in a particular case. CART received no index score in this sub-category, skipping all questions because they are out of the scope for the tribunal’s mandate.

4b. Functionality

As noted earlier, another aspect of this category refers to the extent to which outcomes are useful from the perspective of the parties. Arguably, at the stage in which conflict emerges, people do not immediately think in terms of rights they may or may not have. Rather, people are facing a (legal) problem that they seek to solve, and that triggers them into taking action (starting to travel a path to justice). The indicators in this sub-category explore the usefulness of the outcome in light of these problems. There are five questions. Three were skipped due lack to scope in the tribunal’s mandate. Questions 76a – “Can the administrative body enforce outcomes?” and 76b – “Does the administrative body enforce outcomes?” were answered with the comment, “But our decisions are enforceable.” Those responses were considered “yes.” The Index score is 100.

4c. Transparency of Outcomes

It is important that, whatever the outcome of a complaint, the outcome, and the justification for it, is transparent. If so, the outcome and its justification can be examined and better understood. In addition, transparency is also necessary in order to evaluate the extent to which the outcome is similar to other people’s outcomes in similar cases.

Question 80 asks, “Is the decision or settlement publicly accessible in full or redacted format?” The answer here is “yes” as all decisions and settlements are public.

The final question is “Once issued, are decisions by CART monitored internally for consistency in application?” The answer here is “no”. So for the four questions in this final sub-category, each weighted at five points, the CART scored 75.

Overall, for Category 4, the CART has a score of 83, which is illustrated in Figure 4 below.

Figure 4: Outcomes

Composite Index Score

This last section summarizes the calculations for the individual category Index scores and then calculates the composite Index score for CART.

Figure 5 illustrates the Index scores for the four categories that comprise the Access to Justice for Federal Administrative Bodies Index:

1. Access to the Administrative Body (CART) is 63;
2. Processes (CART) is 26;
3. Costs (CART) is 40; and
4. Outcomes (CART) is 83.

The composite Index score for CART is 43.

Figure 5: Access to Justice Index Scores (Overall)

Return to footnote ¹ referrerSee S. McDonald, Development of An Access to Justice Index for Federal Administrative Bodies (Department of Justice Canada: Ottawa, 2017). Available at: https://www.justice.gc.ca/eng/csj-sjc/access-acces/challenge-defi.html

Return to footnote ² referrerIbid.