An Official Response to the Aboriginal Governance Index

Aboriginal Futures, Frontier Centre, Uncategorized

Text of the letter sent by the Assembly of Manitoba Chiefs Secretariat Inc.:

June 20, 2006

The Assembly of Manitoba Chiefs has recently reviewed the Frontier Centre for Public Policy’s (FCPP) “research” document entitled Aboriginal Governance Index — A 2006 Ranking of Manitobba First Nations — written by Don Sandberg, Dennis Owens and Rebecca Walberg which was released on June 14, 2006.

After our review and analysis of the document, it is our position that the “survey” results clearly indicated a biased view of First Nations governance under the pretense of research. Throughout, the report uses the veneer of research and graphs to purport that the findings are authentic, when even the basic rules of field research are not followed. The following are the most glaring and obvious issues:

  • No statement made with respect to the number of people interviewed per community or even in total is reported, not to mention gender, age, education, income, etc., of the people interviewed (either in person or by mail in)which are the usual descriptive components of data.
  • There is no differentiation between people’s responses in being interviewed versus respondents mailing in self-administered questionnaires. (Note: interviews offer opportunities to clarify questionsand probe more deeply for answers.)
  • The methodology of analysis which translates the responses to the questionnaire into percentages based on “weightings” is never explained except to say the authors held “extensive discussions about scoring methods” but never share those discussions.
  • Despite these major failings of non-identification of the sample population, the FCPP report ranks and names the ten best and ten worst First Nations according to this invented scale and weightings system. Thus it is a shell game whereby perhaps 10-100 people may have been surveyed and registered their complaints and have had their “voices” heard through this scale.
  • This report clearly demonstrates the ignorance of the authors and the FCPP about several areas of First Nations Chief and Council governance and jurisdiction and is simply a public exercise against First Nations. It carries the message once again of unaccountable First Nations leaderhip without the rigor and principles required of true research.

    While we understand that the FCPP states that the authors of the Aboriginal Gvernance Index 2006 “worked independently and the opinions they expressed are therefore their own, and do not necessarily reflect the opinions of the FCPP”, on the other hand, you yourself introduced the report with high praise for its principal author Don Sandberg. It is our position that this report clearly demonstrates an absolute lack of respect for the First Nations leadership by choosing not to discuss the purpose of the methodology of this research with any First Nations leaders or organizations beforehand to ensure that appropriate protocols would be followed or that useful questions would be asked. It may have been possible to develop collaboratively a useful project if more partners and less ideological bent were applied.

    It is too bad that in this day and age, organizations such as your are able to release research reports with such major failings and bias without even considering the harmful impacts of such ill designed and ideologically driven ‘research projects’.

    Sincerely, Ron Evans, Grand Chief, Assembly of Manitoba Chiefs

    The Frontier Centre responds:

    In context, the complaint that the Frontier Centre’s Aboriginal Governance Index indicates a “biased view of First Nations Governance” because it did not follow “even the basic rules of field research” rings hollow. The Frontier Centre does not claim scientific accuracy for its effort to survey First Nations on the quality and effectiveness of their governance. Indeed, this maiden effort was fraught with difficulties which are extensively discussed in the report. The barriers to collecting accurate field data were formidable, and we present our findings as a snapshot of the opinions we gathered, with no claim that our initial rankings are any more than that.

    We would very much have liked to have included more Canadians who belong to First Nations in this survey. We would also have liked to interview representatives from a broad spectrum of aboriginal communities, who are diverse with respect to age, education, and occupation, among other factors. The choice of many reserve authorities actively to hinder the collection of survey data was the biggest obstacle in doing so. In future years, we hope the Manitoba Chiefs will remember our desire to survey a wide swath of reserve residents, and encourage their members to participate freely and in confidence.

    Chief Evans complains that we did not discuss the purposes or the methodology of our survey work with First Nations leaders. Are we to infer from that statement that we can expect the co-operation and participation of the Assembly of Manitoba Chiefs in the next round of surveying? We would welcome that, especially an expansion of our survey teams to cover more territory.

    Significantly more than “10-100 people” were surveyed. Chief Evans’ allusion to a “shell game” is misleading. In a shell game, the idea is to hide the truth from the mark. If there is a party to this research that has an interest in concealing the reality of aboriginal governance, it certainly isn’t the Frontier Centre.

    It is true that there is no reflection in the quantitative analysis of the differences between interviews carried out in person and surveys mailed in. This is why Mr. Sandberg’s fascinating qualitative analysis comprises an important part of the report, so that the differences between different reserves that do not show up in the surveys might be noted. Again, with more co-operation from band authorities, in future years we will be able to survey more extensively in person, in which case we might not need to rely upon any mailed-in surveys.

    The weightings were assigned in such a way as to reflect the areas of governance we judged to be the most significant. Healthy elections, competent administration, protection for human rights, transparency, effective services and a productive economy are, we believe, the key measurements of the health of a system of government. Having said that, it is because of the inherently subjective nature of weightings that we provide the six sub-scores for every reserve surveyed. Those who are interested only in one area of governance, or who would prefer to weight the factors differently, are free to use the data we provide in the report to do so.

    We believe the in-house decisions we made about the weightings of answers to the survey questions written for the Index are well grounded in universal principles of good governance. We gave a fairly strong weighting, for instance, to the question, “Does the Chief or council use band resolutions to force residents to leave the reservation?” We believe that leaders who commonly use methods like exiling or banishing critics in order to solve problems are not following those principles, nor governing in the spirit of our aboriginal peoples, who historically relied on consensus and consent, not arbitrary exclusion.

    What are the “appropriate protocols” necessary to asking people to discuss the quality of their governance? The patronizing tone in this term is itself disturbing. Why would the Assembly of Manitoba Chiefs seek to hinder the ability of their band members to share their stories, experiences and views with anybody at all? In fact, our efforts to follow such protocols and engage Chiefs though official channels were met with silence.

    Individual interviews with many local Chiefs and band officials were conducted successfully, without such prior approvals. In other cases, our staff were told that they needed official permission to conduct such work on First Nations. We believe that all residents of First Nations share in the bedrock Canadian freedom of free speech. We believe that if we had obeyed Chief Evans’ wishes and followed “appropriate protocols” or asked only questions that aboriginal leaders find “useful,” the survey work would never have been conducted. Certainly the results would not have represented the true opinions of First Nations residents.

    Chief Evans criticizes our “choice not to discuss the purpose of the methodology of this research with any First Nations leaders or organizations.” While any methodology can be improved, and while in fact we are interested in improving our research methods and base in subsequent iterations of this work, we have little to learn from this demand about how to conduct good research. To undertake any sort of social science research in a manner that lets the people being evaluated shape the criteria would border on intellectual fraud.

    The ideal in research is to have disinterested observers take objective measurements. This is possible in biology and physics; it is harder in the humanities. Nevertheless, the surveys were designed, collected and analyzed in a manner that would ensure as close a representation of reality as possible. In terms of disinterest, we shall let people decide for themselves who seeks to gain by distorting the truth in this instance: non-partisan researchers who advocate universal liberty and free choice, or Chiefs whose governance has been shown in this work to be unacceptably poor?

    We do have “an absolute lack of respect” for some First Nations leaders, especially those who treat governance as a spoils system, with little regard for the suffering or the ultimate welfare of their people. We believe that such behaviour is common in Manitoba’s First Nations, and all the more odious for that reason. Throughout the existence of our Aboriginal Frontiers Project, we have extensively documented this syndrome, the lack of accountability for spending, and the short shrift given to the value of free and fair elections in some aboriginal communities.

    We have no lack of respect for the progressive Chiefs who have avoided these pitfalls, despite the fact that the legal structure imposed by the Indian Act and the activities of the federal Department of Indian and Northern Affairs have discouraged such enlightenment. Indeed, the main focus of our Aboriginal Governance Index points to them as models for other leaders to emulate.

    We believe the in-house decisions we made about the weightings of answers to the survey questions written for the Index are well grounded in universal principles of good governance. We gave a fairly strong weighting, for instance, to the question, “Does the Chief or council use band resolutions to force residents to leave the reservation?” We believe that leaders who use methods like exiling or banishing critics in order to solve problems are not following those principles, nor governing in the spirit of our aboriginal peoples, who historically relied on consensus and consent, not arbitrary exclusion. Should political leaders have the power to deny citizenship to those who exercise the right of free, peaceful speech? We don’t think so. Perhaps Chief Evans could explain why he disagrees.

    The ultimate welfare of the people in Canada’s First Nations depends on a willingness to reform dysfunctional governance practices such as that. We think that a good source of the values of good governance resides in the wisdom of the people themselves, and consider efforts like the Aboriginal Governance Index to be useful tools for checking the enormous powers of Chiefs and band councils. Indeed, despite the problems we encountered with our survey work, we believe it confirms the worthiness of such bottom-up efforts at consultation. The enthusiastic welcome we received into most First Nations’ homes indicates that the people themselves are eager to have their voices heard through an independent filter.

    Chief Evans says he is concerned about the “harmful impacts” of our Index. Can he identify them? How can the opinions of people about their leaders be considered harmful, except perhaps to the political futures of leaders who ignore their feelings?

    While Chief Evans’ letter is published on Assembly of Manitoba Chiefs’ stationery, it is signed by one person. Do Chiefs of First Nations that ranked highly echo Mr. Evans’ criticisms? If the AMC would like to see more detail in the future about the size and variance of our sample, it is puzzling that they do not provide similar information about the process and level of unanimity that led to this complaint. Does he speak for all Manitoba Chiefs with respect to the contents of his letter? Our contacts with other Chiefs indicate that he does not.

    It was our hope that those First Nations that performed poorly on this survey might take this opportunity to learn from better practices in other communities. Perhaps this is the time for the AMC to invite those successful reserves to speak out about how and why they scored so much higher than the rest of Manitoba’s reserves. We look forward to seeing how human rights and good governance are advanced for all Canadians and Manitobans, including aboriginals, in the future. Insofar as this research was ideologically driven, assisting to this end was its only goal.

    Don Sandberg, Dennis Owens and Rebecca Walberg