February 26, 2010
Interagency Secretariat on Research Ethics
350 Albert Street
Ottawa, ON
Canada K1A 1H5
Tel: 613-996-0072
Fax: 613-996-7117
draft2e@pre.ethics.gc.ca (www.pre.ethics.gc.ca)
Dear Panel Members:
Re: TCPS Revised Draft 2nd Edition (December 2009)
The Society for Academic Freedom and Scholarship (SAFS) is a national organization dedicated to academic freedom and scholarship (http://www.safs.ca). On behalf of SAFS, we are writing in response to a call for comments on an on-line document entitled “Draft 2nd Edition of the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS),” dated December, 2009, and available on-line at http://pre.ethics.gc.ca/eng/resources-ressources/news-nouvelles/nr-cp/2009-12-18/
SAFS provided comments on the first draft of the Tri-Council Policy Statement (TCPS, 1998) which we have appended. Since that time we have continued to express our concerns about the expansion of research ethics regulations given the lack of demonstrated effectiveness for such regulations (e.g., http://safs.ca/interagencymain.html, http://safs.ca/issuescases/ethics.html, and http://safs.ca/issuescases/ncehr.html). In essence, what we see as having occurred is that the research ethics review system has greatly increased the burdens on individual researchers in the social sciences and humanities (SSH) in particular and on local university administrations, with no empirically documented benefit to the genuine protection of research participants. As will become clear based on our comments below, this version of the TCPS does virtually nothing to stop that process, leaving us with limited feelings of progress in regard to our past concerns. However, before discussing those issues, we would like to acknowledge some of the ways in which this version does represent an improvement on the previous version.
Although flexibility and delegation of decision-making is commendable in many ways, local REBs have often seemed to abuse open-ended policies. Therefore, it is good to see some advice to REBs about what not to do (e.g., not using the campus legal/risk office, line 2127). Likewise, we’re pleased to see the admonition about the undesirability of administrators being present (line 2978), although, unfortunately, they don't have to be formally present to have undue influence. Nonetheless, an explicit admonishment for them to not interfere cannot hurt. In fact, an explicit "do not" at various places in the document helps clarify things just as much as "shall" does. The new TCPS would benefit further from more such explicit denials of the propriety of practices implemented by local REBs and administrators.
It is also good to see "academic freedom" explicitly acknowledged (line 64), and to see explicit inducements for reviewers to admit intellectual conflicts of interest (line 2953). It is to be hoped that local REBs and reviewers will take these points seriously.
We’re also pleased that this draft of the TCPS explicitly encourages institutions to stand with the researcher in defending the requirement of confidentiality (line 1755), although it remains to be seen how that will ultimately be realized in practice. The Russ Ogden case (line 1650) demonstrates the double-bind that researchers are in re confidentiality: institutions and REBs demand that the researcher assure confidentiality (and anonymity), but then when government demands to know identities, they pressure the researcher to reveal those identities. Hopefully, by following the recommendations contained in this version of the TCPS, these types of situations can be avoided in the future.
Finally, it is commendable to explicitly identify intellectual (and presumably ideological) differences as grounds for disqualification on an REB review (line 2953). Now, if only local REBs will follow through on this and if individual reviewers will openly and honestly acknowledge such differences with the researcher, progress on this front can certainly be made.
These are not the only improvements in this version of the TCPS, however, we certainly see them as useful and important. Nonetheless, there are still some serious problems with this version, not the least of which is that several of the key questions that we raised in 1996 have not been addressed and are still quite worrisome.
(1) Policies continue to be derived largely from issues based in medical research, with only post hoc efforts being made in order to then make those policies fit other disciplines. A patient-care model is not typical of research in general. Likewise, the harm-benefit analysis is very subjective outside a medical setting, and can be readily adapted by REBs to block research that is really merely "unpopular" in the SSH. There is somewhat more explicit discussion of SSH issues, but still in the context of the broader constraints derived from medical research, typically clinical trials. It is still true that one size does not fit all, and efforts must continue to be made to deal with resolving the mismatches.
(2) We continue to be concerned that allowing research participants to determine research outcomes, by removing their data after the fact (lines 727-728, 837, 1100), will destroy the scientific validity of research carried out under the umbrella of the TCPS document. Much is made in the document about the necessity of obtaining appropriate samples. It doesn’t take much effort to see how allowing research participants to self-select after they are fully debriefed can severely bias even the most carefully selected sample. If the TCPS continues to allow and, in fact, encourage, them to do this, it seems likely that the international community will, at some point, require Canadian researchers to report the number of people who took that action, potentially prohibiting publication if the number is deemed to be too large.
(3) Local REBs are required to evaluate the scientific "value" of methods and design (line 522). We still see this as an invitation for REBs to control unpopular research and promote currently popular agendas. Local REBs do not have the requisite expertise, especially when the REB is responsible for evaluating submissions from a wide array of disciplines as in the SSH. In fact, mandates on REB composition work against expertise (line 2118) -- 3 of 5 are to be non-experts. More importantly, scientific value is not something that can typically be determined in advance, even by peer review. This requirement of REBs serves to shift the focus of research from discovery to political activism at the local level, as the REBs become relatively more concerned with building a better world than with subject safety.
(4) The "rights of groups" (collectivities, communities) is a further invitation to political activism, and is not related to the core mandate of subject safety. This drive seems to occur notably in connection with stigmatized groups, which seems to say "we can't have negative findings about <certain groups>"? In contrast, it is noted that when studying certain organizations (line 987) it is expected they will be offended from time to time. It is problematic that some groups must be protected, whereas others will not be, which is part of what renders "rights of groups" a political issue rather than a question of scientific validity or subject safety. Perhaps doing something like this could be argued to be appropriate for some research topics (e.g., Chapter 9), however, there is certainly no justification to layer it throughout the entire TCPS.
(5) The Policy's effort to define and develop "Ideal Researchers" is purely based on speculation and is driven by ideological agendas. REBs should focus on the behaviour of researchers, not their beliefs and values. Educational efforts by the TCPS and REBs should focus on good/bad behaviours rather than ideological initiatives.
In addition to these issues that were of concern to us in our original submission, we would like to raise a number of others that weren’t obvious to us then but have surfaced over the intervening time period.
(1) The scope of the document.
One issue that has been difficult to resolve to date has been just what activities, and whom, the TCPS actually covers, that is, its limits and scope. This is related to but separate from the Mandate question (below), and concerns the question of whether the Tri-council has the authority to monitor the activities of researchers who do not have funding from any of the agencies. Many people and REBs seem to think or behave as if TCPS control extends to virtually every activity that could be even vaguely defined as “research” including data collection that is completely outside the umbrella of the granting agencies. (Indeed, such a position seems to be being endorsed in the section beginning on line 319, although, at the same time, line 40 implies that such is not necessarily the case.) Nonetheless, in spite of a number of requests for the justification for such a view, we have yet to receive that justification.
It is crucial that the claims contained in the section beginning on line 319 be defended. That is, if one is to accept those claims, the document must clearly state what is the formal/legal basis for the claims. In essence, what must be clearly spelled out is, what authority does this document have to assert such control over research that is not funded by any of the granting agencies?
(2) The metrics being used to evaluate research.
The TCPS continues to talk about "balancing risks and benefits" and "probability of harm," but there are no empirical data suggesting that REBs could do anything of the sort. That is, there are no concrete metrics for any of these constructs in most research proposals, so there is no objective basis for REBs to make any such decisions. Instead, REBs are being allowed to make decisions on an intuitive basis which cannot be challenged or corrected by researchers whose proposals are being evaluated. Thus, reviewers continue to decide "risk" and "benefit" on emotional and ideological grounds, which invites the suppression of innovation and unpopular research. This problem is compounded by the fact that the review must assess risk and benefit as they appear at the moment, whereas the actual benefits of much research may not be known until well after the data have been collected. However commendable such a process may sound in principle, in practice such decision-making is quite vulnerable to very subjective biases and quite likely to lead to a number of bad decisions.
(3) The mandate for the REBs.
A clear mandate for reviewing research proposals would be nice (line 2020), however, "Respect for persons, concern for welfare, and justice" hardly qualifies as clear, no matter how commendable such objectives may be. Obviously, there is a problem here.
We would like to describe our concern about this issue by noting what may seem to be an unrelated fact, that the TCPS has noticeably grown in size. This is in part necessary due to the greater variety and scope of research activities now being undertaken. It is acknowledged in the TCPS that no single document can cover all circumstances (line 85). However, the fact that the current draft is 160+ pages easily leads to the conclusion that the document is likely trying to do too much. One set of problems that loom for an ever-expanding Policy Statement is captured by the colloquial observation that "Given any sufficiently large body of rules, sufficient contradictions exist that any course of action can be deflected." However, that is not the main issue here. The main issue is that much of this growth has been driven by the problems created by the TCPS’s shifting mandate.
As those connected with the process have discovered, the original mandate, subject safety, was difficult enough to work with. What we’ve seen is that it has, therefore, been increasingly supplanted by this grandiose and ill-defined objective of building a better world, especially in SSH, as exemplified in the newest mandate statement (quoted above), a mandate that requires many more instructions and many more regulations (and, hence, a longer document). Further, this change of mandate has occurred in spite of the fact that REBs routinely lack the required expertise to evaluate even the scientific methods that are proposed, much less the expertise to design a better world, and that there is no realistic way to overcome either of these shortfalls. Thus, the REB process becomes increasingly expensive in terms of time and money required, but more adept at stifling unpopular inquiry as more constraints need to be accommodated, all as a result of this attempt to change the mandate from protecting subjects to changing the world.
The increasing size of the TCPS is, for example, directly traceable to the idea that "good research" must produce certain outcomes. Specifically, "… research can benefit human society (line 64) …" is true enough, but this point seems to often be interpreted to mean that "research must benefit". It is acknowledged in the TCPS that "… researchers are not aid agencies …" (line 1586). True as this is, it does not go far enough: neither are REBs , nor is the research enterprise in general, and this fact ought to be made explicit as well.
Justice, fairness, equity, and so forth were not the original rationale for REBs, and these should not become the paramount objectives for researchers, REBs or the TCPS. The endnote for Chapter 1 (line 306) captures much of our frustration with the expanded mandate. There it states that “broader and more encompassing core principles” are said to provide a “more focused framework”. These statements would seem to be completely contradictory, making their merger virtually impossible to realize in practice.
(4) “Participant” not “Subject”.
It is puzzling that we continue to belabour what to call the focus of our research (line 344). There is nothing fundamentally wrong with the term "participant," but the extended justification for using participant everywhere seems to derive from an underlying creeping ideological bias (see Chapter 10, e.g., line 4341) that goes beyond semantics and methodology. The participants in some types of research are much less participatory than imagined, and “subject” is, therefore, actually more precise. In any case, the issue seems, to us, to get more attention than it warrants. Sometimes participant is accurate, sometimes subject, sometimes patient, sometimes client, and so forth, thus subsuming all cases as "participant" seems to be predicated on an underlying view that the research enterprise is a one-size-fits-all enterprise after all. To repeat, continued preoccupation with this terminology gains little, and the defensiveness around the preoccupation seems overdone.
(5) Issues surrounding sampling.
While it is always appropriate for a researcher to make sure that his or her sample is done in the best way possible, using a "convenience sample" does not seem at all unreasonable in many circumstances (line 1541). Why choose an inconvenient sample (B) to spend more time and money traveling when a convenient sample (A) is available, given that in many circumstances (in SSH) samples A and B will be equal in all other respects and thus convenience (time and money) is the sole difference? That is, if sample A and B are equally appropriate on all dimensions other than convenience, it is economical and efficient for a researcher to go with the more convenient sample (i.e., to pick the sample solely on the basis of convenience). It makes absolutely no sense that a decision like that would be prohibited by this version of the TCPS (lines 1541-1543).
Certainly, choosing a convenience sample when a better sampling technique is required should be discouraged, but that is not what is being said in the present draft (line 1543). This section needs to be deleted or at least worded more carefully.
(6) The nature of the appeal process.
The "principles of natural and procedural justice" (line 2571) probably would lead to the creation of an appeal process whereby the first appeal would be to the group that rendered the decision. However, if that is the "first and last" place to appeal, as in the current draft of the TCPS (line 2608), then the situation is more akin to a "kangaroo court". Further, that the federal agencies would want to avoid routine involvement in appeals (line 2606) is understandable at one level, but in a broader sense, for the federal agencies to completely deny responsibility for the policies they have created seems irresponsible and even wasteful. In fact, such appeals would be a rich opportunity to articulate the policies, and to educate the research and ethics review communities.
This list of problems with the current draft of the TCPS is a long one and, indeed, there are more issues that we could raise. However, what also needs to be said is that this draft of the TCPS does contain some definite improvements on the last version. Therefore, some progress is being made, however slowly. We would hope that at least some of the problems we have laid out here can be addressed in the final version of TCPS-2, particularly those that were described in response to the first version of the TCPS, problems that, over 10 years on, are still waiting to be addressed.
Sincerely,
John Mueller
Steve Lupker
Members of the Board of Directors of SAFS