SOCIETY FOR ACADEMIC FREEDOM AND SCHOLARSHIP (SAFS)

1673 Richmond Street #344, London, Ontario, N6G 2N3 email: safs@safs.ca

November 23, 2007

Sponsors Table Secretariat on Research Ethics

Experts Committee -- Moving Ahead

350 Albert Street

Ottawa, Canada K1A 1H5

secretariat@hrppc-pphrc.ca

(Fax: 613-996-7117)

     Re: Moving Ahead Proposal

Dear Committee Members:

We are a national organization dedicated to academic freedom and scholarship (http://www.safs.ca).  We are writing in response to a call for comment on a draft of "Moving Ahead," as per documents posted August 15, 2007, available on-line at < http://www.hrppc-pphrc.ca/english/movingahead.pdf > and developed by the Experts Committee for the Sponsors' Table for Human Research Participant Protection. 

We have in the past expressed concerns about the expansion of research ethics regulations, given the lack of demonstrated effectiveness for such regulations (cf. http://www.safs.ca/issuescases/ethics.html, http://www.safs.ca/interagencymain.html and http://www.safs.ca/issuescases/ncehr.html).  As we have noted previously, changes in the research ethics review system have consistently increased the burdens on individual researchers, in the social sciences and humanities (SSH) in particular, and on local university administrations, with no documented benefit in terms of the protection of research participants. 

We have examined the Moving Ahead (MA) proposal, and we offer some reactions to various aspects of that proposal, then a summary at the end.  The MA proposal, along with its companion documents (e.g., the costing document <http://www.hrppc-pphrc.ca/english/costing.html>) and the Public Assurance System (e.g., line 643 ff.), clearly has involved a great deal of work.  In fact, there has been a great deal of other activity re research ethics in the past year or so, to the point that it is a full-time job to stay abreast of it all.  This prolific pace makes it difficult for those of us outside the research ethics industry to assess the particulars, which, frankly, is a statement in its own right.  What are scholars supposed to gain from all this effort, especially those not doing medical research, and how would we know if we ever actually achieved that gain? 

The MA proposal seems to derive in part from a sense that research ethics is a wide-spread and often disjoint enterprise, a point on which we concur.  Over the past 30 years or so, one agency after another has appeared, and over time each agency has expanded its reach (the current euphemism being "creep").  Although there is some shared connection with the Tri-Council Policy Statement (TCPS), the proliferation of agencies seems to have occurred "spontaneously" rather than from legislative mandate or serious attempts at coordination.  

To deal with the proliferation of agencies that exist in the research ethics industry, the MA document proposes the creation of yet another agency, supposedly to coordinate the many activities that have emerged in the various agencies.  We think this should give everyone occasion to pause for reflection.  It is one thing if such an über-agency were to truly streamline the research ethics industry, that is, if it were to replace some or all of the existing agencies.  However, that is not the typical history of the evolution and organization of bureaucracies, not in general and certainly not in research ethics.  The lack of any conclusive discussion of just which existing agencies are to shrink or disappear (and how and when) is more consistent with the idea that this new agency will be merely an added layer of bureaucracy.  If the result of the proposal is that we just get "more" oversight (and we do fear that that will be the result), we would strongly argue against the creation of this new agency.    

One thing that we found unsettling is that there seems such reluctance in the MA proposal to procure a legislative or statutory mandate for this über-agency (l. 824 ff. and elsewhere).  To us it appears that without such a mandate, the agency will be just another agency created "spontaneously" and with no true authority over the existing agencies.  Why would the other agencies subordinate themselves to this agency?   If the existing agencies do have some legislated basis themselves, it would seem especially odd that they would defer to a new agency that lacked such statutory authority.  Further, if the existing agencies do have such a legislative authority, then does it not require a similar legislative act to create this über-agency?  We are not lawyers, but, given these issues, it would seem almost imperative that the consultation with the government that is noted but repeatedly dismissed here (e.g., l. 824) be undertaken before any of the ideas advanced in this document are put into practice.  Several places in the MA document it is noted that some possible direction might be difficult or impossible without statutory support, but then the proposal argues to go ahead anyway without seeking such authority.  If we are to reorganize the research ethics industry, we ought to do a thorough job and there should be clear authority for the changes.  Everyone, not just the agencies, would benefit from a clarification of the statutory basis for the research ethics industry. 

In essence then, although it is not impossible that a new organization might reduce some of the perceived problems with the present system, our impression is that it is highly unlikely.  The present conceptualization in particular seems to be a scheme merely for more rule/forms/agencies which will only serve to heighten the climate of compliance, and the time that dealing with the added complexity entails will further usurp the time needed for any culture of ethics to develop (not to mention unduly delaying innocuous research).   More rules and more oversight will not instill a climate favorable for researchers to contemplate ethics, it will just generate more rule-focused frustration.  The "inclusive environment" described in line 756, and the "virtuous learning loop" in line 815, and so forth, will not be achieved by ratcheting up the complexity of the ethics regulations and creating wonderment at just which agency has the "final say," especially not in those areas where research is of minimal risk in the first place (i.e., SSH). 

More detailed comments on these and other issues:

Pages 19-20, lines 486-491 and 503-508: We found it interesting that the TCPS covers only funded research.  Although the committee may view this situation as a "gap" (that is, not covering unfunded or privately funded research is a problem to be solved), to us this information is a welcome clarification.  We have made inquiries about this in recent years, especially in the wake of the recent developments in the USA.  Universities and other institutions in the USA have recently rediscovered that only funded research is legally covered by their Federal Wide Assurance agreements with the OHRP, and that they could handle unfunded research proposals differently.  The results of our queries, locally and federally, invariably have been assurances that the TCPS covered all research -- sometimes communicated with considerable indignation.  However, no one could provide the precise legal citation, which is now explained by the fact that there, apparently, is none.  Whether it is a good idea or not to cover all research as the MA document argues (and we certainly believe it is not), it will come as a surprise to many that a new agency (in fact, one with statutory authority) will be required to achieve what we've been told "everyone knows" is already true. 

p. 19, line 482:  Our dictionary definition of "battery" is "The unlawful beating of another".  We are not lawyers, but it seems quite an overstatement to equate the absence of a consent form with a "beating"?   (Preoccupation with hypothetical and hyperbolic outcomes has become a distracting indulgence in the research ethics industry.  Perhaps it is necessary in order to fill the gap created by the absence of real concerns in SSH research?)  Similarly, "negligence" (line 483) is an ad hoc assessment, which makes it a tort lawyer's dream, hindsight being 20-20.  Unfortunately, yet another agency will not transfer the benefits of hindsight to the preliminary review process.  Comments such as "could be sued" (l. 481) are essentially meaningless, because anyone can sue anyone about anything.  It is our belief that such matters of law must be left to the courts, or the institution's lawyers.  REBs simply are not qualified to make decisions about liability, negligence, and other legal issues, and the proposed über-agency will likewise not be qualified to make liability, negligence, or other legal decisions.  (A lawyer on an REB to handle such decisions should remind one of the old joke: when a town has one lawyer he is poor, but when there are two lawyers each will be rich.  The advice that "He who has himself as a lawyer has a fool for a client" also seems to apply here.)

p. 21, line 532 (and elsewhere, e.g., l. 622):  This point about a potential conflict of interest for the federal granting agencies to both provide the funds and also make decisions about ethics seems well taken.  Nonetheless, agencies with such self-policing powers invariably assure us they can handle such conflicts, because they absolutely do not want to give up self-policing.  What is there to make us think that anything would be different here, particularly if the über-agency lacks statutory authority?  Footnote 15 (p. 21) notes that there is a lack of support at "higher levels of government".  We don’t find that surprising, but why are no efforts being made to obtain such support so as to assure that this separation of powers will be obligatory? 

p. 22, line 553: Although it may be somewhat tangential to the MA document, the discussion of a "lack of harmonization" internationally has always struck us as very paternalistic.  What it inevitably comes down to is us imposing our ethics across all situations and nations, a blatant contradiction to the diversity of international research and scholarship.  Frankly, if the über-agency could first get Canadian REBs to trust one another from campus to campus, so as to stop re-reviewing something already approved elsewhere, that itself would be a miraculous achievement!  This nearly ubiquitous attitude that, "It isn't OK until we say it's OK", which we see in this document as well, has generated untold duplication of efforts and expense over the years.  Its existence here is just one more reason to believe adding on a new agency will never achieve any economy (especially without that agency having legislative authority).   

p. 23, line 570: This statement seems to imply that there are data on REB effectiveness somewhere.  If so, we would very much like to see them.  The word "transparency" is mentioned several times as a benefit of the über-agency (e.g., l. 519) and sharing such data on the effectiveness of REBs would certainly be a demonstration of such transparency.  Similar comments apply to statements about "best practices" (l. 662) and various claims of increased efficiency sprinkled throughout the proposal. 

p. 28: In passing, it seems to us that lines 683 and 688 are contradictory.

p. 28, l. 687: Are we to return to an expertise-based REB? 

p. 29, lines 716-736 (and lines 986-992):  Having identified the conflict of interest inherent in having federal granting agencies controlling both dollars and ethics, isn't there an equally obvious conflict of interest in having the education function of the REBs and the evaluation of the REBs carried out by the same body (a point that is, apparently, acknowledged later in the document)?  As noted above, of course, if both functions were to reside in the same body, we have no doubt that the über-agency will assure us that it can handle the conflict.  However, it is always better when the two potentially conflicting objectives are handled at arm's length from one another.  Why is conflict of interest a problem with the TCPS and federal agencies, but not to be a problem within the new über-agency? 

p. 32, lines 777-780:  Given that there will be an interim period of "two-tiered" systems, we would like to point out that this provides an excellent naturalistic opportunity to collect data to compare the two "tiers," to see if accreditation actually makes any difference?  We would expect the data to be shared publicly, of course. 

p. 32, l. 785 ff.:  In theory, Regional REBs could replace local REBs, however history suggests that won't happen.  We believe that, in actual practice, local REBs will continue to review or prescreen proposals, officially or unofficially.  In that case there are no savings, just the added cost of the Regional REBs.  (This seems especially likely given that local REBs are to be retained for student projects, lines 813-814.)

p. 33, l. 802: In regard to the members of the Regional REBs, where are these people to come from?  Appointed?  By whom?  Hired?  Paid by whom?  What expertise or credentials?  Will this be another agency with 90%-plus members from medical research backgrounds?  

p. 34, l. 818-819:  Institutions must accept that REBs can't make liability assessments, the TCPS can't cover liability, and the über-agency will likewise be out of its depth re liability.  Liability is not a public safety issue, nor is it an ethical issue.  Liability is a legal decision and its evaluation should be removed explicitly from REBs and turned over to the institution's legal staff.  (Likewise, REBs should be cautioned against evaluating methodology, especially given the continued insistence that REBs be populated on grounds other than expertise.)  

p. 34, lines 824-856.  We have noted above our concern with this hesitation to get statutory authority for the proposed new agency (and apparently also for the existing agencies?).  In addition to the advantages of getting such authority (cited above), a legislative authority would clarify just what rebs are to do, but also what they are not to do, something that is lacking at the moment.  Without the force of law to dictate economizing (e.g., eliminating local REBs or other federal agencies), the über-agency will just be an added expense and an already complex system will become even more unwieldy. 

p. 34, l. 831: In regard to "increased compliance," just what is the present rate of compliance?  How will we document improvement?  This could be read as implying that non-compliance is rampant, which we do not perceive either to be:  a) true or b) one of the present problems. 

p. 34, l. 834:  In regard to "all research could be captured" -- couldn't that only happen with a legislative edict, something that this proposal seems determined to avoid? 

p. 35, l. 847: As part of being accountable to Parliament, one presumes this also means accountable to the Auditor General? 

p. 40, line 959:  One oversight system may in theory be workable, but we have two concerns, based substantially on the fact that efforts to make one size fit all have not worked so far.  First, we are still of the opinion that there are differences between medical and SSH research having to do with the "patient" role and other conceptual differences, not just in terms of probability and degree of risk.  With respect to the issue of risk, what is noted here (l. 967-968) may be true, that there may be "some" high-risk SSH research, but as stated in the MA proposal that characterization is so misleading that it trivializes fundamental differences.  It would be just as accurate and far more insightful to say that there is almost no high-risk SSH research.  SSH research is virtually all low risk research, whereas the bulk of medical research is by comparison very high risk (a good bit even potentially life threatening).  The relative concentration of research by degree of harm may be represented in this rough diagram, with box size illustrating variations in proportion of cases in the four categories.  Medical and non-medical research are simply not interchangeable in terms of risk profile, and it is not unreasonable to wonder why they should be scrutinized as if they are interchangeable.

We think SSH research involves important differences from medical research, but even if one system theoretically could work, the present effort to do so has, in fact, not worked.  One reason for this is that at the local level the present system encourages REBs to treat all SSH research as comparable to high-risk medical research, in spite of the explicit categories for exempt, expedited, and proportionate review in the TCPS.  We already have, in principle, ways to review by relative risk, however, these have not been followed in local practice, so why will local REBs now more rationally adjust reviews on the grounds of degree of risk?  It is not that reviews should necessarily differ by discipline per se (l. 971), but that discipline is thoroughly confounded with risk.  Nonetheless, local REBs have strong incentives to treat everything as high risk and they still will.  The task demand for local REBs has been "There is a problem, find it," which is a very different task than "Is there undue risk?"  "Find the problem" may be justified in high-risk research, but it is nonsense in low-risk research, which is the bulk of SSH research.  Why/how is this wasted effort going to change with a new über-agency (especially one without statutory authority)? 

In essence, it seems clear that the risk profiles are sufficiently different that it is not inappropriate for ordinary discussion to differentiate SSH from medical research by reference to disciplines.  Further, there are other reasons to do so as well.  For example, there is the medical perspective of "subject as patient," a conception that paints the subject as a vulnerable victim in a very unequal power relationship.  That is, the patient is indeed at the mercy of the physician as healer.   However appropriate it might be in medical research, the patient model ignores the fact that the bulk of SSH research is between parties with no such imbalance of power based on life and death cooperation.  Instead, the relationship between the investigator and the subject in SSH research is very similar to an encounter between a journalist and an interviewee, a relationship that manages to work quite well without a priori review. 

We think there are further reasons to be concerned with one oversight system for all disciplines, including the pattern that the existing agencies are so overwhelmingly populated by people who are medical practitioners or bureaucrats.  That may be appropriate given the locus of the true risks (in other words, it tacitly acknowledges the high-risk proportionality in the diagram above), but it also speaks volumes to a disconnect between medical research and other research.  To be credible, a single oversight system would have to have a far more balanced representation from non-medical research than has been the case historically.  After all, if one system can do it all, as implied by the present document, why can't an über-agency of 90% Historians work as readily as one with 90% medical members?  If that seems an outrageous proposition, why is it completely acceptable in reverse?  And, if 90% Historians does seem absurd, then it should be clear that deep down that we all realize that there are very good grounds for questioning an über-agency.

Having gone to some length at this point in the MA proposal to make the case for "one size fits all," later on the proposal does seem to suggest that there are differences among disciplines (p. 49, lines 1164-1180).   We find the latter discussion more encouraging, though it does seem somewhat at odds with the argument for one oversight agency.  Is this section perhaps merely a token statement to appease the multitude of us who have complained about this issue since the creation of the TCPS? 

p. 41, lines 986-992 (and l. 1040-1092, and clearly in l. 1240-1257):  Although there may be an argument to combine the three functions (policy, education, accreditation) as discussed, there is an inherent conflict of interest.  Although this is acknowledged in the MA proposal, we need to state, once again, that we are far less optimistic than the proposal's authors that this conflict is manageable.  Organizations in a conflict of interest invariably think that they can manage the conflict, but from outside the organization such self-policing is invariably a suspect policy.  Self-policing actually works against accountability and transparency, in part because it tends to deflect external criticism in an unresponsive manner.  We have discussed this issue further above. 

p. 42, lines 1023-1024 (also lines 1051-1052):  The culture of ethics might be a worthy goal, but we think that an added layer of bureaucracy will have just the opposite effect to cultivating a culture of ethics and instead the new layer will simply push researchers even further into a culture of rule-driven frustration, especially in low-risk research.  Another bureaucracy will only increase the duplication and inconsistencies referred to in lines 1021-1022, and not reduce these as claimed.  In our view, the much discussed "culture of ethics" has not been achieved so far in part because of disconnects between federal and local operations, but also because of inconsistencies inherent at each level.  Added complexity will only exacerbate the preoccupation with "rules" when one's main objective is a low-risk research project instead of a full-time career in finding ways to satisfy regulations. 

p. 43, l. 1030: Was the intent to also cover non-funded research in this statement?  Doing so would actually seem to give this new agency control over activities far afield from current coverage, such as investigative journalists, think tanks, and so forth.  Just what are the limits of control in the new scheme: what information can be evaluated, and by whom, without the intervention of the new agency?   Given the history of "creep" some clear boundaries should be laid out at the outset. 

p. 43, l. 1035:  The meaning of "peer" here is ambiguous.  For example, does it include researchers as well as "bureaucrats"?  Does it include individuals or just agencies?  And so forth.  The same questions arise for line 1036, also re the expertise referred to in lines 1102 and 1114, the meaning of "research community" in l. 1124, the meaning of the word "all" in l. 1137, the meaning of "representatives" in l. 1363, "knowledgeable individuals" in l. 1412.  We think that most researchers would be surprised to learn that "We are all in this together."  It is certainly far from the experience that most of us have lived with in our interactions with local REBs.   

p. 43, l. 1037 (and l. 1086, and elsewhere):  How is accountability to be demonstrated?  By reports?  To whom?  How often?  Who has access to the data?  And so forth.

p. 43, l. 1038: This is another of several claims of "greater efficiency".  Without an evaluation plan to document the gain, these claims are just meaningless rhetorical assertions.  See also, e.g., line 1053, line 1064, line 1074, and elsewhere.

p. 44, lines 1051-1052:  We have noted above that the proliferation of rules seems likely to work against a "culture of ethics," but we also have some further concerns with the expression "responsible conduct of research."  This appears to be an obvious goal, and perhaps it is only intended as a paraphrase of a "culture of ethics," but it is actually quite ambiguous.  Our concern is that it opens the door to evaluating research in terms of whether it is "worthwhile," that is, are the research outcomes socially and/or politically desirable.  This interpretation has nothing to do with public safety, but it surely provides ample "justification" for censorship.  Unqualified, this expression, "responsible research," invites further local distortion, and so it requires further thought and articulation to distance it from pandering to "desirable" research.

p. 49, lines 1182-1222:  This section in general would seem to us to beg for seeking a bona fide legislative authority for this enterprise.  The case for getting such approval is clear, why avoid it?

p. 52, l. 1236: Here and elsewhere the proposal seems to depend on the judgment that things are not working now in the sense that there is some large problem with public trust.  Just how strong is public distrust, and where can we find the evidence that documents this distrust?  Personally, we have never seen any serious public concern, but we do see a lot of "internal" frustration on the part of researchers.  The MA proposal may well have no impact on alleviating whatever public concern actually exists (if any does), all the while increasing the more readily apparent internal frustrations. 

p. 58, l. 1376:  Just what will be the consequences of an "adverse decision"?  The same question exists for "withdrawing accreditation" in l. 1406.  Does this mean that local REBs that over-reach and distort their responsibility will be "disciplined" somehow?  If so, what will be the implications for all the rest of the research they have approved?  What will be the implications for the research that is about to be approved?

p. 60, l. 1433:  This is the point where it is critical that the über-agency be clear that it will not be just an add-on, and that some of the existing agencies be abolished as would be expected if the objective is greater efficiency.  This is not a specific recommendation to abolish PRE or NCEHR in particular, but just a reminder that the elimination of some of the existing operations must be what any new agency is all about.  The intent may not be a larger bureaucracy (l. 1427), but one reasonably predicts that one will be created unless a plan for elimination is put in place.  (As noted on p. 61 (l. 1445), NCHER is an independent group, and in such cases there would be real benefits from genuine legislative authority to expedite some reduction.) 

p. 64, l. 1508-1517:  The case here is again clear: statutory authority should be sought (likewise l. 1488-1489).

Key Points: A Summary

Skepticism re universal model.  The basic assumption of the MA proposal continues to be that a single set of regulations can cover all areas of research scholarship.   We are quite convinced that this is simply wrong.  In particular, the medical research bias that has been repeatedly criticized is again promoted, both in terms of the substance of the regulations and the personnel.  We have been going in a very questionable direction and it appears that the authors of this document are content to go even further in that direction. 

Lack of authority.  In spite of making the case more than once for the benefits of statutory authorization, the MA proposal repeatedly backs away from seeking legislative authentication.  We are not sure that getting such authority would be such a daunting task as is implied in the proposal, and it would seem to have benefits.  Greater effort should be directed to realizing this objective, rather than giving up without seriously pursuing it.  As it stands, a reader may rightly wonder why it is being avoided (e.g., is there concern on the part of the authors that, in doing so, some of the authority currently claimed by the ethics bureaucracy might be scaled back?).

Self-policing.  One raison d' etat for the MA proposal is an existing conflict of interest, but then the proposal ends up proposing a situation involving a different conflict of interest.  How can a conflict of interest in the federal agencies be the reason that an über-agency is needed, but then conflict of interest is an acceptable operating plan for the über-agency?  We have some sympathy with correcting the existing conflict of interest, but we do not believe that replacing it with another one is any improvement. 

Expansion without assessment.  There are various claims of improved protection for subjects, improved efficiencies of operations, greater accountability and so forth, but with no assessment scheme.  We anticipate that it will be easy to document the added costs of the über-agency, but impossible to verify the claimed improvements.  This has been the history of the research ethics enterprise. 

Accreditation.  Having commented on this in the past, we will only reiterate that accreditation will not prevent research problems arising from human error, nor will it prevent deliberate efforts to circumvent the system.  Its costs will certainly exceed its benefits, which may be why there seems so little interest in documenting an assessment scheme for it.  However, we expect that this über-agency will eventually come to be, solely for its showcase value.  As we have seen time and time again, the appearance of doing something is actually much more important than actually solving any problems. 

Proliferation.  We have recently commented upon another proposal, the Continuing Ethics Review (CER) document, and we are struck by a parallel in the two documents.  The CER proposes an additional committee at the local level, and this MA document proposes as additional committee at the national level.  This is precisely the type of expansion that we have been referring to above, coming as it does with no evidence that added committees protect subjects better, but certainly with more local expenses.  These expansions are especially worrisome to us given that our concerns focus on research in the social sciences and humanities, where the hazards are minimal and rare.  Although there is some expectation in the MA document for federal funding (l. 1469 ff. and 1519 ff.) of the new agency and some recognition of the risks of overloading the volunteer base (p. 62, l. 1474-1483), it is obvious that, if the proposals here are implemented, there will be more hard expenses for local operations that have to come from local budgets (and not just indirect costs as implied, l. 1579-1581). 

In short, we are very unsupportive of the proposed "changes" in the MA document.  While the current system is far from perfect, these changes certainly don't address the concerns that we continue to have with that system.  Neither expansion implements an assessment scheme to document that subject protection has been improved, certainly not one that would satisfy the Auditor General, and in both expansions there are many questions apparent to an outsider.  The most obvious results of these two expansions (CER and MA) will be to create more jobs in the "ethics industry" (e.g., l. 1519), complicate the lines of authority, and usurp more local resources, and to do so without first gaining statutory authority. 

Sincerely,

John Mueller PhD

Stephen Lupker PhD

Members of SAFS Board of Directors