President's Letter: Joint committee report tough on metrics, but admin refused language protecting Members
I’m writing you today with an update on our agreement with Administration on Research metrics, contained in Appendix H of the Collective Agreement.
The agreement was to strike a joint committee to study research metrics, including bibliometrics, and have the committee determine whether to adopt additional proposed language into the collective agreement (also found in Appendix ‘H’). That language would give each individual Member the right to choose whether or not metrics would be used to assess their work. Administration and UMFA each appointed three members to the committee, and four or more votes were needed to officially adopt the language in the CA.
Following an investigation of the existing research on bibliometrics, the working group unanimously agreed to several principles, including:
- Regular assessment and evaluation is important for both guidance and career progression;
- The h-index and its relatives do not indicate the quality of the work of an individual researcher;
- Bibliometrics alter the focus of research in undesirable ways when maximizing one’s bibliometric rankings becomes a goal;
- Components of bibliometrics are gender-biased;
- Databases used to calculate bibliometrics are not well-correlated at the individual level; and
- There are currently no metrics that suit the criteria for usability in the evaluation of individuals.
The UMFA Members of the committee devoted incredible time and energy to this work. Please join me in the thanking our UMFA appointees in particular: Fletcher Baragar (Faculty of Arts), Colin Garroway (Faculty of Science), and Sherri Vokey (Health Sciences Libraries). Julie Gibbings (Faculty of Arts) also contributed.
Despite this strong indictment of research metrics, however, the three administration appointees voted against the inclusion of the language in Appendix H due to disagreement “centred around the specific language being proposed”. That collective agreement language would:
- prohibit criteria for promotion, tenure, continuing appointments, and performance evaluations that include a standard or expectation based on a quantitative measure or research metric;
- prohibit requiring research metrics and other quantitative measures for performance evaluations, tenure, promotion, and continuing appointments;
- only allow use of research metrics if personally and voluntarily submitted by the Member being evaluated or assessed, and;
- prohibit anyone from penalizing or making adverse inference about a Member or their performance for non-submission of research metrics.
Given the report’s indictments of research metrics, we were disappointed with the administration’s refusal to give Members the right to decide when, if ever, metrics could be used to assess their own work. No details regarding the disagreements with these proposals were provided in the report. So, we sought clarity from the administration.
We requested a president’s liaison meeting in April 2018 pursuant to Section 5.4 of the collective agreement. After much back and forth, we finally met with the president’s delegates, not the president himself, on October 26, 2018. We argued that the language in Appendix H did not prohibit members from submitting research metrics, but protected members from effectively having inappropriate and biased metrics imposed upon them.
This was an opportunity for UMFA and UM to demonstrate to our members and the community that we can work together collaboratively. The language would have helped to ensure that evaluation at our university is fair, accurate, and based on a full consideration of research quality. Our stand on performance metrics, and this language, drew support from faculty associations all over the world during our strike. We would have welcomed seeing UM take a leadership role in promoting academic freedom, protecting members of equity-seeking groups, and helping ensure academic assessment continues to be based on rigourous peer evaluation.
We had originally asked to hear the administration’s answer by November 16, 2018. The administration asked for several extensions and we checked back with them several times over the intervening three months. We finally said that if we heard nothing by February 13, we would have to assume that they are rejecting our proposal and proceed accordingly.
We received their response on the day of our deadline. While we hoped otherwise, we were not surprised that administration rejected our proposal to incorporate the appendix H language. They have instead suggested that we bring the committee back to work on alternative language that can serve as a headstart in our next bargaining round.
This is not a workable solution: the language in Appendix H was the compromise language, produced after months of negotiations. It was accepted by both parties because it responded to each side’s respective concerns by leaving the question up to the discretion of each individual member rather than the administration or the union. It was to be adopted if the committee concluded that metrics were problematic, and the committee did conclude that metrics were problematic. We’re incredibly disappointed that the administration has rejected language that in 2016 they agreed could reasonably resolve the issue.
We encourage all of you to read the report (located here). We all serve on promotion, tenure, grant, and award committees, so it is essential that we understand the problems with research metrics, many of which are also applicable to the use of SEEQ’s. It is also important that we educate our graduate students as future faculty members of the dangers of research metrics. They need to be trained to perform rigourous, fair peer evaluation, and need to learn how to explain the significance of their contributions to people inside and outside their discipline without resorting to the use of research metrics.
While we were not successful in persuading UM administration to protect vulnerable members from the imposition of research metrics, we did gain language in the 2016 bargaining round that gave each of us the power to improve the situation by protecting each other. Sections 20.C.2 and 19.D.5.2, adopted in 2016, allow promotion and tenure committee members to report concerns about procedural defects to the chair of the tenure committee, the dean/director (or University Librarian in the case of a committee for an academic librarian), the VP Academic, Staff Relations, or to UMFA. If, as a committee member, you feel that the committee failed to do an assessment based on a full review of the quality of the member’s contributions, and that research metrics were used as a substitute for a more comprehensive assessment of quality and quantity, please speak up.
An even more powerful way to prevent the incursion of bibliometrics into our academic lives is through your unit’s promotion and tenure guidelines. In 2016, we won far greater control over their development: guidelines will be produced by an advisory committee elected by members. The dean ensures that the promotion/tenure criteria are fair and appropriate for all disciplines, consistent with the CA, university policies and the law: as long as this is the case, your dean/director must adopt the guidelines if the majority of members vote for their adoption. Therefore, the language on research metrics proposed in 2016 can form part of your unit’s tenure and promotion guidelines even if they are not in the collective agreement: have the advisory committee incorporate them and the members voting to approve them. You’ll find details on how to do this in a document that I’m circulating with this letter.
The collective effort of improving our working lives doesn’t end in bargaining, but continues as we exercise the rights that together we’ve won. Please read through not only the report, but the attached description of how to revisit and revise your tenure and promotion guidelines. Understanding and exercising your rights is one of the most powerful ways to extend and improve them.