Download Lectures on Biostatistics (1971). Corrected and searchable version of Google books edition
Download review of Lectures on Biostatistics (THES, 1973).
Academic staff are going to be fired at Queen Mary University of London (QMUL). It’s possible that universities may have to contract a bit in hard times, so what’s wrong?
What’s wrong is that the victims are being selected in a way that I can describe only as insane. The criteria they use are guaranteed to produce a generation of second-rate spiv scientists, with a consequent progressive decline in QMUL’s reputation.
The firings, it seems, are nothing to do with hard financial times, but are a result of QMUL’s aim to raise its ranking in university league tables.
In the UK university league table, a university’s position is directly related to its government research funding. So they need to do well in the 2014 ‘Research Excellence Framework’ (REF). To achieve that they plan to recruit new staff with high research profiles, take on more PhD students and post-docs, obtain more research funding from grants, and get rid of staff who are not doing ‘good’ enough research.
So far, that’s exactly what every other university is trying to do. This sort of distortion is one of the harmful side-effects of the REF. But what’s particularly stupid about QMUL’s behaviour is the way they are going about it. You can assess your own chances of survival at QMUL’s School of Biological and Chemical Sciences from the following table, which is taken from an article by Jeremy Garwood (Lab Times Online. July 4, 2012). The numbers refer to the four year period from 2008 to 2011.
|
Category of staff |
Research Output Quantity |
Research Output |
Research Income (£) |
Research Income (£) |
|
Professor |
11 |
2 |
400,000 |
at least 200,000 |
|
Reader |
9 |
2 |
320,000 |
at least 150,000 |
|
Senior Lecturer |
7 |
1 |
260,000 |
at least 120,000 |
|
Lecturer |
5 |
1 |
200,000 |
at least 100,000 |
|
In addition to the three criteria, ‘Research Output ‐ quality’, ‘Research Output – quantity’, and ‘Research Income’, there is a minimum threshold of 1 PhD completion for staff at each academic level. All this data is “evidenced by objective metrics; publications cited in Web of Science, plus official QMUL metrics on grant income and PhD completion.” To survive, staff must meet the minimum threshold in three out of the four categories, except as follows: Demonstration of activity at an exceptional level in either ‘research outputs’ or ‘research income’, termed an ‘enhanced threshold’, is “sufficient” to justify selection regardless of levels of activity in the other two categories. And what are these enhanced thresholds? |
|
The university notes that the above criteria “are useful as entry standards into the new school, but they fall short of the levels of activity that will be expected from staff in the future. These metrics should not, therefore, be regarded as targets for future performance.” This means that those who survived the redundancy criteria will simply have to do better. But what is to reassure them that it won’t be their turn next time should they fail to match the numbers? To help them, Queen Mary is proposing to introduce ‘D3’ performance management (www.unions.qmul.ac.uk/ucu/docs/d3-part-one.doc). Based on more ‘administrative physics’, D3 is shorthand for ‘Direction × Delivery × Development.’ Apparently “all three are essential to a successful team or organisation. The multiplication indicates that where one is absent/zero, then the sum is zero!” D3 is based on principles of accountability: “A sign of a mature organisation is where its members acknowledge that they face choices, they make commitments and are ready to be held to account for discharging these commitments, accepting the consequences rather than seeking to pass responsibility.” Inspired? |
I presume the D3 document must have been written by an HR person. It has all the incoherent use of buzzwords so typical of HR. And it says "sum" when it means "product" (oh dear, innumeracy is rife).
The criteria are utterly brainless. The use of impact factors for assessing people has been discredited at least since Seglen (1997) showed that the number of citations that a paper gets is not perceptibly correlated with the impact factor of the journal in which it’s published. The reason for this is the distribution of the number of citations for papers in a particular journal is enormously skewed. This means that high-impact journals get most of their citations from a few articles.
|
The distribution for Nature is shown in Fig. 1. Far from being gaussian, it is even more skewed than a geometric distribution; the mean number of citations is 114, but 69% of papers have fewer than the mean, and 24% have fewer than 30 citations. One paper has 2,364 citations but 35 have 10 or fewer. ISI data for citations in 2001 of the 858 papers published in Nature in 1999 show that the 80 most-cited papers (16% of all papers) account for half of all the citations (from Colquhoun, 2003)
|
|
The Institute of Scientific Information, ISI, is guilty of the unsound statistical practice of characterizing a distribution by its mean only, with no indication of its shape or even its spread. School of Biological and Chemical Sciences-QMUL is expecting everyone has to be above average in the new regime. Anomalously, the thresholds for psychologists are lower because it is said that it’s more difficult for them to get grants. This undermines even the twisted logic applied at the outset.
All this stuff about skewed distributions is, no doubt, a bit too technical for HR people to understand. Which, of course, is precisely why they should have nothing to do with assessing people.
At a time when so may PhDs fail to get academic jobs we should be limiting the numbers. But QMUL requires everyone to have a PhD student, not for the benefit of the student, but to increase its standing in league tables. That is deeply unethical.
The demand to have two papers in journals with impact factor greater than seven is nonsense. In physiology, for example, there are only four journals with an impact factor greater that seven and three of them are review journals that don’t publish original research. The two best journals for electrophysiology are Journal of Physiology (impact factor 4.98, in 2010) and Journal of General Physiology (IF 4.71). These are the journals that publish papers that get you into the Royal Society or even Nobel prizes. But for QMUL, they don’t count.
I have been lucky to know well three Nobel prize winners. Andrew Huxley. Bernard Katz, and Bert Sakmann. I doubt that any of them would pass the criteria laid down for a professor by QMUL. They would have been fired.
The case of Sakmann is analysed in How to Get Good Science, [pdf version]. In the 10 years from 1976 to 1985, when Sakmann rose to fame, he published an average of 2.6 papers per year (range 0 to 6). In two of these 10 years he had no publications at all. In the 4 year period (1976 – 1979 ) that started with the paper that brought him to fame (Neher & Sakmann, 1976) he published 9 papers, just enough for the Reader grade, but in the four years from 1979 – 1982 he had 6 papers, in 2 of which he was neither first nor last author. His job would have been in danger if he’d worked at QMUL. In 1991 Sakmann, with Erwin Neher, got the Nobel Prize for Physiology or Medicine.
The most offensive thing of the lot is the way you can buy yourself out if you publish 26 papers in the 4 year period. Sakmann came nowhere near this. And my own total, for the entire time from my first paper (1963) until I was elected to the Royal Society (May 1985) was 27 papers (and 7 book chapters). I would have been fired.
Peter Higgs had no papers at all from the time he moved to Edinburgh in 1960, until 1964 when his two paper’s on what’s now called the Higgs’ Boson were published in Physics Letters. That journal now has an impact factor less than 7 so Queen Mary would not have counted them as “high quality” papers, and he would not have been returnable for the REF. He too would have been fired.
The encouragement to publish large numbers of papers is daft. I have seen people rejected from the Royal Society for publishing too much. If you are publishing a paper every six weeks, you certainly aren’t writing them, and possibly not even reading them. Most likely you are appending your name to somebody else’s work with little or no checking of the data. Such numbers can be reached only by unethical behaviour, as described by Peter Lawrence in The Mismeasurement of Science. Like so much managerialism, the rules provide an active encouragement to dishonesty.
In the face of such a boneheaded approach to assessment of your worth, it’s the duty of any responsible academic to point out the harm that’s being done to the College. Richard Horton, in the Lancet, did so in Bullying at Barts. There followed quickly letters from Stuart McDonald and Nick Wright, who used the Nuremburg defence, pointing out that the Dean (Tom Macdonald) was just obeying orders from above. That has never been as acceptable defence. If Macdonald agreed with the procedure, he should be fired for incompetence. If he did not agree with it he should have resigned.
It’s a pity, because Tom Macdonald was one of the people with whom I corresponded in support of Barts’ students who, very reasonably, objected to having course work marked by homeopaths (see St Bartholomew’s teaches antiscience, but students revolt, and, later, Bad medicine. Barts sinks further into the endarkenment). In that case he was not unreasonable, and, a mere two years later I heard that he’d taken action.
To cap it all, two academics did their job by applying a critical eye to what’s going on at Queen Mary. They wrote to the Lancet under the title Queen Mary: nobody expects the Spanish Inquisition
"For example, one of the “metrics” for research output at professorial level is to have published at least two papers in journals with impact factors of 7 or more. This is ludicrous, of course—a triumph of vanity as sensible as selecting athletes on the basis of their brand of track suit. But let us follow this “metric” for a moment. How does the Head of School fair? Zero, actually. He fails. Just consult Web of Science. Take care though, the result is classified information. HR’s “data” are marked Private and Confidential. Some things must be believed. To question them is heresy."
Astoundingly, the people who wrote this piece are now under investigation for “gross misconduct”. This is behaviour worthy of the University of Poppleton, as pointed out by the inimitable Laurie Taylor, in Times Higher Education (June 7)
|
The rustle of censorship It appears that last week’s edition of our sister paper, The Poppleton Evening News, carried a letter from Dr Gene Ohm of our Biology Department criticising this university’s metrics-based redundancy programme. We now learn that, following the precedent set by Queen Mary, University of London, Dr Ohm could be found guilty of “gross misconduct” and face “disciplinary proceedings leading to dismissal” for having the effrontery to raise such issues in a public place. Louise Bimpson, the corporate director of our ever-expanding human resources team, admitted that this response might appear “severe” but pointed out that Poppleton was eager to follow the disciplinary practices set by such soon-to-be members of the prestigious Russell Group as Queen Mary. Thus it was only to be expected that we would seek to emulate its espousal of draconian censorship. She hoped this clarified the situation. |
David Bignell, emeritus professor of zoology at Queen Mary hit the nail on the head.
"These managers worry me. Too many are modest achievers, retired from their own studies, intoxicated with jargon, delusional about corporate status and forever banging the metrics gong. Crucially, they don’t lead by example."
What the managers at Queen Mary have failed to notice is that the best academics can choose where to go.
People are being told to pack their bags and move out with one day’s notice. Access to journals stopped, email address removed, and you may need to be accompanied to your (ex)-office. Good scientists are being treated like criminals.
What scientist in their right mind would want to work at QMUL, now that their dimwitted assessment methods, and their bullying tactics, are public knowledge?
The responsibility must lie with the principal, Simon Gaskell. And we know what the punishment is for bringing your university into disrepute.
Follow-up
Send an email. You may want to join the many people who have already written to QMUL’s principal, Simon Gaskell (principal@qmul.ac.uk), and/or to Sir Nicholas Montagu, Chairman of Council, n.montagu@qmul.ac.uk.
Sunday 1 July 2012. Since this blog was posted after lunch on Friday 29th June, it has had around 9000 visits from 72 countries. Here is one of 17 maps showing the origins of 200 of the hits in the last two days
The tweets about QMUL are collected in a Storify timeline.
I’m reminded of a 2008 comment, on a post about the problems imposed by HR, In-human resources, science and pizza.
Thanks for that – I LOVED IT. It’s fantastic that the truth of HR (I truly hate that phrase) has been so ruthlessly exposed. Should be part of the School Handbook. Any VC who stripped out all the BS would immediately retain and attract good people and see their productivity soar.
That’s advice that Queen Mary should heed.
Part of the reason for that popularity was Ben Goldacre’s tweet, to his 201,000 followers
“destructive, unethical and crude metric incentives in academia (spotlight QMUL) bit.ly/MFHk2H by @david_colquhoun”
3 July 2012. I have come by a copy of this email, which was sent to Queen Mary by a senior professor from the USA (word travels fast on the web). It shows just how easy it is to destroy the reputation of an institution.
|
Sir Nicholas Montagu, Chairman of Council, and Principal Gaskell, I was appalled to read the criteria devised by your University to evaluate its faculty. There are so flawed it is hard to know where to begin. Your criteria are antithetical to good scientific research. The journals are littered with weak publications, which are generated mainly by scientists who feel the pressure to publish, no matter whether the results are interesting, valid, or meaningful. The literature is flooded by sheer volume of these publications. Your attempt to require “quality” research is provided by the requirement for publications in “high Impact Factor” journals. IF has been discredited among scientists for many reasons: it is inaccurate in not actually reflecting the merit of the specific paper, it is biased toward fields with lots of scientists, etc. The demand for publications in absurdly high IF journals encourages, and practically enforces scientific fraud. I have personally experienced those reviews from Nature demanding one or two more “final” experiments that will clinch the publication. The authors KNOW how these experiments MUST turn out. If they want their Nature paper (and their very academic survival if they are at a brutal, anti-scientific university like QMUL), they must get the “right” answer. The temptation to fudge the data to get this answer is extreme. Some scientists may even be able to convince themselves that each contrary piece of data that they discard to ensure the “correct” answer is being discarded for a valid reason. But the result is that scientific misconduct occurs. I did not see in your criteria for “success” at QMUL whether you discount retracted papers from the tally of high IF publications, or perhaps the retraction itself counts as yet another high IF publication! Your requirement for each faculty to have one or more postdocs or students promotes the abusive exploitation of these individuals for their cheap labor, and ignores the fact that they are being “trained” for jobs that do not exist. The “standards” you set are fantastically unrealistic. For example, funding is not graded, but a sharp step function – we have 1 or 2 or 0 grants and even if the average is above your limits, no one could sustain this continuously. Once you have fired every one of your faculty, which will almost certainly happen within 1-2 rounds of pogroms, where will you find legitimate scientists who are willing to join such a ludicrous University? |
4 July 2012.
Professor John F. Allen is Professor of Biochemistry at Queen Mary, University of London, and distinguished in the fields of Photosynthesis, Chloroplasts, Mitochondria, Genome function and evolution and Redox signalling. He, with a younger colleague, wrote a letter to the Lancet, Queen Mary: nobody expects the Spanish Inquisition. It is an admirable letter, the sort of thing any self-respecting academic should write. But not according to HR. On 14 May, Allen got a letter from HR, which starts thus.
|
14th May 2012 Dear Professor Allen I am writing to inform you that the College had decided to commence a factfinding investigation into the below allegation: That in writing and/or signing your name to a letter entitled "Queen Mary: nobody expects the Spanish Inquisition," (enclosed) which was published in the Lancet online on 4th May 2012, you sought to bring the Head of School of Biological and Chemical Sciences and the Dean for Research in the School of Medicine and Dentistry into disrepute. . . . . Sam Holborn |
Download the entire letter. It is utterly disgraceful bullying. If anyone is bringing Queen Mary into disrepute, it is Sam Holborn and the principal, Simon Gaskell.
Here’s another letter, from the many that have been sent. This is from a researcher in the Netherlands.
|
Dear Sir Nicholas,
I am addressing this to you in the hope that you were not directly involved in creating this extremely stupid set of measures that have been thought up, not to improve the conduct of science at QMUL, but to cheat QMUL’s way up the league tables over the heads of the existing academic staff. Others have written more succinctly about the crass stupidity of your Human Resources department than I could, and their apparent ignorance of how science actually works. As your principal must bear full responsibility for the introduction of these measures, I am not sending him a copy of this mail. I am pretty sure that his “principal” mail address will no longer be operative. We have had a recent scandal in the Netherlands where a social psychology professor, who even won a national “Man of the Year” award, as well as as a very large amount of research money, was recently exposed as having faked all the data that went into a total number of articles running into three figures. This is not the sort of thing one wants to happen to one’s own university. He would have done well according to your REF .. before he was found out. Human Resources departments have gained too much power, and are completely incompetent when it comes to judging academic standards. Let them get on with the old dull, and gobbledigook-free, tasks that personnel departments should be carrying out. |
5 July 2012.
Here’s another letter. It’s from a member of academic staff at QMUL, someone who is not himself threatened with being fired. It certainly shows that I’m not making a fuss about nothing. Rather, I’m the only person old enough to say what needs to be said without fear of losing my job and my house.
|
Dear Prof. Colquhoun,
I am an academic staff member in SBCS, QMUL. I am writing from my personal email account because the risks of using my work account to send this email are too great. I would like to thank you for highlighting our problems and how we have been treated by our employer (Queen Mary University of London), in your blog. I would please urge you to continue to tweet and blog about our plight, and staff in other universities experiencing similarly horrific working conditions. I am not threatened with redundancy by QMUL, and in fact my research is quite successful. Nevertheless, the last nine months have been the most stressful of all my years of academic life. The best of my colleagues in SBCS, QMUL are leaving already and I hope to leave, if I can find another job in London. Staff do indeed feel very unfairly treated, intimidated and bullied. I never thought a job at a university could come to this.
Thank you again for your support. It really does matter to the many of us who cannot really speak out openly at present.
Best regards,
|
In a later letter, the same person pointed out
"There are many of us who would like to speak more openly, but we simply cannot."
"I have mortgage . . . . Losing my job would probably mean losing my home too at this point."
"The plight of our female staff has not even been mentioned. We already had very few female staff. And with restructuring, female staff are more likely to be forced into teaching-only contracts or indeed fired"."
"total madness in the current climate – who would want to join us unless desperate for a job!"
“fuss about nothing” – absolutely not. It is potentially a perfect storm leading to teaching and research disaster for a university! Already the reputation of our university has been greatly damaged. And senior staff keep blaming and targeting the “messengers"."
6 July 2012.
Througn the miracle of WiFi, this is coming from Newton, MA. The Lancet today has another editorial on the Queen Mary scandal.
"As hopeful scientists prepare their applications to QMUL, they should be aware that, behind the glossy advertising, a sometimes harsh, at times repressive, and disturbingly unforgiving culture awaits them."
That sums it up nicely.
24 July 2012. I’m reminded by Nature writer, Richard van Noorden (@Richvn) that Nature itself has wriiten at least twice about the iniquity of judging people by impact factors. In 2005 Not-so-deep impact said
"Only 50 out of the roughly 1,800 citable items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations."
"None of this would really matter very much, were it not for the unhealthy reliance on impact factors by administrators and researchers’ employers worldwide to assess the scientific quality of nations and institutions, and often even to judge individuals."
And, more recently, in Assessing assessment” (2010).
29 July 2012. Jonathan L Rees. of the University of Edinburgh, ends his blog:
"I wonder what career advice I should offer to a young doctor circa 2012. Apart from not taking a job at Queen Mary of course. "
How to select candidates
I have, at various times, been asked how I would select candidates for a job, if not by counting papers and impact factors. This is a slightly modified version of a comment that I left on a blog, which describes roughly what I’d advocate
After a pilot study the entire Research Excellence Framework (which attempts to assess the quality of research in every UK university) made the following statement.
“No sub-panel will make any use of journal impact factors, rankings, lists or the perceived standing of publishers in assessing the quality of research outputs”
It seems that the REF is paying attention to the science not to bibliometricians.
It has been the practice at UCL to ask people to nominate their best papers (2 -4 papers depending on age). We then read the papers and asked candidates hard questions about them (not least about the methods section). It’s a method that I learned a long time ago from Stephen Heinemann, a senior scientist at the Salk Institute. It’s often been surprising to learn how little some candidates know about the contents of papers which they themselves select as their best. One aim of this is to find out how much the candidate understands the principles of what they are doing, as opposed to following a recipe.
Of course we also seek the opinions of people who know the work, and preferably know the person. Written references have suffered so much from ‘grade inflation’ that they are often worthless, but a talk on the telephone to someone that knows both the work, and the candidate, can be useful, That, however, is now banned by HR who seem to feel that any knowledge of the candidate’s ability would lead to bias.
It is not true that use of metrics is universal and thank heavens for that. There are alternatives and we use them.
Incidentally, the reason that I have described the Queen Mary procedures as insane, brainless and dimwitted is because their aim to increase their ratings is likely to be frustrated. No person in their right mind would want to work for a place that treats its employees like that, if they had any other option. And it is very odd that their attempt to improve their REF rating uses criteria that have been explicitly ruled out by the REF. You can’t get more brainless than that.
This discussion has been interesting to me, if only because it shows how little bibliometricians understand how to get good science.
76 Responses to Is Queen Mary University of London trying to commit scientific suicide?
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Sickening. I’m going to QM in a few weeks time for my PhD graduation. That’s the only consideration stopping me from sticking the boot in any further.
Mind you, the spangly new buildings that have sprung up in the most visible parts of the campus look nice. From the outside, at least…
This story is doubly depressing. It is not just the ill-judged threat to academic jobs that causes concern, it is the acceptance by QM of misguided managerial practices that have no place in academia. QM senior management are completely deluding themselves if they think that any successful academic researcher would willingly move to an institution that functions in this way.
@StephenEMoss
I agree. The result will be the opposite of what they hope. What’s so weird is that they can’t see this. You just can’t get the (senior) staff these days.
A good article. However, I take issue with the comment:
“If you are publishing a paper every six weeks, you certainly aren’t writing them, and possibly not even reading them. Most likely you are appending your name to somebody else’s work with little or no checking of the data. Such numbers can be reached only by unethical behaviour.”
If one has lots of ideas and good postdocs/students, it’s more than feasible to publish large numbers of papers per year, to all of which one has made a substantial contribution.
@bgaensler
In some sorts of physics it’s common for papers to have 10 or even 100 authors. Such papers, however good the content, are useless for assessing individuals, because there is no easy way to discover what the contribution of each author is. The practice seems to be to put everyone in the lab as an author, and I’ll bet some of them haven’t even read the paper on which their name appears.
Experience of interviewing job candidates suggests that’s true even in my area, where the number of authors is much more modest. When asked questions about their self-nominated 4 best papers, it is quite amazing how little some candidates know about what’s in them.
Ideas are not enough to make a paper (in the natural sciences). One needs data and analysis. Collecting data and checking analysis takes time. I don’t believe that it’s possible to check thoroughly what postdocs or PhD students have done in a few weeks.
In the case of recent paper of ours, it took four years to iron out technical snags, get the data and analyse it. Eventually it was published as a Nature article, but it could just as well have turned out the experiments failed or that the data were uninterpretable. In that case we’d have been left with little to publish after 4 years work. That’s the gamble -it’s why it’s called research. If the experiments don’t come out as one hoped, that is not grounds for firing a person.
It is certainly not uncommon in my area for a lab head to take a draft of paper written by a junior, make a few corrections to the English and add his/her name as “senior” author, with next to no checking of original data or calculations.
This is a form of fraud, but the really bad part of it is that universities reward that sort of unethical behaviour. In the case of QMUL, they make it compulsory.
Dear David,
It is courageous of you to speak out. I hope Queen Mary colleagues will take your contribution as an opportunity to engage in a dialogue that will help the College find its way in the upper echelons of the Russell Group Universities, by virtue and not by a cheap make up.
Achieving excellence in research and in teaching is no mean goal, and one needs to administer a complex institution in parallel. When things go wrong, repair mechanisms may be very difficult to implement.
Let us keep up the pressure to uphold core academic values; I cannot overemphasize that the claim in Queen Mary’s Strategic Plan that “a constantly improving process of knowledge creation and dissemination, together with the values and supporting activities that are critical to success… will require a high level of commitment and determination to make it a reality. We are lucky to have those qualities in abundance among our staff” is correct.
In the meantime, visitors who wish to read more can look here:
qmucu.wordpress.com
and sign our petition here
http://www.thepetitionsite.com/1/save-sbcs/
@Fanis
Thanks, but it doesn’t need any courage for me to speak out. The courage is needed by the people at QMUL who run a real danger of being fired if they point out the brainless procedures that are being used.
The people who are bringing QMUL into disrepute are not the scientists who speak out. It is the senior management, and hence Simon Gaskell, who are bringing QMUL into disrepute. If anyone is to be fired, it should be them.
[…] motivated to make a quick post in order to direct you to a blog post by David Colquhoun that describes the horrendous behaviour of the management at Queen Mary, […]
@David Colquhon
‘The practice seems to be to put everyone in the lab as an author, and I’ll bet some of them haven’t even read the paper on which their name appears’
You win your bet.
‘I Authored 700 Papers. Did I Read Them All ? No.’
http://www.science20.com/quantum_diaries_survivor/i_authored_700_papers_did_i_read_them_all_no-90480
@Acleron
Thanks for that reference. I’ve never seen anyone admit openly that they haven’t even read most of the papers on which their name appears. It’s worth a longer quotation.
It seems that physics is as corrupt as much of medical science when it comes to being honest about authorship. It most certainly makes a nonsense of QMUL’s criteria. Unless, of course, QMUL wishes to encourage corrupt practice in order to increase their income.
I know you won’t believe it -I can’t- but I discovered that I ended up as a second author on a paper, after it was accepted for publication. That without being able to read the first draft or the referee comments, let alone doing even the smallest experiment. The sad thing is that I will never be credible on the “no figure, no authorship” principle.
Well, Acleron, you have succeeded. I am shocked. And to say “I did not ask to sign those papers – it is automatic” seems rather weak-kneed. Nobody can force you to put your name on a paper. Most journals have criteria for authorship, so anyone who puts you as author is having to lie. Here’s the guidelines of the International Committee of Medical Journal Editors: “Authorship credit should be based on 1) substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published. Authors should meet conditions 1, 2, and 3.” http://www.icmje.org/ethical_1author.html
Your defence souds rather like the bankers: “everyone does it”. Well some of us don’t, not least because it dilutes the value that genuine authors have contributed. Another reason for keeping to the rules is that if you aren’t careful, you may find your name associated with papers that are bad or dishonest.
One way of stopping this sort of thing would be if the metric-merchants started dividing a paper by the number of authors when computing things like H index. I don’t like metrics any more than DC, but I suspect such a move might achieve the result that hordes of authors would melt away.
[…] Wissenschaftliche Disqualifikation durch Quantifikation: “Is Queen Mary University of London trying to commit scientific suicide?” […]
[…] sacking a large number of academic staff. There is much more detail in David Colquhoun’s post, but essentially the management has defined targets for number of publications, number of […]
As a particle physicist, I feel like I should defend the authorship practices in the field. The results in a particle physics paper really do depend on the work of a large group of scientists: those who designed the detectors, constructed the detectors, calibrated the outputs, designed and ran the simulations used to derive the results, and so on. Giving appropriate credit to all of those people is not an easy problem to solve, and so the approach in particle physics is to maintain a list of everyone who’s contributed in some way, and put them all on the author list, usually alphabetically.
It’s maybe not the best possible system, but I don’t think it’s motivated by corruption, dishonesty, or an attempt to game any particular metrics; rather, by a desire to include all those who have made a contribution, without large amounts of drama and wrangling about the relative contributions of hundreds (or even thousands, in the case of LHC experiments) of scientists.
As to the “everyone does it” argument, I think there is some validity in it as far as it is the standard in field, so other particle physicists with whom one might be competing for jobs, etc, will have comparable publication lists – the practice doesn’t give anyone an unfair advantage (over another person in the same field, at least)
@phil
I can see that “the person who designed the detector” deserves some credit. But surely you must agree that counting papers which an ‘author’ hasn’t even read is not a good way to assess anyone’s ability. That’s what they are doing at QMUL.
In some journals in my field, it has become common for authors to have to state what contribution they made to a paper. But with physics papers that have hundreds or even thousands of authors, that is barely feasible (and, in any case, it invites exaggeration).
Since it appears that QMUL counts as a publication only first author, last author and corresponding author, let’s hope they don’t start applying their ill-informed methods to your field.
I had an email that objected to my claim that
The author’s web site claims that he has 1053 publications. Over 20 years that’s about one per week.
He claims
Further enquiries revealed that
So that reduces the total to 180-odd research papers as opposed to reviews and commentaries.
Oddly, Pubmed reveals only 46 publications, but it’s possible that papers on gambling studies are not all listed in Pubmed.
David – it seems the comment thread has moved slightly away from the main topic of the original blog. It’s not that the issue of authorship is not important, but for the managers who fail to understand how research and research publication work, I have an alternative to the D3 formula, namely the (IF)3 rule.
IF = Impact Factor = Intrinsically Flawed = Institutional Failings.
@David:
Yes, I absolutely agree that solely counting papers is a poor way of assessing academic ability/achievement; as you say, blanket metrics like the QMUL ones work just as badly for particle physics as they do in other fields of science.
David
Perhaps we should encourage the Russell Group to come up with an acceptable way to measure academic performance and ensure that Universities within the Russell Group then use it rather than making it up as they go along.
Like Phil, I’m a particle physicist and feel as if I should defend our authorship policies. Nobody on a large collaboration would simply put forward a publication list of x hundred publications in an application and expect to be taken seriously. The convention is to give the complete list and then make a smaller list of papers to which the applicant has made a substantial contribution. A description of the contributions must also be given.
Regarding the main topic of your post, I don’t agree that QMUL will deter high quality applicants. There are, unfortunately, very few permanent jobs around and they will get in researchers who are able to get them the publications and citations they so crave. Unfortunately, they’ll do so at the expense of academic freedom and long term innovation.
Its a hard argument to make but failure in research i.e. going down a blind alley is essential for progress. A university should be proud that some research lines stop because the ideas failed. Finding out what doesn’t work is an indispensable step towards finding what does work. QMUL seems to want to leave the speculative risky research, which usually ends in failure but sometimes ends in paradigm-shifting progress, to others. The problem is that the others are increasingly adopting the same attitude.
If I’d known that academic freedom was going to be eroded to this extent, I’d have made a career in another area. For me, the freedom to explore what I think is important is the key attribute of my job.
@robbo
I doubt very much whether any automated method will ever be satisfactory.
We already had the best method, and that is to ask people to nominate their best papers. Then read the papers and guess, as best they can, whether the author is likely to make progress in the future. Of course many of the guesses will be wrong. That’s why it’s called research. You don’t know what will work out and what won’t.
Nominating your best papers will encourage fewer, more complete, papers. That would help to alleviate the deluge of trivia and salami-sliced papers. In contrast QMUL’s policy encourages it.
@physo
I agree with everything you say, provided that interviews are used to check that the claimed contribution has been stated accurately (in my experience that is not always the case).
Of course QMUL will get applications, but they won’t get the best people if they treat them as they seem to have been doing.
They’ll encourage what I called a “generation of second-rate spiv scientists”. That’s much the same as what you said, a bit more politely “they will get in researchers who are able to get them the publications and citations they so crave. Unfortunately, they’ll do so at the expense of academic freedom and long term innovation”.
David C.
I agree that QMUL won’t get the best but I think their recruited staff will be good enough such that QMUL move up the league table and these policies will then be deemed a success.
Of course I could be wrong. In this case, it would be nice if QMUL could outline the criteria against which senior managers would be expected to lose their jobs…
@physo
Yes they might indeed push up their REF score in the short run, though REF claims that it looks at your four best papers, and that means it’s a disadvantage to have a large number of short papers. Even by REF standards, QMUL’s counting is daft.
The problem is likely to arise over a longer time scale. When the morale of a place falls, and people feel bullied, they will look around for somewhere more pleasant to work, and of course it’s the best people who will find alternative homes, and leave. This takes time and is likely to lead to slow loss of good people over a 5 – 10 year time scale.
The fact that REF focusses attention on the next two years is one of the reasons why its effect on universities is to do harm.
David
A non-automated system is the best way, as you suggest but that needs the involvement of thought, careful condideration and the acceptance that you cannot predict the future, as you point out. Not things that go down well in academic management in my experince. The reality is that there is going to be a daft measuring system so let it be one we (the people being meaured) think is the best we can come up with.
@robbo
Well, I don’t know what goes on at your place, but at mine we have always thought, when making appointments, that it’s important enough to expend “thought, careful consideration”, and I guess everyone knows that, however hard you try you have to accept “that you cannot predict the future”.
I don’t believe for a moment that we have to accept that it’s inevitable that there will be “a daft measuring system”. That will happen only if we keep quiet when they are proposed.
Metrics are only used so that management can point and say look, we have achieved something. As such they are only another form of management-speak, not designed to be understandable or useful but as a code of recognition for each other. It thus gives them that fuzzy and warm feeling of being ‘in’. It serves a double purpose as they are only applied to ‘them’, so usefully excluding the very people who are doing the work from the decision making process.
It is a shame it has reached the halls of academia, it is a technique long used in other organisations, and I wish Dr Colquhoun the very best in combating it.
David
Come on, as a man who has spent his life measuring the activity of single ion channels in the sub ms time domain, pA current range and developing mathematical models to explain their behaviour, you with others who read this blog could come up with a reasonable (not perfect) way of measuring academic performance. I like most of your readers, I suspect, would prefer you (or someone like you) to do it rather someone in HR!
@Robbo
It’s kind of you to say so, but part of the art of science is to know when to say “I don’t know” or “I can’t do that”. I certainly can’t imagine how an automated system could be devised.
My system is to ask candidates to nominate their four best papers (fewer if they are young), read the papers, and ask them hard questions (specially about the methods section). I suspect that this is the best that can be done, but it isn’t infallible by a long chalk.
David C
Academic mobility is limited for a variety of reasons. I don’t foresee the attenuation in staff that you do especially if other universities follow the same path.
Unfortunately, I think QMUL’s policy will get them what they want, even if its not desirable for them (or the wider sector).
@physo
It will certainly be interesting to see whether they achieve their short term aim (assuming that they won’t have the sense to change the policy). It will be even more interesting to see what long term damage the present policy has on QMUL’s fortunes/
I’m afraid that physo has a point, in that I predict versions of this will be widespread across the Russell Group ahead of the REF.
My feeling is that all the research-intensive Univs will be trying feverishly to reduce the number of people on their books who are ‘eligible’ for REF return (i.e. have a ‘research & teaching’ contract) but who they think are not “REF returnable”. The likely means will be reclassification to some sort of ‘teaching & scholarship’ role, as described by the two academics who wrote to the Lancet.
Whether Univs actually try straight redundancy will likely depend on (i) how many early retirement and voluntary severance campaigns a given Univ has already had; and (ii) how many people they can find who can be ‘persuaded’ to accept the move to a ‘teaching + scholarship’ / ‘teaching -focussed’ track.
[…] was reminded of this when I read David Colquhoun’s post about new criteria for academic staff at Queen Mary University of London, where all staff will need […]
Your comment is very timely and important in times of squeezing out senseless publications of moderately interesting data. QMUL seems to exagerate however most Medical Universities in Germany work already in the same way. A department has to achieve a high account of IF, elsewise the head of the departments receives no bonus at the end of the year. Everyone can imagine what this means in terms of publication pressure, quality of the publication and prolonging work contracts. This system is highly sensitive to misconduct of data. Our medical University had just two months ago a huge scandal because the professor for bariatric surgery had to retract six publications because of faked data (http://retractionwatch.wordpress.com/?s=edward+shang). The damage is high for the reputation of the university, the work groups around him and the surgeon himself, who lost his job immediately. Sadly, he was an excellent surgeon, but obviously a scientist driven by IF and publications.
Grading the worth of a scientist is an impossible task; it is so skewed towards particular fields. For example, in my field, paleontology, a couple of papers a year is considered fairly prodigious, and single author papers are common. Also, a large body of work on a group might be condensed into a single, quality, whopper of a paper.
However, Tommaso Dorigo, a middle career particle physicist, has said that he has authored over 700 papers, of which he hasn’t even read the vast majority of; perhaps a third at most (http://www.science20.com/quantum_diaries_survivor/i_authored_700_papers_did_i_read_them_all_no-90480). Even H-indexes and other indices are dramatically skewed towards fields characterized by vast collaborations and publishing diarrhoea. I don’t think there’s any way of separating it.
[…] happens around citation: people cite themselves, or friends, to bump up their H-index, and other deceptively numerical bibliometric tools for divining the quality of an […]
Quite apart from the substance of the letter to Professor Allen, what amazes me, as an arts graduate, is that no one seems to have proofed it.
For example in the very first sentence it says “the college had decided” – presumably this should read “has decided”. In the third paragraph “investigating officer” first has initial capitals and then does not. The paragraphs beginning “This investigation” and “As it is alleged” repeat the same information. The next paragraph has another typo “I will provided”. In the paragraph beginning “You are entitled”, the brackets are unnecessary and potentially ambiguous. The first sentence of the paragraph after that is also badly drafted: it is perfectly possible for Professor Allen to make himself available for both dates and not to attend the meetings. In the paragraph about confidentiality, the words beginning “with the exception of” if they mean anything, suggest that normal practice does not apply to the trade union or other representatives.
Nor is the letter even laid out correctly: on the first page the first lines of the paragraphs beginning “An Investigating Officer” and “As it is alleged” are indented, while all the other paragraphs are not; there is a space missing after “(Appendix 1.3)” (and one would normally expect a paragraph that is a quotation from a document to be indented); and there is a comma missing after “Yours sincerely” and a space missing after “HR Consultant” which would usually be followed by an en-rule rather than a hyphen.
It is bad enough to threaten someone with dismissal for expressing an academic opinion, but to do so in such a slip-shod way must be inexcusable.
[…] Is Queen Mary University of London trying to commit scientific suicide? by David Colquhoun is a defence of core academic values against the absurdity of crude metric applications, which lead to bad science. […]
[…] face it, you would have to be pretty seriously committed to it to put up with stuff like this. Or this. Or […]
I am still trying to work my mind against a number of contradictions (how many can bloggers spot on the small texts provided below?).
The minutes of the faculty panel that evaluated whether I met the above mentioned metrics include:
“However, he contested the definition of a quality paper. Dr Missirlis believes all of his 6 papers to be of high quality… Dr Missirlis also argued that he cannot be assessed by a non-expert. The panel concluded that this was not relevant… Dr Missirlis remains at risk”
The letter I received from the redundancy committee includes:
“…the redundancy committee delegated the responsibility to the faculty panel to evaluate staff against the criteria. The Faculty Panel was responsible for consistency and in the case of SBCS this exercise was done anonymously… You feel that the College is proposing to breach your contract and that you have complied with the main responsibilities of your role. The redundancy procedure is designed to bring contracts of employment to an end fairly and in accordance with procedure, when the role an individual performs is no longer needed, and without any reference to the performance of the individual concerned.”
I could stop here, but take also this one, also from the Letter I received from the redundancy committee (this time it should concern our students – how many of them reached to read this post? – let me know if you are a student at QMUL and you have done so):
“You asked the redundancy committee to consider whether Cell Dynamics/The Human Cell would be taught next year. Although we understand this teaching will be required it was only a part of your role and can be readily fulfilled by other means.”
I hope this was not an abuse of this space, which has raised many broader issues. Sometimes the devil is in the detail. How would all of the be viewed in a Court of Law? I hope my appeal will be successful, so we don’t need to find out.
Fanis
I’m so sorry this is happening to you. As an academic outside of the UK, I’ve done what I can (not zero, but not very much) to help. I just hope the UK community can apply the necessary pressure.
As I mentioned earlier, if I’d known how things would turn out (and I foresee many uni’s following QMUL’s lead) I’d have made a career in a different area.
Best of luck.
On a semi-related note, I received my instructions for what to do on the day of my graduation ceremony.
http://www.qmul.ac.uk/graduation/day/index.html
I thought that the third paragraph under “What to wear” was rather curious:
[…] offer. Perhaps not a sensible thing to in the light of the current entertainment of REF (http://www.dcscience.net/?p=5388), but I feel the right thing to […]
[…] level we just can’t afford that luxury. And if you happen to work at Queen Mary in London you may be screwed on that front even if you have a permanent […]
Thanks for the encouraging comments. Appreciated. Many were surprised when they saw that Queen Mary is advertising 30 positions in the very department under discussion…
http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=420637&c=1
Funny how that happens. In one Biosci Faculty near me a full-press ERVS scheme (‘Very Last chance! No More Mr Nice Guy after this!), complete with lots of “We’ve not met the target yet!”, was followed by a big advertising campaign for new ‘world class’ (inevitably) people.
Still, no hint of compulsories yet anywhere in my Univ, though the pre-REF rumour mill is (also inevitably) churning away merrily.
[…] brings me nicely to a couple of posts that shine light in this area. The first, via David Colquhoun (DC), is on the recent decisions by Queen Mary to get rid of staff. The Lancet have been running […]
[…] sc_project=233721; sc_invisible=0; sc_partition=0; sc_security=""; ← Is Queen Mary University of London trying to commit scientific suicide? […]
This is a particularly gross example of a tendency/trajectory that just seems to be ripping through just about any particular university you look at. What is happening at QMUL is crazy. In my own field, philosophy, one of the most important thinkers of the last 400 years, Immanuel Kant spent a decade where he wrote almost nothing; it’s dubbed ‘the silent decade’ because basically we hear nothing from him. In that decade ideas were brewing and Kant was agonising over them as he saw them as vitally important to human life, both intellectual and otherwise. However one might want to evaluate it all today what flowed out from it had massive impact on the philosophical world an impact that flowed out into many other disciplines. I can say without any fear of controversy that he is on of the most significant figures in the history of my field. Needless to say, the significance of his work was not immediately felt. He would have been sacked from QMUL. I am sure every field of enquiry has more than their share of similar examples.
I suspect that if Andrew Wiles had been at a UK university, rather than at Princeton, he would have had rather less time to withdraw to his attic and beaver away on Fermat’s last theorem until he cracked it.
[…] already written about it twice in Is Queen Mary University of London trying to commit scientific suicide?, and in Queen Mary, University of London in The Times. Does Simon Gaskell care?. But wait, there is […]
Those who wish to have more details about the restructuring process at QMUL can visit
http://qmucu.wordpress.com/
Be advised this is the UCU (lecturers’ union) website. There is no equivalent public place where the management’s full case is put.
In some respects the two sides are not so far apart, in that both agree that improvements are needed in some Schools, and that it is valid to identify individuals who could do better. The argument is about how the identification is done, and then about what to do next. There is no complacency on either side.
Management has used controversial, retrospective metrics, conflated into tortuous algorithms, to identify underperformers, whereas the union favours the existing negociated appraisal system. To deal with these individuals, management proposes redundancy or mandatory demotion to teaching-only status; the union proposes mentoring accompanied by relatively small investments of extra resources and studentships, and a reduction of teaching load. Currently, the management rejects the union plan as “a recipe for mediocrity”, and is moving to the next stage of its own agenda, which is to hire in new staff with “REF-returnable” credentials. Such a process is expensive and a lot of start-up money has been earmarked for newcomers as an incentive for them to leave their existing institutions. However, there is no announced budget, and it is not known how the costs of the management prosals and UCU’s plan would compare.
In all cases where restructuring has been initiated, the Schools concerned are expected to take more students at both UG and PG level. An argument made by UCU that excessive UG recruitment was in some cases the cause of underperformance was dismissed as “an historical complaint”. To support the case for more undergraduates, the College produced new estimates of student:staff ratios, revised downwards and therefore just falling within the range of SSRs in existing equivalent Russell Group departments. The new data were then alleged to demonstrate that the problem of underperformance in research was due to weak individual staff rather than too much teaching or a lack of internal investment.
* “An argument made by UCU that excessive UG recruitment was in some cases the cause of underperformance was dismissed as “an historical complaint”.
No, SBCS has pulled out all the stops again this year in clearing, and “we have recruited our target number of students to start in September” (message to all staff, 17 August). SBCS seems set to continue as a teaching department, with many academic staff now contractually forbidden from doing research at all. The declared aim of the restructuring was to improve research. It has had precisely the opposite effect. REF 2014 will inevitably be another indictment. Unless there is a total change of strategy, and the restructuring halted, there now seems little chance of SBCS becoming a research-led Department by REF 2020 (if there is one).
[…] move has brought essentially universal condemnation. In particular, see David Colquhoun’s blogpost “Is Queen Mary University of London trying to commit scientific suicide?”. The reliance on […]
[…] [not entirely sensible] actions performed at Queen Mary’s Physics/Astronomy department (more information). Essentially, the issue breaks down as […]
[…] Is Queen Mary University of London trying to commit scientific suicide? I meant to highlight this back in June when it was first posted, apologies if this is just old news for some of you. […]
[…] It’s weird that medicine, the most caring profession, is more corrupt than any other branch of science. The reason, needless to say, is money. Well, money and vanity. The publish or perish mentality of senior academics encourages dishonesty. It is a threat to honest science. […]
[…] research career levels; • Universities treat successful grant applications as outputs, upon which continued careers depend; • Statistical analyses are hard, and sometimes researchers get it wrong; • Journals favour […]
[…] levels; • Universities treat successful grant applications as outputs, upon whichcontinued careers depend; • Statistical analyses are hard, and sometimes researchers get it wrong; • Journals favour […]
[…] levels; • Universities treat successful grant applications as outputs, upon which continued careers depend; • Statistical analyses are hard, and sometimes researchers get it wrong; • Journals favour […]
[…] career levels;• Universities treat successful grant applications as outputs, upon which continued careers depend;• Statistical analyses are hard, and sometimes researchers get it wrong;• Journals favour […]
[…] research career levels; • Universities treat successful grant applications as outputs, upon which continued careers depend; • Statistical analyses are hard, and sometimes researchers get it wrong; • Journals favour […]
[…] But it’s not taken off yet. A common argument is that scientists are under pressure and the system encourages fraud (see letter, section entitled 3rd July). To me, people with no principles are undermining our […]
[…] better than me, publications in NPG journals can make or break the career of a young researcher – or, indeed, more established scientists – because an inordinate amount of prestige and importance (and, via the UK REF, financial […]
[…] 1) The pressure to publish in “Top Journals” means it’s worth massaging the data (or just making it up). That this obsession with counting (and with Impact Factors) will lead to fraud has been well enough discussed elsewhere (see letter, 3rd July). […]
[…] the reason that I have described the Queen Mary procedures as insane, brainless and dimwitted is because their aim to increase their ratings is likely to be […]
[…] which journals they published in, or how much grant income they generated. But it’s the metrics, rather than the advancement of knowledge, that scientists are increasingly incentivized to focus […]
[…] grants, research time) for the good of the cause, keeping in mind that doing so might also mean losing your job. […]
[…] grants, research time) for the good of the cause, keeping in mind that doing so might also mean losing your job. […]
[…] grants, research time) for the good of the cause, keeping in mind that doing so might also mean losing your job. […]
[…] We have to; it’s part, indeed, the main part of our assessment (http://www.dcscience.net/?p=5388). As I have said before, this is now the only reason I publish in this way […]
[…] not publishing well, and bringing in grant funding, you don’t have tenure to fall back on and could find yourself out of work after a seemingly good […]
[…] high level esteem markers. Some Institutions have come under scrutiny and (from some quarters) significant criticism for their managerial tactics. Some suggestions of bullying tactics have been […]
[…] departments are quite happy about how they are run. Kings College London, Warwick University and Queen Mary College London have been just as brutal as Imperial. But in these places nobody has died. Not […]
[…] are even setting research grant targets. Don’t achieve the target, don’t get promoted…or worse. Conversely, achieve the target, you get promoted. But here’s the thing. I’m not […]