Ranking strategy
Where should effort go under a rebuilt methodology, and which sustainability goals should we submit?
Sixteen candidate goals reduced to three. It led the category on debut.
The position
As-Is, and To-Be.
- As-Is
- A rebuilt ranking methodology had reset where effort earned position. Sixteen candidate goals were in scope for a category the university had not yet entered, with no basis to choose among them.
- To-Be
- Effort concentrated on the heaviest-weighted pillars; entry limited to goals the institution's own publication record could defend.
A publicly funded research university in Asia
The analysis
What we did.
Both major ranking systems had rebuilt their methodologies within a year of each other. The arithmetic of where marginal effort earned marginal position had to be redone from scratch, and the university was moving quickly on several fronts at once. The question was never whether to compete on rankings, but where effort would actually be repaid.
We took both methodologies apart weight by weight, from teaching reputation down to patents. Citation-weighted measures carry the greatest combined weight across both schemes, which made citation impact the single highest-leverage variable. That claim was then tested against the institution's own publication record rather than asserted: publication trends, journal profile, collaboration patterns, and citation impact charted as average against median, deliberately, to expose the skew that an average alone conceals.
The centrepiece was goal selection. The sustainability rankings ask an institution to choose which of the UN Sustainable Development Goals it will be assessed against, and the choice is consequential: a goal chosen badly costs a cycle. Partnerships for the Goals is assessed for every participating institution, which leaves sixteen in the choice set. We ranked all sixteen on scholarly output, field-weighted citation impact and citation volume, then eliminated them against average and median thresholds. Three survived every cut.
The shortlist is where the data stopped and judgement began. Research output accounts for only about a quarter of the sustainability computation, which argues directly against selecting on bibliometrics alone. The final choice had to read the institution's own direction of travel as carefully as the scoreboard: an organisation should compete where its identity and its evidence converge, not merely where its citation counts are highest.
Running alongside the analysis was a strand about thinking itself. Correlation is not causation, more data is not better data, and one should not spend two million dollars on a one million dollar problem. It included an error of our own: we had previously dismissed a health-related goal on the assumption it required a medical school, until benchmarking showed competitors without one had selected it anyway. Check the field before trusting the desk.
Candidate goals, eliminated against average and median thresholds. Order is illustrative; the goals themselves are not disclosed.
The outcome
What happened.
Three goals survived every cut, two years before the institution entered the category. It entered on one of the three, and led the category on debut.
The claim here is deliberately narrow. The rise belongs to the university and the working group that carried it through. What the engagement contributed was the analysis that reduced sixteen options to three, and the framework to make that kind of choice again without us.
Start a conversation about your own benchmark.
Start a conversation