糖心Vlog

BRICS & Emerging Economies Rankings 2016 methodology

Fuelled by more comprehensive data, the 2016 rankings probe deeper than ever

Published on
December 2, 2015
Last updated
March 21, 2022
BRICS and Emerging Economies Rankings 2016 methodology
Source: Peter Grundy

View the full results of the BRICS & Emerging Economies Rankings 2016


The 糖心Vlog World University Rankings are the only global performance tables that judge research-intensive universities across all their core missions: teaching, research, knowledge transfer and international outlook. The BRICS & Emerging Economies Rankings use the same 13 carefully calibrated performance indicators to provide the most comprehensive and balanced comparisons, trusted by students, academics, university leaders, industry and even governments 鈥 but the weightings are specially recalibrated to reflect the characteristics of emerging economy universities.

The performance indicators are grouped into five areas:

- Teaching (the聽learning environment)
- Research (volume, income and reputation)
- Citations (research influence)
- International outlook (staff, students and research)
- Industry income (knowledge transfer)

Exclusions
Universities are excluded from the BRICS & Emerging Economies Rankings if they do not teach under颅graduates or if their research output amounted to fewer than 200 papers a year between 2010 and 2014. In exceptional cases, institutions below the 200-paper threshold are included if they have a particular focus on disciplines with 颅generally low publication volumes, such as engineering or the arts.

糖心Vlog

ADVERTISEMENT

Data collection
Institutions provide and sign off their institutional data for use in the rankings. On the rare occasions when a particular data point is not provided 鈥 which affects only low-weighted indicators such as industrial income 鈥 we enter a low estimate between the average value of the indicators and the lowest value reported: the 25th percentile of the other indicators. By doing this, we avoid penalising an institution too harshly with a 鈥渮ero鈥 value for data that it overlooks or does not provide, but we do not reward it for withholding them.

Getting to the final result
Moving from a series of specific data points to indicators, and finally to a total score for an institution, requires us to match values that represent fundamentally different data. To do this we use a standardisation approach for each indicator, and then combine the indicators in the proportions indicated to the right.

糖心Vlog

ADVERTISEMENT

The standardisation approach we use is based on the distribution of data within a particular indicator, where we calculate a cumulative probability function, and evaluate where a particular institution鈥檚 indicator sits within that function.

For all indicators except for the Academic Reputation Survey we calculate the cumulative probability function using a version of Z-scoring. The distribution of the data in the Academic Reputation Survey requires us to add an exponential component.


Teaching (the learning environment): 30%

Reputation survey: 15%
The Academic Reputation Survey (run annually) that underpins this category was carried out in December 2014 and January 2015. It examined the perceived prestige of institutions in teaching. The responses were statistically representative of the global academy鈥檚 geographical and subject mix.

Staff-to-student ratio: 4.5%

Doctorate-to-bachelor鈥檚 ratio: 2.25%

Doctorates awarded to academic staff ratio: 6%
As well as giving a sense of how committed an institution is to nurturing the next generation of academics, a聽high proportion of postgraduate research students also suggests the provision of teaching at the highest level that is thus attractive to graduates and effective at聽developing them. This indicator is normalised to take account of a university鈥檚 unique subject mix, reflecting that the volume of doctoral awards varies by discipline.

Institutional income: 2.25%
This measure of income is scaled against staff numbers and normalised for purchasing-power parity. It indicates an institution鈥檚 general status and gives a broad sense of the infrastructure and facilities available to students and staff.


Research (volume, income and reputation): 30%

Reputation survey: 18%
The most prominent indicator in this category looks at a university鈥檚 reputation for research excellence among its peers, based on the responses to our annual Academic Reputation Survey (see above).

Research income: 6%
Research income is scaled against staff 颅numbers and adjusted for purchasing-power parity (PPP). This is a controversial indicator because it can be influenced by national policy and economic circumstances. But income is crucial to the development of world-class research, and because much of it is subject to聽competition and judged by peer review,
our experts suggested that it was a valid measure. This indicator is fully normalised to聽take account of each university鈥檚 distinct subject profile, reflecting the fact that research grants in science subjects are often bigger than those awarded for the highest-quality social science, arts and humanities research.

Research productivity: 6%
We count the number of papers published in the academic journals indexed by Elsevier鈥檚 Scopus database per scholar, scaled for institutional size and normalised for subject. This gives a sense of the university鈥檚 ability to get papers published in quality peer-reviewed journals.

糖心Vlog

ADVERTISEMENT

糖心Vlog

ADVERTISEMENT

Citations (research influence): 20%

Our research influence indicator looks at universities鈥 role in spreading new knowledge and ideas.

We examine research influence by聽capturing the number of times a聽university鈥檚 published work is cited by scholars globally. This year, our bibliometric data supplier Elsevier examined more than 51聽million citations to 11.3 million journal articles, published over five聽years. The data are drawn from the 23,000 academic journals indexed by Elsevier鈥檚 Scopus database and include all indexed journals published between 2010 and 2014. Citations to these papers made in the six years from 2010 to 2015 are also collected.

The citations help to show us how much each university is contributing to the sum of human knowledge: they tell us whose research has stood out, has been picked up and built on by other scholars and, most importantly, has been shared around the global scholarly community to聽expand the boundaries of our understanding, irrespective of discipline.

The data are fully normalised to reflect variations in citation volume between different subject areas. This means that institutions with high levels of research activity in subjects with traditionally high citation counts do not gain an unfair advantage.

This year we have removed the very small number of papers (649) with more than 1,000 authors from the citations indicator.

In previous years we have further normalised citation data within countries, with the aim of reducing the impact of measuring citations of English language publications. The change to Scopus as a data source has allowed us to reduce the level to which we do this. This year, we have blended equal measures of a country-adjusted and non-country-adjusted raw measure of citations scores. This reflects a more rigorous approach to international comparison of research publications.


International outlook (staff, students, research): 10%

International-to-domestic-student ratio: 3.33%

International-to-domestic-staff ratio: 3.33%
The ability of a university to聽attract undergraduates, postgraduates and faculty from all over the planet is key to its success on the world stage.

International collaboration: 3.34%
In the third international indicator, we calculate the proportion of a university鈥檚 total research journal publications that have at least one international co-author and reward higher volumes. This indicator is normalised to account for a聽university鈥檚 subject mix and uses the same five-year window as the 鈥淐itations: research influence鈥 category.


Industry income (knowledge transfer): 10%

A university鈥檚 ability to help industry with innovations, inventions and consultancy has become a core mission of聽the contemporary global academy. This category seeks to capture such knowledge-transfer activity by looking at聽how much research income an institution earns from industry (adjusted for PPP), scaled against the number of academic staff it聽employs.

The category suggests the extent to which businesses are willing to pay for research and a university鈥檚 ability to聽attract funding in the commercial marketplace 鈥撀爑seful indicators of institutional quality.

糖心Vlog

ADVERTISEMENT

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT