Impact of African Journals in ISI Databases

----------------------------------------------------------

The STIMULATE 4 Group

Vrije Universiteit Brussel
Campus Etterbeek, Pleinlaan 2
1050 Brussel, Belgium *

Abstract

The calculation of percentile impact factors and their use are illustrated for all ISI-covered journals published in Africa or carrying the words Africa or African in their title. For each African journal we selected a Western journal (defined as a journal published in North America or Western Europe) belonging to the same journal category in ISI’s Journal Citation Reports^® and having a similar ISI impact factor.

For the groups of journals studied here, we did not find a significant difference between any of the studied impact factors for African journals and for matched Western ones. Surprisingly, for these journals we did not even find a statistically significant difference between the average ISI impact factor, the first quartile impact factor, and the median impact factor. These results indicate that for journals with relatively low impact factors there is little difference between the various ways in which synchronous impact factors are calculated.

* All correspondence should be sent to:

Ronald Rousseau

KHBO, Zeedijk 101, 8400 Oostende, Belgium

E-mail: ronald.rousseau@khbo.be

Introduction

Journal impact factors continue to attract a lot of attention. Indeed, for better or for worse, no journal editor or publisher can afford to ignore them. Journal editors and publishers are not the only ones interested in impact factors. Librarians may use the ISI impact factor as one element in selection and de-selection procedures; scientists may be interested in journals with high impact factors in order to reach the highest possible visibility for their published results; funding agencies may consider the impact factors of the journals in which their grantees publish funded research; and university research councils may use journal impact factors as indices in local evaluation studies (Rousseau, 2002).

The Garfield or ISI impact factor of journal J in the year Y, denoted as IF_J(Y), is defined as (Garfield & Sher, 1963)

Here, PUB_J(X) denotes the number of articles (more correctly, citable items) published in journal J during the year X; CIT_J(Y, X) denotes the number of citations received by journal J in the year with Y referring to items published in the year X. Citations used for the determination of the Garfield or ISI impact factor are always collected from the so-called ISI source journals, a select group of highly visible journals.

Although this impact factor is the best known and the most used, it suffers from a number of drawbacks. This is not surprising as one single number cannot possibly describe all aspects related to the visibility, let alone the quality, of a scientific journal. Consequently, many other proposals have been made in the informetric literature. One of the simplest ones is that of extending the citation window from two years to three, four, or any other number of years. Another proposal consists of using a diachronous approach instead of the synchronous way in which the ISI impact factor is calculated. Recall that the term synchronous means that all citations are collected in the same year, i.e., Y. A diachronous approach, keeping the publication year fixed and collecting citations in subsequent years, is much better suited for research evaluation purposes (Moed et al., 1985). For a detailed description of the difference between synchronous and diachronous impact factors and their use in research evaluation, the reader is referred to Ingwersen et al., 2001. Other proposals change the citation pool either by restricting journals to a specific domain (Hirst, 1978) or by considering a totally different pool as in the case of the Chinese citation indices (Jin & Wang, 1999; Wu et al., 2004). A recent new proposal, the so-called percentile and median impact factor, takes the form of the citation curve into account and does not use a fixed number of years. Details for its calculation are given in Sombatsompop et al., 2004 and Rousseau, 2005 and are described briefly later in this article. Studying the pros and cons of impact-related indicators is certainly a scientifically and practically useful activity.

Purpose and research questions

In this article we illustrate the calculation of percentile impact factors and their use. This is accomplished by studying ISI-covered journals published in Africa or carrying the words Africa or African in their title. This group of journals is referred to as African journals. As this study was performed by a group of international students who, for the most part, were living in Africa, choosing African journals increased the involvement of the participants. Moreover, as African journals are rarely studied, this introduced another element of value to the article. For each African journal we selected a Western journal (defined as a journal published in North America or Western Europe) that belongs to the same JCR (Journal Citation Reports^®, ISI) journal category and has a similar ISI impact factor, i.e., with the smallest possible difference. We then attempted to determine if using percentile impact factors makes a difference in the comparison of African journals and their matched Western ones. More precisely, we calculated the first quartile impact factor and the median impact factor in order to compare the African and the matching Western journals. Calculations for the median cited age were performed only when the age was less than ten. Indeed, the JCR^® provides cited data for only a 10-year period; and although it is possible to estimate the median cited age when it is more than ten (Rousseau, 2005), we stuck to only the known (not estimated) data.

Data collection

From the JCR^® sciences and social sciences editions covering the year 2003, we collected all journals that are either published in Africa or carry the words Africa or African in their title. Later we removed from this list those journals for which it was impossible to find a sufficient number of citation years (this happened when the first quartile cited age was more than ten) or for which it was impossible to collect the number of publications during the period corresponding to the first quartile cited age (because the journal has recently changed its name or was only recently covered by ISI). This resulted in the following table of 28 journals (Table 1).

Table 1. African journals (abbreviated as in the JCR ^® ) and country of publication

Journal Name	File	Country of Publication
AFR ENTOMOL	Science	South Africa
B CHEM SOC ETHIOPIA	Science	Ethiopia
BOTHALIA	Science	South Africa
DISCOV INNOVAT	Science	Kenya
J S AFR I MIN METALL	Science	South Africa
J S AFR VET ASSOC	Science	South Africa
ONDERSTEPOORT J VET	Science	South Africa
OSTRICH	Science	South Africa
S AFR J ANIM SCI	Science	South Africa
S AFR J BOT	Science	South Africa
S AFR J CHEM-S-AFR T	Science	South Africa
S AFR J GEOL	Science	South Africa
S AFR J SCI	Science	South Africa
S AFR J SURG	Science	South Africa
S AFR J WILDL RES	Science	South Africa
SAMJ S AFR MED J	Science	South Africa
WATER SA	Science	South Africa
AFR J ECOL	Science	England
J AFR EARTH SCI	Science	England
S AFR J ECON	Soc. Sciences	South Africa
S AFR J PSYCHOL	Soc. Sciences	South Africa
AFR AFFAIRS	Soc. Sciences	England
AFR TODAY	Soc. Sciences	USA
AFRICA	Soc. Sciences	Scotland
J AFR ECON	Soc. Sciences	England
J AFR HIST	Soc. Sciences	England
J MOD AFR STUD	Soc. Sciences	England
J S AFR STUD	Soc. Sciences	England

Table 1 shows that the ISI view of African publishers is largely a South African view with only two exceptions. If African journals are published outside Africa, such publications generally occur in England.

Next we determined for each of these 28 journals the ISI journal category and picked a matching Western journal. Besides being published in North America or Western Europe, this match was guided by the ISI-impact factor. The journal with the nearest impact factor was chosen. In this process, we tried to alternate higher and lower impact factors. This resulted in Table 2. For one journal (S AFR J GEOL) the matching journal was later found not to have a sufficient amount of publication data. A similar thing happened the other way around for ENVIRON BIOL FISH (we had to remove its African partner). So we matched those two journals although they belong to different categories. They do have similar impact factors and did not turn out to be outliers in our further investigations.

Table 2 Matching African and Western journals

African Journal	JCR^® Category	Matching Western Journal
AFR ENTOMOL	Entomology	SOCIOBIOLOGY
B CHEM SOC ETHIOPIA	Chem. multidiscipl.	ACTUAL CHIMIQUE
BOTHALIA	Plant sc.	BOT HELV
DISCOV INNOVAT	Multidiscipl. sc.	R&D MAG
J S AFR I MIN METALL	Metallurg.	T I MIN METALL C
J S AFR VET ASSOC	Vet. sc.	VLAAMS DIERGEN TIJDS
ONDERSTEPOORT J VET	Vet. sc.	J VET MED A
OSTRICH	Ornithology	WILSON BULL
S AFR J ANIM SCI	Agriculture, dairy, and animal science	ARCH TIERZUCHT
S AFR J BOT	Plant sc.	CRYPTOGAM BRYOL
S AFR J CHEM-S-AFR T	Chem. multidiscipl.	AFINIDAD
S AFR J GEOL	Geology / Marine & Freshwater	ENVIRON BIOL FISH
S AFR J SCI	Multidiscipl. sc.	SCI ENG ETHICS
S AFR J SURG	Surgery	J CARDIAC SURG
S AFR J WILDL RES	Ecology	NORTHWEST SCI
SAMJ S AFR MED J	Medicine, general & internal	AVIAT SPACE ENVIR MD
WATER SA	Water resources	ENVIRON GEOL
AFR J ECOL	Ecology	COMPOST SCI UTIL
J AFR EARTH SCI	Geosc. multidiscipl.	NAT HAZARDS
S AFR J ECON	Economics	EASTERN EUR ECON
S AFR J PSYCHOL	Psychology, multidiscpl.	SWISS J PSYCHOL
AFR AFFAIRS	Area studies	J ASIAN STUD
AFR TODAY	Political sc.	INT POLITIK
AFRICA	Anthropology	ETHNOLOGY
J AFR ECON	Economics	JAHRB NATL STAT
J AFR HIST	History	J AM HIST
J MOD AFR STUD	Area Studies	EUROPE-ASIA STUD
J S AFR STUD	Area Studies	J LAT AM STUD

Methods

We denote by TOT_J(Y) the total number of citations received by journal J in the year Y. These citations refer to all articles published in journal J since its starting date. The symbol X_q, 0 < q < 1, denotes the number of publication years from the year Y which account for q x 100% of current, i.e., in the year Y, citations received. Time is expressed here in years.

Further, the cumulative number of articles published by journal J during the period [Z₁, Z₂] is denoted as CPUB_J(Z₁,Z₂). Then, the q-percentile impact factor of journal J in the year Y, denoted as qIF_J(Y), is defined as (Rousseau, 2005)

Note that (discrete) counting is performed as follows in the JCR^®. Articles published during the year Y are said to be the articles published during year one. This means that the standard ISI-impact factor takes publications in the years two and three into account. If q = 0.5, this percentile impact factor is called the median impact factor, denoted as MIF. The MIF has, essentially, been introduced by Sombatsompop et al. (2004) and generalized further by Rousseau (2005). More details about its calculation and some examples can be found in Rousseau, 2005. If q = 0.25, we obtain the impact factor corresponding to the first quartile, denoted as Q₁IF. Percentile impact factors have led Egghe (20054) to introduce and model fractional relative impact factors.

Results

Table 3 shows the ISI impact factor (IF), the first quartile impact factor (Q₁IF), and the median impact factor (MIF) for the year 2003. If the median cited age is more than ten, no MIF has been calculated. Table 4 shows average impact factors (three types) and corresponding standard deviations (stdev) for the two groups of journals studied in this article.

Table 3 Impact factors for African and matched Western journals

African Journal	IF	Q1IF	MIF	Western Journal	IF	Q1IF	MIF
AFR ENTOMOL	0.577	0.31	0.28	SOCIOBIOLOGY	0.590	0.47	0.60
B CHEM SOC ETHIOPIA	0.190	0.16	0.20	ACTUAL CHIMIQUE	0.112	0.07	0.10
BOTHALIA	0.281	0.23	----	BOT HELV	0.280	0.45	-----
DISCOV INNOVAT	0.013	0.06	0.07	R&D MAG	0.015	0.01	0.01
J S AFR I MIN METALL	0.061	0.07	----	T I MIN METALL C	0.057	0.12	-----
J S AFR VET ASSOC	0.265	0.38	----	VLAAMS DIERGEN TIJDS	0.259	0.18	----
ONDERSTEPOORT J VET	0.548	0.47	----	J VET MED A	0.558	0.48	----
OSTRICH	0.187	0.30	----	WILSON BULL	0.268	0.49	----
S AFR J ANIM SCI	0.143	0.30	0.33	ARCH TIERZUCHT	0.267	0.21	0.23
S AFR J BOT	0.462	0.38	0.36	CRYPTOGAM BRYOL	0.536	0.38	0.35
S AFR J CHEM-S-AFR T	0.240	0.26	0.31	AFINIDAD	0.157	0.13	0.15
S AFR J GEOL	1.021	1.01	1.05	ENVIRON BIOL FISH	0.883	0.98	1.12
S AFR J SCI	0.930	0.82	0.64	SCI ENG ETHICS	0.548	0.49	0.56
S AFR J SURG	0.119	0.21	0.23	J CARDIAC SURG	0.086	0.41	0.59
S AFR J WILDL RES	0.341	0.39	----	NORTHWEST SCI	0.349	0.59	----
SAMJ S AFR MED J	0.989	0.60	----	AVIAT SPACE ENVIR MD	0.946	0.75	----
WATER SA	0.600	0.49	0.56	ENVIRON GEOL	0.605	0.43	0.58
AFR J ECOL	0.479	0.44	0.51	COMPOST SCI UTIL	0.500	0.65	0.82
J AFR EARTH SCI	0.652	0.79	0.96	NAT HAZARDS	0.655	0.34	0.49
S AFR J ECON	0.295	0.20	0.24	EASTERN EUR ECON	0.293	0.15	0.19
S AFR J PSYCHOL	0.164	0.34	0.43	SWISS J PSYCHOL	0.158	0.17	0.23
AFR AFFAIRS	0.820	0.58	0.56	J ASIAN STUD	0.894	0.69	0.62
AFR TODAY	0.075	0.12	0.21	INT POLITIK	0.082	0.10	0.09
AFRICA	0.204	0.50	----	ETHNOLOGY	0.209	0.34	----
J AFR ECON	0.094	0.23	0.33	JAHRB NATL STAT	0.122	0.09	0.10
J AFR HIST	0.459	0.40	----	J AM HIST	0.587	0.59	----
J MOD AFR STUD	0.511	0.44	0.46	EUROPE-ASIA STUD	0.475	0.31	0.38
J S AFR STUD	0.333	0.34	0.39	J LAT AM STUD	0.326	0.37	0.46

Table 4. Average impact factors

	Averages	Standard Deviations (stdev)
Average IF and stdev of African journals	0.395	0.288
Average IF and stdev of matching Western journals	0.386	0.265
Average Q1IF and stdev of African journals	0.386	0.223
Average Q1IF and stdev of matching Western journals	0.373	0.236
Average MIF and stdev of African journals	0.427	0.250
Average MIF and stdev of matching Western journals	0.404	0.287

Statistical tests were performed using StatGraphics Plus. First a two-sided t-test for the difference (H₀: no difference) between the ISI impact factors of the African and the matched Western journals was performed. The results showed that the difference is not statistically significant (p = 0.77). This test is a validation of the matching procedure. Recall that, conventionally, a p-value smaller than 0.05 is considered to indicate a statistically significant result. Next, similar t-tests, based on paired data, were performed for the difference between the African and Western quartile impact factors (p = 0.67) and for the difference between African and Western median impact factors (p = 0.61). Clearly, none of these differences are statistically significant.

In an attempt to find differences among groups, we performed the same test but now for sciences and social sciences journals separately and for African journals published in Africa or in the West separately. Again, none of the differences were found to be statistically significant. Tables 5 and 6 summarize these results.

Table 5. Average impact factors for different subgroups

	Averages	Standard Deviations (stdev)
Average IF and stdev of African journals—sciences	0.426	0.310
Average IF and stdev of matching Western journals—sciences	0.404	0.272
Average IF and stdev of African journals—social sciences	0.328	0.238
Average IF and stdev of matching Western journals—social sciences	0.350	0.263
Average Q1IF and stdev of African journals—sciences	0.404	0.253
Average Q1IF and stdev of matching Western journals—sciences	0.402	0.245
Average Q1IF and stdev of African journals—social sciences	0.350	0.148
Average Q1IF and stdev of matching Western journals—social sciences	0.312	0.214
Average MIF and stdev of African journals—sciences	0.458	0.301
Average MIF and stdev of matching Western journals—sciences	0.467	0.319
Average MIF and stdev of African journals—social sciences	0.374	0.124
Average MIF and stdev of matching Western journals—social sciences	0.296	0.198
Average IF and stdev of African journals—published in Africa	0.391	0.311
Average IF and stdev of matching Western journals—(matched to African journals published in Africa)	0.367	0.269
Average IF and stdev of African journals—published in the West	0.403	0.251
Average IF and stdev of matching Western journals—(matched to African journals published in the West)	0.428	0.268
Average Q1IF and stdev of African journals—published in Africa	0.367	0.238
Average Q1IF and stdev of matching Western journals—(matched to African journals published in Africa)	0.366	0.249
Average Q1IF and stdev of African journals—social sciences—published in the West	0.427	0.194
Average Q1IF and stdev of matching Western journals—(matched to African journals published in the West)	0.387	0.219
Average MIF and stdev of African journals—published in Africa	0.392	0.259
Average MIF and stdev of matching Western journals—(matched to African journals published in Africa)	0.393	0.311
Average MIF and stdev of African journals—social sciences—published in the West	0.489	0.238
Average MIF and stdev of matching Western journals—(matched to African journals published in the West)	0.423	0.264

Table 6. p-values for average differences studied on the basis of Table 5

Test (number of cases between parentheses)	p-value
Difference between IFs of African journals and matching Western journals—sciences (19)	0.46
Difference between IFs of African journals and matching Western journals—social sciences (9)	0.24
Difference between Q1IFs of African journals and matching Western journals—sciences (19)	0.95
Difference between Q1IFs of African journals and matching Western journals—social sciences (9)	0.40
Difference between MIFs of African journals and matching Western journals—sciences (12)	0.91
Difference between MIFs of African journals and matching Western journals—social sciences (7)	0.12
Difference between IFs of African journals published in Africa and matching Western journals (19)	0.33
Difference between IFs of African journals published in the West and matching Western journals (9)	0.17
Difference between Q1IFs of African journals published in Africa and matching Western journals (19)	0.98
Difference between Q1IFs of African journals published in the West and matching Western journals (9)	0.58
Difference between MIFs of African journals published in Africa and matching Western journals (12)	1.00
Difference between MIFs of African journals published in the West and matching Western journals (7)	0.51

The difference between the MIFs of the African journals and those of the matching Western journals in the social sciences had the smallest p-value. The average MIF of African social sciences journals was 0.37 while that of matched Western ones was 0.30.

In the tests discussed above, we tried to find differences between African journals and matched Western ones. Next we performed tests between the three different types of impact factors. More precisely, we performed paired t-tests between IF and Q₁IF and between IF and MIF. For the African journals, the average IF, Q₁IF, and MIF respectively were equal to 0.395, 0.38, and 0.43. For the matched Western journals the corresponding impact factors were 0.386, 0.373, and 0.40. So, on average, the median impact factor seems larger than the ISI-impact factor. This was a surprising result because, based on previous results (Egghe, 1988; Rousseau, 2005), we expected the MIF to be smaller than the IF. Even without a test it was clear that there was no difference between the ISI impact factor (IF) and the first quartile impact factor (Q₁IF). Also, for the median impact factor, differences with the ISI impact factor were not statistically significant (p = 0.56 for African journals; p = 0.72 for the group of matched Western journals).

As we were unable to find any statistically significant difference, we tried one last approach, namely, comparing the subset of African journals that have a matched Western journal publishing only in English (see Appendix for a list). The idea behind this is that if African journals are somewhat outside ISI mainstream journals, then European journals not publishing in English, e.g., French or German, certainly are. We tested twenty journal pairs in this way.

For this particular paired group we found the following p-values:

Difference in IF: p = 0.51

Difference in Q₁IF: p = 0.93

Difference in MIF: p = 0.76

So again, none of these differences were statistically significant. Note also that several South African journals are officially multi-language (in practice, English and Afrikaans). These are BOTHALIA, J S AFR VET ASSOC, S AFR J BOT, S AFR J SURG, S AFR J WILDL RES, WATER SA and S AFR J ECON.

Conclusion and comments

This article illustrates the use of the first quartile and of the median impact factors. Focusing our attention on African journals, we found no significant differences between these journals’ impact factors and their matched Western ones. Matching has been done on the basis of JCR^® subject category and ISI impact factor. Consideration of subgroups such as sciences journals, social sciences journals, and African journals published in Africa as well as African journals published in the West did not show any difference with the corresponding matched group. This finding indicates that for these journals the classical ISI impact factor gives, on average, sufficient information for comparisons with other journals.

For the groups of journals studied here, we did not even find a significant difference between the average ISI impact factor, the first quartile impact factor, and the median impact factor. These results seem to indicate that for journals with a relatively low impact factor there is little difference between the various ways in which synchronous impact factors are calculated. This observation was very surprising for us, and we do not believe it to be generally true. Moreover, focusing on the ISI journal category seems, on average, to lead to similarly shaped citation curves and hence similar percentile impact factors (because the shape of the distribution function determines percentiles).

This brings us to the following research questions. Are the same observations also true for high impact factor journals? More precisely, are the median impact factors, or other percentile impact factors of journals with high ISI impact factors also statistically the same as the classical IF?

Another question for further research is the following: What can be observed if we compare journals with the same IF but belonging to different subject fields? Will their percentile impact factors diverge?

Acknowledgement

The members of the STIMULATE 4 Group express their sincere thanks to Professor Paul Nieuwenhuysen (VUB, Brussels) and Sahdya Khan who made this multinational collaboration possible. The STIMULATE 4 project (www.vub.ac.be/BIBLIO/itp/stimulate4/) of which this research project was a small part has been supported by the VLIR (Flemish Interuniversity Council). The authors thank Thomson-ISI and Henry Small for the permission to publish the data contained in this article.

The SIMULATE 4 Group consists of (in alphabetical order)

Sainul Abideen P. (India), Orlando Gregorio (Cuba), Virginia Hamwela (Zambia), Ngenjo Kabyema (Zambia), Rahma Kubaiza (Uganda), Henock Legesse (Ethiopia), Happiness Sibongile Mabuza (Swaziland), Balla Elnour Mahasin (Sudan), Mendoza Christine Manglal-lal (Philippines), Juma James Masele (Tanzania), Daisy Mendoza (Philippines), Irvine Mutsungi (Zimbabwe), Juliet Nakasagga (Uganda), Ronald Rousseau (Belgium), Manuel Soto Benavides (Chile).

References

Egghe, L. (1988). Mathematical relations between impact factors and average number of citations. Information Processing and Management, 24, 567-576.

Egghe, L. (2005). Continuous weighted Lorenz theory and applications to the study of fractional relative impact factors. Information Processing and Management

Garfield, E., and Sher, I.H. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14, 195-201.

Hirst, G. (1978). Discipline impact factors: A method for determining core journal listings. Journal of the American Society for Information Science, 29(4), 171-172.

Ingwersen, P., Larsen, B., Rousseau, R, & Russell, J. (2001). The publication-citation matrix and its derived quantities. Chinese Science Bulletin, 46(6), 524-528.

Jin, B., & Wang, B. (1999). Chinese Science Citation Database: Its construction and application. Scientometrics, 45(2), 325-332.

Moed, H.F., Burger, W.J.M., Frankfort, J.G., & van Raan, A. F.J. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14, 131-149.

Rousseau, R. (2002). Journal evaluation: Technical and practical issues. Library Trends, 50, 418-439.

Rousseau, R. (2005). Median and percentile impact factors: A set of new indicators. Scientometrics, 63, 431-441.

Sombatsompop, N., Markpin, T., Premkamolnetr, N. (2004). A modified method for calculating the impact factors of journals in ISI Journal Citation Reports: Polymer science category in 1997-2001. Scientometrics, 60, 217-235.

Wu Y., Pan, Y., Zhang, Y., Ma, Z., Pang, J., Guo, H., Xu, B. and Yang, Z. (2004). China scientific and technical papers and citations (CSTCP): History, impact and outlook. Scientometrics, 60, 385-397.

Appendix

Western Journal	Country of Publication	Language
SOCIOBIOLOGY	USA	English
ACTUAL CHIMIQUE	France	French
BOT HELV	Switzerland	Multi-language
R&D MAG	USA	English
T I MIN METALL C	England	English
VLAAMS DIERGEN TIJDS	Belgium	Multi-language
J VET MED A	Germany	English
WILSON BULL	USA	English
ARCH TIERZUCHT	Germany	German
CRYPTOGAM BRYOL	France	Multi-language
AFINIDAD	Spain	Multi-language
ENVIRON BIOL FISH	Netherlands	English
SCI ENG ETHICS	England	English
J CARDIAC SURG	USA	English
NORTHWEST SCI	USA	English
AVIAT SPACE ENVIR MD	USA	English
ENVIRON GEOL	Germany	English
COMPOST SCI UTIL	USA	English
NAT HAZARDS	Netherlands	English
EASTERN EUR ECON	USA	English
SWISS J PSYCHOL	Switzerland	English
J ASIAN STUD	USA	English
INT POLITIK	Germany	German
ETHNOLOGY	USA	English
JAHRB NATL STAT	Germany	Multi-language
J AM HIST	USA	English
EUROPE-ASIA STUD	England	English
J LAT AM STUD	England	English