Effect of grazing on methane uptake from Eurasian steppe of China
BMC Ecology volume 18, Article number: 11 (2018)
The effects of grazing on soil methane (CH4) uptake in steppe ecosystems are important for understanding carbon sequestration and cycling because the role of grassland soil for CH4 uptake can have major impacts at the global level. Here, a meta-analysis of 27 individual studies was carried out to assess the response patterns of soil CH4 uptake to grazing in steppe ecosystems of China. The weighted log response ratio was used to assess the effect size.
We found that heavy grazing significantly depressed soil CH4 uptake by 36.47%, but light and moderate grazing had no significant effects in grassland ecosystem. The response of grassland soil CH4 uptake to grazing also was found to depend upon grazing intensity, grazing duration and climatic types. The increase in soil temperature and reduced aboveground biomass and soil moisture induced by heavy grazing may be the major regulators of the soil CH4 uptake.
These findings imply that grazing effects on soil CH4 uptake are highly context-specific and that grazing in different grasslands might be managed differently to help mitigate greenhouse gas emissions.
Methane (CH4) is a long-lived greenhouse gas (average atmospheric residence about 7.9 years) , contributing approximately 30% of total net anthropogenic radiative forcing, which is second only to the radiative forcing of carbon dioxide (CO2) . The concentration of atmospheric CH4 has been increasing because of anthropogenic activities over the last 150 years , reaching 1813 ppb in 2011, which is 159% higher than the pre-industrial level . These changes can exert strong effects on terrestrial carbon cycles and global warming. Natural soils are the second largest sink of atmospheric CH4 after oxidation in the troposphere by OH radicals, with an estimated global sink of 20–45 Tg CH4 year−1 . Grassland soils play a considerable role in mitigating greenhouse gas emissions because grasslands are one of the largest terrestrial biomes worldwide [4, 5]. Most studies of CH4 uptake have been conducted in grasslands of North America [6,7,8,9] and Europe [10,11,12]. The Eurasian steppe has only received attention in recent years [13,14,15,16,17,18]. In the Eurasian steppes, unprecedented increase in grazing pressure has led to severe grassland degradation, which in turn decreases CH4 uptake of soil [17, 19, 20].
China’s grasslands are representative of the Eurasian steppe in terms of climate, topography, soils properties, vegetation composition and land use history . Generally, China’s steppe ecosystems are classified into desert steppes, typical steppes, meadow steppes and alpine steppes , and better understanding of soil-atmosphere CH4 exchange dynamics in this region can contribute to broader understanding of such dynamics in arid and semiarid areas. As elsewhere in the Eurasian steppe, grazing is the main land use, and most grasslands have been grazed for several decades to centuries. There is clear evidence that grazing in grassland ecosystems alters the activity or community composition of soil microorganisms and vegetation in ways that lead to decreased soil CH4 uptake [19, 20]. Therefore, grazing may alter the loading of CH4 flux to the atmosphere, which may contribute to further global warming. Understanding how grazing management affects the CH4 budget is thus important, from both scientific and political perspectives.
The potential impacts of grazing on the soil CH4 uptake have been investigated in an increasing number of field experiment studies across the worldwide grasslands. In order to improve understanding of the responses of CH4 uptake of steppe soils to grazing, several studies have been conducted in China [14,15,16,17, 19, 23]. Previous studies have reported inconsistent grazing effects on soil CH4 uptake by grasslands [14, 24, 25]. Liu et al.  suggested that winter grazing decreased soil CH4 uptake during the growing season by 47% in a temperate semiarid steppes. Qi et al.  reported that continuous grazing promoted CH4 uptake during growing season in grassland. Contradictory responses of CH4 uptake to grazing may depend on differences in grazing intensities, grazing duration or soil environmental conditions. Therefore, there are still many uncertainties in the CH4 uptake responses of steppe soils to increased grazing pressure.
Most previous studies on soil-atmospheric CH4 exchange in grazed Eurasian steppes were restricted to a single grassland type [16, 18, 20], or a single grazing intensity in the growing season [17, 23] or spring-thaw period . Few reports are available on soil CH4 uptake in the different grazing intensity and grazing duration . Incomplete considerations of these difference may be increase uncertainty when the overall contribution of grassland ecosystems to the greenhouse effect is assessed . More complete assessments contribute to better understanding of atmospheric CH4 uptake in grazed steppe, and help to identify effective measures to increase the effects of the terrestrial CH4 sink. Therefore, it is needed to compile the available data to reveal the underlying mechanisms of soil CH4 uptake responses to grazing.
To reveal general response patterns of CH4 uptake by steppe soil under grazing, we incorporated factors such as grazing intensity (light, moderate and heavy grazing), grazing duration (< 5 years, 5–10 years and ≥ 10 years), and climatic type (humid/semi-humid, ≥ 400 mm precipitation; arid/semi-arid, < 400 mm precipitation) using data from published papers reporting field experiments conducted in China’s steppes (Additional file 1: Note S1).
Grazing effects showed a strong dependence on grazing intensity (Fig. 1, Table 1). The data suggest that soil CH4 uptake decrease as the grazing intensity increases, however, the effect is significant only under heavy grazing (− 36.47%, p < 0.05) (Fig. 1). The effects of light grazing and moderate on soil CH4 uptake were not significantly different (LG 9.77%; MG 1.22%) (Fig. 1). In addition, heavy grazing significantly reduced soil organic carbon by 5.01%, soil moisture by 16.09% (p < 0.05) and aboveground biomass by 114.83% but increased soil bulk density by 16.84% (p < 0.05), and soil temperature were not significantly different (8.81%, p > 0.5) (Fig. 2a–e).
Grazing also significantly reduced soil CH4 uptake with the different grazing duration (p < 0.05). Significant difference in the soil CH4 uptake response was found between these categories (Qb = 18.42, p = 0.007; Table 1). There is a strong trend indicating that CH4 uptake decreases with increasing duration of grazing activities, and that significant decreases occur when the grazing exceeds 5 years, and especially when it exceeds 10 years duration (Fig. 1). Averaging across studies in different climatic types, grazing significantly decreased soil CH4 uptake by 12.40% in precipitation < 400 mm (p < 0.01) (Table 1), but the effects of grazing on soil CH4 uptake were not significant in precipitation ≥ 400 mm (Fig. 1). In addition, our meta-analysis also showed that aboveground biomass displayed significant correlations with response ratio (RR) of soil CH4 uptake (Fig. 3b). The RR of soil CH4 uptake showed a trend of negative correlation with RR of soil temperature (Fig. 3a), but significantly positive correlation with RR of aboveground biomass was observed (Fig. 3b).
The atmospheric concentration of CH4 has dramatically increased since pre-industrial times because of human activities . Grazing is an important disturbance to grasslands, which could have either positive or negative effects on the consumption of CH4 in grassland ecosystems [17, 19, 23]. Long-term overgrazing is the main cause of grassland degradation and associated dust storms in Eurasian grasslands. About 90% of grasslands in China are degraded to some extent, mainly due to overgrazing . High-intensity ruminant grazing shifts net CH4 exchange in grassland ecosystem. In this study, grazing experiments with duration of longer than 5 years had a significant effect on soil CH4 uptake, while the experiments with duration of less than 5 years were not observed to pose an impact on soil CH4 uptake (Fig. 1). It likely also suggests that the grazing intensity is increased induced by grazing treatment in long term. The results of our study indicate that overgrazing has significant negative effects on the CH4 uptake of grassland soils in China, which would most likely cause a large decrease in soil CH4 uptake and a decrease in carbon sequestration of grassland in China.
This meta-analysis included a relatively small number of studies (n = 27) compared to other meta-analyses, and we were limited to considering interactions among factors with a wide range of values across grassland sites in China. In this study, we did not consider the grazing experiment at different grassland types because few reports were concentrated on the effects of grazing on soil CH4 uptake across different grassland types, and most of the identified studies were carried out in Inner Mongolia. The paucity of data from different grassland types indicates that insufficient research has been conducted on the impact of grazing on CH4 flux of grassland soils in China. However, we found soil CH4 uptake has different response to grazing at different precipitation types. The differential responses among climatic types estimated in our study suggests that arid and semi-arid region are more fragile to grazing practice in grassland ecosystems. Moreover, the vegetation cover also changed completely with different precipitation climatic type. It was found that vegetation influences methane uptake of soils indirectly via possible changes in methanotrophic communities due to changes of plant species . Further research is required on the effects of grazing practice on CH4 fluxes of different grassland ecosystems in China.
Research indicates that light grazing may change the microbial community in grassland soils, which could result in a positive response of CH4 to light grazing [20, 26]. The results of this meta-analysis suggest light grazing has a 10.49% higher CH4 uptake than un-grazing, although this was not statistically significant (Fig. 1). Chen et al.  showed that light-to-moderate grazing with stocking rates of < 1 sheep ha−1 year−1 did not significantly change the annual CH4 uptake. However, heavy grazing reduced 24–31% of annual CH4 uptake in typical grassland of China. Tang et al.  also suggested in their study that grazing affected CH4 uptake fluxes variably in three grassland types (meadow grassland, typical grassland and desert grassland) of Inner Mongolia. Therefore, how CH4 flux responds to grazing is complex and should be studied carefully under different grassland type and conditions.
In this meta-analysis, we found that an increase in grazing intensity induced a reduction in CH4 uptake by grassland soils. Despite some limitations, the analysis revealed several mechanisms that may have contributed to a significant reduction in soil CH4 flux under high-intensity grazing. First, trampling by grazing animals compacts the topsoil and increases soil bulk density (Fig. 2a), which would decrease the diffusion of CH4 from the atmosphere into the soil [20, 23]. This may also reduce the amount of atmospheric O2 diffusing into the soil, which could result in an increase in anaerobic conditions and hence an increase in CH4 production. Second, soil organic carbon (SOC) decreased along the grazing gradient in this meta-analysis (Fig. 2d), which may result in a reduction of CH4 uptake in soil due to reduced soil carbon cycles in heavily grazed sites. Soils contain a large stock of SOC, and slight changes in SOC stock can represent large CO2 and CH4 fluxes . Third, heavy grazing significantly reduced the aboveground biomass, which will decline soil water content (Fig. 2c, e) and increase water stress that could inhibit the activities of methanotrophs [20, 23] and further affect the net CH4 flux. In this study, RR of soil CH4 uptake increased linearly with the increase in RR of aboveground biomass but decreased linearly with RR of soil temperature (Fig. 3a, b). The decrease in aboveground biomass may reduce the soil moisture due to its effect on increasing evaporation . However, more studies should be carried out in different grassland ecosystems to understand how soil CH4 uptake respond to soil water stress under different grazing intensity.
Urine and dung patches in grazed grassland are hotspots of CH4 emission [28, 29]. The higher potential for excreta induced CH4 emissions in grazed sites could offset some of the CH4 uptake by soils [20, 28]. In addition, as has been shown in other studies from a typical steppe in Inner Mongolia, grazing animals change the structure of the methane-oxidizing bacterial community. Zhou et al.  have reported that the community composition of soil methane-oxidizing bacteria was different between grazed and non-grazed sites. However, few studies reported other potentially important factors impacting CH4 fluxes, such as urine and dung patches, or soil microbial community under grazing management, so our analysis was not able to evaluate the effect of these factors on the responses of CH4 flux to grazing. These factors may be important because other studies in temperate grasslands of North America and China [9, 17, 20, 23] have suggested that CH4 flux may be associated with livestock urine and dung patches and soil methane-oxidizing bacteria.
Given the small total sample size, the inclusion of studies from that region of China may have some unclearly influenced the analysis of relationships with environmental variables. Our meta-analysis was limited by the relative paucity of published field studies, incomplete representation of different grassland types, and limited reporting of potentially important variables describing soil and vegetation properties. Filling these data gaps through field experiments and publication will be necessary to provide a stronger empirical basis for future large-scale assessments.
Grazing has the potential to change soil CH4 uptake in steppe ecosystems, with consequent impacts on the carbon cycle and climate change. Our results and previous findings [14, 17, 20, 23] indicate that heavy grazing decreases soil CH4 uptake in steppe ecosystems in China. These findings in this and previous studies [17, 18, 20, 23] imply that grazing effects on soil CH4 uptake are highly context-specific and that grazing in different grasslands might be managed differently to help mitigate greenhouse gas emissions, especially when different grazing intensities are taken into consideration.
In order to identify all relevant studies on the effect of grazing on soil-atmosphere CH4 fluxes in China, a comprehensive search was conducted on the Web of Science and the Chinese magazine network (CNKI) database (before 2015). The search terms were ‘methane’ and‘flux’, ‘uptake’, ‘oxidation’, or ‘consumption’ and ‘grazing’. These searches resulted in over 27 papers that studied soil CH4 dynamics under grazing in steppes of China (Additional file 1: Note S1, Additional file 2: Table S1) by including studies that compared soil CH4 fluxes for grazers plot (different grazing intensity) compared to a paired un-grazed plot. In addition, we collected other variables of the treatment and control plots if reported, such as soil bulk density, soil organic carbon, soil moisture, soil temperature and aboveground biomass values. For CH4 flux, the preferred unit is flux per unit area per day (mg m−2 day−1), and all other flux units (e.g. µg m−2 h−1) were converted if data on plot area was provided in the paper. Mean values for CH4 fluxes were taken directly from the available literature. Data from graphs were extracted by digitizing the figures using a graph data extractor software (Graph Data Extractor by Dr. A J Matthews).
The data were selected according to the following criteria: (1) the studies reported changes in soil-atmosphere CH4 exchange in both grazing and control groups; (2) the means, standard deviations (SD) or standard errors (SE), and sample sizes (n) of the CH4 fluxes were provided or could be calculated from the studies; (3) relevant experimental information was reported, including grazing intensity, steppe type, grazing duration, mean annual precipitation (MAP) and mean annual temperature (MAT).
Due to large variation in both the types of steppe (i.e., desert, typical steppe, meadow and alpine steppe) and the grazing units (i.e., dry sheep equivalent ha−1, ha steer−1, animal unit month ha−1) reported in each study, we characterize grazing intensity based on the individual study authors’ own qualitative classification of grazing intensity as ‘light’, ‘moderate’, or ‘heavy’. If the cases were not given a qualitative grazing level, we classified data based on the authors’ qualitative description of the site. The stocking rates are given in more detail in Additional file 2: Table S1 according to the original studies. Sheep was the main grazing animal at most of studies.
Following the techniques reported in Wan et al.  to calculate the response ratio (RR) of response variables to grazing, the meta-analysis was conducted using MetaWin 2.1 software package (Sinauer Associates, Inc., Sunderland, MA, USA) . The natural log-transformed ratio of response variables in grazed (Xe) to un-grazed (Xc) plots was used to estimate the effect size of the grazing treatment. The means, standard deviations of response variables and sample sizes were reported or could be calculated. SE and confidence interval (CI) were transformed to SD before calculation. For studies that did not report SE, SD was assigned as 1/10 of mean [30, 32]. The weighted response ratio (ln RR) was calculated using MetaWin 2.1. The CI on effect-size estimates was generated by bootstrapping the data. The homogeneity test was used to further examine whether different groups of independent variables would result in different responses. The total heterogeneity (QT) was partitioned into two components, within-group heterogeneity (Qw) and between-group heterogeneity (Qb). The Q statistic approximately followed a χ2 distribution, which allowed a significance test of the null hypothesis that all response ratios are equal [30, 33]. If the value of Qb is larger than a critical value, this indicates that an independent variable had a significant impact on the response ratio . Grazing was considered to have a significant effect on variables if the 95% CI did not overlap zero, whereas the grazing effects of different groups were considered to significantly differ from each other if their bootstrap CIs did not overlap [30, 33]. In addition, the relationships between change in CH4 flux and environmental factors were examined using correlation analysis. The significance of differences was assessed at p < 0.05 level.
- CH4 :
- CO2 :
mean annual precipitation
mean annual temperature
weighted response ratio
soil organic carbon
- QT :
- Qw :
- Qb :
Lelieveld J, Crutzen PJ, Dentener FJ. Changing concentration, lifetime and climate forcing of atmospheric methane. Tellus B. 1998;50(2):128–50.
IPCC. Climate Change 2007: The Physical Science Basis. Contribution of working group I to the fourth assessment report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, United Kingdom and New York; 2007.
Aronson EL, Helliker BR. Methane flux in non-wetland soils in response to nitrogen addition: a meta-analysis. Ecology. 2010;91(11):3242–51.
Wang Y, Chen H, Zhu Q, Peng C, Wu N, Yang G, Zhu D, Tian J, Tian L, Kang X, et al. Soil methane uptake by grasslands and forests in China. Soil Biol Biochem. 2014;74:70–81.
Wang Z, Song Y, Gulledge J, Yu Q, Liu H, Han X. China’s grazed temperate grasslands are a net source of atmospheric methane. Atmos Environ. 2009;43(13):2148–53.
Kinney CA, Mosier AR, Ferrer I, Furlong ET, Mandernack KW. Effects of the herbicides prosulfuron and metolachlor on fluxes of CO2, N2O, and CH4 in a fertilized Colorado grassland soil. J Geophys Res Atmos 2004;109(D5). https://doi.org/10.1029/2003JD003656.
Mosier AR, Pendall E, Morgan JA. Effect of water addition and nitrogen fertilization on the fluxes of CH4, CO2, NOx, and N2O following 5 years of elevated CO2 in the Colorado Shortgrass Steppe. Atmos Chem Phys. 2003;3:1703–8.
Mosier AR, Delgado JA. Methane and nitrous oxide fluxes in grasslands in western Puerto Rico. Chemosphere. 1997;35(9):2059–82.
Del Grosso SJ, Parton WJ, Mosier AR, Ojima DS, Potter CS, Borken W, Brumme R, Butterbach Bahl K, Crill PM, Dobbie K. General CH4 oxidation model and comparisons of CH4 oxidation in natural and managed systems. Global Biogeochem Cycles. 2000;14(4):999–1019.
Soussana JF, Allard V, Pilegaard K, Ambus P, Amman C, Campbell C, Ceschia E, Clifton-Brown J, Czobel S, Domingues R, et al. Full accounting of the greenhouse gas (CO2, N2O, CH4) budget of nine European grassland sites. Agr Ecosyst Environ. 2007;121(1–2):121–34.
van den Pol-van Dasselaar A, van Beusichem ML, Oenema O. Effects of nitrogen input and grazing on methane fluxes of extensively and intensively managed grasslands in the Netherlands. Biol Fert Soils. 1999;29(1):24–30.
Pol-van Dasselaar AVD, Beusichem MLV, Oenema O. Effects of grassland management on the emission of methane from intensively managed grasslands on peat soil. Plant Soil. 1997;189(1):1–9.
Wang YS, Xue M, Zheng XH, Ji BM, Du R, Wang YF. Effects of environmental factors on N2O emission from and CH4 uptake by the typical grasslands in the Inner Mongolia. Chemosphere. 2005;58(2):205–15.
Holst J, Liu C, Yao Z, Brueggemann N, Zheng X, Giese M, Butterbach-Bahl K. Fluxes of nitrous oxide, methane and carbon dioxide during freezing-thawing cycles in an Inner Mongolian steppe. Plant Soil. 2008;308(1–2):105–17.
Chen W, Wolf B, Yao Z, Bruggemann N, Butterbach-Bahl K, Liu C, Han S, Han X, Zheng X. Annual methane uptake by typical semiarid steppe in Inner Mongolia. J Geophys Res Part D Atmos. 2010;115(D15):D15108–10.
Wei D, Xu-Ri, Tenzin-Tarchen, Wang Y, Wang Y. Considerable methane uptake by alpine grasslands despite the cold climate: in situ measurements on the central Tibetan Plateau, 2008–2013. Global Change Biol. 2015;21(2):777–88.
Tang S, Wang C, Wilkes A, Zhou P, Jiang Y, Han G, Zhao M, Huang D, Schoenbach P. Contribution of grazing to soil atmosphere CH4 exchange during the growing season in a continental steppe. Atmos Environ. 2013;67:170–6.
Wang C, Tang S, Wilkes A, Jiang Y, Han G, Huang D. Effect of stocking rate on soil-atmosphere CH4 flux during spring freeze-thaw cycles in a northern desert steppe, China. PLoS ONE. 2012;7:e36794.
Wang C, Han G, Wang S, Zhai X, Brown J, Havstad KM, Ma X, Wilkes A, Zhao M, Tang S, et al. Sound management may sequester methane in grazed rangeland ecosystems. Sci Rep UK. 2014;4:4444. https://doi.org/10.1038/srep04444.
Chen W, Wolf B, Zheng X, Yao Z, Butterbach-Bahl K, Brueggemann N, Liu C, Han S, Han X. Annual methane uptake by temperate semiarid steppes as regulated by stocking rates, aboveground plant biomass and topsoil air permeability. Global Change Biol. 2011;17(9):2803–16.
Bai Y, Wu J, Pan Q, Huang J, Wang Q, Li F, Buyantuyev A, Han X. Positive linear relationship between productivity and diversity: evidence from the Eurasian Steppe. J Appl Ecol. 2007;44(5):1023–34.
Yan L, Zhou G, Zhang F. Effects of different grazing intensities on grassland production in China: a meta-analysis. PLOS ONE. 2013;8:e8146612.
Liu C, Holst J, Brüggemann N, Butterbach-Bahl K, Yao Z, Yue J, Han S, Han X, Krümmelbein J, Horn R, et al. Winter-grazing reduces methane uptake by soils of a typical semi-arid steppe in Inner Mongolia. China Atmos Environ. 2007;41(28):5948–58.
Qi Y, Dong Y, Yang X, Geng Y, Liu L, Li M. Effects of grazing on carbon dioxide and methane fluxes in typical temperate grassland in Inner Mongolia, China. Resour Sci. 2005;27:103–9 (in Chinese).
Praeg N, Wagner AO, Illmer P. Plant species, temperature, and bedrock affect net methane flux out of grassland and forest soils. Plant Soil. 2017;410:193–206.
Zhou X, Wang Y, Huang X, Hao Y, Tian J, Wang J. Effects of grazing by sheep on the structure of methane-oxidizing bacterial community of steppe soil. Soil Biol Biochem. 2008;40(1):258–61.
Krogh L, Noergaard A, Hermansen M, Greve MH, Balstroem T, Breuning-Madsen H. Preliminary estimates of contemporary soil organic carbon stocks in Denmark using multiple datasets and four scaling-up methods. Agr Ecosyst Environ. 2003;96(1–3):19–28.
Lin X, Wang S, Ma X, Xu G, Luo C, Li Y, Jiang G, Xie Z. Fluxes of CO2, CH4, and N2O in an alpine meadow affected by yak excreta on the Qinghai-Tibetan plateau during summer grazing periods. Soil Biol Biochem. 2009;41(4):718–25.
Wang X, Huang D, Zhang Y, Chen W, Wang C, Yang X, Luo W. Dynamic changes of CH4 and CO2 emission from grazing sheep urine and dung patches in typical steppe. Atmos Environ. 2013;79:576–81.
Wan SQ, Hui DF, Luo YQ. Fire effects on nitrogen pools and dynamics in terrestrial ecosystems: a meta-analysis. Ecol Appl. 2001;11(5):1349–65.
Lin D, Xia J, Wan S. Climate warming and biomass accumulation of terrestrial plants: a meta-analysis. New Phytol. 2010;188(1):187–98.
Luo YQ, Hui DF, Zhang DQ. Elevated CO2 stimulates net accumulations of carbon and nitrogen in land ecosystems: a meta-analysis. Ecology. 2006;87(1):53–63.
Gurevitch J, Hedges LV. Meta-analysis: combining the results of independent experiments. In: Scheiner SM, Gurevitch J, editors. Design and analysis of ecological experiments. Oxford: Oxford Univ. Press; 2001. p. 347–69.
SMT, YJZ, KW, CJW designed the study, SMT and XJZ conducted the study, SMT, YJZ, XJZ, wrote the paper, SMT, KW and AW revised the paper. All authors read and approved the final manuscript.
We are grateful to all the scientists who contribute to the collection of data used in this meta-analysis.
The authors declare that they have no competing interests.
Availability of data and materials
All the data are summarized in the manuscript itself. Please contact the corresponding author regarding any additional queries related to the dataset generated and analyzed during the current study. The datasets in this study are available from the corresponding author on reasonable request.
Consent for publication
Ethics approval and consent to participate
This study was financially supported by Inner Mongolia Natural Science Foundation (2016BS0320), National Natural Science Foundation (31772654) and National Scientific Research Institutions to Important Agricultural Extension Service Pilot in Hebei Province -Northern Ecology Function Area (29015122).
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tang, S., Zhang, Y., Zhai, X. et al. Effect of grazing on methane uptake from Eurasian steppe of China. BMC Ecol 18, 11 (2018). https://doi.org/10.1186/s12898-018-0168-x