A multiverse approach to assessing the impacts of analysis choice on estimates of habitat selection

Abstract

Possibly the best way to determine an animal’s needs is to follow their movements. Once we have learnt of the animal’s movements we can infer habitat requirements, behaviour, and potential threats. Combined, the movement data and inferences can inform decisions on animal conservation. However, extracting useful information from animal movement requires many steps, from sampling to analysis. Other studies have shown that a single dataset can result in many different answers in the hands of different researchers, so how can we be confident the results from animal movement are leading to the correct decisions in animal conservation? One approach to explore this question is to perform a wide range of alternative analyses on conservation-relevant datasets and examine whether the resulting conclusions would differ. The combination of all these alternative analyses can be called a multiverse. We used a multiverse approach to explore thousands of ways of extracting the habitat selection estimates from the movement of simulated animals with a pre-defined selection. We found that despite different sampling approaches, and completely different analysis methods, the results agree and correctly identify habitat selection. The agreement between different habitat selection estimates tended to be better with more data, and when using more modern analysis methods. We applied this same multiverse of analysis methods to four case studies of snake movement data from Thailand, with the aim of determining the robustness of the originally published estimates of habitat selection. The spread of results from the multiverse indicated that the original conclusions were in agreement with the vast majority of analysis endpoints, and not the product of a particularly peculiar combination of analysis decisions. However, it did reveal the species-specific impacts of certain analysis decisions indicating that it would be unwise to discount analysis choice as a potential source of disagreement between studies examining the same or similar questions.