Exact bounds of Spearman’s footrule in the presence of missing data with applications to independence testing

This work studies exact bounds of Spearman’s footrule in the presence of missing data for two n-dimensional distinct real-valued vectors X and Y. The lower bound is obtained by sequentially constructing imputations of the partially observed vectors, each with a non-increasing value of Spearman’s footrule. The upper bound is found by first considering the set of all possible values of Spearman’s footrule for imputations of X and Y, and then the size of this set is gradually reduced using several constraints. Algorithms with computational complexities 𝑂⁡(𝑛^2) and 𝑂⁡(𝑛^3) are provided for computing these tight lower and upper bounds, respectively. As an application, we propose a novel two-sample independence testing method for data with missing values. Improving on all existing approaches, our method controls the Type I error under arbitrary missingness. Simulation results demonstrate that our method has good power, typically when the proportion of pairs containing missing data is below 15%. We illustrate our method on real-world data by comparing the dependence of yearly precipitation levels in the United Kingdom, Germany and the Unites States.