Many current multivariate filter feature selection approaches consider redundancy and relevance between features and class vectors simultaneously. However, these multivariate filter algorithms calculate the suitability of features by only the intrinsic characteristics of the data. In this paper, we suggest a new distributed framework to offset the multivariate feature selection problem. We propose the interaction with classifiers in multivariate filter feature selection. Our proposed framework calculates the relevance of each feature to class labels by embedded algorithms. Then, this technique examines redundancy among features through multivariate filter algorithms. In addition, in the proposed framework, we use horizontal distribution of data instead of using all them once. This approach reduces the runtime of the process in datasets with many samples and environments without centralized data. The results of the evaluation show that the proposed framework can improve classification accuracy compared with the methods just based on multivariate filters. In addition, the experimental results demonstrate that our algorithm outperforms compared approaches in precision, recall, and runtime.