Unstructured text documents generated daily by millions of internet users have allocated a considerable volume of data in this digital age. Text document clustering is widely regarded as a highly effective method for analyzing text documents, particularly in response to the growing prevalence of big data. This technique is employed to group documents based on their content. Many text clustering algorithms commonly employ a single-criterion optimization strategy, which frequently falls short of generating effective clustering solutions across diverse datasets exhibiting various clustering characteristics. To address this challenge, the multi-objective meta-heuristic approach is employed to achieve optimal clustering outcomes by maximizing or minimizing multiple objective functions. Balancing exploitation and exploration is a crucial aspect of the meta-heuristic approaches. To enhance this balance, we propose a Multi-objective Firefly Differential Jaya (MFDJ) evolutionary algorithm. MFDJ enhances the quest for optimal clustering by improving the equilibrium between exploitation and exploration. We evaluate MFDJ on some text datasets. As the experimental results show, the MFDJ algorithm outperforms new document clustering algorithms.