Introduction

Clustering is one of the most fundamental tasks in the field of data mining and machine learning that aims at uncovering the inherent patterns and structures in data, providing valuable insights in diverse applications. In recent years, deep clustering models [20, 25, 40] have emerged as a major trend in clustering techniques for complex data due to their superior feature extraction capabilities compared to traditional shallow methods. Generally, a feature extracting encoder such as deep neural networks is first applied to map the input data to an embedding space, then traditional clustering techniques such as k-means are applied to the embeddings to facilitate the downstream clustering tasks [10, 28]. There are also several recent works [15, 33, 35–37] that integrate the feature learning process and clustering into an end-to-end framework, which yield high performance for large-scale datasets.

While existing deep approaches have achieved notable success on clustering, they primarily focus on training a model to obtain optimal clustering performance on the data from a given domain. When data from a new domain is present, an interesting question is can we leverage the acquired knowledge from the learned model on trained domains to guide the clustering process in new domains. Unfortunately, existing deep clustering models can be hardly transferred from one domain to another. This limitation arises primarily from the fixed centroid-based learning approach employed by these methods. As illustrated in Figure 1, discrepancies often exist between the distributions of the source and target domains. Consequently, the learned fixed centroids may no longer be suitable for the target domain, leading to suboptimal clustering results. However, the process of training a new model from scratch for each domain incurs a substantial computational burden. More importantly, the acquired knowledge pertaining to the intraand inter-clusters structure and patterns remains underutilized, impeding its potential to guide the clustering process on new data from similar domains. These limitations significantly hinder the practicability of deep clustering methods.

To address these limitations, there is a need for transferable deep clustering models that can leverage acquired knowledge from trained domains to guide clustering in new domains. By transferring the underlying principles of clustering on trained source domains, the model could learn how to cluster better and adapt such knowledge to clustering new data in the target domains. Unfortunately, there exists no trivial way to directly generalize existing deep clustering methods due to several major challenges: (1) Difficulty in unsupervisely learning the shared knowledge among different domains. In clustering scenarios, where labeled data is unavailable, extracting meaningful and transferable knowledge that capture the commonalities of underlying cluster structures across domains is challenging. (2) Difficulty in ensuring the learned knowledge can be adapted and customized to target domains. As shown in Figure 1(b), the distribution discrepancies between source and target domains can significantly harm the clustering performance of existing deep clustering models. Adapting the shared knowledge to new domains remains a challenging task in order to mitigate the negative impact of these distribution discrepancies. (3) Difficulty in theoretically ensuring a stable learning process of clustering module. Unlike supervised learning tasks, clustering models lack labeled data to provide guidance during training, making it even more crucial to establish theoretical guarantees for stability. Addressing this challenge requires developing theoretical frameworks that can provide insights into the stability and convergence properties of clustering algorithms.

In order to address the above metioned challenges, in this paper we propose a novel method named Transferable Deep Clustering Model (TDCM). To address the first challenge, we introduce an end-to-end learning framework that can jointly optimize the feature extraction encoder and a learnable clustering module. This framework aims to leverage the learned model parameter to capture the shared intra-cluster and inter-cluster structure derived from trained cluster patterns. Therefore, the shared knowledge can be effectively transferred to unseen data from new domains. To solve the second challenge, instead of optimizing a fixed set of centroids, a novel learnable attention-based module is proposed for the clustering process to automatically adapt centroids to the new domains, as illustrated in the Figure 1(c). Therefore, the learned clustering model is not limited to the trained source domains and can be easily generalized to other domains. Specifically, this module enables the updating of centroids through a cluster-driven bi-partite attention block, allowing the model to be aware of the similarity relationships among data samples and capture the underlying structures and patterns. Furthermore, we provide theoretical evidence to demonstrate the strong expressive power of the proposed attention-based module in representing the relationships among data samples. Our theoretical analysis reveals that traditional centroid-based clustering models like k-means or GMM can be considered as special cases of our model. This theoretical proof highlights the enhanced capabilities of our approach compared to traditional clustering methods, emphasizing its potential for mining complex cluster patterns from data. Finally, we demonstrate the effectiveness of our proposed framework on both synthetic and real-world datasets. The experimental results show that our method can achieve strongly competitive clustering performance on unseen data by a single forward pass.