Collaborative Deep Learning
for Recommender Systems

Abstract

Collaborative filtering (CF) is a successful approach commonly used by many recommender systems. Conventional CF-based methods use the ratings given to items by users as the sole source of information for learning to make recommendation. However, the ratings are often very sparse in many applications, causing CF-based methods to degrade significantly in their recommendation performance. To address this sparsity problem, auxiliary information such as item content information may be utilized. Collaborative topic regression (CTR) is an appealing recent method taking this approach which tightly couples the two components that learn from two different sources of information. Nevertheless, the latent representation learned by CTR may not be very effective when the auxiliary information is very sparse. To address this problem, we generalize recently advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. Extensive experiments on three real-world datasets from different domains show that CDL can significantly advance the state of the art.

Related Works

CDL Variants (Direct Extensions of CDL)

• Deep Collaborative Filtering via Marginalized Denoising Auto-encoder
  by Li et al., CIKM 2015.
• Collaborative Deep Ranking: A Hybrid Pair-Wise Recommendation Algorithm with Implicit Feedback
  by Ying et al., PAKDD 2016.
• Hybrid Recommender System based on Autoencoders
  by Strub et al., RecSys DLRS Workshop 2016.
• Collaborative Knowledge Base Embedding for Recommender Systems
  by Zhang et al., KDD 2016.
• Ask the GRU: Multi-Task Learning for Deep Text Recommendations
  by Bansal et al., RecSys 2016.
• Convolutional Matrix Factorization for Document Context-Aware Recommendation
  by Kim et al., RecSys 2016.
• Collaborative Denoising Auto-Encoders for Top-N Recommender Systems
  by Wu et al., WSDM 2016.
• Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks
  by Wang et al., NIPS 2016.
• Joint Deep Modeling of Users and Items Using Reviews for Recommendation
  by Zheng et al., WSDM 2017.
• Collaborative Metric Learning
  by Hsieh et al., WWW 2017.
• A Hybrid Collaborative Filtering Model with Deep Structure for Recommender Systems
  by Dong et al., AAAI 2017.
• Neural Rating Regression with Abstractive Tips Generation for Recommendation
  by Li et al., SIGIR 2017.
• AutoSVD++: An Efficient Hybrid Collaborative Filtering Model via Contractive Auto-encoders
  by Zhang et al., SIGIR 2017.
• Collaborative Variational Autoencoder for Recommender Systems
  by Li et al., KDD 2017.
• Spatial-Aware Hierarchical Collaborative Deep Learning for POI Recommendation
  by Yin et al., TKDE 2017.
• DropoutNet: Addressing Cold Start in Recommender Systems
  by Volkovs et al., NIPS 2017.
• Unified Personalized Video Recommendation via Dynamic Recurrent Neural Networks
  by Gao et al., ACM MM 2017.
• Visually-Aware Fashion Recommendation and Design with Generative Image Models
  by Kang et al., ICDM 2017.
• IDAE: Imputation-boosted Denoising Autoencoder for Collaborative Filtering
  by Lee et al., CIKM 2017.
• Augmented Variational Autoencoders for Collaborative Filtering with Auxiliary Information
  by Lee et al., CIKM 2017.
• Collaborative Topic Regression with Denoising AutoEncoder for Content and Community Co-Representation
  by Nguyen et al., CIKM 2017.
• Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources
  by Zhang et al., CIKM 2017.
• Discrete Deep Learning for Fast Content-Aware Recommendation
  by Zhang et al., WSDM 2018.
• Neural Personalized Ranking for Image Recommendation
  by Niu et al., WSDM 2018.
• OpenRec: A Modular Framework for Extensible and Adaptable Recommendation Algorithms
  by Yang et al., WSDM 2018.
• NAIS: Neural Attentive Item Similarity Model for Recommendation
  by He et al., TKDE 2018.
• Unified Personalized Video Recommendation via Dynamic Recurrent Neural Networks
  by Gao et al., ACM MM 2017.
• Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding
  by Tang et al., WSDM 2018.
• Neural Attentional Rating Regression with Review-level Explanations
  by Chen et al., WWW 2018.
• CoupledCF: Learning Explicit and Implicit User-item Couplings in Recommendation for Deep Collaborative Filtering
  by Zhang et al., IJCAI 2018.
• Aspect-Level Deep Collaborative Filtering via Heterogeneous Information Networks
  by Han et al., IJCAI 2018.
• Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System
  by Tang et al., KDD 2018.
• Review Sentiment-Guided Scalable Deep Recommender System
  by Hyun et al., SIGIR 2018.
• Attention-driven Factor Model for Explainable Personalized Recommendation
  by Chen et al., SIGIR 2018.
• Probabilistic Collaborative Representation Learning for Personalized Item Recommendation
  by Salah et al., UAI 2018.
• Adaptive Collaborative Topic Modeling for Online Recommendation
  by Al-Ghossein et al., RecSys 2018.
• Deep Heterogeneous Autoencoders for Collaborative Filtering
  by Li et al., ICDM 2018.
• A General Cross-Domain Recommendation Framework via Bayesian Neural Network
  by He et al., ICDM 2018.
• Partial VAE for Hybrid Recommender System
  by Ma et al., BDL@NIPS 2018.
• Deep Heterogeneous Autoencoders for Collaborative Filtering
  by Li et al., ICDM 2018.
• A Knowledge-Enhanced Deep Recommendation Framework Incorporating GAN-Based Models
  by Yang et al., ICDM 2018.
• Heterogeneous Neural Attentive Factorization Machine for Rating Prediction
  by Chen et al., CIKM 2018.
• Word-Driven and Context-Aware Review Modeling for Recommendation
  by Wang et al., CIKM 2018.
• Content-Based Video Relevance Prediction with Second-Order Relevance and Attention Modeling
  by Chen et al., ACM MM 2018.
• Deep User Modeling for Content-based Event Recommendation in Event-based Social Networks
  by Wang et al., INFOCOM 2018.
• CHAMELEON: a deep learning meta-architecture for news recommender systems
  by de Souza et al., RecSys 2018.
• Neural gaussian mixture model for review-based rating prediction
  by Deng et al., RecSys 2018.
• Recommendation Through Mixtures of Heterogeneous Item Relationships
  by Kang et al., CIKM 2018.
• Neural Relational Topic Models for Scientific Article Analysis
  by Bai et al., CIKM 2018.
• Bayesian Deep Collaborative Matrix Factorization
  by Xiao et al., AAAI 2019.
• Linked Variational AutoEncoders for Inferring Substitutable and Supplementary Items
  by Rakesh et al., WSDM 2019.
• Gated Attentive-Autoencoder for Content-Aware Recommendation
  by Ma et al., WSDM 2019.
• Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation
  by Wang et al., WWW 2019.
• NPA: Neural News Recommendation with Personalized Attention
  by Wu et al., KDD 2019.
• Learning Disentangled Representations for Recommendation
  by Ma et al., NIPS 2019.
• Hierarchical Gating Networks for Sequential Recommendation
  by Ma et al., KDD 2019.
• A Context-Aware User-Item Representation Learning for Item Recommendation
  by Wu et al., TIST 2019.
• STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems
  by Zhang et al., IJCAI 2019.
• DAML: Dual Attention Mutual Learning between Ratings and Reviews for Item Recommendation
  by Liu et al., KDD 2019.
• Explainable Recommendation through Attentive Multi-View Learning
  by Gao et al., AAAI 2019.
• Deeply Fusing Reviews and Contents for Cold Start Users in Cross-Domain Recommendation Systems
  by Fu et al., AAAI 2019.
• Sequential Variational Autoencoders for Collaborative Filtering
  by Sachdeva et al., WSDM 2019.
• Transfer Meets Hybrid: A Synthetic Approach for Cross-Domain Collaborative Filtering with Text
  by Hu et al., WWW 2019.
• Collaborative Similarity Embedding for Recommender Systems
  by Chen et al., WWW 2019.
• CB2CF: A Neural Multiview Content-to-Collaborative Filtering Model for Completely Cold Item Recommendations
  by Barkan et al., RecSys 2019.
• Multimodal Review Generation for Recommender Systems
  by Truong et al., WWW 2019.
• Beyond Personalization: Social Content Recommendation for Creator Equality and Consumer Satisfaction
  by Xiao et al., KDD 2019.
• Improving Top-K Recommendation via Joint Collaborative Autoencoders
  by Zhu et al., WWW 2019.
• On Both Cold-Start and Long-Tail Recommendation with Social Data
  by Li et al., TKDE 2019.
• Deep Collaborative Filtering with Multi-Aspect Information in Heterogeneous Networks
  by Shi et al., TKDE 2019.
• Bayesian Deep Collaborative Matrix Factorization
  by Xiao et al., AAAI 2019.
• Enhancing VAEs for Collaborative Filtering: Flexible Priors & Gating Mechanisms
  by Kim et al., RecSys 2019.
• Attribute-aware Non-linear Co-embeddings of Graph Features
  by Rashed et al., RecSys 2019.
• Representation Interpretation with Spatial Encoding and Multimodal Analytics
  by Liu et al., WSDM 2019.
• Adversarial Training for Review-Based Recommendations
  by Rafailidis et al., SIGIR 2019.
• Linked Variational AutoEncoders for Inferring Substitutable and Supplementary Items
  by Rakesh et al., WSDM 2019.
• Preliminary Investigation of Alleviating User Cold-Start Problem in E-commerce with Deep Cross-Domain Recommender System
  by Wang et al., WWW 2019.
• Recommender System Using Sequential and Global Preference via Attention Mechanism and Topic Modeling
  by Kang et al., CIKM 2019.
• Dataset Recommendation via Variational Graph Autoencoder
  by Altaf et al., ICDM 2019.
• A Generative Model for Review-based Recommendations
  by Shalom et al., RecSys 2019.
• Pcard: Personalized Restaurants Recommendation from Card Payment Transaction Records
  by Du et al., WWW 2019.
• CAMO: A Collaborative Ranking Method for Content Based Recommendation
  by Wang et al., AAAI 2019.
• Dual Neural Personalized Ranking
  by Kim et al., WWW 2019.
• Variational Session-based Recommendation Using Normalizing Flows
  by Zhou et al, WWW 2019.
• Assessing Personally Perceived Image Quality via Image Features and Collaborative Filtering
  by Korhonen, CVPR 2019.
• Constructing High Precision Knowledge Bases with Subjective and Factual Attributes
  by Kobren et al., KDD 2019.
• Using Structured Knowledge and Traditional Word Embeddings to Generate Concept Representations in the Educational Domain
  by Anuyah et al., WWW 2019.
• DDTCDR: Deep Dual Transfer Cross Domain Recommendation
  by Li et al., WSDM 2020.
• Learning a Joint Search and Recommendation Model from User-Item Interactions
  by Zamani et al., WSDM 2020.
• A Knowledge-Aware Attentional Reasoning Network for Recommendation
  by Zhu et al., AAAI 2020.
• News Recommendation with Topic-Enriched Knowledge Graphs
  by Lee et al., CIKM 2020.
• Probabilistic Metric Learning with Adaptive Margin for Top-K Recommendation
  by Ma et al., KDD 2020.
• AR-CF: Augmenting Virtual Users and Items in Collaborative Filtering for Addressing Cold-Start Problems
  by Chae et al., SIGIR 2020.
• paper2repo: GitHub Repository Recommendation for Academic Papers
  by Shao et al., WWW 2020.
• Cornac: A Comparative Framework for Multimodal Recommender Systems
  by Salah et al., JMLR 2020.
• Personalized Employee Training Course Recommendation with Career Development Awareness
  by Wang et al., WWW 2020.
• SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback
  by Wang et al., AAAI 2020.
• Context-Dependent Propagating-Based Video Recommendation in Multimodal Heterogeneous Information Networks
  by Sang et al., TMM 2020.
• DRprofiling: Deep Reinforcement User Profiling for Recommendations in Heterogenous Information Networks
  by Liang, TKDE 2020.
• Attribute Graph Neural Networks for Strict Cold Start Recommendation
  by Qian et al., TKDE 2020.
• TAFA: Two-headed Attention Fused Autoencoder for Context-Aware Recommendations
  by Zhou et al., RecSys 2020.
• Leveraging Title-Abstract Attentive Semantics for Paper Recommendation
  by Guo et al., AAAI 2020.
• Learning Latent Multi-Criteria Ratings from User Reviews for Recommendations
  by Li et al., TKDE 2020.
• TemPEST: Soft Template-Based Personalized EDM Subject Generation through Collaborative Summarization
  by Chen et al., AAAI 2020.
• Multi-Scale Generalized Attention-Based Regional Maximum Activation of Convolutions for Beauty Product Retrieval
  by Xu et al., ACM MM 2020.
• Correcting Exposure Bias for Link Recommendation
  by Wang et al., ICML 2021.
• Bilateral Variational Autoencoder for Collaborative Filtering
  by Truong et al., WSDM 2021.
• A Cooperative Memory Network for Personalized Task-oriented Dialogue Systems with Incomplete User Profiles
  by Pei et al., WWW 2021.
• Task-adaptive Neural Process for User Cold-Start Recommendation
  by Lin et al., WWW 2021.
• Variable Interval Time Sequence Modeling for Career Trajectory Prediction: Deep Collaborative Perspective
  by Wang et al., WWW 2021.
• Dual Metric Learning for Effective and Efficient Cross-Domain Recommendations
  by Li et al., TKDE 2021.
• Local Collaborative Autoencoders
  by Choi et al., WSDM 2021.
• Privileged Graph Distillation for Cold Start Recommendation
  by Wang et al., SIGIR 2021.
• A Semi-Personalized System for User Cold Start Recommendation on Music Streaming Apps
  by Briand et al., KDD 2021.
• Cold Start Revisited: A Deep Hybrid Recommender with Cold-Warm Item Harmonization
  by Barkan et al., ICASSP 2021.
• Local Factor Models for Large-Scale Inductive Recommendation
  by Yang et al., RecSys 2021.
• Learning Hierarchical Review Graph Representations for Recommendation
  by Liu et al., TKDE 2021.
• Shared Neural Item Representations for Completely Cold Start Problem
  by Raziperchikolaei et al., RecSys 2021.
• Dual-Embedding based Deep Latent Factor Models for Recommendation
  by Cheng et al., TKDD 2021.
• Linear, or Non-Linear, That is the Question!
  by Kong et al., WSDM 2022.
• Multi-Modal Recommender Systems: Hands-On Exploration
  by Truong et al., RecSys 2021.
• Hybrid Variational Autoencoder for Recommender Systems
  by Zhang et al., TKDD 2022.
• Multi-modal Discrete Collaborative Filtering for Efficient Cold-start Recommendation
  by Xu et al., TKDE 2022.
• Variational Bandwidth Auto-encoder for Hybrid Recommender Systems
  by Zhu et al., TKDE 2022.
• Itinerary-aware Personalized Deep Matching at Fliggy
  by Xu et al., WWW 2021.
• Neural Representations in Hybrid Recommender Systems: Prediction versus Regularization
  by Raziperchikolaei et al., SIGIR 2021.
• Learning Audio Embeddings with User Listening Data for Content-Based Music Recommendation
  by Chen et al., ICASSP 2021.
• Variational Bayesian Graph Convolutional Network for Robust Collaborative Filtering
  by Onodera et al., ICASSP 2022.
• PNMTA: A Pretrained Network Modulation and Task Adaptation Approach for User Cold-Start Recommendation
  by Pang et al., WWW 2022.
• Mutually-Regularized Dual Collaborative Variational Auto-encoder for Recommendation Systems
  by Zhu et al., WWW 2022.

Earlier Related Works and Other Follow-up Works

• Restricted Boltzmann Machines for Collaborative Filtering
  by Salakhutdinov et al., ICML 2007.
• A non-IID Framework for Collaborative Filtering with Restricted Boltzmann Machines
  by Georgiev and Nakov, ICML 2013.
• Deep Content-based Music Recommendation
  by van den Oord et al., NIPS 2013.
• Improving Content-based and Hybrid Music Recommendation using Deep Learning
  by Wang and Wang, ACM MM 2014.
• Deep Coevolutionary Network: Embedding User and Item Features for Recommendation
  by Dai et al., RecSys DLRS Workshop 2016.
• Improved Recurrent Neural Networks for Session-based Recommendations
  by Tan et al., RecSys DLRS Workshop 2016.
• Wide & Deep Learning for Recommender Systems
  by Cheng et al., RecSys DLRS Workshop 2016.
• Triply Stochastic Variational Inference for Non-linear Beta Process Factor Analysis
  by Fan et al., ICDM 2016.
• Session-based Recommendations with Recurrent Neural Networks
  by Hidasi et al., ICLR 2016.
• Expert Finding for Community-Based Question Answering via Ranking Metric Network Learning
  by Zhao et al., IJCAI 2016.
• Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations
  by Hidasi et al., RecSys 2016.
• Deep Neural Networks for YouTube Recommendations
  by Covington et al., RecSys 2016.
• Multi-Rate Deep Learning for Temporal Recommendation
  by Song et al., SIGIR 2016.
• Neural Collaborative Filtering
  by He et al., WWW 2017.
• A Neural Collaborative Filtering Model with Interaction-based Neighborhood
  by Bai et al., CIKM 2017.
• Micro Behaviors: A New Perspective in E-commerce Recommender Systems
  by Zhou et al., WSDM 2018.
• LSTM Networks for Online Cross-Network Recommendations
  by Perera et al., IJCAI 2018.
• Buy It Again: Modeling Repeat Purchase Recommendations
  by Bhagat et al., KDD 2018.
• Unbiased Offline Recommender Evaluation for Missing-not-at-random Implicit Feedback
  by Yang et al., RecSys 2018.
• MTNet: A Neural Approach for Cross-Domain Recommendation with Unstructured Text
  by Hu et al., KDD 2018.
• A Reinforcement Learning Framework for Explainable Recommendation
  by Wang et al., ICDM 2018.
• CFGAN: A Generic Collaborative Filtering Framework based on Generative Adversarial Networks
  by Chae et al., CIKM 2018.
• PARL: Let Strangers Speak Out What You Like
  by Wu et al., CIKM 2018.
• Point-of-Interest Recommendation: Exploiting Self-Attentive Autoencoders with Neighbor-Aware Influence
  by Ma et al., CIKM 2018.
• Modularizing deep neural network-inspired recommendation algorithms
  by Yang et al., RecSys 2018.
• ANR: Aspect-based Neural Recommender
  by Chin et al., CIKM 2018.
• Self-Attentive Sequential Recommendation
  by Kang et al., ICDM 2018.
• Neural Collaborative Ranking
  by Song et al., CIKM 2018.
• Enhancing the Robustness of Neural Collaborative Filtering Systems Under Malicious Attacks
  by Du et al., TMM 2019.
• Sequential Variational Autoencoders for Collaborative Filtering
  by Sachdeva et al., WSDM 2019.
• Neural Tensor Factorization for Temporal Interaction Learning
  by Wu et al., WSDM 2019.
• Neural Graph Collaborative Filtering
  by Wang et al., SIGIR 2019.
• Graph Neural Networks for Social Recommendation
  by Fan et al., WWW 2019.
• BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
  by Sun et al., CIKM 2019.
• A Neural Influence Diffusion Model for Social Recommendation
  by Wu et al., SIGIR 2019.
• Spatiotemporal Representation Learning for Translation-Based POI Recommendation
  by Qian et al., TIST 2019.
• A Collaborative Session-based Recommendation Approach with Parallel Memory Modules
  by Wang et al., SIGIR 2019.
• Towards Knowledge-Based Recommender Dialog System
  by Chen et al., EMNLP 2019.
• A Capsule Network for Recommendation and Explaining What You Like and Dislike
  by Li et al., SIGIR 2019.
• DeepGBM: A Deep Learning Framework Distilled by GBDT for Online Prediction Tasks
  by Ke et al., KDD 2019.
• Joint Neural Collaborative Filtering for Recommender Systems
  by Chen et al., TIST 2019.
• Neural Tensor Factorization for Temporal Interaction Learning
  by Wu et al., WSDM 2019.
• Deep Social Collaborative Filtering
  by Fan et al., RecSys 2019.
• Multi-Level Deep Cascade Trees for Conversion Rate Prediction in Recommendation System
  by Wen et al., AAAI 2019.
• Value-aware Recommendation based on Reinforcement Profit Maximization
  by Pei et al., WWW 2019.
• Rating Augmentation with Generative Adversarial Networks towards Accurate Collaborative Filtering
  by Chae et al., WWW 2019.
• BPAM: Recommendation Based on BP Neural Network with Attention Mechanism
  by Xi et al., AAAI 2019.
• To Index or Not to Index: Optimizing Exact Maximum Inner Product Search
  by Abuzaid et al., ICDE 2019.
• Collaborative Distillation for Top-N Recommendation
  by Lee et al., ICDM 2019.
• Unified Collaborative Filtering over Graph Embeddings
  by Wang et al., SIGIR 2019.
• Discrete Matrix Factorization and Extension for Fast Item Recommendation
  by Lian et al., TKDE 2019.
• Deep Learning Driven Venue Recommender for Event-Based Social Networks
  by Pramanik et al., TKDE 2019.
• TourSense: A Framework for Tourist Identification and Analytics Using Transport Data
  by Lu et al., TKDE 2019.
• Neural Attention Frameworks for Explainable Recommendation
  by Tal et al., TKDE 2019.
• Hybrid Item-Item Recommendation via Semi-Parametric Embedding
  by Hu et al., IJCAI 2019.
• Leveraging Ratings and Reviews with Gating Mechanism for Recommendation
  by Xia et al., CIKM 2019.
• Sublinear Time Nearest Neighbor Search over Generalized Weighted Space
  by Lei et al., ICML 2019.
• User-Aware Folk Popularity Rank: User-Popularity-Based Tag Recommendation That Can Enhance Social Popularity
  by Wang et al., ACM MM 2019.
• A Spatio-temporal Recommender System for On-demand Cinemas
  by Xue et al., CIKM 2019.
• Fine-Grained Air Quality Inference with Remote Sensing Data and Ubiquitous Urban Data
  by Xu et al., TKDD 2019.
• Disentangled Self-Supervision in Sequential Recommenders
  by Ma et al., KDD 2020.
• Citywide Traffic Flow Prediction Based on Multiple Gated Spatio-temporal Convolutional Neural Networks
  by Chen et al., TKDD 2020.
• Neural Interactive Collaborative Filtering
  by Zou et al., SIGIR 2020.
• How Dataset Characteristics Affect the Robustness of Collaborative Recommendation Models
  by Deldjoo et al., SIGIR 2020.
• Personalized Ranking with Importance Sampling
  by Lian et al., WWW 2020.
• LightRec: A Memory and Search-Efficient Recommender System
  by Lian et al., WWW 2020.
• MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection
  by Luo et al., WWW 2020.
• Adversarial Machine Learning in Recommender Systems (AML-RecSys)
  by Deldjoo et al., WSDM 2020.
• A Survey on Representation Learning for User Modeling
  by Li et al., IJCAI 2020.
• Privacy-Preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation
  by Wang et al., TSP 2020.
• Neural Attentive Multiview Machines
  by Barkan et al., ICASSP 2020.
• Beyond User Embedding Matrix: Learning to Hash for Modeling Large-Scale Users in Recommendation
  by Shi et al., SIGIR 2020.
• Learning Feature Interactions with Lorentzian Factorization Machine
  by Xu et al., AAAI 2020.
• Multiplex Memory Network for Collaborative Filtering
  by Jiang et al., SDM 2020.
• E-commerce Recommendation with Weighted Expected Utility
  by Xu et al., CIKM 2020.
• MindReader: Recommendation over Knowledge Graph Entities with Explicit User Ratings
  by Brams et al., CIKM 2020.
• Product Quantized Collaborative Filtering
  by Lian et al., TKDE 2020.
• Cold Item Recommendations via Hierarchical Item2vec
  by Barkan et al. ICDM 2020.
• MultiRec: A Multi-Relational Approach for Unique Item Recommendation in Auction Systems
  by Rashed et al., RecSys 2020.
• Addressing the Target Customer Distortion Problem in Recommender Systems
  by Zhao et al., WWW 2020.
• Understanding and Exploring the Network with Stochastic Architectures
  by Deng et al., NIPS 2020.
• Modeling Dynamic Missingness of Implicit Feedback for Sequential Recommendation
  by Zheng et al., TKDE 2020.
• Deep Pairwise Hashing for Cold-start Recommendation
  by Zhang et al., TDKE 2020.
• A Deep Neural Network for Crossing-City POI Recommendations
  by Li et al., TDKE 2020.
• J-Recs: Principled and Scalable Recommendation Justification
  by Park et al., ICDM 2020.
• Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches
  by Zhang et al., VLDB 2021.
• Neural Collaborative Reasoning
  by Chen et al., WWW 2021.
• Leave No User Behind: Towards Improving the Utility of Recommender Systems for Non-mainstream Users
  by Li et al., WSDM 2021.
• Deep Transfer Tensor Decomposition with Orthogonal Constraint for Recommender Systems
  by Chen et al., AAAI 2021.
• Intention-aware Sequential Recommendation with Structured Intent Transition
  by Li et al., TKDE 2021.
• Scalable and Explainable 1-Bit Matrix Completion via Graph Signal Learning
  by Chen et al., AAAI 2021.
• Learning to Continuously Optimize Wireless Resource in Episodically Dynamic Environment
  by Sun et al., ICASSP 2021.
• ClickTrain: Efficient and Accurate End-to-end Deep Learning Training via Fine-grained Architecture-preserving Pruning
  by Zhang et al., ICS 2021.
• A Novel Memory-efficient Deep Learning Training Framework via Error-bounded Lossy Compression
  by Jin et al., PPoPP 2021.
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective
  by Sun et al., TSP 2021.
• SamWalker++: Recommendation with Informative Sampling Strategy
  by Wang et al., TKDE 2021.
• Reverse Attack: Black-box Attacks on Collaborative Recommendation
  by Zhang et al., CCS 2021.
• Learning User Representations with Hypercuboids for Recommender Systems
  by Zhang et al., WSDM 2021.
• Improving Sales Forecasting Accuracy: A Tensor Factorization Approach with Demand Awareness
  by Bi et al., INFORMS Journal on Computing 2022.

Please do not hesitate to inform us of any missing works.

Quantitative Results

Below is the recall@M for different methods including our CDL. M is the number recommended items.

Here is the mean average precision (mAP) with the cut-off point set to 500.

Qualitative Results

Interpretability and Accuracy

Below are two case studies where CDL and CTR recommend articles to User I and User II.

Sensitivity to Users' Interest Change and Accuracy

Below is a third case study where an example user (User III) 'watched' 10 movies one by one. During the process, CDL and CTR recommend movies to the user.

Movie 1 and Movie 2: The first 2 movies User III watched are 'Moonstruck' and 'True Romance', both of which are romance movies. Naturally, both CDL and CTR recommend some related romance movie to the user.

Movie 3 and Movie 4: The next 2 movies User III watched are 'Johnny English' and 'American Beauty'. These are drama and action movies, which are very different from the previous two. Note that this indicates a change of user interest. CDL sensitive enought to capture this change and recommend both romance and drama/action movies to User III, boosting the accuracy from 30% to 50%. However, CTR fails to do so and the accuracy remains 20%.

Other Movies: After User III watched and liked 10 movies, the accuracy of CDL's and CTR's recommendations is 50% and 90%, respectively.

Brief Model Introduction

In CDL, a probabilistic stacked denoising autoencoder (pSDAE) is connected to a regularized probabilistic matrix factorization (PMF) component to form a unified probabilistic graphical model.

Model training will alternate between pSDAE and regularized PMF. In each epoch, a pSDAE with a reconstruction target at the end and a regression target in the bottleneck will be udpated before updating the latent factors U and V in the regularized PMF.

Below is the graphical model for CDL. The part in the red rectangle is pSDAE and the rest is the PMF component regularized by pSDAE. Essentially, the updating will alternate between pSDAE (updating W and b) and the regularized PMF component (updating U and V).

For more details on the notation, please refer to this page or the paper.

Code and Data

The code is available on Github or here, and the data is also available.

You can also check out our MXNet code for a simplied version of CDL on Github or here.

We also made a python notebook as part of the KDD 2016 MXNet tutorial, which is available on Github.

Other implementations (third-party):
Tensorflow code by gtshs2.
Another Tensorflow code by preferred.ai.
Keras code by zoujun123.
Python code by xiaoouzhang.

Also Check Out

• Collaborative recurrent autoencoder: recommend while learning to fill in the blanks.
  Hao Wang, Xingjian Shi, Dit-Yan Yeung.
  Thirtieth Annual Conference on Neural Information Processing Systems (NIPS), 2016.
  [pdf] [supplementary] [spotlight video] [code and data]
  
  
• Relational stacked denoising autoencoder for tag recommendation.
  Hao Wang, Xingjian Shi, Dit-Yan Yeung.
  Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), 2015.
  [pdf] [supplementary] [code] [data]
  
  
• Towards Bayesian deep learning: a framework and some existing methods.
  Hao Wang, Dit-Yan Yeung.
  IEEE Transactions on Knowledge and Data Engineering (TKDE), 28(12):3395-3408, 2016.
  [pdf]
  
  
• Relational deep learning: A deep latent variable model for link prediction.
  Hao Wang, Xingjian Shi, Dit-Yan Yeung.
  Thirty-First AAAI Conference on Artificial Intelligence (AAAI), 2017.
  [pdf] [supplementary] [code] [data]
  
  
• Natural parameter networks: a class of probabilistic neural networks.
  Hao Wang, Xingjian Shi, Dit-Yan Yeung.
  Thirtieth Annual Conference on Neural Information Processing Systems (NIPS), 2016.
  [pdf] [supplementary] [spotlight video] [code and data]
  
  

Acknowledgements

This research has been partially supported by research grant FSGRF14EG36.