Resources

Survey

• [arxiv.org][Retrieval] A Comprehensive Survey on Retrieval Methods in Recommender Systems

• [ijcai.org][UBM] A Survey on User Behavior Modeling in Recommender Systems

• [arxiv.org][CTR] Deep Learning for Click-Through Rate Estimation

• [arxiv.org][SSL] Self-Supervised Learning for Recommender Systems: A Survey

• [arxiv.org][Embedding] Embedding in Recommender Systems: A Survey

• [arxiv.org][CTR] Click-Through Rate Prediction in Online Advertising: A Literature Review

• [le-wu.com][Ranking] A Survey on Accuracy-Oriented Neural Recommendation

• [mdpi.com] A Comprehensive Survey of Recommender Systems Based on Deep Learning

• [youtube.com] TUTORIAL: Neural Contextual Bandits for Personalized Recommendation (arxiv)

• [youtube.com] TUTORIAL: Privacy in Web Advertising: Analytics and Modeling

• [youtube.com] TUTORIAL: Multimodal Pretraining and Generation for Recommendation

• [youtube.com] TUTORIAL: Large Language Models for Recommendation Progresses and Future Directions

Metrics & QA

Important

• [evidentlyai.com] 10 metrics to evaluate recommender and ranking systems

• [docs.evidentlyai.com] Ranking metrics

• [arize.com] A Quick Survey of Drift Metrics

• [github.com] 50 Fundamental Recommendation Systems Interview Questions

• [devinterview.io] 50 Recommendation Systems interview questions

Videos

• [youtube.com] Stanford CS224W Machine Learning w/ Graphs I 2023 I GNNs for Recommender Systems

Note

• Mapped as an edge prediction problem in a bipartite graph

• Ranking

  • Metric Recall@k (non differentiable)

  • Other metrics HR@k, nDCG

  • Differentiable Discriminative loss - binary loss (similar to cross entropy), Bayesian prediction loss (BPR)

  • Issue with binary, BPR solves the ranking problem better

  • Trick to choose neg samples

  • Not suitable for ANN

• Collaborative filtering

  • DNN to capture user item similarity with cosine or InfoNCE loss

  • ANN friendly

  • Doesn’t consider longer than 1 hop in the bipartite graph

• GCN

  • Smoothens the embeddings by GCN layer interactions using undirected edges to enforce similar user and similar item signals

  • Neural GCN or LightGCN

  • Application similar image recommendation in Pinterest

  • Issue doesn’t have contextual awareness or session/temporal awareness

Course, Books & Papers

CTR Prediction Papers

• [paperswithcode.com] CTR Prediction

Technique	Resource
LR	Distributed training of Large-scale Logistic models
Embed + MLP	Deep Neural Networks for YouTube Recommendations
Embed + MLP	Real-time Personalization using Embeddings for Search Ranking at Airbnb
Wide & Deep	Wide & Deep Learning for Recommender Systems
DeepFM	DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
xDeepFM	xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
DCN	Deep & Cross Network for Ad Click Predictions
DCNv2	DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
DIN	Deep Interest Network for Click-Through Rate Prediction
BST	Behavior Sequence Transformer for E-commerce Recommendation in Alibaba

Embeddings Papers

Technique	Resource
Hash	Hash Embeddings for EfficientWord Representations
Deep Hash	Learning to Embed Categorical Features without Embedding Tables for Recommendation
Survey	Embedding in Recommender Systems: A Survey

Modeling Methods Papers

• BOF = Bag of features

• NG = N-Gram

• CM = Causal Models (autoregressive)

Tag	Title
QU;Search	Better search through query understanding
IR;QU;Search	Using Query Contexts in Information Retrieval
IR;Course;Stanford	CS 276 / LING 286 Information Retrieval and Web Search
IR;Book	Introduction to Information Retrieval
Retrival;RS	Simple but Efficient A Multi-Scenario Nearline Retrieval Framework for Recommendation on Taobao
Retrival;Ranking;Embed+MLP	Neural Collaborative Filtering
Retrival;Two Tower;BOF	StarSpace Embed All The Things!
Retrival;Ranking;Two Tower;NG+BOF	Embedding-based Retrieval in Facebook Search
Ranking;L2R	DeepRank: Learning to rank with neural networks for recommendation
GCN	Graph Convolutional Neural Networks for Web-Scale Recommender Systems
GCN	LightGCN - Simplifying and Powering Graph Convolution Network for Recommendation
CM;Session	Transformers4Rec Bridging the Gap between NLP and Sequential / Session-Based Recommendation
Diversity;DPP	Improving the Diversity of Top-N Recommendation via Determinantal Point Process
Diversity;DPP	Practical Diversified Recommendations on YouTube with Determinantal Point Processes
Diversity;DPP	Personalized Re-ranking for Improving Diversity in Live Recommender Systems
Diversity;DPP	Fast Greedy MAP Inference for Determinantal Point Process to Improve Recommendation Diversity
Diversity;Multi-Stage	Representation Online Matters Practical End-to-End Diversification in Search and Recommender Systems
Polularity Bias	Managing Popularity Bias in Recommender Systems with Personalized Re-Ranking
Polularity Bias	User-centered Evaluation of Popularity Bias in Recommender Systems
Polularity Bias	Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System
Fairness	Fairness in Ranking Part II Learning-to-Rank and Recommender Systems
Fairness	Fairness Definitions Explained
LLM	A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
LLM	Collaborative Large Language Model for Recommender Systems
LLM	Recommendation as Instruction Following A Large Language Model Empowered Recommendation Approach

More Papers

Year	Title
2001	Item-Based Collaborative Filtering Recommendation Algorithms – Sarwar et al.
2003	Amazon.com Recommendations Item-to-Item Collaborative Filtering – Linden et al.
2007	Link Prediction Approaches and Applications – Liben-Nowell et al.
2008	An Introduction to Information Retrieval – Manning et al.
2009	BM25 and Beyond – Robertson et al.
2009	Matrix Factorization Techniques for Recommender Systems – Koren et al.
2010	Who to Follow Recommending People in Social Networks – Twitter Research
2014	DeepWalk Online Learning of Social Representations – Perozzi et al.
2015	Learning Deep Representations for Content-Based Recommendation – Wang et al.
2015	Netflix Recommendations Beyond the 5 Stars – Gomez-Uribe et al.
2016	Deep Neural Networks for YouTube Recommendations – Covington et al.
2016	Wide & Deep Learning for Recommender Systems – Cheng et al.
2016	Session-Based Recommendations with Recurrent Neural Networks – Hidasi et al.
2017	DeepRank A New Deep Architecture for Relevance Ranking in Information Retrieval – Pang et al.
2017	Neural Collaborative Filtering – He et al.
2017	A Guide to Neural Collaborative Filtering – He et al.
2018	BERT Pre-training of Deep Bidirectional Transformers for Language Understanding – Devlin et al.
2018	PinSage Graph Convolutional Neural Networks for Web-Scale Recommender Systems – Ying et al.
2018	Neural Architecture for Session-Based Recommendations – Tang & Wang
2018	SASRec Self-Attentive Sequential Recommendation – Kang & McAuley
2018	Graph Convolutional Neural Networks for Web-Scale Recommender Systems – Ying et al.
2019	Deep Learning Based Recommender System A Survey and New Perspectives – Zhang et al.
2019	Session-Based Recommendation with Graph Neural Networks – Wu et al.
2019	Next Item Recommendation with Self-Attention – Sun et al.
2019	BERT4Rec Sequential Recommendation with Bidirectional Encoder Representations – Sun et al.
2020	Dense Passage Retrieval for Open-Domain Question Answering – Karpukhin et al.
2020	ColBERT Efficient and Effective Passage Search via Contextualized Late Interaction Over BERT – Khattab et al.
2020	T5 for Information Retrieval – Nogueira et al.
2021	CLIP Learning Transferable Visual Models from Natural Language Supervision – Radford et al.
2021	Transformers4Rec Bridging the Gap Between NLP and Sequential Recommendation – De Souza et al.
2021	Graph Neural Networks A Review of Methods and Applications – Wu et al.
2021	Next-Item Prediction Using Pretrained Language Models – Sun et al.
2022	Unified Vision-Language Pretraining for E-Commerce Recommendations – Wang et al.
2022	Contextual Item Recommendation with Pretrained LLMs – Li et al.
2023	InstructGPT for Information Retrieval – Ouyang et al.
2023	GPT-4 for Web Search Augmentation – Bender et al.
2023	CLIP-Recommend Multimodal Learning for E-Commerce Recommendations – Xu et al.
2023	Semantic-Aware Item Matching with Large Language Models – Chen et al.
2023	GPT4Rec A Generative Framework for Personalized Recommendation – Wang et al.
2023	LLM-based Collaborative Filtering Enhancing Recommendations with Large Language Models – Liu et al.
2023	LLM-Powered Dynamic Personalized Recommendations – Guo et al.
2023	Real-Time Recommendation with Large Language Models – Zhang et al.
2023	Graph Neural Networks Meet Large Language Models A Survey – Wu et al.
2023	LLM-powered Social Graph Completion for Friend Recommendations – Huang et al.
2023	LLM-Augmented Node Classification in Social Networks – Zhang et al.