############################################################### Canonical Micro-problems ############################################################### .. contents:: Table of Contents :depth: 2 :local: :backlinks: none *************************************************************** 1. Natural Language Processing *************************************************************** Extractive QA =============================================================== - We want to build a customer support tool where agents can paste in a long policy document and ask a question - the system should highlight the exact sentence or phrase in the document that answers it - BERT + span head (start/end token classification) - SQuAD-style span labels - answer not present in context, requires "no answer" head Open-domain QA =============================================================== - Build an internal knowledge base assistant for a company with thousands of wiki pages. An employee asks a natural language question and expects a direct answer, not a list of links. - DPR retriever + BERT reader (RAG) - Natural Questions, TriviaQA - retrieval recall is the bottleneck; reader is useless if retriever misses Abstractive QA / Summarization =============================================================== - Our legal team receives hundreds of contracts per day. Build a system that, given a contract and a question like 'what are the termination conditions?', returns a concise natural language answer even when the answer is spread across multiple clauses. - BART, T5 seq2seq - abstractive QA pairs, CNN/DM - hallucination, faithfulness to source Semantic textual similarity =============================================================== - We have a support ticket system with 10M historical tickets. When a new ticket comes in, surface the 5 most similar resolved tickets so the agent can see how they were handled. - Siamese BERT (sentence-transformers), contrastive loss - STS-B, NLI pairs as soft supervision - domain shift; cosine similarity doesn't respect asymmetry (entailment is directional) Natural language inference =============================================================== - Given a product claim made in an ad (e.g. 'clinically proven to reduce wrinkles in 7 days') and a scientific paper, determine whether the paper supports, contradicts, or is neutral about the claim. - BERT + 3-way classification head - SNLI, MultiNLI - annotation artifacts, hypothesis-only shortcuts Named entity recognition =============================================================== - We're ingesting millions of news articles per day. Extract all mentioned companies, people, and locations so they can be linked to structured entities in our knowledge graph - BERT + token classification + CRF - CoNLL-2003 BIO tags - nested entities, boundary ambiguity, cross-sentence coreference Relation extraction =============================================================== - Given a news article, automatically populate a structured event database: who acquired whom, at what price, on what date - BERT with entity markers + classification head - TACRED, DocRED - implicit relations not surfaced in a single sentence Coreference resolution =============================================================== - In a long earnings call transcript, every time 'they', 'the company', 'its CEO' is mentioned, resolve it to the correct named entity so downstream extraction is accurate. - SpanBERT + span scoring - OntoNotes - long-document coreference, pronoun ambiguity Intent classification =============================================================== - We're building a voice assistant for a food delivery app. Given a user utterance like 'I want to change my order', route it to the correct backend handler. - fine-tuned encoder + softmax head - Dialog datasets (ATIS, SNIPS) - out-of-scope intents, intent overlap Slot filling =============================================================== - In the same voice assistant, after classifying the intent as 'book a restaurant', extract the structured fields: restaurant name, date, time, party size. - BERT + BIO token classifier (joint with intent) - ATIS - unseen slot values, multi-value slots Machine translation =============================================================== - Our marketplace operates in 30 countries. Automatically translate seller-generated product listings into the buyer's language with minimal human review. - Transformer seq2seq - parallel corpora (WMT) - low-resource languages, rare token translation Text generation / controlled generation =============================================================== - Given a product's structured spec sheet (dimensions, material, price), generate a compelling marketing description in the brand's tone of voice - GPT-style decoder, CTRL - LM pretraining + RLHF - repetition, degeneration, constraint satisfaction *************************************************************** 2. Information Retrieval *************************************************************** Sparse retrieval =============================================================== - Build a search bar for an internal HR portal. Employees type queries like 'parental leave policy UK' and expect the most relevant policy document to surface at the top. - BM25 - TF-IDF statistics, no training - vocabulary mismatch, acronyms, synonyms Dense retrieval =============================================================== - Our e-commerce site has 50M products. A user types 'something cozy to wear on a winter hike' - match them to relevant products even when none of those words appear in the product title. - Bi-encoder (DPR, sentence-BERT) - in-batch negatives on MS-MARCO / NQ - rare term recall; hard negatives needed for quality Learned sparse retrieval =============================================================== - We need retrieval that handles both exact keyword matching (SKU numbers, product codes) and semantic matching (synonyms, paraphrases) in the same system. - SPLADE - distillation from cross-encoder - slower than BM25, complex training Hybrid retrieval =============================================================== - Build a search system for a legal research platform where users sometimes search by citation ('Section 12(b) of the Securities Exchange Act') and sometimes by concept ('insider trading safe harbor'). - BM25 + dense, score fusion (RRF or learned) - combined supervision - interpolation weight is domain-sensitive Cross-encoder reranking =============================================================== - Our first-stage retrieval returns 100 candidates. Build a second-stage model that reorders them to maximize the chance the top result is the one the user clicks. - BERT cross-encoder, MonoT5 - pointwise or pairwise MS-MARCO labels - latency; can only run on top-k from first stage Query understanding / expansion =============================================================== - Users on our medical platform often use lay terms like 'heart attack' instead of clinical terms like 'myocardial infarction'. Improve retrieval by reformulating queries before they hit the index. - T5 for query rewriting, docT5query for document expansion - weak supervision from click logs - expansion can introduce noise Learned index / ANN =============================================================== - We have 1B product embeddings that need to be searched in under 20ms at query time. Design the indexing and retrieval infrastructure. - FAISS (IVF, HNSW), ScaNN - embedding quality upstream - recall-latency tradeoff; HNSW strong on recall, IVF better on memory *************************************************************** 3. Computer Vision *************************************************************** Image classification =============================================================== - We run an online marketplace for secondhand goods. When a seller uploads a photo, automatically categorize the item into our product taxonomy so it appears in the right browse category. - ResNet / ViT + softmax - ImageNet labels - long tail, fine-grained distinctions Object detection =============================================================== - We're building a retail shelf monitoring system. Given a photo of a store shelf, identify which products are present, their positions, and whether any slots are empty. - DETR, Faster R-CNN - COCO bounding box labels - small object detection, dense scenes Semantic segmentation =============================================================== - We're building an autonomous vehicle perception system. Given a dashcam frame, label every pixel as road, pedestrian, vehicle, or obstacle. - SegFormer, Mask2Former - pixel-level labels (Cityscapes, ADE20K) - label cost is extreme; boundary ambiguity Instance segmentation =============================================================== - For a fashion try-on app, given a photo of a person, precisely cut out individual clothing items so they can be replaced with items from our catalog. - Mask R-CNN, SAM - COCO instance masks - occlusion, overlapping instances Optical character recognition (OCR) =============================================================== - We process millions of scanned invoices from vendors. Extract the invoice number, line items, and total amount so they can be ingested into our accounting system - CNN encoder + CTC / attention decoder - synthetic + real text pairs - arbitrary fonts, low resolution, curved text Document layout analysis =============================================================== - We're digitizing a large archive of academic papers. Given a scanned PDF, identify which regions are body text, which are figures, which are tables, and which are captions. - LayoutLM, DiT - PubLayNet, FUNSD - mixed layout (tables inside prose), multi-column Image captioning =============================================================== - We're building an accessibility feature for a social platform. Automatically generate alt text for user-uploaded images so screen readers can describe them. - CNN encoder + transformer decoder (BLIP) - COCO captions - hallucination of objects not in image Visual QA =============================================================== - Build a feature for a home improvement app where a user can photograph a room and ask 'what style is this?' or 'does this furniture match?' - ViLBERT, BLIP-2, LLaVA - VQA v2, GQA - language bias (models answer from prior without looking) Image-text retrieval / grounding =============================================================== - A user types 'minimalist kitchen with marble countertops' into our interior design platform - return the most visually matching photos from our catalog. - CLIP contrastive - (image, alt-text) web pairs - loose pairing; spatial/compositional reasoning fails Phrase grounding / referring expression =============================================================== - In a warehouse robotics system, a human operator says 'pick up the small red box on the left side of the shelf' - the robot must identify the exact object being referred to. - MDETR, Grounding DINO - RefCOCO - complex relational expressions ("the leftmost red cup") Face verification =============================================================== - Build a re-authentication system for a banking app: when a user attempts a high-value transaction, verify that the live selfie matches the ID photo on file. - ArcFace / CosFace (margin-based softmax) - labeled identity pairs - demographic bias, pose/lighting variation Anomaly detection (visual) =============================================================== - We run a manufacturing line producing circuit boards. Flag any board coming off the line that has a visual defect, without having labeled examples of every defect type. - PatchCore, normalizing flows - only normal images (no anomaly labels needed) - defining "normal" distribution; novel anomaly types *************************************************************** 4. Multimodal *************************************************************** Image-text alignment (document) =============================================================== - We're building a search engine over scientific papers. When a user queries 'transformer attention visualization', return not just relevant papers but the specific figures within them that show attention maps, with their associated captions. - CLIP + layout proximity score - web-scale (image, alt-text) + structural heuristics - loose web pairs; ebook/PDF layout requires spatial features Video-text retrieval =============================================================== - Our video platform has 100M videos with auto-generated transcripts. A user types 'how to change a bike tire' - surface the most relevant videos and start playback at the relevant segment. - VideoClip, CLIP4Clip - HowTo100M, MSR-VTT - temporal aggregation; which frames matter Audio-text (speech recognition) =============================================================== - We run a call center with 10,000 agents. Transcribe every call in real time so that compliance violations can be flagged and agent coaching can be automated. - Whisper (encoder-decoder) - weakly supervised web audio + transcripts - accented speech, overlapping speakers Speech + NLP (spoken QA) =============================================================== - Build a voice interface for a car that can answer 'what's the nearest EV charging station with a 150kW charger?' from spoken input alone, without a screen. - cascaded ASR → QA or end-to-end - SpokenSQuAD - ASR error propagation into downstream Multimodal sentiment =============================================================== - We're building a brand monitoring tool that analyzes video reviews on social media. Given a video, determine overall sentiment accounting for tone of voice, facial expression, and spoken words. - Late fusion or cross-attention over text + audio + video - CMU-MOSI, CMU-MOSEI - modality alignment, missing modality at inference *************************************************************** 5. Graphs & Structured Data *************************************************************** Node classification =============================================================== - We have a transaction graph where nodes are accounts and edges are transfers. Classify each account as legitimate, suspicious, or fraudulent. - GraphSAGE, GCN + softmax - node labels (Cora, citation graphs) - over-smoothing at depth; transductive only Link prediction =============================================================== - On a professional networking platform, recommend people a user might know, based on the structure of their mutual connections and shared group memberships. - GNN encoder + dot-product decoder - positive/negative edge sampling - negative sampling strategy dominates performance Graph classification =============================================================== - Given the molecular graph of a drug candidate, predict whether it will be toxic so we can filter out bad candidates early in the pipeline. - GIN, hierarchical pooling - TUD benchmarks - scalability to large graphs Knowledge graph completion =============================================================== - Our knowledge graph has entities for companies, people, and products but many relationships are missing. Infer likely missing relationships (e.g. 'person X is likely the CEO of company Y') from existing graph structure. - TransE, RotatE, ComplEx - (head, relation, tail) triples - multi-relational reasoning, unseen entities Tabular classification =============================================================== - Given a user's historical transaction features, predict whether the next transaction is fraudulent before it is authorized. - XGBoost / LightGBM (tree ensemble), TabNet (attention) - structured labels - feature leakage, distribution shift between train/serve Anomaly detection (tabular) =============================================================== - We run a cloud infrastructure platform. Given per-machine metrics (CPU, memory, disk I/O, network), flag machines that are behaving anomalously compared to their historical baseline. - Isolation Forest, autoencoders, COPOD - unlabeled normal data - defining anomaly in high-dimensional spaces *************************************************************** 6. Recommender Systems *************************************************************** Collaborative filtering =============================================================== - On a streaming music platform, surface songs a user is likely to enjoy that they haven't heard before, based purely on listening history patterns across all users. - Matrix factorization (ALS), Neural CF - implicit feedback (clicks, purchases) - cold start, popularity bias Two-tower retrieval =============================================================== - We have 500M items in our catalog. Given a user's interaction history, retrieve the top 500 candidate items in under 50ms to feed into a downstream ranker. - Dual encoder, learned embeddings - in-batch negatives from interaction logs - false negatives in batch; popularity collapse Session-based recommendation =============================================================== - A user lands on our e-commerce site and browses 4 items in the first 3 minutes. With no account or history, recommend the next item they're most likely to engage with. - GRU4Rec, SASRec (transformer) - next-item prediction from session - short sessions, position bias Multi-task ranking =============================================================== - We want to rank the home feed to jointly maximize short-term engagement (clicks, likes) and long-term satisfaction (survey scores, return visits) without sacrificing one entirely for the other. - MMOE, PLE - multi-objective labels (click, dwell, share) - task conflict, negative transfer Bandit / exploration =============================================================== - We're launching a new content format. We want to learn which users respond well to it as fast as possible, while not degrading overall engagement during the exploration period. - LinUCB, Thompson Sampling, contextual bandits - online reward signals - exploration cost in production, delayed rewards *************************************************************** 7. Sequence & Time Series *************************************************************** Time series forecasting =============================================================== - We operate a ride-sharing platform. Forecast demand at the city-grid level for the next 4 hours so we can pre-position drivers and offer surge pricing signals. - Temporal Fusion Transformer, N-BEATS, PatchTST - univariate/multivariate targets - distribution shift, irregular sampling Anomaly detection (time series) =============================================================== - We monitor payment processing infrastructure. Alert on-call engineers when transaction throughput, error rates, or latency deviate from expected patterns, with low false positive rate. - LSTM autoencoder, SARIMA residuals, Informer - reconstruction error on normal windows - threshold sensitivity, seasonality confusion Event sequence modeling =============================================================== - Given a user's sequence of in-app actions (viewed product → added to cart → abandoned → returned next day), predict the probability they will convert in the next 24 hours. - Temporal point processes (Neural Hawkes), Transformer on event logs - timestamped event sequences - irregular intervals, long-horizon dependencies