Semantic Understanding, Quality & Integrity Systems¶

Table of Contents

Product Understanding¶

Goal:
- Generate high-quality item profiles
- Automatic categorization and taxonomy classification
- Enrich metadata (category, brand, attributes, etc.) for search, ads, and recommendations
- Product catalog matching or enrichment
- Power auto-suggestions, filters, and query understanding
Tasks
- Multimodal product classification
- Attribute extraction (title -> category/brand/specs)
- Description generation (captioning)
Data Sources:
- User-uploaded listings on Marketplace (text, image, video)
- Crawled product pages (for domain transfer or contrastive learning)
- External e-commerce catalogs (structured taxonomy)
- Golden set of curated listings (human or LLM curated)
Models & Techniques:
- Multimodal encoders:
  - Vision: ResNet, ViT
  - Language: XLM-R, mT5, LLaMA
  - Multimodal: CLIP, BLIP, Flamingo
- Multi-modal fusion: Late fusion or co-attention models
- Contrastive pretraining (image-text pairs)
- Product clustering based on semantic embeddings

Goal: Automatically generate ad creatives to promote listings.
Techniques:
- NLG: Description and tag generation from images/metadata (Fine-tune T5/mT5 for text generation)
- Use BLIP or diffusion models for image enhancement or captioning
- Visual Highlight Extraction: Object detection or saliency-based summarization
- Multi-modal ad copy generation using BLIP-2 or LLaMA variants
- Retrieval-augmented generation (e.g., “Find similar items with good text/images and copy structure”)
Inputs:
- Item metadata
- Product image or video
- Reference creatives from similar listings

Goal:
- Score each listing for visual/textual quality and completeness.
- Feed into reranking, filtering, or even seller coaching
Features:
- Image quality (blur, resolution, brightness), presence of key views
- Text quality (length, grammar, informativeness)
- Engagement features (CTR, time spent)
- Metadata completeness
- Seller reputation/activity
Models:
- Multimodal deep scoring model. Train using human-labeled quality scores or engagement proxy labels
- Lightweight MLPs or GBDTs (LightGBM). Joint vision-language encoders with quality classification heads

Goal: Detect policy-violating, fraudulent, or misleading listings.
Use-cases:
- Spam/fraud detection (duplicate listings, fake pricing, keyword abuse)
- Policy violation detection (prohibited items, explicit content)
- Outlier detection on price by category/location
Data Sources:
- Flagged listings
- Policy template examples
- Human moderation data
Models:
- Classifiers on image + text (BERT + ViT) using weak supervision
- Graph-based anomaly detection (e.g., same phone used across accounts)
- Use pretrained vision-language models fine-tuned on policy rules
- Siamese or contrastive networks for duplicate/fake listing detection

Goal: Score sellers based on activity, responsiveness, listing quality, and user interactions.
Use: Input to ranking model, fraud detection, seller badges
Features: Response rate, listing quality, community rating, past violations.
ML: Seller scoring model that feeds into ranking pipeline.

Use-cases: Recommend a price based on similar listings or detect outlier pricing.
Models: Regression over embeddings + location + metadata.

Goal: Detect price manipulation
Approaches:
- Regression models using category, item embeddings, location
- Price range outlier detection