Machine Learning Tech

• Language Understanding and Language Models
  • Activations
  • Normalisation
  • Tokenizers
    • WordPiece
    • SentencePiece
    • Byte-Pair Encoding (BPE)
  • Word Embeddings
  • Sequence Modeling
    • RNN
    • LSTM
  • Transformer
    • General Resources
    • Position Encoding
    • Attention
    • Decoding
  • Transformer Architecture
    • Encoder [BERT]
    • Decoder [GPT]
    • Encoder-Decoder [T5]
    • Autoencoder [BART]
    • Cross-Lingual
• Large Language Models
  • Training
    • Engineering
    • Objectives
    • Long Context LLMs
    • Special Techniques
    • Resources
  • Applied LLMs
    • Prompt Engineering
    • Embeddings for Retrieval
    • Retrieval Augmented Generation (RAG)
    • Notes: Modeling
    • Notes: Index Choice
    • Notes: RAG vs Long Context
    • Notes: LLM and KG
  • Known Issues
    • Hallucination
    • Sycophany
    • Monosemanticity
• Natural Language Processing
  • Notes from Common NLP Tasks
    • Using In-Context Capability
    • NER
    • Disambiguation
    • Entity Linking
    • Relation Extraction
    • Link Prediction
    • Graph Completion
    • Question Answering
    • GraphRAG
    • MultiModal
    • Other NLP Tasks
  • Information Retrieval (IR)
    • 0. Language Models in IR
    • 1. Document Retrieval
    • 2. Passage Retrieval
    • 3. Query Expansion
    • 4. Question Answering (QA)
  • Information Extraction (IE) Tasks
    • 0. Language Models in IE
    • 1. Named Entity Recognition (NER)
    • 2. Relation Extraction
    • 3. Event Extraction
    • 4. Coreference Resolution
  • Classification Tasks
    • 1. Sentiment Analysis
  • Sequence to Sequence Tasks
    • 1. Machine Translation
    • 2. Text Summarization
    • 3. Text Generation
  • Multimodal Tasks
    • 1. Text-to-Speech (TTS)
    • 2. Speech Recognition
  • Extending Vocab for Domain-Adaptation or Fine-Tuning
    • 1. Extend the Tokenizer Vocabulary
    • 2. Tinkering with the Embedding Matrix
    • Notes
• Vision Models
  • Resources
  • Industry Resources
• Graph Networks
  • Resources
• Multimodal
  • Diffusion
  • Tech
  • Resources
    • Industry
    • Misc
    • Paper List