Review Topics

• constitutional ai

• guardrails

• https://github.com/microsoft/unilm

• eval for ie tasks - open vs supervised

• llm evals: https://github.com/openai/evals

• multimodal ie

• multimodal: text + image

  • classification: - clip: https://github.com/openai/CLIP

    Learning Transferable Visual Models From Natural Language Supervision

    • cnn

  • generation

    • dall-e: https://github.com/openai/dall-e

      Zero-Shot Text-to-Image Generation

    • latent-diffusion: https://github.com/CompVis/latent-diffusion

      • High-Resolution Image Synthesis with Latent Diffusion Models

      • Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models

    • stable diffusion: https://github.com/CompVis/stable-diffusion

      • Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

    • vision transformers and diffusion models

  • eval

• cnn: - image classification - object detection (bounding box):

  https://paperswithcode.com/task/object-detection YOLOv4: Optimal Speed and Accuracy of Object Detection

  • image segmentation:

    • GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer

• recsys - context based (in session rec - llm), interaction based (collaborative filtering - mf, gcn)

• nlp downstream tasks

• hardware p40, v100, a100 - arch, cost

• training: domain adaptation (mlm/rtd/ssl-kl/clm), finetuning (sft/it), alignment and preference optim (rhlf/dpo)

• domain understanding

• design e2e: integrate user feedback