Distribution Shift

Definitions

Note

• Distribution shift: \(p_{\text{train}}(\mathbf{x},y)\neq p_{\text{test}}(\mathbf{x},y)\)

• Covariate shift:

  • \(p_{\text{train}}(\mathbf{x})\neq p_{\text{test}}(\mathbf{x})\)

  • \(p_{\text{train}}(y|\mathbf{x})=p_{\text{test}}(y|\mathbf{x})\)

• Concept shift:

  • \(p_{\text{train}}(\mathbf{x})=p_{\text{test}}(\mathbf{x})\)

  • \(p_{\text{train}}(y|\mathbf{x})\neq p_{\text{test}}(y|\mathbf{x})\)

• Label shift:

  • Only in \(y\implies\mathbf{x}\) problems.

  • \(p_{\text{train}}(y)\neq p_{\text{test}}(y)\)

  • \(p_{\text{train}}(\mathbf{x}|y)=p_{\text{test}}(\mathbf{x}|y)\)