ML Breadth¶
Table of Contents
Resources¶
Misc Resources
[github.com] Machine-Learning-Interview-Prep
Specific Topics
[youtube.com] ML interviews
[blog.paperspace.com] Intro to Optimization in Deep Learning: Busting the Myth About Batch Normalization
[medium.com] A Visual Explanation of Gradient Descent Methods (Momentum, AdaGrad, RMSProp, Adam)
Interview Guide
Revision Topics¶
Fundamentals¶
Math
Matrix Factorizations - Eigendecomposition, SVD, QR, Cholesky, PD/PSD
Matrix Calculus, Vector Calculus, Multivariate Chain Rule, Backprop derivation
Taylor’s Expansion, First and second order Approximation
Convex Optimization: GD, Newton’s Mehtod, Gauss-Newton’s Approximation
Optimizers: SGD, AdaGrad, RMSProp, Adam - Learning Rate, Schedule, Choice of parameters, Effect of Regularisation
Constrained Optimization: Lagrange Multipliers, KKT Conditions
Mercer Kernels, Reproducing Property
Stat
Expectations, Higher Moments, Skew, Kurtoisis, Conditional, Marginal, Joint, Bayes Theorem
Variance Covariance Matrix, Scatter Matrix, Correlation Matrix, Eigendecomposition
Distributions and moments - Bernoulli, Binomial, Categorical, Multinomial, Normal, Poisson, Exponential, Logistic
Frequentis Estimation Theory - Point Estimation, Confidence Interval, Hypothesis Testing
Point Estimation, Bayes Estimator, Bias Variance Decomposition
Minmax Theory, Empirical Risk Minimization
Bayesian Estimaton Theory, Conjugate Priors - Normal, Beta-Binomial, Dirichlet-Multinomial
Sampling Techniques: CDF-Jacobian, Monte Carlo Estimators, Metropolis Hasting, Adaptive Metropolis Hasting, Ancestor Sampling
Learning Theory
KL divergence, Entropy, MaxEnt, Cross-Entropy, NLL
Graphical Models - BN, MRF, CRF
Variational Inference, Belief Propagation, Deep Belief Net
Regression and Classification
Bayes Estimator - Estimator for conditional mean (regression) or conditional mode (classification)
Basis Expansion, Gram matrix, Feature Maps
Linear Regression, Logistic Regression, Naive Bayes, SVM
Tree Based Methods: DT, Bagging, Boosting, XGBoost
Clustering
Distance Based: K-Means, Density Based: DBSCAN
Hierarchical Clustering, Self-Organizing Maps
Metrics - Distance Based (Silhoutte coefficient), DB index, CH index
Manifold Learning
t-SNE
Spectral Clustering
Latent Variable Models
GMM, PCA, Kernel-PCA, ICA, CCA
NMF, LDA
Outlier prediction
Isolation Forest
One-Class SVM
Density Estimation
Linear/Quadratic Discriminator Analysis
Kernel Density Estimator
Practical
Feature Engineering
Reinforcement learning
SARSA
Explore-exploit, bandits (eps-greedy, UCB, Thompson sampling),
Q-learning, DQN
Learning To Rank
Predicts a relative-order
Metrics: MAP, Precision@k, Recall@k, DCG@k/NDCG@k, MRR)
Common Approaches: Pairwise
Study Framework¶
Problem
Problem description and assumptions for simplicity.
Approach and Assumptions
Theoretical framework & motivation.
Mathematical derivation of training objective (loss) with boundary conditions.
What-if scenarios where training fails - mathematical issues (check stack-exchange).
Training and Validation
Design the training algorithm
Implementation and computational considerations including complexity.
Convergence checks.
What-if scenarios where training fails - computational issues (check stack-exchange).
Testing and Model Selection
Overfitting/underfitting checks and remedies.
Metrics to check - different choices and trade-offs.
Hyperparameter tuning and model selection.
Inference
Computational considerations.
Model degradation monitoring and remedies.
Sample Questions¶
Sample questions moved to the question bank in docs/source/gen/interviews/qb.rst.
Statistics¶
Question prompts moved to the question bank in docs/source/gen/interviews/qb.rst.
Classical ML¶
Question prompts moved to the question bank in docs/source/gen/interviews/qb.rst.
Applied ML¶
Question prompts moved to the question bank in docs/source/gen/interviews/qb.rst.
Mixture¶
Question prompts moved to the question bank in docs/source/gen/interviews/qb.rst.