Sure Pass Exam AIP-210 PDF

Overfitting is a common negative side effect of not using regularization. Regularization is a technique that reduces the complexity of a model by adding a penalty term to the loss function, which prevents the model from learning too many parameters that may fit the noise in the training data. Overfitting occurs when the model performs well on the training data but poorly on the test data or new data, because it has memorized the training data and cannot generalize well. References: Regularization (mathematics) - Wikipedia, Overfitting in Machine Learning: What It Is and How to Prevent It

Lasso regression is a type of linear regression that adds a regularization term to the loss function to reduce overfitting and improve generalization. Lasso regression uses an L1 norm as the regularization term, which is the sum of the absolute values of the coefficients. Lasso regression can shrink some of the coefficients to zero, which effectively eliminates some of the features from the model. Lasso regression is preferable over ridge regression when there are many features with no association with the dependent variable, as it can perform feature selection and reduce the complexity and noise of the model.

Normalization is a method that can help us to solve the problem of features being on different scales when using the k-nearest neighbors algorithm. Normalization is a technique that rescales the values of features to a common range, such as [0, 1] or [-1, 1]. Normalization can help reduce the influence or dominance of some features over others, as well as improve the accuracy and performance of the algorithm2.

Ridge regression is a type of regularization technique that can help reduce collinearity among independent variables. It does this by adding a penalty term to the ordinary least squares (OLS) objective function, which shrinks the coefficients of highly correlated variables towards zero. This reduces the variance of the coefficient estimates and improves the stability and accuracy of the regression model. References: Multicollinearity in Regression Analysis: Problems, Detection, and Solutions - Statistics By Jim, A Beginner’s Guide to Collinearity: What it is and How it affects our regression model - StrataScratch