Logistic Regression 2 – Cost Function, Gradient Descent and Other Optimization Algorithms

We have discussed the basic ideas of logistic regression in previous post.  The purpose of logistic regression is to find the optimal decision boundary which can classify the data with different categorical target feature into different classes.  We also introduced the logistic function or sigmoid function as the regression model to find the optimal decision… Read More »

Logistic Regression 1 – Classification, Logistic Regression and Sigmoid Function

In previous series of posts we discussed simple and multivariate linear regression that can be used to predict target features with continuous values.  Besides that, there are other prediction problems with categorical target features and we want to train a model so that we can use it to predict the class of unknown data.  Logistic… Read More »

Linear Regression 5 – Other Practical Issues

In the previous articles we have discussed the basic concept of simple linear regression; how to measure the error of the regression model so that we can use the gradient descent method to find the global optimum of the regression problem; develop the multivariate linear regression model for real world problems; and how to choose… Read More »

Linear Regression 4 – Learning Rate and Initial Weights

Choosing Learning Rate We introduced an important parameter, the learning rate α, in Linear Regression 2 – Gradient Descent without discussing how to choose its value.  In fact, the choice of the learning rate affects the performance of the algorithm significantly.  It determines the convergence speed of the gradient descent algorithm, which is the number… Read More »