Support Vector Machine, Decision Boundary

Support Vector Machine: Linearly Separable case

This is an example of Large Margin Classifier, the gap between A and B is called the margin. This case is called the Large Margin because the margin is large. In this case, the SVM will learn line C.

Recall that the cost function for SVM was:

\[C⅀ycost1(theta.T@X) + (1-y)cost0(theta.T@X)\] 

Where the summation runs over all the examples.

We are going to focus on the parameter C:

A large C will make sure that it classifies perfectly, will make sure that each example has a huge weight when deciding the decision boundary. In case this can be useful, but in most case, you don't want it. At least when the training set is very very large.