As we know Support Vector Machine is a supervised machine learning algorithm which can be used for both classification or regression challenges. And mostly it is used in classification problems. In Support Vector Machine, we simply perform classification by finding the hyper-plane that differentiate the two classes very well.
Support Vector Machine Working
Now how we will identify the right hyper-plane?
Scenario 1 :
From these three hyper plane A,B & C we need to identify the right hyper plane to classify cross and circle. In this scenario, hyper plane B has excellently performed this job.
Thumb-Rule: Select the hyper plane which segregates the two classes better.
Here maximizing the distance between nearest data point (either class) and hyper-plane will help us to decide the right hyper – plane . This distance is called as margin.
As we can see margin of hyper – plane B is high compared to both A & C. Hence we select hyper-plane B as the correct hyper-plane in this case. Another reason of selecting the hyper plane with higher margin is robustness. If we select a hyper plane having low margin then there is high chance of misclassification.
Some individuals would select the hyper plane B as it has higher margin compared to A. But here is the catch, Support Vector Machine selects the hyper-plane which classifies the classes accurately prior to maximizing margin. Here hyper plane B has a classification error & A has classified all correctly. So the right hyper plane is A here.
Scenario 4: Below we are unable to segregate the two classes using a straight line as one of the star lies in the territory of other (circle) class as an outlier.
As said earlier one star at other end is like an outlier for star class in this scenario of support machine working. Support Vector Machine has a feature to ignore outliers & find hyper-plane that has maximum margin in support vector machine working. Hence we can say Support Vector Machine is robust to outliers.
In Support Vector Machine working, it is easy to have a linear hyper plane between two classes.But another question arise is , should we need to add this feature manually to have a hyper-plane. No, SVM has a technique called the kernal trick.
Implementation of Support Vector Machine using R
e1071 package in R is used to create SVM with ease. It has helper functions as well as code for the Naive Bayes classifier.
Pros of Support Vector Machine
- It works really well with clear margin of seperation
- It is effective in high dimensional spaces.
Cons of Support Vector Machine
- It doesn’t perform well, when we have large data set because the required training time is higher.
- It doesn’t perform well when the data set has more noise i.e. target classes are overlapping.