官术网_书友最值得收藏!

Support vector

We draw two boundary lines passing through feature vectors of one class closest to the feature vectors of another class. The center line of these boundary lines is the hyperplane we have been talking about. For example, for Class B, a boundary line is passing through p and q along the way and another boundary line through r and s because p and q are the closest to the feature vectors of Class B and so are r and s. These are called support vectors. We will understand now why these are called support vectors:

Let's say that if we try to remove one of the feature vectors that is not so close to the boundary line, we will not have an impact on the position or orientation of the hyperplane because the hyperplane's position is decided by boundary lines crossing through vectors p, q, r, and s. And, since these are the points holding (supporting) the hyperplane together, they have been named support vectors.

So, this equation yi (w.xi + b) = 1 holds true when xi is pqr, or s.

We will go back to the equation w.x + b/ ||w|| ≥ γ; here, we are trying to maximize γ, and in order to do so either we need to maximize b or minimize ||w||.

Or we can say we have to minimize w.w. If we convert that into a function, Φ(w) = w.w has to be minimized. Φ(w) =1/2( w.w) (here 1/2 has been added for mathematical convenience).

So, the objective function of SVM becomes Φ(w) =1/2( w.w), which has to be minimized subject to constraints, as follows:

yi (w.xi + b) = 1

Since it is a constrained optimization problem, it can be converted into an unconstrained optimization problem using the Lagrangian multiplier.

Hence, L(w,b)= 1/2(w.w) - ∑ αi [yi(w.xi+b) - 1] where αi is the Lagrangian multiplier, L(w,b)= 1/2(w.w) - ∑ αi yi (w.xi) -∑ αi yi b + ∑ αi.

Let's find out w and b by using maxima and minima calculus:

δL/δb = 0 

It results in ∑ αi yi=0, δL/δw = 0 would result in ∑ αi yi xi = w. Now, putting these results back into the Lagrangian function yields the following:

L= ∑ αi - 1/2 ∑ αi αj yi yj (xj.xi)

It means that if the value of αi is very high then the corresponding x. There will be a lot of influence on the position of the hyperplane. Hence, for classification and for unknown feature vector z, the required equation would be the following:

D(z) = Sign( ∑ αi xi yi z + b)

If D(z) >0, then z would belong to class A and if D(z)<0, z ∈ class B. Let's try to perform a case study in Python:

主站蜘蛛池模板: 苏尼特左旗| 江门市| 浦江县| 滨州市| 长乐市| 湖州市| 张北县| 左贡县| 车险| 嘉峪关市| 广水市| 鄂尔多斯市| 鞍山市| 乳山市| 碌曲县| 玛纳斯县| 宜阳县| 忻城县| 鄯善县| 宜兴市| 富顺县| 潞西市| 磴口县| 嘉黎县| 铁岭县| 都安| 南江县| 永寿县| 武功县| 兴文县| 泸州市| 青阳县| 麻城市| 宁都县| 衢州市| 佛学| 略阳县| 台前县| 读书| 巧家县| 惠安县|