官术网_书友最值得收藏!

Support vector

We draw two boundary lines passing through feature vectors of one class closest to the feature vectors of another class. The center line of these boundary lines is the hyperplane we have been talking about. For example, for Class B, a boundary line is passing through p and q along the way and another boundary line through r and s because p and q are the closest to the feature vectors of Class B and so are r and s. These are called support vectors. We will understand now why these are called support vectors:

Let's say that if we try to remove one of the feature vectors that is not so close to the boundary line, we will not have an impact on the position or orientation of the hyperplane because the hyperplane's position is decided by boundary lines crossing through vectors p, q, r, and s. And, since these are the points holding (supporting) the hyperplane together, they have been named support vectors.

So, this equation yi (w.xi + b) = 1 holds true when xi is pqr, or s.

We will go back to the equation w.x + b/ ||w|| ≥ γ; here, we are trying to maximize γ, and in order to do so either we need to maximize b or minimize ||w||.

Or we can say we have to minimize w.w. If we convert that into a function, Φ(w) = w.w has to be minimized. Φ(w) =1/2( w.w) (here 1/2 has been added for mathematical convenience).

So, the objective function of SVM becomes Φ(w) =1/2( w.w), which has to be minimized subject to constraints, as follows:

yi (w.xi + b) = 1

Since it is a constrained optimization problem, it can be converted into an unconstrained optimization problem using the Lagrangian multiplier.

Hence, L(w,b)= 1/2(w.w) - ∑ αi [yi(w.xi+b) - 1] where αi is the Lagrangian multiplier, L(w,b)= 1/2(w.w) - ∑ αi yi (w.xi) -∑ αi yi b + ∑ αi.

Let's find out w and b by using maxima and minima calculus:

δL/δb = 0 

It results in ∑ αi yi=0, δL/δw = 0 would result in ∑ αi yi xi = w. Now, putting these results back into the Lagrangian function yields the following:

L= ∑ αi - 1/2 ∑ αi αj yi yj (xj.xi)

It means that if the value of αi is very high then the corresponding x. There will be a lot of influence on the position of the hyperplane. Hence, for classification and for unknown feature vector z, the required equation would be the following:

D(z) = Sign( ∑ αi xi yi z + b)

If D(z) >0, then z would belong to class A and if D(z)<0, z ∈ class B. Let's try to perform a case study in Python:

主站蜘蛛池模板: 明星| 永福县| 十堰市| 牡丹江市| 上饶市| 长葛市| 东海县| 新昌县| 临朐县| 磐石市| 鄂托克前旗| 杭州市| 克拉玛依市| 遂昌县| 建德市| 呈贡县| 文成县| 玉树县| 肥西县| 金昌市| 芜湖市| 盘山县| 曲周县| 三门峡市| 朝阳市| 长汀县| 泽普县| 铁岭市| 郁南县| 白河县| 繁昌县| 缙云县| 浪卡子县| 阳城县| 五常市| 清水河县| 麻阳| 龙泉市| 玉环县| 怀仁县| 南华县|