官术网_书友最值得收藏!

Support vector

We draw two boundary lines passing through feature vectors of one class closest to the feature vectors of another class. The center line of these boundary lines is the hyperplane we have been talking about. For example, for Class B, a boundary line is passing through p and q along the way and another boundary line through r and s because p and q are the closest to the feature vectors of Class B and so are r and s. These are called support vectors. We will understand now why these are called support vectors:

Let's say that if we try to remove one of the feature vectors that is not so close to the boundary line, we will not have an impact on the position or orientation of the hyperplane because the hyperplane's position is decided by boundary lines crossing through vectors p, q, r, and s. And, since these are the points holding (supporting) the hyperplane together, they have been named support vectors.

So, this equation yi (w.xi + b) = 1 holds true when xi is pqr, or s.

We will go back to the equation w.x + b/ ||w|| ≥ γ; here, we are trying to maximize γ, and in order to do so either we need to maximize b or minimize ||w||.

Or we can say we have to minimize w.w. If we convert that into a function, Φ(w) = w.w has to be minimized. Φ(w) =1/2( w.w) (here 1/2 has been added for mathematical convenience).

So, the objective function of SVM becomes Φ(w) =1/2( w.w), which has to be minimized subject to constraints, as follows:

yi (w.xi + b) = 1

Since it is a constrained optimization problem, it can be converted into an unconstrained optimization problem using the Lagrangian multiplier.

Hence, L(w,b)= 1/2(w.w) - ∑ αi [yi(w.xi+b) - 1] where αi is the Lagrangian multiplier, L(w,b)= 1/2(w.w) - ∑ αi yi (w.xi) -∑ αi yi b + ∑ αi.

Let's find out w and b by using maxima and minima calculus:

δL/δb = 0 

It results in ∑ αi yi=0, δL/δw = 0 would result in ∑ αi yi xi = w. Now, putting these results back into the Lagrangian function yields the following:

L= ∑ αi - 1/2 ∑ αi αj yi yj (xj.xi)

It means that if the value of αi is very high then the corresponding x. There will be a lot of influence on the position of the hyperplane. Hence, for classification and for unknown feature vector z, the required equation would be the following:

D(z) = Sign( ∑ αi xi yi z + b)

If D(z) >0, then z would belong to class A and if D(z)<0, z ∈ class B. Let's try to perform a case study in Python:

主站蜘蛛池模板: 湖南省| 铜川市| 长子县| 兴城市| 台州市| 四子王旗| 太谷县| 新河县| 洛川县| 郴州市| 岱山县| 汝南县| 丰台区| 资中县| 万盛区| 宝山区| 旅游| 祁门县| 林芝县| 金华市| 酒泉市| 兖州市| 杨浦区| 武城县| 宣汉县| 南丹县| 沂南县| 永城市| 汉源县| 凯里市| 太白县| 龙胜| 稷山县| 木里| 呼伦贝尔市| 津市市| 昭觉县| 太白县| 仪陇县| 寿光市| 宁都县|