- Deep Learning with Keras
- Antonio Gulli Sujit Pal
- 267字
- 2021-07-02 23:58:02
Problems in training the perceptron and a solution
Let's consider a single neuron; what are the best choices for the weight w and the bias b? Ideally, we would like to provide a set of training examples and let the computer adjust the weight and the bias in such a way that the errors produced in the output are minimized. In order to make this a bit more concrete, let's suppose we have a set of images of cats and another separate set of images not containing cats. For the sake of simplicity, assume that each neuron looks at a single input pixel value. While the computer processes these images, we would like our neuron to adjust its weights and bias so that we have fewer and fewer images wrongly recognized as non-cats. This approach seems very intuitive, but it requires that a small change in weights (and/or bias) causes only a small change in outputs.
If we have a big output jump, we cannot progressively learn (rather than trying things in all possible directions—a process known as exhaustive search—without knowing if we are improving). After all, kids learn little by little. Unfortunately, the perceptron does not show this little-by-little behavior. A perceptron is either 0 or 1 and that is a big jump and it will not help it to learn, as shown in the following graph:

We need something different, smoother. We need a function that progressively changes from 0 to 1 with no discontinuity. Mathematically, this means that we need a continuous function that allows us to compute the derivative.
- Raspberry Pi 3 Cookbook for Python Programmers
- Istio入門與實(shí)戰(zhàn)
- 計算機(jī)組裝與系統(tǒng)配置
- 電腦常見故障現(xiàn)場處理
- 數(shù)字邏輯(第3版)
- Learning Stencyl 3.x Game Development Beginner's Guide
- Building 3D Models with modo 701
- 筆記本電腦維修實(shí)踐教程
- Managing Data and Media in Microsoft Silverlight 4:A mashup of chapters from Packt's bestselling Silverlight books
- RISC-V處理器與片上系統(tǒng)設(shè)計:基于FPGA與云平臺的實(shí)驗(yàn)教程
- Java Deep Learning Cookbook
- The Deep Learning with PyTorch Workshop
- 微服務(wù)實(shí)戰(zhàn)
- Deep Learning with Keras
- The Reinforcement Learning Workshop