官术网_书友最值得收藏!

Autoregressive models

Autoregressive models estimate the conditional distribution of some data  , given some other values of y. For example, in image synthesis, it estimates the conditional distribution of pixels given surrounding or previous pixels; in audio synthesis, it estimates the conditional distribution of audio samples given previous audio samples and spectrograms.

In its simplest linear form, with dependency on the previous time-step only and time-invariant bias term, an autoregressive model can be defined with the following equation:

is a constant term that represents the model's bias,  represents the model's coefficients, Yt-1 represents the previous output vector, and is assumed to be white noise. The dependency of the current output on the previous output is explicit in this equation.

Although autoregressive models are sequential in nature, given that the training data is available beforehand, they are normally trained in parallel using the teacher-forcing procedure. In this procedure, the model is not conditioned on its output, but on the real output obtained from the training data.

During inference, the model's output must be used, because we do not have access to the correct output – that is, the model must do autoregression on its own output, hence the name autoregressive model.

Autoregressive models have the advantage of being trained with simple and stable maximum likelihood estimates. This simplicity is counterbalanced by the limited capability of autoregressive models to perform inference in parallel, thus potentially requiring long wait times to generate data.

PixelCNN is one of the most famous autoregressive models for image synthesis. You can refer to the following paper for more details: https://www.semanticscholar.org/paper/Conditional-Image-Generation-with-PixelCNN-Decoders-Oord-Kalchbrenner/8e4ab54564fb492dcae9a1e862aedd3e52fb258b.

In the following figure, we show an image of faces generated with PixelCNN:

Source: Conditional Image Generation with PixelCNN Decoders ( https://arxiv.org/abs/1606.05328)

WaveNet is one of the most famous autoregressive generative models for audio-synthesis. You can refer to the following paper for more details about WaveNet, available at: https://arxiv.org/pdf/1609.03499.pdf. The following diagram describes WaveNet's graph:

Source: WaveNet: A Generative Model for Raw Audio ( https://arxiv.org/abs/1609.03499)
主站蜘蛛池模板: 乐都县| 和静县| 博罗县| 尚义县| 石楼县| 九寨沟县| 黔西县| 小金县| 潢川县| 托克逊县| 清丰县| 方山县| 黔东| 宝兴县| 怀宁县| 北辰区| 万山特区| 兖州市| 广西| 神池县| 迁安市| 广元市| 盖州市| 佛冈县| 莎车县| 高尔夫| 根河市| 邯郸县| 来安县| 台州市| 淳安县| 白银市| 抚州市| 兴国县| 太仓市| 栖霞市| 青阳县| 宁安市| 满城县| 满洲里市| 五寨县|