自编码器 可视化

Auto-Encoders are one of the most useful unsupervised algorithms, to gain insight into the data, and therefore optimize the learning algorithm that trains the algorithm.

自动编码器是最有用的无监督算法之一，可以深入了解数据，从而优化训练算法的学习算法。

什么是自动编码器？ (What are Auto-Encoders?)

Auto-Encoders are a neural network, that takes the data as its input, and the data as its output. At first, this might seem ridiculous: The relationship between a number and itself is simply just 1, why is a neural network necessary? This is true, but auto-encoders have created a way around it.

自动编码器是一个神经网络，它将数据作为输入，并将数据作为输出。 起初，这似乎很荒谬：数字与自身之间的关系仅仅是1，为什么需要神经网络？ 的确如此，但是自动编码器已经创建了一种解决方法。

Using Bottlenecks. Bottlenecks, from its commonplace definition, means a narrow gap of the necks of glass bottles. In this case, bottlenecks refer to decreasing the number of neurons in the hidden layer. This, therefore, forces the data to be represented into a smaller number of neurons. The network, by default, will learn a way to compress data and decompress data, to encode and decode the data into bigger and smaller representations of data.

使用瓶颈。 从常见的定义来看，瓶颈是指玻璃瓶的颈部狭窄的缝隙。 在这种情况下，瓶颈指的是减少隐藏层中神经元的数量。 因此，这迫使将数据表示为较少数量的神经元。 默认情况下，网络将学习一种压缩和解压缩数据，将数据编码和解码为更大或更小的数据表示形式的方法。

为什么自动编码器如此有用？ (Why are Auto-Encoders so useful?)

In a world where datasets go up into the terabytes, we need to find a way to efficiently decrease the memory taken up by the data. Additionally, there is limited to computational power to process information. The smaller representation created by Auto-Encoders becomes so important, as the data it takes up is much less.

在数据集达到TB的世界中，我们需要找到一种方法来有效减少数据占用的内存。 另外，限于处理信息的计算能力。 自动编码器创建的较小表示形式变得非常重要，因为占用的数据要少得多。

Additionally, with a visualization technique I will show you, you can actually visualize what each neuron is looking for.

此外，借助可视化技术，我将向您展示，您实际上可以可视化每个神经元在寻找什么。

代码： (The Code:)

Now that the theory of the Auto-Encoder is clear, we can move on to the code. For the actual auto-encoder, I will be using Keras, but for the visualization of what each layer is looking for, I will use my own neural network framework.

现在，自动编码器的理论已经很清楚了，我们可以继续进行编码。 对于实际的自动编码器，我将使用Keras，但是对于可视化每个层的查找，我将使用我自己的神经网络框架。

from keras.layers import Densefrom keras.models import Sequentialimport requestsimport json

Along with the standard keras layers and models, I will be using requests and json, to access financial data, using the alpha-vantage API.

除了标准的keras层和模型外，我还将使用alpha-vantage API使用请求和json访问财务数据。

API_KEY = 'XXXXXXX'from_symbol = 'EUR'to_symbol = 'USD'close_price = [r = requests.get( 'https://www.alphavantage.co/query?function=FX_INTRADAY&from_symbol=' + from_symbol + '&to_symbol=' + to_symbol + '&interval=1min&outputsize=full&apikey=' + API_KEY)jsondata = json.loads(r.content)pre_data = list(jsondata['Time Series FX (1min)'].values())fx_data = []for data in pre_data: fx_data.append(list(data.values()))fx_data.reverse()for term in fx_data: close_price.append(float(term[-1]))

This is the script I use to access the dataset from alpha-vantage. However, to utilize this script, change the API key to a valid key, that you can get here. The script also only picks out the closing price, for simplicity’s sake.

这是我用来从alpha-vantage访问数据集的脚本。 但是，要利用此脚本，请将API密钥更改为有效密钥，您可以在此处获得。 为了简单起见，脚本也只选择收盘价。

model = Sequential() model.add(Dense(10,input_shape = (None,10),activation = 'relu'))model.add(Dense(1))model.compile(optimizer = 'adam', loss = 'mse')model.fit(close_price,close_price,epochs = 100)

This script creates the model, just using Dense layers. You could use KL-Divergence a means of preventing the model from just memorizing the input data. It penalizes the network by the number of activated neurons, so as to decrease the number of neurons needed to represent the data.

该脚本仅使用密集层创建模型。 您可以使用KL-散度来防止模型仅记住输入数据。 它通过激活神经元的数量来惩罚网络，从而减少表示数据所需的神经元的数量。

That is the full program! Simple right?

那是完整的程序！ 简单吧？

可视化： (Visualization:)

The main advantage of auto-encoders is the ability to visualize what each neuron is looking for. First, copy this framework into your program.

自动编码器的主要优点是能够可视化每个神经元在寻找什么。 首先，将此框架复制到您的程序中。

import numpy as npfrom matplotlib import pyplot as pltdef sigmoid(x): return 1/(1+np.exp(-x))def sigmoid_p(x): return sigmoid(x)*(1 -sigmoid(x))def relu(x): return np.maximum(x, 0)def relu_p(x): return np.heaviside(x, 0)def tanh(x): return np.tanh(x)def tanh_p(x): return 1.0 - np.tanh(x)**2def flatten(itr): t = tuple() for e in itr: try: t += flatten(e) except: t += (e,) return tdef deriv_func(z,function): if function == sigmoid: return sigmoid_p(z) elif function == relu: return relu_p(z) elif function == tanh: return tanh_p(z)class NeuralNetwork: def __init__(self): self.layers = [] self.weights = [] self.loss = [] def add(self,layer_function): self.layers.append(layer_function) def initialize_weights(self): for layer in self.layers: index = self.layers.index(layer) weights = layer.initialize_weights(self.layers,index) self.weights.append(weights) def propagate(self,X): As,Zs = [],[] input_data = X for layer in self.layers: a,z = layer.propagate(input_data) As.append(a) Zs.append(z) input_data = a return As,Zs def train(self,X,y,iterations): loss = [] for i in range(iterations): As,Zs = self.propagate(X) loss.append(sum(np.square(flatten(y - As[-1])))) As.insert(0,X) g_wm = [0] * len(self.layers) for i in range(len(g_wm)): pre_req = (y-As[-1])*2 a_1 = As[-(i+2)] z_index = -1 w_index = -1 if i == 0: range_value = 1 else: range_value = 2*i for j in range(range_value): if j% 2 == 0: pre_req = pre_req * sigmoid_p(Zs[z_index]) z_index -= 1 else: pre_req = np.dot(pre_req,self.weights[w_index].T) w_index -= 1 gradient = np.dot(a_1.T,pre_req) g_wm[-(i+1)] = gradient for i in range(len(self.layers)): self.layers[i].network_train(g_wm[i]) return loss class Perceptron: def __init__(self,nodes,input_shape= None,activation = None): self.nodes = nodes self.input_shape = input_shape self.activation = activation def initialize_weights(self,layers,index): if self.input_shape: self.weights = np.random.randn(self.input_shape[-1],self.nodes) else: self.weights = np.random.randn(layers[index-1].weights.shape[-1],self.nodes) return self.weights def propagate(self,input_data): z = np.dot(input_data,self.weights) if self.activation: a = self.activation(z) else: a = z return a,z def network_train(self,gradient): self.weights += gradientmodel = NeuralNetwork()Perceptron = model.Perceptron

I won’t get into the code of the framework, but it essentially works as a transparent version of Keras, that can give the propagated value of each layer.

我不会涉及框架的代码，但是它实质上是作为Keras的透明版本工作的，它可以提供每一层的传播值。

X = np.array([[1, 1, 1, 1, 0], [1, 1, 0, 0, 1], [0, 0, 0, 1, 0], [1, 0, 0, 0, 0], [0, 0, 1, 1, 1], [1, 1, 0, 0, 0], [1, 0, 0, 1, 0], [1, 0, 1, 1, 0], [0, 0, 0, 0, 0], [1, 1, 0, 0, 1]])model.add(Perceptron(2,input_shape = (None,5),activation = sigmoid))model.add(Perceptron(5,activation = sigmoid))model.initialize_weights()loss = model.train(X,X,1000)

This Auto-Encoder is trained with just MSE for 1000 epochs.

仅使用MSE即可训练此自动编码器1000个纪元。

As,Zs = model.propagate(X)full_set = []for a in range(len(As)): a_set_values = [] for z in range(5): zeroes =np.zeros(5) zeroes[z] = 1 As,Zs = model.propagate(zeroes) a_set_values.append(As[a]) full_set.append(a_set_values)full_setimport matplotlib.pyplot as pltimport numpy as npplt.subplot(212)for sets in full_set: plt.imshow(sets, cmap='Greys', interpolation='nearest') plt.show()full_set

Then, propagate the network with the data. Since the data here is simple, we iterate through each term in the data, changes it to 1, and see how it changes each layer of the data. We then plot the black and white images of it, to see the effect

然后，将数据传播到网络中。 由于此处的数据很简单，因此我们遍历数据中的每个术语，将其更改为1，然后查看其如何更改数据的每一层。 然后，我们绘制它的黑白图像，以查看效果

Thank you for reading my article!

感谢您阅读我的文章！

翻译自: https://towardsdatascience.com/an-intuitive-guide-to-auto-encoders-theory-code-and-visualization-3cc2a6a30d2c

自编码器 可视化