Arjun Kumar and Tim Oates
Abstract—Neural networks have attracted significant interest in recent years due to their exceptional performance in various domains ranging from natural language processing to image identification and classification. Modern deep neural networks demonstrate state-of-the-art results in complex tasks such as epileptic seizure detection  and time series classification . The internal architecture of these networks, in terms of earned representations, still remains opaque. This research addresses the first step towards the long term goal of constructing a bidirectional connection between raw input data and symbolic representations. In this research, we examined whether a denoising autoencoder can internally find correlated principal features from input images and their symbolic representations that can be used to generate one from the other. Our results indicate that using symbolic representations along with the raw inputs generates better reconstructions. Our network was able to construct the symbolic representations from the input as well as input instances from their symbolic representations.
Deep neural networks are a form of artificial neural network that consists of many layers of hidden units between their input and output layers with the aim to capture a complex hierarchy of features of the inputs . Using multiple hidden layers, deep neural networks extract features of inputs at multiple levels, allowing the network to learn complex mappings between these inputs and their expected outputs . Recent advancements in the architecture and training mechanisms of deep neural networks have made them replace state-of-the-art systems in many fields. Although deep neural networks present world class results in many domains, the internal representations these networks learn are still opaque. We intend to address this problem by trying to connect symbolic representations of the input to neural networks to understand and reason about what is learned by them.
An autoencoder is an artificial neural network that consists of identical input and output layers with one or more hidden layers that present themselves as limited capacity channels used to abstract complex features of the input space ….
Complete technical paper available as a PDF.