Understanding state space organization in recurrent neural networks with iterative function systems dynamics

Abstract

We study a novel recurrent network architecture with dyna-mics of iterative function systems used in chaos game representations of DNA sequences [16,11]. We show that such networks code the tempo-ral and statistical structure of input sequences in a strict mathematical sense: generalized dimensions of network states are in direct correspon-dence with statistical properties of input sequences expressed via genera-lized Renyi entropy spectra. We also argue and experimentally illustrate that the commonly used heuristic of finite state machine extraction by network state space quantization corresponds in this case to variable memory length Markov model construction.