Planning rearrangements in a blocks world with the Neurosolver

The Neurosolver, a neural network based on the architecture and connection patterns of cortical columns, is applied as a solver of rearrangement problems in a version of the blocks world. The rearrangement problems are expressed in terms of state spaces that are suitable for the Neurosolver. Automatic ways of generating state spaces are discussed. Several presented experiments illustrate the solving capabilities of the Neurosolver. Figure 1. A workbench to explore the blocks world by neuromorphic agents. Figure 2. The solution to the problem in a global sense is a sequence of configurations between the current configuration (state) and the goal. one mapping of the input into the output. One can start with a large number of available nodes and end up only with some of them being actually used for storing states, because some inputs may never be activated. The unused nodes can be pruned to limit the size of the network. Some of the unused nodes may be left for the future to accommodate any inputs that have not been accounted for so far, but may be activated later on. A supervised algorithm, for example the backpropagation (Rumelhart, [5]) or Kohonen’s LVQ (Kohonen, [4]), could also be used, but unsupervised algorithms are superior in the sense that they do not need any a priori knowledge and can adapt autonomously; i.e., without a teacher. In the course of this research, both, the Kohonen and backpropagation, methods have been used to confirm that a state space can be obtained in an automatic way. The Stuttgart Neural Network Simulator ([6]) was used in both cases. The results conHence, the dimension of the input vector is (3x3)x2=18. Any configuration of the blocks can be encoded using that number of nodes. An example is illustrated in Figure 4. The input nodes are connected to state nodes in such a way that one state node is activated for each block configuration. The connections could have been determined by one of the techniques described in the previous section. In the case of the blocks world, the connections may have also been established manually as the results of the analysis of the state space for the blocks rearrangement problems that has been presented as well in the previous section. In all but the backpropagation case the resulting wiring is of the type illustrated in Figure 5. The only difference in the network obtained by ... ... i1 ij iM c1 c2 ci cN ... ... (grandmother cells) input nodes ijci W weight matrix output nodes the backpropagation method would be the presence of intermediate hidden units between the input and output nodes. In every case, one and only one output unit is set in response to any pattern. It should be noted that, although there are efferent connections going from each input node to every output node, some of the connections, those that have strength equal to 0, are in fact not present. They have to be considered only during the learning process. 3.2 Output representation In the opposite direction, whenever a node fires, the state associated with that node is translated into a binary representation of the block configuration, analogous to the input vector. The blocks in the workspace are re-arranged accordingly by the system of manipulators. An example is presented in Figure 5. firmed the claims. In the Kohonen case, a larger number of nodes than the cardinality of the state space had to be used. In one of the experiments, from 255 (15x15 grid) original nodes, after a number of training epochs (in the range of hundreds of thousands), 60 distinct nodes started to respond to 60 input vectors encoding the configuration of the world. Using only 60 nodes for the algorithm has not yielded desired effects; i.e. one node would respond to more than one input pattern with certain nodes remaining unused. In spite of trying various parameters that impact the learning in the Kohonen network and running the training for very long, the desired one-to-one mapping has not been obtained. The backpropagation method required that input-output pairs were presented at the same time. After the training a distinct output node responded to each input vector. A network with a hidden layer of 15 units was trained in ten thousands epochs using the traditional learning schema. The output layer was an array of 60 nodes. Figure 3. There are 10 possible arrangements of blocks if the color is disregarded. Three block can be arranged in 6 ways, there are 10x6 possible configurations taking into account the three colors. Figure 4. Encoding the configuration of the blocks in the blocks world into an input vector to the Neurosolver. Figure 5. Every input cell is connected to all grandmother cells. With a certain matrix of the connection weights W, each combination of input signals activates just one of the grandmother cells representing the state (concept) Ci of the input vector I. C A