A Modification of Backpropagation Enables Neural Networks to Learn Preferences

To help a person make proper decisions, we must first understand the person’s preferences. A natural way to determine these preferences is to learn them from the person’s choices. In principle, we can use the traditional machine learning techniques: we start with all the pairs (x, y) of options for which we know the person’s choices, and we train, e.g., the neural network to recognize these choices. However, this process does not take into account that a rational person’s choices are consistent: e.g., if a person prefers a to b and b to c, this person should also prefer a and c. Since the usual learning algorithms do not take this consistency into account, the resulting choice-prediction algorithm may be inconsistent. It is therefore desirable to explicitly take consistency into account when training the network. In this paper, we show how this can be done. 1 Formulation of the Problem Need to learn preferences. To help a person make decisions, we must first understand this person’s preferences. Sometimes, a person can describe his or her preferences in precise terms. However, in many cases, a person cannot describe these preferences in precise terms, so we must elicit such preferences from him or her. Elicitation such preferences is an important task in decision making; see, e.g., [5, 6, 8]. Elicitation such preferences is also an important part of recommender systems. It is natural to use machine learning to learn preferences. The only way to learn a person’s preferences is to provide this person with several pairs of options, record the person’s preferences for all these pairs, and then use this information to predict how this person will react to other pairs.