A key experimental tool for studying infants' cognitive development is their exploratory behavior, such as gaze. Such behavior may, in turn, reflect rational information gathering and thereby connect to theoretical accounts of exploration in other settings. For instance, a preference for novel stimuli that habituates with exposure might reflect the diminishing value of information over repeated samples. From this perspective, though, it is surprising that novelty preferences are not universal, and that instead the opposite preference – favoring more familiar stimuli – is also observed. In a classic phenomenological model, Hunter and Ames (1988) suggest that a progression from familiarity to novelty preference arises due to an inverted U-shaped preference for a stimulus with exposure. This poses a puzzle for connecting infant exploration to rational information gathering- Why should the value of information not decline monotonically as a stimulus is sampled? We propose a computational theory for the Hunter and Ames (1988) phenomena by connecting them to a line of work in which we have analyzed value of information in sequential decision tasks, in terms of the expected improvement in future returns. The insight of this work is that this value can be decomposed into the product of two terms, Gain and Need. Gain reflects the additional rewards due to exploration producing better decisions at the explored state- this reflects the information received and is monotonically decreasing over exposures. But in a sequential task, these rewards are only realized when that state is revisited- thus value of exploration depends on Need, i.e. expected future occupancy of the explored state. We propose that infant exploration can be explained in the same terms, with familiarity preference arising due to Need. This perspective offers a new connection between infant and computational exploration, and new interpretations and predictions about the factors that impact infants' exploratory attention.