Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes from preceding cues or actions. For example, reward outcomes may be linked to specific stimuli and actions, corresponding to stimulus- and action-based learning. But how does the brain arbitrate between such models? Here, we combined multiple computational approaches to quantify concurrent learning in male monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for a dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala in model arbitration. We demonstrated that the amygdala adjusts the initial balance between the two learning systems and is essential for updating arbitration according to the correct model, which in turn alters the interaction between arbitration and learning that governs the time course of learning and choice behavior. In contrast, VS lesions lead to an overall reduction in stimulus-value signals. This role of the amygdala reconciles existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.