In order to improve the performance of image annotation, recently proposed methods build their model combining multiple dependencies from relations between image and label (image/label), between images (image/image) and between labels (label/label). However, most of these methods cannot make multiple dependencies work jointly, and their performances is largely depending on the results predicted by image/label dependency. To address this problem, we propose an end-to-
end image annotation model to associate these dependencies with the prediction path, which is composed of a series of labels in the order they are detected. Specially, our model can adaptively adjust the prediction path: from those easy-to-detect relevant labels to these hard-to-detect relevant ones. To validate the effective of the model, we conduct experiments on three well-known public datasets, COCO 2014, IAPR TC-12 and NUSWIDE, and achieve better performance than the state-of-the-art methods.