Landscape heterogeneity, including compositional and spatial configuration, changes soil erosion and sediment yield (SESY) in watersheds by regulating hydrological processes. However, existing research has paid little attention to landscape composition. The Lower Jinsha River Basin (LJRB) has significant landscape heterogeneity and the highest sediment yields in the Yangtze River Basin, China. The spatially distributed soil erosion model WaTEM/SEDEM was used to evaluate the spatio-temporal changes in SESY in the LJRB from 1990 to 2020. Area information conservation method and semi-variogram were utilized to determine the optimal grain and extent size of landscape analysis. The landscape heterogeneity characteristics of the LJRB were analyzed for landscape composition and spatial configuration. Finally, statistical analysis was performed to determine the differences in SESY among different landscape composition types (LCTs) and to explore the relationship between landscape configuration and SESY. The optimal grain and extent size for analysis of the landscape pattern in the LJRB were 150 and 2100 m, respectively. Based on this, three types and ten subtypes of landscape composition were identified. Landscapes composed of forest, shrubs, grassland, and farmland were predominant in the study area, with a relatively even distribution of patches and minimum dominance of specific patch types. Additionally, the landscapes composed of forest, shrubs, grassland, farmland, and impervious surfaces were diverse, with the highest fragmentation. Landscapes comprising farmland and impervious surfaces occupied a minor proportion of the study area, but specific patches exhibited higher landscape connectivity. The magnitude of SESY in the LJRB gradually decreased during 1990-2020. Farmland and grassland were the land types with the highest magntidue of SESY in this region. Substantial sediment has been deposited in Xiangjiaba and Xiluodu reservoirs since they commenced operations. Increased proportions of farmland patches in some LCTs have significantly enhanced SESY. For LCTs dominated by farmland and impervious surfaces, reducing the dominance of the largest patch, decreasing patch aggregation, and increasing landscape-type diversity and evenness have helped mitigate SESY. The opposite is true for LCTs dominated by forests, shrubs, and grasslands. Collectively, the results of this study provide valuable progress in the capacity to support landscape restoration and comprehensive management of soil and water degradation in mountainous watersheds with similar characteristics to the LJRB.