When the boundaries of a district which is a component of a high school district are for any cause changed to include territory in, or to exclude from, the district, the territory added to, or removed from, the district shall, at the same time, be included in, or excluded from, the high school district.