No person may serve as a director unless he is a resident and voter of the district as finally determined. Any vacancies on the board caused by the elimination of territory shall be filled by appointment by the remaining directors, in which case ward lines may be disregarded.