Any vacancy occurring during the unexpired term may be filled by the secretary for the unexpired term. In filling a vacancy, the secretary may consider recommendations from the remaining members of the council. The director may remove any member of the council for cause deemed sufficient by him and upon such removal there shall be a vacancy which shall be filled in the manner as provided in this section for the unexpired term.