Any party to the proceeding may object to the reference of the proceeding to a particular hearing officer upon any one or more of the grounds specified in Section 641 of the Code of Civil Procedure and such objection shall be heard and disposed of by the appeals board. Affidavits may be read and witnesses examined as to such objections.