More races than just the humans refused to join the Conclave. The Rraey, the Enesha, and the Obin were three examples.

But only two races refused to even send delegates to the conference where the Conclave was being planned, and humans were one of those. (The other being the Consu, because, well, they're so advanced, why would they bother?)