I just added the customers to newly created Customer Groups: Group A, Group B Group C etc.
I set all the users Status to OFF
Then when I need to have the specific group access the site, I sort by group, then change their status to ON
Being that I only have 100 users, its not a daunting task to set them manually when they are in smaller groups.