like for example if i clicked on the pool and chose swim, then the game changes the sim into swimwear 1 - but if that sim had 3 swimwear outfits, is there any way for the game to choose whether it changes the sim into swimwear 1, 2, or 3 at random?

I think NRAAS Dresser has a function like this.
It could do with a description to be sure, but the 'Rotation' feature seems to be referring to cycling a sim's outfits each day. Rather than every change, it should shuffle the order randomly overnight.

Yes, Dresser does that. Its enabled on non-active sims by default, you have to change it to true to affect your active sims.

Dresser-Interactions#Rotation

As a follow up question, is it possible to make it so that instead of randomizing each outfit individually, it selects a group of outfits? Like, for example, instead of the roll deciding to go with everyday 1, formal 3, swimsuit 2, etc. it decides to go with everyday 2, formal 2, swimsuit 2, etc.?

On MTS try consorts Dresscode sign.