Every Coach might have a different view on this, but in my opinion on those days I’d rather lighten the weights than rest. Those days are more about getting blood in the muscles and moving, not necessarily pushing heavy weights.
If you are getting gassed out try lowering the weights.
Another tactic is to intentionally set up three exercises in the circuit geographically far from each other while training, so the walk between exercises takes 15 seconds instead of 5.

I don’t write rest times much.

Rest on accessory should be as little as you need.
Usually 60-90 seconds is plenty.

Move quickly.
It’s not cardio so you shouldn’t be sucking wind, but you also shouldn’t be sitting around looking at a clock.
Go as soon as you can.