If I was part of a MILSIM team that actually utilized ranks, I would wear mine on the field. Off the field, however, I'm rarely even in uniform, let alone pulling rank.
Wearing a rank on the field for a purpose (i.e. team rank structure, re-enactment, etc.) can sometimes be acceptable, and different people will tell you different things. But IMO, wearing anything you didn't earn off the field is NEVER acceptable. If you're on your way back from the field, take the rank off (thanks to ACUs having velcro ranks), or take the blouse, hat, shirt, etc. with the rank off.
You'll find a lot of vets don't care if you wear the uniform in public, but you'll be asking for a new ass if you wear ranks, patches, etc. that aren't yours.