I've bowed out of many "mandatory" events by claiming family obligations. Since I'm the sole driver in the family, it's usually true.

There's very little I can do to avoid these things when they hold them during work hours.

At least they're paying you for them!

Maybe you should renegotiate your contract: time and half or double time for "fun"....