I can't decide if he's naïve or stupid.

Yes.

You'd be so proud of me!

We are, though we'd have been proud of you if you'd actually said something, too.

If there were some chance that you'd take a job offered by someone at the conference, it would have made perfect sense to wear the F.I.E. even if you'd known that people were dressing down. The advice I was given many years ago (and mostly ignore, because I'm in academia) was to "dress for the job you want, not the job you're in." You lose nothing by looking professional.