I would be very suspicious. I might watch the drawer for a while and see if any more go missing. Did you check the floor, behind the drawer, etc. in case one condom fell out?
Confronting is good but I tend to want to lie low and gather more information before confronting because once you do, if he is cheating, he knows and he'll just cover his tracks a bit better.
Would he have had a friend over at any point who might have asked him for one?