Why are you upset that he took extra money out? Did a bill not get paid? Was savings compromised? What were the consequences?
If there were none - besides the fact that you discovered he handled something differently than you expected - you need to go apologize to him. You are not the boss of him. You have no right to be upset with him because he handles money differently than you. As he asked, so what if he used cash for some things and debit for other? Who said your way is right and his is wrong?
Do you see how this could make a person angry? To be treated like a child?
Now, if there were repercussions, like you couldn't pay a bill, that's different. Calmly talk to him about your concerns about that and ask him if he has ideas or solutions. Work together to come up with a plan.
And frankly, considering your reaction I can see why he didn't rush to confront you. Sheesh. You know, sometimes people are just trying to be nice when they first get married, are trying to make it work and not complain.