I can relate. My DH had a pretty flexible job where he can work from home. But his hours vary a lot and he can easily take half days and run errands during the day that I can't. So I get jealous because he doesn't have to commute and some days only works like 4 hours. Everyone in his dept does it so it is not an issue, whereas for me, if I leave 30 mins early my boss is watching me and I know it.
I get jealous too, but I'm happy for him and his flexibility is great for us as a family. He can do errands during the day for us, be home for repairmen, etc. When we have kids, it will probably be easier for him to do the drop off/pick up or stay home when our kid is sick. So overall, I know it benefits both of us. I rag him sometimes (when I'm particularly stressed) and thankfully, he doesn't hold it against me.
But more than that, I've realized that I'm really jealous because he really likes his job and I really don't like mine. Which is making me realize, I need to make a change in my life. I'm just like you, I have good pay, decent commute and good vacation, so it is hard to give that up. But I'm realizing being happy is more important so if that means taking a pay cut to make a career change, so be it. If I don't do anything to change my situation, then I can't really complain.