I think that considering that you were a virgin before starting birth control, you really can't say that the pill is affecting your sex drive........I don't think that you really have any clue what kind of sex drive you have.
Sure, you wanted sex and thought about it more before you were married.......because it was the forbidden fruit, something that you had built up in your head to be this wonderfully amazing experience.
Then you actually had sex, it wasn't what you expected, and now that reality didn't meet your expectations you think about it/crave it less. I doubt that getting off birth control pills will help you at all - you'll still be disappointed for awhile with sex, and you'll also be hyper worried about getting pregnant accidentally (I'm assuming that you went on birth control because you're not ready for a child yet)......that fear will certainly make it harder to get in the mood for sex.
Call you GYN and tell them your concerns, and ask about trying a different pill. Everyone reacts to every pill differently, and maybe this one isn't right for you. But don't discredit the idea that you just have to keep practicing at sex.....the more you practice, the better it will feel......and the better it feels, the more you'll want it. And if you don't already, start masturbating. If you don't know and understand your own body and what makes you feel good, it's going to be that much harder for your DH to know these things.