I think you need to ultimately bring it up. I know it's hard, but try to find a way of wording it that it isn't "confrontation". You've been given a couple ideas already.
Another idea is to be VERY honest w/ them, but said in a very gentle way - the next time they ask for you to sit, tell them "I have to be honest about this because I'm finding myself getting upset and I REALLY don't want to be upset. We started swapping sitting w/ each other to help one another out, and I really enjoyed doing this. However, lately, it seems as if you all aren't available and you always offer up your DD. Clearly, we love using your DD. But I feel that our original terms have changed - we sit for you for free, but we have to pay for your DD to watch. I fully understand that you all are busy and if the swap will no longer work, that's fine. But in turn, we need to work something else out.".
Good luck - I know this is hard. :(