I had this exact problem this morning! And at the beginning, it's hard to stir all the oil to the bottom without it making a big mess. They should leave more room at the top!
I often just don't refrigerate. I don't know if that's bad or not, but it sounds pretty common.