Is it reasonable for an anthology to expect first publication rights when a) 'all rights remain with the author' is specified in the submission guidelines and b) they are not paying you anything? It seems from what I read that first publication rights are something you specifically sell in a contract. A short story of mine was recently withdrawn from an anthology at the last minute because it appeared in another publication (I wasn't aware of the latter until after I okayed the former). Neither of these publications asked me to sign anything. I understand the courtesy, but isn't the onus on the publisher to specify what 'previously unpublished' means?
It does sound as if you've ended up in a tangle entirely not of your own making. (And for clarity let's call the first publisher you mention - the one who ended up saying no - Publisher A, and the other is Publisher B.) It's hard to know exactly what to say without knowing the specifics of each publication, but it does seem a bit draconian of Publisher A to withdraw your story. The idea of 'first publication' rights would be for the first publisher, whoever that is, to have the commercial advantage - but unless your story's presence in Publisher B's anthology was going to mean that no one would buy Publisher A's anthology, there was no commercial reason to pull your story. Especially when, as you point out, they're not paying you and you haven't assigned any rights in a contract. And I'm presuming Publisher A didn't say 'you cannot submit your story to anyone else while you wait for our decision'. So they're being churlish. And there's not much you can do about it.