Whenever someone tells me "I'm sorry you feel that way", my response is always the same:
It's not a feeling; it's a fact.
I go on to support my position with objective, factual details. Often, saying "I'm sorry you feel that way" is simply a tactic to replace an objective fact with a subjective feeling to avoid being wrong or taking responsibility. Recently, I told a customer service rep that my order didn't arrive on time. Her response? "I'm sorry you feel that way." Huh? It wasn't my gut that told me the package would arrive within a week; it was your website. She didn't want to admit wrongdoing or take responsibility, so she shifted from fact to feelings, where she could apologize and be right!
Certainly I feel disappointed, angry, or some other negative emotion as a result of your failure to meet my expectation, but by focusing on my emotional response, you're dismissing the underlying issue and, as a result, offering an empty, insincere apology.
Think about it. "I'm sorry you're angry" is essentially the same as saying "I'm right, I'll do the same thing next time, and it's unfortunate you're experiencing an irrational emotional response to the situation."
I'm sorry ... you feel that way. I'm not sorry. I'm sorry for myself, that I now have to deal with your unjust, negative emotions. Sure, I could make you feel better, but I'm not going to. Why? Because I disagree with you, and since I'm definitely right, there's no point confronting the issue; instead, I'm sorry you're not smart enough to realize you're wrong, which is causing you to be angry. I'm sorry your being wrong makes you angry. Kind of smug, arrogant, and condescending, right?
Maybe that's not how you intend to come across, but can you see how it can be taken that way? I think some people confuse apologizing for someone's feelings with relating to or understanding that person's circumstance. There's a big difference between saying "I'm sorry you feel that way" and "I understand how this outcome may make you angry." The later is a great alternative. People want to be understood, even if that understanding won't change their circumstance.
What's a better alternative? Well, if you messed up, just say, "I'm sorry." No qualifier, no deflecting, no minimizing, no excuses. Swallow your pride, show humility, and own your mistake. We all make them. Even better, go on to explain why you're sorry and what you're going to do to remedy the situation and/or make sure it doesn't happen again.
But let's be honest, when you're tempted to say "I'm sorry you feel that way," you're not interested in apologizing because you don't think you messed up. In that case, don't offer an empty apology! Don't apologize at all! Instead, explain your position as objectively as possible. If you reach an impasse, agree to disagree, get a third party involved, or employ any number of problem solving techniques, but don't dismiss my concerns by saying, "I'm sorry you feel that way." Of course I'm angry — you're not listening!