Normally it will be difficult to tell who unfollowed you on twitter, and there can be many good reasons why you want to unfollow someone – don't hold yourself back because you are afraid that they'd might find out that you unfollowed them.
See also: Unfollow inactive twitter users
Unfollowing users can be a way to make more "room" to follow more people, inactive accounts can generally be unfollowed without the risk of having those annoying @mentions making everyone aware that you just unfollowed someone.
How can people see if you unfollow them on twitter?
There are many different tools to manage twitter followers, some of which will automatically send out a @mention to people who unfollows you – this dose discourage a lot of spam accounts from following and unfollowing large numbers of people – but it can also be annoying and spammy to real users. Who cares that someone just unfollowed you!? This is twitter after all! People follow and unfollow each other all the time! Maybe they simply got tired of seeing your automatic @mention spam in their timelines?!
To get rid of these @mentions, one may choose to respond with hostility towards such automatic spam, since this would discourage people from using these useless tools. Another way to avoid such uncomfortable situations on twitter, would be to only unfollow inactive accounts – like there are tools to do so many other cool things on twitter, there are also tools that will check which accounts that you are following that are inactive – this is typically done by checking the date on their last tweet.