You know I am learning my second language, and in this long way I realized we have to be more that patience. Many of different things have to go together, our mains need to be under our control too, because some emotions can go against us if we have not taken the controls of our thoughts.
I remember myself feeling like embarrassed ans sweating in a group of people, literally some times I wan to run away still. If we are learning something we need to challenge ourselves, I mean I can understand it is not great feeling awkward when you are in a group of people and you can not understand what they are talking about, or when you are watching a cartoon or a TV commercial, believe me I feel you. Well I have to tell you there is where you should be, yes feeling uncomfortable because there will be when your brain will give a step forward, you have to win the battle against your lazy brain and your thoughts so please be conscientious about that time and just don’t run away, don’t change the TV channel etc. Keeping listening and facing that awkward time is a key to enhance, You will gain confidence slowly and you will surprise when one day you start to understand that TV show or that song or the surrounding people.
One of those challenge you have to face can be listening o watching a video where the speaker has an accent, yes a teacher with Australian accent or British accent or a native Spanish speaker talking in that language you are learning. There are many kinds of accents, these are some examples, I know you got the point. This challenge will help you to feel less frustration.
I want to recommend one resource I use to training my ears. This is the site to hear different accents http://elllo.org/, so take a look and listen, listen and listen.