Understanding Les Grosses Tetes is pretty impressive actually. Speaking is always the last part of learning in any case.
I listen to it religiously so I've found it's a matter of getting used to accents and speed more so that vocabulary or grammar. The closest I've come to a real conversation was apologizing to a Belgian for my French but he said I was fine. Who knows if he was just being nice though haha!