It’s pretty simple. Play the game in offline mode and it all goes away and you’re playing with no framerate hiccups, even in large crowd.
I first found the thread about it on reddit here, then immediately tested it tonight myself and it worked like a charm. Just put your PS4 offline (I’m assuming this also works on Xbox One, not so sure about PC).