Happend to me, too. Refreshing server list or restarting game didn't work. Although nothing was selected in my modlist, I had to click on "Clear all" and restart. After that most servers (about 2/3 maybe?) did not have the red cross anymore. Agreeing with Jimmy, smells like a bug.
Never had a mod, clean install on a couple of PC's. No refreshing or restarting fixes it.
Fixed itself over time.