I think it's more an "Us/Them" issue, with "Us" being "respectable, normal, safe" people, and "Them" being disruptive, unreliable, and of questionable ethics.

He has become *much* more open-minded as he's aged, but he still has a very strong sense that a middle-ground, middle-class, conventional public face is essential to social and career success.