Part of the difficulty of addressing privilege and power is the way in which oppressive structures are built up over time, and at the same time, we must interact with one another in the present.
One is born into a social formation and socialized with a certain set of expectations, and a baseline understanding of how one can expect to be treated, and how to treat others, and what sorts of technologies and resources are going to be at one's disposal.
The nature of this formation takes its character over time due to complex interactions of individuals, environments, and histories, among other things.
Two individuals from two completely different backgrounds or formations may interact with each other with a differing set of expectations, but the human base is relatively the same: one knows when one is slighted, and one knows when to slight. One knows what pain and pleasure are, and they happen immediately.
And the problem is, pain and pleasure can mean very different things for those in different tiers of privilege.
It hurts someone of higher privilege to be told to step down, while a person of lower privilege is always hurting, and looking to step up.
For both individuals to understand the world in the same framework, a decent understanding can be worked out.
But there is still the matter of the more privileged individual willingly stepping out of the way of her or his own ego, and willingly accepting a lower standard of living. A little more difficult to do than simply understanding these things intellectually. It happens easier when it happens apart from personal choice, but it can happen by choice.
The mountain crumbles rock by rock, where it is disturbed or where parts of it are already tumbling off and connected.