Thanks for the replies, guys. I've already tried a couple of washers variations, top/bottom, stock/bones etc... Nothing really worked. I guess the edge bushing seat is just too worn-out to hold it from popping.
The thing is, it doesn't affect my skating at all. It just looks dumb. Might be the time to finally switch to Ace