People in Australia tend to forget/ignore the fact that the Indigenous people were invaded, and then massacred, treated as 'flora and fauna' (and up until 1970 or so they were still regarded so in the constitution) and generally just shit upon by the colonial invaders.
To then expect them to just adhere to a new society and fit in, with people who destroyed their home, killed their relatives and just totally destroyed their way of life is ridiculous.
Even today, they are still treated like shit, have high rates of incarceration, and a much lower life expectancy.
I dont have the answer, but i use this argument with people who shit talk indigenous australians over here. It works ok when you ask them something like 'If a group of people came to your land, turfed you out, killed some of your relatives, and then treated you like shit, would you want to be a member of that society? Would you teach your kids that those were good people and to be trusted?' Sometimes it resonates, but mostly people racist against indigenous australians are just fucked in the head and its not worth wasting time on them.
Thats my 2c. Not the most eloquent rant, but you get the picture i hope.