You are 1000% percent right about this caveat. And more so.

I did not write about accessibility in this article. Related, my advice on accessibility is often two parted:

1) Make sure the palette has been designed with accessibility in mind.

2) Make sure each visual is accessible - using an accessibility assessment tool.

Just because the palette is accessible, does not automatically mean all visuals will be accessible too.