Near as I can tell, there's the startup state of the game palette, which pictures can override. Which makes sense. The problem is that if a picture doesn't do so, it will be drawn with whatever palette settings are already there. Could be the default, could be whatever fancy setup the previous picture set up.
It's like a box of chocolates uninitialized variables.
At least that's how I read it.
As for CGA and such, the palette in question is itself an abstraction of sorts that, no matter what combinations you put in there, is ultimately mapped down to the standard 16 for rendering, and anything with fewer than 16 colors maps it even further down. But until it reaches the video driver, the color values you're assigning are effectively always the standard 16 under the hood.