What it changes
A lower-bit format needs less memory and less data to read per token. A higher-bit format stays closer to the original model. How much the difference shows depends on the model and the task, so test the formats you are choosing between on your own prompts.