[bitcoin/bitcoin] Add checks for settxfee reasonableness (#13988)

@ajtowns feel free to pick or squash ad367e1cb.