[input-output-hk/cardano-sl] [CBR-389] Don’t construct WalletAddress IxSet (#3439)

Rebased and addressed PR feedback (inserted comments, removed `sliceList`).