Below I propose three policies to help guard against the mempool being flooded. The three would be used in conjunction with each other.
**Problem 1: A user can send large transactions instead of multiple small ones, which also hogs memory.**
Solution: The bigger a transaction is, the more in fees it should cost to send. This can be done using the txsize.
**Problem2: When the network is being spammed, nodes will make it worse by propagating the malicious transactions.**
Solution: Each node will have a minRelayFee. This fee will be based on the size of the mempool and the rate at which transactions are being added. If a lot of transactions are added in a small amount of time and they all have low fees, they will not pass the minRelayFee threshold and will not be propagated. If the nodes mempool is full, then it will take a large fee for that node to relay the transaction. Importantly, he will not discard it.
**Problem3: When the mempool is full, it stays full for a long period of time.**
Solution3: Transactions upon being added can be stamped with a time, If a transaction has been in the mempool for longer than 5/10 minutes, it should be flushed, and the user would need to re-submit their transaction.
— During the time that the attacker is attacking the network, fees will be high and so good transactions trying to get in, will be delayed or will need to pay high fees.
— One issue brought up by Igor, is that some attackers would be willing to pay millions to take down promising blockchains.
With the above model it would make it expensive for an attacker to do so, the fees would go back into the economy, however it still stands that the more money the attacker has to waste the longer the attack would last for.
— Nodes may find it hard to agree on a set of transactions.
With the model, the subset of transactions that would be agreed upon would be the ones with the highest fees, as these would be the ones that are being propagated.
Implementation would not require a fork.
In order for a malicious actor to spam the network, he will need to keep increasing the fee, the faster he spams, the more he has to pay, in order for his transaction to propagate. The attacker would need to adjust the fee for each node as node mempool sizes are different, so they will have different minRelayFees. The attacker could mitigate this by attaching a large than normal fee, and increasing it by some factor.
The amount being sent could also be added as a factor, when deciding on whether to relay. A case would be where the txsize is small, however the amount is large because there are not many inputs/outputs.