I think the whole thing will end up revolving around whether or not giving paypal access to your bank account gives them carte-blanche to remove and / or deposit money at their whim.

I would argue that they don't, that they need an explicit instruction from me to do so, they will probably argue that they don't need any permission at all, that the permission once granted stays granted until explicitly revoked.