How can I safely update the data on a node behind my node balancer?
I need to upload data to my nodes one at a time, but I don't want to upload data while my customers may be using that data for calculations.
I know I can set the state of any node to "reject", but does this take effect immediately (i.e. kill all currently processing requests)?
Alternatively, if I were to "drain" the connections, how can I know when the number of connections is zero so that I can safely update the data on this node?
For context, these nodes are hosting an API, not serving HTML.
Setting the NodeBalancer Mode to Reject will reject new incoming connections but current connections will remain active.
We suggest using the Drain Mode to prep nodes for maintenance since this will only accept connections that have a session stickiness pointing to the backend node.
To determine when your backend is successfully drained, you can use a command such as
tcpdump or other more sophisticated methods to check your node's traffic. Generally speaking you will want to wait a day or so before taking the node down after enabling Drain mode.