uWSGI allows to dynamically scale the number of running workers using pluggable algorithms. This allows to run only required number of workers based on some algorithm specific criteria.
Use uwsgi --cheaper-algos-list to get the list of available algorithms.
To enable cheaper mode You need to add cheaper = N option to uWSGI configuration file, where N is the minimum number of workers uWSGI can run.
cheaper value needs to be lower than the maximum number of configured workers (workers or processes option).
# set cheaper algorithm to use, if not set default will be used
cheaper-algo = spare
# minimum number of workers to keep at all times
cheaper = 2
# number of workers to spawn at startup
cheaper-initial = 5
# maximum number of workers that can be spawned
workers = 10
# how many workers should be spawned at a time
cheaper-step = 1This will tell uWSGI to run up to 10 workers under load, if app is idle uWSGI will stop workers but it will always leave at least 2 of them running.
With cheaper-initial You can control how many workers should be spawned at startup, so if Your average load requires more than minimum number of workers You can have them spawned right away and than cheaped if load is low enough.
When cheaper algorithm decides that it needs more worker it will spawn cheaper-step number of them, this is useful if You have high maximum number of workers -- if load spike hit Your app it could take a lot of time to spawn all of them one by one.
This is default algorithm.
If all workers are busy for cheaper_overload seconds than uWSGI will spawn new workers. If load is gone it will start to stop processes one at a time.
Only available on Linux.
If listen queue has more than cheaper_overload requests waiting to be processed uWSGI will spawn new workers.
If it's lower it will start to stop processes one at a time.
This algorithm is optional, it is only available if cheaper_busyness plugin is compiled and loaded.
This plugin implements cheaper algorithm that adds or remove workers based on average utilization for given time period. It's goal is to keep more workers than the minimum needed at given time, so that app always have some room for new requests. If You want to run only minimum number of workers than use spare or backlog algorithms.
This plugin exist mostly because the way spare and backlog plugins works causes very aggressive scaling behavior, if You set low cheaper value (for example 1), than uWSGI will keep only 1 worker running and spawn new workers only when running worker is overloaded. If Your app requires more workers, than uWSGI will be spawning and stopping workers all the time, only during night time when load is very low minimum number of workers will be enough. Busyness algorithm tries to do the opposite: spawn as many workers as needed and stop some of them only when there is a good chance that they are not needed. This means more stable worker count and much less respawns, and since for most of the time we have little more workers than actually needed, average application respond times should be lower than with other plugins.
Options:
Specify intervals (in seconds) for tracking average busyness of workers. Example:
cheaper-overload = 30will to check busyness every 30 seconds, if during last 30 seconds all workers were running for 3 seconds and they were idle for remaining 27 seconds, than calculated busyness will be at 10% (3/30). This value will decide how fast uWSGI can respond to load spikes, new workers will be spawned at most every cheaper-overload seconds unless You are running uWSGI on linux (see cheaper-busyness-backlog-alert for details). If You want to react faster to load spikes than keep this value low, so that busyness will be calculated more often and proper action can be taken, but keep in mind that this might cause workers to be started/stopped more often since every minor spike might spawn new workers. With high cheaper-overload worker count will change much less since longer cycles will eat all short spikes of load and extreme values.
How many workers to spawn when any cheaper algorithm decide that it is needed. Default is 1.
How many workers should be started when starting application, after app is started cheaper algorithm can stop or start workers if needed.
This is maximum busyness we allow, every time current calculated busyness for last cheaper-overload seconds is higher than this value, than uWSGI will spawn new workers (cheaper-step value tells uWSGI how many workers will be spawned).
Default is 50.
This is minimum busyness, if current calculated busyness is below this value, than it is considered idle cycle and uWSGI will start counting. Once we reach needed number of subsequent idle cycles than uWSGI will cheap one worker. Default is 25.
This option tells uWSGI how many subsequent idle cycles we need before stopping (cheaping) one worker. After reaching required number of idle cycles and stopping one worker, we reset this counter so to stop next worker we need to wait the same amount of time.
Example:
cheaper-overload = 10
cheaper-busyness-multiplier = 20
cheaper-busyness-min = 25If average workers busyness is under 25% for 20 checks in a row, executed every 10 seconds (so we need to wait 200 seconds, 10*20), than one worker will be stopped. Idle cycles counter will be reset if average busyness jump above cheaper-busyness-max and we spawn new worker. If during idle cycle counting average busyness jumps above cheaper-busyness-min but still below cheaper-busyness-max, than idle cycles counter is adjusted and we need to wait extra one idle cycle. If during idle cycle counting average busyness jumps above cheaper-busyness-min but still below cheaper-busyness-max three times in a row, than we must reset idle cycle counter and start from scratch.
uWSGI will auto tune number of idle cycles needed to stop worker when worker is stopped due to enough idle cycles and than spawned back to fast (less than the same time we need to cheap worker), than we will increment the cheaper-busyness-multiplier value this value.
Default is 1.
Example:
cheaper-overload = 10
cheaper-busyness-multiplier = 20
cheaper-busyness-min = 25
cheaper-busyness-penalty = 2If average workers busyness is under 25% for 20 checks in a row, executed every 10 seconds (so we need to wait 200 seconds, 10*20), than one worker will be stopped. If new worker be spawned in less than 200 seconds (counting from the time when we spawned last worker), than cheaper-busyness-multiplier value will be incremented up to 22 (20+2). Now we will need to wait 220 seconds (22*10) to cheap another worker.
This option is used to prevent workers from being started and stopped all the time since once we stop one worker busyness might jump up enough to hit cheaper-busyness-max, and then new worker will be spawned, once we have new worker busyness will go down and another worker will be stopped.
This option will enable debug logs from cheaper_busyness plugin, enable them to debug and understand it. Default is false.
This option is only available on linux, it is used to allow quick response to load spikes even with high cheaper-overload values. Every uWSGI master cycle (1 second) current listen queue is checked, if it is bigger than this value than 1 emergency worker is spawned. When using this option it is safe to use high cheaper-overload values to have more smooth scaling of workers count.
Default is 33.
This option is only available on linux, it works just like cheaper-busyness-multiplier except that it is used only for emergency workers spawned when listen queue was higher than cheaper-busyness-backlog-alert. Emergency workers are spawned in case of big load spike to prevent currently running workers from being overloaded (it takes some time to spawn new workers due to high average busyness), and sometimes those load spike are random, short and they can spawn a lot of such workers. In such case we would need to wait many cycles before cheaping all those workers, so to cheap them faster we use different multiplier in such case.
Default is 3.
Notes:
experiment with settings, there is no one golden rule of what values should be used for everyone, test and pick values that are best for You, carbon stats will make it easy to decide so use them
don't expect busyness to be constant value, it will change a lot jumping up and down, real users interact with Your apps in very random way, it's recommended to use longer --cheaper-overload values (>=30) to have less spikes
if You want to run some benchmarks with this plugin than use tools that add randomness to the work load
with low number of workers (2-3) starting new worker or stopping one might affect busyness a lot, if You have 2 workers with busyness of 50%, than stopping one of them will increase busyness to 100%. Keep that in mind when picking min and max levels, with only few workers running most of the time max should be more than double of min, otherwise every time one worker is stopped it might increase busyness to above max level.
with low number of workers (1-4) and default settings expect that this plugin will keep average busyness below min level, adjust levels to compensate that
with higher number of workers required to handle load, workers count should stabilize somewhere near minimum busyness level, jumping a little bit around this value
when experimenting with this plugin it is advised to enable --cheaper-busyness-verbose to get an idea of what it is doing, example log below:
# These messages are logged at startup to show current settings [busyness] settings: min=20%, max=60%, overload=20, multiplier=15, respawn penalty=3 [busyness] backlog alert is set to 33 request(s) # With --cheaper-busyness-verbose enabled You can monitor calculated busyness [busyness] worker nr 1 20s average busyness is at 11% [busyness] worker nr 2 20s average busyness is at 11% [busyness] worker nr 3 20s average busyness is at 20% [busyness] 20s average busyness of 3 worker(s) is at 14% # Average busyness is under 20%, we start counting idle cycles # we have overload=20 and multiplier=15 so we need to wait 300 seconds before we can stop worker # cycle we just had was counted as idle so we need to wait another 280 seconds # 1 missing second below is just from rounding, master cycle is every 1 second but it also takes some time, this is normal [busyness] need to wait 279 more second(s) to cheap worker # We waited long enough and we can stop one worker [busyness] worker nr 1 20s average busyness is at 6% [busyness] worker nr 2 20s average busyness is at 22% [busyness] worker nr 3 20s average busyness is at 19% [busyness] 20s average busyness of 3 worker(s) is at 15% [busyness] 20s average busyness is at 15%, cheap one of 3 running workers # After stopping one worker average busyness is now higher, which is no surprise [busyness] worker nr 2 20s average busyness is at 36% [busyness] worker nr 3 20s average busyness is at 24% [busyness] 20s average busyness of 2 worker(s) is at 30% # 30% is above our minimum (20%), but it's still far from our maximum (60%) # since this is not idle cycle uWSGI will ignore it when counting when to stop worker [busyness] 20s average busyness is at 30%, 1 non-idle cycle(s), adjusting cheaper timer # After a while our average busyness is still low enough, so we stop another worker [busyness] 20s average busyness is at 3%, cheap one of 2 running workers # With only one worker running we won't see per worker busyness since it's the same as total average [busyness] 20s average busyness of 1 worker(s) is at 16% [busyness] 20s average busyness of 1 worker(s) is at 17% # Shortly after stopping second worker and with only one running we have load spike that is enough to hit our maximum level # this was just few cycles after stopping worker so uWSGI will increase multiplier # now we need to wait extra 3 cycles before stopping worker [busyness] worker(s) respawned to fast, increasing cheaper multiplier to 18 (+3) # Initially we needed to wait only 300 seconds, now we need to have 360 subsequent seconds when workers busyness is below minimum level # 10*20 + 3*20 = 360 [busyness] worker nr 1 20s average busyness is at 9% [busyness] worker nr 2 20s average busyness is at 17% [busyness] worker nr 3 20s average busyness is at 17% [busyness] worker nr 4 20s average busyness is at 21% [busyness] 20s average busyness of 4 worker(s) is at 16% [busyness] need to wait 339 more second(s) to cheap worker