Balanced distribution of tasks among shards #2

[4meta5]

There is a tradeoff between the computational efficiency of assigning up to the task limit to the first shard online vs the socioeconomic benefit of dynamically assigning tasks to the least assigned shard (one at a time).

WIP Closes #998

• confirmed that task assignment is balanced iff the network task limit is set artificially low, see pallet/tasks test balanced_distribution_when_more_unassigned_tasks_than_task_limit
• confirmed that task assignment prior to this PR is/was greedy up until the limit set for the network, see pallet/tasks test first_shard_online_assigned_up_to_task_limit
• choose "solution" and implement it

[4meta5]

@penumbra23 @dvc94ch @0x1100010010 Part of the problem is that we sort the shards in order of least assigned tasks before iterating through them and assigning up to the task limit to each shard. This separation of sorting before batch assignment of tasks up to the limit via schedule_tasks_shard leads to the first shard being assigned the task limit number of tasks regardless of whether during its batch assignment it is assigned more tasks than other shards (the sorting is not weaved into the task assignment and this leads to the unequal distribution of tasks assigned to shards).

https://github.com/Analog-Labs/timechain/pull/928/files#r1673262307

proposed change 0

distribute tasks among ALL shards online evenly instead of greedy task assignment to the first shard assigned of up to its limit

impl draft:

1. add input qty to schedule_tasks_shard to replace ShardTasksLimit
2. get qty value passed into schedule_tasks_shard invocations by dividing ShardTasksLimit by the number of shards online

[4meta5]

superseded by #1013