Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Operators work directly on raw arrays if most agents are active #843

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

RomeshA
Copy link
Contributor

@RomeshA RomeshA commented Jan 17, 2025

Description

Operators on Arr and BoolArr use asnew() to return new Arr instances. These work along the lines of

  • Using self.values to create a numpy array with length equal to auids
  • Creating a corresponding array from other.values if other is an Arr
  • Creating an intermediate result array with length equal to auids
  • Creating a new empty array the same size as uids
  • Populating it sparsely with the result array based on auids

The intention is that the alternative of operating on every UID (i.e., on the raw arrays directly) would be unnecessarily slow if there are many inactive agents. However, in practice the overhead associated with creating the intermediate arrays and then sparsely inserting it is fairly significant. For simulations that do not have a significant proportion of inactive agents, it can be faster to just operate on every entry. In that case, no intermediate arrays need to be created - the output of the logical operation of the two raw arrays can be directly used as the raw array for the new Arr instances. In some simulations this can give a speedup of around 40%.

This PR thus adds an attribute for Arr that corresponds to whether this optimization should be used, with a threshold of 50% (i.e., it will use the raw entries if over half the agents are active). This threshold could be tuned as we get more use cases. There is a corresponding update to asnew to optionally take in a raw array, which will be used if provided.

There should be no changes necessary in any user code and simulation output should be the same

Checklist

  • Code commented & docstrings added
  • New tests were needed and have been added
  • A new version number was needed & changelog has been updated
  • A new PyPI version needs to be released

@RomeshA RomeshA requested a review from cliffckerr January 17, 2025 04:54
@RomeshA RomeshA marked this pull request as draft January 17, 2025 07:40
@RomeshA
Copy link
Contributor Author

RomeshA commented Jan 17, 2025

Going to mark this as draft as other changes in our code seem to have made this change have less impact, will continue profiling further

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant