Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add concurreny configuration #2569

Merged

Conversation

ruben-arts
Copy link
Contributor

@ruben-arts ruben-arts commented Nov 26, 2024

Why

Pixi can possibly use a big amount of memory during the solve or network requests during the repodata fetching. While we search for a better/automated solution we want to let the user escape the issue by forcing the amount of concurrent jobs.
The related issue is: #2458

What this PR adds

As a user you can now define the max concurrent solves and max network requests in two ways
CLI

pixi install --concurrent-solves 3
pixi install --concurrent-downloads 12

configuration

pixi config set concurrency.solves 1
pixi config set concurrency.downloads 12

config.toml

[concurrency]
solves = 2
downloads = 12

TODO:

After initial approval of design I'll add the following:

  • : Add documentation
  • : Add basic cli and configuration test to the integration tests

@wolfv
Copy link
Member

wolfv commented Nov 26, 2024

Should we add a concurrency table and then have solve, network, and filesystem entries or something along those lines? For the global config/

@ruben-arts
Copy link
Contributor Author

@wolfv merged in #2388

@ruben-arts ruben-arts changed the title feat: add max_concurrent_solves configuration feat: add concurreny configuration Nov 28, 2024
@wolfv
Copy link
Member

wolfv commented Nov 28, 2024

Thanks! I closed the other one :)

Copy link
Member

@wolfv wolfv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@ruben-arts ruben-arts enabled auto-merge (squash) November 28, 2024 15:51
@ruben-arts ruben-arts merged commit 514ea08 into prefix-dev:main Nov 28, 2024
45 checks passed
jjjermiah pushed a commit to jjjermiah/pixi that referenced this pull request Nov 30, 2024
### Why
Pixi can possibly use a big amount of memory during the solve or network
requests during the repodata fetching. While we search for a
better/automated solution we want to let the user escape the issue by
forcing the amount of concurrent jobs.
The related issue is: prefix-dev#2458

### What this PR adds
As a user you can now define the max concurrent solves and max network
requests in two ways
**CLI**
```
pixi install --concurrent-solves 3
pixi install --concurrent-downloads 12
```
**configuration**
```
pixi config set concurrency.solves 1
pixi config set concurrency.downloads 12
```
`config.toml`
```toml
[concurrency]
solves = 2
downloads = 12
```
### TODO:
After initial approval of design I'll add the following:
- [x] : Add documentation
- [x] : Add basic cli and configuration test to the integration tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants