Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sharing one index between several processes, over time. #135

Closed
Adoni5 opened this issue Jul 6, 2018 · 2 comments
Closed

Sharing one index between several processes, over time. #135

Adoni5 opened this issue Jul 6, 2018 · 2 comments

Comments

@Adoni5
Copy link

Adoni5 commented Jul 6, 2018

Hi, I suppose this is less of issue and more of an open question.

I am trying to run several centrifuge processes over a period of time, but from only one index. So ideally I'd like to load the index and then have all the processes reference that one index during classification. I have limited memory and processing power, so I was looking at the --mm option. However I was a little unclear on how it works.

Does using the --mm flag mean that a centrifuge-class process loads the index into memory and then any subsequent classification process use that pre-loaded index once that process is finished, or do they share it in real time?

Once the classifications are finished I assume the index is removed from memory. Is there a way to leave it loaded?

Also, can someone explain how --mm works if the index doesn't fit into memory? Is the index placed into virtual memory instead?

I know that's a lot of questions, so I appreciate any help!

Thank you

Rory

@mourisl
Copy link
Collaborator

mourisl commented Jul 10, 2018

If you have root right, maybe you can try ramdisk as mentionedn in #65 .

@Adoni5
Copy link
Author

Adoni5 commented Jul 11, 2019

Coming back to this #95 was the solution I needed. Thank you very much, sorry I never closed this!

@Adoni5 Adoni5 closed this as completed Jul 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants