Remove staged threads, immediately create thread objects #5843
Conversation
hkaiser
commented
Apr 11, 2022
hkaiser
commented
- flyby: more noexcept for combined_tagged_state
Replace std::distance with parallel::detail::distance Replace include/datapar.hpp with parallel/datapar.hpp
# Conflicts: # libs/core/hardware/include/hpx/hardware/cpuid/msvc.hpp
Support for data-parallelism for find algorithms
are being built as OBJECT libraries. In this case the fallback is to build STATIC (whole-archive) libraries instead.
5727: Moving compression plugins to components directory r=hkaiser a=hkaiser - flyby: removing obsolete #include files Co-authored-by: Hartmut Kaiser <[email protected]>
5743: Add check for MPICH and set the correct env to suport multi-threaded r=hkaiser a=diehlpk ## Proposed Changes - If MPICH is used we set `export MPICH_MAX_THREAD_SAFETY=multiple` to activate multithreaded MPI. ## Any background context you want to provide? This might solve some hanging, we experienced with MPI recently. Note, this has to happen before `MPI_Init` is being called. Co-authored-by: Patrick <[email protected]>
Remove obsolete files related to cpuid, etc.
Fixing spell-checking errors
Replace std::distance with parallel::detail::distance Replace include/datapar.hpp with parallel/datapar.hpp
Support for data-parallelism for adjacent find
Workaround for CMake/Ninja generator OOM problem
wait_xxx_nothrow functions return whether one of the futures is exceptional
hkaiser
force-pushed
the
no_staged_threads
branch
2 times, most recently
from
1337878 to
0db8264
Compare
- flyby: more noexcept for combined_tagged_state
hkaiser
force-pushed
the
no_staged_threads
branch
from
0db8264 to
25a961e
Compare
|
This might not be viable after all. All threads that are created immediately either use a cached thread object (with an allocated stack) or create a new thread object (without an associated stack) otherwise. This way, even threads that do not require immediate execution might end up with an allocated stack segment (if the thread object was retrieved from the cache). This is prone to causing memory issues as too many threads/stacks may be created (the tests here expose exactly that problem). For now, I don't have any fool-proof idea on how to decide whether a thread needs to run right away, so that it might benefit from being initialized from the thread object cache. The only solution I see is more intrusive: in addition (or instead of) caching the thread objects, we will have to cache the stack segments. That would allow to minimize the number of created stack segments to the minimal number of concurrently active threads. For now, I will convert this PR to 'daft' and get back to it after V1.8 has been released. |
|
What is the status of this PR? Should we push it for 1.10 instead? |
Yes, let's move this to the next milestone. |
