Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ode_norm, ode_unstable_check #1113

Merged
merged 4 commits into from
Apr 7, 2022
Merged

ode_norm, ode_unstable_check #1113

merged 4 commits into from
Apr 7, 2022

Conversation

ranocha
Copy link
Member

@ranocha ranocha commented Apr 5, 2022 •
edited
Loading

An even simpler version of #1104.

TODO:

  • Check step rejections
  • Check some complex setups (MPI shock capturing does not use alpha smoothing! but everything else should work, incl. AMR)

julia --check-bounds=no --threads=4:

julia> trixi_include("examples/tree_2d_dgsem/elixir_euler_astro_jet_amr.jl", alpha_smooth=false)
[...]
────────────────────────────────────────────────────────────────────────────────────────────────────
Trixi simulation finished.  Final time: 0.001  Time steps: 6987 (accepted), 8764 (total)
────────────────────────────────────────────────────────────────────────────────────────────────────

 ──────────────────────────────────────────────────────────────────────────────────────────
                 Trixi.jl                         Time                    Allocations
                                         ───────────────────────   ────────────────────────
            Tot / % measured:                  178s /  89.5%           36.5GiB /  99.6%

 Section                         ncalls     time    %tot     avg     alloc    %tot      avg
 ──────────────────────────────────────────────────────────────────────────────────────────
 rhs!                             35.1k    91.3s   57.3%  2.60ms    182MiB    0.5%  5.31KiB
   volume integral                35.1k    68.7s   43.1%  1.96ms   31.7MiB    0.1%     947B
     blended DG-FV                35.1k    64.0s   40.2%  1.82ms   15.0MiB    0.0%     448B
     blending factors             35.1k    3.76s    2.4%   107μs   16.7MiB    0.0%     499B
     ~volume integral~            35.1k    913ms    0.6%  26.0μs   2.20KiB    0.0%    0.06B
     pure DG                      35.1k   4.12ms    0.0%   118ns     0.00B    0.0%    0.00B
   interface flux                 35.1k    6.19s    3.9%   177μs   15.0MiB    0.0%     448B
   prolong2interfaces             35.1k    4.52s    2.8%   129μs   10.2MiB    0.0%     304B
   surface integral               35.1k    3.91s    2.5%   111μs   12.3MiB    0.0%     368B
   reset ∂u/∂t                    35.1k    2.77s    1.7%  79.1μs     0.00B    0.0%    0.00B
   mortar flux                    35.1k    2.22s    1.4%  63.4μs   20.7MiB    0.1%     620B
   prolong2mortars                35.1k    1.82s    1.1%  51.9μs   14.9MiB    0.0%     445B
   Jacobian                       35.1k    750ms    0.5%  21.4μs   13.4MiB    0.0%     400B
   ~rhs!~                         35.1k    200ms    0.1%  5.70μs   16.1MiB    0.0%     480B
   boundary flux                  35.1k    133ms    0.1%  3.78μs   36.4MiB    0.1%  1.06KiB
   prolong2boundaries             35.1k   90.0ms    0.1%  2.57μs   11.2MiB    0.0%     336B
   source terms                   35.1k   1.95ms    0.0%  55.6ns     0.00B    0.0%    0.00B
 AMR                              6.99k    60.5s   38.0%  8.66ms   36.0GiB   99.1%  5.27MiB
   refine                         6.99k    29.3s   18.4%  4.19ms   13.7GiB   37.8%  2.01MiB
     mesh                         6.33k    16.8s   10.6%  2.66ms   82.0MiB    0.2%  13.3KiB
       refine_unbalanced!         6.33k    15.2s    9.6%  2.40ms   2.24MiB    0.0%     371B
       rebalance!                 10.8k    1.47s    0.9%   136μs   21.6MiB    0.1%  2.05KiB
       ~mesh~                     6.33k    139ms    0.1%  21.9μs   58.1MiB    0.2%  9.40KiB
     solver                       6.33k    12.4s    7.8%  1.96ms   13.7GiB   37.6%  2.21MiB
     ~refine~                     6.99k   46.3ms    0.0%  6.63μs   7.45MiB    0.0%  1.09KiB
   coarsen                        6.99k    28.9s   18.2%  4.14ms   21.4GiB   58.9%  3.13MiB
     mesh                         6.99k    15.0s    9.4%  2.15ms   47.8MiB    0.1%  7.00KiB
     solver                       6.99k    10.9s    6.8%  1.56ms   14.0GiB   38.5%  2.05MiB
     ~coarsen~                    6.99k    3.05s    1.9%   437μs   7.34GiB   20.2%  1.08MiB
   indicator                      6.99k    1.82s    1.1%   261μs    414MiB    1.1%  60.6KiB
   ~AMR~                          6.99k    462ms    0.3%  66.1μs    472MiB    1.3%  69.2KiB
 analyze solution                     3    3.66s    2.3%   1.22s   44.8MiB    0.1%  14.9MiB
 positivity-preserving limiter    35.1k    2.35s    1.5%  66.9μs   22.5MiB    0.1%     672B
 I/O                                  4    1.38s    0.9%   345ms   98.8MiB    0.3%  24.7MiB
   ~I/O~                              4    966ms    0.6%   241ms   57.0MiB    0.2%  14.3MiB
   save solution                      3    384ms    0.2%   128ms   37.7MiB    0.1%  12.6MiB
   get element variables              3   27.3ms    0.0%  9.09ms   3.10MiB    0.0%  1.03MiB
   save mesh                          3   2.97ms    0.0%   989μs    970KiB    0.0%   323KiB
 initial condition AMR                1    356μs    0.0%   356μs    297KiB    0.0%   297KiB
   AMR                                1    355μs    0.0%   355μs    296KiB    0.0%   296KiB
     indicator                        1    290μs    0.0%   290μs   97.4KiB    0.0%  97.4KiB
     ~AMR~                            1   64.1μs    0.0%  64.1μs    199KiB    0.0%   199KiB
     coarsen                          1    239ns    0.0%   239ns     64.0B    0.0%    64.0B
     refine                           1    149ns    0.0%   149ns     64.0B    0.0%    64.0B
   ~initial condition AMR~            1   1.05μs    0.0%  1.05μs      752B    0.0%     752B
 ──────────────────────────────────────────────────────────────────────────────────────────

tmpi 4 julia --check-bounds=no --threads=1 after modifying the file by adding
the keyword arguments internalnorm=ode_norm, unstable_check=ode_unstable_check
to solve:

julia> trixi_include("examples/tree_2d_dgsem/elixir_euler_astro_jet_amr.jl", alpha_smooth=false)
[...]
────────────────────────────────────────────────────────────────────────────────────────────────────
Trixi simulation finished.  Final time: 0.001  Time steps: 6987 (accepted), 8764 (total)
────────────────────────────────────────────────────────────────────────────────────────────────────

 ─────────────────────────────────────────────────────────────────────────────────────────────
                  Trixi.jl                           Time                    Allocations
                                            ───────────────────────   ────────────────────────
              Tot / % measured:                   185s /  94.4%           28.3GiB /  99.7%

 Section                            ncalls     time    %tot     avg     alloc    %tot      avg
 ─────────────────────────────────────────────────────────────────────────────────────────────
 rhs!                                35.1k    96.2s   55.1%  2.74ms    341MiB    1.2%  10.0KiB
   volume integral                   35.1k    73.8s   42.3%  2.10ms   59.3KiB    0.0%    1.73B
     blended DG-FV                   35.1k    70.3s   40.2%  2.00ms     0.00B    0.0%    0.00B
     blending factors                35.1k    3.25s    1.9%  92.6μs   25.1KiB    0.0%    0.73B
     ~volume integral~               35.1k    283ms    0.2%  8.08μs   34.3KiB    0.0%    1.00B
     pure DG                         35.1k   5.51ms    0.0%   157ns     0.00B    0.0%    0.00B
   interface flux                    35.1k    5.18s    3.0%   148μs     0.00B    0.0%    0.00B
   surface integral                  35.1k    4.64s    2.7%   132μs     0.00B    0.0%    0.00B
   prolong2interfaces                35.1k    4.02s    2.3%   115μs     0.00B    0.0%    0.00B
   prolong2mortars                   35.1k    1.80s    1.0%  51.5μs     0.00B    0.0%    0.00B
   mortar flux                       35.1k    1.77s    1.0%  50.4μs     0.00B    0.0%    0.00B
   reset ∂u/∂t                       35.1k    1.72s    1.0%  49.1μs     0.00B    0.0%    0.00B
   Jacobian                          35.1k    958ms    0.5%  27.3μs     0.00B    0.0%    0.00B
   finish MPI receive                35.1k    873ms    0.5%  24.9μs    234MiB    0.8%  6.84KiB
   start MPI send                    35.1k    322ms    0.2%  9.19μs   77.8MiB    0.3%  2.27KiB
   ~rhs!~                            35.1k    289ms    0.2%  8.25μs   16.6MiB    0.1%     496B
   boundary flux                     35.1k    155ms    0.1%  4.41μs     0.00B    0.0%    0.00B
   MPI mortar flux                   35.1k    135ms    0.1%  3.86μs     0.00B    0.0%    0.00B
   MPI interface flux                35.1k    127ms    0.1%  3.62μs     0.00B    0.0%    0.00B
   prolong2mpimortars                35.1k    108ms    0.1%  3.07μs     0.00B    0.0%    0.00B
   prolong2boundaries                35.1k   95.1ms    0.1%  2.71μs     0.00B    0.0%    0.00B
   prolong2mpiinterfaces             35.1k   71.5ms    0.0%  2.04μs     0.00B    0.0%    0.00B
   finish MPI send                   35.1k   56.7ms    0.0%  1.62μs   6.57MiB    0.0%     197B
   start MPI receive                 35.1k   53.9ms    0.0%  1.54μs   5.91MiB    0.0%     177B
   source terms                      35.1k   2.72ms    0.0%  77.6ns     0.00B    0.0%    0.00B
 AMR                                 6.99k    69.9s   40.1%  10.0ms   27.7GiB   97.9%  4.05MiB
   coarsen                           6.99k    31.2s   17.9%  4.47ms   12.0GiB   42.5%  1.76MiB
     mesh                            6.99k    24.1s   13.8%  3.45ms   47.8MiB    0.2%  7.00KiB
     solver                          6.99k    5.12s    2.9%   733μs   4.65GiB   16.5%   698KiB
     ~coarsen~                       6.99k    2.00s    1.1%   287μs   7.32GiB   25.9%  1.07MiB
   refine                            6.99k    24.6s   14.1%  3.52ms   4.65GiB   16.5%   698KiB
     mesh                            6.33k    17.9s   10.3%  2.83ms   82.0MiB    0.3%  13.3KiB
       refine_unbalanced!            6.33k    16.2s    9.3%  2.56ms   2.24MiB    0.0%     371B
       rebalance!                    10.8k    1.52s    0.9%   141μs   21.6MiB    0.1%  2.05KiB
       ~mesh~                        6.33k    185ms    0.1%  29.1μs   58.1MiB    0.2%  9.40KiB
     solver                          6.33k    6.68s    3.8%  1.05ms   4.57GiB   16.2%   757KiB
     ~refine~                        6.99k   26.0ms    0.0%  3.72μs   2.05MiB    0.0%     307B
   dynamic load balancing            6.41k    12.1s    6.9%  1.88ms   10.2GiB   36.2%  1.63MiB
     ~dynamic load balancing~        6.41k    4.97s    2.8%   775μs   9.01GiB   31.9%  1.44MiB
     exchange data                   5.40k    3.60s    2.1%   667μs   10.8MiB    0.0%  2.05KiB
     reinitialize data structures    5.40k    3.48s    2.0%   644μs   1.21GiB    4.3%   235KiB
   indicator                         6.99k    1.22s    0.7%   175μs    103MiB    0.4%  15.1KiB
   ~AMR~                             6.99k    824ms    0.5%   118μs    676MiB    2.3%  99.1KiB
 analyze solution                        3    5.16s    3.0%   1.72s    206MiB    0.7%  68.8MiB
 positivity-preserving limiter       35.1k    1.85s    1.1%  52.7μs     0.00B    0.0%    0.00B
 I/O                                     4    1.46s    0.8%   364ms   61.1MiB    0.2%  15.3MiB
   ~I/O~                                 4    928ms    0.5%   232ms   28.0MiB    0.1%  7.00MiB
   save solution                         3    493ms    0.3%   164ms   29.0MiB    0.1%  9.68MiB
   get element variables                 3   30.9ms    0.0%  10.3ms   3.09MiB    0.0%  1.03MiB
   save mesh                             3   3.04ms    0.0%  1.01ms    970KiB    0.0%   323KiB
 initial condition AMR                   1    295μs    0.0%   295μs    256KiB    0.0%   256KiB
   AMR                                   1    294μs    0.0%   294μs    256KiB    0.0%   256KiB
     indicator                           1    181μs    0.0%   181μs   24.3KiB    0.0%  24.3KiB
     ~AMR~                               1    112μs    0.0%   112μs    231KiB    0.0%   231KiB
     refine                              1    248ns    0.0%   248ns     64.0B    0.0%    64.0B
     coarsen                             1    170ns    0.0%   170ns     64.0B    0.0%    64.0B
   ~initial condition AMR~               1   1.09μs    0.0%  1.09μs      752B    0.0%     752B
 ─────────────────────────────────────────────────────────────────────────────────────────────

@ranocha ranocha requested a review from sloede April 5, 2022 17:29
@ranocha ranocha added the parallelization Related to MPI, threading, tasks etc. label Apr 5, 2022
sloede
sloede reviewed Apr 5, 2022
View reviewed changes
Copy link
Member

@sloede sloede left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this works, it would be a great way to gather some experience with using error-based step size control.

ranocha and others added 2 commits April 5, 2022 21:08
@ranocha ranocha marked this pull request as ready for review April 5, 2022 19:09
@ranocha ranocha requested a review from sloede April 5, 2022 19:09
sloede
sloede approved these changes Apr 5, 2022
View reviewed changes
@codecov
Copy link

codecov bot commented Apr 5, 2022 •
edited
Loading

Codecov Report

Merging #1113 (f471404) into main (84755ae) will decrease coverage by 0.01%.
The diff coverage is 81.25%.

@@            Coverage Diff             @@
##             main    #1113      +/-   ##
==========================================
- Coverage   96.75%   96.74%   -0.01%     
==========================================
  Files         303      303              
  Lines       23876    23892      +16     
==========================================
+ Hits        23099    23112      +13     
- Misses        777      780       +3
Flag Coverage Δ
unittests 96.74% <81.25%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/Trixi.jl 66.67% <ø> (ø)
src/auxiliary/mpi.jl 88.37% <81.25%> (-4.22%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 84755ae...f471404. Read the comment docs.

@ranocha ranocha mentioned this pull request Apr 7, 2022
Draft
4 tasks
@ranocha ranocha enabled auto-merge (squash) April 7, 2022 07:53
@ranocha ranocha merged commit bd39d5a into main Apr 7, 2022
@ranocha ranocha deleted the hr/mpi_ode_norm branch April 7, 2022 10:17
@sloede sloede mentioned this pull request Apr 23, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sloede sloede sloede approved these changes

Assignees
No one assigned
Labels
parallelization Related to MPI, threading, tasks etc.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ranocha @sloede