Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ELBDM/UniformGranule test problem #135

Open
wants to merge 15 commits into
base: psidm
Choose a base branch
from
Open
13 changes: 13 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/Input__Flag_Lohner
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Level Threshold_Refine Threshold_Derefine Filter Soften MinDensity
0 0.80 0.80 0.01 0.00 0.00
1 0.80 0.80 0.01 0.00 0.00
2 0.80 0.80 0.01 0.00 0.00
3 0.80 0.80 0.01 0.00 0.00
4 0.80 0.80 0.01 0.00 0.00
5 0.80 0.80 0.01 0.00 0.00
6 0.80 0.80 0.01 0.00 0.00
7 0.80 0.80 0.01 0.00 0.00
8 0.80 0.80 0.01 0.00 0.00
9 0.80 0.80 0.01 0.00 0.00
10 0.80 0.80 0.01 0.00 0.00
11 0.80 0.80 0.01 0.00 0.00
2 changes: 2 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/Input__Flag_Rho
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Level Density
0 9.0e-4
354 changes: 354 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/Input__Parameter

Large diffs are not rendered by default.

20 changes: 20 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/Input__TestProb
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# problem-specific runtime parameters
ComputeCorrelation 1 # compute correlation function, must set OPT__RECORD_USER=1 [0]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ComputeCorrelation 1 # compute correlation function, must set OPT__RECORD_USER=1 [0]
ComputeCorrelation 1 # compute correlation function (must set OPT__RECORD_USER=1) [0]

ReComputeCorrelation 0 # use the simulation time of RESTART as initial time for computing time correlation; only available for RESTART [0]
FilePath_corr ./Record__Correlation # output path for correlation function text files
MinLv 0 # do statistics from MinLv to MaxLv [0]
MaxLv 0 # do statistics from MinLv to MaxLv [MAX_LEVEL]
StepInitial 0 # inital step for recording correlation function [0]
StepInterval 1 # interval for recording correlation function (only for OutputCorrelationMode=0) [1
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
StepInterval 1 # interval for recording correlation function (only for OutputCorrelationMode=0) [1
StepInterval 1 # interval for recording correlation function (only for OutputCorrelationMode=0) [1]

StepEnd 200 # end step for recording correlation function (only for OutputCorrelationMode=0) [NoMax_int]

# parameters for correlation function profile (only useful when ComputeCorrelation=1)
RadiusMax_corr 0.04 # maximum radius of correlation profile [0.5*BoxSize]
dr_min_corr 0.00234224 # minimum bin size [1e-2*0.5*BoxSize]
LogBin_corr 1 # use log bin [0]
LogBinRatio_corr 1.1940617521534918 # exp((log(RadiusMax_corr)-log(dr_min_corr))/NBin) [2]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
LogBinRatio_corr 1.1940617521534918 # exp((log(RadiusMax_corr)-log(dr_min_corr))/NBin) [2]
LogBinRatio_corr 1.1940617521534918 # ratio of adjacent log bins [2]

RemoveEmpty_corr 0 # remove 0 sample bins; false: Data[empty_bin]=Weight[empty_bin]=NCell[empty_bin]=0 [0]

# parameters for initial density profile (only useful when ComputeCorrelation=1)
dr_min_prof 0.00058556 # minimum bin size [0.25*dr_min_corr]

Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# For loading an AMR initial condition from a file --> OPT__INIT=3 && OPT__UM_IC_NLEVEL>1
#
# dLv : target AMR level = OPT__UM_IC_LEVEL + dLv
# NP_Skip_xL: number of patches on the parent level to be skipped in the x direction
# from the left edge of the parent refinement region
# ==================================================================================
# dLv NP_Skip_xL NP_Skip_xR NP_Skip_yL NP_Skip_yR NP_Skip_zL NP_Skip_zR
1 3 3 3 3 3 3
2 6 6 6 6 6 6
3 19 19 19 19 19 19
32 changes: 32 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/README
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about mentioning that one must turn on OPT__RECORD_USER and set COM_* properly for ComputeCorrelation?

Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Compilation flags:
========================================
Enable : MODEL=ELBDM, GRAVITY, SUPPORT_HDF5, NCOMP_PASSIVE_USER=1
Disable : COMOVING


Default setup:
=======================================
0. Copy "generate_make.sh" to the directory "src/," execute "sh generate_make.sh" to generate "Makefile,"
and then execute "make clean" and "make -j 4" to generate the executable "gamer"
Comment on lines +9 to +10
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
0. Copy "generate_make.sh" to the directory "src/," execute "sh generate_make.sh" to generate "Makefile,"
and then execute "make clean" and "make -j 4" to generate the executable "gamer"
0. Copy "generate_make.sh" to the directory "src/", execute "sh generate_make.sh" to generate "Makefile,"
and then execute "make clean" and "make -j 4" to generate the executable "gamer"


1. Code units
(1) UNIT_L = kpc
(2) UNIT_V = km/s
(3) UNIT_D = 1.0e10 Msun/kpc^3
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(3) UNIT_D = 1.0e10 Msun/kpc^3
(3) UNIT_M = 1.0e10 Msun


2. ELBDM_MASS 3.0e-19
ELBDM_TAYLOR3_AUTO 0
ELBDM_BASE_SPECTRAL 1

3. OPT__BC_FLU_* 1 (periodic)
OPT__BC_POT 1 (periodic)


Default UM_IC info:
=======================================
BoxSize = 0.08 kpc
dimension = 128^3
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dimension = 128^3
dimension = 256^3

average density = 0.25e10 Msun/kpc^3
1-D velocity dispersion = 6 km/s


Empty file.
3 changes: 3 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/clean.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
rm -rf Record__* Data_* PowerSpec_* log
mkdir Record__Correlation
touch Record__Correlation/.empty
5 changes: 5 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/download_ic.sh
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you agree with increasing the default resolution to 256^3, upload a new UM_IC file.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
filename=uniform-granule-ic

curl https://girder.hub.yt/api/v1/item/66e0028e32f323dee1b801dc/download -o ${filename}.tgz
tar -zxvf ${filename}.tgz
rm ${filename}.tgz
6 changes: 6 additions & 0 deletions example/test_problem/ELBDM/UniformGranule/generate_make.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# This script should run in the same directory as configure.py

PYTHON=python3

${PYTHON} configure.py --machine=eureka_intel --mpi=true --hdf5=true --fftw=FFTW3 --gpu=true \
--model=ELBDM --gravity=true --passive=1
Comment on lines +5 to +6
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please merge the latest psidm branch, which supports a default --machine.

Suggested change
${PYTHON} configure.py --machine=eureka_intel --mpi=true --hdf5=true --fftw=FFTW3 --gpu=true \
--model=ELBDM --gravity=true --passive=1
${PYTHON} configure.py --mpi=true --hdf5=true --fftw=FFTW3 --gpu=true \
--model=ELBDM --gravity=true --passive=1 "$@"

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Support MPI parallelization. Check my comment on plot_slice.py.
  • Relatedly, improve output message organization when MPI is enabled. Specifically, refine messages like average density = ... so that they clearly indicate different times. Currently, it's difficult to distinguish between outputs from different times.

Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import yt
import numpy as np
import argparse
import sys

#-------------------------------------------------------------------------------------------------------------------------
# load the command-line parameters
parser = argparse.ArgumentParser( description='Get average velocity dispersion of the entire box' )

parser.add_argument( '-s', action='store', required=True, type=int, dest='idx_start',
help='first data index' )
parser.add_argument( '-e', action='store', required=True, type=int, dest='idx_end',
help='last data index' )
parser.add_argument( '-d', action='store', required=False, type=int, dest='didx',
help='delta data index [%(default)d]', default=1 )

args=parser.parse_args()

idx_start = args.idx_start
idx_end = args.idx_end
didx = args.didx

# print command-line parameters
print( '\nCommand-line arguments:' )
print( '-------------------------------------------------------------------' )
for t in range( len(sys.argv) ):
print( str(sys.argv[t]))
print( '' )
print( '-------------------------------------------------------------------\n' )

for idx in range(idx_start, idx_end+1, didx):
ds = yt.load("../Data_%06d"%idx)
N = ds.parameters["NX0"]
N_tot = N[0]*N[1]*N[2]
UNIT_L = ds.parameters["Unit_L"]
ma = ds.parameters["ELBDM_Mass"]
hbar = ds.parameters["ELBDM_PlanckConst"]
fac = hbar/ma

grad_Real = ds.add_gradient_fields(("Real"))
grad_Imag = ds.add_gradient_fields(("Imag"))

dd = ds.all_data()

if ( N_tot != len(dd["Dens"])):
print('file size not matched!')
sys.exit(1)

dens = np.array(dd["Dens"])
real = np.array(dd["Real"])
imag = np.array(dd["Imag"])
avedens = np.mean(dens)
print("average density = " + str(avedens))
print("minimum density = " + str(min(dens)))

grad_real = np.array([dd["Real_gradient_x"], dd["Real_gradient_y"], dd["Real_gradient_z"]])*UNIT_L
grad_imag = np.array([dd["Imag_gradient_x"], dd["Imag_gradient_y"], dd["Imag_gradient_z"]])*UNIT_L

vx_bk = fac*(imag*grad_real[0] - real*grad_imag[0])/dens
vy_bk = fac*(imag*grad_real[1] - real*grad_imag[1])/dens
vz_bk = fac*(imag*grad_real[2] - real*grad_imag[2])/dens

vx_qp = fac*(imag*grad_imag[0] + real*grad_real[0])/dens
vy_qp = fac*(imag*grad_imag[1] + real*grad_real[1])/dens
vz_qp = fac*(imag*grad_imag[2] + real*grad_real[2])/dens

sigma_x_sqr_bk = np.average(dens*vx_bk**2)/avedens - (np.average(dens*vx_bk))**2/avedens**2
sigma_y_sqr_bk = np.average(dens*vy_bk**2)/avedens - (np.average(dens*vy_bk))**2/avedens**2
sigma_z_sqr_bk = np.average(dens*vz_bk**2)/avedens - (np.average(dens*vz_bk))**2/avedens**2

sigma_x_sqr_qp = np.average(dens*vx_qp**2)/avedens - (np.average(dens*vx_qp))**2/avedens**2
sigma_y_sqr_qp = np.average(dens*vy_qp**2)/avedens - (np.average(dens*vy_qp))**2/avedens**2
sigma_z_sqr_qp = np.average(dens*vz_qp**2)/avedens - (np.average(dens*vz_qp))**2/avedens**2


sigma_bk = ((sigma_x_sqr_bk + sigma_y_sqr_bk +sigma_z_sqr_bk)/3.)**0.5
sigma_qp = ((sigma_x_sqr_qp + sigma_y_sqr_qp +sigma_z_sqr_qp)/3.)**0.5

sigma_total = (sigma_bk**2+sigma_qp**2)**0.5
print('bulk velocity dispersion = %g'%(sigma_bk))
print('quantum pressure velocity dispersion = %g'%(sigma_qp))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
print('quantum pressure velocity dispersion = %g'%(sigma_qp))
print('thermal velocity dispersion = %g'%(sigma_qp))

print('total velocity dispersion = %g'%(sigma_total))

d = 0.35*2*np.pi*hbar/(ma*sigma_total)
print('estimated granule diameter = %g\n'%(d))
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import numpy as np
import matplotlib.pyplot as plt
import glob
import re
import sys

file_dir = '../Record__Correlation/'
files = glob.glob(file_dir + 'correlation_function_t=*.txt')

filename = np.array(files)

r = []
C = []
t = []
for f in filename:
Corr = np.loadtxt(f, skiprows=1, dtype=float)
if not r:
r.append(Corr[:,0])
else:
if not np.array_equal(r[0], Corr[:,0]):
print('radius bin not matched!! filename=\"%s\"'%f)
sys.exit(1)

C.append(Corr[:,1])
match = re.search(r'correlation_function_t=(\d+\.\d+e[+-]\d+)', f)
if match:
time = float(match.group(1))
t.append(time)
else:
print('time pattern not matched!! filename=\"%s\"'%f)
sys.exit(1)

if not r:
print("no file loaded, check ../Record__Correlation/ !!")

t = np.array(t)*1000
C = np.array(C)

d = 0.00234224
sigma = 5.10172

# C(t) decay time scale, C(t_corr)=(1+2*0.35**2)**(-1.5)*C(t=0) = 0.72*C(t=0)
t_corr = d/(2**0.5*np.pi*sigma)*1000

color = plt.cm.turbo(np.linspace(0.1, 0.9, len(r[0])))

plt.figure()
for i in range(len(r[0])):
plt.plot(t, C[:,i], c=color[i], label = r'$r$ = %1.3e kpc'%(r[0][i]))
plt.axvline(x=t_corr, ls='--', lw = '0.9', color = '0.7', label = 'expected correlation time scale')
plt.xlabel('$t$ (Myr)')
plt.ylabel(r'$C(t)$')
plt.xlim(0, t[-1])
plt.legend(bbox_to_anchor=(1.03,0.03), loc='lower left')
plt.savefig('fig_correlation.png', dpi = 150, bbox_inches="tight")
plt.close()


Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support MPI parallelization. Check my comment on plot_slice.py. Note that you have to retrieve idx.

Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import numpy as np
import matplotlib.pyplot as plt
import yt
import argparse
import sys

#-------------------------------------------------------------------------------------------------------------------------
# load the command-line parameters
parser = argparse.ArgumentParser( description='Get power spectrum' )

parser.add_argument( '-s', action='store', required=True, type=int, dest='idx_start',
help='first data index' )
parser.add_argument( '-e', action='store', required=True, type=int, dest='idx_end',
help='last data index' )
parser.add_argument( '-d', action='store', required=False, type=int, dest='didx',
help='delta data index [%(default)d]', default=1 )

args=parser.parse_args()

idx_start = args.idx_start
idx_end = args.idx_end
didx = args.didx

# print command-line parameters
print( '\nCommand-line arguments:' )
print( '-------------------------------------------------------------------' )
for t in range( len(sys.argv) ):
print( str(sys.argv[t]))
print( '' )
print( '-------------------------------------------------------------------\n' )

# plot GAMER generated power spectrum as reference
GAMER_PS = True
print('GAMER_PS='+str(GAMER_PS)+'\n')

for idx in range(idx_start, idx_end+1, didx):
ds = yt.load("../Data_%06d"%idx)
N = ds.parameters["NX0"][0]
BoxSize = ds.parameters["BoxSize"][0]
dh = ds.parameters["CellSize"][0]

dd = ds.covering_grid(level=0, left_edge=[0, 0, 0], dims=ds.domain_dimensions)
rho = dd["Dens"].d
rhok = np.fft.rfftn( rho )

kx = 2*np.pi*np.fft.fftfreq( N, dh )
ky = 2*np.pi*np.fft.fftfreq( N, dh )
kz = 2*np.pi*np.fft.rfftfreq( N, dh )

Pk3d = abs(rhok)**2
Pk_total = np.zeros(N//2+1)
count = np.zeros(N//2+1)

for i in range( N ):
for j in range( N ):
for k in range( N//2+1 ):
l = round((kx[i]**2+ky[j]**2+kz[k]**2)**0.5*BoxSize/(2.0*np.pi))
if (l < N//2+1):
Pk_total[l] = Pk_total[l] + Pk3d[i,j,k]
count[l] = count[l] + 1
Pk1d = np.divide(Pk_total, count, out=np.zeros_like(Pk_total), where=count!=0)
k3Pk = Pk1d*kz**3
d = 2*np.pi/kz[np.argmax(k3Pk)]
print('estimated granule diameter = %g\n'%(d))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about showing the corresponding physical time to better distinguish the outputs from different MPI processes?


if(GAMER_PS):
gamer_ps = np.loadtxt('../PowerSpec_%06d'%idx, skiprows=1, dtype=float)
gamer_k = gamer_ps[:,0]
gamer_Pk1d = gamer_ps[:,1]
gamer_k3Pk = gamer_k**3*gamer_Pk1d

plt.figure()
plt.title("Dimensionless Power Spectrum")
plt.plot(kz[1:], k3Pk[1:]/max(k3Pk), label = 'numpy')
if(GAMER_PS):
plt.plot(gamer_k, gamer_k3Pk/max(gamer_k3Pk), label = 'gamer')
plt.xlabel('$k$')
plt.ylabel(r'$k^3P(k)$')
plt.yscale('log')
plt.legend(loc='lower right')
plt.savefig('fig_powerspectrum_dimensionless_%06d.png'%idx, dpi = 150, bbox_inches="tight")
plt.close()


Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support MPI parallelization by adding something like the following lines. Please refer to Plummer/plot_script/plot_gas.py.

yt.enable_parallelism()
ts = yt.DatasetSeries( [ prefix+'/Data_%06d'%idx for idx in range(idx_start, idx_end+1, didx) ] )
for ds in ts.piter():

Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import yt
import numpy as np
import argparse
import sys

#-------------------------------------------------------------------------------------------------------------------------
# load the command-line parameters
parser = argparse.ArgumentParser( description='Get disk properties' )

parser.add_argument( '-s', action='store', required=True, type=int, dest='idx_start',
help='first data index' )
parser.add_argument( '-e', action='store', required=True, type=int, dest='idx_end',
help='last data index' )
parser.add_argument( '-d', action='store', required=False, type=int, dest='didx',
help='delta data index [%(default)d]', default=1 )

args=parser.parse_args()

idx_start = args.idx_start
idx_end = args.idx_end
didx = args.didx

# print command-line parameters
print( '\nCommand-line arguments:' )
print( '-------------------------------------------------------------------' )
for t in range( len(sys.argv) ):
print( str(sys.argv[t]))
print( '' )
print( '-------------------------------------------------------------------\n' )

field = 'Dens'
colormap = 'algae'
#colormap = 'magma'
dpi = 300


for idx in range(idx_start, idx_end+1, didx):
ds = yt.load("../Data_%06d"%idx)
dd = ds.all_data()
dens = np.array(dd["Dens"])
avedens = np.mean(dens)

plt = yt.SlicePlot( ds, 0, fields = field, center = 'c')
plt.set_zlim( field, avedens*1.0e-4, avedens*1.0e+1, dynamic_range=None)
plt.set_axes_unit( 'kpc' )
plt.set_unit( field, 'Msun/kpc**3')
plt.set_cmap( field, colormap )
plt.annotate_timestamp( time_unit='Myr', corner='upper_right', text_args={'color':'k'} )
plt.save( mpl_kwargs={"dpi":dpi} )

plt = yt.SlicePlot( ds, 1, fields = field, center = 'c')
plt.set_zlim( field, avedens*1.0e-4, avedens*1.0e+1, dynamic_range=None)
plt.set_axes_unit( 'kpc' )
plt.set_unit( field, 'Msun/kpc**3')
plt.set_cmap( field, colormap )
plt.annotate_timestamp( time_unit='Myr', corner='upper_right', text_args={'color':'k'} )
plt.save( mpl_kwargs={"dpi":dpi} )

plt = yt.SlicePlot( ds, 2, fields = field, center = 'c')
plt.set_zlim( field, avedens*1.0e-4, avedens*1.0e+1, dynamic_range=None)
plt.set_axes_unit( 'kpc' )
plt.set_unit( field, 'Msun/kpc**3')
plt.set_cmap( field, colormap )
plt.annotate_timestamp( time_unit='Myr', corner='upper_right', text_args={'color':'k'} )
plt.save( mpl_kwargs={"dpi":dpi} )


Loading