-
-
-
- 
-
- Probabilistic Estimation of Losses, Injuries, and Community resilience Under Natural disasters + Probabilistic Estimation of Losses, Injuries, and Community resilience Under Natural hazard events
## What is it? -`pelicun` is a Python package that provides tools for assessment of damage and losses due to natural hazards. It uses a stochastic damage and loss model that is based on the methodology described in FEMA P58 (FEMA, 2012). While FEMA P58 aims to assess the seismic performance of a building, with `pelicun` we provide a more versatile, hazard-agnostic tool that estimates losses for several types of assets in the built environment. +`pelicun` is a Python package that provides tools for assessment of damage and losses due to natural hazard events. It uses a stochastic damage and loss model that is an extension of the high-resolution PEER performance assessment methodology described in FEMA P58 (FEMA, 2012). While FEMA P58 aims to assess the seismic performance of a building, with `pelicun` we provide a more versatile, hazard-agnostic tool to assess the performance of several types of assets in the built environment. Detailed documentation of the available methods and their use is available at http://nheri-simcenter.github.io/pelicun @@ -28,27 +28,27 @@ Detailed documentation of the available methods and their use is available at ht - **Estimate component damages.** Given a damage and loss model and the joint distribution of EDPs, `pelicun` provides methods to estimate the amount of damaged components and the number of cases with collapse. -- **Estimate consequences.** Using information about collapse and component damages, the following consequences can be estimated with the loss model: reconstruction cost and time, unsafe placarding (red tag), injuries and fatalities. +- **Estimate consequences.** Using information about collapse and component damages, the following consequences can be estimated with the loss model: reconstruction cost and time, unsafe placarding (red tag), injuries and fatalities. ## Why should I use it? -1. It is free and it always will be. +1. It is free and it always will be. 2. It is open source. You can always see what is happening under the hood. 3. It is efficient. The loss assessment calculations in `pelicun` use `numpy`, `scipy`, and `pandas` libraries to efficiently propagate uncertainties and provide detailed results quickly. -4. You can trust it. Every function in `pelicun` is tested after every commit. See the Travis-CI and Coveralls badges at the top for more info. +4. You can trust it. Every function in `pelicun` is tested after every commit. See the Travis-CI and Coveralls badges at the top for more info. 5. You can extend it. If you have other methods that you consider better than the ones we already offer, we encourage you to fork the repo, and extend `pelicun` with your approach. You do not need to share your extended version with the community, but if you are interested in doing so, contact us and we are more than happy to merge your version with the official release. ## Requirements `pelicun` runs under Python 3.6+ . The following packages are required for it to work properly: -- `numpy` >= 1.19.0 +- `numpy` >= 1.21.0 -- `scipy` >= 1.5.0 +- `scipy` >= 1.7.0 -- `pandas` >= 1.1.0 +- `pandas` >= 1.3.0 -We recommend installing these using `pip`. +We recommend installing each of these using `pip`. For convenience, we also published a nheri_simcenter python package that gets all of these and a few additional dependencies installed to initialize the Python environment for SimCenter tools. ## Installation @@ -58,8 +58,44 @@ We recommend installing these using `pip`. pip install pelicun ``` +Although v3.0 of pelicun is released here on Github, it is currently meant for testing by advanced, power-users. With v3.0, pelicun received a major architectural update and we want to wait a few weeks to collect feedback from users and prepare a set of rigorous tests before releasing a build on PyPI for the general researcher community. Until then, pip will keep installing the previous version (v2.6). If you are interested in using v3.0, you can download the source files from GitHub and install it using setup.py. + ## Changelog +### Changes in v3.0 + +* The architecture was redesigned to better support interactive calculation and provide a low-level integration across all supported methods. This is the first release with the new architecture. Frequent updates are planned to provide additional examples, tests, and bugfixes in the next few months. + +* New `assessment` module introduced to replace `control` module: + * Provides a high-level access to models and their methods + * Integrates all types of assessments into a uniform approach + * Most of the methods from the earlier `control` module were moved to the `model` module + +* Decoupled demand, damage, and loss calculations: + * Fragility functions and consequence functions are stored in separate files. Added new methods to the `db` module to prepare the corresponding data files and re-generated such data for FEMA P58 and Hazus earthquake assessments. Hazus hurricane data will be added in a future release. + * Decoupling removed a large amount of redundant data from supporting databases and made the use of HDF and json files for such data unnecessary. All data are stored in easy-to-read csv files. + * Assessment workflows can include all three steps (i.e., demand, damage, and loss) or only one or two steps. For example, damage estimates from one analysis can drive loss calculations in another one. + +* Integrated damage and loss calculation across all methods and components: + * This includes phenomena such as collapse, including various collapse modes, and irreparable damage. + * Cascading damages and other interdependencies between various components can be introduced using a damage process file. + * Losses can be driven by damages or demands. The former supports the conventional damage->consequence function approach, while the latter supports the use of vulnerability functions. These can be combined within the same analysis, if needed. + * The same loss component can be driven by multiple types of damages. For example, replacement can be triggered by either collapse or irreparable damage. + +* Introduced *Options* in the configuration file and in the `base` module: + * These options handle settings that concern pelicun behavior; + * general preferences that might affect multiple assessment models; + * and settings that users would not want to change frequently. + * Default settings are provided in a `default_config.json` file. These can be overridden by providing any of the prescribed keys with a user-defined value assigned to them in the configuration file for an analysis. + +* Introduced consistent handling of units. Each csv table has a standard column to describe units of the data in it. If the standard column is missing, the table is assumed to use SI units. + +* Introduced consistent handling of pandas MultiIndex objects in headers and indexes. When tabular data is stored in csv files, MultiIndex objects are converted to simple indexes by concatenating the strings at each level and separating them with a `-`. This facilitates post-processing csv files in pandas without impeding post-processing those files in non-Python environments. + +* Updated the DL_calculation script to support the new architecture. Currently, only the config file input is used. Other arguments were kept in the script for backwards compatibility; future updates will remove some of those arguments and introduce new ones. + +* The log files were redesigned to provide more legible and easy-to-read information about the assessment. + ### Changes in v2.6 * Support EDPs with more than 3 characters and/or a variable in their name. For example, SA_1.0 or SA_T1 @@ -73,7 +109,7 @@ pip install pelicun ## Acknowledgement -This material is based upon work supported by the National Science Foundation under Grant No. 1612843. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. +This material is based upon work supported by the National Science Foundation under Grants No. 1612843 2131111. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. ## Contact diff --git a/doc_src/Makefile b/doc_src/Makefile deleted file mode 100644 index f684cd4b2..000000000 --- a/doc_src/Makefile +++ /dev/null @@ -1,61 +0,0 @@ -# Minimal makefile for Sphinx documentation -# - -# You can set these variables from the command line, and also -# from the environment for the first two. -SPHINXOPTS ?= -SPHINXBUILD ?= sphinx-build -#SPHINXBUILD ?= python3 -m sphinx -SOURCEDIR = . -BUILDDIR = ./web - -# Put it first so that "make" without argument is like "make help". -help: - @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) - -RDT: - make clean - rm -fr ../../RDT/docs/_sources - rm -fr ../../RDT/docs/_static - rm -fr ../../RDT/docs/_images - rm -fr ../../RDT/docs/common - @$(SPHINXBUILD) -b html "$(SOURCEDIR)" ../../PBE-Documentation/docs $(O) - -PBE: - make clean - rm -fr ../.../PBE-Documentation/docs/_sources - rm -fr ../../PBE-Documentation/docs/_static - rm -fr ../../PBE-Documentation/docs/_images - rm -fr ../../PBE-Documentation/docs/common - @$(SPHINXBUILD) -b html "$(SOURCEDIR)" ../../PBE-Documentation/docs $(O) - -quoFEM: - make clean - rm -fr ../.../quoFEM-Documentation/docs/_sources - rm -fr ../../quoFEM-Documentation/docs/_static - rm -fr ../../quoFEM-Documentation/docs/_images - rm -fr ../../quoFEM-Documentation/docs/common - @$(SPHINXBUILD) -b html "$(SOURCEDIR)" ../../quoFEM-Documentation/docs $(O) - -EE: - make clean - rm -fr ../../EE-UQ-Documentation/docs/_sources - rm -fr ../../EE-UQ-Documentation/docs/_static - rm -fr ../../EE-UQ-Documentation/docs/_images - rm -fr ../../EE-UQ-Documentation/docs/common - @$(SPHINXBUILD) -b html "$(SOURCEDIR)" ../../EE-UQ-Documentation/docs $(O) - -WE: - make clean - rm -fr ../../WE-UQ-Documentation/docs/_sources - rm -fr ../../WE-UQ-Documentation/docs/_static - rm -fr ../../WE-UQ-Documentation/docs/_images - rm -fr ../../WE-UQ-Documentation/docs/common - @$(SPHINXBUILD) -b html "$(SOURCEDIR)" ../../WE-UQ-Documentation/docs $(O) - -.PHONY: help Makefile - -# Catch-all target: route all unknown targets to Sphinx using the new -# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). -%: Makefile - @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/doc_src/_static/css/custom.css b/doc_src/_static/css/custom.css deleted file mode 100644 index 8884404a2..000000000 --- a/doc_src/_static/css/custom.css +++ /dev/null @@ -1,34 +0,0 @@ -wy-nav-content { - max-width: none; -} - - -.math { - text-align: left; -} - -.eqno { - float: right; -} - - -#div.wy-side-scroll{ -# background:#cb463f; -#} - -div.wy-menu.wy-menu-vertical > .caption { - color: #cb463f; -} - -# LIGHT BLUE background:#0099ff -# BLUE: background:#0B619C -# ADAM RED: background:#cb463f; - -span.caption.caption-text{ - color: #000000; -} - -td{ - white-space: normal !important; -} - diff --git a/doc_src/_static/css/fmkPlay.css b/doc_src/_static/css/fmkPlay.css deleted file mode 100644 index a55a15f96..000000000 --- a/doc_src/_static/css/fmkPlay.css +++ /dev/null @@ -1,19 +0,0 @@ -div.wy-side-scroll{ - background:#cb463f; -} - -# LIGHT BLUE background:#0099ff -# BLUE: background:#0B619C -# RED: background:#cb463f; - -#span.caption-text{ -# color: #000000; -#} - -#.wy-menu-vertical header, .wy-menu-vertical p.caption { -# color: #000000; -#} - -td{ - white-space: normal !important; -} \ No newline at end of file diff --git a/doc_src/_static/plots/plotstyle.mplstyle b/doc_src/_static/plots/plotstyle.mplstyle deleted file mode 100644 index 6d6dd74e4..000000000 --- a/doc_src/_static/plots/plotstyle.mplstyle +++ /dev/null @@ -1,62 +0,0 @@ -# Claudio Perez -# claudio_perez@berkeley.edu -# SimCenter plotting style sheet - -## Default figure size -# figure.figsize : 3.5, 2.625 -figure.dpi: 600 - -## AXES -axes.facecolor : ffffff # axes background color -axes.edgecolor : bcbcbc # axes edge color -axes.grid : True -axes.axisbelow : True -axes.prop_cycle : cycler('color', ['CA463E', '136AC9','48A59E','999999', 'cb463f']) -# axes.linewidth : 0.5 - -# axes3d.facecolor : 'w' - -## GRID -grid.color : eeeeee #ffffff -grid.alpha : 1.0 -grid.linestyle : dotted # solid, dashed, dashdot, dotted, none, ' ', '' -# grid.linewidth : 0.5 - -## X axis -# xtick.direction : in -# xtick.major.size : 3 -# xtick.major.width : 0.5 -# xtick.minor.size : 1.5 -# xtick.minor.width : 0.5 -# xtick.minor.visible : True -# xtick.top : True - -## Y axis -# ytick.direction : in -# ytick.major.size : 3 -# ytick.major.width : 0.5 -# ytick.minor.size : 1.5 -# ytick.minor.width : 0.5 -# ytick.minor.visible : True -# ytick.right : True - -## Lines -# lines.linewidth : 1. -lines.markersize: 6 - -# LEGEND -legend.frameon : False - -# Save Layout -savefig.bbox : tight -savefig.pad_inches : 0.05 - -# FONT -font.serif : Computer Modern -font.family : serif -font.weight : light -mathtext.fontset : cm - -# LaTeX -text.usetex : True -text.latex.preamble : \usepackage{amsmath} \ No newline at end of file diff --git a/doc_src/common/developer_manual/.DS_Store b/doc_src/common/developer_manual/.DS_Store deleted file mode 100644 index 422ac4049..000000000 Binary files a/doc_src/common/developer_manual/.DS_Store and /dev/null differ diff --git a/doc_src/common/developer_manual/API/pelicun/API.rst b/doc_src/common/developer_manual/API/pelicun/API.rst deleted file mode 100644 index a0cd66acb..000000000 --- a/doc_src/common/developer_manual/API/pelicun/API.rst +++ /dev/null @@ -1,21 +0,0 @@ -.. _lbl_API: - -================= -API Documentation -================= - -.. automodule:: pelicun - :members: - :no-undoc-members: - :show-inheritance: - -.. toctree:: - :maxdepth: 1 - - auto
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-This module has classes and methods that auto-populate DL models.
-
-.. rubric:: Contents
-
-.. autosummary::
-
- auto_populate
-
-"""
-
-from .base import *
-import importlib
-import json
-from pathlib import Path
-
-ap_DesignLevel = {
- 1940: 'Pre-Code',
- 1940: 'Low-Code',
- 1975: 'Moderate-Code',
- 2100: 'High-Code'
-}
-
-ap_DesignLevel_W1 = {
- 0: 'Pre-Code',
- 0: 'Low-Code',
- 1975: 'Moderate-Code',
- 2100: 'High-Code'
-}
-
-ap_Occupancy = {
- 'Other/Unknown': 'RES3',
- 'Residential - Single-Family': 'RES1',
- 'Residential - Town-Home': 'RES3',
- 'Residential - Multi-Family': 'RES3',
- 'Residential - Mixed Use': 'RES3',
- 'Office': 'COM4',
- 'Hotel': 'RES4',
- 'School': 'EDU1',
- 'Industrial - Light': 'IND2',
- 'Industrial - Warehouse': 'IND2',
- 'Industrial - Heavy': 'IND1',
- 'Retail': 'COM1',
- 'Parking' : 'COM10'
-}
-
-convert_design_level = {
- 'High-Code' : 'HC',
- 'Moderate-Code': 'MC',
- 'Low-Code' : 'LC',
- 'Pre-Code' : 'PC'
- }
-
-def story_scale(stories, comp_type):
- if comp_type == 'NSA':
- if stories == 1:
- return 1.00
- elif stories == 2:
- return 1.22
- elif stories == 3:
- return 1.40
- elif stories == 4:
- return 1.45
- elif stories == 5:
- return 1.50
- elif stories == 6:
- return 1.90
- elif stories == 7:
- return 2.05
- elif stories == 8:
- return 2.15
- elif stories == 9:
- return 2.20
- elif (stories >= 10) and (stories < 30):
- return 2.30 + (stories-10)*0.04
- elif stories >= 30:
- return 3.10
- else:
- return 1.0
-
- elif comp_type in ['S', 'NSD']:
- if stories == 1:
- return 1.45
- elif stories == 2:
- return 1.90
- elif stories == 3:
- return 2.50
- elif stories == 4:
- return 2.75
- elif stories == 5:
- return 3.00
- elif stories == 6:
- return 3.50
- elif stories == 7:
- return 3.50
- elif stories == 8:
- return 3.50
- elif stories == 9:
- return 4.50
- elif (stories >= 10) and (stories < 50):
- return 4.50 + (stories-10)*0.07
- elif stories >= 50:
- return 7.30
- else:
- return 1.0
-
-[docs]def auto_populate(DL_input_path, EDP_input_path,
- DL_method, realization_count, coupled_EDP, event_time,
- ground_failure, auto_script_path = None):
- """
- Short description
-
- Assumptions:
- - The BIM is stored under 'GeneralInformation' or 'GI' in the root of
- DL_input
-
- Parameters
- ----------
- DL_input_path:
-
- Returns
- -------
- DL_input
- DL_ap_path
- """
-
-
- # load the available DL input information
- with open(DL_input_path, 'r') as f:
- DL_input = json.load(f)
-
- # get the BIM data
- BIM = DL_input.get('GeneralInformation', None)
- if BIM is None:
- raise ValueError(
- "No Building Information provided for the auto-population routine."
- )
-
- if auto_script_path is not None: # if an external auto pop script is provided
-
- # load the module
- ASP = Path(auto_script_path).resolve()
- sys.path.insert(0, str(ASP.parent)+'/')
- auto_script = importlib.__import__(ASP.name[:-3], globals(), locals(), [], 0)
- auto_populate_ext = auto_script.auto_populate
-
- # generate the DL input data
- BIM_ap, DL_ap = auto_populate_ext(BIM = BIM)
-
- # add the response model information
- DL_ap.update({
- 'ResponseModel': {
- 'ResponseDescription': {
- 'EDP_Distribution': 'empirical',
- 'Realizations' : realization_count
- }
- }
- })
-
- # add the even time information - if needed
- if event_time is not None:
- DL_ap['LossModel']['Inhabitants'].update({'EventTime': event_time})
-
- # assemble the extended DL input
- DL_input.update({'BIM_Inferred': BIM_ap})
- DL_input.update({'DamageAndLoss': DL_ap})
-
- # save it to the DL file with the ap suffix
- DL_ap_path = DL_input_path[:-5] + '_ap.json'
-
- with open(DL_ap_path, 'w') as f:
- json.dump(DL_input, f, indent=2)
-
- # and also return these information
- return DL_input, DL_ap_path
-
- else: # otherwise, use the old autopop method
-
- EDP_input = pd.read_csv(EDP_input_path, sep='\s+', header=0,
- index_col=0)
-
- is_IM_based = DL_method[-2:] == 'IM'
-
- stories = BIM['NumberofStories']
- # use only 1 story if DM is based on IM
- if DL_method == 'HAZUS MH EQ IM':
- stories = 1
- BIM.update({'NumberofStories':stories})
-
- # HAZUS Earthquake
- if DL_method in ['HAZUS MH EQ', 'HAZUS MH EQ IM']:
-
- bt = BIM['StructureType']
-
- if bt == 'RV.structType':
- bt = EDP_input['structType'].values[0]
-
- year_built = BIM['YearBuilt']
-
- if bt not in ['W1', 'W2', 'S3', 'PC1', 'MH']:
- if bt not in ['URM']:
- if stories <= 3:
- bt += 'L'
- elif stories <= 7:
- bt += 'M'
- else:
- if bt in ['RM']:
- bt += 'M'
- else:
- bt += 'H'
- else:
- if stories <= 2:
- bt += 'L'
- else:
- bt += 'M'
-
- if BIM['OccupancyClass'] in ap_Occupancy.keys():
- ot = ap_Occupancy[BIM['OccupancyClass']]
- else:
- ot = BIM['OccupancyClass']
-
- replacementCost = BIM.get('ReplacementCost', 1.0)
- replacementTime = BIM.get('ReplacementTime', 1.0)
- population = BIM.get('Population', 1.0)
-
- loss_dict = {
- '_method': DL_method,
- 'DamageModel': {
- 'StructureType': bt
- },
- 'LossModel': {
- 'DecisionVariables': {
- 'ReconstructionCost': True,
- 'ReconstructionTime': True,
- 'Injuries': True
- },
- 'Inhabitants': {
- 'OccupancyType': ot,
- 'PeakPopulation': f'{population}'
- },
- 'ReplacementCost': replacementCost,
- 'ReplacementTime': replacementTime
- },
- 'ResponseModel': {
- 'ResponseDescription': {
- 'Realizations': realization_count,
- "CoupledAssessment": coupled_EDP
- }
- },
- "Dependencies": {
- "Fragilities": "btw. Performance Groups"
- }
- }
-
- # add uncertainty if the EDPs are not coupled
- if not coupled_EDP:
- loss_dict['ResponseModel'].update({
- "AdditionalUncertainty": {
- "GroundMotion": "0.10",
- "Modeling" : "0.20"
- }})
-
- if is_IM_based:
- loss_dict.update({
- "ComponentDataFolder": pelicun_path+"/resources/HAZUS_MH_2.1_EQ_eqv_PGA.hdf"
- })
- else:
- loss_dict['ResponseModel'].update({
- 'DetectionLimits': {
- "PFA": "100.0",
- "PID": "0.20",
- "PRD": "0.20"
- }})
- loss_dict.update({
- "ComponentDataFolder": pelicun_path+"/resources/HAZUS_MH_2.1_EQ_story.hdf"
- })
-
- if 'W1' in bt:
- DesignL = ap_DesignLevel_W1
- else:
- DesignL = ap_DesignLevel
-
- for year in sorted(DesignL.keys()):
- if year_built <= year:
- loss_dict['DamageModel'].update(
- {'DesignLevel': DesignL[year]})
- break
-
- dl = convert_design_level[loss_dict['DamageModel']['DesignLevel']]
- if 'C3' in bt:
- if dl not in ['LC', 'PC']:
- dl = 'LC'
-
- # only one structural component for IM-based approach
- if is_IM_based:
-
- FG_S = f'S-{bt}-{dl}-{ot}'
-
- loss_dict.update({
- 'Components': {
- FG_S: [
- {'location': '1',
- 'direction': '1',
- 'median_quantity': '1.0',
- 'unit': 'ea',
- 'distribution': 'N/A'
- }]
- }})
-
- # story-based approach
- else:
-
- FG_S = f'S-{bt}-{dl}-{ot}'
- FG_NSD = f'NSD-{ot}'
- FG_NSA = f'NSA-{dl}-{ot}'
-
- loss_dict.update({
- 'Components': {
- FG_S: [
- {'location': 'all',
- 'direction': '1, 2',
- #'median_quantity': '{q}'.format(q = 0.5), #/stories),
- 'median_quantity': '{q}'.format(q = story_scale(stories, 'S')/stories/2.),
- 'unit': 'ea',
- 'distribution': 'N/A'
- }],
- FG_NSA: [
- {'location': 'all',
- 'direction': '1',
- #'median_quantity': '{q}'.format(q = 1.0), #/stories),
- 'median_quantity': '{q}'.format(q = story_scale(stories, 'NSA')/stories),
- 'unit': 'ea',
- 'distribution': 'N/A'
- }],
- FG_NSD: [
- {'location': 'all',
- 'direction': '1, 2',
- #'median_quantity': '{q}'.format(q = 0.5), #/stories),
- 'median_quantity': '{q}'.format(q = story_scale(stories, 'NSD')/stories/2.),
- 'unit': 'ea',
- 'distribution': 'N/A'
- }]
- }})
-
- # if damage from ground failure is included
- if ground_failure:
-
- foundation_type = 'S'
-
- FG_GF_H = f'GF-H_{foundation_type}-{bt}'
- FG_GF_V = f'GF-V_{foundation_type}-{bt}'
-
- loss_dict['Components'].update({
- FG_GF_H: [
- {'location': '1',
- 'direction': '1',
- 'median_quantity': '1.0',
- 'unit': 'ea',
- 'distribution': 'N/A'
- }],
- FG_GF_V: [
- {'location': '1',
- 'direction': '3',
- 'median_quantity': '1.0',
- 'unit': 'ea',
- 'distribution': 'N/A'
- }]
- })
-
- # define logic that connects ground failure with building damage
- loss_dict.update({
- 'DamageLogic': [
- {'type': 'propagate',
- 'source_FG': FG_GF_H,
- 'target_FG': FG_S,
- 'DS_links': {
- '1_1': '3_1',
- '2_1': '4_1',
- '2_2': '4_2'
- }
- },
- {'type': 'propagate',
- 'source_FG': FG_GF_V,
- 'target_FG': FG_S,
- 'DS_links': {
- '1_1': '3_1',
- '2_1': '4_1',
- '2_2': '4_2'
- }
- }
- ]
- })
-
- # HAZUS Hurricane
- elif DL_method == 'HAZUS MH HU':
-
- #TODO: use the HU NJ autopop script by default
- pass
-
- elif DL_method == 'FEMA P58':
- if BIM.get('AssetType',None) == 'Water_Pipe':
-
- material = BIM['Material']
-
- if material in ['Asbestos cement', 'Cast iron']:
- # brittle material
- config = 'P0001a'
- else:
- # ductile material
- config = 'P0001b'
-
- segment_count = BIM['SegmentCount']
- segment_length = BIM['Segments'][0]['length']
- cg_count = int(segment_length / (100 * ft))
- quantities = '1'
- for s in range(1, cg_count):
- quantities += ', 1'
-
- loss_dict = {
- "_method" : "FEMA P58",
- "ResponseModel" : {
- "ResponseDescription": {
- "EDP_Distribution" : "empirical",
- "Realizations" : "1000", # need to fix this later
- "CoupledAssessment": True
- }
- },
- "DamageModel" : {
- "CollapseProbability": {
- "Value": "0.0",
- },
- },
- "LossModel" : {
- "ReplacementCost" : "1",
- "ReplacementTime" : "180",
- "DecisionVariables": {
- "Injuries" : False,
- "ReconstructionCost": True,
- "ReconstructionTime": True,
- "RedTag" : False
- },
- },
- "Dependencies" : {
- "CostAndTime" : True,
- "Fragilities" : "btw. Damage States",
- "Quantities" : "Independent",
- "ReconstructionCosts": "Independent",
- "ReconstructionTimes": "Independent",
- },
- "ComponentDataFolder": "c:/Adam/Dropbox/Kutatas/2019 SC Testbeds/Memphis/",
- "Components" : {
- config: [
- {
- "location" : "all",
- "direction" : "1",
- "median_quantity": quantities,
- "unit" : "ea",
- "distribution" : "N/A",
- }
- ],
- }
- }
-
- if event_time is not None:
- loss_dict['LossModel']['Inhabitants'].update({'EventTime': event_time})
-
- DL_input.update({'DamageAndLoss':loss_dict})
-
- DL_ap_path = DL_input_path[:-5]+'_ap.json'
-
- with open(DL_ap_path, 'w') as f:
- json.dump(DL_input, f, indent = 2)
-
- return DL_input, DL_ap_path
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-This module defines constants, basic classes and methods for pelicun.
-
-"""
-
-import os, sys, time
-import warnings
-from datetime import datetime
-from time import strftime
-from copy import deepcopy
-
-# import libraries for other modules
-import numpy as np
-import pandas as pd
-
-idx = pd.IndexSlice
-
-# set printing options
-import pprint
-pp = pprint.PrettyPrinter(indent=4, width=300)
-
-pd.options.display.max_rows = 20
-pd.options.display.max_columns = None
-pd.options.display.expand_frame_repr = True
-pd.options.display.width = 300
-
-log_file = None
-
-log_div = '-' * (80-21) # 21 to have a total length of 80 with the time added
-
-# get the absolute path of the pelicun directory
-pelicun_path = os.path.dirname(os.path.abspath(__file__))
-
-# print a matrix in a nice way using a DataFrame
-def show_matrix(data, describe=False):
- if describe:
- pp.pprint(pd.DataFrame(data).describe(percentiles=[0.01,0.1,0.5,0.9,0.99]))
- else:
- pp.pprint(pd.DataFrame(data))
-
-# Monkeypatch warnings to get prettier messages
-def _warning(message, category, filename, lineno, file=None, line=None):
- if '\\' in filename:
- file_path = filename.split('\\')
- elif '/' in filename:
- file_path = filename.split('/')
- python_file = '/'.join(file_path[-3:])
- print('WARNING in {} at line {}\n{}\n'.format(python_file, lineno, message))
-warnings.showwarning = _warning
-
-def show_warning(warning_msg):
- warnings.warn(UserWarning(warning_msg))
-
-def set_log_file(filepath):
- globals()['log_file'] = filepath
- with open(filepath, 'w') as f:
- f.write('pelicun\n')
-
-[docs]def log_msg(msg='', prepend_timestamp=True):
- """
- Print a message to the screen with the current time as prefix
-
- The time is in ISO-8601 format, e.g. 2018-06-16T20:24:04Z
-
- Parameters
- ----------
- msg: string
- Message to print.
-
- """
- if prepend_timestamp:
- formatted_msg = '{} {}'.format(datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S:%fZ')[:-4], msg)
- else:
- formatted_msg = msg
-
- #print(formatted_msg)
-
- if globals()['log_file'] is not None:
- with open(globals()['log_file'], 'a') as f:
- f.write('\n'+formatted_msg)
-
-def describe(df):
-
- if isinstance(df, (pd.Series, pd.DataFrame)):
- vals = df.values
- if isinstance(df, pd.DataFrame):
- cols = df.columns
- elif df.name is not None:
- cols = df.name
- else:
- cols = 0
- else:
- vals = df
- cols = np.arange(vals.shape[1]) if vals.ndim > 1 else 0
-
- if vals.ndim == 1:
- df_10, df_50, df_90 = np.nanpercentile(vals, [10, 50, 90])
- desc = pd.Series({
- 'count': np.sum(~np.isnan(vals)),
- 'mean': np.nanmean(vals),
- 'std': np.nanstd(vals),
- 'min': np.nanmin(vals),
- '10%': df_10,
- '50%': df_50,
- '90%': df_90,
- 'max': np.nanmax(vals),
- }, name=cols)
- else:
- df_10, df_50, df_90 = np.nanpercentile(vals, [10, 50, 90], axis=0)
- desc = pd.DataFrame({
- 'count': np.sum(~np.isnan(vals), axis=0),
- 'mean': np.nanmean(vals, axis=0),
- 'std': np.nanstd(vals, axis=0),
- 'min': np.nanmin(vals, axis=0),
- '10%': df_10,
- '50%': df_50,
- '90%': df_90,
- 'max': np.nanmax(vals, axis=0),
- }, index=cols).T
-
- return desc
-
-def str2bool(v):
- # courtesy of Maxim @ stackoverflow
-
- if isinstance(v, bool):
- return v
- if v.lower() in ('yes', 'true', 'True', 't', 'y', '1'):
- return True
- elif v.lower() in ('no', 'false', 'False', 'f', 'n', '0'):
- return False
- else:
- raise argparse.ArgumentTypeError('Boolean value expected.')
-
-# Constants for unit conversion
-
-# time
-sec = 1.
-
-minute = 60. * sec
-h = 60. * minute
-day = 24. * h
-
-sec2 = sec**2.
-
-# distance, area, volume
-m = 1.
-
-mm = 0.001 * m
-cm = 0.01 * m
-km = 1000. * m
-
-inch = 0.0254
-ft = 12. * inch
-mile = 5280. * ft
-
-# area
-m2 = m**2.
-
-mm2 = mm**2.
-cm2 = cm**2.
-km2 = km**2.
-
-inch2 = inch**2.
-ft2 = ft**2.
-mile2 = mile**2.
-
-# volume
-m3 = m**3.
-
-inch3 = inch**3.
-ft3 = ft**3.
-
-
-# speed / velocity
-cmps = cm / sec
-mps = m / sec
-mph = mile / h
-
-inchps = inch / sec
-ftps = ft / sec
-
-# acceleration
-mps2 = m / sec2
-
-inchps2 = inch / sec2
-ftps2 = ft / sec2
-
-g = 9.80665 * mps2
-
-# mass
-kg = 1.
-
-ton = 1000. * kg
-
-lb = 0.453592 * kg
-
-# force
-N = kg * m / sec2
-
-kN = 1e3 * N
-
-lbf = lb * g
-kip = 1000. * lbf
-kips = kip
-
-# pressure / stress
-Pa = N / m2
-
-kPa = 1e3 * Pa
-MPa = 1e6 * Pa
-GPa = 1e9 * Pa
-
-psi = lbf / inch2
-ksi = 1e3 * psi
-Mpsi = 1e6 * psi
-
-# misc
-A = 1.
-
-V = 1.
-kV = 1000. * V
-
-ea = 1.
-
-rad = 1.
-
-C = 1.
-
-# FEMA P58 specific
-#TODO: work around these and make them available only in the parser methods
-EA = ea
-SF = ft2
-LF = ft
-TN = ton
-AP = A
-CF = ft3 / minute
-KV = kV * A
-
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-# Pouria Kourehpaz
-
-"""
-This module has classes and methods that control the performance assessment.
-
-.. rubric:: Contents
-
-.. autosummary::
-
- Assessment
- FEMA_P58_Assessment
- HAZUS_Assessment
-
-"""
-
-from .base import *
-from .uq import *
-from .model import *
-from .file_io import *
-
-
-[docs]class Assessment(object):
- """
- A high-level class that collects features common to all supported loss
- assessment methods. This class will only rarely be called directly when
- using pelicun.
- """
-
- def __init__(self, log_file=True):
-
- # initialize the basic data containers
- # inputs
- self._AIM_in = None
- self._EDP_in = None
- self._POP_in = None
- self._FG_in = None
-
- # random variables and loss model
- self._RV_reg = RandomVariableRegistry() # object to manage RVs
- self._RV_dict = None # dictionary to store random variables
- self._EDP_dict = None
- self._QNT_dict = None
- self._FF_dict = None
- self._DSG_dict = None
- self._FG_dict = None
- self._DV_RED_dict = None
- self._DV_REP_dict = None
- self._DV_INJ_dict = None
-
- # results
- self._TIME = None
- self._POP = None
- self._COL = None
- self._ID_dict = None
- self._DMG = None
- self._DV_dict = None
- self._SUMMARY = None
-
- self._assessment_type = 'generic'
-
- # initialize the log file
- if log_file:
- set_log_file('pelicun_log.txt')
-
- log_msg(log_div)
- log_msg('Assessement Started')
- log_msg(log_div)
-
- @property
- def beta_tot(self):
- """
- Calculate the total additional uncertainty for post processing.
-
- The total additional uncertainty is the squared root of sum of squared
- uncertainties corresponding to ground motion and modeling.
-
- Returns
- -------
- beta_total: float
- The total uncertainty (logarithmic EDP standard deviation) to add
- to the EDP distribution. Returns None if no additional uncertainty
- is assigned.
- """
-
- AU = self._AIM_in['general']['added_uncertainty']
-
- beta_total = 0.
- if AU['beta_m'] is not None:
- beta_total += AU['beta_m'] ** 2.
- if AU['beta_gm'] is not None:
- beta_total += AU['beta_gm'] ** 2.
-
- # if no uncertainty is assigned, we return None
- if beta_total == 0:
- beta_total = None
- else:
- beta_total = np.sqrt(beta_total)
-
- return beta_total
-
-[docs] def read_inputs(self, path_DL_input, path_EDP_input, verbose=False):
- """
- Read and process the input files to describe the loss assessment task.
-
- Parameters
- ----------
- path_DL_input: string
- Location of the Damage and Loss input file. The file is expected to
- be a JSON with data stored in a standard format described in detail
- in the Input section of the documentation.
- path_EDP_input: string
- Location of the EDP input file. The file is expected to follow the
- output formatting of Dakota. The Input section of the documentation
- provides more information about the expected formatting.
- verbose: boolean, default: False
- If True, the method echoes the information read from the files.
- This can be useful to ensure that the information in the file is
- properly read by the method.
- """
-
- # read SimCenter inputs -----------------------------------------------
- log_msg(log_div)
- log_msg('Reading inputs...')
-
- # BIM file
- log_msg('\tBIM file...')
- self._AIM_in = read_SimCenter_DL_input(
- path_DL_input, assessment_type=self._assessment_type,
- verbose=verbose)
-
- data = self._AIM_in
-
- log_msg()
- log_msg('\t\tGlobal attributes / settings:')
- for att in ['stories', 'coupled_assessment', 'realizations']:
- log_msg('\t\t\t{}: {}'.format(att, data['general'][att]))
-
- log_msg()
- log_msg('\t\tPrescribed Decision Variables:')
- for dv, val in data['decision_variables'].items():
- if val:
- log_msg('\t\t\t{}'.format(dv))
-
- log_msg()
- log_msg("\t\tDamage and Loss Data Dir:")
- log_msg('\t\t\t{}'.format(data['data_sources']['path_CMP_data']))
-
- if data['decision_variables']['injuries']:
- log_msg()
- log_msg("\t\tPopulation Data Dir:")
- log_msg('\t\t\t{}'.format(data['data_sources']['path_POP_data']))
-
- log_msg()
- log_msg('\t\tUnits:')
- for dv, val in data['unit_names'].items():
- log_msg('\t\t\t{}: {} ({})'.format(dv, val, data['units'][dv]))
-
- log_msg()
- log_msg('\t\tResponse Model:')
- log_msg('\t\t\tDetection Limits:')
- for dl_name, dl in data['general']['detection_limits'].items():
- log_msg('\t\t\t\t{}: {}'.format(dl_name, dl))
- for att, val in data['general']['response'].items():
- log_msg()
- log_msg('\t\t\t{}: {}'.format(att, val))
- log_msg()
- log_msg('\t\t\tAdditional Uncertainty:')
- for att, val in data['general']['added_uncertainty'].items():
- log_msg('\t\t\t\t{}: {}'.format(att, val))
-
- log_msg()
- log_msg('\t\tPerformance Model:')
- log_msg('\t\t\t\tloc\tdir\tqnt\tdist\tcov\tcgw')
- for comp_id, comp_data in data['components'].items():
- log_msg('\t\t{} [{}]:'.format(comp_id, comp_data['unit']))
- if False: #TODO: control this with a verbose flag
- for i in range(len(comp_data['locations'])):
- log_msg('\t\t\t\t{}\t{}\t{}\t{}\t{}\t{}'.format(*[comp_data[att][i] for att in ['locations', 'directions', 'quantities', 'distribution', 'cov', 'csg_weights']]))
-
- log_msg()
- log_msg('\t\tDamage Model:')
- if self._assessment_type == 'P58':
- log_msg('\t\t\tCollapse Limits:')
- for cl_name, cl in data['general']['collapse_limits'].items():
- log_msg('\t\t\t\t{}: {}'.format(cl_name, cl))
- log_msg()
- log_msg('\t\t\tIrreparable Residual Drift:')
- if 'irreparable_res_drift' in data['general']:
- for att, val in data['general']['irreparable_res_drift'].items():
- log_msg('\t\t\t\t{}: {}'.format(att, val))
- else:
- log_msg('\t\t\t\tnot considered')
- log_msg()
- log_msg('\t\t\tCollapse Probability:')
- if data['general']['response']['coll_prob'] == 'estimated':
- log_msg('\t\t\t\tEstimated based on {}'.format(data['general']['response']['CP_est_basis']))
- else:
- log_msg('\t\t\t\tPrescribed: {}'.format(data['general']['response']['coll_prob']))
-
- log_msg()
- log_msg('\t\tLoss Model:')
- for att in ['replacement_cost', 'replacement_time', 'population']:
- if att in data['general'].keys():
- log_msg('\t\t\t{}: {}'.format(att, data['general'][att]))
-
- log_msg()
- log_msg('\t\tCollapse Modes:')
- for cmode, cmode_data in data['collapse_modes'].items():
- log_msg('\t\t\t{}'.format(cmode))
- for att, val in cmode_data.items():
- log_msg('\t\t\t {}: {}'.format(att, val))
-
- log_msg()
- log_msg('\t\tDependencies:')
- for att, val in data['dependencies'].items():
- log_msg('\t\t\t{}: {}'.format(att, val))
-
- # EDP file
- log_msg('\tEDP file...')
- if self._hazard == 'EQ':
- self._EDP_in = read_SimCenter_EDP_input(
- path_EDP_input,
- EDP_kinds=('PID', 'PRD', 'RID', 'PFA', 'PMD',
- 'PGA', 'PGV', 'SA', 'SV', 'SD',
- 'PGD','DWD', 'RDR'),
- units=dict(PID=1.,
- PRD=1.,
- RID=1.,
- DWD=1.,
- RDR=1.,
- PFA=self._AIM_in['units']['acceleration'],
- PMD=1.,
- PGA=self._AIM_in['units']['acceleration'],
- PGV=self._AIM_in['units']['speed'],
- SA =self._AIM_in['units']['acceleration'],
- SV =self._AIM_in['units']['speed'],
- SD =self._AIM_in['units']['length'],
- PGD = self._AIM_in['units']['length']),
- verbose=verbose)
- elif self._hazard == 'HU':
- self._EDP_in = read_SimCenter_EDP_input(
- path_EDP_input, EDP_kinds=('PWS',),
- units=dict(PWS=self._AIM_in['units']['speed']),
- verbose=verbose)
-
- data = self._EDP_in
-
- log_msg('\t\tEDP types:')
- for EDP_kind in data.keys():
- log_msg('\t\t\t{}'.format(EDP_kind))
- for EDP_data in data[EDP_kind]:
- if False: #TODO: control this with a verbose flag
- log_msg('\t\t\t\t{} {}'.format(EDP_data['location'], EDP_data['direction']))
-
- log_msg()
- log_msg('\t\tnumber of samples: {}'.format(len(data[list(data.keys())[0]][0]['raw_data'])))
-
-[docs] def define_random_variables(self):
- """
- Define the random variables used for loss assessment.
-
- """
- log_msg(log_div)
- log_msg('Defining random variables...')
-
-[docs] def define_loss_model(self):
- """
- Create the stochastic loss model based on the inputs provided earlier.
-
- """
- log_msg(log_div)
- log_msg('Creating the damage and loss model...')
-
-[docs] def calculate_damage(self):
- """
- Characterize the damage experienced in each random event realization.
-
- """
- log_msg(log_div)
- log_msg('Calculating damage...')
- self._ID_dict = {}
-
-[docs] def calculate_losses(self):
- """
- Characterize the consequences of damage in each random event realization.
-
- """
- log_msg(log_div)
- log_msg('Calculating losses...')
- self._DV_dict = {}
-
-[docs] def save_outputs(self, output_path, EDP_file, DM_file, DV_file,
- suffix="", detailed_results=True):
- """
- Export the results.
-
- """
- def replace_FG_IDs_with_FG_names(df):
- FG_list = sorted(self._FG_dict.keys())
- new_col_names = dict(
- (fg_id, fg_name) for (fg_id, fg_name) in
- zip(np.arange(1, len(FG_list) + 1), FG_list))
-
- return df.rename(columns=new_col_names)
-
- log_msg(log_div)
- log_msg('Saving outputs...')
-
- log_msg('\tConverting EDP samples to input units...')
- EDPs = sorted(self._EDP_dict.keys())
- EDP_samples = pd.DataFrame({edp: self._EDP_dict[edp].samples for edp in EDPs})
- #EDP_samples = self._EDP_dict[EDPs[0]].samples_DF.copy()
- cols = EDP_samples.columns
- col_info = [col.split('-') for col in cols]
- EDP_samples.columns = [
- '1-{}-{}-{}'.format(col[1], col[3], col[5]) for col in col_info]
-
- # TODO: use some global vars to identify EDP units because this is a mess
- for col_i, col in enumerate(cols):
- if ('PFA' in col) or ('PGA' in col) or ('SA' in col):
- scale_factor = self._AIM_in['units']['acceleration']
- elif (('PFV' in col) or ('PGV' in col) or ('SV' in col) or
- ('PWS' in col)):
- scale_factor = self._AIM_in['units']['speed']
- elif ('PGD' in col):
- scale_factor = self._AIM_in['units']['length']
- else:
- scale_factor = 1.0
-
- if scale_factor != 1.0:
- EDP_samples.iloc[:, col_i] = EDP_samples.iloc[:, col_i].div(scale_factor)
-
- log_msg('\tConverting damaged quantities to input units...')
- DMG_scaled = self._DMG.copy()
- cols = DMG_scaled.columns.get_level_values(0)
- FG_list = sorted(self._FG_dict.keys())
- for col_i, col in enumerate(cols):
- FG_name = FG_list[col-1]
- scale_factor = self._FG_dict[FG_name]._unit
- if scale_factor != 1.0:
- DMG_scaled.iloc[:,col_i] = DMG_scaled.iloc[:,col_i].div(scale_factor)
-
- log_msg('\tReplacing headers with FG names...')
- DMG_mod = replace_FG_IDs_with_FG_names(DMG_scaled)
- DV_mods, DV_names = [], []
- for key in self._DV_dict.keys():
- if key != 'injuries':
- DV_mods.append(replace_FG_IDs_with_FG_names(self._DV_dict[key]))
- DV_names.append('{}DV_{}'.format(suffix, key))
- else:
- for i in range(2 if self._assessment_type == 'P58' else 4):
- DV_mods.append(replace_FG_IDs_with_FG_names(self._DV_dict[key][i]))
- DV_names.append('{}DV_{}_{}'.format(suffix, key, i+1))
-
- try:
- #if True:
- log_msg('\tSaving files:')
-
- log_msg('\t\tSummary')
- write_SimCenter_DL_output(
- output_path, '{}DL_summary.csv'.format(suffix),
- self._SUMMARY, index_name='#Num', collapse_columns=True)
-
- if not detailed_results:
- log_msg('\t\tOnly saving the main results.')
- else:
- log_msg('\t\tSummary statistics')
- write_SimCenter_DL_output(
- output_path, '{}DL_summary_stats.csv'.format(suffix),
- self._SUMMARY, index_name='attribute', collapse_columns=True,
- stats_only=True)
-
- log_msg('\t\tEDP values')
- write_SimCenter_DL_output(
- output_path, '{}EDP_.csv'.format(suffix),
- EDP_samples, index_name='#Num',
- collapse_columns=False)
-
- log_msg('\t\tEDP statistics')
- write_SimCenter_DL_output(
- output_path, '{}EDP_stats.csv'.format(suffix),
- EDP_samples, index_name='#Num',
- collapse_columns=False, stats_only=True)
-
- log_msg('\t\tDamaged quantities')
- write_SimCenter_DL_output(
- output_path, '{}DMG.csv'.format(suffix),
- DMG_mod, index_name='#Num', collapse_columns=False)
-
- log_msg('\t\tDamage statistics')
- write_SimCenter_DL_output(
- output_path, '{}DMG_stats.csv'.format(suffix),
- DMG_mod, index_name='#Num',
- collapse_columns=False, stats_only=True)
-
- log_msg('\t\tDamaged quantities - aggregated')
- write_SimCenter_DL_output(
- output_path, '{}DMG_agg.csv'.format(suffix),
- DMG_mod.T.groupby(level=0).aggregate(np.sum).T,
- index_name='#Num', collapse_columns=False)
-
- for DV_mod, DV_name in zip(DV_mods, DV_names):
- log_msg('\t\tDecision variable {}'.format(DV_name))
- write_SimCenter_DL_output(
- output_path, '{}{}.csv'.format(suffix, DV_name),
- DV_mod, index_name='#Num', collapse_columns=False)
-
- DV_mod_agg = DV_mod.T.groupby(level=0).aggregate(np.sum).T
-
- log_msg('\t\tDecision variable {} - aggregated'.format(DV_name))
- write_SimCenter_DL_output(
- output_path, '{}{}_agg.csv'.format(suffix, DV_name),
- DV_mod_agg, index_name='#Num', collapse_columns=False)
-
- log_msg('\t\tAggregated statistics for {}'.format(DV_name))
- write_SimCenter_DL_output(
- output_path, '{}{}_agg_stats.csv'.format(suffix, DV_name),
- DV_mod_agg, index_name='#Num', collapse_columns=False,
- stats_only=True)
-
- #if True:
- # create the EDP file
- if self._assessment_type.startswith('HAZUS'):
- log_msg('\t\tSimCenter EDP file')
- write_SimCenter_EDP_output(
- output_path, suffix + EDP_file,
- EDP_samples)
-
- # create the DM file
- if self._assessment_type.startswith('HAZUS'):
- log_msg('\t\tSimCenter DM file')
- write_SimCenter_DM_output(
- output_path, suffix+DM_file, self._SUMMARY,
- DMG_mod)
-
- # create the DV file
- if self._assessment_type.startswith('HAZUS'):
- log_msg('\t\tSimCenter DV file')
- write_SimCenter_DV_output(
- output_path, suffix+DV_file, self._AIM_in['general'],
- self._SUMMARY, dict(zip(DV_names, DV_mods)))
-
- except:
- print("ERROR when trying to create DL output files.")
-
- def _create_RV_demands(self):
-
- # Unlike other random variables, the demand RV is based on raw data.
-
- # First, collect the raw values from the EDP dict...
- demand_data = []
- d_tags = []
- detection_limits = []
- collapse_limits = []
- GI = self._AIM_in['general']
- s_edp_keys = sorted(self._EDP_in.keys())
-
- for d_id in s_edp_keys:
- d_list = self._EDP_in[d_id]
- for i in range(len(d_list)):
- demand_data.append(d_list[i]['raw_data'])
- d_tags.append(f"EDP-{d_id}-LOC-{d_list[i]['location']}" +
- f"-DIR-{d_list[i]['direction']}")
-
- det_lim = GI['detection_limits'].get(d_id, None)
- if det_lim is None:
- det_lim = np.inf
-
- if GI['response']['EDP_dist_basis'] == 'non-collapse results':
- coll_lim = GI['collapse_limits'][d_id]
- if coll_lim is None:
- coll_lim = np.inf
- elif GI['response']['EDP_dist_basis'] == 'all results':
- coll_lim = np.inf
-
- detection_limits.append([None, det_lim])
- collapse_limits.append([None, coll_lim])
-
- # detection_limits = np.transpose(np.asarray(detection_limits))
- # collapse_limits = np.transpose(np.asarray(collapse_limits))
- # demand_data = np.transpose(np.asarray(demand_data))
- detection_limits = np.asarray(detection_limits)
- collapse_limits = np.asarray(collapse_limits)
- demand_data = np.atleast_2d(demand_data)
-
- # In a coupled assessment only the raw data needs to be stored
- if GI['coupled_assessment']:
-
- # Create the random variable
- # demand_RV = RandomVariable(ID=200, dimension_tags=d_tags,
- # raw_data=np.transpose(demand_data))
- [self._RV_reg.add_RV(
- RandomVariable(name=d_tag, distribution='coupled_empirical',
- raw_samples=demand_data[i]))
- for i,d_tag in enumerate(d_tags)]
-
- # Otherwise, a distribution is fit to the raw data
- else:
- # If more than one sample is available...
- if demand_data.shape[1] > 1:
-
- # Second, we discard the collapsed EDPs if the fitted
- # distribution shall represent non-collapse EDPs.
- demand_data_T = demand_data.T
- # EDP_filter = np.all(
- # [np.all(demand_data_T > collapse_limits.T[0], axis=1),
- # np.all(demand_data_T < collapse_limits.T[1], axis=1)], axis=0)
- EDP_filter = np.all(demand_data_T < collapse_limits.T[1], axis=1)
- demand_data_T = demand_data_T[EDP_filter]
-
- log_msg(
- '\t\t{} considered collapsed out of {} raw samples'.format(
- list(EDP_filter).count(False), len(EDP_filter)))
-
- # Third, we censor the EDPs that are beyond the detection limit.
- # EDP_filter = np.all(
- # [np.all(demand_data_T > detection_limits.T[0], axis=1),
- # np.all(demand_data_T < detection_limits.T[1], axis=1)], axis=0)
- EDP_filter = np.all(demand_data_T < detection_limits.T[1], axis=1)
-
- log_msg(
- '\t\t{} are beyond the detection limits out of {} '
- 'non-collapse samples'.format(
- list(EDP_filter).count(False), len(EDP_filter)))
-
- censored_count = len(EDP_filter) - sum(EDP_filter)
- demand_data_T = demand_data_T[EDP_filter]
- demand_data = demand_data_T.T
-
- log_msg('\t\tNumber of EDP dimensions: {}'.format(len(d_tags)))
-
- # Fourth, if requested, we fit a multivariate lognormal
- # distribution to the data and create the random variables
- target_dist = GI['response']['EDP_distribution']
-
- if target_dist == 'lognormal':
- if censored_count > 0:
- log_msg('\t\tFitting a censored multivariate lognormal '
- 'distribution to EDP samples...')
- EDP_theta, EDP_rho = fit_distribution(
- demand_data, 'lognormal',
- censored_count = censored_count,
- detection_limits = detection_limits)
-
- else:
- log_msg('\t\tFitting a multivariate lognormal '
- 'distribution to EDP samples...')
- EDP_theta, EDP_rho = fit_distribution(
- demand_data, 'lognormal')
-
- # Create the RVs
- for d, (d_tag, theta_d) in enumerate(
- zip(d_tags, EDP_theta)):
- self._RV_reg.add_RV(RandomVariable(
- name=d_tag, distribution='lognormal',
- theta=theta_d
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'EDP_set', list(self._RV_reg.RVs(d_tags).values()),
- EDP_rho))
-
- elif target_dist == 'truncated lognormal':
- if censored_count > 0:
- log_msg('\t\tFitting a censored truncated multivariate '
- 'lognormal distribution to EDP samples...')
- EDP_theta, EDP_rho = fit_distribution(
- demand_data, 'lognormal',
- truncation_limits=collapse_limits)
- else:
- log_msg('\t\tFitting a truncated multivariate '
- 'lognormal distribution to EDP samples...')
- EDP_theta, EDP_rho = fit_distribution(
- demand_data, 'lognormal',
- truncation_limits=collapse_limits)
-
- # Create the RVs
- for d, (d_tag, theta_d) in enumerate(
- zip(d_tags, EDP_theta)):
- self._RV_reg.add_RV(RandomVariable(
- name=d_tag, distribution='lognormal',
- theta=theta_d,
- truncation_limits=collapse_limits[d]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'EDP_set', list(self._RV_reg.RVs(d_tags).values()),
- EDP_rho))
-
- else: # this is when fitting is not requested (i.e. empirical)
- [self._RV_reg.add_RV(
- RandomVariable(name=d_tag,
- distribution='empirical',
- raw_samples=demand_data[i]))
- for i, d_tag in enumerate(d_tags)]
-
- # demand_RV = RandomVariable(ID=200, dimension_tags=d_tags,
- # raw_data=demand_data,
- # detection_limits=detection_limits,
- # censored_count=censored_count
- # )
-
- # This is a special case when only a one sample is provided.
- else:
- # TODO: what to do when the sample is larger than the collapse or detection limit and when truncated distribution is prescribed
-
- # Since we only have one data point, the best we can do is assume
- # it is the median of the multivariate distribution. The dispersion
- # is assumed to be negligible.
-
- # dim = demand_data.shape[0]
- # if dim > 1:
- # sig = np.abs(demand_data.T[0])*1e-6
- # rho = np.zeros((dim,dim))
- # np.fill_diagonal(rho, 1.0)
- # else:
- # COV = np.abs(demand_data[0][0])*(1e-6)**2.0
-
- [self._RV_reg.add_RV(
- RandomVariable(name=d_tag, distribution='lognormal',
- theta=[demand_data[i,0],
- demand_data[i,0]*1e-6]))
- for i, d_tag in enumerate(d_tags)]
-
- # demand_RV = RandomVariable(ID=200, dimension_tags=d_tags,
- # distribution_kind='lognormal',
- # theta=demand_data[0],
- # COV=COV)
-
- # To consider additional uncertainty in EDPs, we need to increase
- # the log standard deviation of the lognormal RVs. If the EDP
- # distribution is set to 'empirical' then we do not consider the
- # prescribed additional uncertainty
- if ((self.beta_tot is not None) and
- (GI['response']['EDP_distribution'] != 'empirical')):
- log_msg('Considering additional sources of uncertainty...')
-
- for d_tag in d_tags:
- RV = self._RV_reg.RV[d_tag]
-
- theta_0 = RV.theta
- theta_0[1] = np.sqrt(theta_0[1] ** 2. + self.beta_tot ** 2.)
- RV.theta = theta_0
-
- # determine the covariance matrix with added uncertainty
- # if demand_RV.COV.shape != ():
- # sig_mod = np.sqrt(demand_RV.sig ** 2. + self.beta_tot ** 2.)
- # COV_mod = np.outer(sig_mod, sig_mod) * demand_RV.corr
- # else:
- # COV_mod = np.sqrt(demand_RV.COV**2. + self.beta_tot**2.)
- #
- # # redefine the random variable
- # demand_RV = RandomVariable(
- # ID=200,
- # dimension_tags=demand_RV.dimension_tags,
- # distribution_kind=demand_RV.distribution_kind,
- # theta=demand_RV.theta,
- # COV=COV_mod,
- # truncation_limits=demand_RV.tr_limits_pre)
-
- return d_tags
-
-
-[docs]class FEMA_P58_Assessment(Assessment):
- """
- An Assessment class that implements the loss assessment method in FEMA P58.
- """
- def __init__(self, inj_lvls = 2, log_file=True):
- super(FEMA_P58_Assessment, self).__init__(log_file)
-
- # constants for the FEMA-P58 methodology
- self._inj_lvls = inj_lvls
- self._hazard = 'EQ'
- self._assessment_type = 'P58'
-
- log_msg('type: FEMA P58 Assessment')
- log_msg('hazard: {}'.format(self._hazard))
- log_msg(log_div)
-
-[docs] def read_inputs(self, path_DL_input, path_EDP_input, verbose=False):
- """
- Read and process the input files to describe the loss assessment task.
-
- Parameters
- ----------
- path_DL_input: string
- Location of the Damage and Loss input file. The file is expected to
- be a JSON with data stored in a standard format described in detail
- in the Input section of the documentation.
- path_EDP_input: string
- Location of the EDP input file. The file is expected to follow the
- output formatting of Dakota. The Input section of the documentation
- provides more information about the expected formatting.
- verbose: boolean, default: False
- If True, the method echoes the information read from the files.
- This can be useful to ensure that the information in the file is
- properly read by the method.
-
- """
-
- super(FEMA_P58_Assessment, self).read_inputs(path_DL_input,
- path_EDP_input, verbose)
-
- # assume that the asset is a building
- # TODO: If we want to apply FEMA-P58 to non-building assets, several parts of this methodology need to be extended.
- BIM = self._AIM_in
-
- # read component and population data ----------------------------------
- # components
- log_msg('\tDamage and Loss data files...')
- self._FG_in = read_component_DL_data(
- self._AIM_in['data_sources']['path_CMP_data'],
- BIM['components'],
- assessment_type=self._assessment_type, verbose=verbose)
-
- data = self._FG_in
-
- log_msg('\t\tAvailable Fragility Groups:')
- for key, val in data.items():
- log_msg('\t\t\t{} demand:{} PGs: {}'.format(key, val['demand_type'], len(val['locations'])))
-
- # population (if needed)
- if self._AIM_in['decision_variables']['injuries']:
-
- if BIM['general']['event_time'] is None:
- log_msg('\tPopulation data files...')
- POP = read_population_distribution(
- self._AIM_in['data_sources']['path_POP_data'],
- BIM['general']['occupancy_type'],
- assessment_type=self._assessment_type,
- verbose=verbose)
- else:
- POP = {'peak': None}
-
- POP['peak'] = BIM['general']['population']
- self._POP_in = POP
-
-[docs] def define_random_variables(self):
- """
- Define the random variables used for loss assessment.
-
- Following the FEMA P58 methodology, the groups of parameters below are
- considered random. Simple correlation structures within each group can
- be specified through the DL input file. The random decision variables
- are only created and used later if those particular decision variables
- are requested in the input file.
-
- 1. Demand (EDP) distribution
-
- Describe the uncertainty in the demands. Unlike other random variables,
- the EDPs are characterized by the EDP input data provided earlier. All
- EDPs are handled in one multivariate lognormal distribution. If more
- than one sample is provided, the distribution is fit to the EDP data.
- Otherwise, the provided data point is assumed to be the median value
- and the additional uncertainty prescribed describes the dispersion. See
- _create_RV_demands() for more details.
-
- 2. Component quantities
-
- Describe the uncertainty in the quantity of components in each
- Performance Group. All Fragility Groups are handled in the same
- multivariate distribution. Consequently, correlation between various
- groups of component quantities can be specified. See
- _create_RV_quantities() for details.
-
- 3. Fragility EDP limits
-
- Describe the uncertainty in the EDP limit that corresponds to
- exceedance of each Damage State. EDP limits are grouped by Fragility
- Groups. Consequently, correlation between fragility limits are
- currently limited within Fragility Groups. See
- _create_RV_fragilities() for details.
-
- 4. Reconstruction cost and time
-
- Describe the uncertainty in the cost and duration of reconstruction of
- each component conditioned on the damage state of the component. All
- Fragility Groups are handled in the same multivariate distribution.
- Consequently, correlation between various groups of component
- reconstruction time and cost estimates can be specified. See
- _create_RV_repairs() for details.
-
- 5. Damaged component proportions that trigger a red tag
-
- Describe the uncertainty in the amount of damaged components needed to
- trigger a red tag for the building. All Fragility Groups are handled in
- the same multivariate distribution. Consequently, correlation between
- various groups of component proportion limits can be specified. See
- _create_RV_red_tags() for details.
-
- 6. Injuries
-
- Describe the uncertainty in the proportion of people in the affected
- area getting injuries exceeding a certain level of severity. FEMA P58
- uses two severity levels: injury and fatality. Both levels for all
- Fragility Groups are handled in the same multivariate distribution.
- Consequently, correlation between various groups of component injury
- expectations can be specified. See _create_RV_injuries() for details.
- """
- super(FEMA_P58_Assessment, self).define_random_variables()
-
- # create the random variables -----------------------------------------
- DEP = self._AIM_in['dependencies']
-
- self._RV_dict = {}
-
- # quantities 100
- log_msg('\tQuantities...')
- QNT_tags = self._create_RV_quantities(DEP['quantities'])
-
- # self._RV_dict.update({'QNT':
- # self._create_RV_quantities(DEP['quantities'])})
-
- if QNT_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(QNT_tags)))
- self._QNT_dict = self._RV_reg.RVs(QNT_tags)
- else:
- log_msg('\t\tNone of the components have random quantities assigned')
-
- # fragilities 300
- log_msg('\tDamage State Limits...')
- s_fg_keys = sorted(self._FG_in.keys())
- self._FF_dict = {}
- log_msg('\t\tRV dimensions:')
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- FF_tags = self._create_RV_fragilities(c_id, comp, DEP['fragilities'])
- self._FF_dict.update(self._RV_reg.RVs(FF_tags))
-
- # self._RV_dict.update({
- # 'FR-' + c_name:
- # self._create_RV_fragilities(c_id, comp,
- # DEP['fragilities'])})
- log_msg('\t\t\t{}: {}'.format(c_name, len(FF_tags)))
-
-
- # for key, val in self._RV_dict.items():
- # if 'FR-' in key:
- # log_msg('\t\t\t{}: {}'.format(key, len(val.theta)))
-
- # irreparability
- if 'irreparable_res_drift' in self._AIM_in['general']:
- irrep_frag = self._AIM_in['general']['irreparable_res_drift']
- RV_irrep = RandomVariable(
- name='RED_irrep', distribution='lognormal',
- theta=[irrep_frag['Median'], irrep_frag['Beta']])
- self._RV_reg.add_RV(RV_irrep)
-
- # collapse modes
- coll_modes = self._AIM_in['collapse_modes']
- P_keys = [cmk for cmk in coll_modes.keys()]
- P_modes = [coll_modes[k]['w'] for k in P_keys]
- RV_CM = RandomVariable(
- name='CM', distribution='multinomial', theta=P_modes)
- self._RV_reg.add_RV(RV_CM)
-
- # damages (mutually exclusive cases and later simultaneous ones too)
- log_msg('\tDamage States...')
- DS_tags = self._create_RV_damage_states()
-
- if DS_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(DS_tags)))
- self._DSG_dict = self._RV_reg.RVs(DS_tags)
- else:
- log_msg('\t\tNone of the components have random damage states assigned')
-
- # consequences 400
- DVs = self._AIM_in['decision_variables']
-
- if DVs['red_tag']:
- log_msg('\tRed Tag Thresholds...')
- RED_tags = self._create_RV_red_tags(DEP['red_tags'])
-
- # self._RV_dict.update({'DV_RED':
- # self._create_RV_red_tags(DEP['red_tags'])})
-
- if RED_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(RED_tags)))
- self._DV_RED_dict = self._RV_reg.RVs(RED_tags)
- else:
- log_msg('\t\tNone of the component damage states trigger red tags')
-
- if DVs['rec_time'] or DVs['rec_cost']:
- log_msg('\tReconstruction Costs and Times...')
- REP_tags = self._create_RV_repairs(
- DEP['rec_costs'], DEP['rec_times'], DEP['cost_and_time'])
-
- # self._RV_dict.update({'DV_REP':
- # self._create_RV_repairs(
- # DEP['rec_costs'],
- # DEP['rec_times'],
- # DEP['cost_and_time'])})
-
- if REP_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(REP_tags)))
- self._DV_REP_dict = self._RV_reg.RVs(REP_tags)
- else:
- log_msg('\t\tNone of the components have probabilistic consequence functions')
-
-
- if DVs['injuries']:
- log_msg('\tInjury Probabilities...')
- INJ_tags = self._create_RV_injuries(
- DEP['injuries'], DEP['injury_lvls'])
-
- # self._RV_dict.update({'DV_INJ':
- # self._create_RV_injuries(
- # DEP['injuries'],
- # DEP['injury_lvls'])})
-
- if INJ_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(INJ_tags)))
- self._DV_INJ_dict = self._RV_reg.RVs(INJ_tags)
- else:
- log_msg('\t\tNone of the component damage states trigger injuries')
-
- # demands 200
- log_msg('\tEDPs...')
-
- GR = self._AIM_in['general']['response']
- if GR['EDP_dist_basis'] == 'non-collapse results':
- discard_limits = self._AIM_in['general']['collapse_limits']
- else:
- discard_limits = None
-
- EDP_tags = self._create_RV_demands()
- self._EDP_dict = self._RV_reg.RVs(EDP_tags)
- #self._RV_dict.update({'EDP': self._create_RV_demands()})
-
- # sample the random variables -----------------------------------------
- log_msg()
- log_msg('Sampling the random variables...')
-
- realization_count = self._AIM_in['general']['realizations']
-
- self._RV_reg.generate_samples(sample_size=realization_count)
-
- # is_coupled = self._AIM_in['general']['coupled_assessment']
- #
- # s_rv_keys = sorted(self._RV_dict.keys())
- # for r_i in s_rv_keys:
- # rv = self._RV_dict[r_i]
- # if rv is not None:
- # log_msg('\t{}...'.format(r_i))
- # rv.sample_distribution(
- # sample_size=realization_count,
- # preserve_order=((r_i=='EDP') and is_coupled))
-
- log_msg('Sampling completed.')
-
-[docs] def define_loss_model(self):
- """
- Create the stochastic loss model based on the inputs provided earlier.
-
- Following the FEMA P58 methodology, the components specified in the
- Damage and Loss input file are used to create Fragility Groups. Each
- Fragility Group corresponds to a component that might be present in
- the building at several locations. See _create_fragility_groups() for
- more details about the creation of Fragility Groups.
-
- """
- super(FEMA_P58_Assessment, self).define_loss_model()
-
- # fragility groups
- self._FG_dict = self._create_fragility_groups()
-
- # demands
- # self._EDP_dict = dict(
- # [(tag, RandomVariableSubset(self._RV_dict['EDP'],tags=tag))
- # for tag in self._RV_dict['EDP']._dimension_tags])
-
-[docs] def calculate_damage(self):
- """
- Characterize the damage experienced in each random event realization.
-
- First, the time of the event (month, weekday/weekend, hour) is randomly
- generated for each realization. Given the event time, the number of
- people present at each floor of the building is calculated.
-
- Second, the realizations that led to collapse are filtered. See
- _calc_collapses() for more details on collapse estimation.
-
- Finally, the realizations that did not lead to building collapse are
- further investigated and the quantities of components in each damage
- state are estimated. See _calc_damage() for more details on damage
- estimation.
-
- """
- super(FEMA_P58_Assessment, self).calculate_damage()
-
- # event time - month, weekday, and hour realizations
- log_msg('\tSampling event time...')
- self._TIME = self._sample_event_time()
-
- # get the population conditioned on event time (if needed)
- if self._AIM_in['decision_variables']['injuries']:
- log_msg('\tSampling the population...')
- self._POP = self._get_population()
-
- # collapses
- log_msg('\tCalculating the number of collapses...')
- self._COL, collapsed_IDs = self._calc_collapses()
- self._ID_dict.update({'collapse':collapsed_IDs})
- log_msg('\t\t{} out of {} collapsed.'.format(
- len(collapsed_IDs),
- self._AIM_in['general']['realizations']))
-
- # select the non-collapse cases for further analyses
- non_collapsed_IDs = self._TIME[
- ~self._TIME.index.isin(collapsed_IDs)].index.values.astype(int)
- self._ID_dict.update({'non-collapse': non_collapsed_IDs})
-
- # damage in non-collapses
- log_msg('\tCalculating the damage in the non-collapsed cases...')
- self._DMG = self._calc_damage()
-
-[docs] def calculate_losses(self):
- """
- Characterize the consequences of damage in each random event realization.
-
- For the sake of efficiency, only the decision variables requested in
- the input file are estimated. The following consequences are handled by
- this method:
-
- Reconstruction time and cost
- Estimate the irreparable cases based on residual drift magnitude and
- the provided irreparable drift limits. Realizations that led to
- irreparable damage or collapse are assigned the replacement cost and
- time of the building when reconstruction cost and time is estimated.
- Repairable cases get a cost and time estimate for each Damage State in
- each Performance Group. For more information about estimating
- irreparability see _calc_irreparable() and reconstruction cost and
- time see _calc_repair_cost_and_time() methods.
-
- Injuries
- Collapse-induced injuries are based on the collapse modes and
- corresponding injury characterization. Injuries conditioned on no
- collapse are based on the affected area and the probability of
- injuries of various severity specified in the component data file. For
- more information about estimating injuries conditioned on collapse and
- no collapse, see _calc_collapse_injuries() and
- _calc_non_collapse_injuries, respectively.
-
- Red Tag
- The probability of getting an unsafe placard or red tag is a function
- of the amount of damage experienced in various Damage States for each
- Performance Group. The damage limits that trigger an unsafe placard are
- specified in the component data file. For more information on
- assigning red tags to realizations see the _calc_red_tag() method.
-
- """
- super(FEMA_P58_Assessment, self).calculate_losses()
- DVs = self._AIM_in['decision_variables']
-
- # red tag probability
- if DVs['red_tag']:
- log_msg('\tAssigning Red Tags...')
- DV_RED = self._calc_red_tag()
-
- self._DV_dict.update({'red_tag': DV_RED})
-
- # reconstruction cost and time
- if DVs['rec_cost'] or DVs['rec_time']:
- # irreparable cases
- if 'irreparable_res_drift' in self._AIM_in['general']:
- log_msg('\tIdentifying Irreparable Cases...')
- irreparable_IDs = self._calc_irreparable()
- log_msg('\t\t{} out of {} non-collapsed cases are irreparable.'.format(
- len(irreparable_IDs), len(self._ID_dict['non-collapse'])))
- else:
- irreparable_IDs = np.array([])
-
- # collect the IDs of repairable realizations
- P_NC = self._TIME.loc[self._ID_dict['non-collapse']]
- repairable_IDs = P_NC[
- ~P_NC.index.isin(irreparable_IDs)].index.values.astype(int)
-
- self._ID_dict.update({'repairable': repairable_IDs})
- self._ID_dict.update({'irreparable': irreparable_IDs})
-
- # reconstruction cost and time for repairable cases
- log_msg('\tCalculating Reconstruction cost and time...')
- DV_COST, DV_TIME = self._calc_repair_cost_and_time()
-
- if DVs['rec_cost']:
- self._DV_dict.update({'rec_cost': DV_COST})
-
- if DVs['rec_time']:
- self._DV_dict.update({'rec_time': DV_TIME})
-
- # injuries due to collapse
- if DVs['injuries']:
- log_msg('\tCalculating Injuries in Collapsed Cases...')
- COL_INJ = self._calc_collapse_injuries()
-
- # injuries in non-collapsed cases
- log_msg('\tCalculating Injuries in Non-Collapsed Cases...')
- DV_INJ_dict = self._calc_non_collapse_injuries()
-
- # store results
- if COL_INJ is not None:
- self._COL = pd.concat([self._COL, COL_INJ], axis=1)
-
- self._DV_dict.update({'injuries': DV_INJ_dict})
-
-[docs] def aggregate_results(self):
- """
-
- Returns
- -------
-
- """
-
- log_msg(log_div)
- log_msg('Aggregating results...')
-
- DVs = self._AIM_in['decision_variables']
-
- MI_raw = [
- ('inhabitants', ''),
- ('collapses', 'collapsed'),
- ('collapses', 'mode'),
- ('red tagged', ''),
- ('reconstruction', 'irreparable'),
- ('reconstruction', 'cost impractical'),
- ('reconstruction', 'cost'),
- ('reconstruction', 'time impractical'),
- ('reconstruction', 'time-sequential'),
- ('reconstruction', 'time-parallel'),
- ('injuries', 'sev1'), # thanks, Laura S.!
- ('injuries', 'sev2'),
- ]
-
- if self._AIM_in['general'].get('event_time', None) != 'off':
- MI_raw += [
- ('event time', 'month'),
- ('event time', 'weekday?'),
- ('event time', 'hour')
- ]
-
- ncID = self._ID_dict['non-collapse']
- colID = self._ID_dict['collapse']
- if DVs['rec_cost'] or DVs['rec_time']:
- repID = self._ID_dict['repairable']
- irID = self._ID_dict['irreparable']
-
- MI = pd.MultiIndex.from_tuples(MI_raw)
-
- SUMMARY = pd.DataFrame(np.empty((
- self._AIM_in['general']['realizations'],
- len(MI))), columns=MI)
- SUMMARY[:] = np.NaN
-
- # event time (if needed)
- if self._AIM_in['general'].get('event_time', None) != 'off':
- for prop in ['month', 'weekday?', 'hour']:
- offset = 0
- if prop == 'month':
- offset = 1
- SUMMARY.loc[:, ('event time', prop)] = \
- self._TIME.loc[:, prop] + offset
-
- # collapses
- SUMMARY.loc[:, ('collapses', 'collapsed')] = self._COL.iloc[:, 0]
-
- # red tag
- if DVs['red_tag']:
- SUMMARY.loc[ncID, ('red tagged', '')] = \
- self._DV_dict['red_tag'].max(axis=1)
-
- # reconstruction cost
- if DVs['rec_cost']:
- SUMMARY.loc[ncID, ('reconstruction', 'cost')] = \
- self._DV_dict['rec_cost'].sum(axis=1)
-
- repl_cost = self._AIM_in['general']['replacement_cost']
- SUMMARY.loc[colID, ('reconstruction', 'cost')] = repl_cost
-
- if DVs['rec_cost'] or DVs['rec_time']:
- SUMMARY.loc[ncID, ('reconstruction', 'irreparable')] = 0
- SUMMARY.loc[irID, ('reconstruction', 'irreparable')] = 1
-
- if DVs['rec_cost']:
- SUMMARY.loc[irID, ('reconstruction', 'cost')] = repl_cost
-
- repair_impractical_IDs = SUMMARY.loc[
- SUMMARY.loc[:, ('reconstruction', 'cost')] > repl_cost].index
- SUMMARY.loc[repID, ('reconstruction', 'cost impractical')] = 0
- SUMMARY.loc[repair_impractical_IDs,
- ('reconstruction', 'cost impractical')] = 1
- SUMMARY.loc[
- repair_impractical_IDs, ('reconstruction', 'cost')] = repl_cost
-
- # reconstruction time
- if DVs['rec_time']:
- SUMMARY.loc[ncID, ('reconstruction', 'time-sequential')] = \
- self._DV_dict['rec_time'].sum(axis=1)
- SUMMARY.loc[ncID, ('reconstruction', 'time-parallel')] = \
- self._DV_dict['rec_time'].max(axis=1)
-
- rep_time = self._AIM_in['general']['replacement_time']
-
- for t_label in ['time-sequential', 'time-parallel']:
- SUMMARY.loc[colID, ('reconstruction', t_label)] = rep_time
- SUMMARY.loc[irID, ('reconstruction', t_label)] = rep_time
-
- repair_impractical_IDs = \
- SUMMARY.loc[SUMMARY.loc[:, ('reconstruction',
- 'time-parallel')] > rep_time].index
- SUMMARY.loc[repID, ('reconstruction', 'time impractical')] = 0
- SUMMARY.loc[repair_impractical_IDs,('reconstruction',
- 'time impractical')] = 1
- SUMMARY.loc[repair_impractical_IDs, ('reconstruction',
- 'time-parallel')] = rep_time
-
- # injuries
- if DVs['injuries']:
-
- # inhabitants
- SUMMARY.loc[:, ('inhabitants', '')] = self._POP.sum(axis=1)
-
- if 'CM' in self._COL.columns:
- SUMMARY.loc[colID, ('collapses', 'mode')] = self._COL.loc[:, 'CM']
-
- SUMMARY.loc[colID, ('injuries', 'sev1')] = \
- self._COL.loc[:, 'INJ-0']
- SUMMARY.loc[colID, ('injuries', 'sev2')] = \
- self._COL.loc[:, 'INJ-1']
-
- SUMMARY.loc[ncID, ('injuries', 'sev1')] = \
- self._DV_dict['injuries'][0].sum(axis=1)
- SUMMARY.loc[ncID, ('injuries', 'sev2')] = \
- self._DV_dict['injuries'][1].sum(axis=1)
-
- self._SUMMARY = SUMMARY.dropna(axis=1,how='all')
-
-[docs] def save_outputs(self, *args, **kwargs):
- """
-
- Returns
- -------
-
- """
- super(FEMA_P58_Assessment, self).save_outputs(*args, **kwargs)
-
- def _create_correlation_matrix(self, rho_target, c_target=-1,
- include_CSG=False,
- include_DSG=False, include_DS=False):
- """
-
- Parameters
- ----------
- rho_target
- c_target
- include_CSG
- include_DSG
- include_DS
-
- Returns
- -------
-
- """
-
- # set the correlation structure
- rho_FG, rho_PG, rho_LOC, rho_DIR, rho_CSG, rho_DS = np.zeros(6)
-
- if rho_target in ['FG', 'PG', 'DIR', 'LOC', 'CSG', 'ATC', 'DS']:
- rho_DS = 1.0
- if rho_target in ['FG', 'PG', 'DIR', 'LOC', 'CSG']:
- rho_CSG = 1.0
- if rho_target in ['FG', 'PG', 'DIR']:
- rho_DIR = 1.0
- if rho_target in ['FG', 'PG', 'LOC']:
- rho_LOC = 1.0
- if rho_target in ['FG', 'PG']:
- rho_PG = 1.0
- if rho_target == 'FG':
- rho_FG = 1.0
-
- L_D_list = []
- dims = []
- DS_list = []
- ATC_rho = []
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- if ((c_target == -1) or (c_id == c_target)):
- c_L_D_list = []
- c_DS_list = []
- ATC_rho.append(comp['correlation'])
-
- if include_DSG:
- DS_count = 0
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- if include_DS:
- DS_count += len(DSG['DS_set'])
- else:
- DS_count += 1
- else:
- DS_count = 1
-
- #for loc in comp['locations']:
- # if include_CSG:
- # u_dirs = comp['directions']
- # else:
- # u_dirs = np.unique(comp['directions'])
- # c_L_D_list.append([])
- # for dir_ in u_dirs:
- # c_DS_list.append(DS_count)
- # for ds_i in range(DS_count):
- # c_L_D_list[-1].append(dir_)
-
- for loc_u in np.unique(comp['locations']):
- c_L_D_list.append([])
- for loc, dir, csg_weights in zip(comp['locations'],
- comp['directions'],
- comp['csg_weights']):
- if loc == loc_u:
- if include_CSG:
- csg_list = csg_weights
- else:
- csg_list = [1.0,]
- for csg_ in csg_list:
- c_DS_list.append(DS_count)
- for ds_i in range(DS_count):
- c_L_D_list[-1].append(dir)
-
- c_dims = sum([len(loc) for loc in c_L_D_list])
- dims.append(c_dims)
- L_D_list.append(c_L_D_list)
- DS_list.append(c_DS_list)
-
- rho = np.ones((sum(dims), sum(dims))) * rho_FG
-
- f_pos_id = 0
- for c_id, (c_L_D_list, c_dims, c_DS_list) in enumerate(
- zip(L_D_list, dims, DS_list)):
- c_rho = np.ones((c_dims, c_dims)) * rho_PG
-
- # dependencies btw directions
- if rho_DIR != 0:
- c_pos_id = 0
- for loc_D_list in c_L_D_list:
- l_dim = len(loc_D_list)
- c_rho[c_pos_id:c_pos_id + l_dim,
- c_pos_id:c_pos_id + l_dim] = rho_DIR
- c_pos_id = c_pos_id + l_dim
-
- # dependencies btw locations
- if rho_LOC != 0:
- flat_dirs = []
- [[flat_dirs.append(dir_i) for dir_i in dirs] for dirs in
- c_L_D_list]
- flat_dirs = np.array(flat_dirs)
- for u_dir in np.unique(flat_dirs):
- dir_ids = np.where(flat_dirs == u_dir)[0]
- for i in dir_ids:
- for j in dir_ids:
- c_rho[i, j] = rho_LOC
-
- if ((rho_CSG != 0) or (rho_target == 'ATC')):
- c_pos_id = 0
- if rho_target == 'ATC':
- rho_to_use = float(ATC_rho[c_id])
- else:
- rho_to_use = rho_CSG
- for loc_D_list in c_L_D_list:
- flat_dirs = np.array(loc_D_list)
- for u_dir in np.unique(flat_dirs):
- dir_ids = np.where(flat_dirs == u_dir)[0]
- for i in dir_ids:
- for j in dir_ids:
- c_rho[c_pos_id + i, c_pos_id + j] = rho_to_use
- c_pos_id = c_pos_id + len(loc_D_list)
-
- if rho_DS != 0:
- c_pos_id = 0
- for l_dim in c_DS_list:
- c_rho[c_pos_id:c_pos_id + l_dim,
- c_pos_id:c_pos_id + l_dim] = rho_DS
- c_pos_id = c_pos_id + l_dim
-
- rho[f_pos_id:f_pos_id + c_dims,
- f_pos_id:f_pos_id + c_dims] = c_rho
- f_pos_id = f_pos_id + c_dims
-
- np.fill_diagonal(rho, 1.0)
-
- return rho
-
- def _create_RV_quantities(self, rho_qnt):
- """
-
- Parameters
- ----------
- rho_qnt
-
- Returns
- -------
-
- """
-
- q_theta, q_sig, q_tags, q_dist = [np.array([]) for i in range(4)]
-
- # collect the parameters for each quantity dimension
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id in s_fg_keys:
- comp = self._FG_in[c_id]
-
- u_dirs = np.unique(comp['directions'])
-
- #dir_weights = comp['dir_weights']
- #theta_list = []
- #[[theta_list.append(qnt * dw)
- # for dw in dir_weights] for qnt in comp['quantities']]
-
- theta_list = comp['quantities']
- q_theta = np.append(q_theta, theta_list)
-
- dist_list = comp['distribution_kind']
- q_dist = np.append(q_dist, dist_list)
-
- cov_list = comp['cov']
- for theta, dk, cov in list(zip(theta_list, dist_list, cov_list)):
- if dk == 'normal':
- q_sig = np.append(q_sig, [cov*theta,])
- else:
- q_sig = np.append(q_sig, [cov, ])
-
- q_tags = np.append(q_tags, [f'{c_id}-QNT-{s_i}-{d_i}' for s_i, d_i
- in list(zip(comp['locations'],
- comp['directions']))])
-
- rho = self._create_correlation_matrix(rho_qnt)
-
- if not np.all(q_dist=='N/A'):
- # remove the unnecessary fields
- to_remove = np.where(q_dist=='N/A')[0]
- rho = np.delete(rho, to_remove, axis=0)
- rho = np.delete(rho, to_remove, axis=1)
-
- q_theta, q_sig, q_dist, q_tags = [
- np.delete(q_vals, to_remove) for q_vals in [
- q_theta, q_sig, q_dist, q_tags]]
-
- dims = len(q_theta)
-
- # add lower limits to ensure only positive quantities
- # zero is probably too low, and it might make sense to introduce upper
- # limits as well
- tr_lower = [0. for d in range(dims)]
- tr_upper = [None for d in range(dims)]
-
- # Create the RVs
- for q, (q_tag, theta_q, sig_q, dist_q) in enumerate(
- zip(q_tags, q_theta, q_sig, q_dist)):
- self._RV_reg.add_RV(RandomVariable(
- name=q_tag, distribution=dist_q,
- theta=[theta_q, sig_q],
- truncation_limits = [tr_lower[q], tr_upper[q]]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'QNT_set', list(self._RV_reg.RVs(q_tags).values()),
- rho
- ))
-
- # q_COV = np.outer(q_sig, q_sig) * rho
- #
- # # to avoid truncations affecting other dimensions when rho_QNT is large,
- # # assign a post-truncation correlation structure
- # corr_ref = 'post'
- #
- # quantity_RV = RandomVariable(ID=100,
- # dimension_tags=q_tag,
- # distribution_kind=q_dist,
- # theta=q_theta,
- # COV=q_COV,
- # truncation_limits=[tr_lower, tr_upper],
- # corr_ref=corr_ref)
-
- else:
- q_tags = None
-
- return q_tags
-
- def _create_RV_fragilities(self, c_id, comp, rho_fr):
- """
-
- Parameters
- ----------
- c_id
- comp
- rho_fr
-
- Returns
- -------
-
- """
-
- # prepare the basic multivariate distribution data for one component subgroup considering all damage states
- d_theta, d_sig, d_tags, d_distr_kind = [np.array([]) for i in range(4)]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for d_id in s_dsg_keys:
- DSG = comp['DSG_set'][d_id]
- d_theta = np.append(d_theta, DSG['theta'])
- d_sig = np.append(d_sig, DSG['sig'])
- d_tags = np.append(d_tags, f"FF-{comp['ID']}-{d_id}")
- d_distr_kind = np.append(d_distr_kind, DSG['distribution_kind'])
- dims = len(d_theta)
-
- # get the total number of random variables for this fragility group
- #rv_count = len(comp['locations']) * len(comp['directions']) * dims
- rv_count = sum([len(csg_w) for csg_w in comp['csg_weights']]) * dims
-
- # create the (empty) input arrays for the RV
- c_theta = np.zeros(rv_count)
- c_tags = np.empty(rv_count, dtype=object)
- c_sig = np.zeros(rv_count)
- c_distr_kind = np.empty(rv_count, dtype=object)
-
- pos_id = 0
- #for l_id in comp['locations']:
- # # for each location-direction pair)
- # for d_id, __ in enumerate(comp['directions']):
- # # for each component-subgroup
- # c_theta[pos_id:pos_id + dims] = d_theta
- # c_sig[pos_id:pos_id + dims] = d_sig
- # c_tags[pos_id:pos_id + dims] = [
- # t + '-LOC-{}-CSG-{}'.format(l_id, d_id) for t in d_tags]
- # c_distr_kind[pos_id:pos_id + dims] = d_distr_kind
- # pos_id += dims
-
- for l_id, d_id, csg_list in zip(comp['locations'], comp['directions'],
- comp['csg_weights']):
- # for each location-direction pair)
- for csg_id, __ in enumerate(csg_list):
- # for each component-subgroup
- c_theta[pos_id:pos_id + dims] = d_theta
- c_sig[pos_id:pos_id + dims] = d_sig
- c_tags[pos_id:pos_id + dims] = [
- t + '-LOC-{}-DIR-{}-CSG-{}'.format(l_id, d_id, csg_id)
- for t in d_tags]
- c_distr_kind[pos_id:pos_id + dims] = d_distr_kind
- pos_id += dims
-
- # create the covariance matrix
- c_rho = self._create_correlation_matrix(rho_fr, c_target=c_id,
- include_DSG=True,
- include_CSG=True)
-
- if c_tags.size > 0:
- # Create the RVs
- for c, (c_tag, sig_c, theta_c, dkind_c) in enumerate(
- zip(c_tags, c_sig, c_theta, c_distr_kind)):
- self._RV_reg.add_RV(RandomVariable(
- name=c_tag, distribution=dkind_c,
- theta=[theta_c, sig_c]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- f'FF_set_{c_id}', list(self._RV_reg.RVs(c_tags).values()),
- c_rho))
-
- # c_COV = np.outer(c_sig, c_sig) * c_rho
- #
- # fragility_RV = RandomVariable(ID=300 + c_id,
- # dimension_tags=c_tags,
- # distribution_kind=c_distr_kind,
- # theta=c_theta,
- # COV=c_COV)
- else:
- c_tags = None
-
- return c_tags
-
- def _create_RV_damage_states(self):
-
- d_tags, d_theta = [[] for i in range(2)]
-
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- if DSG['DS_set_kind'] == 'mutually exclusive':
-
- s_ds_keys = sorted(DSG['DS_set'].keys())
- DS_set = DSG['DS_set']
- DS_weights = [DS_set[ds_i]['weight'] for ds_i in s_ds_keys]
-
- for loc, dir_, csg_list in zip(comp['locations'],
- comp['directions'],
- comp['csg_weights']):
-
- for csg_id, __ in enumerate(csg_list):
-
- d_tags.append(f'DSG-{comp["ID"]}-{dsg_i}-LOC-{loc}-'
- f'DIR-{dir_}-CSG-{csg_id}')
- d_theta.append(DS_weights)
-
- if len(d_tags) > 0:
-
- # Create the RVs
- for d, (d_tag, theta_d) in enumerate(zip(d_tags, d_theta)):
- self._RV_reg.add_RV(RandomVariable(
- name=d_tag, distribution='multinomial',
- theta=theta_d))
-
- else:
- d_tags = None
-
- return d_tags
-
- def _create_RV_red_tags(self, rho_target):
-
- f_theta, f_sig, f_tags = [np.array([]) for i in range(3)]
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- d_theta, d_sig, d_tag = [np.array([]) for i in range(3)]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- s_ds_keys = sorted(DSG['DS_set'].keys())
- for ds_i in s_ds_keys:
- DS = DSG['DS_set'][ds_i]
- if 'red_tag' in DS.keys():
- d_theta = np.append(d_theta, DS['red_tag']['theta'])
- d_sig = np.append(d_sig, DS['red_tag']['cov'])
- else:
- # if there are no injuries assigned to this DS
- d_theta = np.append(d_theta, 0)
- d_sig = np.append(d_sig, 0.0001)
- d_tag = np.append(d_tag, f"RED-{comp['ID']}-{dsg_i}-{ds_i}")
-
- for loc, dir_ in zip(comp['locations'], comp['directions']):
- f_theta = np.append(f_theta, d_theta)
- f_sig = np.append(f_sig, d_sig)
- f_tags = np.append(f_tags, [t + f'-LOC-{loc}-DIR-{dir_}'
- for t in d_tag])
-
- rho = self._create_correlation_matrix(rho_target, c_target=-1,
- include_DSG=True,
- include_DS=True)
-
- if not np.all(f_theta==0.):
- # remove the unnecessary fields
- to_remove = np.where(f_theta == 0)[0]
- rho = np.delete(rho, to_remove, axis=0)
- rho = np.delete(rho, to_remove, axis=1)
-
- f_theta, f_sig, f_tags = [np.delete(f_vals, to_remove)
- for f_vals in [f_theta, f_sig, f_tags]]
-
- tr_upper = 1. + (1. - f_theta) / f_theta
-
- # Create the RVs
- for f, (f_tag, theta_f, sig_f) in enumerate(
- zip(f_tags, f_theta, f_sig)):
- self._RV_reg.add_RV(RandomVariable(
- name=f_tag, distribution='normal',
- theta=[1.0, sig_f],
- truncation_limits=[0., tr_upper[f]]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'DV_RED_set', list(self._RV_reg.RVs(f_tags).values()), rho))
-
- # f_COV = np.outer(f_sig, f_sig) * rho
- #
- # red_tag_RV = RandomVariable(ID=400,
- # dimension_tags=f_tag,
- # distribution_kind='normal',
- # theta=np.ones(len(f_theta)),
- # COV=f_COV,
- # corr_ref='post',
- # truncation_limits=[np.zeros(len(f_theta)),
- # tr_upper])
- else:
- f_tags = None
-
- return f_tags
-
- def _create_RV_repairs(self, rho_cost, rho_time, rho_cNt):
-
- # prepare the cost and time parts of the data separately
- ct_sig, ct_tags, ct_dkind = [np.array([]) for i in range(3)]
- for rho_target, name in zip([rho_cost, rho_time], ['cost', 'time']):
-
- f_sig, f_tag, f_dkind = [np.array([]) for i in range(3)]
-
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- d_sig, d_tag, d_dkind = [np.array([]) for i in range(3)]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- s_ds_keys = sorted(DSG['DS_set'].keys())
- for ds_i in s_ds_keys:
- DS = DSG['DS_set'][ds_i]
- if ((f'repair_{name}' in DS.keys()) and
- (DS[f'repair_{name}']['distribution_kind'] is not None)):
- data = DS[f'repair_{name}']
- d_sig = np.append(d_sig, data['cov'])
- d_dkind = np.append(d_dkind,
- data['distribution_kind'])
- else:
- d_sig = np.append(d_sig, 0.0001)
- d_dkind = np.append(d_dkind, None)
-
- d_tag = np.append(
- d_tag, f'REP-{comp["ID"]}-{dsg_i}-{ds_i}-{name}')
- #comp['ID'] + '-' + str(
- # dsg_i) + '-' + str(
- # ds_i) + '-{}'.format(name))
-
- for loc, dir_ in zip(comp['locations'], comp['directions']):
- f_sig = np.append(f_sig, d_sig)
- f_dkind = np.append(f_dkind, d_dkind)
- f_tag = np.append(
- f_tag, [t + f'-LOC-{loc}-DIR-{dir_}' for t in d_tag])
-
- ct_sig = np.append(ct_sig, f_sig)
- ct_tags = np.append(ct_tags, f_tag)
- ct_dkind = np.append(ct_dkind, f_dkind)
-
- rho_c = self._create_correlation_matrix(rho_cost, c_target=-1,
- include_DSG=True,
- include_DS=True)
- rho_t = self._create_correlation_matrix(rho_time, c_target=-1,
- include_DSG=True,
- include_DS=True)
-
- dims = len(ct_tags)
- ct_rho = np.zeros((dims, dims))
-
- dims = dims // 2
- if rho_cNt == False:
- ct_rho[:dims, :dims] = rho_c
- ct_rho[dims:, dims:] = rho_t
- else:
- # In the special case of mixing perfect correlation between
- # locations and directions, taking the envelope is not the
- # appropriate solution. Instead, the LOC & DIR -> PG approach is
- # used.
- if (((rho_cost == 'LOC') and (rho_time =='DIR')) or
- ((rho_cost == 'DIR') and (rho_time == 'LOC'))):
- rho_ct = self._create_correlation_matrix('PG', c_target=-1,
- include_DSG=True,
- include_DS=True)
- else:
- # We use the envelope in every other case.
- rho_ct = np.maximum(rho_c, rho_t)
-
- ct_rho[:dims, :dims] = rho_ct
- ct_rho[dims:, dims:] = rho_ct
-
- # apply the same blocks to the off-diagonal positions
- ct_rho[:dims, dims:] = rho_ct
- ct_rho[dims:, :dims] = rho_ct
-
- # now remove the unnecessary fields
- if not np.all(ct_dkind == None):
-
- to_remove = np.where(ct_dkind == None)[0]
- ct_rho = np.delete(ct_rho, to_remove, axis=0)
- ct_rho = np.delete(ct_rho, to_remove, axis=1)
-
- ct_dkind, ct_sig, ct_tags = [np.delete(ct_vals, to_remove)
- for ct_vals in [ct_dkind, ct_sig, ct_tags]]
-
- # Create the RVs
- for ct, (ct_tag, sig_ct, dkind_ct) in enumerate(
- zip(ct_tags, ct_sig, ct_dkind)):
- self._RV_reg.add_RV(RandomVariable(
- name=ct_tag, distribution=dkind_ct,
- theta=[1.0, sig_ct],
- truncation_limits=[0., None]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'DV_REP_set', list(self._RV_reg.RVs(ct_tags).values()), ct_rho))
-
- # ct_COV = np.outer(ct_sig, ct_sig) * ct_rho
- #
- # repair_RV = RandomVariable(ID=401,
- # dimension_tags=ct_tags,
- # distribution_kind=ct_dkind,
- # theta=np.ones(len(ct_sig)),
- # COV=ct_COV,
- # corr_ref='post',
- # truncation_limits=[np.zeros(len(ct_sig)),
- # None])
- else:
- ct_tags = None
-
- return ct_tags
-
- def _create_RV_injuries(self, rho_target, rho_lvls):
-
- inj_lvls = self._inj_lvls
-
- # prepare the parts for different levels of injury separately
- full_theta, full_sig, full_tags = [np.array([]) for i in range(3)]
- for i_lvl in range(inj_lvls):
-
- f_theta, f_sig, f_tag = [np.array([]) for i in range(3)]
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- d_theta, d_sig, d_tag = [np.array([]) for i in range(3)]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- s_ds_keys = sorted(DSG['DS_set'].keys())
- for ds_i in s_ds_keys:
- DS = DSG['DS_set'][ds_i]
- if 'injuries' in DS.keys():
- d_theta = np.append(
- d_theta, DS['injuries']['theta'][i_lvl])
- d_sig = np.append(
- d_sig, DS['injuries']['cov'][i_lvl])
- else:
- # if there are no injuries assigned to this DS
- d_theta = np.append(d_theta, 0)
- d_sig = np.append(d_sig, 0.0001)
- d_tag = np.append(
- d_tag, f"INJ-{comp['ID']}-{dsg_i}-{ds_i}-{i_lvl}")
-
- for loc, dir_ in zip(comp['locations'], comp['directions']):
- f_theta = np.append(f_theta, d_theta)
- f_sig = np.append(f_sig, d_sig)
- f_tag = np.append(f_tag,
- [t + '-LOC-{}-DIR-{}'.format(loc,
- dir_)
- for t in d_tag])
-
- full_theta = np.append(full_theta, f_theta)
- full_sig = np.append(full_sig, f_sig)
- full_tags = np.append(full_tags, f_tag)
-
- dims = len(full_tags)
- full_rho = np.zeros((dims, dims))
- dims = dims // inj_lvls
-
- # if correlation between different levels of severity is considered, add that to the matrix
- if rho_lvls:
- rho_i = self._create_correlation_matrix(rho_target, c_target=-1,
- include_DSG=True,
- include_DS=True)
- for i in range(inj_lvls):
- for j in range(inj_lvls):
- full_rho[i * dims:(i + 1) * dims,
- j * dims:(j + 1) * dims] = rho_i
-
- # and now add the values around the main diagonal
- for i in range(inj_lvls):
- rho_i = self._create_correlation_matrix(rho_target, c_target=-1,
- include_DSG=True,
- include_DS=True)
- full_rho[i * dims:(i + 1) * dims, i * dims:(i + 1) * dims] = rho_i
-
- # finally, remove the unnecessary lines
- to_remove = np.where(full_theta == 0)[0]
- full_rho = np.delete(full_rho, to_remove, axis=0)
- full_rho = np.delete(full_rho, to_remove, axis=1)
-
- full_theta, full_sig, full_tags = [np.delete(f_vals, to_remove)
- for f_vals in
- [full_theta, full_sig, full_tags]]
-
- tr_upper = 1. + (1. - full_theta) / full_theta
-
- if full_tags.size > 0:
- # Create the RVs
- for ii, (ii_tag, sig_ii, theta_ii) in enumerate(
- zip(full_tags, full_sig, full_theta)):
- self._RV_reg.add_RV(RandomVariable(
- name=ii_tag, distribution='normal',
- theta=[1.0, sig_ii],
- truncation_limits=[0., tr_upper[ii]]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'DV_INJ_set', list(self._RV_reg.RVs(full_tags).values()),
- full_rho))
-
- # full_COV = np.outer(full_sig, full_sig) * full_rho
- #
- # if full_tags.size > 0:
- # injury_RV = RandomVariable(ID=402,
- # dimension_tags=full_tags,
- # distribution_kind='normal',
- # theta=np.ones(len(full_sig)),
- # COV=full_COV,
- # corr_ref='post',
- # truncation_limits=[np.zeros(len(full_sig)),
- # tr_upper])
- else:
- full_tags = None
-
- return full_tags
-
- def _create_fragility_groups(self):
-
- RVd = self._RV_dict
- DVs = self._AIM_in['decision_variables']
-
- # create a list for the fragility groups
- FG_dict = dict()
-
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id in s_fg_keys:
- log_msg('\t{}...'.format(c_id))
- comp = self._FG_in[c_id]
-
- FG_ID = len(FG_dict.keys())+1
-
- # create a list for the performance groups
- performance_groups = []
-
- # one group for each of the stories prescribed by the user
- PG_locations = comp['locations']
- PG_directions = comp['directions']
- PG_csg_lists = comp['csg_weights']
- PG_dists = comp['distribution_kind']
- PG_qnts = comp['quantities']
- for loc, dir_, csg_list, dist, qnt, in zip(
- PG_locations, PG_directions, PG_csg_lists, PG_dists, PG_qnts):
- PG_ID = 10000 * FG_ID + 10 * loc + dir_
-
- # get the quantity
- if dist == 'N/A':
- QNT = qnt
- else:
- # QNT = RandomVariableSubset(RVd['QNT'],
- # tags=[f'{c_id}-QNT-{loc}-{dir_}', ])
- QNT = self._QNT_dict[f'{c_id}-QNT-{loc}-{dir_}']
-
- # create the damage objects
- # consequences are calculated on a performance group level
-
- # create a list for the damage state groups and their tags
- DSG_list = []
- d_tags = []
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i, DSG_ID in enumerate(s_dsg_keys):
- DSG = comp['DSG_set'][DSG_ID]
- d_tags.append('FF-' + c_id + '-' + DSG_ID)
-
- # create a list for the damage states
- DS_set = []
-
- s_ds_keys = sorted(DSG['DS_set'].keys())
- for ds_i, DS_ID in enumerate(s_ds_keys):
- DS = DSG['DS_set'][DS_ID]
-
- # create the consequence functions
- if DVs['rec_cost']:
- data = DS['repair_cost']
- f_median = prep_bounded_multilinear_median_DV(
- **{k: data.get(k, None) for k in
- ('medians', 'quantities')})
- #('median_max', 'median_min',
- # 'quantity_lower', 'quantity_upper')})
- cf_tag = 'REP-' + c_id + '-' + DSG_ID + '-' + DS_ID + \
- '-cost' + \
- '-LOC-{}-DIR-{}'.format(loc, dir_)
- CF_RV = self._DV_REP_dict[cf_tag]
- # CF_RV = RandomVariableSubset(RVd['DV_REP'],
- # tags=cf_tag)
- CF_cost = ConsequenceFunction(DV_median=f_median,
- DV_distribution=CF_RV)
- else:
- CF_cost = None
-
- if DVs['rec_time']:
- data = DS['repair_time']
- f_median = prep_bounded_multilinear_median_DV(
- **{k: data.get(k, None) for k in
- ('medians', 'quantities')})
- #('median_max', 'median_min',
- # 'quantity_lower', 'quantity_upper')})
- cf_tag = 'REP-' + c_id + '-' + DSG_ID + '-' + DS_ID + \
- '-time' + \
- '-LOC-{}-DIR-{}'.format(loc, dir_)
- CF_RV = self._DV_REP_dict[cf_tag]
- # CF_RV = RandomVariableSubset(RVd['DV_REP'],
- # tags=cf_tag)
- CF_time = ConsequenceFunction(DV_median=f_median,
- DV_distribution=CF_RV)
- else:
- CF_time = None
-
- if (DVs['red_tag']) and ('red_tag' in DS.keys()):
- data = DS['red_tag']
- if data['theta'] > 0:
- f_median = prep_constant_median_DV(data['theta'])
- cf_tag = 'RED-' + c_id + '-' + DSG_ID + '-' + DS_ID + \
- '-LOC-{}-DIR-{}'.format(loc, dir_)
- CF_RV = self._DV_RED_dict[cf_tag]
- # CF_RV = RandomVariableSubset(RVd['DV_RED'],
- # tags=cf_tag)
- CF_red_tag = ConsequenceFunction(DV_median=f_median,
- DV_distribution=CF_RV)
- else:
- CF_red_tag = None
- else:
- CF_red_tag = None
-
- if (DVs['injuries']) and ('injuries' in DS.keys()):
- CF_inj_set = []
- for inj_i, theta in enumerate(DS['injuries']['theta']):
- if theta > 0.:
- f_median = prep_constant_median_DV(theta)
- cf_tag = 'INJ-' + c_id + '-' + DSG_ID + '-' + DS_ID + \
- '-{}-LOC-{}-DIR-{}'.format(inj_i, loc, dir_)
- CF_RV = self._DV_INJ_dict[cf_tag]
- # CF_RV = RandomVariableSubset(RVd['DV_INJ'],
- # tags=cf_tag)
- CF_inj_set.append(ConsequenceFunction(
- DV_median=f_median,
- DV_distribution=CF_RV))
- else:
- CF_inj_set.append(None)
- else:
- CF_inj_set = [None,]
-
- # add the DS to the list
- if 'affected_area' in DS.keys():
- AA = DS['affected_area']
- else:
- AA = 0.0
- # TODO: make this smarter by making affected_area a property of DS
- DS_set.append(DamageState(
- ID=ds_i + 1,
- description=DS['description'],
- weight=DS['weight'],
- affected_area=AA,
- repair_cost_CF=CF_cost,
- reconstruction_time_CF=CF_time,
- red_tag_CF=CF_red_tag,
- injuries_CF_set=CF_inj_set))
-
- # add the DSG to the list
- DSG_list.append(DamageStateGroup(
- ID=dsg_i + 1,
- DS_set=DS_set,
- DS_set_kind=DSG['DS_set_kind']))
-
- # create the fragility functions
- FF_set = []
- #CSG_this = np.where(comp['directions']==dir_)[0]
- #PG_weights = np.asarray(comp['csg_weights'])[CSG_this]
- # normalize the weights
- #PG_weights /= sum(PG_weights)
- for csg_id, __ in enumerate(csg_list):
- # assign the appropriate random variable to the fragility
- # function
- ff_tags = [t + f'-LOC-{loc}-DIR-{dir_}-CSG-{csg_id}'
- for t in d_tags]
- EDP_limit = [self._FF_dict[ff_tag] for ff_tag in ff_tags]
- # RandomVariableSubset(RVd['FR-' + c_id],
- # tags=ff_tags)
- FF_set.append(FragilityFunction(EDP_limit))
-
- # create the performance group
- PG = PerformanceGroup(ID=PG_ID,
- location=loc,
- quantity=QNT,
- fragility_functions=FF_set,
- DSG_set=DSG_list,
- csg_weights=csg_list,
- direction=dir_
- )
- performance_groups.append(PG)
-
- # create the fragility group
- FG = FragilityGroup(ID=FG_ID,
- #kind=comp['kind'],
- demand_type=comp['demand_type'],
- performance_groups=performance_groups,
- directional=comp['directional'],
- correlation=comp['correlation'],
- demand_location_offset=comp['offset'],
- incomplete=comp['incomplete'],
- name=str(FG_ID) + ' - ' + comp['ID'],
- description=comp['description'],
- unit=comp['unit']
- )
-
- FG_dict.update({comp['ID']:FG})
-
- return FG_dict
-
- def _sample_event_time(self):
-
- sample_count = self._AIM_in['general']['realizations']
-
- # month - uniform distribution over [0,11]
- month = np.random.randint(0, 12, size=sample_count)
-
- # weekday - binomial with p=5/7
- weekday = np.random.binomial(1, 5. / 7., size=sample_count)
-
- # hour - uniform distribution over [0,23]
- hour = np.random.randint(0, 24, size=sample_count)
-
- data = pd.DataFrame(data={'month' : month,
- 'weekday?': weekday,
- 'hour' : hour},
- dtype=int)
-
- return data
-
- def _get_population(self):
- """
- Use the population characteristics to generate random population samples.
-
- Returns
- -------
-
- """
- POPin = self._POP_in
- TIME = self._TIME
-
- POP = pd.DataFrame(
- np.ones((len(TIME.index), len(POPin['peak']))) * POPin['peak'],
- columns=['LOC' + str(loc + 1)
- for loc in range(len(POPin['peak']))])
-
- # if there is a temporal population model available...
- if 'weekday' in POPin.keys():
- weekdays = TIME[TIME['weekday?'] == 1].index
- weekends = TIME[~TIME.index.isin(weekdays)].index
-
- for col in POP.columns.values:
- POP.loc[weekdays, col] = (
- POP.loc[weekdays, col] *
- np.array(POPin['weekday']['daily'])[
- TIME.loc[weekdays, 'hour'].values.astype(int)] *
- np.array(POPin['weekday']['monthly'])[
- TIME.loc[weekdays, 'month'].values.astype(int)])
-
- POP.loc[weekends, col] = (
- POP.loc[weekends, col] *
- np.array(POPin['weekend']['daily'])[
- TIME.loc[weekends, 'hour'].values.astype(int)] *
- np.array(POPin['weekend']['monthly'])[
- TIME.loc[weekends, 'month'].values.astype(int)])
-
- return POP
-
- def _calc_collapses(self):
-
- # There are three options for determining which realizations ended in
- # collapse.
- GI = self._AIM_in['general']
- GR = GI['response']
- realizations = self._AIM_in['general']['realizations']
-
- # 1, The simplest case: prescribed collapse rate
- if GR['coll_prob'] != 'estimated':
- collapsed_IDs = np.random.choice(
- realizations,
- size=int(GR['coll_prob']*realizations),
- replace=False)
-
- # 2, Collapses estimated using EDP results
- elif GR['CP_est_basis'] == 'raw EDP':
- demand_data = []
- collapse_limits = []
- s_edp_keys = sorted(self._EDP_in.keys())
- for d_id in s_edp_keys:
- d_list = self._EDP_in[d_id]
- for i in range(len(d_list)):
- demand_data.append(d_list[i]['raw_data'])
-
- coll_lim = GI['collapse_limits'][d_id]
- if coll_lim is None:
- coll_lim = np.inf
-
- collapse_limits.append([0., coll_lim])
-
- collapse_limits = np.transpose(np.asarray(collapse_limits))
- demand_data = np.transpose(np.asarray(demand_data))
-
- EDP_filter = np.all(
- [np.all(demand_data > collapse_limits[0], axis=1),
- np.all(demand_data < collapse_limits[1], axis=1)],
- axis=0)
- coll_prob = 1.0 - sum(EDP_filter)/len(EDP_filter)
- collapsed_IDs = np.random.choice(
- realizations,
- size=int(coll_prob * realizations),
- replace=False)
-
- # 3, Collapses estimated using sampled EDP distribution
- elif GR['CP_est_basis'] == 'sampled EDP':
- collapsed_IDs = np.array([])
- s_edp_keys = sorted(self._EDP_dict.keys())
- for demand_ID in s_edp_keys:
- demand = self._EDP_dict[demand_ID]
- kind = demand_ID[4:7]
- collapse_limit = self._AIM_in['general']['collapse_limits'][kind]
- if collapse_limit is not None:
- EDP_samples = demand.samples_DF
- coll_df = EDP_samples[EDP_samples > collapse_limit]
- collapsed_IDs = np.concatenate(
- (collapsed_IDs, coll_df.index.values))
-
- # get a list of IDs of the collapsed cases
- collapsed_IDs = np.unique(collapsed_IDs).astype(int)
-
- COL = pd.DataFrame(np.zeros(realizations), columns=['COL', ])
- COL.loc[collapsed_IDs, 'COL'] = 1
-
- return COL, collapsed_IDs
-
- def _calc_damage(self):
-
- ncID = self._ID_dict['non-collapse']
- NC_samples = len(ncID)
- DMG = pd.DataFrame()
-
- s_fg_keys = sorted(self._FG_dict.keys())
- for fg_id in s_fg_keys:
- log_msg('\t\t{}...'.format(fg_id))
- FG = self._FG_dict[fg_id]
-
- PG_set = FG._performance_groups
-
- DS_list = []
- for DSG in PG_set[0]._DSG_set:
- for DS in DSG._DS_set:
- DS_list.append(str(DSG._ID) + '_' + str(DS._ID))
- d_count = len(DS_list)
-
- MI = pd.MultiIndex.from_product([[FG._ID, ],
- [pg._ID for pg in PG_set],
- DS_list],
- names=['FG', 'PG', 'DSG_DS'])
-
- FG_damages = pd.DataFrame(np.zeros((NC_samples, len(MI))),
- columns=MI,
- index=ncID)
-
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
- if isinstance(PG._quantity, RandomVariable):
- PG_qnt = PG._quantity.samples_DF.loc[ncID]
- else:
- PG_qnt = pd.Series(np.ones(NC_samples) * PG._quantity,
- index=ncID)
-
- # get the corresponding demands
- if not FG._directional:
- demand_ID_list = []
-
- for demand_ID in self._EDP_dict.keys():
- if demand_ID[4:7] == FG._demand_type:
- demand_data = demand_ID.split('-')
- if int(demand_data[3]) == PG._location + FG._demand_location_offset:
- demand_ID_list.append(demand_ID)
-
- EDP_samples = self._EDP_dict[demand_ID_list[0]].samples_DF.loc[ncID]
- if len(demand_ID_list)>1:
- for demand_ID in demand_ID_list[1:]:
- new_samples = self._EDP_dict[demand_ID].samples_DF.loc[ncID]
- EDP_samples.update(
- pd.Series(np.maximum(new_samples.values,EDP_samples.values),
- index=EDP_samples.index))
-
- # scale the max of inputs by 1.2 as per FEMA P58 vol 2 3.2.3
- EDP_samples *= 1.2
-
- else:
- demand_ID = ('EDP-' + FG._demand_type +
- '-LOC-' + str(PG._location + FG._demand_location_offset) +
- '-DIR-' + str(PG._direction))
-
- if demand_ID in self._EDP_dict.keys():
- EDP_samples = self._EDP_dict[demand_ID].samples_DF.loc[ncID]
- else:
- # If the required demand is not available, then we are most
- # likely analyzing a 3D structure using results from a 2D
- # simulation. The best thing we can do in that particular
- # case is to use the EDP from the 1 direction for all other
- # directions.
- demand_ID = ('EDP-' + FG._demand_type +
- '-LOC-' + str(PG._location + FG._demand_location_offset) + '-DIR-1')
- EDP_samples = self._EDP_dict[demand_ID].samples_DF.loc[ncID]
-
- csg_w_list = PG._csg_weights
-
- for csg_i, csg_w in enumerate(csg_w_list):
- DSG_df = PG._FF_set[csg_i].DSG_given_EDP(EDP_samples)
-
- for DSG in PG._DSG_set:
- in_this_DSG = DSG_df[DSG_df.values == DSG._ID].index
- if DSG._DS_set_kind == 'single':
- DS = DSG._DS_set[0]
- DS_tag = str(DSG._ID) + '_' + str(DS._ID)
- FG_damages.loc[in_this_DSG,
- (FG._ID, PG_ID, DS_tag)] += csg_w
- elif DSG._DS_set_kind == 'mutually exclusive':
- mut_ex_id = f'DSG-{fg_id}-DSG-{DSG._ID}-' \
- f'LOC-{PG._location}-' \
- f'DIR-{PG._direction}-CSG-{csg_i}'
- DS_df = self._DSG_dict[mut_ex_id].samples_DF.loc[in_this_DSG]
-
- # DS_weights = [DS._weight for DS in DSG._DS_set]
- # DS_RV = RandomVariable(
- # ID=-1, dimension_tags=['me_DS', ],
- # distribution_kind='multinomial',
- # p_set=DS_weights)
- # DS_df = DS_RV.sample_distribution(
- # len(in_this_DSG)) + 1
-
- for DS in DSG._DS_set:
- DS_tag = str(DSG._ID) + '_' + str(DS._ID)
- in_this_DS = DS_df[DS_df.values == DS._ID-1].index
- FG_damages.loc[in_this_DS,
- (FG._ID, PG_ID, DS_tag)] += csg_w
- elif DSG._DS_set_kind == 'simultaneous':
- DS_weights = [DS._weight for DS in DSG._DS_set]
- DS_df = np.random.uniform(
- size=(len(in_this_DSG), len(DS_weights)))
- which_DS = DS_df < DS_weights
- any_DS = np.any(which_DS, axis=1)
- no_DS_ids = np.where(any_DS == False)[0]
-
- while len(no_DS_ids) > 0:
- DS_df_add = np.random.uniform(
- size=(len(no_DS_ids), len(DS_weights)))
- which_DS_add = DS_df_add < DS_weights
- which_DS[no_DS_ids] = which_DS_add
-
- any_DS = np.any(which_DS_add, axis=1)
- no_DS_ids = no_DS_ids[
- np.where(any_DS == False)[0]]
-
- for ds_i, DS in enumerate(DSG._DS_set):
- DS_tag = str(DSG._ID) + '_' + str(DS._ID)
- in_this_DS = which_DS[:, ds_i]
- FG_damages.loc[in_this_DSG[in_this_DS], (
- FG._ID, PG_ID, DS_tag)] += csg_w
-
- else:
- raise ValueError(
- "Unknown damage state type: {}".format(
- DSG._DS_set_kind)
- )
-
- FG_damages.iloc[:,pg_i * d_count:(pg_i + 1) * d_count] = \
- FG_damages.mul(PG_qnt, axis=0)
-
- DMG = pd.concat((DMG, FG_damages), axis=1)
-
- DMG.index = ncID
-
- # sort the columns to enable index slicing later
- DMG = DMG.sort_index(axis=1, ascending=True)
-
- return DMG
-
- def _calc_red_tag(self):
- idx = pd.IndexSlice
-
- ncID = self._ID_dict['non-collapse']
- NC_samples = len(ncID)
- DV_RED = pd.DataFrame()
-
- s_fg_keys = sorted(self._FG_dict.keys())
- for fg_id in s_fg_keys:
- FG = self._FG_dict[fg_id]
-
- PG_set = FG._performance_groups
-
- DS_list = self._DMG.loc[:, idx[FG._ID, PG_set[0]._ID, :]].columns
- DS_list = DS_list.levels[2][DS_list.codes[2]].values
-
- MI = pd.MultiIndex.from_product([[FG._ID, ],
- [pg._ID for pg in PG_set],
- DS_list],
- names=['FG', 'PG', 'DSG_DS'])
-
- FG_RED = pd.DataFrame(np.zeros((NC_samples, len(MI))),
- columns=MI,
- index=ncID)
-
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
- if isinstance(PG._quantity, RandomVariable):
- PG_qnt = PG._quantity.samples_DF.loc[ncID]
- else:
- PG_qnt = pd.Series(np.ones(NC_samples) * PG._quantity,
- index=ncID)
-
- PG_DMG = self._DMG.loc[:, idx[FG._ID, PG_ID, :]].div(PG_qnt,axis=0)
-
- for d_i, d_tag in enumerate(DS_list):
- dsg_i = int(d_tag[0]) - 1
- ds_i = int(d_tag[-1]) - 1
-
- DS = PG._DSG_set[dsg_i]._DS_set[ds_i]
-
- if DS._red_tag_CF is not None:
- RED_samples = DS.red_tag_dmg_limit(
- sample_size=NC_samples)
- RED_samples.index = ncID
-
- is_red = PG_DMG.loc[:, (FG._ID, PG_ID, d_tag)].sub(
- RED_samples, axis=0)
-
- FG_RED.loc[:, (FG._ID, PG_ID, d_tag)] = (
- is_red > 0.).astype(int)
- else:
- FG_RED.drop(labels=[(FG._ID, PG_ID, d_tag), ], axis=1,
- inplace=True)
-
- if FG_RED.size > 0:
- DV_RED = pd.concat((DV_RED, FG_RED), axis=1)
-
- # sort the columns to enable index slicing later
- DV_RED = DV_RED.sort_index(axis=1, ascending=True)
-
- return DV_RED
-
- def _calc_irreparable(self):
-
- ncID = self._ID_dict['non-collapse']
- NC_samples = len(ncID)
-
- # determine which realizations lead to irreparable damage
- # get the max residual drifts
- RID_max = None
- PID_max = None
- s_edp_keys = sorted(self._EDP_dict.keys())
- for demand_ID in s_edp_keys:
- demand = self._EDP_dict[demand_ID]
- kind = demand_ID[4:7]
- if kind == 'RID':
- r_max = demand.samples_DF.loc[ncID].values
- if RID_max is None:
- RID_max = r_max
- else:
- RID_max = np.max((RID_max, r_max), axis=0)
- elif kind == 'PID':
- d_max = demand.samples_DF.loc[ncID].values
- if PID_max is None:
- PID_max = d_max
- else:
- PID_max = np.max((PID_max, d_max), axis=0)
-
- if RID_max is None:
- if PID_max is not None:
- # we need to estimate residual drifts based on peak drifts
- RID_max = np.zeros(NC_samples)
-
- # based on FEMA P-58 Vol. 1 5.4
- delta_y = self._AIM_in['general']['yield_drift']
- small = PID_max < delta_y
- medium = PID_max < 4 * delta_y
- large = PID_max >= 4 * delta_y
-
- RID_max[large] = PID_max[large] - 3 * delta_y
- RID_max[medium] = 0.3 * (PID_max[medium] - delta_y)
- RID_max[small] = 0.
-
- # add extra uncertainty
- eps = np.random.normal(scale=0.2, size=len(ncID) - np.sum(small))
- RID_max[RID_max>0] = np.exp(np.log(RID_max[RID_max>0]) + eps)
-
- else:
- # If no drift data is available, then we cannot provide an estimate
- # of irreparability. We assume that all non-collapse realizations
- # are repairable in this case.
- return np.array([])
-
- # get the probabilities of irreparability
- # irrep_frag = self._AIM_in['general']['irreparable_res_drift']
- # RV_irrep = RandomVariable(ID=-1, dimension_tags=['RED_irrep', ],
- # distribution_kind='lognormal',
- # theta=irrep_frag['Median'],
- # COV=irrep_frag['Beta'] ** 2.
- # )
- # RED_irrep = RV_irrep.sample_distribution(NC_samples)['RED_irrep'].values
-
- RED_irrep = self._RV_reg.RV['RED_irrep'].samples_DF.loc[ncID].values
-
- # determine if the realizations are repairable
- irreparable = RID_max > RED_irrep
- irreparable_IDs = ncID[np.where(irreparable)[0]]
-
- return irreparable_IDs
-
- def _calc_repair_cost_and_time(self):
-
- idx = pd.IndexSlice
- DVs = self._AIM_in['decision_variables']
-
- DMG_by_FG_and_DS = self._DMG.groupby(level=[0, 2], axis=1).sum()
-
- repID = self._ID_dict['repairable']
- REP_samples = len(repID)
- DV_COST = pd.DataFrame(np.zeros((REP_samples, len(self._DMG.columns))),
- columns=self._DMG.columns, index=repID)
- DV_TIME = DV_COST.copy()
-
- s_fg_keys = sorted(self._FG_dict.keys())
- for fg_id in s_fg_keys:
- FG = self._FG_dict[fg_id]
-
- PG_set = FG._performance_groups
-
- DS_list = self._DMG.loc[:, idx[FG._ID, PG_set[0]._ID, :]].columns
- DS_list = DS_list.levels[2][DS_list.codes[2]].values
-
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
-
- for d_i, d_tag in enumerate(DS_list):
- dsg_i = int(d_tag[0]) - 1
- ds_i = int(d_tag[-1]) - 1
-
- DS = PG._DSG_set[dsg_i]._DS_set[ds_i]
-
- TOT_qnt = DMG_by_FG_and_DS.loc[repID, (FG._ID, d_tag)]
- PG_qnt = self._DMG.loc[repID,
- (FG._ID, PG_ID, d_tag)]
-
- # repair cost
- if DVs['rec_cost']:
- COST_samples = DS.unit_repair_cost(quantity=TOT_qnt)
- DV_COST.loc[:,
- (FG._ID, PG_ID, d_tag)] = COST_samples * PG_qnt
-
- if DVs['rec_time']:
- # repair time
- TIME_samples = DS.unit_reconstruction_time(quantity=TOT_qnt)
- DV_TIME.loc[:,
- (FG._ID, PG_ID, d_tag)] = TIME_samples * PG_qnt
-
- # sort the columns to enable index slicing later
- if DVs['rec_cost']:
- DV_COST = DV_COST.sort_index(axis=1, ascending=True)
- else:
- DV_COST = None
- if DVs['rec_time']:
- DV_TIME = DV_TIME.sort_index(axis=1, ascending=True)
- else:
- DV_TIME = None
-
- return DV_COST, DV_TIME
-
- def _calc_collapse_injuries(self):
-
- inj_lvls = self._inj_lvls
-
- # calculate injuries for the collapsed cases
- # generate collapse modes
- colID = self._ID_dict['collapse']
- C_samples = len(colID)
-
- if C_samples > 0:
-
- inj_lvls = self._inj_lvls
- coll_modes = self._AIM_in['collapse_modes']
- P_keys = [cmk for cmk in coll_modes.keys()]
- # P_modes = [coll_modes[k]['w'] for k in P_keys]
-
- # create the DataFrame that collects the decision variables
- inj_cols = ['CM',]
- for i in range(inj_lvls):
- inj_cols.append('INJ-{}'.format(i))
- COL_INJ = pd.DataFrame(np.zeros((C_samples, inj_lvls + 1)),
- columns=inj_cols, index=colID)
-
- # CM_RV = RandomVariable(ID=-1, dimension_tags=['CM', ],
- # distribution_kind='multinomial',
- # p_set=P_modes)
- #COL_INJ['CM'] = CM_RV.sample_distribution(C_samples).values
- COL_INJ['CM'] = self._RV_reg.RV['CM'].samples_DF.loc[colID].values
-
- # get the popoulation values corresponding to the collapsed cases
- P_sel = self._POP.loc[colID]
-
- # calculate the exposure of the popoulation
- for cm_i, cmk in enumerate(P_keys):
- mode_IDs = COL_INJ[COL_INJ['CM'] == cm_i].index
- CFAR = coll_modes[cmk]['affected_area']
- INJ = coll_modes[cmk]['injuries']
- for loc_i in range(len(CFAR)):
- loc_label = 'LOC{}'.format(loc_i + 1)
- if loc_label in P_sel.columns:
- for inj_i in range(inj_lvls):
- INJ_i = P_sel.loc[mode_IDs, loc_label] * CFAR[loc_i] * \
- INJ[inj_i]
- COL_INJ.loc[mode_IDs, 'INJ-{}'.format(inj_i)] = (
- COL_INJ.loc[mode_IDs, 'INJ-{}'.format(inj_i)].add(INJ_i, axis=0).values)
-
- return COL_INJ
-
- else:
- return None
-
- def _calc_non_collapse_injuries(self):
-
- idx = pd.IndexSlice
-
- ncID = self._ID_dict['non-collapse']
- NC_samples = len(ncID)
- DV_INJ_dict = dict([(i, pd.DataFrame(np.zeros((NC_samples,
- len(self._DMG.columns))),
- columns=self._DMG.columns,
- index=ncID))
- for i in range(self._inj_lvls)])
- s_fg_keys = sorted(self._FG_dict.keys())
- for fg_id in s_fg_keys:
- FG = self._FG_dict[fg_id]
-
- PG_set = FG._performance_groups
-
- DS_list = self._DMG.loc[:, idx[FG._ID, PG_set[0]._ID, :]].columns
- DS_list = DS_list.levels[2][DS_list.codes[2]].values
-
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
-
- for d_i, d_tag in enumerate(DS_list):
- dsg_i = int(d_tag[0]) - 1
- ds_i = int(d_tag[-1]) - 1
-
- DS = PG._DSG_set[dsg_i]._DS_set[ds_i]
-
- if DS._affected_area > 0.:
- P_affected = (self._POP.loc[ncID]
- * DS._affected_area /
- self._AIM_in['general']['plan_area'])
-
- QNT = self._DMG.loc[:, (FG._ID, PG_ID, d_tag)]
-
- # estimate injuries
- for i in range(self._inj_lvls):
- INJ_samples = DS.unit_injuries(severity_level=i,
- sample_size=NC_samples)
- if INJ_samples is not None:
- INJ_samples.index = ncID
- P_aff_i = P_affected.loc[:,
- 'LOC{}'.format(PG._location)]
- INJ_i = INJ_samples * P_aff_i * QNT
- DV_INJ_dict[i].loc[:,
- (FG._ID, PG_ID, d_tag)] = INJ_i
-
- # remove the useless columns from DV_INJ
- for i in range(self._inj_lvls):
- DV_INJ = DV_INJ_dict[i]
- DV_INJ_dict[i] = DV_INJ.loc[:, (DV_INJ != 0.0).any(axis=0)]
-
- # sort the columns to enable index slicing later
- for i in range(self._inj_lvls):
- DV_INJ_dict[i] = DV_INJ_dict[i].sort_index(axis=1, ascending=True)
-
- return DV_INJ_dict
-
-
-[docs]class HAZUS_Assessment(Assessment):
- """
- An Assessment class that implements the damage and loss assessment method
- following the HAZUS Technical Manual and the HAZUS software.
-
- Parameters
- ----------
- hazard: {'EQ', 'HU'}
- Identifies the type of hazard. EQ corresponds to earthquake, HU
- corresponds to hurricane.
- default: 'EQ'.
- inj_lvls: int
- Defines the discretization used to describe the severity of injuries.
- The HAZUS earthquake methodology uses 4 levels.
- default: 4
- """
- def __init__(self, hazard='EQ', inj_lvls = 4, log_file=True):
- super(HAZUS_Assessment, self).__init__(log_file)
-
- self._inj_lvls = inj_lvls
- self._hazard = hazard
- self._assessment_type = 'HAZUS_{}'.format(hazard)
-
- log_msg('type: HAZUS Assessment')
- log_msg('hazard: {}'.format(self._hazard))
- log_msg(log_div)
-
-[docs] def read_inputs(self, path_DL_input, path_EDP_input, verbose=False):
- """
- Read and process the input files to describe the loss assessment task.
-
- Parameters
- ----------
- path_DL_input: string
- Location of the Damage and Loss input file. The file is expected to
- be a JSON with data stored in a standard format described in detail
- in the Input section of the documentation.
- path_EDP_input: string
- Location of the EDP input file. The file is expected to follow the
- output formatting of Dakota. The Input section of the documentation
- provides more information about the expected formatting.
- verbose: boolean, default: False
- If True, the method echoes the information read from the files.
- This can be useful to ensure that the information in the file is
- properly read by the method.
-
- """
-
- super(HAZUS_Assessment, self).read_inputs(path_DL_input,
- path_EDP_input, verbose)
-
- # assume that the asset is a building
- # TODO: If we want to apply HAZUS to non-building assets, several parts of this methodology need to be extended.
- BIM = self._AIM_in
-
- # read component and population data ----------------------------------
- # components
- log_msg('\tDamage and Loss data files...')
- self._FG_in = read_component_DL_data(
- self._AIM_in['data_sources']['path_CMP_data'], BIM['components'],
- assessment_type=self._assessment_type, verbose=verbose)
-
- data = self._FG_in
- log_msg('\t\tAvailable Fragility Groups:')
- for key, val in data.items():
- log_msg('\t\t\t{} demand:{} PGs: {}'.format(key, val['demand_type'], len(val['locations'])))
-
- # population (if needed)
- if self._AIM_in['decision_variables']['injuries']:
-
- if BIM['general']['event_time'] is None:
- log_msg('\tPopulation data files...')
- POP = read_population_distribution(
- self._AIM_in['data_sources']['path_POP_data'],
- BIM['general']['occupancy_type'],
- assessment_type=self._assessment_type,
- verbose=verbose)
- else:
- POP = {'peak': None}
-
- POP['peak'] = BIM['general']['population']
- self._POP_in = POP
-
-[docs] def define_random_variables(self):
- """
- Define the random variables used for loss assessment.
-
- Following the HAZUS methodology, only the groups of parameters below
- are considered random. Correlations within groups are not considered
- because each Fragility Group has only one Performance Group with a
- in this implementation.
-
- 1. Demand (EDP) distribution
-
- Describe the uncertainty in the demands. Unlike other random variables,
- the EDPs are characterized by the EDP input data provided earlier. All
- EDPs are handled in one multivariate lognormal distribution. If more
- than one sample is provided, the distribution is fit to the EDP data.
- Otherwise, the provided data point is assumed to be the median value
- and the additional uncertainty prescribed describes the dispersion. See
- _create_RV_demands() for more details.
-
- 2. Fragility EDP limits
-
- Describe the uncertainty in the EDP limit that corresponds to
- exceedance of each Damage State. EDP limits are grouped by Fragility
- Groups. See _create_RV_fragilities() for details.
-
- """
- super(HAZUS_Assessment, self).define_random_variables()
-
- DEP = self._AIM_in['dependencies']
-
- # create the random variables -----------------------------------------
- self._RV_dict = {}
-
- # quantities 100
- log_msg('\tQuantities...')
- QNT_tags = self._create_RV_quantities(DEP['quantities'])
-
- # self._RV_dict.update({
- # 'QNT': self._create_RV_quantities(DEP['quantities'])})
-
- if QNT_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(QNT_tags)))
- self._QNT_dict = self._RV_reg.RVs(QNT_tags)
- else:
- log_msg('\t\tNone of the components have random quantities assigned')
-
- # fragilities 300
- log_msg('\tDamage State Limits...')
- s_fg_keys = sorted(self._FG_in.keys())
- self._FF_dict = {}
- log_msg('\t\tRV dimensions:')
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- FF_tags = self._create_RV_fragilities(c_id, comp,
- DEP['fragilities'])
- self._FF_dict.update(self._RV_reg.RVs(FF_tags))
-
- # self._RV_dict.update({
- # 'FR-' + c_name:
- # self._create_RV_fragilities(c_id, comp,DEP['fragilities'])})
- log_msg('\t\t\t{}: {}'.format(c_name, len(FF_tags)))
-
- # for key, val in self._RV_dict.items():
- # if 'FR-' in key:
- # log_msg('\t\t\t{}: {}'.format(key, len(val.theta)))
-
- # damages (mutually exclusive cases and later simultaneous ones too)
- log_msg('\tDamage States...')
- DS_tags = self._create_RV_damage_states()
-
- if DS_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(DS_tags)))
- self._DSG_dict = self._RV_reg.RVs(DS_tags)
- else:
- log_msg(
- '\t\tNone of the components have random damage states assigned')
-
- # decision variables
- DVs = self._AIM_in['decision_variables']
-
- if DVs['rec_time'] or DVs['rec_cost']:
- log_msg('\tReconstruction Costs and Times...')
- REP_tags = self._create_RV_repairs(
- DEP['rec_costs'], DEP['rec_times'], DEP['cost_and_time'])
-
- # self._RV_dict.update({
- # 'DV_REP': self._create_RV_repairs(DEP['rec_costs'],
- # DEP['rec_times'],
- # DEP['cost_and_time'])})
-
- if REP_tags is not None:
- log_msg('\t\tRV dimensions: {}'.format(len(REP_tags)))
- self._DV_REP_dict = self._RV_reg.RVs(REP_tags)
- else:
- log_msg('\t\tNone of the components have probabilistic '
- 'consequence functions')
-
- # demands 200
- log_msg('\tEDPs...')
-
- EDP_tags = self._create_RV_demands()
- self._EDP_dict = self._RV_reg.RVs(EDP_tags)
- #self._RV_dict.update({'EDP': self._create_RV_demands()})
-
- # sample the random variables -----------------------------------------
- log_msg()
- log_msg('Sampling the random variables...')
-
- realization_count = self._AIM_in['general']['realizations']
-
- self._RV_reg.generate_samples(sample_size=realization_count)
-
- # is_coupled = self._AIM_in['general']['coupled_assessment']
- #
- # s_rv_keys = sorted(self._RV_dict.keys())
- # for r_i in s_rv_keys:
- # rv = self._RV_dict[r_i]
- # if rv is not None:
- # log_msg('\t{}...'.format(r_i))
- # rv.sample_distribution(
- # sample_size=realization_count,
- # preserve_order=((r_i=='EDP') and is_coupled))
-
- log_msg('Sampling completed.')
-
-[docs] def define_loss_model(self):
- """
- Create the stochastic loss model based on the inputs provided earlier.
-
- Following the HAZUS methodology, the component assemblies specified in
- the Damage and Loss input file are used to create Fragility Groups.
- Each Fragility Group corresponds to one assembly that represents every
- component of the given type in the structure. See
- _create_fragility_groups() for more details about the creation of
- Fragility Groups.
-
- """
- super(HAZUS_Assessment, self).define_loss_model()
-
- # fragility groups
- self._FG_dict = self._create_fragility_groups()
-
- # demands
- # self._EDP_dict = dict(
- # [(tag, RandomVariableSubset(self._RV_dict['EDP'], tags=tag))
- # for tag in self._RV_dict['EDP']._dimension_tags])
-
-[docs] def calculate_damage(self):
- """
- Characterize the damage experienced in each random event realization.
-
- First, the time of the event (month, weekday/weekend, hour) is randomly
- generated for each realization. Given the event time, if we are interested
- in injuries, the number of people present at each floor of the building is
- sampled. The event time is only important if we are interested in injuries,
- but it is calculated every time because it is not a large overhead and it
- serves as the basis of indexing every other array.
-
- Second, the quantities of components in each damage state are estimated.
- See _calc_damage() for more details on damage estimation.
-
- Finally, the realizations that led to collapse are filtered from the damage
- data.
-
- """
- super(HAZUS_Assessment, self).calculate_damage()
-
- # event time - month, weekday, and hour realizations
- log_msg('\tSampling event time...')
- self._TIME = self._sample_event_time()
-
- # if we are interested in injuries...
- if self._AIM_in['decision_variables']['injuries']:
- # get the population conditioned on event time
- log_msg('\tSampling the population...')
- self._POP = self._get_population()
-
- # assume that all cases are non-collapse for damage assessment
- non_collapsed_IDs = self._TIME.index.values.astype(int)
- self._ID_dict.update({'non-collapse': non_collapsed_IDs})
-
- # calculate damage
- log_msg('\tCalculating the damage in the non-collapsed cases...')
- self._DMG = self._calc_damage()
-
- # apply the prescribed damge logic
- if self._AIM_in['damage_logic'] is not None:
- for DL in self._AIM_in['damage_logic']:
- if DL['type'] == 'propagate':
- # identify the source and target FG ids
- source_id = self._FG_dict[DL['source_FG']]._ID
- target_id = self._FG_dict[DL['target_FG']]._ID
-
- # get the source DMG info
- source_DMG = self._DMG.loc[:,source_id].groupby(level='DSG_DS', axis=1).sum()
-
- # get the PGs in the target FG
- target_PGs = self._FG_dict[DL['target_FG']]._performance_groups
-
- # for each PG
- for target_PG in target_PGs:
-
- # get the total quantity
- if isinstance(target_PG._quantity, RandomVariable):
- qnt_tot = target_PG._quantity.samples_DF.loc[self._DMG.index]
- else:
- qnt_tot = pd.DataFrame(np.ones(self._DMG.shape[0]) * target_PG._quantity,
- index=self._DMG.index)
-
- # get all DSG_DS combinations in the target_PG
- target_DSG_DS_list = self._DMG.loc[:, (target_id, target_PG._ID)].columns
-
- for source_DS, target_DS in DL['DS_links'].items():
-
- # get the damaged quantity
- qnt_dmg = self._DMG.loc[:, (target_id, target_PG._ID)].sum(axis=1).to_frame()
-
- # get the undamaged quantity
- qnt_undmg = qnt_tot - qnt_dmg
-
- # get the mapping based on source_dmg and source DS
- dmg_map = source_DMG.index.values[source_DMG.loc[:, source_DS] > 0.0]
-
- # get the damage states exceeded by target_DS
- exc_DS = target_DSG_DS_list[:np.where(target_DSG_DS_list == target_DS)[0][0]+1]
-
- # sum up the damage in the exceeded DSs + no damage
- exc_dmg = self._DMG.loc[dmg_map, idx[target_id, target_PG._ID, exc_DS]].sum(axis=1).to_frame()
-
- exc_dmg = exc_dmg + qnt_undmg.loc[dmg_map]
-
- # save this damage to the target_DS and zero to lower DSs
- for ds_i in exc_DS:
- self._DMG.loc[dmg_map, (target_id, target_PG._ID, ds_i)] = 0.0
- self._DMG.loc[dmg_map, (target_id, target_PG._ID, target_DS)] = exc_dmg.iloc[:,0].values
-
- else:
- log_msg(f'Unkown damage logic: {DL["type"]}')
-
- # collapses are indicated by the ultimate DS in HAZUS
- DMG_agg = self._DMG.groupby(level=2, axis=1).sum()
- if '4_2' in DMG_agg.columns:
- collapse_flag = DMG_agg['4_2']>0.
- else:
- collapse_flag = [False] * len(DMG_agg.index)
- self._ID_dict.update({'collapse':
- self._DMG[collapse_flag].index.values.astype(int)})
- # Note: Non-collapse IDs are not updated because we use the same
- # procedure to estimate injuries (and potentially other decision vars)
- # under collapse and non-collapse cases
-
- self._COL = pd.DataFrame(
- np.zeros(self._AIM_in['general']['realizations']),
- columns=['COL', ])
- self._COL.loc[collapse_flag, 'COL'] = 1
-
-[docs] def calculate_losses(self):
- """
- Characterize the consequences of damage in each random event realization.
-
- For the sake of efficiency, only the decision variables requested in
- the input file are estimated. The following consequences are handled by
- this method for a HAZUS assessment:
-
- Reconstruction time and cost
- Get a cost and time estimate for each Damage State in each Performance
- Group. For more information about estimating reconstruction cost and
- time see _calc_repair_cost_and_time() methods.
-
- Injuries
- The number of injuries are based on the probability of injuries of
- various severity specified in the component data file. For more
- information about estimating injuries _calc_non_collapse_injuries.
-
- """
- super(HAZUS_Assessment, self).calculate_losses()
- DVs = self._AIM_in['decision_variables']
-
- # reconstruction cost and time
- if DVs['rec_cost'] or DVs['rec_time']:
- # all damages are considered repairable in HAZUS
- repairable_IDs = self._ID_dict['non-collapse']
- self._ID_dict.update({'repairable': repairable_IDs})
- self._ID_dict.update({'irreparable': []})
-
- # reconstruction cost and time for repairable cases
- log_msg('\tCalculating Reconstruction cost and time...')
- DV_COST, DV_TIME = self._calc_repair_cost_and_time()
-
- if DVs['rec_cost']:
- self._DV_dict.update({'rec_cost': DV_COST})
-
- if DVs['rec_time']:
- self._DV_dict.update({'rec_time': DV_TIME})
-
- # injuries due to collapse
- if DVs['injuries']:
-
- # injuries in non-collapsed cases
- log_msg('\tCalculating Injuries in Non-Collapsed Cases...')
- DV_INJ_dict = self._calc_non_collapse_injuries()
-
- self._DV_dict.update({'injuries': DV_INJ_dict})
-
-[docs] def aggregate_results(self):
- """
-
- Returns
- -------
-
- """
-
- log_msg(log_div)
- log_msg('Aggregating results...')
-
- DVs = self._AIM_in['decision_variables']
-
- MI_raw = [
- ('collapses', 'collapsed'),
- ('highest damage state', 'S'),
- ('highest damage state', 'NSA'),
- ('highest damage state', 'NSD'),
- ('reconstruction', 'cost impractical'),
- ('reconstruction', 'cost'),
- ]
-
- if DVs['rec_time']:
- MI_raw += [
- #('reconstruction', 'time impractical?'),
- ('reconstruction', 'time'),
- ]
-
- if DVs['injuries']:
- MI_raw += [
- ('inhabitants', ''),
- ('injuries', 'sev1'),
- ('injuries', 'sev2'),
- ('injuries', 'sev3'),
- ('injuries', 'sev4'),
- ]
-
- if (DVs['injuries'] and
- (self._AIM_in['general']['event_time'] != 'off')):
- MI_raw += [
- ('event time', 'month'),
- ('event time', 'weekday?'),
- ('event time', 'hour')
- ]
-
- ncID = self._ID_dict['non-collapse']
- colID = self._ID_dict['collapse']
- if DVs['rec_cost'] or DVs['rec_time']:
- repID = self._ID_dict['repairable']
- irID = self._ID_dict['irreparable']
-
- MI = pd.MultiIndex.from_tuples(MI_raw)
-
- SUMMARY = pd.DataFrame(
- np.empty((self._AIM_in['general']['realizations'], len(MI))),
- columns=MI)
- SUMMARY[:] = np.NaN
-
- # event time (if needed)
- if (DVs['injuries'] and
- (self._AIM_in['general']['event_time'] != 'off')):
- for prop in ['month', 'weekday?', 'hour']:
- offset = 0
- if prop == 'month':
- offset = 1
- SUMMARY.loc[:, ('event time', prop)] = \
- self._TIME.loc[:, prop] + offset
-
- # collapses
- SUMMARY.loc[:, ('collapses', 'collapsed')] = self._COL.iloc[:, 0]
-
- # damage
- # remove the ground failure FGs first
- DMG = self._DMG.copy()
- for FG_name in self._FG_dict.keys():
- if FG_name.startswith('GF'):
- del DMG[self._FG_dict[FG_name]._ID]
-
- for comp_type in ['S', 'NSA', 'NSD']:
- fg_list = [self._FG_dict[fg]._ID for fg in self._FG_dict.keys() if fg.startswith(comp_type)]
-
- if len(fg_list)>0:
-
- DMG_agg = DMG.loc[:, fg_list].groupby(level=['DSG_DS',], axis=1).sum()
-
- DMG_agg['DS'] = 0
- for c_i, col in enumerate(DMG_agg.columns):
- if col != 'DS':
- DMG_agg.loc[DMG_agg.loc[:,col] > 0.0, 'DS'] = int(col[0])
-
- SUMMARY.loc[:, ('highest damage state', comp_type)] = DMG_agg['DS']
-
- # reconstruction cost
- if DVs['rec_cost']:
- SUMMARY.loc[ncID, ('reconstruction', 'cost')] = \
- self._DV_dict['rec_cost'].sum(axis=1)
-
- repl_cost = self._AIM_in['general']['replacement_cost']
- SUMMARY.loc[colID, ('reconstruction', 'cost')] = repl_cost
-
- SUMMARY.loc[:, ('reconstruction', 'cost impractical')] = 0
- repair_impractical_IDs = SUMMARY.loc[ncID, ('reconstruction', 'cost')] > repl_cost
- SUMMARY.loc[repair_impractical_IDs,
- ('reconstruction', 'cost impractical')] = 1
- SUMMARY.loc[repair_impractical_IDs,
- ('reconstruction', 'cost')] = repl_cost
-
- # only keep the non-collapsed cases in the DVs
- self._DV_dict['rec_cost'] = self._DV_dict['rec_cost'].loc[self._COL['COL'] == 0]
-
- # reconstruction time
- if DVs['rec_time']:
- SUMMARY.loc[ncID, ('reconstruction', 'time')] = \
- self._DV_dict['rec_time'].sum(axis=1)
-
- repl_time = self._AIM_in['general']['replacement_time']
- SUMMARY.loc[colID, ('reconstruction', 'time')] = repl_time
-
- SUMMARY.loc[repair_impractical_IDs, ('reconstruction', 'time')] = repl_time
-
- # only keep the non-collapsed cases in the DVs
- self._DV_dict['rec_time'] = self._DV_dict['rec_time'].loc[self._COL['COL'] == 0]
-
- # injuries
- if DVs['injuries']:
-
- # inhabitants
- SUMMARY.loc[:, ('inhabitants', '')] = self._POP.sum(axis=1)
-
- for sev_id in range(4):
- # both collapse and non-collapse cases
- sev_tag = 'sev{}'.format(sev_id+1)
- SUMMARY.loc[ncID, ('injuries', sev_tag)] = \
- self._DV_dict['injuries'][sev_id].sum(axis=1)
-
- # keep only the non-collapse damage data
- self._DMG = self._DMG.loc[self._COL['COL'] == 0]
-
- self._ID_dict['non-collapse'] = self._DV_dict['rec_cost'].index.values.astype(int)
-
- self._SUMMARY = SUMMARY.dropna(axis=1, how='all')
-
-[docs] def save_outputs(self, *args, **kwargs):
- """
-
- Returns
- -------
-
- """
- super(HAZUS_Assessment, self).save_outputs(*args, **kwargs)
-
- def _create_correlation_matrix(self, rho_target, c_target=-1,
- include_CSG=False,
- include_DSG=False, include_DS=False):
- """
-
- Parameters
- ----------
- rho_target
- c_target
- include_CSG
- include_DSG
- include_DS
-
- Returns
- -------
-
- """
-
- # set the correlation structure
- rho_FG, rho_PG, rho_LOC, rho_DIR, rho_CSG, rho_DS = np.zeros(6)
-
- if rho_target in ['FG', 'PG', 'DIR', 'LOC', 'CSG', 'ATC', 'DS']:
- rho_DS = 1.0
- if rho_target in ['FG', 'PG', 'DIR', 'LOC', 'CSG']:
- rho_CSG = 1.0
- if rho_target in ['FG', 'PG', 'DIR']:
- rho_DIR = 1.0
- if rho_target in ['FG', 'PG', 'LOC']:
- rho_LOC = 1.0
- if rho_target in ['FG', 'PG']:
- rho_PG = 1.0
- if rho_target == 'FG':
- rho_FG = 1.0
-
- L_D_list = []
- dims = []
- DS_list = []
- ATC_rho = []
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- if ((c_target == -1) or (c_id == c_target)):
- c_L_D_list = []
- c_DS_list = []
- ATC_rho.append(comp['correlation'])
-
- if include_DSG:
- DS_count = 0
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- if include_DS:
- DS_count += len(DSG['DS_set'])
- else:
- DS_count += 1
- else:
- DS_count = 1
-
- #for loc in comp['locations']:
- # if include_CSG:
- # u_dirs = comp['directions']
- # else:
- # u_dirs = np.unique(comp['directions'])
- # c_L_D_list.append([])
- # for dir_ in u_dirs:
- # c_DS_list.append(DS_count)
- # for ds_i in range(DS_count):
- # c_L_D_list[-1].append(dir_)
-
- for loc_u in np.unique(comp['locations']):
- c_L_D_list.append([])
- for loc, dir, csg_weights in zip(comp['locations'],
- comp['directions'],
- comp['csg_weights']):
- if loc == loc_u:
- if include_CSG:
- csg_list = csg_weights
- else:
- csg_list = [1.0,]
- for csg_ in csg_list:
- c_DS_list.append(DS_count)
- for ds_i in range(DS_count):
- c_L_D_list[-1].append(dir)
-
- c_dims = sum([len(loc) for loc in c_L_D_list])
- dims.append(c_dims)
- L_D_list.append(c_L_D_list)
- DS_list.append(c_DS_list)
-
- rho = np.ones((sum(dims), sum(dims))) * rho_FG
-
- f_pos_id = 0
- for c_id, (c_L_D_list, c_dims, c_DS_list) in enumerate(
- zip(L_D_list, dims, DS_list)):
- c_rho = np.ones((c_dims, c_dims)) * rho_PG
-
- # dependencies btw directions
- if rho_DIR != 0:
- c_pos_id = 0
- for loc_D_list in c_L_D_list:
- l_dim = len(loc_D_list)
- c_rho[c_pos_id:c_pos_id + l_dim,
- c_pos_id:c_pos_id + l_dim] = rho_DIR
- c_pos_id = c_pos_id + l_dim
-
- # dependencies btw locations
- if rho_LOC != 0:
- flat_dirs = []
- [[flat_dirs.append(dir_i) for dir_i in dirs] for dirs in
- c_L_D_list]
- flat_dirs = np.array(flat_dirs)
- for u_dir in np.unique(flat_dirs):
- dir_ids = np.where(flat_dirs == u_dir)[0]
- for i in dir_ids:
- for j in dir_ids:
- c_rho[i, j] = rho_LOC
-
- if ((rho_CSG != 0) or (rho_target == 'ATC')):
- c_pos_id = 0
- if rho_target == 'ATC':
- rho_to_use = float(ATC_rho[c_id])
- else:
- rho_to_use = rho_CSG
- for loc_D_list in c_L_D_list:
- flat_dirs = np.array(loc_D_list)
- for u_dir in np.unique(flat_dirs):
- dir_ids = np.where(flat_dirs == u_dir)[0]
- for i in dir_ids:
- for j in dir_ids:
- c_rho[c_pos_id + i, c_pos_id + j] = rho_to_use
- c_pos_id = c_pos_id + len(loc_D_list)
-
- if rho_DS != 0:
- c_pos_id = 0
- for l_dim in c_DS_list:
- c_rho[c_pos_id:c_pos_id + l_dim,
- c_pos_id:c_pos_id + l_dim] = rho_DS
- c_pos_id = c_pos_id + l_dim
-
- rho[f_pos_id:f_pos_id + c_dims,
- f_pos_id:f_pos_id + c_dims] = c_rho
- f_pos_id = f_pos_id + c_dims
-
- np.fill_diagonal(rho, 1.0)
-
- return rho
-
- def _create_RV_quantities(self, rho_qnt):
- """
-
- Parameters
- ----------
- rho_qnt
-
- Returns
- -------
-
- """
-
- q_theta, q_sig, q_tags, q_dist = [np.array([]) for i in range(4)]
-
- # collect the parameters for each quantity dimension
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id in s_fg_keys:
- comp = self._FG_in[c_id]
-
- u_dirs = np.unique(comp['directions'])
-
- #dir_weights = comp['dir_weights']
- #theta_list = []
- #[[theta_list.append(qnt * dw)
- # for dw in dir_weights] for qnt in comp['quantities']]
-
- theta_list = comp['quantities']
- q_theta = np.append(q_theta, theta_list)
-
- dist_list = comp['distribution_kind']
- q_dist = np.append(q_dist, dist_list)
-
- cov_list = comp['cov']
- for theta, dk, cov in list(zip(theta_list, dist_list, cov_list)):
- if dk == 'normal':
- q_sig = np.append(q_sig, [cov*theta,])
- else:
- q_sig = np.append(q_sig, [cov, ])
-
- q_tags = np.append(q_tags, [f'{c_id}-QNT-{s_i}-{d_i}' for s_i, d_i
- in list(zip(comp['locations'],
- comp['directions']))])
-
- rho = self._create_correlation_matrix(rho_qnt)
-
- if not np.all(q_dist=='N/A'):
- # remove the unnecessary fields
- to_remove = np.where(q_dist=='N/A')[0]
- rho = np.delete(rho, to_remove, axis=0)
- rho = np.delete(rho, to_remove, axis=1)
-
- q_theta, q_sig, q_dist, q_tags = [
- np.delete(q_vals, to_remove) for q_vals in [
- q_theta, q_sig, q_dist, q_tags]]
-
- dims = len(q_theta)
-
- # add lower limits to ensure only positive quantities
- # zero is probably too low, and it might make sense to introduce upper
- # limits as well
- tr_lower = [0. for d in range(dims)]
- tr_upper = [None for d in range(dims)]
-
- # Create the RVs
- for q, (q_tag, theta_q, sig_q, dist_q) in enumerate(
- zip(q_tags, q_theta, q_sig, q_dist)):
- self._RV_reg.add_RV(RandomVariable(
- name=q_tag, distribution=dist_q,
- theta=[theta_q, sig_q],
- truncation_limits=[tr_lower[q], tr_upper[q]]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'QNT_set', list(self._RV_reg.RVs(q_tags).values()), rho))
-
- # q_COV = np.outer(q_sig, q_sig) * rho
- #
- # # to avoid truncations affecting other dimensions when rho_QNT is large,
- # # assign a post-truncation correlation structure
- # corr_ref = 'post'
- #
- # quantity_RV = RandomVariable(ID=100,
- # dimension_tags=q_tag,
- # distribution_kind=q_dist,
- # theta=q_theta,
- # COV=q_COV,
- # truncation_limits=[tr_lower, tr_upper],
- # corr_ref=corr_ref)
- else:
- q_tags = None
-
- return q_tags
-
- def _create_RV_fragilities(self, c_id, comp, rho_fr):
- """
-
- Parameters
- ----------
- c_id
- comp
- rho_fr
-
- Returns
- -------
-
- """
-
- # prepare the basic multivariate distribution data for one component subgroup considering all damage states
- d_theta, d_sig, d_tags, d_distr_kind = [np.array([]) for i in range(4)]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for d_id in s_dsg_keys:
- DSG = comp['DSG_set'][d_id]
- d_theta = np.append(d_theta, DSG['theta'])
- d_sig = np.append(d_sig, DSG['sig'])
- d_tags = np.append(d_tags, f"FF-{comp['ID']}-{d_id}")
- d_distr_kind = np.append(d_distr_kind, DSG['distribution_kind'])
- dims = len(d_theta)
-
- # get the total number of random variables for this fragility group
- # TODO: add the possibility of multiple locations and directions
- #rv_count = len(comp['locations']) * len(comp['directions']) * dims
- rv_count = sum([len(csg_w) for csg_w in comp['csg_weights']]) * dims
-
- # create the (empty) input arrays for the RV
- c_theta = np.zeros(rv_count)
- c_tags = np.empty(rv_count, dtype=object)
- c_sig = np.zeros(rv_count)
- c_distr_kind = np.empty(rv_count, dtype=object)
-
- pos_id = 0
- #for l_id in comp['locations']:
- # # for each location-direction pair)
- # for d_id, __ in enumerate(comp['directions']):
- # # for each component-subgroup
- # c_theta[pos_id:pos_id + dims] = d_theta
- # c_sig[pos_id:pos_id + dims] = d_sig
- # c_tags[pos_id:pos_id + dims] = [
- # t + '-LOC-{}-CSG-{}'.format(l_id, d_id) for t in d_tags]
- # c_distr_kind[pos_id:pos_id + dims] = d_distr_kind
- # pos_id += dims
-
- for l_id, d_id, csg_list in zip(comp['locations'], comp['directions'],
- comp['csg_weights']):
- # for each location-direction pair)
- for csg_id, __ in enumerate(csg_list):
- # for each component-subgroup
- c_theta[pos_id:pos_id + dims] = d_theta
- c_sig[pos_id:pos_id + dims] = d_sig
- c_tags[pos_id:pos_id + dims] = [
- t + '-LOC-{}-DIR-{}-CSG-{}'.format(l_id, d_id, csg_id)
- for t in d_tags]
- c_distr_kind[pos_id:pos_id + dims] = d_distr_kind
- pos_id += dims
-
- # create the covariance matrix
- #c_rho = self._create_correlation_matrix(rho_fr, c_target=c_id,
- # include_DSG=True,
- # include_CSG=True)
- c_rho = np.ones((rv_count, rv_count))
-
- if c_tags.size > 0:
-
- # Create the RVs
- for c, (c_tag, sig_c, theta_c, dkind_c) in enumerate(
- zip(c_tags, c_sig, c_theta, c_distr_kind)):
- self._RV_reg.add_RV(RandomVariable(
- name=c_tag, distribution=dkind_c,
- theta=[theta_c, sig_c]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- f'FF_set_{c_id}', list(self._RV_reg.RVs(c_tags).values()),
- c_rho))
-
- # c_COV = np.outer(c_sig, c_sig) * c_rho
- #
- # fragility_RV = RandomVariable(ID=300 + c_id,
- # dimension_tags=c_tags,
- # distribution_kind=c_distr_kind,
- # theta=c_theta,
- # COV=c_COV)
- else:
- c_tags = None
-
- return c_tags
-
- def _create_RV_damage_states(self):
-
- d_tags, d_theta = [[] for i in range(2)]
-
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- if DSG['DS_set_kind'] == 'mutually exclusive':
-
- s_ds_keys = sorted(DSG['DS_set'].keys())
- DS_set = DSG['DS_set']
- DS_weights = [DS_set[ds_i]['weight'] for ds_i in s_ds_keys]
-
- for loc, dir_, csg_list in zip(comp['locations'],
- comp['directions'],
- comp['csg_weights']):
-
- for csg_id, __ in enumerate(csg_list):
-
- d_tags.append(f'DSG-{comp["ID"]}-{dsg_i}-LOC-{loc}-'
- f'DIR-{dir_}-CSG-{csg_id}')
- d_theta.append(DS_weights)
-
- if len(d_tags) > 0:
-
- # Create the RVs
- for d, (d_tag, theta_d) in enumerate(zip(d_tags, d_theta)):
- self._RV_reg.add_RV(RandomVariable(
- name=d_tag, distribution='multinomial',
- theta=theta_d))
-
- else:
- d_tags = None
-
- return d_tags
-
- def _create_RV_repairs(self, rho_cost, rho_time, rho_cNt):
-
- # prepare the cost and time parts of the data separately
- ct_sig, ct_tags, ct_dkind = [np.array([]) for i in range(3)]
- for rho_target, name in zip([rho_cost, rho_time], ['cost', 'time']):
-
- f_sig, f_tag, f_dkind = [np.array([]) for i in range(3)]
-
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id, c_name in enumerate(s_fg_keys):
- comp = self._FG_in[c_name]
-
- d_sig, d_tag, d_dkind = [np.array([]) for i in range(3)]
-
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i in s_dsg_keys:
- DSG = comp['DSG_set'][dsg_i]
- s_ds_keys = sorted(DSG['DS_set'].keys())
- for ds_i in s_ds_keys:
- DS = DSG['DS_set'][ds_i]
- if ((f'repair_{name}' in DS.keys()) and
- (DS[f'repair_{name}']['distribution_kind'] is not None)):
- data = DS[f'repair_{name}']
- d_sig = np.append(d_sig, data['cov'])
- d_dkind = np.append(d_dkind,
- data['distribution_kind'])
- else:
- d_sig = np.append(d_sig, 0.0001)
- d_dkind = np.append(d_dkind, None)
-
- d_tag = np.append(
- d_tag, f'REP-{comp["ID"]}-{dsg_i}-{ds_i}-{name}')
- #comp['ID'] + '-' + str(
- # dsg_i) + '-' + str(
- # ds_i) + '-{}'.format(name))
-
- for loc, dir_ in zip(comp['locations'], comp['directions']):
- f_sig = np.append(f_sig, d_sig)
- f_dkind = np.append(f_dkind, d_dkind)
- f_tag = np.append(
- f_tag, [t + f'-LOC-{loc}-DIR-{dir_}' for t in d_tag])
-
- ct_sig = np.append(ct_sig, f_sig)
- ct_tags = np.append(ct_tags, f_tag)
- ct_dkind = np.append(ct_dkind, f_dkind)
-
- rho_c = self._create_correlation_matrix(rho_cost, c_target=-1,
- include_DSG=True,
- include_DS=True)
- rho_t = self._create_correlation_matrix(rho_time, c_target=-1,
- include_DSG=True,
- include_DS=True)
-
- dims = len(ct_tags)
- ct_rho = np.zeros((dims, dims))
-
- dims = dims // 2
- if rho_cNt == False:
- ct_rho[:dims, :dims] = rho_c
- ct_rho[dims:, dims:] = rho_t
- else:
- # In the special case of mixing perfect correlation between
- # locations and directions, taking the envelope is not the
- # appropriate solution. Instead, the LOC & DIR -> PG approach is
- # used.
- if (((rho_cost == 'LOC') and (rho_time =='DIR')) or
- ((rho_cost == 'DIR') and (rho_time == 'LOC'))):
- rho_ct = self._create_correlation_matrix('PG', c_target=-1,
- include_DSG=True,
- include_DS=True)
- else:
- # We use the envelope in every other case.
- rho_ct = np.maximum(rho_c, rho_t)
-
- ct_rho[:dims, :dims] = rho_ct
- ct_rho[dims:, dims:] = rho_ct
-
- # apply the same blocks to the off-diagonal positions
- ct_rho[:dims, dims:] = rho_ct
- ct_rho[dims:, :dims] = rho_ct
-
- # now remove the unnecessary fields
- if not np.all(ct_dkind == None):
-
- to_remove = np.where(ct_dkind == None)[0]
- ct_rho = np.delete(ct_rho, to_remove, axis=0)
- ct_rho = np.delete(ct_rho, to_remove, axis=1)
-
- ct_dkind, ct_sig, ct_tags = [np.delete(ct_vals, to_remove)
- for ct_vals in [ct_dkind, ct_sig, ct_tags]]
-
- # Create the RVs
- for ct, (ct_tag, sig_ct, dkind_ct) in enumerate(
- zip(ct_tags, ct_sig, ct_dkind)):
- self._RV_reg.add_RV(RandomVariable(
- name=ct_tag, distribution=dkind_ct,
- theta=[1.0, sig_ct],
- truncation_limits=[0., None]
- ))
-
- self._RV_reg.add_RV_set(RandomVariableSet(
- 'DV_REP_set', list(self._RV_reg.RVs(ct_tags).values()), ct_rho))
-
- # ct_COV = np.outer(ct_sig, ct_sig) * ct_rho
- #
- # repair_RV = RandomVariable(ID=401,
- # dimension_tags=ct_tags,
- # distribution_kind=ct_dkind,
- # theta=np.ones(len(ct_sig)),
- # COV=ct_COV,
- # corr_ref='post',
- # truncation_limits=[np.zeros(len(ct_sig)),
- # None])
-
- else:
- ct_tags = None
-
- return ct_tags
-
- def _create_fragility_groups(self):
-
- RVd = self._RV_dict
- DVs = self._AIM_in['decision_variables']
-
- # use the building replacement cost to calculate the absolute
- # reconstruction cost for component groups
- repl_cost = self._AIM_in['general']['replacement_cost']
-
- # create a list for the fragility groups
- FG_dict = dict()
-
- s_fg_keys = sorted(self._FG_in.keys())
- for c_id in s_fg_keys:
- log_msg('\t{}...'.format(c_id))
- comp = self._FG_in[c_id]
-
- FG_ID = len(FG_dict.keys()) + 1
-
- # create a list for the performance groups
- performance_groups = []
-
- # one group for each of the stories prescribed by the user
- PG_locations = comp['locations']
- PG_directions = comp['directions']
- PG_csg_lists = comp['csg_weights']
- PG_dists = comp['distribution_kind']
- PG_qnts = comp['quantities']
- for loc, dir_, csg_list, dist, qnt, in zip(
- PG_locations, PG_directions, PG_csg_lists, PG_dists, PG_qnts):
- PG_ID = 10000 * FG_ID + 10 * loc + dir_
-
- # get the quantity
- if dist == 'N/A':
- QNT = qnt
- else:
- QNT = self._QNT_dict[f'{c_id}-QNT-{loc}-{dir_}']
- # QNT = RandomVariableSubset(RVd['QNT'],
- # tags=[f'{c_id}-QNT-{loc}-{dir_}', ])
-
- # create the damage objects
- # consequences are calculated on a performance group level
-
- # create a list for the damage state groups and their tags
- DSG_list = []
- d_tags = []
- s_dsg_keys = sorted(comp['DSG_set'].keys())
- for dsg_i, DSG_ID in enumerate(s_dsg_keys):
- DSG = comp['DSG_set'][DSG_ID]
- d_tags.append('FF-' + c_id + '-' + DSG_ID)
-
- # create a list for the damage states
- DS_set = []
-
- s_ds_keys = sorted(DSG['DS_set'].keys())
- for ds_i, DS_ID in enumerate(s_ds_keys):
- DS = DSG['DS_set'][DS_ID]
-
- # create the consequence functions
- # note: consequences in HAZUS are conditioned on
- # damage with no added uncertainty
-
- if DVs['rec_cost'] and ('repair_cost' in DS.keys()):
- data = DS['repair_cost']
- data_scaled = deepcopy(data)
- data_scaled['medians'] = np.array(data_scaled['medians'])
- data_scaled['medians'] *= repl_cost
-
- if len(data['medians']) > 1:
- f_median = prep_bounded_multilinear_median_DV(
- **{k: data_scaled.get(k, None) for k in
- ('medians', 'quantities')})
- else:
- f_median = prep_constant_median_DV(
- data_scaled['medians'][0])
-
- if data['distribution_kind'] is not None:
-
- cf_tag = 'REP-' + c_id + '-' + DSG_ID + '-' + DS_ID + \
- '-cost' + f'-LOC-{loc}-DIR-{dir_}'
-
- CF_RV = self._DV_REP_dict[cf_tag]
- # CF_RV = RandomVariableSubset(RVd['DV_REP'],
- # tags=cf_tag)
- else:
- CF_RV = None
-
- CF_cost = ConsequenceFunction(DV_median=f_median,
- DV_distribution=CF_RV)
- else:
- CF_cost = None
-
- if DVs['rec_time'] and ('repair_time' in DS.keys()):
- data = DS['repair_time']
-
- if len(data['medians']) > 1:
- f_median = prep_bounded_multilinear_median_DV(
- **{k: data.get(k, None) for k in
- ('medians', 'quantities')})
- else:
- f_median = prep_constant_median_DV(
- data['medians'][0])
-
- if data['distribution_kind'] is not None:
-
- cf_tag = 'REP-' + c_id + '-' + DSG_ID + '-' + DS_ID + \
- '-time' + f'-LOC-{loc}-DIR-{dir_}'
-
- CF_RV = self._DV_REP_dict[cf_tag]
- # CF_RV = RandomVariableSubset(RVd['DV_REP'],
- # tags=cf_tag)
- else:
- CF_RV = None
-
- CF_time = ConsequenceFunction(DV_median=f_median,
- DV_distribution=CF_RV)
- else:
- CF_time = None
-
- # note: no red tag in HAZUS assessments
-
- if (DVs['injuries']) and ('injuries' in DS.keys()):
- CF_inj_set = []
- for inj_i, theta in enumerate(
- DS['injuries']):
- if theta > 0.:
- f_median = prep_constant_median_DV(
- theta)
- CF_inj_set.append(ConsequenceFunction(
- DV_median=f_median,
- DV_distribution=None))
- else:
- CF_inj_set.append(None)
- else:
- CF_inj_set = [None, ]
-
- DS_set.append(DamageState(ID=ds_i + 1,
- description=DS[
- 'description'],
- weight=DS['weight'],
- repair_cost_CF=CF_cost,
- reconstruction_time_CF=CF_time,
- injuries_CF_set=CF_inj_set
- ))
-
- # add the DSG to the list
- DSG_list.append(DamageStateGroup(ID=dsg_i + 1,
- DS_set=DS_set,
- DS_set_kind=DSG[
- 'DS_set_kind']
- ))
-
- # create the fragility functions
- FF_set = []
- #CSG_this = np.where(comp['directions'] == dir_)[0]
- #PG_weights = np.asarray(comp['csg_weights'])[CSG_this]
- # normalize the weights
- #PG_weights /= sum(PG_weights)
- for csg_id, __ in enumerate(csg_list):
- # assign the appropriate random variable to the fragility
- # function
- ff_tags = [t + f'-LOC-{loc}-DIR-{dir_}-CSG-{csg_id}'
- for t in d_tags]
- EDP_limit = [self._FF_dict[ff_tag] for ff_tag in ff_tags]
- # EDP_limit = RandomVariableSubset(RVd['FR-' + c_id],
- # tags=ff_tags)
- FF_set.append(FragilityFunction(EDP_limit))
-
- # create the performance group
- PG = PerformanceGroup(ID=PG_ID,
- location=loc,
- quantity=QNT,
- fragility_functions=FF_set,
- DSG_set=DSG_list,
- csg_weights=csg_list,
- direction=dir_
- )
- performance_groups.append(PG)
-
- # create the fragility group
- FG = FragilityGroup(ID=FG_ID,
- #kind=comp['kind'],
- demand_type=comp['demand_type'],
- performance_groups=performance_groups,
- directional=comp['directional'],
- correlation=comp['correlation'],
- demand_location_offset=comp['offset'],
- incomplete=comp['incomplete'],
- name=str(FG_ID) + ' - ' + comp['ID'],
- description=comp['description'],
- unit=comp['unit']
- )
-
- FG_dict.update({comp['ID']: FG})
-
- return FG_dict
-
- def _sample_event_time(self):
-
- sample_count = self._AIM_in['general']['realizations']
-
- # month - uniform distribution over [0,11]
- month = np.random.randint(0, 12, size=sample_count)
-
- # weekday - binomial with p=5/7
- weekday = np.random.binomial(1, 5. / 7., size=sample_count)
-
- # hour - uniform distribution over [0,23]
- hour = np.random.randint(0, 24, size=sample_count)
-
- data = pd.DataFrame(data={'month' : month,
- 'weekday?': weekday,
- 'hour' : hour},
- dtype=int)
-
- return data
-
- def _get_population(self):
- """
- Use the population characteristics to generate random population samples.
-
- Returns
- -------
-
- """
- POPin = self._POP_in
- TIME = self._TIME
-
- POP = pd.DataFrame(
- np.ones((len(TIME.index), len(POPin['peak']))) * POPin['peak'],
- columns=['LOC' + str(loc + 1)
- for loc in range(len(POPin['peak']))])
-
- # if there is a temporal population model available...
- if 'weekday' in POPin.keys():
- weekdays = TIME[TIME['weekday?'] == 1].index
- weekends = TIME[~TIME.index.isin(weekdays)].index
-
- for col in POP.columns.values:
- POP.loc[weekdays, col] = (
- POP.loc[weekdays, col] *
- np.array(POPin['weekday']['daily'])[
- TIME.loc[weekdays, 'hour'].values.astype(int)] *
- np.array(POPin['weekday']['monthly'])[
- TIME.loc[weekdays, 'month'].values.astype(int)])
-
- POP.loc[weekends, col] = (
- POP.loc[weekends, col] *
- np.array(POPin['weekend']['daily'])[
- TIME.loc[weekends, 'hour'].values.astype(int)] *
- np.array(POPin['weekend']['monthly'])[
- TIME.loc[weekends, 'month'].values.astype(int)])
-
- return POP
-
- def _calc_damage(self):
-
- ncID = self._ID_dict['non-collapse']
- NC_samples = len(ncID)
-
- FG_dmg_list = []
-
- s_fg_keys = sorted(self._FG_dict.keys())
- for fg_id in s_fg_keys:
- log_msg('\t\t{}...'.format(fg_id))
- FG = self._FG_dict[fg_id]
-
- PG_set = FG._performance_groups
-
- DS_list = []
- for DSG in PG_set[0]._DSG_set:
- for DS in DSG._DS_set:
- DS_list.append(str(DSG._ID) + '_' + str(DS._ID))
- d_count = len(DS_list)
-
- MI = pd.MultiIndex.from_product([[FG._ID, ],
- [pg._ID for pg in PG_set],
- DS_list],
- names=['FG', 'PG', 'DSG_DS'])
-
- FG_damages = pd.DataFrame(np.zeros((NC_samples, len(MI))),
- columns=MI,
- index=ncID)
-
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
- if isinstance(PG._quantity, RandomVariable):
- PG_qnt = PG._quantity.samples_DF.loc[ncID]
- else:
- PG_qnt = pd.Series(np.ones(NC_samples) * PG._quantity,
- index=ncID)
-
- # get the corresponding demands
- if not FG._directional:
- demand_ID_list = []
-
- for demand_ID in self._EDP_dict.keys():
- if demand_ID[4:7] == FG._demand_type:
- demand_data = demand_ID.split('-')
- if int(demand_data[3]) == PG._location + FG._demand_location_offset:
- demand_ID_list.append(demand_ID)
-
- EDP_samples = self._EDP_dict[demand_ID_list[0]].samples_DF.loc[ncID]
- if len(demand_ID_list)>1:
- for demand_ID in demand_ID_list[1:]:
- new_samples = self._EDP_dict[demand_ID].samples_DF.loc[ncID]
- EDP_samples = np.maximum(new_samples.values,
- EDP_samples.values)
-
- else:
- demand_ID = ('EDP-' + FG._demand_type +
- '-LOC-' + str(PG._location + FG._demand_location_offset) +
- '-DIR-' + str(PG._direction))
-
- if demand_ID in self._EDP_dict.keys():
- EDP_samples = self._EDP_dict[demand_ID].samples_DF.loc[ncID]
- else:
- # If the required demand is not available, then we are most
- # likely analyzing a 3D structure using results from a 2D
- # simulation. The best thing we can do in that particular
- # case is to use the EDP from the 1 direction for all other
- # directions.
- demand_ID = ('EDP-' + FG._demand_type +
- '-LOC-' + str(PG._location + FG._demand_location_offset) + '-DIR-1')
- EDP_samples = self._EDP_dict[demand_ID].samples_DF.loc[ncID]
-
- csg_w_list = np.array(PG._csg_weights)
- for csg_i, csg_w in enumerate(csg_w_list):
- DSG_df = PG._FF_set[csg_i].DSG_given_EDP(EDP_samples)
-
- for DSG in PG._DSG_set:
- in_this_DSG = DSG_df[DSG_df.values == DSG._ID].index
- if DSG._DS_set_kind == 'single':
- DS = DSG._DS_set[0]
- DS_tag = str(DSG._ID) + '_' + str(DS._ID)
- FG_damages.loc[in_this_DSG,
- (FG._ID, PG_ID, DS_tag)] += csg_w
- elif DSG._DS_set_kind == 'mutually exclusive':
- mut_ex_id = f'DSG-{fg_id}-DSG-{DSG._ID}-' \
- f'LOC-{PG._location}-' \
- f'DIR-{PG._direction}-CSG-{csg_i}'
- DS_df = self._DSG_dict[mut_ex_id].samples_DF.loc[
- in_this_DSG]
-
- # DS_weights = [DS._weight for DS in DSG._DS_set]
- # DS_RV = RandomVariable(
- # ID=-1, dimension_tags=['me_DS', ],
- # distribution_kind='multinomial',
- # p_set=DS_weights)
- # DS_df = DS_RV.sample_distribution(
- # len(in_this_DSG)) + 1
-
- for DS in DSG._DS_set:
- DS_tag = str(DSG._ID) + '_' + str(DS._ID)
- in_this_DS = DS_df[DS_df.values == DS._ID-1].index
- FG_damages.loc[in_this_DS,
- (FG._ID, PG_ID, DS_tag)] += csg_w
- elif DSG._DS_set_kind == 'simultaneous':
- DS_weights = [DS._weight for DS in DSG._DS_set]
- DS_df = np.random.uniform(
- size=(len(in_this_DSG), len(DS_weights)))
- which_DS = DS_df < DS_weights
- any_DS = np.any(which_DS, axis=1)
- no_DS_ids = np.where(any_DS == False)[0]
-
- while len(no_DS_ids) > 0:
- DS_df_add = np.random.uniform(
- size=(len(no_DS_ids), len(DS_weights)))
- which_DS_add = DS_df_add < DS_weights
- which_DS[no_DS_ids] = which_DS_add
-
- any_DS = np.any(which_DS_add, axis=1)
- no_DS_ids = no_DS_ids[
- np.where(any_DS == False)[0]]
-
- for ds_i, DS in enumerate(DSG._DS_set):
- DS_tag = str(DSG._ID) + '_' + str(DS._ID)
- in_this_DS = which_DS[:, ds_i]
- FG_damages.loc[in_this_DSG[in_this_DS], (
- FG._ID, PG_ID, DS_tag)] += csg_w
-
- else:
- raise ValueError(
- "Unknown damage state type: {}".format(
- DSG._DS_set_kind)
- )
-
- FG_damages.iloc[:, pg_i * d_count:(pg_i + 1) * d_count] = \
- FG_damages.iloc[:, pg_i * d_count:(pg_i + 1) * d_count].values * PG_qnt.values.reshape(-1, *[1])
-
-
- FG_dmg_list.append(FG_damages)
-
- DMG = pd.concat(FG_dmg_list, axis=1)
-
- DMG.index = ncID
-
- # sort the columns to enable index slicing later
- DMG = DMG.sort_index(axis=1, ascending=True)
-
- return DMG
-
- def _calc_repair_cost_and_time(self):
-
- idx = pd.IndexSlice
- DVs = self._AIM_in['decision_variables']
-
- DMG_by_FG_and_DS = self._DMG.groupby(level=[0, 2], axis=1).sum()
-
- repID = self._ID_dict['repairable']
- DV_COST = self._DMG.loc[repID, :].copy()
- DV_TIME = DV_COST.copy()
-
- s_fg_keys = sorted(self._FG_dict.keys())
- for fg_id in s_fg_keys:
- log_msg('\t\t{}...'.format(fg_id))
- FG = self._FG_dict[fg_id]
-
- PG_set = FG._performance_groups
-
- DS_list = self._DMG.loc[:, idx[FG._ID, PG_set[0]._ID, :]].columns
- DS_list = DS_list.levels[2][DS_list.codes[2]].values
-
- for d_i, d_tag in enumerate(DS_list):
-
- dsg_i = int(d_tag[0]) - 1
- ds_i = int(d_tag[-1]) - 1
-
- TOT_qnt = DMG_by_FG_and_DS.loc[repID, (FG._ID, d_tag)]
-
- # check what can we expect later
- # pull the DS from the first PG
- DS_test = PG_set[0]._DSG_set[dsg_i]._DS_set[ds_i]
-
- if DVs['rec_cost']:
- COST_samples = DS_test.unit_repair_cost(quantity=TOT_qnt)
-
- if COST_samples is None:
- # there are no costs assigned to this DS
- DV_COST.drop(DV_COST.loc[:, idx[FG._ID, :, d_tag]].columns, axis=1, inplace=True)
-
- elif isinstance(COST_samples, pd.Series):
- # the assigned costs are random numbers
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
- DS = PG._DSG_set[dsg_i]._DS_set[ds_i]
-
- COST_samples = DS.unit_repair_cost(quantity=TOT_qnt).values
-
- DV_COST.loc[:,(FG._ID, PG_ID, d_tag)] = DV_COST.loc[:, (FG._ID, PG_ID, d_tag)].values * COST_samples
-
- else:
- # the assigned costs are identical for all realizations
- DV_COST.loc[:, idx[FG._ID, :, d_tag]] *= COST_samples
-
- if DVs['rec_time']:
- TIME_samples = DS_test.unit_reconstruction_time(quantity=TOT_qnt)
-
- if TIME_samples is None:
- # there are no repair times assigned to this DS
- DV_TIME.drop(DV_TIME.loc[:, idx[FG._ID, :, d_tag]].columns, axis=1, inplace=True)
-
- elif isinstance(TIME_samples, pd.Series):
- # the assigned repair times are random numbers
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
- DS = PG._DSG_set[dsg_i]._DS_set[ds_i]
-
- TIME_samples = DS.unit_reconstruction_time(quantity=TOT_qnt).values
-
- DV_TIME.loc[:, (FG._ID, PG_ID, d_tag)] = DV_TIME.loc[:, (FG._ID, PG_ID, d_tag)].values * TIME_samples
-
- else:
- # the assigned repair times are identical for all realizations
- DV_TIME.loc[:, idx[FG._ID, :, d_tag]] *= TIME_samples
-
- # sort the columns to enable index slicing later
- if DVs['rec_cost']:
- DV_COST = DV_COST.sort_index(axis=1, ascending=True)
- else:
- DV_COST = None
- if DVs['rec_time']:
- DV_TIME = DV_TIME.sort_index(axis=1, ascending=True)
- else:
- DV_TIME = None
-
- return DV_COST, DV_TIME
-
- def _calc_non_collapse_injuries(self):
-
- idx = pd.IndexSlice
-
- ncID = self._ID_dict['non-collapse']
- P_affected = self._POP.loc[ncID]
-
- NC_samples = len(ncID)
- DV_INJ_dict = dict(
- [(i, self._DMG.loc[ncID, :].copy()) for i in range(self._inj_lvls)]
- )
-
- s_fg_keys = sorted(self._FG_dict.keys())
- for fg_id in s_fg_keys:
- log_msg('\t\t{}...'.format(fg_id))
- FG = self._FG_dict[fg_id]
-
- PG_set = FG._performance_groups
-
- DS_list = self._DMG.loc[:, idx[FG._ID, PG_set[0]._ID, :]].columns
- DS_list = DS_list.levels[2][DS_list.codes[2]].values
-
- for i in range(self._inj_lvls):
-
- for d_i, d_tag in enumerate(DS_list):
- dsg_i = int(d_tag[0]) - 1
- ds_i = int(d_tag[-1]) - 1
-
- # check what can we expect later
- # pull the DS from the first PG
- DS_test = PG_set[0]._DSG_set[dsg_i]._DS_set[ds_i]
- INJ_samples = DS_test.unit_injuries(severity_level=i,
- sample_size=NC_samples)
-
- if INJ_samples is None:
- # there are no injuries assigned to this DS
- DV_INJ_dict[i].drop(DV_INJ_dict[i].loc[:, idx[FG._ID, :, d_tag]].columns, axis=1, inplace=True)
- continue
-
- elif isinstance(INJ_samples, pd.Series):
- # the assigned injuries are random numbers
- rnd_inj = True
-
- else:
- # the assigned injuries are identical for all realizations
- rnd_inj = False
-
- for pg_i, PG in enumerate(PG_set):
-
- PG_ID = PG._ID
- DS = PG._DSG_set[dsg_i]._DS_set[ds_i]
-
- # get injury samples if needed
- if rnd_inj:
- INJ_samples = DS.unit_injuries(
- severity_level=i, sample_size=NC_samples).values
-
- P_aff_i = P_affected.loc[:,'LOC{}'.format(PG._location)].values * INJ_samples
- DV_INJ_dict[i].loc[:, (FG._ID, PG_ID, d_tag)] = DV_INJ_dict[i].loc[:, (FG._ID, PG_ID, d_tag)].values * P_aff_i
-
- # remove the useless columns from DV_INJ
- for i in range(self._inj_lvls):
- DV_INJ = DV_INJ_dict[i]
- DV_INJ_dict[i] = DV_INJ.loc[:, (DV_INJ != 0.0).any(axis=0)]
-
- # sort the columns to enable index slicing later
- for i in range(self._inj_lvls):
- DV_INJ_dict[i] = DV_INJ_dict[i].sort_index(axis=1, ascending=True)
-
- return DV_INJ_dict
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-This module has classes and methods to manage databases used by pelicun.
-
-.. rubric:: Contents
-
-.. autosummary::
-
- convert_jsons_to_table
- save_to_standard_HDF
- convert_json_files_to_HDF
- convert_Series_to_dict
-
- convert_P58_data_to_json
- create_HAZUS_EQ_json_files
- create_HAZUS_EQ_story_json_files
- create_HAZUS_EQ_PGA_json_files
- create_HAZUS_HU_json_files
-
-"""
-
-from .base import *
-from pathlib import Path
-import json
-import xml.etree.ElementTree as ET
-import shutil
-
-
-[docs]def dict_generator(indict, pre=None):
- """
- Lists all branches of a tree defined by a dictionary.
-
- The dictionary can have nested dictionaries and lists. When encountering a
- list, its elements are returned as separate branches with each element's id
- created as a combination of the parent key and #i where i stands for the
- element number in the list.
-
- This method can process a json file and break it up into independent
- branches.
-
- """
- pre = pre[:] if pre else []
- if isinstance(indict, dict):
- for key, value in indict.items():
- if isinstance(value, dict):
- for d in dict_generator(value, pre + [key]):
- yield d
- elif isinstance(value, list) or isinstance(value, tuple):
- for v_id, v in enumerate(value):
- for d in dict_generator(v, pre + [key + f'#{v_id}']):
- yield d
- else:
- yield pre + [key, value]
- else:
- yield pre + [indict]
-
-
-[docs]def get_val_from_dict(indict, col):
- """
- Gets the value from a branch of a dictionary.
-
- The dictionary can have nested dictionaries and lists. When walking through
- lists, #i in the branch data identifies the ith element of the list.
-
- This method can be used to travel branches of a dictionary previously
- defined by the dict_generator method above.
-
- """
-
- val = indict
-
- for col_i in col:
- if col_i != ' ':
- if '#' in col_i:
- col_name, col_id = col_i.split('#')
- col_id = int(col_id)
- if (col_name in val.keys()) and (col_id < len(val[col_name])):
- val = val[col_name][int(col_id)]
- else:
- return None
-
- elif col_i in val.keys():
- val = val[col_i]
- else:
- return None
-
- return val
-
-def convert_jsons_to_table(json_id_list, json_list, json_template):
- # Define the header for the data table based on the template structure
- header = np.array(
- [[col[:-1], len(col[:-1])] for col in dict_generator(json_template)])
- lvls = max(np.transpose(header)[1])
- header = [col + (lvls - size) * [' ', ] for col, size in header]
-
- # Use the header to initialize the DataFrame that will hold the data
- MI = pd.MultiIndex.from_tuples(header)
-
- json_DF = pd.DataFrame(columns=MI, index=json_id_list)
- json_DF.index.name = 'ID'
-
- # Load the data into the DF
- for json_id, json_data in zip(json_id_list, json_list):
-
- for col in json_DF.columns:
-
- val = get_val_from_dict(json_data, col)
-
- if val is not None:
- json_DF.at[json_id, col] = val
-
- # Remove empty rows and columns
- json_DF = json_DF.dropna(axis=0, how='all')
- json_DF = json_DF.dropna(axis=1, how='all')
-
- # Set the dtypes for the columns based on the template
- for col in json_DF.columns:
- dtype = get_val_from_dict(json_template, col)
-
- if dtype != 'string':
- try:
- json_DF[col] = json_DF[col].astype(dtype)
- except:
- print(col, dtype)
- else:
- json_DF[col] = json_DF[col].apply(str)
-
- return json_DF
-
-
-[docs]def save_to_standard_HDF(df, name, target_path, mode='w'):
- """
- Saves a DataFrame in a standard HDF format using h5py.
-
- """
- import h5py # import here to avoid an issue on Stampede2
-
- df = df.T
-
- hf = h5py.File(target_path, mode)
-
- #try:
- if True:
- # save each row (i.e., column of FG_df) as a separate dataset in the file
- for row_id, row in df.iterrows():
- row = row.convert_dtypes()
-
- # create a tree that identifies the column in the hierarchy
- row_name = name
- for label in row_id:
- if label != " ":
- row_name += f'/{label}'
-
- # perform the appropriate type conversion before saving
- if row.dtype == np.float64:
- values = row.values.astype(float)
-
- hf.create_dataset(row_name, data=values)
-
- elif row.dtype == pd.StringDtype():
- # Strings are saved as ASCII strings so that the files can be
- # opened with any tool on any platform. Non-ASCII characters
- # are replaced by a backslash-escaped UTF8 identifier.
- values = row.values.astype(str)
- values = np.char.encode(values, encoding='ASCII',
- errors='backslashreplace')
-
- hf.create_dataset(row_name, data=values)
-
- elif row.dtype in [pd.BooleanDtype(), pd.Int64Dtype()]:
- row.fillna(-1, inplace=True)
- values = row.values.astype(int)
-
- hf.create_dataset(row_name, data=values)
-
- else:
- print("unknown dtype: ", row.dtype)
-
- # finally, save the index
- values = df.columns.values.astype(str)
- values = np.char.encode(values, encoding='ASCII',
- errors='backslashreplace')
- hf.create_dataset(f'{name}/index', data=values)
-
- #except:
- # show_warning("Error while trying to save standard HDF5 file.")
-
- hf.close()
-
-
-[docs]def convert_json_files_to_HDF(data_source_dir, DL_dir, db_name):
- """
- Converts data from json files to a single HDF5 file
-
- """
-
- # Start with the fragility and consequence data - we'll call it data
-
- DL_dir = Path(DL_dir).resolve()
- data_source_dir = Path(data_source_dir).resolve()
-
- # get a list of json files to convert
- FG_ID_list = [filename[:-5] for filename in os.listdir(DL_dir / 'DL json')]
-
- # We will use a template.json to define the schema for the jsons and the
- # header for the data table.
- with open(data_source_dir / 'DL_template.json', 'r') as f:
- FG_template = json.load(f)
-
- FG_list = []
- DL_json_dir = DL_dir / 'DL json'
- for FG_i in FG_ID_list:
- with open(DL_json_dir / f'{FG_i}.json', 'r') as f:
- FG_list.append(json.load(f))
-
- FG_df = convert_jsons_to_table(FG_ID_list, FG_list, FG_template)
-
- # start with saving the data in standard HDF5 format
- save_to_standard_HDF(FG_df, name='data_standard',
- target_path=DL_dir / f'{db_name}.hdf')
-
- # then also save it using PyTables for quick access and slicing
- FG_df.to_hdf(DL_dir / f'{db_name}.hdf', 'data', mode='a', format='table',
- complevel=1, complib='blosc:snappy')
-
- # Now add the population distribution data - we'll call it pop
-
- # Only do this if there is population data
- try:
- with open(DL_dir / 'population.json', 'r') as f:
- pop = json.load(f)
-
- pop_ID_list = list(pop.keys())
-
- pop_data = [pop[key] for key in pop.keys()]
-
- with open(data_source_dir / 'pop_template.json', 'r') as f:
- pop_template = json.load(f)
-
- pop_df = convert_jsons_to_table(pop_ID_list, pop_data, pop_template)
-
- save_to_standard_HDF(pop_df, name='pop_standard',
- target_path=DL_dir / f'{db_name}.hdf', mode='a')
-
- pop_df.to_hdf(DL_dir / f'{db_name}.hdf', 'pop', mode='a', format='table',
- complevel=1, complib='blosc:snappy')
-
- except:
- pass
-
-[docs]def convert_Series_to_dict(comp_Series):
- """
- Converts data from a table to a json file
-
- """
-
- comp_Series = comp_Series.dropna(how='all')
-
- comp_dict = {}
-
- for branch in comp_Series.index:
-
- nested_dict = comp_dict
- parent_dict = None
- parent_val = None
- parent_id = None
-
- for val in branch:
- if val != ' ':
- if '#' in val:
- val, list_id = val.split('#')
- list_id = int(list_id)
- else:
- list_id = None
-
- if val not in nested_dict.keys():
- if list_id is not None:
- nested_dict.update({val: []})
- else:
- nested_dict.update({val: {}})
-
- if list_id is not None:
- if list_id > len(nested_dict[val]) - 1:
- nested_dict[val].append({})
- parent_dict = nested_dict
- nested_dict = nested_dict[val][list_id]
-
- parent_id = list_id
-
- else:
- parent_dict = nested_dict
- nested_dict = nested_dict[val]
-
- parent_val = val
-
- if isinstance(parent_dict[parent_val], dict):
- parent_dict[parent_val] = comp_Series[branch]
- else:
- parent_dict[parent_val][parent_id] = comp_Series[branch]
-
- return comp_dict
-
-[docs]def convert_P58_data_to_json(data_dir, target_dir):
- """
- Create JSON data files from publicly available P58 data.
-
- FEMA P58 damage and loss information is publicly available in an Excel
- spreadsheet and also in a series of XML files as part of the PACT tool.
- Those files are copied to the resources folder in the pelicun repo. Here
- we collect the available information on Fragility Groups from those files
- and save the damage and loss data in the common SimCenter JSON format.
-
- A large part of the Fragility Groups in FEMA P58 do not have complete
- damage and loss information available. These FGs are clearly marked with
- an incomplete flag in the JSON file and the 'Undefined' value highlights
- the missing pieces of information.
-
- Parameters
- ----------
- data_dir: string
- Path to the folder with the FEMA P58 Excel file and a 'DL xml'
- subfolder in it that contains the XML files.
- target_dir: string
- Path to the folder where the JSON files shall be saved.
-
- """
-
- data_dir = Path(data_dir).resolve()
- target_dir = Path(target_dir).resolve()
- DL_dir = None
-
- convert_unit = {
- 'Unit less': 'ea',
- 'Radians' : 'rad',
- 'g' : 'g',
- 'meter/sec': 'mps'
-
- }
-
- convert_DSG_type = {
- 'MutEx': 'MutuallyExclusive',
- 'Simul': 'Simultaneous'
- }
-
- def decode_DS_Hierarchy(DSH):
-
- if 'Seq' == DSH[:3]:
- DSH = DSH[4:-1]
-
- DS_setup = []
-
- while len(DSH) > 0:
- if DSH[:2] == 'DS':
- DS_setup.append(DSH[:3])
- DSH = DSH[4:]
- elif DSH[:5] in ['MutEx', 'Simul']:
- closing_pos = DSH.find(')')
- subDSH = DSH[:closing_pos + 1]
- DSH = DSH[closing_pos + 2:]
-
- DS_setup.append([subDSH[:5]] + subDSH[6:-1].split(','))
-
- return DS_setup
-
- def parse_DS_xml(DS_xml):
- CFG = DS_xml.find('ConsequenceGroup')
- CFG_C = CFG.find('CostConsequence')
- CFG_T = CFG.find('TimeConsequence')
-
- repair_cost = dict(
- Amount=[float(CFG_C.find('MaxAmount').text),
- float(CFG_C.find('MinAmount').text)],
- Quantity=[float(CFG_C.find('LowerQuantity').text),
- float(CFG_C.find('UpperQuantity').text)],
- CurveType=CFG_C.find('CurveType').text,
- Beta=float(CFG_C.find('Uncertainty').text),
- Bounds=[0., 'None']
- )
- if repair_cost['Amount'] == [0.0, 0.0]:
- repair_cost['Amount'] = 'Undefined'
-
- repair_time = dict(
- Amount=[float(CFG_T.find('MaxAmount').text),
- float(CFG_T.find('MinAmount').text)],
- Quantity=[float(CFG_T.find('LowerQuantity').text),
- float(CFG_T.find('UpperQuantity').text)],
- CurveType=CFG_T.find('CurveType').text,
- Beta=float(CFG_T.find('Uncertainty').text),
- Bounds=[0., 'None']
- )
- if repair_time['Amount'] == [0.0, 0.0]:
- repair_time['Amount'] = 'Undefined'
-
- return repair_cost, repair_time
-
- def is_float(s):
- try:
- if type(s) == str and s[-1] == '%':
- s_f = float(s[:-1]) / 100.
- else:
- s_f = float(s)
- if np.isnan(s_f):
- return False
- else:
- return True
- except ValueError:
- return False
-
- src_df = pd.read_excel(data_dir / 'PACT_fragility_data.xlsx')
- ID_list = src_df['NISTIR Classification']
-
- XML_list = [f for f in os.listdir(data_dir / 'DL xml') if f.endswith('.xml')]
-
- incomplete_count = 0
-
- for filename in XML_list:
-
- comp_ID = filename[:-4]
-
- #try:
- if True:
- tree = ET.parse((data_dir / 'DL xml') / f'{comp_ID}.xml')
- root = tree.getroot()
-
- # correct for the error in the numbering of RC beams
- if (comp_ID[:5] == 'B1051') and (
- comp_ID[-1] not in str([1, 2, 3, 4])):
- comp_ID = 'B1042' + comp_ID[5:]
-
- row = src_df.loc[np.where(ID_list == comp_ID)[0][0], :]
-
- json_output = {}
- incomplete = False
-
- json_output.update({'Name': row['Component Name']})
-
- QU = row['Fragility Unit of Measure']
- QU = QU.split(' ')
- if is_float(QU[1]):
- if QU[0] in ['TN', 'AP', 'CF', 'KV']:
- QU[0] = 'ea'
- QU[1] = 1
- json_output.update({'QuantityUnit': [int(QU[1]), QU[0]]})
- else:
- json_output.update({'QuantityUnit': [0., 'Undefined']})
- incomplete = True
-
- json_output.update({'Directional': row['Directional?'] in ['YES']})
- json_output.update({'Correlated': row['Correlated?'] in ['YES']})
-
- json_output.update({
- 'EDP': {
- 'Type' : row['Demand Parameter (value):'],
- 'Unit' : [1,
- convert_unit[row['Demand Parameter (unit):']]],
- 'Offset': int(
- row['Demand Location (use floor above? Yes/No)'] in [
- 'Yes'])
- }
- })
-
- json_output.update({
- 'GeneralInformation': {
- 'ID' : row['NISTIR Classification'],
- 'Description': row['Component Description'],
- 'Author' : row['Author'],
- 'Official' : root.find('Official').text in ['True',
- 'true'],
- 'DateCreated': root.find('DateCreated').text,
- 'Approved' : root.find('Approved').text in ['True',
- 'true'],
- 'Incomplete' : root.find('Incomplete').text in ['True',
- 'true'],
- 'Notes' : row['Comments / Notes']
- }
- })
- for key in json_output['GeneralInformation'].keys():
- if json_output['GeneralInformation'][key] is np.nan:
- json_output['GeneralInformation'][key] = 'Undefined'
-
- json_output.update({
- 'Ratings': {
- 'DataQuality' : row['Data Quality'],
- 'DataRelevance': row['Data Relevance'],
- 'Documentation': row['Documentation Quality'],
- 'Rationality' : row['Rationality'],
- }
- })
- for key in json_output['Ratings'].keys():
- if json_output['Ratings'][key] is np.nan:
- json_output['Ratings'][key] = 'Undefined'
-
- DSH = decode_DS_Hierarchy(row['DS Hierarchy'])
-
- json_output.update({'DSGroups': []})
-
- for DSG in DSH:
- if DSG[0] in ['MutEx', 'Simul']:
- mu = row['DS {}, Median Demand'.format(DSG[1][-1])]
- beta = row[
- 'DS {}, Total Dispersion (Beta)'.format(DSG[1][-1])]
- if is_float(mu) and is_float(beta):
- json_output['DSGroups'].append({
- 'MedianEDP' : float(mu),
- 'Beta' : float(beta),
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : convert_DSG_type[DSG[0]],
- 'DamageStates': DSG[1:]
- })
- else:
- json_output['DSGroups'].append({
- 'MedianEDP' : float(mu) if is_float(
- mu) else 'Undefined',
- 'Beta' : float(beta) if is_float(
- beta) else 'Undefined',
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : convert_DSG_type[DSG[0]],
- 'DamageStates': DSG[1:]
- })
- incomplete = True
- else:
- mu = row['DS {}, Median Demand'.format(DSG[-1])]
- beta = row['DS {}, Total Dispersion (Beta)'.format(DSG[-1])]
- if is_float(mu) and is_float(beta):
- json_output['DSGroups'].append({
- 'MedianEDP' : float(mu),
- 'Beta' : float(beta),
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [DSG],
- })
- else:
- json_output['DSGroups'].append({
- 'MedianEDP' : float(mu) if is_float(
- mu) else 'Undefined',
- 'Beta' : float(beta) if is_float(
- beta) else 'Undefined',
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [DSG],
- })
- incomplete = True
-
- need_INJ = False
- need_RT = False
- for DSG in json_output['DSGroups']:
- DS_list = DSG['DamageStates']
- DSG['DamageStates'] = []
- for DS in DS_list:
-
- # avoid having NaN as repair measures
- repair_measures = row['DS {}, Repair Description'.format(DS[-1])]
- if not isinstance(repair_measures, str):
- repair_measures = ""
-
- DSG['DamageStates'].append({
- 'Weight' :
- float(row['DS {}, Probability'.format(DS[-1])]),
- 'LongLeadTime' :
- int(row['DS {}, Long Lead Time'.format(DS[-1])] in [
- 'YES']),
- 'Consequences' : {},
- 'Description' :
- row['DS {}, Description'.format(DS[-1])],
- 'RepairMeasures': repair_measures
- })
-
- IMG = row['DS{}, Illustrations'.format(DS[-1])]
- if IMG not in ['none', np.nan]:
- DSG['DamageStates'][-1].update({'DamageImageName': IMG})
-
- AA = row['DS {} - Casualty Affected Area'.format(DS[-1])]
- if (isinstance(AA, str) and (is_float(AA.split(' ')[0]))):
- AA = AA.split(' ')
- DSG['DamageStates'][-1].update(
- {'AffectedArea': [int(AA[0]), AA[1]]})
- need_INJ = True
- else:
- DSG['DamageStates'][-1].update(
- {'AffectedArea': [0, 'SF']})
-
- DSG['DamageStates'][-1]['Consequences'].update(
- {'Injuries': [{}, {}]})
-
- INJ0 = DSG[
- 'DamageStates'][-1]['Consequences']['Injuries'][0]
- INJ_mu = row[
- 'DS {} Serious Injury Rate - Median'.format(DS[-1])]
- INJ_beta = row[
- 'DS {} Serious Injury Rate - Dispersion'.format(DS[-1])]
- if is_float(INJ_mu) and is_float(INJ_beta):
- INJ0.update({
- 'Amount' : float(INJ_mu),
- 'Beta' : float(INJ_beta),
- 'CurveType': 'Normal',
- 'Bounds' : [0., 1.]
- })
-
- if INJ_mu != 0.0:
- need_INJ = True
- if DSG['DamageStates'][-1]['AffectedArea'][0] == 0:
- incomplete = True
- else:
- INJ0.update({'Amount' :
- float(INJ_mu) if is_float(INJ_mu)
- else 'Undefined',
- 'Beta' :
- float(INJ_beta) if is_float(INJ_beta)
- else 'Undefined',
- 'CurveType': 'Normal'})
- if ((INJ0['Amount'] == 'Undefined') or
- (INJ0['Beta'] == 'Undefined')):
- incomplete = True
-
- INJ1 = DSG[
- 'DamageStates'][-1]['Consequences']['Injuries'][1]
- INJ_mu = row['DS {} Loss of Life Rate - Median'.format(DS[-1])]
- INJ_beta = row['DS {} Loss of Life Rate - Dispersion'.format(DS[-1])]
- if is_float(INJ_mu) and is_float(INJ_beta):
- INJ1.update({
- 'Amount' : float(INJ_mu),
- 'Beta' : float(INJ_beta),
- 'CurveType': 'Normal',
- 'Bounds' : [0., 1.]
- })
- if INJ_mu != 0.0:
- need_INJ = True
- if DSG['DamageStates'][-1]['AffectedArea'][0] == 0:
- incomplete = True
- else:
- INJ1.update({'Amount' :
- float(INJ_mu) if is_float(INJ_mu)
- else 'Undefined',
- 'Beta' :
- float(INJ_beta) if is_float(INJ_beta)
- else 'Undefined',
- 'CurveType': 'Normal',
- 'Bounds': [0., 1.]})
- if ((INJ1['Amount'] == 'Undefined') or
- (INJ1['Beta'] == 'Undefined')):
- incomplete = True
-
- DSG['DamageStates'][-1]['Consequences'].update({'RedTag': {}})
- RT = DSG['DamageStates'][-1]['Consequences']['RedTag']
-
- RT_mu = row['DS {}, Unsafe Placard Damage Median'.format(DS[-1])]
- RT_beta = row['DS {}, Unsafe Placard Damage Dispersion'.format(DS[-1])]
- if is_float(RT_mu) and is_float(RT_beta):
- RT.update({
- 'Amount' : float(RT_mu),
- 'Beta' : float(RT_beta),
- 'CurveType': 'Normal',
- 'Bounds' : [0., 1.]
- })
- if RT['Amount'] != 0.0:
- need_RT = True
- else:
- RT.update({'Amount' :
- float(RT_mu[:-1]) if is_float(RT_mu)
- else 'Undefined',
- 'Beta' :
- float(RT_beta[:-1]) if is_float(RT_beta)
- else 'Undefined',
- 'CurveType': 'Normal',
- 'Bounds': [0., 1.]})
- if ((RT['Amount'] == 'Undefined') or
- (RT['Beta'] == 'Undefined')):
- incomplete = True
-
- # remove the unused fields
- if not need_INJ:
- for DSG in json_output['DSGroups']:
- for DS in DSG['DamageStates']:
- del DS['AffectedArea']
- del DS['Consequences']['Injuries']
-
- if not need_RT:
- for DSG in json_output['DSGroups']:
- for DS in DSG['DamageStates']:
- del DS['Consequences']['RedTag']
-
- # collect the repair cost and time consequences from the XML file
- DSG_list = root.find('DamageStates').findall('DamageState')
- for DSG_i, DSG_xml in enumerate(DSG_list):
-
- if DSG_xml.find('DamageStates') is not None:
- DS_list = (DSG_xml.find('DamageStates')).findall('DamageState')
- for DS_i, DS_xml in enumerate(DS_list):
- r_cost, r_time = parse_DS_xml(DS_xml)
- CONSEQ = json_output['DSGroups'][DSG_i][
- 'DamageStates'][DS_i]['Consequences']
- CONSEQ.update({
- 'ReconstructionCost': r_cost,
- 'ReconstructionTime': r_time
- })
- if ((r_cost['Amount'] == 'Undefined') or
- (r_time['Amount'] == 'Undefined')):
- incomplete = True
-
- else:
- r_cost, r_time = parse_DS_xml(DSG_xml)
- CONSEQ = json_output['DSGroups'][DSG_i][
- 'DamageStates'][0]['Consequences']
- CONSEQ.update({
- 'ReconstructionCost': r_cost,
- 'ReconstructionTime': r_time
- })
- if ((r_cost['Amount'] == 'Undefined') or
- (r_time['Amount'] == 'Undefined')):
- incomplete = True
-
- if incomplete:
- json_output['GeneralInformation']['Incomplete'] = True
- incomplete_count += 1
-
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{comp_ID}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- #except:
- # warnings.warn(UserWarning(
- # 'Error converting data for component {}'.format(comp_ID)))
-
- # finally, copy the population file
- shutil.copy(
- data_dir / 'population.json',
- target_dir / 'population.json'
- )
-
-
-[docs]def create_HAZUS_EQ_json_files(data_dir, target_dir):
- """
- Create JSON data files from publicly available HAZUS data.
-
- HAZUS damage and loss information is publicly available in the technical
- manuals. The relevant tables have been converted into a JSON input file
- (hazus_data_eq.json) that is stored in the 'resources/HAZUS MH 2.1' folder
- in the pelicun repo. Here we read that file (or a file of similar format)
- and produce damage and loss data for Fragility Groups in the common
- SimCenter JSON format.
-
- HAZUS handles damage and losses at the assembly level differentiating only
- structural and two types of non-structural component assemblies. In this
- implementation we consider each of those assemblies a Fragility Group
- and describe their damage and its consequences in a FEMA P58-like framework
- but using the data from the HAZUS Technical Manual.
-
- Parameters
- ----------
- data_dir: string
- Path to the folder with the hazus_data_eq JSON file.
- target_dir: string
- Path to the folder where the results shall be saved. The population
- distribution file will be saved here, the DL JSON files will be saved
- to a 'DL json' subfolder.
-
- """
-
- data_dir = Path(data_dir).resolve()
- target_dir = Path(target_dir).resolve()
- DL_dir = None
-
- convert_design_level = {
- 'High_code' : 'HC',
- 'Moderate_code': 'MC',
- 'Low_code' : 'LC',
- 'Pre_code' : 'PC'
- }
-
- convert_DS_description = {
- 'DS1': 'Slight',
- 'DS2': 'Moderate',
- 'DS3': 'Extensive',
- 'DS4': 'Complete',
- 'DS5': 'Collapse',
- }
-
- # open the raw HAZUS data
- with open(data_dir / 'hazus_data_eq.json', 'r') as f:
- raw_data = json.load(f)
-
- design_levels = list(
- raw_data['Structural_Fragility_Groups']['EDP_limits'].keys())
- building_types = list(
- raw_data['Structural_Fragility_Groups']['P_collapse'].keys())
- occupancy_types = list(raw_data['Structural_Fragility_Groups'][
- 'Reconstruction_cost'].keys())
-
- S_data = raw_data['Structural_Fragility_Groups']
- NSA_data = raw_data[
- 'NonStructural_Acceleration_Sensitive_Fragility_Groups']
- NSD_data = raw_data['NonStructural_Drift_Sensitive_Fragility_Groups']
-
- for ot in occupancy_types:
-
- # first, structural fragility groups
- for dl in design_levels:
- for bt in building_types:
- if bt in S_data['EDP_limits'][dl].keys():
-
- json_output = {}
-
- dl_id = 'S-{}-{}-{}'.format(bt,
- convert_design_level[dl],
- ot)
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': False,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': dl_id,
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Roof Drift Ratio',
- 'Unit': [1, 'rad']
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
- EDP_lim = S_data['EDP_limits'][dl][bt]
-
- for dsg_i in range(4):
- json_output['DSGroups'].append({
- 'MedianEDP' : EDP_lim[dsg_i],
- 'Beta' : S_data['Fragility_beta'][dl],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {},
- 'Description' : 'DS{}'.format(dsg_i + 1),
- }]
- })
- # the last DSG is different
- if dsg_i == 3:
- json_output['DSGroups'][-1][
- 'DSGroupType'] = 'MutuallyExclusive'
- DS5_w = S_data['P_collapse'][bt]
- json_output['DSGroups'][-1][
- 'DamageStates'].append({
- 'Weight' : DS5_w,
- 'Consequences': {},
- 'Description' : 'DS5'
- })
- json_output['DSGroups'][-1]['DamageStates'][0][
- 'Weight'] = 1.0 - DS5_w
-
- for dsg_i, DSG in enumerate(json_output['DSGroups']):
- base_cost = S_data['Reconstruction_cost'][ot][dsg_i] / 100.
- base_time = S_data['Reconstruction_time'][ot][dsg_i]
-
- for DS in DSG['DamageStates']:
- DS_id = DS['Description']
- DS['Consequences'] = {
- # injury rates are provided in percentages of the population
- 'Injuries' : [
- {'Amount': val / 100.} for val in
- S_data['Injury_rates'][DS_id][bt]],
- # reconstruction cost is provided in percentages of replacement cost
- 'ReconstructionCost': {
- "Amount": base_cost,
- "CurveType": "N/A",
- },
- 'ReconstructionTime': {
- "Amount": base_time,
- "CurveType": "N/A",
- }
- }
- DS['Description'] = convert_DS_description[
- DS['Description']]
-
- # create the DL json directory (if it does not exist)
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # second, nonstructural acceleration sensitive fragility groups
- json_output = {}
-
- dl_id = 'NSA-{}-{}'.format(convert_design_level[dl], ot)
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': False,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': dl_id,
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Peak Floor Acceleration',
- 'Unit': [1, 'g'],
- 'Offset': NSA_data.get('Offset', 0)
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
-
- for dsg_i in range(4):
- base_cost = NSA_data['Reconstruction_cost'][ot][dsg_i] / 100.
-
- json_output['DSGroups'].append({
- 'MedianEDP' : NSA_data['EDP_limits'][dl][dsg_i],
- 'Beta' : NSA_data['Fragility_beta'],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {
- # reconstruction cost is provided in percentages of replacement cost
- 'ReconstructionCost': {
- "Amount": base_cost,
- "CurveType": "N/A",
- },
- },
- 'Description' : convert_DS_description[
- 'DS{}'.format(dsg_i + 1)]
- }]
- })
-
- # create the DL json directory (if it does not exist)
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # third, nonstructural drift sensitive fragility groups
- json_output = {}
-
- dl_id = 'NSD-{}'.format(ot)
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': False,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': dl_id,
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Roof Drift Ratio',
- 'Unit': [1, 'rad']
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
-
- for dsg_i in range(4):
- base_cost = NSD_data['Reconstruction_cost'][ot][dsg_i] / 100.
-
- json_output['DSGroups'].append({
- 'MedianEDP' : NSD_data['EDP_limits'][dsg_i],
- 'Beta' : NSD_data['Fragility_beta'],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {
- # reconstruction cost is provided in percentages of replacement cost
- 'ReconstructionCost': {
- "Amount": base_cost,
- "CurveType": "N/A",
- },
- },
- 'Description' : convert_DS_description[
- 'DS{}'.format(dsg_i + 1)]
- }]
- })
-
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # finally, prepare the population distribution data
- PD_data = raw_data['Population_Distribution']
-
- pop_output = {}
- for ot in PD_data.keys():
- night_ids = raw_data['Parts_of_day']['Nighttime']
- day_ids = raw_data['Parts_of_day']['Daytime']
- commute_ids = raw_data['Parts_of_day']['Commute']
-
- daily_pop = np.ones(24)
- daily_pop[night_ids] = PD_data[ot][0]
- daily_pop[day_ids] = PD_data[ot][1]
- daily_pop[commute_ids] = PD_data[ot][2]
- daily_pop = list(daily_pop)
-
- # HAZUS does not introduce monthly and weekend/weekday variation
- pop_output.update({ot: {
- "weekday": {
- "daily" : daily_pop,
- "monthly": list(np.ones(12))
- },
- "weekend": {
- "daily" : daily_pop,
- "monthly": list(np.ones(12))
- }
- }})
-
- with open(target_dir / 'population.json', 'w') as f:
- json.dump(pop_output, f, indent=2)
-
-[docs]def create_HAZUS_EQ_story_json_files(data_dir, target_dir):
- """
- Create JSON data files from publicly available HAZUS data.
-
- HAZUS damage and loss information is publicly available in the technical
- manuals. The relevant tables have been converted into a JSON input file
- (hazus_data_eq.json) that is stored in the 'resources/HAZUS MH 2.1' folder
- in the pelicun repo. Here we read that file (or a file of similar format)
- and produce damage and loss data for Fragility Groups in the common
- SimCenter JSON format.
-
- HAZUS handles damage and losses at the assembly level differentiating only
- structural and two types of non-structural component assemblies. In this
- implementation we consider each of those assemblies a Fragility Group
- and describe their damage and its consequences in a FEMA P58-like framework
- but using the data from the HAZUS Technical Manual.
-
- Parameters
- ----------
- data_dir: string
- Path to the folder with the hazus_data_eq JSON file.
- target_dir: string
- Path to the folder where the results shall be saved. The population
- distribution file will be saved here, the DL JSON files will be saved
- to a 'DL json' subfolder.
-
- """
-
- data_dir = Path(data_dir).resolve()
- target_dir = Path(target_dir).resolve()
- DL_dir = None
-
- convert_design_level = {
- 'High_code' : 'HC',
- 'Moderate_code': 'MC',
- 'Low_code' : 'LC',
- 'Pre_code' : 'PC'
- }
-
- convert_DS_description = {
- 'DS1': 'Slight',
- 'DS2': 'Moderate',
- 'DS3': 'Extensive',
- 'DS4': 'Complete',
- 'DS5': 'Collapse',
- }
-
- # open the raw HAZUS data
- with open(data_dir / 'hazus_data_eq.json', 'r') as f:
- raw_data = json.load(f)
-
- design_levels = list(
- raw_data['Structural_Fragility_Groups']['EDP_limits'].keys())
- building_types = list(
- raw_data['Structural_Fragility_Groups']['P_collapse'].keys())
- occupancy_types = list(raw_data['Structural_Fragility_Groups'][
- 'Reconstruction_cost'].keys())
-
- S_data = raw_data['Structural_Fragility_Groups']
- NSA_data = raw_data[
- 'NonStructural_Acceleration_Sensitive_Fragility_Groups']
- NSD_data = raw_data['NonStructural_Drift_Sensitive_Fragility_Groups']
-
- for ot in occupancy_types:
-
- # first, structural fragility groups
- for dl in design_levels:
- for bt in building_types:
- if bt in S_data['EDP_limits'][dl].keys():
-
- json_output = {}
-
- dl_id = 'S-{}-{}-{}'.format(bt,
- convert_design_level[dl],
- ot)
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': True,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': dl_id,
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Story Drift Ratio',
- 'Unit': [1, 'rad']
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
- EDP_lim = S_data['EDP_limits'][dl][bt]
-
- for dsg_i in range(4):
- json_output['DSGroups'].append({
- 'MedianEDP' : EDP_lim[dsg_i],
- 'Beta' : S_data['Fragility_beta'][dl],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {},
- 'Description' : 'DS{}'.format(dsg_i + 1),
- }]
- })
- # the last DSG is different
- if dsg_i == 3:
- json_output['DSGroups'][-1][
- 'DSGroupType'] = 'MutuallyExclusive'
- DS5_w = S_data['P_collapse'][bt]
- json_output['DSGroups'][-1][
- 'DamageStates'].append({
- 'Weight' : DS5_w,
- 'Consequences': {},
- 'Description' : 'DS5'
- })
- json_output['DSGroups'][-1]['DamageStates'][0][
- 'Weight'] = 1.0 - DS5_w
-
- for dsg_i, DSG in enumerate(json_output['DSGroups']):
- base_cost = S_data['Reconstruction_cost'][ot][dsg_i] / 100.
- base_time = S_data['Reconstruction_time'][ot][dsg_i]
-
- for DS in DSG['DamageStates']:
- DS_id = DS['Description']
- DS['Consequences'] = {
- # injury rates are provided in percentages of the population
- 'Injuries' : [
- {'Amount': val / 100.} for val in
- S_data['Injury_rates'][DS_id][bt]],
- # reconstruction cost is provided in percentages of replacement cost
- 'ReconstructionCost': {
- "Amount": base_cost,
- "CurveType": "N/A",
- },
- 'ReconstructionTime': {
- "Amount": base_time,
- "CurveType": "N/A",
- }
- }
- DS['Description'] = convert_DS_description[
- DS['Description']]
-
- # create the DL json directory (if it does not exist)
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # second, nonstructural acceleration sensitive fragility groups
- json_output = {}
-
- dl_id = 'NSA-{}-{}'.format(convert_design_level[dl], ot)
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': False,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': dl_id,
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Peak Floor Acceleration',
- 'Unit': [1, 'g'],
- 'Offset': NSA_data.get('Offset', 0)
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
-
- for dsg_i in range(4):
- base_cost = NSA_data['Reconstruction_cost'][ot][dsg_i] / 100.
-
- json_output['DSGroups'].append({
- 'MedianEDP' : NSA_data['EDP_limits'][dl][dsg_i],
- 'Beta' : NSA_data['Fragility_beta'],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {
- # reconstruction cost is provided in percentages of replacement cost
- 'ReconstructionCost': {
- "Amount": base_cost,
- "CurveType": "N/A",
- },
- },
- 'Description' : convert_DS_description[
- 'DS{}'.format(dsg_i + 1)]
- }]
- })
-
- # create the DL json directory (if it does not exist)
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # third, nonstructural drift sensitive fragility groups
- json_output = {}
-
- dl_id = 'NSD-{}'.format(ot)
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': True,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': dl_id,
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Story Drift Ratio',
- 'Unit': [1, 'rad']
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
-
- for dsg_i in range(4):
- base_cost = NSD_data['Reconstruction_cost'][ot][dsg_i] / 100.
-
- json_output['DSGroups'].append({
- 'MedianEDP' : NSD_data['EDP_limits'][dsg_i],
- 'Beta' : NSD_data['Fragility_beta'],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {
- # reconstruction cost is provided in percentages of replacement cost
- 'ReconstructionCost': {
- "Amount": base_cost,
- "CurveType": "N/A",
- },
- },
- 'Description' : convert_DS_description[
- 'DS{}'.format(dsg_i + 1)]
- }]
- })
-
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # finally, prepare the population distribution data
- PD_data = raw_data['Population_Distribution']
-
- pop_output = {}
- for ot in PD_data.keys():
- night_ids = raw_data['Parts_of_day']['Nighttime']
- day_ids = raw_data['Parts_of_day']['Daytime']
- commute_ids = raw_data['Parts_of_day']['Commute']
-
- daily_pop = np.ones(24)
- daily_pop[night_ids] = PD_data[ot][0]
- daily_pop[day_ids] = PD_data[ot][1]
- daily_pop[commute_ids] = PD_data[ot][2]
- daily_pop = list(daily_pop)
-
- # HAZUS does not introduce monthly and weekend/weekday variation
- pop_output.update({ot: {
- "weekday": {
- "daily" : daily_pop,
- "monthly": list(np.ones(12))
- },
- "weekend": {
- "daily" : daily_pop,
- "monthly": list(np.ones(12))
- }
- }})
-
- with open(target_dir / 'population.json', 'w') as f:
- json.dump(pop_output, f, indent=2)
-
-[docs]def create_HAZUS_EQ_PGA_json_files(data_dir, target_dir):
- """
- Create JSON data files from publicly available HAZUS data.
-
- HAZUS damage and loss information is publicly available in the technical
- manuals. The relevant tables have been converted into a JSON input file
- (hazus_data_eq.json) that is stored in the 'resources/HAZUS MH 2.1
- earthquake PGA' folder in the pelicun repo. Here we read that file (or a
- file of similar format) and produce damage and loss data for Fragility
- Groups in the common SimCenter JSON format.
-
- HAZUS handles damage and losses at the assembly level. This method assumes
- that we use the fragility curves controlled by equivalent PGA from HAZUS.
- Those curves are only available for structural component assemblies. A
- Fragility Group is created for each of the structural configurations that
- describes their damage and its consequences in a FEMA P58-like framework
- but using the data from the HAZUS Technical Manual.
-
- Parameters
- ----------
- data_dir: string
- Path to the folder with the hazus_data_eq JSON file.
- target_dir: string
- Path to the folder where the results shall be saved. The population
- distribution file will be saved here, the DL JSON files will be saved
- to a 'DL json' subfolder.
-
- """
-
- data_dir = Path(data_dir).resolve()
- target_dir = Path(target_dir).resolve()
- DL_dir = None
-
- convert_design_level = {
- 'High_code' : 'HC',
- 'Moderate_code': 'MC',
- 'Low_code' : 'LC',
- 'Pre_code' : 'PC'
- }
-
- convert_DS_description = {
- 'DS1': 'Slight',
- 'DS2': 'Moderate',
- 'DS3': 'Extensive',
- 'DS4': 'Complete',
- 'DS5': 'Collapse',
- }
-
- convert_GF_description = {
- 'H_S': 'Lateral spreading, shallow foundation',
- 'H_D': 'Lateral spreading, deep foundation',
- 'V_S': 'Ground settlement, shallow foundation',
- 'V_D': 'Ground settlement, deep foundation',
- }
-
- # open the raw HAZUS data
- with open(data_dir / 'hazus_data_eq.json', 'r') as f:
- raw_data = json.load(f)
-
- # First, the ground shaking fragilities
- S_data = raw_data['Structural_Fragility_Groups']
-
- design_levels = list(S_data['EDP_limits'].keys())
- building_types = list(S_data['P_collapse'].keys())
- occupancy_types = list(S_data['Reconstruction_cost'].keys())
-
- for ot in occupancy_types:
- for dl in design_levels:
- for bt in building_types:
- if bt in S_data['EDP_limits'][dl].keys():
-
- json_output = {}
-
- dl_id = 'S-{}-{}-{}'.format(bt,
- convert_design_level[dl],
- ot)
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': True,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': dl_id,
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Peak Ground Acceleration',
- 'Unit': [1, 'g']
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
- EDP_lim = S_data['EDP_limits'][dl][bt]
-
- for dsg_i in range(4):
- json_output['DSGroups'].append({
- 'MedianEDP' : EDP_lim[dsg_i],
- 'Beta' : S_data['Fragility_beta'][dl],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {},
- 'Description' : 'DS{}'.format(dsg_i + 1),
- }]
- })
- # the last DSG is different
- if dsg_i == 3:
- json_output['DSGroups'][-1][
- 'DSGroupType'] = 'MutuallyExclusive'
- DS5_w = S_data['P_collapse'][bt]
- json_output['DSGroups'][-1][
- 'DamageStates'].append({
- 'Weight' : DS5_w,
- 'Consequences': {},
- 'Description' : 'DS5'
- })
- json_output['DSGroups'][-1]['DamageStates'][0][
- 'Weight'] = 1.0 - DS5_w
-
- for dsg_i, DSG in enumerate(json_output['DSGroups']):
- base_cost = S_data['Reconstruction_cost'][ot][dsg_i] / 100.
- base_time = S_data['Reconstruction_time'][ot][dsg_i]
-
- for DS in DSG['DamageStates']:
- DS_id = DS['Description']
- DS['Consequences'] = {
- # injury rates are provided in percentages of the population
- 'Injuries' : [
- {'Amount': val / 100.} for val in
- S_data['Injury_rates'][DS_id][bt]],
- # reconstruction cost is provided in percentages of replacement cost
- 'ReconstructionCost': {
- "Amount": base_cost,
- "CurveType": "N/A",
- },
- 'ReconstructionTime': {
- "Amount": base_time,
- "CurveType": "N/A",
- }
- }
- DS['Description'] = convert_DS_description[
- DS['Description']]
-
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # Second, the ground failure fragilities
- L_data = raw_data['Ground_Failure']
-
- ground_failure_types = list(L_data['EDP_limits'].keys())
-
- for gf in ground_failure_types:
- for bt in building_types:
- if bt in L_data['P_collapse'].keys():
-
- json_output = {}
-
- dl_id = f'GF-{gf}-{bt}'
-
- # this might get replaced by a more descriptive name in the future
- json_output.update({'Name': dl_id})
-
- # General info
- json_output.update({
- 'Directional': True,
- 'GeneralInformation': {
- 'ID' : dl_id,
- 'Description': convert_GF_description[gf],
- # this might get replaced by more details in the future
- # other fields can be added here if needed
- }
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Permanent Ground Deformation',
- 'Unit': [1, 'in']
- }
- })
-
- # Damage State info
- json_output.update({'DSGroups': []})
- EDP_lim = L_data['EDP_limits'][gf]
- beta_vals = L_data['Fragility_beta'][gf]
-
- for dsg_i in range(2):
- json_output['DSGroups'].append({
- 'MedianEDP' : EDP_lim[dsg_i],
- 'Beta' : beta_vals[dsg_i],
- 'CurveType' : 'LogNormal',
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {},
- 'Description' : 'DS{}'.format(dsg_i + 1),
- }]
- })
- # the last DSG is different
- if dsg_i == 1:
- json_output['DSGroups'][-1][
- 'DSGroupType'] = 'MutuallyExclusive'
- DS5_w = L_data['P_collapse'][bt]
- json_output['DSGroups'][-1]['DamageStates'].append({
- 'Weight' : DS5_w,
- 'Consequences': {},
- 'Description' : 'DS5'
- })
- json_output['DSGroups'][-1]['DamageStates'][0]['Weight'] = 1.0 - DS5_w
-
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- # consequences are handled through propagating damage to other components
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
- # prepare the population distribution data
- PD_data = raw_data['Population_Distribution']
-
- pop_output = {}
- for ot in PD_data.keys():
- night_ids = raw_data['Parts_of_day']['Nighttime']
- day_ids = raw_data['Parts_of_day']['Daytime']
- commute_ids = raw_data['Parts_of_day']['Commute']
-
- daily_pop = np.ones(24)
- daily_pop[night_ids] = PD_data[ot][0]
- daily_pop[day_ids] = PD_data[ot][1]
- daily_pop[commute_ids] = PD_data[ot][2]
- daily_pop = list(daily_pop)
-
- # HAZUS does not introduce monthly and weekend/weekday variation
- pop_output.update({ot: {
- "weekday": {
- "daily" : daily_pop,
- "monthly": list(np.ones(12))
- },
- "weekend": {
- "daily" : daily_pop,
- "monthly": list(np.ones(12))
- }
- }})
-
- with open(target_dir / 'population.json', 'w') as f:
- json.dump(pop_output, f, indent=2)
-
-[docs]def create_HAZUS_HU_json_files(data_dir, target_dir):
- """
- Create JSON data files from publicly available HAZUS data.
-
- HAZUS damage and loss information is publicly available in the technical
- manuals and the HAZUS software tool. The relevant data have been collected
- in a series of Excel files (e.g., hu_Wood.xlsx) that are stored in the
- 'resources/HAZUS MH 2.1 hurricane' folder in the pelicun repo. Here we read
- that file (or a file of similar format) and produce damage and loss data
- for Fragility Groups in the common SimCenter JSON format.
-
- The HAZUS hurricane methodology handles damage and losses at the assembly
- level. In this implementation each building is represented by one Fragility
- Group that describes the damage states and their consequences in a FEMA
- P58-like framework but using the data from the HAZUS Technical Manual.
-
- Note: HAZUS calculates lossess independently of damage using peak wind gust
- speed as a controlling variable. We fitted a model to the curves in HAZUS
- that assigns losses to each damage state and determines losses as a function
- of building damage. Results shall be in good agreement with those of HAZUS
- for the majority of building configurations. Exceptions and more details
- are provided in the ... section of the documentation.
-
- Parameters
- ----------
- data_dir: string
- Path to the folder with the hazus_data_eq JSON file.
- target_dir: string
- Path to the folder where the results shall be saved. The population
- distribution file will be saved here, the DL JSON files will be saved
- to a 'DL json' subfolder.
-
- """
-
- data_dir = Path(data_dir).resolve()
- target_dir = Path(target_dir).resolve()
- DL_dir = None
-
- # open the raw HAZUS data
- df_wood = pd.read_excel(data_dir / 'hu_Wood.xlsx')
-
- # some general formatting to make file name generation easier
- df_wood['shutters'] = df_wood['shutters'].astype(int)
- df_wood['terr_rough'] = (df_wood['terr_rough'] * 100.).astype(int)
-
- convert_building_type = {
- 'WSF1' : 'Wood Single-Family Homes 1 story',
- 'WSF2' : 'Wood Single-Family Homes 2+ stories',
- 'WMUH1': 'Wood Multi-Unit or Hotel or Motel 1 story',
- 'WMUH2': 'Wood Multi-Unit or Hotel or Motel 2 stories',
- 'WMUH3': 'Wood Multi-Unit or Hotel or Motel 3+ stories',
- }
-
- convert_bldg_char_names = {
- 'roof_shape' : 'Roof Shape',
- 'sec_water_res' : 'Secondary Water Resistance',
- 'roof_deck_attch': 'Roof Deck Attachment',
- 'roof_wall_conn' : 'Roof-Wall Connection',
- 'garage' : 'Garage',
- 'shutters' : 'Shutters',
- 'roof_cover' : 'Roof Cover Type',
- 'roof_quality' : 'Roof Cover Quality',
- 'terr_rough' : 'Terrain',
- }
-
- convert_bldg_chars = {
- 1 : True,
- 0 : False,
-
- 'gab' : 'gable',
- 'hip' : 'hip',
- 'flt' : 'flat',
-
- '6d' : '6d @ 6"/12"',
- '8d' : '8d @ 6"/12"',
- '6s' : '6d/8d mix @ 6"/6"',
- '8s' : '8D @ 6"/6"',
-
- 'tnail': 'Toe-nail',
- 'strap': 'Strap',
-
- 'no' : 'None',
- 'wkd' : 'Weak',
- 'std' : 'Standard',
- 'sup' : 'SFBC 1994',
-
- 'bur' : 'BUR',
- 'spm' : 'SPM',
-
- 'god' : 'Good',
- 'por' : 'Poor',
-
- 3 : 'Open',
- 15 : 'Light Suburban',
- 35 : 'Suburban',
- 70 : 'Light Trees',
- 100 : 'Trees',
- }
-
- convert_dist = {
- 'normal' : 'Normal',
- 'lognormal': 'LogNormal',
- }
-
- convert_ds = {
- 1: 'Minor',
- 2: 'Moderate',
- 3: 'Severe',
- 4: 'Destruction',
- }
-
- for index, row in df_wood.iterrows():
- #print(index, end=' ')
-
- json_output = {}
-
- # define the name of the building damage and loss configuration
- bldg_type = row["bldg_type"]
-
- if bldg_type[:3] == "WSF":
- cols_of_interest = ["bldg_type", "roof_shape", "sec_water_res",
- "roof_deck_attch", "roof_wall_conn", "garage",
- "shutters", "terr_rough"]
- elif bldg_type[:4] == "WMUH":
- cols_of_interest = ["bldg_type", "roof_shape", "roof_cover",
- "roof_quality", "sec_water_res",
- "roof_deck_attch", "roof_wall_conn", "shutters",
- "terr_rough"]
-
- bldg_chars = row[cols_of_interest]
-
- if np.isnan(bldg_chars["sec_water_res"]):
- bldg_chars["sec_water_res"] = 'null'
- else:
- bldg_chars["sec_water_res"] = int(bldg_chars["sec_water_res"])
-
- if bldg_type[:4] == "WMUH":
- if (not isinstance(bldg_chars["roof_cover"],str)
- and np.isnan(bldg_chars["roof_cover"])):
- bldg_chars["roof_cover"] = 'null'
- if (not isinstance(bldg_chars["roof_quality"], str)
- and np.isnan(bldg_chars["roof_quality"])):
- bldg_chars["roof_quality"] = 'null'
-
- dl_id = "_".join(bldg_chars.astype(str))
-
- json_output.update({'Name': dl_id})
-
- # general information
- json_output.update({
- 'GeneralInformation': {
- 'ID' : str(index),
- 'Description' : dl_id,
- 'Building type': convert_building_type[bldg_type],
- }
- })
- for col in cols_of_interest:
- if (col != 'bldg_type') and (bldg_chars[col] != 'null'):
- json_output['GeneralInformation'].update({
- convert_bldg_char_names[col]: convert_bldg_chars[
- bldg_chars[col]]
- })
-
- # EDP info
- json_output.update({
- 'EDP': {
- 'Type': 'Peak Gust Wind Speed',
- 'Unit': [1, 'mph']
- }
- })
-
- # Damage States
- json_output.update({'DSGroups': []})
-
- for dsg_i in range(1, 5):
- json_output['DSGroups'].append({
- 'MedianEDP' : row['DS{}_mu'.format(dsg_i)],
- 'Beta' : row['DS{}_sig'.format(dsg_i)],
- 'CurveType' : convert_dist[row['DS{}_dist'.format(dsg_i)]],
- 'DSGroupType' : 'Single',
- 'DamageStates': [{
- 'Weight' : 1.0,
- 'Consequences': {
- 'ReconstructionCost': {
- 'Amount': row[
- 'L{}'.format(dsg_i)] if dsg_i < 4 else 1.0
- }
- },
- 'Description' : convert_ds[dsg_i]
- }]
- })
-
- if DL_dir is None:
- DL_dir = target_dir / "DL json"
- DL_dir.mkdir(exist_ok=True)
-
- with open(DL_dir / f'{dl_id}.json', 'w') as f:
- json.dump(json_output, f, indent=2)
-
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-# Pouria Kourehpaz
-# Kuanshi Zhong
-
-"""
-This module has classes and methods that handle file input and output.
-
-.. rubric:: Contents
-
-.. autosummary::
-
- read_SimCenter_DL_input
- read_SimCenter_EDP_input
- read_population_distribution
- read_component_DL_data
- write_SimCenter_DL_output
- write_SimCenter_EDP_output
- write_SimCenter_DM_output
- write_SimCenter_DV_output
-
-"""
-
-from .base import *
-from pathlib import Path
-from .db import convert_Series_to_dict
-
-import json, posixpath
-
-from tables.exceptions import HDF5ExtError
-from time import sleep
-
-
-import warnings
-
-convert_dv_name = {
- 'DV_rec_cost': 'Reconstruction Cost',
- 'DV_rec_time': 'Reconstruction Time',
- 'DV_injuries_0': 'Injuries lvl. 1',
- 'DV_injuries_1': 'Injuries lvl. 2',
- 'DV_injuries_2': 'Injuries lvl. 3',
- 'DV_injuries_3': 'Injuries lvl. 4',
- 'DV_red_tag': 'Red Tag ',
-}
-
-# this is a convenience function for converting strings to float or None
-def float_or_None(string):
- try:
- res = float(string)
- return res
- except:
- return None
-
-def int_or_None(string):
- try:
- res = int(string)
- return res
- except:
- return None
-
-def process_loc(string, stories):
- try:
- res = int(string)
- return [res, ]
- except:
- if "-" in string:
- s_low, s_high = string.split('-')
- s_low = process_loc(s_low, stories)
- s_high = process_loc(s_high, stories)
- return list(range(s_low[0], s_high[0]+1))
- elif string == "all":
- return list(range(1, stories+1))
- elif string == "top":
- return [stories,]
- elif string == "roof":
- return [stories,]
- else:
- return None
-
-
-[docs]def read_SimCenter_DL_input(input_path, assessment_type='P58', verbose=False):
- """
- Read the damage and loss input information from a json file.
-
- The SimCenter in the function name refers to having specific fields
- available in the file. Such a file is automatically prepared by the
- SimCenter PBE Application, but it can also be easily manipulated or created
- manually. The accepted input fields are explained in detail in the Input
- section of the documentation.
-
- Parameters
- ----------
- input_path: string
- Location of the DL input json file.
- assessment_type: {'P58', 'HAZUS_EQ', 'HAZUS_HU'}
- Tailors the warnings and verifications towards the type of assessment.
- default: 'P58'.
- verbose: boolean
- If True, the function echoes the information read from the file. This
- can be useful to ensure that the information in the file is properly
- read by the method.
-
- Returns
- -------
- data: dict
- A dictionary with all the damage and loss data.
-
- """
-
- AT = assessment_type
-
- log_msg('\t\tOpening the configuration file...')
- with open(input_path, 'r') as f:
- jd = json.load(f)
-
- # get the data required for DL
- data = dict([(label, dict()) for label in [
- 'general', 'units', 'unit_names', 'components', 'collapse_modes',
- 'decision_variables', 'dependencies', 'data_sources', 'damage_logic'
- ]])
-
- # create a few internal variables for convenience
- DL_input = jd['DamageAndLoss']
-
- log_msg('\t\tLoading the Models:')
- log_msg('\t\t\tResponse Model')
- response = DL_input.get('ResponseModel',None)
- if response is not None:
- res_description = response.get('ResponseDescription', None)
- det_lims = response.get('DetectionLimits', None)
- uncertainty = response.get('AdditionalUncertainty', None)
-
- else:
- res_description = None
- det_lims = None
- uncertainty = None
-
- log_msg('\t\t\tDamage Model')
- damage = DL_input.get('DamageModel',None)
- if damage is not None:
- irrep_res_drift = damage.get('IrreparableResidualDrift', None)
- coll_prob = damage.get('CollapseProbability', None)
- coll_lims = damage.get('CollapseLimits', None)
- design_lvl = damage.get('DesignLevel', None)
- struct_type = damage.get('StructureType', None)
-
- else:
- irrep_res_drift = None
- coll_prob = None
- coll_lims = None
- design_lvl = None
- struct_type = None
-
- log_msg('\t\t\tLoss Model')
- loss = DL_input.get('LossModel', None)
- if loss is not None:
- repl_cost = loss.get('ReplacementCost', None)
- repl_time = loss.get('ReplacementTime', None)
- dec_vars = loss.get('DecisionVariables', None)
- inhabitants = loss.get('Inhabitants', None)
-
- else:
- repl_cost = None
- repl_time = None
- dec_vars = None
- inhabitants = None
-
- log_msg('\t\t\tPerformance Model')
- components = DL_input.get('Components', None)
-
- depends = DL_input.get('Dependencies', None)
- coll_modes = DL_input.get('CollapseModes', None)
-
- dam_logic = DL_input.get('DamageLogic', None)
-
- # decision variables of interest
- if dec_vars is not None:
- for target_att, source_att in [ ['injuries', 'Injuries'],
- ['rec_cost', 'ReconstructionCost'],
- ['rec_time', 'ReconstructionTime'],
- ['red_tag', 'RedTag'], ]:
- val = bool(dec_vars.get(source_att, False))
- data['decision_variables'].update({target_att: val})
- else:
- show_warning("No decision variables specified in the input file.")
- log_msg("Assuming that only reconstruction cost and time needs to be calculated.")
- data['decision_variables'].update({ 'injuries': False,
- 'rec_cost': True,
- 'rec_time': True,})
- # red tag is only used by P58 now
- if AT == 'P58':
- data['decision_variables'].update({'red_tag': False})
-
- dec_vars = data['decision_variables']
-
- # data sources
- # default data locations
- default_data_name = {
- 'P58' : 'FEMA P58 second edition',
- 'HAZUS_EQ': 'HAZUS MH 2.1 earthquake',
- 'HAZUS_HU': 'HAZUS MH 2.1 hurricane'
- }
-
- # check if the user specified custom data sources
- path_CMP_data = DL_input.get("ComponentDataFolder", "")
-
- if path_CMP_data == "":
- path_CMP_data = pelicun_path
- if AT == 'P58':
- path_CMP_data += '/resources/FEMA_P58_2nd_ed.hdf'
- elif AT == 'HAZUS_EQ':
- path_CMP_data += '/resources/HAZUS_MH_2.1_EQ.hdf'
- elif AT == 'HAZUS_HU':
- path_CMP_data += '/resources/HAZUS_MH_2.1_HU.hdf'
- data['data_sources'].update({'path_CMP_data': path_CMP_data})
-
- # The population data is only needed if we are interested in injuries
- if inhabitants is not None:
- path_POP_data = inhabitants.get("PopulationDataFile", "")
- else:
- path_POP_data = ""
-
- if data['decision_variables']['injuries']:
- if path_POP_data == "":
- path_POP_data = pelicun_path
- if AT == 'P58':
- path_POP_data += '/resources/FEMA_P58_2nd_ed.hdf'
- elif AT == 'HAZUS_EQ':
- path_POP_data += '/resources/HAZUS_MH_2.1_EQ.hdf'
- data['data_sources'].update({'path_POP_data': path_POP_data})
-
- # general information
- GI = jd.get("GeneralInformation", None)
-
- # units
- if (GI is not None) and ('units' in GI.keys()):
- for key, value in GI['units'].items():
- if value == 'in':
- value = 'inch'
- if value in globals().keys():
- data['unit_names'].update({key: value})
- else:
- show_warning("Unknown {} unit: {}".format(key, value))
-
- if 'length' in data['unit_names'].keys():
- if 'area' not in data['unit_names']:
- data['unit_names'].update({
- 'area': data['unit_names']['length']+'2'})
-
- if 'volume' not in data['unit_names']:
- data['unit_names'].update({
- 'volume': data['unit_names']['length']+'3'})
-
- if 'speed' not in data['unit_names'].keys():
- data['unit_names'].update({
- 'speed': data['unit_names']['length']+'ps'})
-
- if 'acceleration' not in data['unit_names'].keys():
- data['unit_names'].update({
- #'acceleration': 1.0 })
- 'acceleration': data['unit_names']['length']+'ps2'})
- else:
- show_warning("No units were specified in the input file.")
- data['unit_names'].update({
- 'force': 'N',
- 'length': 'm',
- 'area': 'm2',
- 'volume': 'm3',
- 'speed': 'mps',
- 'acceleration': 'mps2',
- })
-
- for unit_type, unit_name in data['unit_names'].items():
- data['units'].update({unit_type: globals()[unit_name]})
-
- # other attributes that can be used by a P58 assessment
- if AT == 'P58':
- for target_att, source_att, f_conv, unit_kind, dv_req in [
- ['plan_area', 'PlanArea', float, 'area', 'injuries'],
- ['stories', 'NumberofStories', int, '', 'all'],
- ]:
- if (GI is not None) and (source_att in GI.keys()):
- if unit_kind != '':
- f_unit = data['units'][unit_kind]
- else:
- f_unit = 1
- att_value = f_conv(GI[source_att]) * f_unit
- data['general'].update({target_att: att_value})
- else:
- if (dv_req != '') and ((dv_req == 'all') or dec_vars[dv_req]):
- raise ValueError(
- "{} has to be specified in the DL input file to "
- "estimate {} decision variable(s).".format(source_att,
- dv_req))
- elif AT.startswith('HAZUS'):
- data['general'].update({'stories': int(GI['NumberofStories'])})
-
- # is this a coupled assessment?
- if res_description is not None:
- data['general'].update({'coupled_assessment':
- res_description.get('CoupledAssessment', False)})
- else:
- data['general'].update({'coupled_assessment': False})
-
- # Performance Model
- # Having components defined is not necessary, but if a component is defined
- # then all of its attributes need to be specified. Note that the required
- # set of attributes depends on the type of assessment.
- if components is not None:
- for fg_id, frag_group in components.items():
- if AT == 'P58':
- # TODO: allow location and direction inputs with '-' in them
- comp_data = {
- 'locations' : [],
- 'directions' : [],
- 'quantities' : [],
- 'unit' : [],
- 'distribution': [],
- 'cov' : [],
- 'csg_weights': [],
- }
-
- for comp in frag_group:
- locs = []
- for loc_ in comp['location'].split(','):
- for l in process_loc(loc_, data['general']['stories']):
- locs.append(l)
- locs.sort()
-
- dirs = sorted([int_or_None(dir_)
- for dir_ in comp['direction'].split(',')])
- qnts = [float(qnt)
- for qnt in comp['median_quantity'].split(',')]
- csg_weights = (qnts / np.sum(qnts)).tolist()
- qnts = np.sum(qnts)
-
- pg_count = len(locs) * len(dirs)
-
- comp_data['locations'] = (comp_data['locations'] +
- [l for l in locs for d in dirs])
- comp_data['directions'] = (comp_data['directions'] +
- dirs * len(locs))
-
- unit = comp['unit']
- if unit not in globals().keys():
- raise ValueError(
- "Unknown unit for component {}: {}".format(fg_id,
- unit))
- for i in range(pg_count):
- comp_data['quantities'].append(qnts)
- comp_data['csg_weights'].append(csg_weights)
- comp_data['unit'].append(unit)
- comp_data['distribution'].append(comp.get('distribution', 'N/A'))
- comp_data['cov'].append(comp.get('cov', None))
-
- sorted_ids = np.argsort(comp_data['locations'])
- for key in ['locations', 'directions', 'quantities',
- 'csg_weights', 'distribution', 'cov']:
- comp_data[key] = [comp_data[key][s_id] for s_id in sorted_ids]
-
- if len(set(comp_data['unit'])) != 1:
- raise ValueError(
- "Multiple types of units specified for fragility group "
- "{}. Make sure that every component group in a "
- "fragility group is defined using the same "
- "unit.".format(fg_id))
- comp_data['unit'] = comp_data['unit'][0]
-
- # aggregate PGs that are in the same loc & dir
- PG_loc_dir_list = list(zip(comp_data['locations'], comp_data['directions']))
- to_aggregate = set([x for x in PG_loc_dir_list if PG_loc_dir_list.count(x) > 1])
- for combo in to_aggregate:
- PG_loc_dir_list = list(zip(comp_data['locations'], comp_data['directions']))
- combo_ids = [i for i,e in enumerate(PG_loc_dir_list) if e==combo]
-
- c_base = combo_ids[0]
- comp_data['csg_weights'][c_base] = (np.array(comp_data['csg_weights'][c_base]) * comp_data['quantities'][c_base]).tolist()
- for ci in combo_ids[1:]:
- comp_data['quantities'][c_base] += comp_data['quantities'][ci]
- comp_data['csg_weights'][c_base] += (np.array(comp_data['csg_weights'][ci]) * comp_data['quantities'][ci]).tolist()
- comp_data['csg_weights'][c_base] = (np.array(comp_data['csg_weights'][c_base]) / comp_data['quantities'][c_base]).tolist()
-
- for ci in combo_ids[1:][::-1]:
- for key in ['locations', 'directions', 'quantities',
- 'csg_weights', 'distribution', 'cov']:
- del comp_data[key][ci]
-
- elif AT.startswith('HAZUS'):
- comp_data = {
- 'locations' : [],
- 'directions' : [],
- 'quantities' : [],
- 'unit' : [],
- 'distribution': [],
- 'cov' : [],
- 'csg_weights': [],
- }
-
- for comp in frag_group:
- if 'location' in comp:
- locs = []
- for loc_ in comp['location'].split(','):
- for l in process_loc(loc_, data['general']['stories']):
- locs.append(l)
- locs.sort()
- else:
- locs = [1,]
-
- if 'direction' in comp:
- dirs = sorted([int_or_None(dir_)
- for dir_ in comp['direction'].split(',')])
- else:
- dirs = [1, ]
-
- if 'median_quantity' in comp:
- qnts = [float(qnt)
- for qnt in comp['median_quantity'].split(',')]
- csg_weights = (qnts / np.sum(qnts)).tolist()
- qnts = np.sum(qnts)
-
- pg_count = len(locs) * len(dirs)
-
- comp_data['locations'] = (comp_data['locations'] +
- [l for l in locs for d in dirs])
- comp_data['directions'] = (comp_data['directions'] +
- dirs * len(locs))
-
- if 'unit' in comp:
- unit = comp['unit']
- if unit not in globals().keys():
- raise ValueError(
- "Unknown unit for component {}: {}".format(fg_id,
- unit))
- else:
- unit = 'ea'
-
- for i in range(pg_count):
- comp_data['quantities'].append(qnts)
- comp_data['csg_weights'].append(csg_weights)
- comp_data['unit'].append(unit)
- comp_data['distribution'].append(comp.get('distribution', 'N/A'))
- comp_data['cov'].append(comp.get('cov', None))
-
- sorted_ids = np.argsort(comp_data['locations'])
- for key in ['locations', 'directions', 'quantities',
- 'csg_weights', 'distribution', 'cov']:
- comp_data[key] = [comp_data[key][s_id] for s_id in sorted_ids]
-
- if len(set(comp_data['unit'])) != 1:
- print(comp_data['unit'])
- raise ValueError(
- "Multiple types of units specified for fragility group "
- "{}. Make sure that every component group in a "
- "fragility group is defined using the same "
- "unit.".format(fg_id))
- comp_data['unit'] = comp_data['unit'][0]
-
- # some basic pre-processing
- # sort the dirs and their weights to have better structured
- # matrices later
- #dir_order = np.argsort(comp_data['directions'])
- #comp_data['directions'] = [comp_data['directions'][d_i] for d_i
- # in dir_order]
-
- # get the location(s) of components based on non-zero quantities
- #comp_data.update({
- # 'locations': (np.where(comp_data['quantities'] > 0.)[
- # 0] + 1).tolist()
- #})
- # remove the zeros from the quantities
- #nonzero = comp_data['quantities'] > 0.
- #comp_data['quantities'] = comp_data['quantities'][
- # nonzero].tolist()
-
- # scale the quantities according to the specified unit
-
- # store the component data
- data['components'].update({fg_id: comp_data})
- else:
- show_warning("No components were defined in the input file.")
-
- # collapse modes
- if AT == 'P58':
- # Having collapse modes defined is not necessary, but if a collapse mode
- # is defined, then all of its attributes need to be specified.
- if coll_modes is not None:
- for coll_mode in coll_modes:
- cm_data = {
- 'w' : float(coll_mode['weight']),
- 'injuries' : [float(inj) for inj in
- coll_mode['injuries'].split(',')],
- 'affected_area': [float(cfar) for cfar in
- coll_mode['affected_area'].split(',')],
- }
- if len(cm_data['affected_area']) == 1:
- cm_data['affected_area'] = (np.ones(data['general']['stories'])*cm_data['affected_area']).tolist()
- if len(cm_data['injuries']) == 1:
- cm_data['injuries'] = (np.ones(data['general']['stories'])*cm_data['injuries']).tolist()
- data['collapse_modes'].update({coll_mode['name']: cm_data})
- else:
- show_warning("No collapse modes were defined in the input file.")
-
- # the number of realizations has to be specified in the file
- if res_description is not None:
- realizations = res_description.get("Realizations", None)
- if realizations is not None:
- data['general'].update({'realizations': int(realizations)})
- else:
- raise ValueError(
- "Number of realizations is not specified in the input file.")
-
- EDP_units = dict(
- # PID, PRD, RID, and MID are not here because they are unitless
- PFA = 'acceleration',
- PWS = 'speed',
- PGA = 'acceleration',
- SA = 'acceleration',
- SV = 'speed',
- SD = 'length'
- )
- if AT in ['P58', 'HAZUS_EQ']:
- EDP_keys = ['PID', 'PRD', 'PFA',
- 'PGV', 'RID', 'PMD',
- 'PGA', 'SA', 'SV', 'SD',
- 'RDR','DWD']
- elif AT in ['HAZUS_HU']:
- EDP_keys = ['PWS', ]
-
- # response model info ------------------------------------------------------
- if response is None:
- show_warning("Response model characteristics were not defined in the input "
- "file")
-
- # detection limits
- if ((response is not None) and (det_lims is not None)):
- data['general'].update({
- 'detection_limits':
- dict([(key, float_or_None(value)) for key, value in
- det_lims.items()])})
- DGDL = data['general']['detection_limits']
- # scale the limits by the units
- for EDP_kind, value in DGDL.items():
- if (EDP_kind in EDP_units.keys()) and (value is not None):
- f_EDP = data['units'][EDP_units[EDP_kind]]
- DGDL[EDP_kind] = DGDL[EDP_kind] * f_EDP
- else:
- data['general'].update({'detection_limits':{}})
-
- # make sure that detection limits are initialized
- for key in EDP_keys:
- if key not in data['general']['detection_limits'].keys():
- data['general']['detection_limits'].update({key: None})
-
- # response description
- if ((response is not None) and (res_description is not None)):
- #TODO: move the collapse-related data to another field
- data['general'].update({'response': {
- 'EDP_distribution': res_description.get('EDP_Distribution',
- 'lognormal'),
- 'EDP_dist_basis': res_description.get('BasisOfEDP_Distribution',
- 'all results')}})
- else:
- data['general'].update({'response': {
- 'EDP_distribution': 'lognormal',
- 'EDP_dist_basis' : 'all results'}})
-
- # additional uncertainty
- if ((response is not None) and (uncertainty is not None)):
- data['general'].update({
- 'added_uncertainty': {
- 'beta_gm': float_or_None(uncertainty['GroundMotion']),
- 'beta_m' : float_or_None(uncertainty['Modeling'])}})
- else:
- data['general'].update({
- 'added_uncertainty': {
- 'beta_gm': None,
- 'beta_m': None
- }
- })
-
- # damage model info --------------------------------------------------------
- if damage is None:
- if AT == 'P58':
- show_warning("Damage model characteristics were not defined in the "
- "input file")
- elif AT.startswith('HAZUS'):
- pass
-
- # P58-specific things
- if AT == 'P58':
- # EDP limits for collapse
- if ((damage is not None) and (coll_lims is not None)):
- # load the limits
- data['general'].update({
- 'collapse_limits':
- dict([(key, float_or_None(value)) for key, value
- in coll_lims.items()])})
-
- # scale the limits according to their units
- DGCL = data['general']['collapse_limits']
- for EDP_kind, value in DGCL.items():
- if (EDP_kind in EDP_units.keys()) and (value is not None):
- f_EDP = data['units'][EDP_units[EDP_kind]]
- DGCL[EDP_kind] = DGCL[EDP_kind] * f_EDP
- else:
- data['general'].update({'collapse_limits': {}})
-
- # make sure that collapse limits are initialized
- for key in EDP_keys:
- if key not in data['general']['collapse_limits'].keys():
- data['general']['collapse_limits'].update({key: None})
-
- # irreparable drift
- if ((damage is not None) and (irrep_res_drift is not None)):
- data['general'].update({
- 'irreparable_res_drift':
- dict([(key, float_or_None(value)) for key, value in
- irrep_res_drift.items()])})
- # TODO: move this in the irreparable part of general
- yield_drift = irrep_res_drift.get("YieldDriftRatio", None)
- if yield_drift is not None:
- data['general'].update({
- 'yield_drift': float_or_None(yield_drift)})
- elif ((data['decision_variables']['rec_cost']) or
- (data['decision_variables']['rec_time'])):
- data['general'].update({'yield_drift': 0.01})
-
- elif ((data['decision_variables']['rec_cost']) or
- (data['decision_variables']['rec_time'])):
- pass
- #TODO: show this warning in the log file instead
-
- # show_warning(
- # "Residual drift limits corresponding to irreparable "
- # "damage were not defined in the input file. We assume that "
- # "damage is repairable regardless of the residual drift.")
- # we might need to have a default yield drift here
-
- # collapse probability
- if 'response' not in data['general'].keys():
- data['general'].update({'response': {}})
- if ((damage is not None) and (coll_prob is not None)):
- data['general']['response'].update({
- 'coll_prob' : coll_prob.get('Value', 'estimated'),
- 'CP_est_basis': coll_prob.get('BasisOfEstimate', 'raw EDP')})
- if data['general']['response']['coll_prob'] != 'estimated':
- data['general']['response']['coll_prob'] = \
- float_or_None(data['general']['response']['coll_prob'])
- else:
- data['general']['response'].update({
- 'coll_prob' : 'estimated',
- 'CP_est_basis' : 'raw EDP'})
-
- # loss model info ----------------------------------------------------------
- if loss is None:
- show_warning("Loss model characteristics were not defined in the input file")
-
- # replacement cost
- if ((loss is not None) and (repl_cost is not None)):
- data['general'].update({
- 'replacement_cost': float_or_None(repl_cost)})
- elif data['decision_variables']['rec_cost']:
- if AT == 'P58':
- show_warning("Building replacement cost was not defined in the "
- "input file.")
- elif AT.startswith('HAZUS'):
- raise ValueError(
- "Building replacement cost was not defined in the input "
- "file.")
-
- # replacement time
- if ((loss is not None) and (repl_time is not None)):
- data['general'].update({
- 'replacement_time': float_or_None(repl_time)})
- elif data['decision_variables']['rec_time']:
- if AT == 'P58':
- show_warning("Building replacement cost was not defined in the "
- "input file.")
- elif AT.startswith('HAZUS'):
- raise ValueError(
- "Building replacement cost was not defined in the input "
- "file.")
-
- # inhabitants
- if data['decision_variables']['injuries']:
- if ((loss is not None) and (inhabitants is not None)):
-
- # occupancy type
- occupancy = inhabitants.get("OccupancyType", None)
- if occupancy is not None:
- data['general'].update({'occupancy_type': occupancy})
- else:
- raise ValueError("Occupancy type was not defined in the input "
- "file.")
-
- # event time
- event_time = inhabitants.get("EventTime", None)
- data['general'].update({'event_time': event_time})
-
- # peak population
- peak_pop = inhabitants.get("PeakPopulation", None)
- if peak_pop is not None:
- peak_pop = [float_or_None(pop) for pop in peak_pop.split(',')]
-
- # If the number of stories is specified...
- if 'stories' in data['general'].keys():
- stories = data['general']['stories']
- pop_in = len(peak_pop)
-
- # If only one value is provided then we assume that it
- # corresponds to the whole building
- if pop_in == 1:
- peak_pop = (np.ones(stories)*peak_pop[0]/stories).tolist()
-
- # If more than one value is provided then we assume they
- # define population for every story
- else:
- # If the population list does not provide values
- # for every story the values are assumed to correspond
- # to the lower stories and the upper ones are filled
- # with zeros
- for s in range(pop_in, stories):
- peak_pop.append(0)
-
- if pop_in > 1 and pop_in != stories:
- show_warning(
- "Peak population was specified to some, but not all "
- "stories. The remaining stories are assumed to have "
- "zero population."
- )
-
- data['general'].update({'population': peak_pop})
- else:
- raise ValueError(
- "Peak population was not defined in the input file.")
- else:
- raise ValueError(
- "Information about inhabitants was not defined in the input "
- "file.")
-
- # dependencies -------------------------------------------------------------
-
- # set defaults
- # We assume 'Independent' for all unspecified fields except for the
- # fragilities where 'per ATC recommendation' is the default setting.
- dependency_to_acronym = {
- 'btw. Fragility Groups' : 'FG',
- 'btw. Performance Groups': 'PG',
- 'btw. Floors' : 'LOC',
- 'btw. Directions' : 'DIR',
- 'btw. Component Groups' : 'CSG',
- 'btw. Damage States' : 'DS',
- 'Independent' : 'IND',
- 'per ATC recommendation' : 'ATC',
- }
-
- for target_att, source_att, dv_req in [
- ['quantities', 'Quantities', ''],
- ['fragilities', 'Fragilities', ''],
- ['injuries', 'Injuries', 'injuries'],
- ['rec_costs', 'ReconstructionCosts', 'rec_cost'],
- ['rec_times', 'ReconstructionTimes', 'rec_time'],
- ['red_tags', 'RedTagProbabilities', 'red_tag'],]:
-
- if ((depends is not None) and (source_att in depends.keys())):
- data['dependencies'].update({
- target_att:dependency_to_acronym[depends[source_att]]})
- #elif dv_req == '' or data['decision_variables'][dv_req]:
- else:
- if target_att != 'fragilities':
- data['dependencies'].update({target_att: 'IND'})
- else:
- data['dependencies'].update({target_att: 'ATC'})
-
- log_msg("\t\t\t\t\t"+
- "Correlation between {} was not ".format(source_att)+
- "defined in the input file. Using default values.")
-
- if ((depends is not None) and ('CostAndTime' in depends.keys())):
- data['dependencies'].update({
- 'cost_and_time': bool(depends['CostAndTime'])})
- elif ((data['decision_variables']['rec_cost']) or
- (data['decision_variables']['rec_time'])):
- data['dependencies'].update({'cost_and_time': False})
- log_msg("\t\t\t\t\t"+
- "Correlation between reconstruction cost and time was not "
- "defined in the input file. Using default values.")
-
- if ((depends is not None) and ('InjurySeverities' in depends.keys())):
- data['dependencies'].update({
- 'injury_lvls': bool(depends['InjurySeverities'])})
- elif data['decision_variables']['injuries']:
- data['dependencies'].update({'injury_lvls': False})
- log_msg("\t\t\t\t\t"+
- "Correlation between injury levels was not defined in the "
- "input file. Using default values.")
-
- # damage logic info --------------------------------------------------------
-
- data['damage_logic'] = dam_logic
-
- if verbose: pp.pprint(data)
-
- return data
-
-[docs]def read_SimCenter_EDP_input(input_path, EDP_kinds=('PID', 'PFA'),
- units = dict(PID=1., PFA=1.),
- verbose=False):
- """
- Read the EDP input information from a text file with a tabular structure.
-
- The SimCenter in the function name refers to having specific columns
- available in the file. Currently, the expected formatting follows the
- output formatting of Dakota that is applied for the dakotaTab.out. When
- using pelicun with the PBE Application, such a dakotaTab.out is
- automatically generated. The Input section of the documentation provides
- more information about the expected formatting of the EDP input file.
-
- Parameters
- ----------
- input_path: string
- Location of the EDP input file.
- EDP_kinds: tuple of strings, default: ('PID', 'PFA')
- Collection of the kinds of EDPs in the input file. The default pair of
- 'PID' and 'PFA' can be replaced or extended by any other EDPs.
- units: dict, default: {'PID':1., 'PFA':1}
- Defines the unit conversion that shall be applied to the EDP values.
- verbose: boolean
- If True, the function echoes the information read from the file. This
- can be useful to ensure that the information in the file is properly
- read by the method.
-
- Returns
- -------
- data: dict
- A dictionary with all the EDP data.
- """
-
- # initialize the data container
- data = {}
-
- # read the collection of EDP inputs...
- log_msg('\t\tOpening the input file...')
- # If the file name ends with csv, we assume a standard csv file
- if input_path.endswith('csv'):
- EDP_raw = pd.read_csv(input_path, header=0, index_col=0)
-
- # otherwise, we assume that a dakota file is provided...
- else:
- # the read_csv method in pandas is sufficiently versatile to handle the
- # tabular format of dakota
- EDP_raw = pd.read_csv(input_path, sep=r'\s+', header=0, index_col=0)
-
- # set the index to be zero-based
- if EDP_raw.index[0] == 1:
- EDP_raw.index = EDP_raw.index - 1
-
- # search the header for EDP information
- for column in EDP_raw.columns:
- for kind in EDP_kinds:
- if kind in column:
-
- if kind not in data.keys():
- data.update({kind: []})
-
- # extract info about the location, direction, and scenario
- info = column.split('-')
-
- # get the scale factor to perform unit conversion
- f_unit = units[kind]
-
- # store the data
- data[kind].append(dict(
- raw_data=(EDP_raw[column].values * f_unit).tolist(),
- location=info[2],
- direction=info[3],
- scenario_id=info[0]
- ))
-
- if verbose: pp.pprint(data)
-
- return data
-
-[docs]def read_population_distribution(path_POP, occupancy, assessment_type='P58',
- verbose=False):
- """
- Read the population distribution from an external json file.
-
- The population distribution is expected in a format used in FEMA P58, but
- the list of occupancy categories can be modified and/or extended beyond
- those available in that document. The population distributions for the
- occupancy categories from FEMA P58 and HAZUS MH are provided with pelicun
- in the population.json files in the corresponding folder under resources.
-
- Note: Population distributions in HAZUS do not have a 1:1 mapping to the
- occupancy types provided in the Technical Manual. We expect inputs to
- follow the naming convention in the HAZUS Technical Manual and convert
- those to the broader categories here automatically. During conversion, the
- following assumptions are made about the occupancy classes: i) RES classes
- are best described as Residential; ii) COM and REL as Commercial; iii) EDU
- as Educational; iv) IND and AGR as Industrial; v) Hotels do not have a
- matching occupancy class.
-
- Parameters
- ----------
- path_POP: string
- Location of the population distribution json file.
- occupancy: string
- Identifies the occupancy category.
- assessment_type: {'P58', 'HAZUS_EQ'}
- Tailors the warnings and verifications towards the type of assessment.
- default: 'P58'.
- verbose: boolean
- If True, the function echoes the information read from the file. This
- can be useful to ensure that the information in the file is properly
- read by the method.
-
- Returns
- -------
- data: dict
- A dictionary with the population distribution data.
- """
-
- HAZUS_occ_converter = {
- 'RES' : 'Residential',
- 'COM' : 'Commercial',
- 'REL' : 'Commercial',
- 'EDU' : 'Educational',
- 'IND' : 'Industrial',
- 'AGR' : 'Industrial'
- }
-
- AT = assessment_type
-
- # Convert the HAZUS occupancy classes to the categories used to define
- # population distribution.
- if AT == 'HAZUS_EQ':
- occupancy = HAZUS_occ_converter.get(occupancy[:3], None)
-
- if occupancy is None:
- warnings.warn(UserWarning(
- 'Unknown, probably invalid, occupancy class for HAZUS '
- 'assessment: {}. When defining population distribution, '
- 'assuming RES1 instead.'.format(occupancy)))
- occupancy = 'Residential'
-
- # Load the population data
-
- # If a json file is provided:
- if path_POP.endswith('json'):
- with open(path_POP, 'r') as f:
- jd = json.load(f)
-
- data = jd[occupancy]
-
- # else if an HDF5 file is provided
- elif path_POP.endswith('hdf'):
-
- # this for loop is needed to avoid issues from race conditions on HPC
- for i in range(1000):
- try:
- store = pd.HDFStore(path_POP)
- store.open()
-
- except HDF5ExtError:
- pop_table = None
- sleep(0.01)
- continue
-
- else:
- pop_table = store.select('pop',
- where=f'index in {[occupancy, ]}')
- store.close()
- break
-
- if pop_table is not None:
- data = convert_Series_to_dict(pop_table.loc[occupancy, :])
- else:
- raise IOError("Couldn't read the HDF file for POP data after 20 "
- "tries because it was blocked by other processes.")
-
- # convert peak population to persons/m2
- if 'peak' in data.keys():
- data['peak'] = data['peak'] / (1000. * ft2)
-
- if verbose: # pragma: no cover
- pp.pprint(data)
-
- return data
-
-
-[docs]def read_component_DL_data(path_CMP, comp_info, assessment_type='P58',
- verbose=False):
- """
- Read the damage and loss data for the components of the asset.
-
- DL data for each component is assumed to be stored in a JSON file following
- the DL file format specified by SimCenter. The name of the file is the ID
- (key) of the component in the comp_info dictionary. Besides the filename,
- the comp_info dictionary is also used to get other pieces of data about the
- component that is not available in the JSON files. Therefore, the following
- attributes need to be provided in the comp_info: ['quantities',
- 'csg_weights', 'dirs', 'kind', 'distribution', 'cov', 'unit', 'locations']
- Further information about these attributes is available in the Input
- section of the documentation.
-
- Parameters
- ----------
- path_CMP: string
- Location of the folder that contains the component data in JSON files.
- comp_info: dict
- Dictionary with additional information about the components.
- assessment_type: {'P58', 'HAZUS_EQ', 'HAZUS_HU'}
- Tailors the warnings and verifications towards the type of assessment.
- default: 'P58'.
- verbose: boolean
- If True, the function echoes the information read from the files. This
- can be useful to ensure that the information in the files is properly
- read by the method.
-
- Returns
- -------
- data: dict
- A dictionary with damage and loss data for each component.
-
- """
-
- AT = assessment_type
-
- data = dict([(c_id, dict([(key, None) for key in [
- 'ID',
- 'name',
- 'description',
- #'kind',
- 'demand_type',
- 'directional',
- 'correlation',
- 'offset',
- 'incomplete',
- 'locations',
- 'quantities',
- 'csg_weights',
- #'dir_weights',
- 'directions',
- 'distribution_kind',
- 'cov',
- 'unit',
- 'DSG_set',
- ]])) for c_id in comp_info.keys()])
-
- s_cmp_keys = sorted(data.keys())
- DL_data_dict = {}
-
- # If the path_CMP is a folder we assume it contains a set of json files
- if os.path.isdir(path_CMP):
-
- CMP_dir = Path(path_CMP).resolve()
-
- for c_id in s_cmp_keys:
- with open(CMP_dir / f'{c_id}.json', 'r') as f:
- DL_data_dict.update({c_id: json.load(f)})
-
- # else if an HDF5 file is provided we assume it contains the DL data
- elif path_CMP.endswith('hdf'):
-
- # this for loop is needed to avoid issues from race conditions on HPC
- for i in range(1000):
- try:
- store = pd.HDFStore(path_CMP)
- store.open()
-
- except HDF5ExtError:
- CMP_table = None
- sleep(0.1)
- continue
-
- else:
- CMP_table = store.select('data', where=f'index in {s_cmp_keys}')
- store.close()
- break
-
- if CMP_table is not None:
- for c_id in s_cmp_keys:
- DL_data_dict.update(
- {c_id: convert_Series_to_dict(CMP_table.loc[c_id, :])})
- else:
- raise IOError("Couldn't read the HDF file for DL data after 20 "
- "tries because it was blocked by other processes.")
-
- else:
- raise ValueError(
- "Component data source not recognized. Please provide "
- "either a folder with DL json files or an HDF5 table.")
-
- # for each component
- for c_id in s_cmp_keys:
- c_data = data[c_id]
-
- DL_data = DL_data_dict[c_id]
-
- DL_GI = DL_data['GeneralInformation']
- DL_EDP = DL_data['EDP']
- DL_DSG = DL_data['DSGroups']
-
- # First, check if the DL data is complete. Incomplete data can lead to
- # all kinds of problems, so in such a case we display a warning and do
- # not use the component. This can be relaxed later if someone creates a
- # method to replace unknown values with reasonable estimates.
- if 'Incomplete' in DL_GI.keys():
- c_data['incomplete'] = int(DL_GI['Incomplete'])
- if c_data['incomplete']:
- # show warning
- del data[c_id]
- warnings.warn(UserWarning(
- 'Fragility information for {} is incomplete. The component '
- 'cannot be used for loss assessment.'.format(c_id)))
- continue
-
- # Get the parameters from the BIM component info
- ci_data = comp_info[c_id]
- c_data['locations'] = ci_data['locations']
- c_data['directions'] = ci_data['directions']
-
- c_data['unit'] = globals()[ci_data['unit']]
- c_data['quantities'] = [(np.asarray(qnt) * c_data['unit']).tolist()
- for qnt in ci_data['quantities']]
- c_data['csg_weights'] = ci_data['csg_weights']
-
- c_data['distribution_kind'] = ci_data['distribution']
- c_data['cov'] = [float_or_None(cov) for cov in ci_data['cov']]
-
- # replace N/A distribution with normal and negligible cov
- #c_data['cov'] = [0.0001 if dk == 'N/A' else cov
- # for cov,dk in list(zip(c_data['cov'],
- # c_data['distribution_kind']))]
- #c_data['distribution_kind'] = ['normal' if dk == 'N/A' else dk
- # for dk in c_data['distribution_kind']]
- #c_data['cov'] = [0.0001 if cov==None else cov
- # for cov in c_data['cov']]
-
- #c_data['kind'] = ci_data['kind']
- #c_data['unit'] = ci_data['unit'][0] * globals()[ci_data['unit'][1]]
- #c_data['quantities'] = (np.asarray(ci_data['quantities']) * c_data[
- # 'unit']).tolist()
-
- # calculate the quantity weights in each direction
- #dirs = np.asarray(c_data['directions'], dtype=np.int)
- #u_dirs = np.unique(dirs)
- #weights = np.asarray(c_data['csg_weights'])
- #c_data['dir_weights'] = [sum(weights[np.where(dirs == d_i)])
- # for d_i in u_dirs]
-
- c_data['ID'] = c_id
- c_data['name'] = DL_data['Name']
- c_data['description'] = DL_GI['Description']
- c_data['offset'] =int(DL_EDP.get('Offset', 0))
- c_data['correlation'] = int(DL_data.get('Correlated', False))
- c_data['directional'] = int(DL_data.get('Directional', False))
-
- EDP_type = DL_EDP['Type']
- if DL_EDP['Unit'][1] == 'in':
- DL_EDP['Unit'][1] = 'inch'
- demand_factor = globals()[DL_EDP['Unit'][1]] * DL_EDP['Unit'][0]
- if EDP_type == 'Story Drift Ratio':
- demand_type = 'PID'
- elif EDP_type == 'Roof Drift Ratio':
- demand_type = 'PRD'
- elif EDP_type == 'Damageable Wall Drift':
- demand_type = 'DWD'
- elif EDP_type == 'Racking Drift Ratio':
- demand_type = 'RDR'
- elif EDP_type == 'Peak Floor Acceleration':
- demand_type = 'PFA'
- #demand_factor = g
- # PFA corresponds to the top of the given story. The ground floor
- # has an idex of 0. When damage of acceleration-sensitive components
- # is controlled by the acceleration of the bottom of the story, the
- # corresponding PFA location needs to be reduced by 1. Our framework
- # assumes that PFA corresponds to the bottom of the given story
- # by default, we need to subtract 1 from the location values. Rather
- # than changing the locations themselves, we assign an offset of -1
- # so that the results still get collected at the appropriate story.
- c_data['offset'] = c_data['offset'] - 1
- elif EDP_type == 'Peak Floor Velocity':
- demand_type = 'PFV'
- #demand_factor = mps
- c_data['offset'] = c_data['offset'] - 1
- elif EDP_type == 'Peak Gust Wind Speed':
- demand_type = 'PWS'
- #demand_factor = mph
- elif EDP_type == 'Peak Ground Acceleration':
- demand_type = 'PGA'
- #demand_factor = g
- elif EDP_type == 'Peak Ground Velocity':
- demand_type = 'PGV'
- #demand_factor = cmps
- elif EDP_type == 'Spectral Acceleration':
- demand_type = 'SA'
- #demand_factor = g
- elif EDP_type == 'Spectral Velocity':
- demand_type = 'SV'
- #demand_factor = mps
- elif EDP_type == 'Spectral Displacement':
- demand_type = 'SD'
- #demand_factor = m
- elif EDP_type == 'Permanent Ground Deformation':
- demand_type = 'PGD'
- elif EDP_type == 'Mega Drift Ratio':
- demand_type = 'PMD'
- elif EDP_type == 'Residual Drift Ratio':
- demand_type = 'RID'
- elif EDP_type in [
- 'Link Rotation Angle',
- 'Link Beam Chord Rotation']:
- demand_type = None
- warnings.warn(UserWarning(
- 'Component {} requires {} as EDP, which is not yet '
- 'implemented.'.format(c_data['ID'], EDP_type)))
- else: # pragma: no cover
- demand_type = None
- warnings.warn(UserWarning(
- f'Unexpected EDP type in component {c_id}: {EDP_type}'))
- if demand_type is None:
- del data[c_id]
- continue
- c_data['demand_type'] = demand_type
-
- # dictionary to convert DL data to internal representation
- curve_type = {'LogNormal': 'lognormal',
- 'Normal' : 'normal',
- 'N/A' : None}
- DS_set_kind = {'MutuallyExclusive' : 'mutually exclusive',
- 'mutually exclusive': 'mutually exclusive',
- 'simultaneous' : 'simultaneous',
- 'Simultaneous' : 'simultaneous',
- 'single' : 'single',
- 'Single' : 'single'}
-
- # load the damage state group information
- c_data['DSG_set'] = dict()
- QNT_unit = DL_data.get('QuantityUnit', [1, 'ea'])
- data_unit = QNT_unit[0] * globals()[QNT_unit[1]]
- for DSG_id, DSG_i in enumerate(DL_DSG):
- DSG_data = dict(
- theta=float(DSG_i['MedianEDP']) * demand_factor,
- sig=float(DSG_i['Beta']),
- DS_set_kind=DS_set_kind[DSG_i['DSGroupType']],
- distribution_kind = curve_type[DSG_i['CurveType']],
- DS_set={}
- )
- # sig needs to be scaled for normal distributions
- if DSG_data['distribution_kind'] == 'normal':
- DSG_data['sig'] = DSG_data['sig'] * demand_factor
-
- for DS_id, DS_i in enumerate(DSG_i['DamageStates']):
- DS_data = {'description': DS_i['Description'],
- 'weight' : DS_i['Weight']}
-
- DS_C = DS_i.get('Consequences', None)
- if DS_C is not None:
- if 'ReconstructionCost' in DS_C.keys():
- DS_CC = DS_C['ReconstructionCost']
- if isinstance(DS_CC['Amount'], list):
- DS_data.update({'repair_cost': {
- 'medians' : np.array([float(a) for a in DS_CC['Amount']]),
- 'quantities' : np.array(DS_CC['Quantity']),
- 'distribution_kind': curve_type[DS_CC.get('CurveType','N/A')],
- 'cov' : DS_CC.get('Beta',None),
- }})
-
- # convert the quantity units to standard ones
- DS_data['repair_cost']['quantities'] *= data_unit
- DS_data['repair_cost']['quantities'] = DS_data['repair_cost']['quantities'].tolist()
- else:
- DS_data.update({'repair_cost': {
- 'medians': np.array([float(DS_CC['Amount']),]),
- 'distribution_kind': curve_type[DS_CC.get('CurveType','N/A')],
- 'cov' : DS_CC.get('Beta',None),
- }})
-
- # convert the median units to standard ones
- DS_data['repair_cost']['medians'] /= data_unit
- DS_data['repair_cost']['medians'] = DS_data['repair_cost']['medians'].tolist()
-
- if 'ReconstructionTime' in DS_C.keys():
- DS_CT = DS_C['ReconstructionTime']
- if isinstance(DS_CT['Amount'], list):
- DS_data.update({'repair_time': {
- 'medians' : np.array([float(a) for a in DS_CT['Amount']]),
- 'quantities' : np.array(DS_CT['Quantity']),
- 'distribution_kind': curve_type[DS_CT.get('CurveType','N/A')],
- 'cov' : DS_CT.get('Beta',None),
- }})
-
- # convert the quantity units to standard ones
- DS_data['repair_time']['quantities'] *= data_unit
- DS_data['repair_time']['quantities'] = DS_data['repair_time']['quantities'].tolist()
- else:
- DS_data.update({'repair_time': {
- 'medians': np.array([float(DS_CT['Amount']),]),
- 'distribution_kind': curve_type[DS_CT.get('CurveType','N/A')],
- 'cov' : DS_CT.get('Beta',None),
- }})
-
- # convert the median units to standard ones
- DS_data['repair_time']['medians'] /= data_unit
- DS_data['repair_time']['medians'] = DS_data['repair_time']['medians'].tolist()
-
- if 'RedTag' in DS_C.keys():
- DS_CR = DS_C['RedTag']
- DS_data.update({'red_tag': {
- 'theta': DS_CR['Amount'],
- # 'distribution_kind': curve_type[DS_CR['CurveType']],
- 'cov' : DS_CR['Beta'],
- }})
-
- if 'Injuries' in DS_C.keys():
- DS_CI = DS_C['Injuries']
- if DS_CI[0].get('Beta') is not None:
- DS_data.update({'injuries': {
- 'theta': [float(I_i['Amount']) for I_i in DS_CI],
- # 'distribution_kind': curve_type[DS_CR['CurveType']],
- 'cov' : [I_i['Beta'] for I_i in DS_CI],
- }})
- else:
- DS_data.update({
- 'injuries': [I_i['Amount'] for I_i in DS_CI]})
-
- # if there is a chance of injuries, load the affected floor area
- affected_area, unit = DS_i.get('AffectedArea',
- [0.0, 'SF'])
- if unit == 'SF':
- affected_area = affected_area * SF
- else: # pragma: no cover
- warnings.warn(UserWarning(
- 'Unknown unit for affected floor area: {}'.format(
- unit)))
- affected_area = 0.
- DS_data.update({'affected_area': affected_area})
-
- # convert the units to standard ones
- DS_data['affected_area'] /= data_unit
-
- DSG_data['DS_set'].update({'DS-' + str(DS_id + 1): DS_data})
-
- c_data['DSG_set'].update({'DSG-' + str(DSG_id + 1): DSG_data})
-
- if verbose: # pragma: no cover
- for c_id, c_data in data.items():
- print(c_id)
- pp.pprint(c_data)
-
- return data
-
-def write_SimCenter_DL_output(output_dir, output_filename, output_df, index_name='#Num',
- collapse_columns = True, stats_only=False):
-
- # if the summary flag is set, then not all realizations are returned, but
- # only the first two moments and the empirical CDF through 100 percentiles
- if stats_only:
- #output_df = output_df.describe(np.arange(1, 100)/100.)
- #output_df = output_df.describe([0.1,0.5,0.9])
- if len(output_df.columns) > 0:
- output_df = describe(output_df)
- else:
- output_df = describe(np.zeros(len(output_df.index)))
- else:
- output_df = output_df.copy()
-
- # the name of the index column is replaced with the provided value
- output_df.index.name = index_name
-
- # multiple levels of indices are collapsed into a single level if needed
- # TODO: check for the number of levels and prepare a smarter collapse method
- if collapse_columns:
- output_df.columns = [('{}/{}'.format(s0, s1)).replace(' ', '_')
- for s0, s1 in zip(output_df.columns.get_level_values(0),
- output_df.columns.get_level_values(1))]
-
- # write the results in a csv file
- # TODO: provide other file formats
- log_msg('\t\t\tSaving file {}'.format(output_filename))
- file_path = posixpath.join(output_dir, output_filename)
- output_df.to_csv(file_path)
- # TODO: this requires pandas 1.0+ > wait until next release
- #with open(file_path[:-3]+'zip', 'w') as f:
- # output_df.to_csv(f, compression=dict(mehtod='zip', archive_name=output_filename))
-
-def write_SimCenter_EDP_output(output_dir, EDP_filename, EDP_df):
-
- # initialize the output DF
- col_info = np.transpose([col.split('-')[1:] for col in EDP_df.columns])
-
- EDP_types = np.unique(col_info[0])
- EDP_locs = np.unique(col_info[1])
- EDP_dirs = np.unique(col_info[2])
-
- MI = pd.MultiIndex.from_product(
- [EDP_types, EDP_locs, EDP_dirs, ['median', 'beta']],
- names=['type', 'loc', 'dir', 'stat'])
-
- df_res = pd.DataFrame(columns=MI, index=[0, ])
- if ('PID', '0') in df_res.columns:
- del df_res[('PID', '0')]
-
- # store the EDP statistics in the output DF
- for col in np.transpose(col_info):
- df_res.loc[0, (col[0], col[1], col[2], 'median')] = EDP_df[
- '1-{}-{}-{}'.format(col[0], col[1], col[2])].median()
- df_res.loc[0, (col[0], col[1], col[2], 'beta')] = np.log(
- EDP_df['1-{}-{}-{}'.format(col[0], col[1], col[2])]).std()
-
- df_res.dropna(axis=1, how='all', inplace=True)
-
- df_res = df_res.astype(float) #.round(4)
-
- # save the output
- df_res.to_csv('EDP.csv')
-
-def write_SimCenter_DM_output(output_dir, DM_filename, SUMMARY_df, DMG_df):
-
- # first, get the collapses from the SUMMARY_df
- df_res_c = pd.DataFrame([0,],
- columns=pd.MultiIndex.from_tuples([('probability',' '),]),
- index=[0, ])
- df_res_c['probability'] = SUMMARY_df[('collapses', 'collapsed')].mean()
-
- # aggregate the damage data along Performance Groups
- DMG_agg = DMG_df.groupby(level=['FG', 'DSG_DS'], axis=1).sum()
-
- comp_types = []
- FG_list = [c for c in DMG_agg.columns.get_level_values('FG').unique()]
- for comp_type in ['S', 'NS', 'NSA', 'NSD']:
- if np.sum([fg.startswith(comp_type) for fg in FG_list]) > 0:
- comp_types.append(comp_type)
- if np.sum([np.any([fg.startswith(comp_type) for comp_type in comp_types])
- for fg in FG_list]) != len(FG_list):
- comp_types.append('other')
-
- # second, get the damage state likelihoods
- df_res_l = pd.DataFrame(
- columns=pd.MultiIndex.from_product([comp_types,
- ['0', '1_1', '2_1', '3_1', '4_1', '4_2']],
- names=['comp_type', 'DSG_DS']),
- index=[0, ])
-
- # third, get the damage quantities conditioned on damage state
- # we do not do that for now
- # df_res_q = pd.DataFrame(
- # columns=pd.MultiIndex.from_product([comp_types,
- # ['1_1', '2_1', '3_1', '4_1', '4_2']],
- # names=['comp_type', 'DSG_DS']),
- # index=[0, ])
-
- for comp_type in ['NSA', 'NSD', 'NS', 'other']:
- if comp_type in comp_types:
- del df_res_l[(comp_type, '4_2')]
- # del df_res_q[(comp_type, '4_2')]
-
- # for each type of component...
- for comp_type in comp_types:
-
- # select the corresponding subset of columns
- if comp_type == 'other':
- type_cols = [fg for fg in FG_list
- if np.all([~fg.startswith(comp_type) for comp_type in comp_types])]
- else:
- type_cols = [c for c in DMG_agg.columns.get_level_values('FG').unique()
- if c.startswith(comp_type)]
-
- df_sel = DMG_agg.loc[:, type_cols].groupby(level='DSG_DS',axis=1).sum()
- df_sel = df_sel / len(type_cols)
-
- # calculate the probability of DSG exceedance
- df_sel[df_sel > 0.0] = df_sel[df_sel > 0.0] / df_sel[df_sel > 0.0]
-
- cols = df_sel.columns
- for i in range(len(cols)):
- filter = np.where(df_sel.iloc[:, i].values > 0.0)[0]
- df_sel.iloc[filter, idx[0:i]] = 1.0
-
- df_sel_exc = pd.Series(np.mean(df_sel.values, axis=0),
- index=df_sel.columns)
-
- DS_0 = 1.0 - df_sel_exc['1_1']
- for i in range(len(df_sel_exc.index) - 1):
- df_sel_exc.iloc[i] = df_sel_exc.iloc[i] - df_sel_exc.iloc[i + 1]
-
- # Add the probability of no damage for convenience.
- df_sel_exc.loc['0'] = DS_0
- df_sel_exc = df_sel_exc.sort_index()
-
- # store the results in the output DF
- for dsg_i in df_sel_exc.index:
- if df_sel_exc[dsg_i] > 0.0:
- df_res_l.loc[:, idx[comp_type, dsg_i]] = df_sel_exc[dsg_i]
-
- # get the quantity of components in the highest damage state
- # skip this part for now to reduce file size
- if False:
- df_init = DMG_agg.loc[:, type_cols].groupby(level='DSG_DS', axis=1).sum()
- df_init = (df_init / len(type_cols)).round(2)
-
- df_sel = df_sel.sum(axis=1)
-
- for lvl, lvl_label in zip([1.0, 2.0, 3.0, 4.0, 5.0],
- ['1_1', '2_1', '3_1', '4_1', '4_2']):
-
- df_cond = df_init[df_sel == lvl]
-
- if df_cond.size > 0:
- unique_vals, unique_counts = np.unique(
- df_cond[lvl_label].values, return_counts=True)
- unique_counts = np.around(unique_counts / df_cond.shape[0],
- decimals=4)
- sorter = np.argsort(unique_counts)[::-1][:4]
- DQ = list(zip(unique_vals[sorter], unique_counts[sorter]))
-
- # store the damaged quantities in the output df
- df_res_q.loc[:,idx[comp_type, lvl_label]] = str(DQ)
-
- # join the output dataframes
- #df_res = pd.concat([df_res_c, df_res_l, df_res_q], axis=1,
- # keys=['Collapse','DS likelihood','Damaged Quantities'])
- df_res = pd.concat([df_res_c, df_res_l], axis=1, keys=['Collapse','DS likelihood'])
-
- # save the output
- with open(posixpath.join(output_dir, DM_filename), 'w') as f:
- df_res.to_csv(f)
-
-def write_SimCenter_DM_output_old(output_dir, DM_filename, DMG_df):
-
- # Start with the probability of being in a particular damage state.
- # Here, the damage state of the building (asset) is defined as the highest
- # damage state among the building components/component groups. This works
- # well for a HAZUS assessment, but something more sophisticated is needed
- # for a FEMA P58 assessment.
-
- # Determine the probability of DS exceedance by collecting the DS from all
- # components and assigning ones to all lower damage states.
- DMG_agg = DMG_df.T.groupby('DSG_DS').sum().T
- DMG_agg[DMG_agg > 0.0] = DMG_agg[DMG_agg > 0.0] / DMG_agg[DMG_agg > 0.0]
-
- cols = DMG_agg.columns
- for i in range(len(cols)):
- filter = np.where(DMG_agg.iloc[:,i].values > 0.0)[0]
- DMG_agg.iloc[filter,idx[0:i]] = 1.0
-
- # The P(DS=ds) probability is determined by subtracting consecutive DS
- # exceedance probabilites. This will not work well for a FEMA P58 assessment
- # with Damage State Groups that include multiple Damage States.
- #DMG_agg_mean = DMG_agg.describe().loc['mean',:]
- DMG_agg_mean = pd.Series(np.mean(DMG_agg.values, axis=0), index=DMG_agg.columns)
-
- DS_0 = 1.0 - DMG_agg_mean['1_1']
- for i in range(len(DMG_agg_mean.index)-1):
- DMG_agg_mean.iloc[i] = DMG_agg_mean.iloc[i] - DMG_agg_mean.iloc[i+1]
-
- # Add the probability of no damage for convenience.
- DMG_agg_mean['0'] = DS_0
- DMG_agg_mean = DMG_agg_mean.sort_index()
-
- # Save the results in the output json file
- DM = {'aggregate': {}}
-
- for id in DMG_agg_mean.index:
- DM['aggregate'].update({str(id): DMG_agg_mean[id]})
-
- # Now determine the probability of being in a damage state for individual
- # components / component assemblies...
- #DMG_mean = DMG_df.describe().loc['mean',:]
- DMG_mean = pd.Series(np.mean(DMG_df.values, axis=0), index=DMG_df.columns)
-
- # and save the results in the output json file.
- for FG in sorted(DMG_mean.index.get_level_values('FG').unique()):
- DM.update({str(FG):{}})
-
- for PG in sorted(
- DMG_mean.loc[idx[FG],:].index.get_level_values('PG').unique()):
- DM[str(FG)].update({str(PG):{}})
-
- for DS in sorted(
- DMG_mean.loc[idx[FG],:].loc[idx[:,PG],:].index.get_level_values('DSG_DS').unique()):
- DM[str(FG)][str(PG)].update({str(DS): DMG_mean.loc[(FG,PG,DS)]})
-
- log_msg('\t\t\tSaving file {}'.format(DM_filename))
- with open(posixpath.join(output_dir, DM_filename), 'w') as f:
- json.dump(DM, f, indent = 2)
-
-def write_SimCenter_DV_output(output_dir, DV_filename, GI, SUMMARY_df, DV_dict):
-
- DV_cost = None
- DV_time = None
- DV_inj = [None,]*4
-
- for DV_name, DV_mod in DV_dict.items():
- if 'rec_cost' in DV_name:
- DV_cost = DV_mod
- elif 'rec_time' in DV_name:
- DV_time = DV_mod
- elif 'injuries_1' in DV_name:
- DV_inj[0] = DV_mod
- elif 'injuries_2' in DV_name:
- DV_inj[1] = DV_mod
- elif 'injuries_3' in DV_name:
- DV_inj[2] = DV_mod
- elif 'injuries_4' in DV_name:
- DV_inj[3] = DV_mod
-
- DVs = SUMMARY_df.columns.get_level_values(1)
-
- if DV_cost is not None:
-
- comp_types = []
- FG_list = [c for c in DV_cost.columns.get_level_values('FG').unique()]
- for comp_type in ['S', 'NS', 'NSA', 'NSD']:
- if np.sum([fg.startswith(comp_type) for fg in FG_list]) > 0:
- comp_types.append(comp_type)
-
- repl_cost = GI['replacement_cost']
-
- headers = [['Repair Cost',],
- ['aggregate',],
- [' ',],
- ['mean','std','10%','median','90%']]
-
- MI = pd.MultiIndex.from_product(headers,
- names=['DV', 'comp_type', 'DSG_DS', 'stat'])
-
- df_res_Cagg = pd.DataFrame(columns=MI, index=[0, ])
- df_res_Cagg.fillna(0, inplace=True)
-
- headers = [['Repair Impractical',],
- ['probability',],
- [' ',],
- [' ',]]
-
- MI = pd.MultiIndex.from_product(headers,
- names=['DV', 'comp_type', 'DSG_DS', 'stat'])
-
- df_res_Cimp = pd.DataFrame(columns=MI, index=[0, ])
- df_res_Cimp[('Repair Impractical', 'probability')] = SUMMARY_df[('reconstruction', 'cost impractical')].mean()
- df_res_Cimp = df_res_Cimp.astype(float)
-
- headers = [['Repair Cost',],
- comp_types,
- ['aggregate','1_1', '2_1', '3_1', '4_1', '4_2'],
- ['mean',]]
-
- MI = pd.MultiIndex.from_product(headers,
- names=['DV', 'comp_type', 'DSG_DS', 'stat'])
-
- df_res_C = pd.DataFrame(columns=MI, index=[0, ])
-
- for comp_type in ['NSA', 'NSD', 'NS']:
- if comp_type in comp_types:
- del df_res_C[('Repair Cost', comp_type, '4_2')]
-
- if DV_time is not None:
-
- repl_time = GI['replacement_time']
-
- headers = [['Repair Time',],
- [' ',],
- ['aggregate',],
- ['mean','std','10%','median','90%']]
-
- MI = pd.MultiIndex.from_product(headers,
- names=['DV', 'comp_type', 'DSG_DS', 'stat'])
-
- df_res_Tagg = pd.DataFrame(columns=MI, index=[0, ])
- df_res_Tagg.fillna(0, inplace=True)
-
- if DV_inj[0] is not None:
-
- lvls = []
- [lvls.append(f'sev{i+1}') for i in range(4) if DV_inj[i] is not None]
-
- headers = [['Injuries',],
- lvls,
- ['aggregate',],
- ['mean','std','10%','median','90%']]
-
- MI = pd.MultiIndex.from_product(headers,
- names=['DV', 'comp_type', 'DSG_DS', 'stat'])
-
- df_res_Iagg = pd.DataFrame(columns=MI, index=[0, ])
- df_res_Iagg.fillna(0, inplace=True)
-
- dfs_to_join = []
-
- # start with the disaggregated costs...
- if DV_cost is not None:
- for type_ID in comp_types:
-
- DV_res = DV_cost.groupby(level=['FG', 'DSG_DS'], axis=1).sum()
-
- type_cols = [c for c in DV_res.columns.get_level_values('FG').unique() if c.startswith(type_ID)]
-
- df_cost = DV_res.loc[:, type_cols].groupby(level='DSG_DS',axis=1).sum()
-
- # create a df with 1s at cells with damage and identify the governing DS
- df_sel = df_cost.copy()
- df_sel[df_sel>0.0] = df_sel[df_sel>0.0] / df_sel[df_sel>0.0]
-
- cols = df_sel.columns
- for i in range(len(cols)):
- filter = np.where(df_sel.iloc[:,i].values > 0.0)[0]
- df_sel.iloc[filter,idx[0:i]] = 1.0
- df_sel = df_sel.sum(axis=1)
-
- if type_ID == 'S':
- ds_list = ['1_1', '2_1', '3_1', '4_1', '4_2']
- else:
- ds_list = ['1_1', '2_1', '3_1', '4_1']
-
- # store the results in the output DF
- df_cost = df_cost.sum(axis=1)
- df_cost.loc[:] = np.minimum(df_cost.values, repl_cost)
- mean_costs = [df_cost.loc[df_sel == dsg_i+1].mean() for dsg_i, dsg in enumerate(ds_list)]
-
- df_res_C.loc[:, idx['Repair Cost', type_ID, ds_list, 'mean']] = mean_costs
- df_res_C.loc[:, idx['Repair Cost', type_ID, 'aggregate', 'mean']] = df_cost.mean()
-
- df_res_C = df_res_C.astype(float) #.round(0)
-
- # now store the aggregate results for cost
- DV_res = describe(SUMMARY_df[('reconstruction','cost')])
-
- df_res_Cagg.loc[:, idx['Repair Cost', 'aggregate', ' ', ['mean', 'std','10%','median','90%']]] = DV_res[['mean', 'std','10%','50%','90%']].values
-
- df_res_Cagg = df_res_Cagg.astype(float) #.round(0)
- dfs_to_join = dfs_to_join + [df_res_Cagg, df_res_Cimp, df_res_C]
-
- if DV_time is not None:
- DV_res = describe(SUMMARY_df[('reconstruction','time')])
-
- df_res_Tagg.loc[:, idx['Repair Time', ' ', 'aggregate', ['mean', 'std','10%','median','90%']]] = DV_res[['mean', 'std','10%','50%','90%']].values
-
- df_res_Tagg = df_res_Tagg.astype(float) #.round(1)
- dfs_to_join.append(df_res_Tagg)
-
- if DV_inj[0] is not None:
- for i in range(4):
- if DV_inj[i] is not None:
- DV_res = describe(SUMMARY_df[('injuries',f'sev{i+1}')])
-
- df_res_Iagg.loc[:, idx['Injuries', f'sev{i+1}', 'aggregate', ['mean', 'std','10%','median','90%']]] = DV_res[['mean', 'std','10%','50%','90%']].values
-
- df_res_Iagg = df_res_Iagg.astype(float) #.round(6)
-
- dfs_to_join.append(df_res_Iagg)
-
- df_res = pd.concat(dfs_to_join,axis=1)
-
- # save the output
- with open(posixpath.join(output_dir, DV_filename), 'w') as f:
- df_res.to_csv(f)
-
-def write_SimCenter_DV_output_old(output_dir, DV_filename, DV_df, DV_name):
-
- DV_name = convert_dv_name[DV_name]
-
- DV_file_path = posixpath.join(output_dir, DV_filename)
-
- try:
- with open(DV_file_path, 'r') as f:
- DV = json.load(f)
- except:
- DV = {}
-
- DV.update({DV_name: {}})
-
- DV_i = DV[DV_name]
-
- try:
- #if True:
- #DV_tot = DV_df.sum(axis=1).describe([0.1,0.5,0.9]).drop('count')
- DV_tot = describe(np.sum(DV_df.values, axis=1))
- DV_i.update({'total':{}})
- for stat in DV_tot.index:
- DV_i['total'].update({stat: DV_tot.loc[stat]})
-
- #DV_stats = DV_df.describe([0.1,0.5,0.9]).drop('count')
- DV_stats = describe(DV_df)
- for FG in sorted(DV_stats.columns.get_level_values('FG').unique()):
- DV_i.update({str(FG):{}})
-
- for PG in sorted(
- DV_stats.loc[:,idx[FG]].columns.get_level_values('PG').unique()):
- DV_i[str(FG)].update({str(PG):{}})
-
- for DS in sorted(
- DV_stats.loc[:,idx[FG, PG]].columns.get_level_values('DSG_DS').unique()):
- DV_i[str(FG)][str(PG)].update({str(DS): {}})
- DV_stats_i = DV_stats.loc[:,(FG,PG,DS)]
- for stat in DV_stats_i.index:
- DV_i[str(FG)][str(PG)][str(DS)].update({
- stat: DV_stats_i.loc[stat]})
- except:
- pass
-
- log_msg('\t\t\tSaving file {}'.format(DV_filename))
- with open(DV_file_path, 'w') as f:
- json.dump(DV, f, indent = 2)
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-This module has classes and methods that define and access the model used for
-loss assessment.
-
-.. rubric:: Contents
-
-.. autosummary::
-
- prep_constant_median_DV
- prep_bounded_linear_median_DV
- prep_bounded_multilinear_median_DV
-
- FragilityFunction
- ConsequenceFunction
- DamageState
- DamageStateGroup
- PerformanceGroup
- FragilityGroup
-
-"""
-
-from .base import *
-
-[docs]class FragilityFunction(object):
- """
- Describes the relationship between asset response and damage.
-
- Asset response is characterized by a Demand value that represents an
- engineering demand parameter (EDP). Only a scalar EDP is supported
- currently. The damage is characterized by a set of DamageStateGroup (DSG)
- objects. For each DSG, the corresponding EDP limit (i.e. the EDP at which
- the asset is assumed to experience damage described by the DSG) is
- considered uncertain; hence, it is described by a random variable. The
- random variables that describe EDP limits for the set of DSGs are not
- necessarily independent.
-
- We assume that the EDP limit will be approximated by a probability
- distribution for each DSG and these variables together form a multivariate
- distribution. Following common practice, the correlation between
- variables is assumed perfect by default, but the framework allows the
- users to explore other, more realistic options.
-
- Parameters
- ----------
- EDP_limit: list of RandomVariable
- A list of correlated random variables where each variable corresponds
- to an EDP limit that triggers a damage state. The number of
- list elements shall be equal to the number of DSGs handled by the
- Fragility Function (FF) and they shall be in ascending order of damage
- severity.
-
- """
-
- def __init__(self, EDP_limit):
- self._EDP_limit = EDP_limit
-
- self._EDP_tags = [EDP_lim_i.name for EDP_lim_i in EDP_limit]
-
-[docs] def P_exc(self, EDP, DSG_ID):
- """
- Return the probability of damage state exceedance.
-
- Calculate the probability of exceeding the damage corresponding to the
- DSG identified by the DSG_ID conditioned on a particular EDP value.
-
- Parameters
- ----------
- EDP: float scalar or ndarray
- Single EDP or numpy array of EDP values.
- DSG_ID: int
- Identifies the conditioning DSG. The DSG numbering is 1-based,
- because zero typically corresponds to the undamaged state.
-
- Returns
- -------
- P_exc: float scalar or ndarray
- DSG exceedance probability at the given EDP point(s).
- """
-
- EDP = np.asarray(EDP, dtype=np.float64)
- nvals = EDP.size
-
- # The exceedance probability corresponding to no damage is 1.
- # Although this is trivial, returning 1 for DSG_ID=0 makes using P_exc
- # more convenient.
- if DSG_ID == 0:
- P_exc = np.ones(EDP.size)
- else:
- # prepare the limits for the density calculation
- ndims = len(self._EDP_limit)
-
- limit_list = np.full((ndims, nvals), -np.inf, dtype=np.float64)
- limit_list[DSG_ID - 1:] = EDP
- limit_list[:DSG_ID - 1] = None
-
- P_exc = 1.0 - self._EDP_limit[0].RV_set.orthotope_density(
- lower=limit_list, var_subset = self._EDP_tags)
-
- # if EDP was a scalar, make sure that the result is also a scalar
- if EDP.size == 1:
- return P_exc[0]
- else:
- return P_exc
-
-[docs] def DSG_given_EDP(self, EDP, force_resampling=False):
- """
- Given an EDP, get a damage level based on the fragility function.
-
- The damage is evaluated by sampling the joint distribution of
- fragilities corresponding to all possible damage levels and checking
- which damage level the given EDP falls into. This approach allows for
- efficient damage state evaluation for a large number of EDP
- realizations.
-
- Parameters
- ----------
- EDP: float scalar or ndarray or Series
- Single EDP, or numpy array or pandas Series of EDP values.
- force_resampling: bool, optional, default: False
- If True, the probability distribution is resampled before
- evaluating the damage for each EDP. This is not recommended if the
- fragility functions are correlated with other sources of
- uncertainty because those variables will also be resampled in this
- case. If False, which is the default approach, we assume that
- the random variable has already been sampled and the number of
- samples greater or equal to the number of EDP values.
-
- Returns
- -------
- DSG_ID: Series
- Identifies the damage that corresponds to the given EDP. A DSG_ID
- of 0 means no damage.
-
- """
- # get the number of samples needed
- nsamples = EDP.size
-
- # if there are no samples or resampling is forced, then sample the
- # distribution first
- # TODO: force_resampling is probably not needed
- # if force_resampling or (self._EDP_limit.samples is None):
- # self._EDP_limit.sample_distribution(sample_size=nsamples)
- #
- # # if the number of samples is not sufficiently large, raise an error
- # if self._EDP_limit.samples.shape[0] < nsamples:
- # raise ValueError(
- # 'Damage evaluation requires at least as many samples of the '
- # 'joint distribution defined by the fragility functions as '
- # 'many EDP values are provided to the DSG_given_EDP function. '
- # 'You might want to consider setting force_resampling to True '
- # 'or sampling the distribution before calling the DSG_given_EDP '
- # 'function.')
-
- #samples = pd.DataFrame(self._EDP_limit.samples)
- samples = pd.DataFrame(dict([(lim_i.name, lim_i.samples)
- for lim_i in self._EDP_limit]))
-
- if type(EDP) not in [pd.Series, pd.DataFrame]:
- EDP = pd.Series(EDP, name='EDP')
-
- #nstates = samples.columns.values.size
- nstates = samples.shape[1]
-
- samples = samples.loc[EDP.index,:]
-
- # sort columns
- sample_cols = samples.columns
- samples = samples[sample_cols[np.argsort(sample_cols)]]
-
- # check for EDP exceedance
- EXC = samples.sub(EDP, axis=0) < 0.
-
- DSG_ID = pd.Series(np.zeros(len(samples.index)), name='DSG_ID',
- index=samples.index, dtype=np.int)
-
- for s in range(nstates):
- DSG_ID[EXC.iloc[:,s]] = s + 1
-
- return DSG_ID
-
-[docs]def prep_constant_median_DV(median):
- """
- Returns a constant median Decision Variable (DV) function.
-
- Parameters
- ----------
- median: float
- The median DV for a consequence function with fixed median.
-
- Returns
- -------
- f: callable
- A function that returns the constant median DV for all component
- quantities.
- """
- def f(quantity):
- return median
-
- return f
-
-[docs]def prep_bounded_linear_median_DV(median_max, median_min, quantity_lower,
- quantity_upper):
- """
- Returns a bounded linear median Decision Variable (DV) function.
-
- The median DV equals the min and max values when the quantity is
- outside of the prescribed quantity bounds. When the quantity is within the
- bounds, the returned median is calculated by a linear function with a
- negative slope between max and min values.
-
- Parameters
- ----------
- median_max: float, optional
- median_min: float, optional
- Minimum and maximum limits that define the bounded_linear median DV
- function.
- quantity_lower: float, optional
- quantity_upper: float, optional
- Lower and upper bounds of component quantity that define the
- bounded_linear median DV function.
-
- Returns
- -------
- f: callable
- A function that returns the median DV given the quantity of damaged
- components.
- """
- def f(quantity):
- if quantity is None:
- raise ValueError(
- 'A bounded linear median Decision Variable function called '
- 'without specifying the quantity of damaged components')
-
- q_array = np.asarray(quantity, dtype=np.float64)
-
- # calculate the median consequence given the quantity of damaged
- # components
- output = np.interp(q_array,
- [quantity_lower, quantity_upper],
- [median_max, median_min])
-
- return output
-
- return f
-
-[docs]def prep_bounded_multilinear_median_DV(medians, quantities):
- """
- Returns a bounded multilinear median Decision Variable (DV) function.
-
- The median DV equals the min and max values when the quantity is
- outside of the prescribed quantity bounds. When the quantity is within the
- bounds, the returned median is calculated by linear interpolation.
-
- Parameters
- ----------
- medians: ndarray
- Series of values that define the y coordinates of the multilinear DV
- function.
- quantities: ndarray
- Series of values that define the component quantities corresponding to
- the series of medians and serving as the x coordinates of the
- multilinear DV function.
-
- Returns
- -------
- f: callable
- A function that returns the median DV given the quantity of damaged
- components.
- """
- def f(quantity):
- if quantity is None:
- raise ValueError(
- 'A bounded linear median Decision Variable function called '
- 'without specifying the quantity of damaged components')
-
- q_array = np.asarray(quantity, dtype=np.float64)
-
- # calculate the median consequence given the quantity of damaged
- # components
- output = np.interp(q_array, quantities, medians)
-
- return output
-
- return f
-
-[docs]class ConsequenceFunction(object):
- """
- Describes the relationship between damage and a decision variable.
-
- Indicates the distribution of a quantified Decision Variable (DV)
- conditioned on a component, an element, or the system reaching a given
- damage state (DS). DV can be reconstruction cost, repair time, casualties,
- injuries, etc. Its distribution might depend on the quantity of damaged
- components.
-
- Parameters
- ----------
-
- DV_median: callable
- Describes the median DV as an f(quantity) function of the total
- quantity of damaged components. Use the prep_constant_median_DV, and
- prep_bounded_linear_median_DV helper functions to conveniently
- prescribe the typical FEMA P-58 functions.
- DV_distribution: RandomVariable
- A random variable that characterizes the uncertainty in the DV. The
- distribution shall be normalized by the median DV (i.e. the RV is
- expected to have a unit median). Truncation can be used to
- prescribe lower and upper limits for the DV, such as the (0,1) domain
- needed for red tag evaluation.
-
- """
-
- def __init__(self, DV_median, DV_distribution):
-
- self._DV_median = DV_median
- self._DV_distribution = DV_distribution
-
-[docs] def median(self, quantity=None):
- """
- Return the value of the median DV.
-
- The median DV corresponds to the component damage state (DS). If the
- damage consequence depends on the quantity of damaged components, the
- total quantity of damaged components shall be specified through the
- quantity parameter.
-
- Parameters
- ----------
- quantity: float scalar or ndarray, optional
- Total quantity of damaged components that determines the magnitude
- of median DV. Not needed for consequence functions with a fixed
- median DV.
-
- Returns
- -------
- median: float scalar or ndarray
- A single scalar for fixed median; a scalar or an array depending on
- the shape of the quantity parameter for bounded_linear median.
-
- """
- return self._DV_median(quantity)
-
-[docs] def sample_unit_DV(self, quantity=None, sample_size=1,
- force_resampling=False):
- """
- Sample the decision variable quantity per component unit.
-
- The Unit Decision Variable (UDV) corresponds to the component Damage
- State (DS). It shall be multiplied by the quantity of damaged
- components to get the total DV that corresponds to the quantity of the
- damaged components in the asset. If the DV depends on the total
- quantity of damaged components, that value shall be specified through
- the quantity parameter.
-
- Parameters
- ----------
- quantity: float scalar, ndarray or Series, optional, default: None
- Total quantity of damaged components that determines the magnitude
- of median DV. Not needed for consequence functions with a fixed
- median DV.
- sample_size: int, optional, default: 1
- Number of samples drawn from the DV distribution. The default value
- yields one sample. If quantity is an array with more than one
- element, the sample_size parameter is ignored.
- force_resampling: bool, optional, default: False
- If True, the DV distribution (and the corresponding RV if there
- are correlations) is resampled even if there are samples already
- available. This is not recommended if the DV distribution is
- correlated with other sources of uncertainty because those
- variables will also be resampled in this case. If False, which is
- the default approach, we assume that the random variable has
- already been sampled and the number of samples is greater or equal
- to the number of samples requested.
-
- Returns
- -------
- unit_DV: float scalar or ndarray
- Unit DV samples.
-
- """
- # get the median DV conditioned on the provided quantities
- median = self.median(quantity=np.asarray(quantity))
-
- # if the distribution is None, then there is no uncertainty in the DV
- # and the median values are the samples
- if self._DV_distribution is None:
- return median
-
- else:
- # if there are more than one quantities defined, infer the number of
- # samples from the number of quantities
- if quantity is not None:
- if type(quantity) not in [pd.Series, pd.DataFrame]:
- quantity = pd.Series(quantity, name='QNT')
-
- if quantity.size > 1:
- sample_size = quantity.size
- elif sample_size > 1:
- quantity = pd.Series(np.ones(sample_size) * quantity.values,
- name='QNT')
-
- # if there are no samples or resampling is forced, then sample the
- # distribution first
- # TODO: force_resampling is probably not needed
- # if (force_resampling or
- # (self._DV_distribution.samples is None)):
- # self._DV_distribution.sample_distribution(sample_size=sample_size)
-
- # # if the number of samples is not sufficiently large, raise an error
- # if self._DV_distribution.samples.shape[0] < sample_size:
- # raise ValueError(
- # 'Consequence evaluation requires at least as many samples of '
- # 'the Decision Variable distribution as many samples are '
- # 'requested or as many quantity values are provided to the '
- # 'sample_unit_DV function. You might want to consider setting '
- # 'force_resampling to True or sampling the distribution before '
- # 'calling the sample_unit_DV function.')
-
- # get the samples
- if quantity is not None:
- samples = pd.Series(self._DV_distribution.samples).loc[quantity.index]
- else:
- samples = pd.Series(self._DV_distribution.samples).iloc[:sample_size]
- samples = samples * median
-
- return samples
-
-
-[docs]class DamageState(object):
- """
- Characterizes one type of damage that corresponds to a particular DSG.
-
- The occurrence of damage is evaluated at the DSG. The DS describes one of
- the possibly several types of damages that belong to the same DSG and the
- consequences of such damage.
-
- Parameters
- ----------
-
- ID:int
- weight: float, optional, default: 1.0
- Describes the probability of DS occurrence, conditioned on the damage
- being in the DSG linked to this DS. This information is only used for
- DSGs with multiple DS corresponding to them. The weights of the set of
- DS shall sum up to 1.0 if they are mutually exclusive. When the set of
- DS occur simultaneously, the sum of weights typically exceeds 1.0.
- description: str, optional
- Provides a short description of the damage state.
- affected_area: float, optional, default: 0.
- Defines the area over which life safety hazards from this DS exist.
- repair_cost_CF: ConsequenceFunction, optional
- A consequence function that describes the cost necessary to restore the
- component to its pre-disaster condition.
- reconstruction_time_CF: ConsequenceFunction, optional
- A consequence function that describes the time, necessary to repair the
- damaged component to its pre-disaster condition.
- injuries_CF_set: ConsequenceFunction array, optional
- A set of consequence functions; each describes the number of people
- expected to experience injury of a particular severity when the
- component is in this DS. Any number of injury-levels can be considered.
- red_tag_CF: ConsequenceFunction, optional
- A consequence function that describes the proportion of components
- (within a Performance Group) that needs to be damaged to trigger an
- unsafe placard (i.e. red tag) for the building during post-disaster
- inspection.
-
- """
-
- def __init__(self, ID, weight=1.0, description='',
- repair_cost_CF=None, reconstruction_time_CF=None,
- injuries_CF_set=None, affected_area=0., red_tag_CF=None):
- self._ID = int(ID)
- self._weight = float(weight)
- self._description = description
- self._repair_cost_CF = repair_cost_CF
- self._reconstruction_time_CF = reconstruction_time_CF
- self._injuries_CF_set = injuries_CF_set
- self._affected_area = affected_area
- self._red_tag_CF = red_tag_CF
-
- @property
- def description(self):
- """
- Return the damage description.
- """
- return self._description
-
- @property
- def weight(self):
- """
- Return the weight of DS among the set of damage states in the DSG.
- """
- return self._weight
-
-[docs] def unit_repair_cost(self, quantity=None, sample_size=1, **kwargs):
- """
- Sample the repair cost distribution and return the unit repair costs.
-
- The unit repair costs shall be multiplied by the quantity of damaged
- components to get the total repair costs for the components in this DS.
-
- Parameters
- ----------
- quantity: float scalar, ndarray or Series, optional, default: None
- Total quantity of damaged components that determines the median
- repair cost. Not used for repair cost models with fixed median.
- sample_size: int, optional, default: 1
- Number of samples drawn from the repair cost distribution. The
- default value yields one sample.
-
- Returns
- -------
- unit_repair_cost: float scalar or ndarray
- Unit repair cost samples.
-
- """
- output = None
- if self._repair_cost_CF is not None:
- output = self._repair_cost_CF.sample_unit_DV(quantity=quantity,
- sample_size=sample_size,
- **kwargs)
-
- return output
-
-[docs] def unit_reconstruction_time(self, quantity=None, sample_size=1,
- **kwargs):
- """
- Sample the reconstruction time distribution and return the unit
- reconstruction times.
-
- The unit reconstruction times shall be multiplied by the quantity of
- damaged components to get the total reconstruction time for the
- components in this DS.
-
- Parameters
- ----------
- quantity: float scalar, ndarray or Series, optional, default: None
- Total quantity of damaged components that determines the magnitude
- of median reconstruction time. Not used for reconstruction time
- models with fixed median.
- sample_size: int, optional, default: 1
- Number of samples drawn from the reconstruction time distribution.
- The default value yields one sample.
-
- Returns
- -------
- unit_reconstruction_time: float scalar or ndarray
- Unit reconstruction time samples.
-
- """
- output = None
- if self._reconstruction_time_CF is not None:
- output = self._reconstruction_time_CF.sample_unit_DV(
- quantity=quantity,
- sample_size=sample_size, **kwargs)
-
- return output
-
-[docs] def red_tag_dmg_limit(self, sample_size=1, **kwargs):
- """
- Sample the red tag consequence function and return the proportion of
- components that needs to be damaged to trigger a red tag.
-
- The red tag consequence function is assumed to have a fixed median
- value that does not depend on the quantity of damaged components.
-
- Parameters
- ----------
- sample_size: int, optional, default: 1
- Number of samples drawn from the red tag consequence distribution.
- The default value yields one sample.
-
- Returns
- -------
- red_tag_trigger: float scalar or ndarray
- Samples of damaged component proportions that trigger a red tag.
-
- """
- output = None
- if self._red_tag_CF is not None:
- output = self._red_tag_CF.sample_unit_DV(sample_size=sample_size,
- **kwargs)
-
- return output
-
-[docs] def unit_injuries(self, severity_level=0, sample_size=1, **kwargs):
- """
- Sample the injury consequence function that corresponds to the
- specified level of severity and return the injuries per component unit.
-
- The injury consequence function is assumed to have a fixed median
- value that does not depend on the quantity of damaged components (i.e.
- the number of injuries per component unit does not change with the
- quantity of components.)
-
- Parameters
- ----------
- severity_level: int, optional, default: 1
- Identifies which injury consequence to sample. The indexing of
- severity levels is zero-based.
- sample_size: int, optional, default: 1
- Number of samples drawn from the injury consequence distribution.
- The default value yields one sample.
-
- Returns
- -------
- unit_injuries: float scalar or ndarray
- Unit injury samples.
-
- """
-
-
- output = None
- if len(self._injuries_CF_set) > severity_level:
- CF = self._injuries_CF_set[severity_level]
- if CF is not None:
- output = CF.sample_unit_DV(sample_size=sample_size, **kwargs)
-
- return output
-
-
-[docs]class DamageStateGroup(object):
- """
- A set of similar component damages that are controlled by the same EDP.
-
- Damages are described in detail by the set of Damage State objects.
- Damages in a DSG are assumed to occur at the same EDP magnitude. A Damage
- State Group (DSG) might have only a single DS in the simplest case.
-
- Parameters
- ----------
- ID: int
- DS_set: DamageState array
- DS_set_kind: {'single', 'mutually_exclusive', 'simultaneous'}
- Specifies the relationship among the DS in the set. When only one DS is
- defined, use the 'single' option to improve calculation efficiency.
- When multiple DS are present, the 'mutually_exclusive' option assumes
- that the occurrence of one DS precludes the occurrence of another DS.
- In such a case, the weights of the DS in the set shall sum up to 1.0.
- In a 'simultaneous' case the DS are independent and unrelated. Hence,
- they can occur at the same time and at least one of them has to occur.
- """
-
- def __init__(self, ID, DS_set, DS_set_kind):
- self._ID = ID
- self._DS_set = DS_set
- self._DS_set_kind = DS_set_kind
-
-[docs]class PerformanceGroup(object):
- """
- A group of similar components that experience the same demands.
-
- FEMA P-58: Performance Groups (PGs) are a sub-categorization of fragility
- groups. A performance group is a subset of fragility group components that
- are subjected to the same demands (e.g. story drift, floor acceleration,
- etc.).
-
- In buildings, most performance groups shall be organized by story level.
- There is no need to separate performance groups by direction, because the
- direction of components within a group can be specified during definition,
- and it will be taken into consideration in the analysis.
-
- Parameters
- ----------
- ID: int
- location: int
- Identifies the location of the components that belong to the PG. In a
- building, location shall typically refer to the story of the building.
- The location assigned to each PG shall be in agreement with the
- locations assigned to the Demand objects.
- quantity: RandomVariable
- Specifies the quantity of components that belong to this PG.
- Uncertainty in component quantities is considered by assigning a
- random variable to this property.
- fragility_functions: FragilityFunction list
- Each fragility function describes the probability that the damage in
- a subset of components will meet or exceed the damages described by
- each damage state group in the DSG_set. Each is a multi-dimensional
- function if there is more than one DSG. The number of functions shall
- match the number of subsets defined by the `csg_weights` parameter.
- DSG_set: DamageStateGroup array
- A set of sequential Damage State Groups that describe the plausible set
- of damage states of the components in the FG.
- csg_weights: float ndarray, optional, default: [1.0]
- Identifies subgroups of components within a PG, each of which have
- perfectly correlated behavior. Correlation between the damage and
- consequences among subgroups is controlled by the `correlation`
- parameter of the FragilityGroup that the PG belongs to. Note that if
- the components are assumed to have perfectly correlated behavior at the
- PG level, assigning several subgroups to the PG is unnecessary. This
- input shall be a list of weights that are applied to the quantity
- of components to define the amount of components in each subgroup. The
- sum of assigned weights shall be 1.0.
- directions: int ndarray, optional, default: [0]
- Identifies the direction of each subgroup of components within the PG.
- The number of directions shall be identical to the number of
- csg_weights assigned. In buildings, directions typically correspond to
- the orientation of components in plane. Hence, using 0 or 1 to identify
- 'X' or 'Y' is recommended. These directions shall be in agreement with
- the directions assigned to Demand objects.
- """
-
- def __init__(self, ID, location, quantity, fragility_functions, DSG_set,
- csg_weights=[1.0], direction=0):
- self._ID = ID
- self._location = location
- self._quantity = quantity
- if type(fragility_functions) == FragilityFunction:
- self._FF_set = [fragility_functions,]
- else:
- self._FF_set = fragility_functions
- self._DSG_set = DSG_set
- self._csg_weights = csg_weights
- self._direction = direction
-
-[docs] def P_exc(self, EDP, DSG_ID):
- """
- This is a convenience function that provides a shortcut to
- fragility_function.P_exc(). It calculates the exceedance probability
- of a given DSG conditioned on the provided EDP value(s). The fragility
- functions assigned to the first subset are used for this calculation
- because P_exc shall be identical among subsets.
-
- Parameters
- ----------
- EDP: float scalar or ndarray
- Single EDP or numpy array of EDP values.
- DSG_ID: int
- Identifies the DSG of interest.
-
- Returns
- -------
- P_exc: float scalar or ndarray
- Exceedance probability of the given DSG at the EDP point(s).
- """
- return self._FF_set[0].P_exc(EDP, DSG_ID)
-
-
-[docs]class FragilityGroup(object):
- """
- Groups a set of similar components from a loss-assessment perspective.
-
- Characterizes a set of structural or non-structural components that have
- similar construction characteristics, similar potential modes of damage,
- similar probability of incurring those modes of damage, and similar
- potential consequences resulting from their damage.
-
- Parameters
- ----------
- ID: int
- demand_type: {'PID', 'PFA', 'PSD', 'PSA', 'ePGA', 'PGD'}
- The type of Engineering Demand Parameter (EDP) that controls the damage
- of the components in the FG. See Demand for acronym descriptions.
- performance_groups: PerformanceGroup array
- A list of performance groups that contain the components characterized
- by the FG.
- directional: bool, optional, default: True
- Determines whether the components in the FG are sensitive to the
- directionality of the EDP.
- correlation: bool, optional, default: True
- Determines whether the components within a Performance Group (PG) will
- have correlated or uncorrelated damage. Correlated damage means that
- all components will have the same damage state. In the uncorrelated
- case, each component in the performance group will have its damage
- state evaluated independently. Correlated damage reduces the required
- computational effort for the calculation. Incorrect correlation
- modeling will only slightly affect the mean estimates, but might
- significantly change the dispersion of results.
- demand_location_offset: int, optional, default: 0
- Indicates if the location for the demand shall be different from the
- location of the components. Damage to components of the ceiling, for
- example, is controlled by demands on the floor above the one that the
- components belong to. This can be indicated by setting the
- demand_location_offset to 1 for such an FG.
- incomplete: bool, optional, default: False
- Indicates that the FG information is not complete and corresponding
- results shall be treated with caution.
- name: str, optional, default: ''
- Provides a short description of the fragility group.
- description: str, optional, default: ''
- Provides a detailed description of the fragility group.
- """
-
- def __init__(self, ID, demand_type, performance_groups,
- directional=True, correlation=True, demand_location_offset=0,
- incomplete=False, name='', description='', unit="ea"):
- self._ID = ID
- self._demand_type = demand_type
- self._performance_groups = performance_groups
- self._directional = directional
- self._correlation = correlation
- self._demand_location_offset = demand_location_offset
- self._incomplete = incomplete
- self._name = name
- self._description = description
- self._unit = unit
-
- @property
- def description(self):
- """
- Return the fragility group description.
- """
- return self._description
-
- @property
- def name(self):
- """
- Return the name of the fragility group.
-
- """
- return self._name
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-These are unit and integration tests on the control module of pelicun.
-
-"""
-
-import pytest
-import numpy as np
-from numpy.testing import assert_allclose
-from scipy.stats import truncnorm as tnorm
-from copy import deepcopy
-
-import os, sys, inspect
-current_dir = os.path.dirname(
- os.path.abspath(inspect.getfile(inspect.currentframe())))
-parent_dir = os.path.dirname(current_dir)
-sys.path.insert(0,os.path.dirname(parent_dir))
-
-from pelicun.control import *
-from pelicun.uq import mvn_orthotope_density as mvn_od
-from pelicun.tests.test_pelicun import prob_allclose, prob_approx
-
-# -----------------------------------------------------------------------------
-# FEMA_P58_Assessment
-# -----------------------------------------------------------------------------
-
-[docs]def test_FEMA_P58_Assessment_central_tendencies():
- """
- Perform a loss assessment with customized inputs that reduce the
- dispersion of calculation parameters to negligible levels. This allows us
- to test the results against pre-defined reference values in spite of the
- randomness involved in the calculations.
-
- """
-
- base_input_path = 'resources/'
- DL_input = base_input_path + 'input data/' + "DL_input_test.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())[0]
- assert RV_EDP.theta[0] == pytest.approx(0.5 * g)
- assert RV_EDP.theta[1] == pytest.approx(0.5 * g * 1e-6, abs=1e-7)
- assert RV_EDP._distribution == 'lognormal'
-
- # QNT
- assert A._QNT_dict is None
- #RV_QNT = A._RV_dict['QNT']
- #assert RV_QNT is None
-
- # FRG
- RV_FRG = list(A._FF_dict.values())
-
- thetas, betas = np.array([rv.theta for rv in RV_FRG]).T
- assert_allclose(thetas, np.array([0.444, 0.6, 0.984]) * g, rtol=0.01)
- assert_allclose(betas, np.array([0.3, 0.4, 0.5]), rtol=0.01)
-
- rho = RV_FRG[0].RV_set.Rho()
- assert_allclose(rho, np.ones((3, 3)), rtol=0.01)
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_FRG])
-
- # RED
- RV_RED = list(A._DV_RED_dict.values())
-
- mus, sigmas = np.array([rv.theta for rv in RV_RED]).T
- assert_allclose(mus, np.ones(2), rtol=0.01)
- assert_allclose(sigmas, np.array([1e-4, 1e-4]), rtol=0.01)
-
- rho = RV_RED[0].RV_set.Rho()
- assert_allclose(rho, np.array([[1, 0], [0, 1]]), rtol=0.01)
-
- assert np.all([rv.distribution == 'normal' for rv in RV_RED])
-
- assert_allclose (RV_RED[0].truncation_limits, [0., 2.], rtol=0.01)
- assert_allclose (RV_RED[1].truncation_limits, [0., 4.], rtol=0.01)
-
- # INJ
- RV_INJ = list(A._DV_INJ_dict.values())
-
- mus, sigmas = np.array([rv.theta for rv in RV_INJ]).T
- assert_allclose(mus, np.ones(4), rtol=0.01)
- assert_allclose(sigmas, np.ones(4) * 1e-4, rtol=0.01)
-
- rho = RV_INJ[0].RV_set.Rho()
- rho_target = np.zeros((4, 4))
- np.fill_diagonal(rho_target, 1.)
- assert_allclose(rho, rho_target, rtol=0.01)
-
- assert np.all([rv.distribution == 'normal' for rv in RV_INJ])
-
- assert_allclose(RV_INJ[0].truncation_limits, [0., 10./3.], rtol=0.01)
- assert_allclose(RV_INJ[1].truncation_limits, [0., 10./3.], rtol=0.01)
- assert_allclose(RV_INJ[2].truncation_limits, [0., 10.], rtol=0.01)
- assert_allclose(RV_INJ[3].truncation_limits, [0., 10.], rtol=0.01)
-
- # REP
- RV_REP = list(A._DV_REP_dict.values())
-
- thetas, betas = np.array([rv.theta for rv in RV_REP]).T
- assert_allclose(thetas, np.ones(6), rtol=0.01)
- assert_allclose(betas, np.ones(6) * 1e-4, rtol=0.01)
-
- rho = RV_REP[0].RV_set.Rho()
- rho_target = np.zeros((6, 6))
- np.fill_diagonal(rho_target, 1.)
- assert_allclose(rho, rho_target, rtol=0.01)
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_REP])
-
- # ------------------------------------------------------------------------
-
- A.define_loss_model()
-
- # QNT (deterministic)
- QNT = A._FG_dict['T0001.001']._performance_groups[0]._quantity
- assert QNT == pytest.approx(50., rel=0.01)
-
- A.calculate_damage()
-
- # ------------------------------------------------ check damage calculation
- # TIME
- T_check = A._TIME.describe().T.loc[['hour','month','weekday?'],:]
-
- assert_allclose(T_check['mean'], np.array([11.5, 5.5, 5. / 7.]), rtol=0.05)
- assert_allclose(T_check['min'], np.array([0., 0., 0.]), rtol=0.01)
- assert_allclose(T_check['max'], np.array([23., 11., 1.]), rtol=0.01)
- assert_allclose(T_check['50%'], np.array([12., 5., 1.]), atol=1.0)
- assert_allclose(T_check['count'], np.array([10000., 10000., 10000.]),
- rtol=0.01)
-
- # POP
- P_CDF = A._POP.describe(np.arange(1, 27) / 27.).iloc[:, 0].values[4:]
- vals, counts = np.unique(P_CDF, return_counts=True)
- assert_allclose(vals, np.array([0., 2.5, 5., 10.]), rtol=0.01)
- assert_allclose(counts, np.array([14, 2, 7, 5]), atol=1)
-
- # COL
- COL_check = A._COL.describe().T
- assert COL_check['mean'].values[0] == pytest.approx(0.5, rel=0.05)
- assert len(A._ID_dict['non-collapse']) == pytest.approx(5000, rel=0.05)
- assert len(A._ID_dict['collapse']) == pytest.approx(5000, rel=0.05)
-
- # DMG
- DMG_check = A._DMG.describe().T
- assert_allclose(DMG_check['mean'], np.array([17.074, 17.074, 7.9361]),
- rtol=0.1, atol=1.0)
- assert_allclose(DMG_check['min'], np.zeros(3), rtol=0.01)
- assert_allclose(DMG_check['max'], np.ones(3) * 50.0157, rtol=0.05)
-
- # ------------------------------------------------------------------------
-
- A.calculate_losses()
-
- # -------------------------------------------------- check loss calculation
-
- # RED
- DV_RED = A._DV_dict['red_tag'].describe().T
- assert_allclose(DV_RED['mean'], np.array([0.341344, 0.1586555]), rtol=0.1)
-
- # INJ - collapse
- DV_INJ_C = deepcopy(A._COL[['INJ-0', 'INJ-1']])
- DV_INJ_C.dropna(inplace=True)
- NC_count = DV_INJ_C.describe().T['count'][0]
- assert_allclose(NC_count, np.ones(2) * 5000, rtol=0.05)
- # lvl 1
- vals, counts = np.unique(DV_INJ_C.iloc[:, 0].values, return_counts=True)
- assert_allclose(vals, np.array([0., 2.5, 5., 10.]) * 0.1, rtol=0.01)
- assert_allclose(counts / NC_count, np.array([14, 2, 7, 5]) / 28., atol=0.01, rtol=0.1)
- # lvl 2
- vals, counts = np.unique(DV_INJ_C.iloc[:, 1].values, return_counts=True)
- assert_allclose(vals, np.array([0., 2.5, 5., 10.]) * 0.9, rtol=0.01)
- assert_allclose(counts / NC_count, np.array([14, 2, 7, 5]) / 28., atol=0.01, rtol=0.1)
-
- # INJ - non-collapse
- DV_INJ_NC = deepcopy(A._DV_dict['injuries'])
- DV_INJ_NC[0].dropna(inplace=True)
- assert_allclose(DV_INJ_NC[0].describe().T['count'], np.ones(2) * 5000,
- rtol=0.05)
- # lvl 1 DS2
- I_CDF = DV_INJ_NC[0].iloc[:, 0]
- I_CDF = np.around(I_CDF, decimals=3)
- vals, counts = np.unique(I_CDF, return_counts=True)
- assert_allclose(vals, np.array([0., 0.075, 0.15, 0.3]), rtol=0.01)
- target_prob = np.array(
- [0.6586555, 0., 0., 0.] + 0.3413445 * np.array([14, 2, 7, 5]) / 28.)
- assert_allclose(counts / NC_count, target_prob, atol=0.01, rtol=0.1)
- # lvl 1 DS3
- I_CDF = DV_INJ_NC[0].iloc[:, 1]
- I_CDF = np.around(I_CDF, decimals=3)
- vals, counts = np.unique(I_CDF, return_counts=True)
- assert_allclose(vals, np.array([0., 0.075, 0.15, 0.3]), rtol=0.01)
- target_prob = np.array(
- [0.8413445, 0., 0., 0.] + 0.1586555 * np.array([14, 2, 7, 5]) / 28.)
- assert_allclose(counts / NC_count, target_prob, atol=0.01, rtol=0.1)
- # lvl 2 DS2
- I_CDF = DV_INJ_NC[1].iloc[:, 0]
- I_CDF = np.around(I_CDF, decimals=3)
- vals, counts = np.unique(I_CDF, return_counts=True)
- assert_allclose(vals, np.array([0., 0.025, 0.05, 0.1]), rtol=0.01)
- target_prob = np.array(
- [0.6586555, 0., 0., 0.] + 0.3413445 * np.array([14, 2, 7, 5]) / 28.)
- assert_allclose(counts / NC_count, target_prob, atol=0.01, rtol=0.1)
- # lvl2 DS3
- I_CDF = DV_INJ_NC[1].iloc[:, 1]
- I_CDF = np.around(I_CDF, decimals=3)
- vals, counts = np.unique(I_CDF, return_counts=True)
- assert_allclose(vals, np.array([0., 0.025, 0.05, 0.1]), rtol=0.01)
- target_prob = np.array(
- [0.8413445, 0., 0., 0.] + 0.1586555 * np.array([14, 2, 7, 5]) / 28.)
- assert_allclose(counts / NC_count, target_prob, atol=0.01, rtol=0.1)
-
- # REP
- assert len(A._ID_dict['non-collapse']) == len(A._ID_dict['repairable'])
- assert len(A._ID_dict['irreparable']) == 0
- # cost
- DV_COST = A._DV_dict['rec_cost']
- # DS1
- C_CDF = DV_COST.iloc[:, 0]
- C_CDF = np.around(C_CDF / 10., decimals=0) * 10.
- vals, counts = np.unique(C_CDF, return_counts=True)
- assert_allclose(vals, [0, 2500], rtol=0.01)
- t_prob = 0.3413445
- assert_allclose(counts / NC_count, [1. - t_prob, t_prob], rtol=0.1)
- # DS2
- C_CDF = DV_COST.iloc[:, 1]
- C_CDF = np.around(C_CDF / 100., decimals=0) * 100.
- vals, counts = np.unique(C_CDF, return_counts=True)
- assert_allclose(vals, [0, 25000], rtol=0.01)
- t_prob = 0.3413445
- assert_allclose(counts / NC_count, [1. - t_prob, t_prob], rtol=0.1)
- # DS3
- C_CDF = DV_COST.iloc[:, 2]
- C_CDF = np.around(C_CDF / 1000., decimals=0) * 1000.
- vals, counts = np.unique(C_CDF, return_counts=True)
- assert_allclose(vals, [0, 250000], rtol=0.01)
- t_prob = 0.1586555
- assert_allclose(counts / NC_count, [1. - t_prob, t_prob], rtol=0.1)
- # time
- DV_TIME = A._DV_dict['rec_time']
- # DS1
- T_CDF = DV_TIME.iloc[:, 0]
- T_CDF = np.around(T_CDF, decimals=1)
- vals, counts = np.unique(T_CDF, return_counts=True)
- assert_allclose(vals, [0, 2.5], rtol=0.01)
- t_prob = 0.3413445
- assert_allclose(counts / NC_count, [1. - t_prob, t_prob], rtol=0.1)
- # DS2
- T_CDF = DV_TIME.iloc[:, 1]
- T_CDF = np.around(T_CDF, decimals=0)
- vals, counts = np.unique(T_CDF, return_counts=True)
- assert_allclose(vals, [0, 25], rtol=0.01)
- t_prob = 0.3413445
- assert_allclose(counts / NC_count, [1. - t_prob, t_prob], rtol=0.1)
- # DS3
- T_CDF = DV_TIME.iloc[:, 2]
- T_CDF = np.around(T_CDF / 10., decimals=0) * 10.
- vals, counts = np.unique(T_CDF, return_counts=True)
- assert_allclose(vals, [0, 250], rtol=0.01)
- t_prob = 0.1586555
- assert_allclose(counts / NC_count, [1. - t_prob, t_prob], rtol=0.1)
-
- # ------------------------------------------------------------------------
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- S = A._SUMMARY
- SD = S.describe().T
-
- assert_allclose(S[('event time', 'month')], A._TIME['month'] + 1)
- assert_allclose(S[('event time', 'weekday?')], A._TIME['weekday?'])
- assert_allclose(S[('event time', 'hour')], A._TIME['hour'])
- assert_allclose(S[('inhabitants', '')], A._POP.iloc[:, 0])
-
- assert SD.loc[('collapses', 'collapsed'), 'mean'] == pytest.approx(0.5,
- rel=0.05)
- assert SD.loc[('collapses', 'mode'), 'mean'] == 0.
- assert SD.loc[('collapses', 'mode'), 'count'] == pytest.approx(5000,
- rel=0.05)
-
- assert SD.loc[('red tagged', ''), 'mean'] == pytest.approx(0.5, rel=0.05)
- assert SD.loc[('red tagged', ''), 'count'] == pytest.approx(5000, rel=0.05)
-
- for col in ['irreparable', 'cost impractical', 'time impractical']:
- assert SD.loc[('reconstruction', col), 'mean'] == 0.
- assert SD.loc[('reconstruction', col), 'count'] == pytest.approx(5000,
- rel=0.05)
-
- RC = deepcopy(S.loc[:, ('reconstruction', 'cost')])
- RC_CDF = np.around(RC / 1000., decimals=0) * 1000.
- vals, counts = np.unique(RC_CDF, return_counts=True)
- assert_allclose(vals, np.array([0, 2., 3., 25., 250., 300.]) * 1000.)
- t_prob1 = 0.3413445 / 2.
- t_prob2 = 0.1586555 / 2.
- assert_allclose(counts / 10000.,
- [t_prob2, t_prob1 / 2., t_prob1 / 2., t_prob1, t_prob2,
- 0.5], atol=0.01, rtol=0.1)
-
- RT = deepcopy(S.loc[:, ('reconstruction', 'time-parallel')])
- RT_CDF = np.around(RT, decimals=0)
- vals, counts = np.unique(RT_CDF, return_counts=True)
- assert_allclose(vals, np.array([0, 2., 3., 25., 250., 300.]))
- t_prob1 = 0.3413445 / 2.
- t_prob2 = 0.1586555 / 2.
- assert_allclose(counts / 10000.,
- [t_prob2, t_prob1 / 2., t_prob1 / 2., t_prob1, t_prob2,
- 0.5], atol=0.01, rtol=0.1)
-
- assert_allclose(S.loc[:, ('reconstruction', 'time-parallel')],
- S.loc[:, ('reconstruction', 'time-sequential')])
-
- CAS = deepcopy(S.loc[:, ('injuries', 'sev1')])
- CAS_CDF = np.around(CAS, decimals=3)
- vals, counts = np.unique(CAS_CDF, return_counts=True)
- assert_allclose(vals, [0, 0.075, 0.15, 0.25, 0.3, 0.5, 1.])
- assert_allclose(counts / 10000.,
- np.array([35, 1, 3.5, 2, 2.5, 7, 5]) / 56., atol=0.01,
- rtol=0.1)
-
- CAS = deepcopy(S.loc[:, ('injuries', 'sev2')])
- CAS_CDF = np.around(CAS, decimals=3)
- vals, counts = np.unique(CAS_CDF, return_counts=True)
- assert_allclose(vals, [0, 0.025, 0.05, 0.1, 2.25, 4.5, 9.])
- assert_allclose(counts / 10000.,
- np.array([35, 1, 3.5, 2.5, 2, 7, 5]) / 56., atol=0.01,
- rtol=0.1)
-
-[docs]def test_FEMA_P58_Assessment_EDP_uncertainty_basic():
- """
- Perform a loss assessment with customized inputs that focus on testing the
- methods used to estimate the multivariate lognormal distribution of EDP
- values. Besides the fitting, this test also evaluates the propagation of
- EDP uncertainty through the analysis. Dispersions in other calculation
- parameters are reduced to negligible levels. This allows us to test the
- results against pre-defined reference values in spite of the randomness
- involved in the calculations.
- """
-
- base_input_path = 'resources/'
- DL_input = base_input_path + 'input data/' + "DL_input_test_2.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_2.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())
-
- thetas, betas = np.array([rv.theta for rv in RV_EDP]).T
- assert_allclose(thetas, [9.80665, 12.59198, 0.074081, 0.044932], rtol=0.02)
- assert_allclose(betas, [0.25, 0.25, 0.3, 0.4], rtol=0.02)
-
- rho = RV_EDP[0].RV_set.Rho()
- rho_target = [
- [1.0, 0.6, 0.3, 0.3],
- [0.6, 1.0, 0.3, 0.3],
- [0.3, 0.3, 1.0, 0.7],
- [0.3, 0.3, 0.7, 1.0]]
- assert_allclose(rho, rho_target, atol=0.05)
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_EDP])
-
- # ------------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # ------------------------------------------------ check damage calculation
- # COL
- COL_check = A._COL.describe().T
- col_target = 1.0 - mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer(
- [0.3, 0.4], [0.3, 0.4]),
- upper=np.log([0.1, 0.1]))[0]
- assert COL_check['mean'].values[0] == pytest.approx(col_target, rel=0.1)
-
- # DMG
- DMG_check = [len(np.where(A._DMG.iloc[:, i] > 0.0)[0]) / 10000. for i in
- range(8)]
-
- DMG_1_PID = mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([0.05488, 1e-6]), upper=np.log([0.1, 0.1]))[
- 0]
- DMG_2_PID = mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([1e-6, 0.05488]), upper=np.log([0.1, 0.1]))[
- 0]
- DMG_1_PFA = mvn_od(np.log([0.074081, 9.80665]),
- np.array([[1, 0.3], [0.3, 1]]) * np.outer([0.3, 0.25],
- [0.3, 0.25]),
- lower=np.log([1e-6, 9.80665]),
- upper=np.log([0.1, np.inf]))[0]
- DMG_2_PFA = mvn_od(np.log([0.074081, 12.59198]),
- np.array([[1, 0.3], [0.3, 1]]) * np.outer([0.3, 0.25],
- [0.3, 0.25]),
- lower=np.log([1e-6, 9.80665]),
- upper=np.log([0.1, np.inf]))[0]
-
- assert DMG_check[0] == pytest.approx(DMG_check[1], rel=0.01)
- assert DMG_check[2] == pytest.approx(DMG_check[3], rel=0.01)
- assert DMG_check[4] == pytest.approx(DMG_check[5], rel=0.01)
- assert DMG_check[6] == pytest.approx(DMG_check[7], rel=0.01)
-
- assert DMG_check[0] == pytest.approx(DMG_1_PID, rel=0.10)
- assert DMG_check[2] == pytest.approx(DMG_2_PID, rel=0.10)
- assert DMG_check[4] == pytest.approx(DMG_1_PFA, rel=0.10)
- assert DMG_check[6] == pytest.approx(DMG_2_PFA, rel=0.10)
-
- # ------------------------------------------------------------------------
-
- A.calculate_losses()
-
- # -------------------------------------------------- check loss calculation
- # COST
- DV_COST = A._DV_dict['rec_cost']
- DV_TIME = A._DV_dict['rec_time']
-
- C_target = [0., 250., 1250.]
- T_target = [0., 0.25, 1.25]
-
- # PG 1011 and 1012
- P_target = [
- mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([1e-6, 1e-6]), upper=np.log([0.05488, 0.1]))[0],
- mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([0.05488, 0.05488]), upper=np.log([0.1, 0.1]))[0],
- mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([0.05488, 1e-6]), upper=np.log([0.1, 0.05488]))[0],
- ]
-
- for i in [0, 1]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(P_target, P_test, atol=0.02)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 1021 and 1022
- P_target = [
- mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([1e-6, 1e-6]), upper=np.log([0.1, 0.05488]))[0],
- mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([0.05488, 0.05488]), upper=np.log([0.1, 0.1]))[0],
- mvn_od(np.log([0.074081, 0.044932]),
- np.array([[1, 0.7], [0.7, 1]]) * np.outer([0.3, 0.4],
- [0.3, 0.4]),
- lower=np.log([1e-6, 0.05488]), upper=np.log([0.05488, 0.1]))[0],
- ]
-
- for i in [2, 3]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(P_target, P_test, atol=0.02)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 2011 and 2012
- P_target = [
- mvn_od(np.log([0.074081, 9.80665, 12.59198]),
- np.array([[1.0, 0.3, 0.3], [0.3, 1.0, 0.6],
- [0.3, 0.6, 1.0]]) * np.outer([0.3, 0.25, 0.25],
- [0.3, 0.25, 0.25]),
- lower=np.log([1e-6, 1e-6, 1e-6]),
- upper=np.log([0.1, 9.80665, np.inf]))[0],
- mvn_od(np.log([0.074081, 9.80665, 12.59198]),
- np.array([[1.0, 0.3, 0.3], [0.3, 1.0, 0.6],
- [0.3, 0.6, 1.0]]) * np.outer([0.3, 0.25, 0.25],
- [0.3, 0.25, 0.25]),
- lower=np.log([1e-6, 9.80665, 9.80665]),
- upper=np.log([0.1, np.inf, np.inf]))[0],
- mvn_od(np.log([0.074081, 9.80665, 12.59198]),
- np.array([[1.0, 0.3, 0.3], [0.3, 1.0, 0.6],
- [0.3, 0.6, 1.0]]) * np.outer([0.3, 0.25, 0.25],
- [0.3, 0.25, 0.25]),
- lower=np.log([1e-6, 9.80665, 1e-6]),
- upper=np.log([0.1, np.inf, 9.80665]))[0],
- ]
-
- for i in [4, 5]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(P_target, P_test, atol=0.02)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 2021 and 2022
- P_target = [
- mvn_od(np.log([0.074081, 9.80665, 12.59198]),
- np.array([[1.0, 0.3, 0.3], [0.3, 1.0, 0.6],
- [0.3, 0.6, 1.0]]) * np.outer([0.3, 0.25, 0.25],
- [0.3, 0.25, 0.25]),
- lower=np.log([1e-6, 1e-6, 1e-6]),
- upper=np.log([0.1, np.inf, 9.80665]))[0],
- mvn_od(np.log([0.074081, 9.80665, 12.59198]),
- np.array([[1.0, 0.3, 0.3], [0.3, 1.0, 0.6],
- [0.3, 0.6, 1.0]]) * np.outer([0.3, 0.25, 0.25],
- [0.3, 0.25, 0.25]),
- lower=np.log([1e-6, 9.80665, 9.80665]),
- upper=np.log([0.1, np.inf, np.inf]))[0],
- mvn_od(np.log([0.074081, 9.80665, 12.59198]),
- np.array([[1.0, 0.3, 0.3], [0.3, 1.0, 0.6],
- [0.3, 0.6, 1.0]]) * np.outer([0.3, 0.25, 0.25],
- [0.3, 0.25, 0.25]),
- lower=np.log([1e-6, 1e-6, 9.80665]),
- upper=np.log([0.1, 9.80665, np.inf]))[0],
- ]
-
- for i in [6, 7]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(P_target, P_test, atol=0.02)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # RED TAG
- RED_check = A._DV_dict['red_tag'].describe().T
- RED_check = (RED_check['mean'] * RED_check['count'] / 10000.).values
-
- assert RED_check[0] == pytest.approx(RED_check[1], rel=0.01)
- assert RED_check[2] == pytest.approx(RED_check[3], rel=0.01)
- assert RED_check[4] == pytest.approx(RED_check[5], rel=0.01)
- assert RED_check[6] == pytest.approx(RED_check[7], rel=0.01)
-
- assert RED_check[0] == pytest.approx(DMG_1_PID, rel=0.10)
- assert RED_check[2] == pytest.approx(DMG_2_PID, rel=0.10)
- assert RED_check[4] == pytest.approx(DMG_1_PFA, rel=0.10)
- assert RED_check[6] == pytest.approx(DMG_2_PFA, rel=0.10)
-
- DMG_on = np.where(A._DMG > 0.0)[0]
- RED_on = np.where(A._DV_dict['red_tag'] > 0.0)[0]
- assert_allclose(DMG_on, RED_on)
-
- # ------------------------------------------------------------------------
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- P_no_RED_target = mvn_od(np.log([0.074081, 0.044932, 9.80665, 12.59198]),
- np.array(
- [[1.0, 0.7, 0.3, 0.3], [0.7, 1.0, 0.3, 0.3],
- [0.3, 0.3, 1.0, 0.6],
- [0.3, 0.3, 0.6, 1.0]]) * np.outer(
- [0.3, 0.4, 0.25, 0.25],
- [0.3, 0.4, 0.25, 0.25]),
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [0.05488, 0.05488, 9.80665, 9.80665]))[0]
- S = A._SUMMARY
- SD = S.describe().T
-
- P_no_RED_test = (1.0 - SD.loc[('red tagged', ''), 'mean']) * SD.loc[
- ('red tagged', ''), 'count'] / 10000.
-
-[docs]def test_FEMA_P58_Assessment_EDP_uncertainty_detection_limit():
- """
- Perform a loss assessment with customized inputs that focus on testing the
- methods used to estimate the multivariate lognormal distribution of EDP
- values. Besides the fitting, this test also evaluates the propagation of
- EDP uncertainty through the analysis. Dispersions in other calculation
- parameters are reduced to negligible levels. This allows us to test the
- results against pre-defined reference values in spite of the randomness
- involved in the calculations.
- This test differs from the basic case in having unreliable EDP values above
- a certain limit - a typical feature of interstory drifts in dynamic
- simulations. Such cases should not be a problem if the limits can be
- estimated and they are specified as detection limits in input file.
- """
-
- base_input_path = 'resources/'
- DL_input = base_input_path + 'input data/' + "DL_input_test_3.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_3.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())
-
- thetas, betas = np.array([rv.theta for rv in RV_EDP]).T
- EDP_theta_test = thetas
- EDP_beta_test = betas
- EDP_theta_target = [9.80665, 12.59198, 0.074081, 0.044932]
- EDP_beta_target = [0.25, 0.25, 0.3, 0.4]
- assert_allclose(EDP_theta_test, EDP_theta_target, rtol=0.025)
- assert_allclose(EDP_beta_test, EDP_beta_target, rtol=0.1)
-
- rho = RV_EDP[0].RV_set.Rho()
- EDP_rho_test = rho
- EDP_rho_target = [
- [1.0, 0.6, 0.3, 0.3],
- [0.6, 1.0, 0.3, 0.3],
- [0.3, 0.3, 1.0, 0.7],
- [0.3, 0.3, 0.7, 1.0]]
- EDP_COV_test = EDP_rho_test * np.outer(EDP_beta_test, EDP_beta_test)
- assert_allclose(EDP_rho_test, EDP_rho_target, atol=0.15)
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_EDP])
-
- # ------------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # ------------------------------------------------ check damage calculation
- # COL
- COL_check = A._COL.describe().T
-
- col_target = 1.0 - mvn_od(np.log(EDP_theta_test[2:]),
- EDP_COV_test[2:, 2:],
- upper=np.log([0.1, 0.1]))[0]
-
- assert COL_check['mean'].values[0] == prob_approx(col_target, 0.03)
-
- # DMG
- DMG_check = [len(np.where(A._DMG.iloc[:, i] > 0.0)[0]) / 10000.
- for i in range(8)]
-
- DMG_1_PID = mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([0.05488, 1e-6]),
- upper=np.log([0.1, 0.1]))[0]
-
- DMG_2_PID = mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 0.05488]),
- upper=np.log([0.1, 0.1]))[0]
-
- DMG_1_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0]
-
- DMG_2_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 9.80665, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0]
-
- assert DMG_check[0] == pytest.approx(DMG_check[1], rel=0.01)
- assert DMG_check[2] == pytest.approx(DMG_check[3], rel=0.01)
- assert DMG_check[4] == pytest.approx(DMG_check[5], rel=0.01)
- assert DMG_check[6] == pytest.approx(DMG_check[7], rel=0.01)
-
- assert DMG_check[0] == prob_approx(DMG_1_PID, 0.03)
- assert DMG_check[2] == prob_approx(DMG_2_PID, 0.03)
- assert DMG_check[4] == prob_approx(DMG_1_PFA, 0.03)
- assert DMG_check[6] == prob_approx(DMG_2_PFA, 0.03)
-
- # ------------------------------------------------------------------------
-
- A.calculate_losses()
-
- # -------------------------------------------------- check loss calculation
- # COST
- DV_COST = A._DV_dict['rec_cost']
- DV_TIME = A._DV_dict['rec_time']
-
- C_target = [0., 250., 1250.]
- T_target = [0., 0.25, 1.25]
-
- # PG 1011 and 1012
- P_target = [
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 1e-6]), upper=np.log([0.05488, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([0.05488, 0.05488]), upper=np.log([0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([0.05488, 1e-6]), upper=np.log([0.1, 0.05488]))[0],
- ]
-
- for i in [0, 1]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # PG 1021 and 1022
- P_target = [
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 1e-6]), upper=np.log([0.1, 0.05488]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([0.05488, 0.05488]), upper=np.log([0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 0.05488]), upper=np.log([0.05488, 0.1]))[0],
- ]
-
- for i in [2, 3]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # PG 2011 and 2012
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([9.80665, np.inf, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 9.80665, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, 9.80665, 0.1, 0.1]))[0],
- ]
-
- for i in [4, 5]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # PG 2021 and 2022
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, 9.80665, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 9.80665, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 9.80665, 1e-6, 1e-6]),
- upper=np.log([9.80665, np.inf, 0.1, 0.1]))[0],
- ]
-
- for i in [6, 7]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # RED TAG
- RED_check = A._DV_dict['red_tag'].describe().T
- RED_check = (RED_check['mean'] * RED_check['count'] / 10000.).values
-
- assert RED_check[0] == pytest.approx(RED_check[1], rel=0.01)
- assert RED_check[2] == pytest.approx(RED_check[3], rel=0.01)
- assert RED_check[4] == pytest.approx(RED_check[5], rel=0.01)
- assert RED_check[6] == pytest.approx(RED_check[7], rel=0.01)
-
- assert RED_check[0] == prob_approx(DMG_1_PID, 0.03)
- assert RED_check[2] == prob_approx(DMG_2_PID, 0.03)
- assert RED_check[4] == prob_approx(DMG_1_PFA, 0.03)
- assert RED_check[6] == prob_approx(DMG_2_PFA, 0.03)
-
- DMG_on = np.where(A._DMG > 0.0)[0]
- RED_on = np.where(A._DV_dict['red_tag'] > 0.0)[0]
- assert_allclose(DMG_on, RED_on)
-
- # ------------------------------------------------------------------------
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- P_no_RED_target = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([9.80665, 9.80665, 0.05488, 0.05488]))[0]
-
- S = A._SUMMARY
- SD = S.describe().T
-
- P_no_RED_test = ((1.0 - SD.loc[('red tagged', ''), 'mean'])
- * SD.loc[('red tagged', ''), 'count'] / 10000.)
-
- assert P_no_RED_target == prob_approx(P_no_RED_test, 0.04)
-
-[docs]def test_FEMA_P58_Assessment_EDP_uncertainty_failed_analyses():
- """
- Perform a loss assessment with customized inputs that focus on testing the
- methods used to estimate the multivariate lognormal distribution of EDP
- values. Besides the fitting, this test also evaluates the propagation of
- EDP uncertainty through the analysis. Dispersions in other calculation
- parameters are reduced to negligible levels. This allows us to test the
- results against pre-defined reference values in spite of the randomness
- involved in the calculations.
- Here we use EDP results with unique values assigned to failed analyses.
- In particular, PID=1.0 and PFA=100.0 are used when an analysis fails.
- These values shall be handled by detection limits of 10 and 100 for PID
- and PFA, respectively.
- """
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_4.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_4.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())
-
- thetas, betas = np.array([rv.theta for rv in RV_EDP]).T
- EDP_theta_test = thetas
- EDP_beta_test = betas
- EDP_theta_target = [9.80665, 12.59198, 0.074081, 0.044932]
- EDP_beta_target = [0.25, 0.25, 0.3, 0.4]
- assert_allclose(EDP_theta_test, EDP_theta_target, rtol=0.025)
- assert_allclose(EDP_beta_test, EDP_beta_target, rtol=0.1)
-
- rho = RV_EDP[0].RV_set.Rho()
- EDP_rho_test = rho
- EDP_rho_target = [
- [1.0, 0.6, 0.3, 0.3],
- [0.6, 1.0, 0.3, 0.3],
- [0.3, 0.3, 1.0, 0.7],
- [0.3, 0.3, 0.7, 1.0]]
- EDP_COV_test = EDP_rho_test * np.outer(EDP_beta_test, EDP_beta_test)
- assert_allclose(EDP_rho_test, EDP_rho_target, atol=0.15)
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_EDP])
-
- # ------------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # ------------------------------------------------ check damage calculation
- # COL
- COL_check = A._COL.describe().T
- col_target = 1.0 - mvn_od(np.log(EDP_theta_test[2:]),
- EDP_COV_test[2:,2:],
- upper=np.log([0.1, 0.1]))[0]
-
- assert COL_check['mean'].values[0] == prob_approx(col_target, 0.03)
-
- # DMG
- DMG_check = [len(np.where(A._DMG.iloc[:, i] > 0.0)[0]) / 10000.
- for i in range(8)]
-
- DMG_1_PID = mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:,2:],
- lower=np.log([0.05488, 1e-6]),
- upper=np.log([0.1, 0.1]))[0]
-
- DMG_2_PID = mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 0.05488]),
- upper=np.log([0.1, 0.1]))[0]
-
- DMG_1_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0]
-
- DMG_2_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 9.80665, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0]
-
- assert DMG_check[0] == pytest.approx(DMG_check[1], rel=0.01)
- assert DMG_check[2] == pytest.approx(DMG_check[3], rel=0.01)
- assert DMG_check[4] == pytest.approx(DMG_check[5], rel=0.01)
- assert DMG_check[6] == pytest.approx(DMG_check[7], rel=0.01)
-
- assert DMG_check[0] == prob_approx(DMG_1_PID, 0.03)
- assert DMG_check[2] == prob_approx(DMG_2_PID, 0.03)
- assert DMG_check[4] == prob_approx(DMG_1_PFA, 0.03)
- assert DMG_check[6] == prob_approx(DMG_2_PFA, 0.03)
-
- # ------------------------------------------------------------------------
-
- A.calculate_losses()
-
- # -------------------------------------------------- check loss calculation
- # COST
- DV_COST = A._DV_dict['rec_cost']
- DV_TIME = A._DV_dict['rec_time']
-
- C_target = [0., 250., 1250.]
- T_target = [0., 0.25, 1.25]
-
- # PG 1011 and 1012
- P_target = [
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 1e-6]), upper=np.log([0.05488, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([0.05488, 0.05488]), upper=np.log([0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([0.05488, 1e-6]), upper=np.log([0.1, 0.05488]))[0],
- ]
-
- for i in [0, 1]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # PG 1021 and 1022
- P_target = [
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 1e-6]), upper=np.log([0.1, 0.05488]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([0.05488, 0.05488]), upper=np.log([0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test[2:]), EDP_COV_test[2:, 2:],
- lower=np.log([1e-6, 0.05488]), upper=np.log([0.05488, 0.1]))[0],
- ]
-
- for i in [2, 3]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # PG 2011 and 2012
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([9.80665, np.inf, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 9.80665, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, 9.80665, 0.1, 0.1]))[0],
- ]
-
- for i in [4, 5]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # PG 2021 and 2022
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, 9.80665, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 9.80665, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, 0.1, 0.1]))[0],
-
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 9.80665, 1e-6, 1e-6]),
- upper=np.log([9.80665, np.inf, 0.1, 0.1]))[0],
- ]
-
- for i in [6, 7]:
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, i].values / 10., decimals=0) * 10.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(
- np.around(DV_TIME.iloc[:, i].values * 100., decimals=0) / 100.,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / 10000.
-
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
- prob_allclose(P_target, P_test, 0.04)
-
- # RED TAG
- RED_check = A._DV_dict['red_tag'].describe().T
- RED_check = (RED_check['mean'] * RED_check['count'] / 10000.).values
-
- assert RED_check[0] == pytest.approx(RED_check[1], rel=0.01)
- assert RED_check[2] == pytest.approx(RED_check[3], rel=0.01)
- assert RED_check[4] == pytest.approx(RED_check[5], rel=0.01)
- assert RED_check[6] == pytest.approx(RED_check[7], rel=0.01)
-
- assert RED_check[0] == prob_approx(DMG_1_PID, 0.03)
- assert RED_check[2] == prob_approx(DMG_2_PID, 0.03)
- assert RED_check[4] == prob_approx(DMG_1_PFA, 0.03)
- assert RED_check[6] == prob_approx(DMG_2_PFA, 0.03)
-
- DMG_on = np.where(A._DMG > 0.0)[0]
- RED_on = np.where(A._DV_dict['red_tag'] > 0.0)[0]
- assert_allclose(DMG_on, RED_on)
-
- # ------------------------------------------------------------------------
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- P_no_RED_target = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([9.80665, 9.80665, 0.05488, 0.05488]))[0]
-
- S = A._SUMMARY
- SD = S.describe().T
-
- P_no_RED_test = ((1.0 - SD.loc[('red tagged', ''), 'mean'])
- * SD.loc[('red tagged', ''), 'count'] / 10000.)
-
- assert P_no_RED_target == prob_approx(P_no_RED_test, 0.04)
-
-[docs]def test_FEMA_P58_Assessment_EDP_uncertainty_3D():
- """
- .
-
- Perform a loss assessment with customized inputs that focus on testing the
- methods used to estimate the multivariate lognormal distribution of EDP
- values. Besides the fitting, this test also evaluates the propagation of
- EDP uncertainty through the analysis. Dispersions in other calculation
- parameters are reduced to negligible levels. This allows us to test the
- results against pre-defined reference values in spite of the randomness
- involved in the calculations.
- In this test we look at the propagation of EDP values provided for two
- different directions. (3D refers to the numerical model used for response
- estimation.)
- """
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_5.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_5.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_EDP])
-
- thetas, betas = np.array([rv.theta for rv in RV_EDP]).T
- EDP_theta_test = thetas
- EDP_beta_test = betas
- EDP_theta_target = [9.80665, 8.65433, 12.59198, 11.11239,
- 0.074081, 0.063763, 0.044932, 0.036788]
- EDP_beta_target = [0.25, 0.25, 0.25, 0.25, 0.3, 0.3, 0.4, 0.4]
- assert_allclose(EDP_theta_test, EDP_theta_target, rtol=0.05)
- assert_allclose(EDP_beta_test, EDP_beta_target, rtol=0.1)
-
- rho = RV_EDP[0].RV_set.Rho()
- EDP_rho_test = rho
- EDP_rho_target = np.array([
- [1.0, 0.8, 0.6, 0.5, 0.3, 0.3, 0.3, 0.3],
- [0.8, 1.0, 0.5, 0.6, 0.3, 0.3, 0.3, 0.3],
- [0.6, 0.5, 1.0, 0.8, 0.3, 0.3, 0.3, 0.3],
- [0.5, 0.6, 0.8, 1.0, 0.3, 0.3, 0.3, 0.3],
- [0.3, 0.3, 0.3, 0.3, 1.0, 0.8, 0.7, 0.6],
- [0.3, 0.3, 0.3, 0.3, 0.8, 1.0, 0.6, 0.7],
- [0.3, 0.3, 0.3, 0.3, 0.7, 0.6, 1.0, 0.8],
- [0.3, 0.3, 0.3, 0.3, 0.6, 0.7, 0.8, 1.0]])
- large_rho_ids = np.where(EDP_rho_target >= 0.5)
- small_rho_ids = np.where(EDP_rho_target < 0.5)
- assert_allclose(EDP_rho_test[large_rho_ids], EDP_rho_target[large_rho_ids],
- atol=0.1)
- assert_allclose(EDP_rho_test[small_rho_ids], EDP_rho_target[small_rho_ids],
- atol=0.2)
-
- EDP_COV_test = EDP_rho_test * np.outer(EDP_beta_test, EDP_beta_test)
-
- # ------------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # ------------------------------------------------ check damage calculation
- theta_PID = np.log(EDP_theta_target[4:])
- COV_PID = EDP_COV_test[4:, 4:]
-
- # COL
- COL_check = A._COL.describe().T
- col_target = 1.0 - mvn_od(theta_PID, COV_PID,
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0]
-
- assert COL_check['mean'].values[0] == pytest.approx(col_target, rel=0.1, abs=0.05)
-
- # DMG
- realization_count = float(A._AIM_in['general']['realizations'])
- DMG_check = [len(np.where(A._DMG.iloc[:, i] > 0.0)[0]) / realization_count for i in
- range(8)]
-
- DMG_1_1_PID = mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0]
- DMG_1_2_PID = mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0]
- DMG_2_1_PID = mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 1e-6, 0.05488, 1e-6]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0]
- DMG_2_2_PID = mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 1e-6, 1e-6, 0.05488]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0]
- DMG_1_1_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([9.80665, 1e-6, 1e-6, 1e-6,
- 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, np.inf, np.inf,
- 0.1, 0.1, 0.1, 0.1]))[0]
- DMG_1_2_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 9.80665, 1e-6, 1e-6,
- 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, np.inf, np.inf,
- 0.1, 0.1, 0.1, 0.1]))[0]
- DMG_2_1_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 9.80665, 1e-6,
- 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, np.inf, np.inf,
- 0.1, 0.1, 0.1, 0.1]))[0]
- DMG_2_2_PFA = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 9.80665,
- 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([np.inf, np.inf, np.inf, np.inf,
- 0.1, 0.1, 0.1, 0.1]))[0]
-
- DMG_ref = [DMG_1_1_PID, DMG_1_2_PID, DMG_2_1_PID, DMG_2_2_PID,
- DMG_1_1_PFA, DMG_1_2_PFA, DMG_2_1_PFA, DMG_2_2_PFA]
-
- assert_allclose(DMG_check, DMG_ref, rtol=0.10, atol=0.01)
-
- # ------------------------------------------------------------------------
-
- A.calculate_losses()
-
- # -------------------------------------------------- check loss calculation
- # COST
- DV_COST = A._DV_dict['rec_cost']
- DV_TIME = A._DV_dict['rec_time']
-
- C_target = [0., 249., 624., 1251., 1875.]
- T_target = [0., 0.249, 0.624, 1.251, 1.875]
-
- # PG 1011
- P_target = [
- mvn_od(theta_PID, COV_PID, lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([0.05488, 0.1, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 0.05488, 0.05488]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 0.05488, 0.05488]),
- upper=np.log([0.1, 0.05488, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 1e-6, 0.05488]),
- upper=np.log([0.1, 0.1, 0.05488, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 0.05488, 1e-6]),
- upper=np.log([0.1, 0.1, 0.1, 0.05488]))[0], ]),
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 1e-6, 0.05488]),
- upper=np.log([0.1, 0.05488, 0.05488, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.1, 0.05488, 0.05488]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 0.05488, 1e-6]),
- upper=np.log([0.1, 0.05488, 0.1, 0.05488]))[0], ]),
- mvn_od(theta_PID, COV_PID, lower=np.log([0.05488, 1e-6, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.05488, 0.05488, 0.05488]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 0].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 0].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / realization_count
-
- assert_allclose(P_target, P_test, atol=0.05)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 1012
- P_target = [
- mvn_od(theta_PID, COV_PID, lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.05488, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 0.05488, 0.05488]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 0.05488, 0.05488]),
- upper=np.log([0.05488, 0.1, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 1e-6, 0.05488]),
- upper=np.log([0.1, 0.1, 0.05488, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 0.05488, 1e-6]),
- upper=np.log([0.1, 0.1, 0.1, 0.05488]))[0], ]),
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 1e-6, 0.05488]),
- upper=np.log([0.05488, 0.1, 0.05488, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.1, 0.05488, 0.05488]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 0.05488, 1e-6]),
- upper=np.log([0.05488, 0.1, 0.1, 0.05488]))[0], ]),
- mvn_od(theta_PID, COV_PID, lower=np.log([1e-6, 0.05488, 1e-6, 1e-6]),
- upper=np.log([0.05488, 0.1, 0.05488, 0.05488]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 1].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 1].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / realization_count
-
- assert_allclose(P_target, P_test, atol=0.05)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 1021
- P_target = [
- mvn_od(theta_PID, COV_PID, lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.1, 0.05488, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 0.05488, 0.05488]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 0.05488, 0.05488]),
- upper=np.log([0.05488, 0.1, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 0.05488, 0.05488]),
- upper=np.log([0.1, 0.05488, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 0.05488, 1e-6]),
- upper=np.log([0.1, 0.1, 0.1, 0.05488]))[0], ]),
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 1e-6, 0.05488, 0.05488]),
- upper=np.log([0.05488, 0.05488, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 0.05488, 1e-6]),
- upper=np.log([0.1, 0.05488, 0.1, 0.05488]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 0.05488, 1e-6]),
- upper=np.log([0.05488, 0.1, 0.1, 0.05488]))[0], ]),
- mvn_od(theta_PID, COV_PID, lower=np.log([1e-6, 1e-6, 0.05488, 1e-6]),
- upper=np.log([0.05488, 0.05488, 0.1, 0.05488]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 2].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 2].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / realization_count
-
- #print('------------------------')
- #print('P_target')
- #print(P_target)
- #print('------------------------')
-
- assert_allclose(P_target, P_test, atol=0.05)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 1022
- P_target = [
- mvn_od(theta_PID, COV_PID, lower=np.log([1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log([0.1, 0.1, 0.1, 0.05488]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 0.05488, 0.05488]),
- upper=np.log([0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 0.05488, 0.05488]),
- upper=np.log([0.05488, 0.1, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 0.05488, 0.05488]),
- upper=np.log([0.1, 0.05488, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 0.05488, 1e-6, 0.05488]),
- upper=np.log([0.1, 0.1, 0.05488, 0.1]))[0], ]),
- np.sum([
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 1e-6, 0.05488, 0.05488]),
- upper=np.log([0.05488, 0.05488, 0.1, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([0.05488, 1e-6, 1e-6, 0.05488]),
- upper=np.log([0.1, 0.05488, 0.05488, 0.1]))[0],
- mvn_od(theta_PID, COV_PID,
- lower=np.log([1e-6, 0.05488, 1e-6, 0.05488]),
- upper=np.log([0.05488, 0.1, 0.05488, 0.1]))[0], ]),
- mvn_od(theta_PID, COV_PID, lower=np.log([1e-6, 1e-6, 1e-6, 0.05488]),
- upper=np.log([0.05488, 0.05488, 0.05488, 0.1]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 3].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 5)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 3].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 5)]
- P_test = P_test[np.where(P_test > 5)]
- P_test = P_test / realization_count
-
- assert_allclose(P_target[:-1], P_test[:4], atol=0.05)
- assert_allclose(C_target[:-1], C_test[:4], rtol=0.001)
- assert_allclose(T_target[:-1], T_test[:4], rtol=0.001)
-
- # PG 2011
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, 9.80665, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, 9.80665, 9.80665, 0.1, 0.1, 0.1, 0.1]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 4].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 4].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / realization_count
-
- assert_allclose(P_target, P_test, atol=0.05)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 2012
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, 9.80665, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, 9.80665, 9.80665, 0.1, 0.1, 0.1, 0.1]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 5].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 5].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / realization_count
-
- assert_allclose(P_target[:4], P_test[:4], atol=0.05)
- assert_allclose(C_target[:4], C_test[:4], rtol=0.001)
- assert_allclose(T_target[:4], T_test[:4], rtol=0.001)
-
- # PG 2021
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 1e-6, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, 9.80665, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, 9.80665, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 6].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 6].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / realization_count
-
- assert_allclose(P_target, P_test, atol=0.05)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # PG 2022
- P_target = [
- mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- lower=np.log([1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, 9.80665, 0.1, 0.1, 0.1, 0.1]))[0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 9.80665, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, np.inf, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- np.sum([
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 1e-6, 9.80665, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, 9.80665, np.inf, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [9.80665, 1e-6, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [np.inf, 9.80665, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0],
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 9.80665, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, np.inf, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[
- 0], ]),
- mvn_od(np.log(EDP_theta_test), EDP_COV_test, lower=np.log(
- [1e-6, 1e-6, 1e-6, 9.80665, 1e-6, 1e-6, 1e-6, 1e-6]),
- upper=np.log(
- [9.80665, 9.80665, 9.80665, np.inf, 0.1, 0.1, 0.1, 0.1]))[0],
- ]
-
- C_test, P_test = np.unique(
- np.around(DV_COST.iloc[:, 7].values / 3., decimals=0) * 3.,
- return_counts=True)
- C_test = C_test[np.where(P_test > 10)]
- T_test, P_test = np.unique(np.around(DV_TIME.iloc[:, 7].values * 333.33333,
- decimals=0) / 333.33333,
- return_counts=True)
- T_test = T_test[np.where(P_test > 10)]
- P_test = P_test[np.where(P_test > 10)]
- P_test = P_test / realization_count
-
- assert_allclose(P_target, P_test, atol=0.05)
- assert_allclose(C_target, C_test, rtol=0.001)
- assert_allclose(T_target, T_test, rtol=0.001)
-
- # RED TAG
- RED_check = A._DV_dict['red_tag'].describe().T
- RED_check = (RED_check['mean'] * RED_check['count'] / realization_count).values
-
- assert_allclose(RED_check, DMG_ref, atol=0.02, rtol=0.10)
-
- DMG_on = np.where(A._DMG > 0.0)[0]
- RED_on = np.where(A._DV_dict['red_tag'] > 0.0)[0]
- assert_allclose(DMG_on, RED_on)
-
- # ------------------------------------------------------------------------
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- P_no_RED_target = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- upper=np.log(
- [9.80665, 9.80665, 9.80665, 9.80665, 0.05488,
- 0.05488, 0.05488, 0.05488]))[0]
- S = A._SUMMARY
- SD = S.describe().T
-
- P_no_RED_test = (1.0 - SD.loc[('red tagged', ''), 'mean']) * SD.loc[
- ('red tagged', ''), 'count'] / realization_count
-
- assert P_no_RED_target == pytest.approx(P_no_RED_test, abs=0.03)
-
-[docs]def test_FEMA_P58_Assessment_EDP_uncertainty_single_sample():
- """
- Perform a loss assessment with customized inputs that focus on testing the
- methods used to estimate the multivariate lognormal distribution of EDP
- values. Besides the fitting, this test also evaluates the propagation of
- EDP uncertainty through the analysis. Dispersions in other calculation
- parameters are reduced to negligible levels. This allows us to test the
- results against pre-defined reference values in spite of the randomness
- involved in the calculations.
- In this test we provide only one structural response result and see if it
- is properly handled as a deterministic value or a random EDP using the
- additional sources of uncertainty.
- """
- print()
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_6.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_6.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_EDP])
-
- thetas, betas = np.array([rv.theta for rv in RV_EDP]).T
- EDP_theta_test = thetas
- EDP_beta_test = betas
- EDP_theta_target = np.array(
- [7.634901, 6.85613, 11.685934, 10.565554,
- 0.061364, 0.048515, 0.033256, 0.020352])
- EDP_beta_target = EDP_theta_target * 1e-6
- assert_allclose(EDP_theta_test, EDP_theta_target, rtol=0.05)
- assert_allclose(EDP_beta_test, EDP_beta_target, rtol=0.1)
-
- assert RV_EDP[0].RV_set == None
-
- # ------------------------------------------------- perform the calculation
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- A.calculate_losses()
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- S = A._SUMMARY
- SD = S.describe().T
-
- P_no_RED_test = (1.0 - SD.loc[('red tagged', ''), 'mean']) * SD.loc[
- ('red tagged', ''), 'count'] / 10000.
-
- assert P_no_RED_test == 0.0
-
- # -------------------------------------------------------------------------
- # now do the same analysis, but consider additional uncertainty
- # -------------------------------------------------------------------------
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- AU = A._AIM_in['general']['added_uncertainty']
-
- AU['beta_m'] = 0.3
- AU['beta_gm'] = 0.4
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_EDP])
-
- thetas, betas = np.array([rv.theta for rv in RV_EDP]).T
- EDP_theta_test = thetas
- EDP_beta_test = betas
- EDP_beta_target = np.sqrt((EDP_theta_target * 1e-6)**2. +
- np.ones(8)*(0.3**2. + 0.4**2.))
- assert_allclose(EDP_theta_test, EDP_theta_target, rtol=0.05)
- assert_allclose(EDP_beta_test, EDP_beta_target, rtol=0.1)
-
- assert RV_EDP[0].RV_set == None
- EDP_rho_target = np.zeros((8, 8))
- np.fill_diagonal(EDP_rho_target, 1.0)
-
- EDP_COV_test = EDP_rho_target * np.outer(EDP_beta_test, EDP_beta_test)
-
- # ------------------------------------------------- perform the calculation
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- A.calculate_losses()
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- P_no_RED_target = mvn_od(np.log(EDP_theta_test), EDP_COV_test,
- upper=np.log(
- [9.80665, 9.80665, 9.80665, 9.80665, 0.05488,
- 0.05488, 0.05488, 0.05488]))[0]
-
- S = A._SUMMARY
- SD = S.describe().T
-
- P_no_RED_test = (1.0 - SD.loc[('red tagged', ''), 'mean']) * SD.loc[
- ('red tagged', ''), 'count'] / 10000.
-
- assert P_no_RED_target == pytest.approx(P_no_RED_test, abs=0.01)
-
-[docs]def test_FEMA_P58_Assessment_EDP_uncertainty_zero_variance():
- """
- Perform a loss assessment with customized inputs that focus on testing the
- methods used to estimate the multivariate lognormal distribution of EDP
- values. Besides the fitting, this test also evaluates the propagation of
- EDP uncertainty through the analysis. Dispersions in other calculation
- parameters are reduced to negligible levels. This allows us to test the
- results against pre-defined reference values in spite of the randomness
- involved in the calculations.
- This test simulates a scenario when one of the EDPs is identical in all
- of the available samples. This results in zero variance in that dimension
- and the purpose of the test is to ensure that such cases are handled
- appropriately.
- """
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_7.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_7.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # EDP
- RV_EDP = list(A._EDP_dict.values())
-
- assert np.all([rv.distribution == 'lognormal' for rv in RV_EDP])
-
- thetas, betas = np.array([rv.theta for rv in RV_EDP]).T
- EDP_theta_test = thetas
- EDP_beta_test = betas
- assert EDP_theta_test[4] == pytest.approx(0.061364, rel=0.05)
- assert EDP_beta_test[4] < 0.061364 * 1e-3
-
- rho = RV_EDP[0].RV_set.Rho()
- EDP_rho_test = rho
- EDP_rho_target = np.zeros((8, 8))
- np.fill_diagonal(EDP_rho_target, 1.0)
- assert_allclose(EDP_rho_test[4], EDP_rho_target[4], atol=1e-6)
-
- # ------------------------------------------------- perform the calculation
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- A.calculate_losses()
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- S = A._SUMMARY
- SD = S.describe().T
-
- P_no_RED_test = (1.0 - SD.loc[('red tagged', ''), 'mean']) * SD.loc[
- ('red tagged', ''), 'count'] / 10000.
-
- assert P_no_RED_test == 0.0
-
-[docs]def test_FEMA_P58_Assessment_QNT_uncertainty_independent():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component quantities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations.
- This test assumes that component quantities are independent.
- """
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_8.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_8.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A.define_random_variables()
-
- # -------------------------------------------------- check random variables
-
- # QNT
- RV_QNT = list(A._QNT_dict.values())
-
- QNT_theta_test, QNT_beta_test = np.array([rv.theta for rv in RV_QNT]).T
- QNT_theta_target = np.ones(8) * 25.
- QNT_beta_target = [25.0] * 4 + [0.4] * 4
- assert_allclose(QNT_theta_test, QNT_theta_target, rtol=0.001)
- assert_allclose(QNT_beta_test, QNT_beta_target, rtol=0.001)
-
- for i in range(4):
- assert RV_QNT[i].distribution == 'normal'
- for i in range(4, 8):
- assert RV_QNT[i].distribution == 'lognormal'
-
- QNT_rho_target = [
- [1, 0, 0, 0, 0, 0, 0, 0],
- [0, 1, 0, 0, 0, 0, 0, 0],
- [0, 0, 1, 0, 0, 0, 0, 0],
- [0, 0, 0, 1, 0, 0, 0, 0],
- [0, 0, 0, 0, 1, 0, 0, 0],
- [0, 0, 0, 0, 0, 1, 0, 0],
- [0, 0, 0, 0, 0, 0, 1, 0],
- [0, 0, 0, 0, 0, 0, 0, 1],
- ]
- QNT_rho_test = RV_QNT[0].RV_set.Rho()
- assert_allclose(QNT_rho_test, QNT_rho_target, atol=0.001)
-
- # ------------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # ------------------------------------------------ check damage calculation
-
- # COL
- # there shall be no collapses
- assert A._COL.describe().T['mean'].values == 0
-
- # DMG
- DMG_check = A._DMG.describe().T
- mu_test = DMG_check['mean']
- sig_test = DMG_check['std']
- rho_test = A._DMG.corr()
-
- mu_target_1 = 25.0 + 25.0 * norm.pdf(-1.0) / (1.0 - norm.cdf(-1.0))
- sig_target_1 = np.sqrt(25.0 ** 2.0 * (
- 1 - norm.pdf(-1.0) / (1.0 - norm.cdf(-1.0)) - (
- norm.pdf(-1.0) / (1.0 - norm.cdf(-1.0))) ** 2.0))
- mu_target_2 = np.exp(np.log(25.0) + 0.4 ** 2. / 2.)
- sig_target_2 = np.sqrt(
- (np.exp(0.4 ** 2.0) - 1.0) * np.exp(2 * np.log(25.0) + 0.4 ** 2.0))
-
- assert_allclose(mu_test[:4], mu_target_1, rtol=0.05)
- assert_allclose(mu_test[4:], mu_target_2, rtol=0.05)
- assert_allclose(sig_test[:4], sig_target_1, rtol=0.05)
- assert_allclose(sig_test[4:], sig_target_2, rtol=0.05)
- assert_allclose(rho_test, QNT_rho_target, atol=0.05)
-
- # ------------------------------------------------------------------------
-
- A.calculate_losses()
-
- # -------------------------------------------------- check loss calculation
-
- DV_COST = A._DV_dict['rec_cost'] / A._DMG
-
- rho_DV_target = [
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- ]
-
- assert_allclose(DV_COST.corr(), rho_DV_target, atol=0.05)
-
- # Uncertainty in decision variables is controlled by the correlation
- # between damages
- RND = [tnorm.rvs(-1., np.inf, loc=25, scale=25, size=10000) for i in
- range(4)]
- RND = np.sum(RND, axis=0)
- P_target_PID = np.sum(RND > 90.) / 10000.
- P_test_PID = np.sum(DV_COST.iloc[:, 0] < 10.01) / 10000.
- assert P_target_PID == pytest.approx(P_test_PID, rel=0.02)
-
- RND = [np.exp(norm.rvs(loc=np.log(25.), scale=0.4, size=10000)) for i in
- range(4)]
- RND = np.sum(RND, axis=0)
- P_target_PFA = np.sum(RND > 90.) / 10000.
- P_test_PFA = np.sum(DV_COST.iloc[:, 4] < 10.01) / 10000.
- assert P_target_PFA == pytest.approx(P_test_PFA, rel=0.02)
-
- # the same checks can be performed for reconstruction time
- DV_TIME = A._DV_dict['rec_time'] / A._DMG
-
- assert_allclose(DV_TIME.corr(), rho_DV_target, atol=0.05)
-
- P_test_PID = np.sum(DV_TIME.iloc[:, 0] < 0.0101) / 10000.
- assert P_target_PID == pytest.approx(P_test_PID, rel=0.02)
-
- P_test_PFA = np.sum(DV_TIME.iloc[:, 4] < 0.0101) / 10000.
- assert P_target_PFA == pytest.approx(P_test_PFA, rel=0.02)
-
- # injuries...
- DV_INJ_dict = deepcopy(A._DV_dict['injuries'])
- DV_INJ0 = (DV_INJ_dict[0] / A._DMG).describe()
- DV_INJ1 = (DV_INJ_dict[1] / A._DMG).describe()
-
- assert_allclose(DV_INJ0.loc['mean', :][:4], np.ones(4) * 0.025, rtol=0.001)
- assert_allclose(DV_INJ0.loc['mean', :][4:], np.ones(4) * 0.1, rtol=0.001)
- assert_allclose(DV_INJ1.loc['mean', :][:4], np.ones(4) * 0.005, rtol=0.001)
- assert_allclose(DV_INJ1.loc['mean', :][4:], np.ones(4) * 0.02, rtol=0.001)
-
- assert_allclose(DV_INJ0.loc['std', :], np.zeros(8), atol=1e-4)
- assert_allclose(DV_INJ1.loc['std', :], np.zeros(8), atol=1e-4)
-
- # and for red tag...
- # Since every component is damaged in every realization, the red tag
- # results should all be 1.0
- assert_allclose(A._DV_dict['red_tag'], np.ones((10000, 8)))
-
- # ------------------------------------------------------------------------
-
- A.aggregate_results()
-
- # ------------------------------------------------ check result aggregation
-
- S = A._SUMMARY
- SD = S.describe().T
-
- assert SD.loc[('inhabitants', ''), 'mean'] == 20.0
- assert SD.loc[('inhabitants', ''), 'std'] == 0.0
-
- assert SD.loc[('collapses', 'collapsed'), 'mean'] == 0.0
- assert SD.loc[('collapses', 'collapsed'), 'std'] == 0.0
-
- assert SD.loc[('red tagged', ''), 'mean'] == 1.0
- assert SD.loc[('red tagged', ''), 'std'] == 0.0
-
- assert np.corrcoef(S.loc[:, ('reconstruction', 'cost')],
- S.loc[:, ('reconstruction', 'time-sequential')])[
- 0, 1] == pytest.approx(1.0)
-
- assert_allclose(A._DV_dict['rec_cost'].sum(axis=1),
- S.loc[:, ('reconstruction', 'cost')])
- assert_allclose(A._DV_dict['rec_time'].sum(axis=1),
- S.loc[:, ('reconstruction', 'time-sequential')])
- assert_allclose(A._DV_dict['rec_time'].max(axis=1),
- S.loc[:, ('reconstruction', 'time-parallel')])
- assert_allclose(A._DV_dict['injuries'][0].sum(axis=1),
- S.loc[:, ('injuries', 'sev1')])
- assert_allclose(A._DV_dict['injuries'][1].sum(axis=1),
- S.loc[:, ('injuries', 'sev2')])
-
-[docs]def test_FEMA_P58_Assessment_QNT_uncertainty_dependencies():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component quantities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations.
- This test checks if dependencies between component quantities are handled
- appropriately.
- """
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_8.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_8.out"
-
- for dep in ['FG', 'PG', 'DIR', 'LOC']:
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A._AIM_in['dependencies']['quantities'] = dep
-
- A.define_random_variables()
-
- # ---------------------------------------------- check random variables
-
- # QNT
- RV_QNT = list(A._QNT_dict.values())
-
- QNT_theta_test, QNT_beta_test = np.array([rv.theta for rv in RV_QNT]).T
- QNT_theta_target = np.ones(8) * 25.
- QNT_beta_target = [25.0] * 4 + [0.4] * 4
- assert_allclose(QNT_theta_test, QNT_theta_target, rtol=0.001)
- assert_allclose(QNT_beta_test, QNT_beta_target, rtol=0.001)
-
- for i in range(4):
- assert RV_QNT[i].distribution == 'normal'
- for i in range(4, 8):
- assert RV_QNT[i].distribution == 'lognormal'
-
- if dep == 'FG':
- QNT_rho_target = np.array([
- [1, 1, 1, 1, 1, 1, 1, 1],
- [1, 1, 1, 1, 1, 1, 1, 1],
- [1, 1, 1, 1, 1, 1, 1, 1],
- [1, 1, 1, 1, 1, 1, 1, 1],
- [1, 1, 1, 1, 1, 1, 1, 1],
- [1, 1, 1, 1, 1, 1, 1, 1],
- [1, 1, 1, 1, 1, 1, 1, 1],
- [1, 1, 1, 1, 1, 1, 1, 1],
- ])
- elif dep == 'PG':
- QNT_rho_target = np.array([
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- ])
- elif dep == 'DIR':
- QNT_rho_target = np.array([
- [1, 1, 0, 0, 0, 0, 0, 0],
- [1, 1, 0, 0, 0, 0, 0, 0],
- [0, 0, 1, 1, 0, 0, 0, 0],
- [0, 0, 1, 1, 0, 0, 0, 0],
- [0, 0, 0, 0, 1, 1, 0, 0],
- [0, 0, 0, 0, 1, 1, 0, 0],
- [0, 0, 0, 0, 0, 0, 1, 1],
- [0, 0, 0, 0, 0, 0, 1, 1],
- ])
- elif dep == 'LOC':
- QNT_rho_target = np.array([
- [1, 0, 1, 0, 0, 0, 0, 0],
- [0, 1, 0, 1, 0, 0, 0, 0],
- [1, 0, 1, 0, 0, 0, 0, 0],
- [0, 1, 0, 1, 0, 0, 0, 0],
- [0, 0, 0, 0, 1, 0, 1, 0],
- [0, 0, 0, 0, 0, 1, 0, 1],
- [0, 0, 0, 0, 1, 0, 1, 0],
- [0, 0, 0, 0, 0, 1, 0, 1],
- ])
-
- QNT_rho_test = RV_QNT[0].RV_set.Rho()
- assert_allclose(QNT_rho_test, QNT_rho_target, atol=0.001)
-
- # ---------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # -------------------------------------------- check damage calculation
-
- # COL
- # there shall be no collapses
- assert A._COL.describe().T['mean'].values == 0
-
- # DMG
- # Because the correlations are enforced after truncation, the marginals
- # shall be unaffected by the correlation structure. Hence, the
- # distribution of damaged quantities within a PG shall be identical in
- # all dep cases.
- # The specified dependencies are apparent in the correlation between
- # damaged quantities in various PGs.
-
- DMG_check = A._DMG.describe().T
- mu_test = DMG_check['mean']
- sig_test = DMG_check['std']
- rho_test = A._DMG.corr()
-
- mu_target_1 = 25.0 + 25.0 * norm.pdf(-1.0) / (1.0 - norm.cdf(-1.0))
- sig_target_1 = np.sqrt(25.0 ** 2.0 * (
- 1 - norm.pdf(-1.0) / (1.0 - norm.cdf(-1.0)) - (
- norm.pdf(-1.0) / (1.0 - norm.cdf(-1.0))) ** 2.0))
- mu_target_2 = np.exp(np.log(25.0) + 0.4 ** 2. / 2.)
- sig_target_2 = np.sqrt(
- (np.exp(0.4 ** 2.0) - 1.0) * np.exp(2 * np.log(25.0) + 0.4 ** 2.0))
-
- assert_allclose(mu_test[:4], mu_target_1, rtol=0.05)
- assert_allclose(mu_test[4:], mu_target_2, rtol=0.05)
- assert_allclose(sig_test[:4], sig_target_1, rtol=0.05)
- assert_allclose(sig_test[4:], sig_target_2, rtol=0.05)
- assert_allclose(rho_test, QNT_rho_target, atol=0.05)
-
- # ---------------------------------------------------------------------
-
- A.calculate_losses()
-
- # ---------------------------------------------- check loss calculation
-
- DV_COST = A._DV_dict['rec_cost'] / A._DMG
-
- # After the DVs are normalized by the damaged quantities, the resulting
- # samples show the correlations between the DV_measure (such as
- # reconstruction cost) / 1 unit of damaged component. Because this
- # consequences are perfectly correlated among the components of a
- # fragility group by definition, the quadrants on the main diagonal
- # will follow the matrix presented below. If there are additional
- # correlations defined between component quantities in different
- # fragility groups (i.e. the off-diagonal quadrants of the rho matrix),
- # those will be preserved in the consequences. Therefore, the
- # off-diagonal quadrants need to be updated with those from QNT_rho_target
- # to get an appropriate rho_DV_target.
-
- rho_DV_target = np.array([
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [1, 1, 1, 1, 0, 0, 0, 0],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- [0, 0, 0, 0, 1, 1, 1, 1],
- ])
- rho_DV_target[:4, 4:] = QNT_rho_target[:4, 4:]
- rho_DV_target[4:, :4] = QNT_rho_target[:4, 4:]
-
- assert_allclose(DV_COST.corr(), rho_DV_target, atol=0.05)
-
- # uncertainty in decision variables is controlled by the correlation
- # between damages
- P_test_PID = np.sum(DV_COST.iloc[:, 0] < 10.01) / 10000.
- P_test_PFA = np.sum(DV_COST.iloc[:, 4] < 10.01) / 10000.
-
- # the first component quantities follow a truncated multivariate normal
- # distribution
- mu_target_PID = mu_target_1 * 4.
- sig_target_PID = np.sqrt(
- sig_target_1 ** 2. * np.sum(QNT_rho_target[:4, :4]))
- mu_target_PID_b = mu_target_PID
- sig_target_PID_b = sig_target_PID
- alpha = 100.
- i = 0
- while (np.log(
- np.abs(alpha / (mu_target_PID_b / sig_target_PID_b))) > 0.001) and (
- i < 10):
- alpha = -mu_target_PID_b / sig_target_PID_b
- mu_target_PID_b = mu_target_PID - sig_target_PID_b * norm.pdf(
- alpha) / (1.0 - norm.cdf(alpha))
- sig_target_PID_b = sig_target_PID / np.sqrt(
- (1.0 + alpha * norm.pdf(alpha) / (1.0 - norm.cdf(alpha))))
- i += 1
- xi = (90 - mu_target_PID_b) / sig_target_PID_b
- P_target_PID = 1.0 - (norm.cdf(xi) - norm.cdf(alpha)) / (
- 1.0 - norm.cdf(alpha))
-
- assert P_target_PID == pytest.approx(P_test_PID, rel=0.05)
-
- # the second component quantities follow a multivariate lognormal
- # distribution
- mu_target_PFA = mu_target_2 * 4.
- sig_target_PFA = np.sqrt(
- sig_target_2 ** 2. * np.sum(QNT_rho_target[4:, 4:]))
- sig_target_PFA_b = np.sqrt(
- np.log(sig_target_PFA ** 2.0 / mu_target_PFA ** 2.0 + 1.0))
- mu_target_PFA_b = np.log(mu_target_PFA) - sig_target_PFA_b ** 2.0 / 2.
- xi = np.log(90)
- P_target_PFA = 1.0 - norm.cdf(xi, loc=mu_target_PFA_b,
- scale=sig_target_PFA_b)
-
- assert P_target_PFA == pytest.approx(P_test_PFA, rel=0.05)
-
- # the same checks can be performed for reconstruction time
- DV_TIME = A._DV_dict['rec_time'] / A._DMG
-
- assert_allclose(DV_TIME.corr(), rho_DV_target, atol=0.05)
-
- P_test_PID = np.sum(DV_TIME.iloc[:, 0] < 0.0101) / 10000.
- assert P_target_PID == pytest.approx(P_test_PID, rel=0.05)
-
- P_test_PFA = np.sum(DV_TIME.iloc[:, 4] < 0.0101) / 10000.
- assert P_target_PFA == pytest.approx(P_test_PFA, rel=0.05)
-
- # injuries...
- # Every component is damaged in every realization in this test. Once
- # normalized by the quantity of components, the number of injuries
- # shall be identical and unaffected by the correlation between
- # component quantities.
-
- DV_INJ_dict = deepcopy(A._DV_dict['injuries'])
- DV_INJ0 = (DV_INJ_dict[0] / A._DMG).describe()
- DV_INJ1 = (DV_INJ_dict[1] / A._DMG).describe()
-
- assert_allclose(DV_INJ0.loc['mean', :][:4], np.ones(4) * 0.025,
- rtol=0.001)
- assert_allclose(DV_INJ0.loc['mean', :][4:], np.ones(4) * 0.1,
- rtol=0.001)
- assert_allclose(DV_INJ1.loc['mean', :][:4], np.ones(4) * 0.005,
- rtol=0.001)
- assert_allclose(DV_INJ1.loc['mean', :][4:], np.ones(4) * 0.02,
- rtol=0.001)
-
- assert_allclose(DV_INJ0.loc['std', :], np.zeros(8), atol=1e-4)
- assert_allclose(DV_INJ1.loc['std', :], np.zeros(8), atol=1e-4)
-
- # and for red tag...
- # since every component is damaged in every realization, the red tag
- # results should all be 1.0
- assert_allclose(A._DV_dict['red_tag'], np.ones((10000, 8)))
-
- # ---------------------------------------------------------------------
-
- A.aggregate_results()
-
- # -------------------------------------------- check result aggregation
-
- S = A._SUMMARY
- SD = S.describe().T
-
- assert SD.loc[('inhabitants', ''), 'mean'] == 20.0
- assert SD.loc[('inhabitants', ''), 'std'] == 0.0
-
- assert SD.loc[('collapses', 'collapsed'), 'mean'] == 0.0
- assert SD.loc[('collapses', 'collapsed'), 'std'] == 0.0
-
- assert SD.loc[('red tagged', ''), 'mean'] == 1.0
- assert SD.loc[('red tagged', ''), 'std'] == 0.0
-
- assert np.corrcoef(S.loc[:, ('reconstruction', 'cost')],
- S.loc[:, ('reconstruction', 'time-sequential')])[
- 0, 1] == pytest.approx(1.0)
-
- assert_allclose(A._DV_dict['rec_cost'].sum(axis=1),
- S.loc[:, ('reconstruction', 'cost')])
- assert_allclose(A._DV_dict['rec_time'].sum(axis=1),
- S.loc[:, ('reconstruction', 'time-sequential')])
- assert_allclose(A._DV_dict['rec_time'].max(axis=1),
- S.loc[:, ('reconstruction', 'time-parallel')])
- assert_allclose(A._DV_dict['injuries'][0].sum(axis=1),
- S.loc[:, ('injuries', 'sev1')])
- assert_allclose(A._DV_dict['injuries'][1].sum(axis=1),
- S.loc[:, ('injuries', 'sev2')])
-
-[docs]def test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies(dep='IND'):
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component fragilities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations. Component fragilites are assumed
- independent in this test.
-
- """
- print()
- idx = pd.IndexSlice
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_9.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_9.out"
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- A._AIM_in['dependencies']['fragilities'] = dep
-
- A.define_random_variables()
-
- # ---------------------------------------------- check random variables
- RV_FF = list(A._FF_dict.values())
- fr_names = np.unique([rv.name[3:12] for rv in RV_FF])
- fr_keys = {}
- for fr_name in fr_names:
- fr_list = [rv.name for rv in RV_FF if fr_name in rv.name]
- fr_keys.update({fr_name: fr_list})
-
- # fr_keys = []
- # for key in A._RV_dict.keys():
- # if 'FR' in key:
- # fr_keys.append(key)
-
- dimtag_target = [4 * 2 * 3, 20 * 2 * 3 * 3, 20 * 2 * 3 * 3,
- 20 * 2 * 3 * 3]
- theta_target = [[0.048, 0.096], [0.048, 0.072, 0.096],
- [2.9419, 5.8840, 11.7680], [2.9419, 5.8840, 11.7680]]
- sig_target = [[0.5, 0.25], [1.0, 0.5, 0.25], [1.0, 0.5, 0.25],
- [1.0, 0.5, 0.25]]
-
- if dep == 'IND':
- rho_target = np.zeros((24, 24))
- np.fill_diagonal(rho_target, 1.0)
-
- rho_sum = 360
-
- elif dep == 'PG':
- rho_target = np.ones((24, 24))
-
- rho_sum = 360 ** 2.
-
- elif dep == 'DIR':
- rho_target = [
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.]]
-
- rho_sum = (20 * 2 * 3) ** 2. * 3
-
- elif dep == 'LOC':
- rho_target = [
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1., 1.]]
-
- rho_sum = (20 * 3) ** 2. * (2 * 9)
-
- elif dep in ['ATC', 'CSG']:
- rho_target = [
- [1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.]]
-
- rho_sum = (20 * 3) ** 2. * (2 * 3)
-
- elif dep == 'DS':
- rho_target = [
- [1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.]]
-
- rho_sum = 3 ** 2 * (20 * 2 * 3)
-
- for k, key in enumerate(sorted(fr_keys.keys())):
- RV_FF_i = [A._FF_dict[rv_i] for rv_i in fr_keys[key]]
-
- assert len(RV_FF_i) == dimtag_target[k]
-
- FF_theta_test, FF_beta_test = np.array([rv.theta for rv in RV_FF_i]).T
-
- if k == 0:
- FF_theta_test = pd.DataFrame(
- np.reshape(FF_theta_test, (12, 2))).describe()
- FF_beta_test = pd.DataFrame(
- np.reshape(FF_beta_test, (12, 2))).describe()
- else:
- FF_theta_test = pd.DataFrame(
- np.reshape(FF_theta_test, (120, 3))).describe()
- FF_beta_test = pd.DataFrame(
- np.reshape(FF_beta_test, (120, 3))).describe()
-
- assert_allclose(FF_theta_test.loc['mean', :].values, theta_target[k],
- rtol=1e-4)
- assert_allclose(FF_theta_test.loc['std', :].values,
- np.zeros(np.array(theta_target[k]).shape),
- atol=1e-10)
-
- assert_allclose(FF_beta_test.loc['mean', :].values, sig_target[k],
- rtol=1e-4)
- assert_allclose(FF_beta_test.loc['std', :].values,
- np.zeros(np.array(sig_target[k]).shape), atol=1e-10)
-
- rho_test = RV_FF_i[0].RV_set.Rho(fr_keys[fr_names[k]])
- if k == 0:
- # we perform the detailed verification of rho for the first case
- # only (because the others are 360x360 matrices)
- assert_allclose(rho_test, rho_target)
-
- else:
- # for the other cases we check the number of ones in the matrix
- assert np.sum(rho_test) == rho_sum
-
- # ---------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # -------------------------------------------- check damage calculation
- # COL
- # there shall be no collapses
- assert A._COL.describe().T['mean'].values == 0
-
- # DMG
- DMG_check = A._DMG
-
- # start with checking the damage correlations
- for k in range(4):
- DMG_corr = DMG_check.loc[:, idx[k + 1, :, :]].corr()
-
- if k == 0:
- DMG_corr = DMG_corr.iloc[:8, :8]
-
- if dep in ['IND', 'ATC', 'CSG', 'DS']:
- DMG_corr_ref = np.array([
- [ 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 1.0,-0.1, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0],
- ])
- elif dep == 'PG':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1],
- [-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0],
- [ 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1],
- [-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0],
- [ 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1],
- [-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0],
- [ 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1],
- [-0.1, 1.0,-0.1, 1.0,-0.1, 1.0,-0.1, 1.0],
- ])
- elif dep == 'DIR':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0],
- [ 1.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 1.0,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 1.0],
- [ 0.0, 0.0, 0.0, 0.0, 1.0,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 1.0],
- ])
- elif dep == 'LOC':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1, 0.0, 0.0, 1.0,-0.1, 0.0, 0.0],
- [-0.1, 1.0, 0.0, 0.0,-0.1, 1.0, 0.0, 0.0],
- [ 0.0, 0.0, 1.0,-0.1, 0.0, 0.0, 1.0,-0.1],
- [ 0.0, 0.0,-0.1, 1.0, 0.0, 0.0,-0.1, 1.0],
- [ 1.0,-0.1, 0.0, 0.0, 1.0,-0.1, 0.0, 0.0],
- [-0.1, 1.0, 0.0, 0.0,-0.1, 1.0, 0.0, 0.0],
- [ 0.0, 0.0, 1.0,-0.1, 0.0, 0.0, 1.0,-0.1],
- [ 0.0, 0.0,-0.1, 1.0, 0.0, 0.0,-0.1, 1.0],
- ])
-
- if k == 1:
- DMG_corr = DMG_corr.iloc[:12, :12]
-
- if dep in ['IND', 'ATC', 'CSG', 'DS']:
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0],
- ])
- elif dep == 'PG':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1],
- [-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1],
- [-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1],
- [-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1],
- [-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1],
- [-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1],
- [-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1],
- [-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1],
- [-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0],
- ])
- elif dep == 'DIR':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1, 1.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 1.0,-0.1,-0.1, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1, 1.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 1.0,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0,-0.1,-0.1, 1.0],
- ])
- elif dep == 'LOC':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0],
- [-0.1,-0.1, 1.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1],
- [ 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0],
- [-0.1,-0.1, 1.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1],
- [ 0.0, 0.0, 0.0,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0],
- ])
-
- if k == 2:
- DMG_corr = DMG_corr.iloc[:20, :20]
-
- if dep in ['IND', 'DS']:
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1, 1.0,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep == 'PG':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 1.0, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1],
- [-0.1, 0.5, 1.0, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1],
- [-0.1, 0.5, 0.5, 1.0,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 1.0, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1],
- [-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 1.0, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1],
- [-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 1.0,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 1.0, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1],
- [-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 1.0, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1],
- [-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 1.0,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 1.0, 0.5, 0.5,-0.1],
- [-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 1.0, 0.5,-0.1],
- [-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 1.0,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep == 'DIR':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 1.0, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 0.5, 1.0,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 1.0, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 1.0, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.5, 0.5,-0.1,-0.1, 0.8, 0.5, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 1.0, 0.5,-0.1,-0.1, 0.5, 0.6, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 1.0,-0.1,-0.1, 0.5, 0.5, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.5, 0.5,-0.1,-0.1, 1.0, 0.5, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.6, 0.5,-0.1,-0.1, 0.5, 1.0, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 0.5,-0.1,-0.1, 0.5, 0.5, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep == 'LOC':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 1.0, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.6, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 0.5, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.5, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 1.0, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.6, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 0.6, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 1.0, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.5, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.6, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 1.0, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep in ['ATC', 'CSG']:
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 1.0, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.5, 0.5, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 1.0, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.5, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 1.0, 0.5,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.5, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 1.0, 0.5,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.5, 0.5, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
-
- if k == 3:
- DMG_corr = DMG_corr.iloc[:20, :20]
-
- if dep in ['IND', 'DS']:
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.0, 0.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.0, 1.0, 0.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.0, 0.0, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.0, 0.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.0, 1.0, 0.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.0, 0.0, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.0, 0.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.0, 1.0, 0.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.0, 0.0, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.0, 0.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.0, 1.0, 0.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.0, 0.0, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep == 'PG':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 1.0, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1],
- [-0.1, 0.8, 1.0, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1],
- [-0.1, 0.7, 0.6, 1.0,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 1.0, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1],
- [-0.1, 0.8, 0.6, 0.6,-0.1,-0.1, 0.8, 1.0, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1],
- [-0.1, 0.7, 0.6, 0.5,-0.1,-0.1, 0.7, 0.6, 1.0,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 1.0, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1],
- [-0.1, 0.8, 0.7, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1,-0.1, 0.8, 1.0, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1],
- [-0.1, 0.7, 0.6, 0.6,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1,-0.1, 0.7, 0.6, 1.0,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 1.0, 0.8, 0.7,-0.1],
- [-0.1, 0.8, 0.7, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1,-0.1, 0.8, 0.6, 0.6,-0.1,-0.1, 0.8, 1.0, 0.6,-0.1],
- [-0.1, 0.7, 0.6, 0.6,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1,-0.1, 0.7, 0.6, 0.5,-0.1,-0.1, 0.7, 0.6, 1.0,-0.1],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep == 'DIR':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 1.0, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.7, 0.6, 1.0,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 1.0, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 0.6, 0.6,-0.1,-0.1, 0.8, 1.0, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.7, 0.6, 0.5,-0.1,-0.1, 0.7, 0.6, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.8, 0.7,-0.1,-0.1, 0.8, 0.8, 0.7,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 1.0, 0.6,-0.1,-0.1, 0.8, 0.7, 0.6,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 1.0,-0.1,-0.1, 0.7, 0.6, 0.6,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.8, 0.7,-0.1,-0.1, 1.0, 0.8, 0.7,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.6, 0.6,-0.1,-0.1, 0.8, 1.0, 0.6,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 0.5,-0.1,-0.1, 0.7, 0.6, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep == 'LOC':
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 1.0, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.7, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.7, 0.6, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.8, 0.7,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 1.0, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.7, 0.6,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 0.6,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 0.7, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 1.0, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.7, 0.6, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.8, 0.7,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 0.7, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 1.0, 0.6,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
- elif dep in ['ATC', 'CSG']:
- DMG_corr_ref = np.array([
- [ 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 1.0, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.8, 1.0, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1, 0.7, 0.6, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 1.0, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.8, 0.7,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 1.0, 0.6,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 1.0,-0.1, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,-0.1,-0.1,-0.1,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 1.0, 0.8, 0.7,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.8, 1.0, 0.6,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1, 0.7, 0.6, 1.0,-0.1],
- [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,-0.1,-0.1,-0.1,-0.1, 1.0],
- ])
-
- for i in range(len(DMG_corr.index)):
- for j in range(len(DMG_corr.columns)):
- ref_i = DMG_corr_ref[i, j]
- if ref_i != 0.0:
- if ref_i > 0.0:
- assert DMG_corr.iloc[i, j] > 0.97 * ref_i
- else:
- assert DMG_corr.iloc[i, j] < 0.0
- else:
- assert DMG_corr.iloc[i, j] == pytest.approx(ref_i,
- abs=0.15)
-
- # then check the distribution of damage within each performance group
- EDP_list = np.array(
- [[[0.080000, 0.080000], [0.080000, 0.080000], [0.040000, 0.040000]],
- [[7.845320, 7.845320], [7.845320, 7.845320],
- [2.942000, 2.942000]]])
-
- fr_keys = []
- for key in A._RV_dict.keys():
- if 'FR' in key:
- fr_keys.append(key)
-
- for k, key in enumerate(sorted(fr_keys)):
- # print(key)
-
- RV_FR = A._RV_dict[key]
-
- # only third of the data is unique because of the 3 stories
- rel_len = int(len(RV_FR._dimension_tags) / 3)
-
- COV_test = RV_FR.COV[:rel_len, :rel_len]
- theta_test = RV_FR.theta[:rel_len]
-
- lims = np.unique(theta_test)
- ndims = len(lims)
- if k in [2, 3]:
- ndims += 2
-
- if (dep in ['DS', 'IND']) or k > 1:
- DMG_vals = [[[0., 5., 7.5, 12.5, 17.5, 20., 25.], [0., 25.]],
- [[0., 1.5, 3., 4.5, 6., 7.5, 9., 10.5, 12., 13.5,
- 15.,
- 16.5, 18., 19.5, 21., 22.5, 24., 25.5, 27., 28.5,
- 30.0],
- [0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.,
- 11., 12., 13., 14., 15., 16., 17., 18., 19.,
- 20.]]]
- else:
- DMG_vals = [[[0., 25.], [0., 25.]],
- [[0., 30.], [0., 20.]]]
- DMG_vals = np.array(DMG_vals)
-
- for story in [0, 1, 2]:
- for dir_ in [0, 1]:
- # print(story, dir_)
-
- idx = pd.IndexSlice
- DMG_check_FG = DMG_check.loc[:, idx[k + 1, :, :]]
- DMG_check_PG = DMG_check_FG.iloc[:,
- story * 2 * ndims + dir_ * ndims:story * 2 * ndims + (
- dir_ + 1) * ndims]
-
- DMG_val_test = np.unique(
- np.around(DMG_check_PG.values * 10., decimals=0) / 10.,
- return_counts=True)
- DMG_val_test = DMG_val_test[0][DMG_val_test[1] > 10]
-
- # only check at most the first 10 elements, because the
- # higher values have extremely low likelihood
- ddim = min(len(DMG_val_test), 10)
- DMG_val_ref = DMG_vals[np.sign(k), dir_]
- for v in DMG_val_test:
- assert v in DMG_val_ref
-
- # additional tests for mutually exclusive DS2 in FG3
- if (k == 2) and (dep not in ['DS', 'IND']):
- DMG_tot = [[0., 30.], [0., 20.]][dir_]
- DMG_DS2_test = DMG_check_PG.iloc[:, [1, 2, 3]].sum(
- axis=1)
-
- # the proportion of each DS in DS2 shall follow the
- # pre-assigned weights
- ME_test = \
- DMG_check_PG.iloc[DMG_DS2_test.values > 0].iloc[:,
- [1, 2, 3]].describe().T['mean'].values / DMG_tot[-1]
- assert_allclose(ME_test, [0.5, 0.3, 0.2], atol=0.01)
-
- # the sum of DMG with correlated CSGs shall be either 0.
- # or the total quantity
- DMG_DS2_test = np.unique(
- np.around(DMG_DS2_test * 10., decimals=0) / 10.,
- return_counts=True)
- DMG_DS2_test = DMG_DS2_test[0][DMG_DS2_test[1] > 10]
- assert_allclose(DMG_DS2_test, DMG_tot, atol=0.01)
-
- # additional tests for simultaneous DS2 in FG4
- if (k == 3) and (dep not in ['DS', 'IND']):
- DMG_tot = [30.0, 20.0][dir_]
- DMG_DS2_test = DMG_check_PG.iloc[:, [1, 2, 3]].sum(
- axis=1)
-
- # the proportion of each DS in DS2 shall follow the
- # pre-assigned weights considering replacement
- SIM_test = \
- DMG_check_PG.iloc[DMG_DS2_test.values > 0].iloc[:,
- [1, 2, 3]].describe().T['mean'].values / DMG_tot
- P_rep = 0.5 * 0.7 * 0.8
- SIM_ref = np.array([0.5, 0.3, 0.2]) * (
- 1.0 + P_rep / (1.0 - P_rep))
- assert_allclose(SIM_test, SIM_ref, atol=0.02)
-
- # the sum of DMG with correlated CSGs shall be either
- # 0. or more than the total quantity
- DMG_DS2_test = DMG_DS2_test.iloc[
- DMG_DS2_test.values > 0]
- # Even with perfect correlation, the generated random
- # samples will not be identical. Hence, one of the 20
- # CSGs in FG4, very rarely will belong to a different
- # DS than the rest. To avoid false negatives, we test
- # the third smallest value.
- assert DMG_DS2_test.sort_values().iloc[
- 2] >= DMG_tot * 0.99
- assert np.max(DMG_DS2_test.values) > DMG_tot
-
- # the first component has 3-1 CSGs in dir 1 and 2,
- # respectively
- if k == 0:
- dir_len = int(rel_len * 3 / 4)
- # the other components have 20-20 CSGs in dir 1 and 2,
- # respectively
- else:
- dir_len = int(rel_len / 2)
-
- if dir_ == 0:
- theta_t = theta_test[:dir_len]
- COV_t = COV_test[:dir_len, :dir_len]
-
- else:
- theta_t = theta_test[dir_len:]
- COV_t = COV_test[dir_len:, dir_len:]
-
- lim_ds1 = np.where(theta_t == lims[0])[0]
- lim_ds2 = np.where(theta_t == lims[1])[0]
- if k > 0:
- lim_ds3 = np.where(theta_t == lims[2])[0]
-
- ndim = len(theta_t)
-
- EDP = EDP_list[int(k > 1), story, dir_]*1.2
-
- DS_ref_all = []
- DS_ref_any = []
- DS_test_all = []
- DS_test_any = []
- # DS0
- DS_ref_all.append(mvn_od(np.log(theta_t), COV_t,
- lower=np.log(np.ones(ndim) * EDP),
- upper=np.ones(ndim) * np.inf)[0])
-
- if k == 0:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, 1] == 0.],
- axis=0)) / 10000.)
- elif k == 1:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, 1] == 0.,
- DMG_check_PG.iloc[:, 2] == 0.],
- axis=0)) / 10000.)
- else:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, 1] == 0.,
- DMG_check_PG.iloc[:, 2] == 0.,
- DMG_check_PG.iloc[:, 3] == 0.,
- DMG_check_PG.iloc[:, 4] == 0.],
- axis=0)) / 10000.)
-
- # DS1
- lower_lim = -np.ones(ndim) * np.inf
- upper_lim = np.ones(ndim) * np.inf
- lower_lim[lim_ds2] = np.log(EDP)
- upper_lim[lim_ds1] = np.log(EDP)
- if k > 0:
- lower_lim[lim_ds3] = np.log(EDP)
- DS_ref_all.append(mvn_od(np.log(theta_t), COV_t,
- lower=lower_lim, upper=upper_lim)[
- 0])
-
- lower_lim = -np.ones(ndim) * np.inf
- upper_lim = np.ones(ndim) * np.inf
- lower_lim[lim_ds2[0]] = np.log(EDP)
- upper_lim[lim_ds1[0]] = np.log(EDP)
- if k > 0:
- lower_lim[lim_ds3[0]] = np.log(EDP)
- P_any = mvn_od(np.log(theta_t), COV_t, lower=lower_lim,
- upper=upper_lim)[0]
- if (dep in ['DS', 'IND']):
- P_any = 1.0 - (1.0 - P_any) ** len(lim_ds1)
- DS_ref_any.append(P_any)
-
- if k == 0:
- DS_test_all.append(np.sum(np.all(
- [DMG_check_PG.iloc[:, 0] > DMG_val_ref[-1] - 0.1,
- DMG_check_PG.iloc[:, 1] == 0.], axis=0)) / 10000.)
- elif k == 1:
- DS_test_all.append(np.sum(np.all(
- [DMG_check_PG.iloc[:, 0] > DMG_val_ref[-1] - 0.1,
- DMG_check_PG.iloc[:, 1] == 0.,
- DMG_check_PG.iloc[:, 2] == 0.], axis=0)) / 10000.)
- else:
- DS_test_all.append(np.sum(np.all(
- [DMG_check_PG.iloc[:, 0] > DMG_val_ref[-1] - 0.1,
- DMG_check_PG.iloc[:, 1] == 0.,
- DMG_check_PG.iloc[:, 2] == 0.,
- DMG_check_PG.iloc[:, 3] == 0.,
- DMG_check_PG.iloc[:, 4] == 0.], axis=0)) / 10000.)
-
- DS_test_any.append(np.sum(
- np.all([DMG_check_PG.iloc[:, 0] > 0.],
- axis=0)) / 10000.)
-
- # DS2
- lower_lim = -np.ones(ndim) * np.inf
- upper_lim = np.ones(ndim) * np.inf
- upper_lim[lim_ds2] = np.log(EDP)
- if k > 0:
- lower_lim[lim_ds3] = np.log(EDP)
- if k < 3:
- DS_ref_all.append(mvn_od(np.log(theta_t), COV_t,
- lower=lower_lim,
- upper=upper_lim)[0])
- else:
- DS_ref_all.append(0.0)
-
- lower_lim = -np.ones(ndim) * np.inf
- upper_lim = np.ones(ndim) * np.inf
- upper_lim[lim_ds2[0]] = np.log(EDP)
- if k > 0:
- lower_lim[lim_ds3[0]] = np.log(EDP)
- P_any = mvn_od(np.log(theta_t), COV_t, lower=lower_lim,
- upper=upper_lim)[0]
- if (dep in ['DS', 'IND']):
- P_any = 1.0 - (1.0 - P_any) ** len(lim_ds1)
- DS_ref_any.append(P_any)
-
- if k == 0:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, 1] >
- DMG_val_ref[-1] - 0.1],
- axis=0)) / 10000.)
- elif k == 1:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, 1] >
- DMG_val_ref[-1] - 0.1,
- DMG_check_PG.iloc[:, 2] == 0.],
- axis=0)) / 10000.)
- elif k == 2:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, [1, 2, 3]].sum(
- axis=1) > DMG_val_ref[-1] - 0.1,
- DMG_check_PG.iloc[:, 4] == 0.],
- axis=0)) / 10000.)
- elif k == 3:
- # skip this case
- DS_test_all.append(0.0)
-
- if k < 2:
- DS_test_any.append(np.sum(
- np.all([DMG_check_PG.iloc[:, 1] > 0.],
- axis=0)) / 10000.)
- else:
- DS_test_any.append(np.sum(np.all(
- [DMG_check_PG.iloc[:, [1, 2, 3]].sum(axis=1) > 0.],
- axis=0)) / 10000.)
-
- # DS3
- if k > 0:
-
- lower_lim = -np.ones(ndim) * np.inf
- upper_lim = np.ones(ndim) * np.inf
- upper_lim[lim_ds3] = np.log(EDP)
- DS_ref_all.append(mvn_od(np.log(theta_t), COV_t,
- lower=lower_lim,
- upper=upper_lim)[0])
-
- lower_lim = -np.ones(ndim) * np.inf
- upper_lim = np.ones(ndim) * np.inf
- upper_lim[lim_ds3[0]] = np.log(EDP)
- P_any = mvn_od(np.log(theta_t), COV_t, lower=lower_lim,
- upper=upper_lim)[0]
- if (dep in ['DS', 'IND']):
- P_any = 1.0 - (1.0 - P_any) ** len(lim_ds1)
- DS_ref_any.append(P_any)
-
- if k == 1:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, 1] == 0.,
- DMG_check_PG.iloc[:, 2] >
- DMG_val_ref[-1] - 0.1],
- axis=0)) / 10000.)
- else:
- DS_test_all.append(
- np.sum(np.all([DMG_check_PG.iloc[:, 0] == 0.,
- DMG_check_PG.iloc[:, 1] == 0.,
- DMG_check_PG.iloc[:, 2] == 0.,
- DMG_check_PG.iloc[:, 3] == 0.,
- DMG_check_PG.iloc[:, 4] >
- DMG_val_ref[-1] - 0.1],
- axis=0)) / 10000.)
- if k == 1:
- DS_test_any.append(np.sum(
- np.all([DMG_check_PG.iloc[:, 2] > 0.],
- axis=0)) / 10000.)
-
- else:
- DS_test_any.append(np.sum(
- np.all([DMG_check_PG.iloc[:, 4] > 0.],
- axis=0)) / 10000.)
-
- assert_allclose(DS_ref_all, DS_test_all, atol=0.02)
- assert_allclose(DS_ref_any, DS_test_any, atol=0.02)
-
- # ---------------------------------------------------------------------
-
- A.calculate_losses()
-
- # ---------------------------------------------- check loss calculation
-
- # No additional uncertainty is introduced when it comes to losses in
- # this test. The decision variables and the damaged quantities shall
- # follow the same distribution and have the same correlation structure.
- # The damaged quantities have already been verified, so now we use them
- # as reference values for testing the decision variables.
-
- # COST and TIME and INJ
- DV_COST = A._DV_dict['rec_cost']
- DV_TIME = A._DV_dict['rec_time']
- DV_INJ_dict = deepcopy(A._DV_dict['injuries'])
- DV_INJ0 = DV_INJ_dict[0]
- DV_INJ1 = DV_INJ_dict[1]
-
- DMG_check = A._DMG
-
- for k in range(4):
- # Start with checking the correlations...
- dmg = DMG_check.loc[:, (DMG_check != 0.0).any(axis=0)]
- dmg_corr = dmg.loc[:, idx[k + 1, :, :]].corr()
- for dv in [DV_COST, DV_TIME, DV_INJ0, DV_INJ1]:
- dv = dv.loc[:, (dv != 0.0).any(axis=0)]
- dv_corr = dv.loc[:, idx[k + 1, :, :]].corr()
-
- assert_allclose(dmg_corr.values, dv_corr.values, atol=0.001)
-
- # then check the distribution.
- # After normalizing with the damaged quantities all decision
- # variables in a given DS shall have the same value.
- dv = ((dv / dmg).describe().T).fillna(0.0)
-
- assert_allclose(dv['std'], np.zeros(len(dv.index)), atol=1.0)
-
- # red tags require special checks
- for f, fg_id in enumerate(sorted(A._FG_dict.keys())):
- dims = [2, 3, 5, 5][f]
-
- # take the total quantity of each performance group
- FG = A._FG_dict[fg_id]
- qnt = []
- for PG in FG._performance_groups:
- if isinstance(PG._quantity, RandomVariable):
- qnt.append((PG._quantity.samples[:dims]).flatten())
- else:
- qnt.append(np.ones(dims) * PG._quantity)
- qnt = np.array(qnt).flatten()
-
- # flag the samples where the damage exceeds the pre-defined limit
- # for red tagging
- dmg = DMG_check.loc[:, idx[FG._ID, :, :]]
- red_ref = dmg > 0.489 * qnt
-
- # collect the red tag results from the analysis
- red_test = A._DV_dict['red_tag'].loc[:, idx[FG._ID, :, :]]
-
- # compare
- red_diff = (red_ref - red_test).describe().T
- assert_allclose(red_diff['mean'].values, 0.)
- assert_allclose(red_diff['std'].values, 0.)
-
- # ---------------------------------------------------------------------
-
- A.aggregate_results()
-
- # -------------------------------------------- check result aggregation
-
- # Aggregate results are checked in detail by other tests.
- # Here we only focus on some simple checks to make sure the results
- # make sense.
-
- S = A._SUMMARY
- SD = S.describe().T
-
- assert SD.loc[('inhabitants', ''), 'mean'] == 10.0
- assert SD.loc[('inhabitants', ''), 'std'] == 0.0
-
- assert SD.loc[('collapses', 'collapsed'), 'mean'] == 0.0
- assert SD.loc[('collapses', 'collapsed'), 'std'] == 0.0
-
- assert_allclose(A._DV_dict['rec_cost'].sum(axis=1),
- S.loc[:, ('reconstruction', 'cost')])
- assert_allclose(A._DV_dict['rec_time'].sum(axis=1),
- S.loc[:, ('reconstruction', 'time-sequential')])
- assert_allclose(A._DV_dict['rec_time'].max(axis=1),
- S.loc[:, ('reconstruction', 'time-parallel')])
- assert_allclose(A._DV_dict['injuries'][0].sum(axis=1),
- S.loc[:, ('injuries', 'sev1')])
- assert_allclose(A._DV_dict['injuries'][1].sum(axis=1),
- S.loc[:, ('injuries', 'sev2')])
-
-[docs]def test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies_PG():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component fragilities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations. Component fragilites are assumed
- perfectly correlated between performance groups in this test.
-
- """
-
- test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies('PG')
-
-[docs]def test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies_DIR():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component fragilities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations. Component fragilites are assumed
- perfectly correlated between performance groups controlled by the same EDPs
- in identical directions in this test.
-
- """
-
- test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies('DIR')
-
-[docs]def test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies_LOC():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component fragilities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations. Component fragilites are assumed
- perfectly correlated between performance groups at identical locations in
- this test.
-
- """
-
- test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies('LOC')
-
-[docs]def test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies_ATC():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component fragilities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations. Component fragilites are assumed
- perfectly correlated within performance groups if such correlation is
- prescribed by ATC in the FEMA P58 document.
-
- """
-
- test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies('ATC')
-
-[docs]def test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies_CSG():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component fragilities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations. Component fragilites are assumed
- perfectly correlated within a performance group in this test.
-
- """
-
- test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies('CSG')
-
-[docs]def test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies_DS():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in component fragilities. Dispersions in other
- calculation parameters are reduced to negligible levels. This allows us to
- test the results against pre-defined reference values in spite of the
- randomness involved in the calculations. Component fragilites are assumed
- perfectly correlated between damage states in this test.
-
- """
-
- test_FEMA_P58_Assessment_FRAG_uncertainty_dependencies('DS')
-
-[docs]def test_FEMA_P58_Assessment_DV_uncertainty_dependencies():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in consequence functions and decision variables.
- Dispersions in other calculation parameters are reduced to negligible
- levels. This allows us to test the results against pre-defined reference
- values in spite of the randomness involved in the calculations.
- """
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_10.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_10.out"
-
- dep_list = ['IND', 'FG', 'PG', 'DIR', 'LOC', 'DS']
-
- for d in range(7):
-
- if d > 0:
- dep_COST = dep_list[[0, 1, 2, 3, 4, 5][d - 1]]
- dep_TIME = dep_list[[1, 2, 3, 4, 5, 0][d - 1]]
- dep_RED = dep_list[[2, 3, 4, 5, 0, 1][d - 1]]
- dep_INJ = dep_list[[3, 4, 5, 0, 1, 2][d - 1]]
- else:
- dep_COST = np.random.choice(dep_list)
- dep_TIME = np.random.choice(dep_list)
- dep_RED = np.random.choice(dep_list)
- dep_INJ = np.random.choice(dep_list)
- dep_CT = np.random.choice([True, False])
- dep_ILVL = np.random.choice([True, False])
-
- #print([dep_COST, dep_TIME, dep_RED, dep_INJ, dep_CT, dep_ILVL], end=' ')
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- # set the dependencies
- A._AIM_in['dependencies']['rec_costs'] = dep_COST
- A._AIM_in['dependencies']['rec_times'] = dep_TIME
- A._AIM_in['dependencies']['red_tags'] = dep_RED
- A._AIM_in['dependencies']['injuries'] = dep_INJ
-
- A._AIM_in['dependencies']['cost_and_time'] = dep_CT
- A._AIM_in['dependencies']['injury_lvls'] = dep_ILVL
-
- A.define_random_variables()
-
- # ---------------------------------------------- check random variables
-
- rho_ref = dict(
- IND=np.zeros((16, 16)),
- FG=np.ones((16, 16)),
- PG=np.array([
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- ]),
- LOC=np.array([
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- ]),
- DIR=np.array([
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- ]),
- DS=np.array([
- [1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.],
- ])
- )
- np.fill_diagonal(rho_ref['IND'], 1.0)
-
- # RV_REP = deepcopy(A._RV_dict['DV_REP'])
- # RV_RED = deepcopy(A._RV_dict['DV_RED'])
- # RV_INJ = deepcopy(A._RV_dict['DV_INJ'])
-
- RV_REP = list(A._DV_REP_dict.values())
- RV_RED = list(A._DV_RED_dict.values())
- RV_INJ = list(A._DV_INJ_dict.values())
-
- for r, (RV_DV, RV_tag) in enumerate(
- zip([RV_REP, RV_RED, RV_INJ], ['rep', 'red', 'inj'])):
-
- # assert len(RV_DV._dimension_tags) == [32, 16, 32][r]
- assert len(RV_DV) == [32, 16, 32][r]
-
- DV_theta_test, DV_beta_test = np.array([rv.theta for rv in RV_DV]).T
- DV_rho_test = RV_DV[0].RV_set.Rho([rv.name for rv in RV_DV])
-
- # COV_test = RV_DV.COV
- # sig_test = np.sqrt(np.diagonal(COV_test))
- # rho_test = COV_test / np.outer(sig_test, sig_test)
-
- if RV_tag == 'rep':
-
- assert_allclose(DV_theta_test, np.ones(32))
- assert_allclose(DV_beta_test, np.array(
- [0.31, 0.71] * 8 + [0.32, 0.72] * 8))
-
- if dep_CT == True:
- if (((dep_COST == 'LOC') and (dep_TIME == 'DIR')) or
- ((dep_COST == 'DIR') and (dep_TIME == 'LOC'))):
- rho_ref_CT = rho_ref['PG']
- else:
- rho_ref_CT = np.maximum(rho_ref[dep_COST],
- rho_ref[dep_TIME])
-
- assert_allclose(DV_rho_test[:16, :16], rho_ref_CT)
- assert_allclose(DV_rho_test[16:, 16:], rho_ref_CT)
- assert_allclose(DV_rho_test[:16, 16:], rho_ref_CT)
- assert_allclose(DV_rho_test[16:, :16], rho_ref_CT)
-
- else:
- assert_allclose(DV_rho_test[:16, :16], rho_ref[dep_COST])
- assert_allclose(DV_rho_test[16:, 16:], rho_ref[dep_TIME])
- assert_allclose(DV_rho_test[:16, 16:], np.zeros((16, 16)))
- assert_allclose(DV_rho_test[16:, :16], np.zeros((16, 16)))
-
- elif RV_tag == 'red':
-
- assert_allclose(DV_theta_test, np.ones(16))
- assert_allclose(DV_beta_test, np.array([0.33, 0.73] * 8))
-
- assert_allclose(DV_rho_test, rho_ref[dep_RED])
-
- elif RV_tag == 'inj':
-
- assert_allclose(DV_theta_test, np.ones(32))
- assert_allclose(DV_beta_test, np.array(
- [0.34, 0.74] * 8 + [0.35, 0.75] * 8))
-
- if dep_ILVL == True:
- assert_allclose(DV_rho_test[:16, :16], rho_ref[dep_INJ])
- assert_allclose(DV_rho_test[16:, 16:], rho_ref[dep_INJ])
- assert_allclose(DV_rho_test[:16, 16:], rho_ref[dep_INJ])
- assert_allclose(DV_rho_test[16:, :16], rho_ref[dep_INJ])
- else:
- assert_allclose(DV_rho_test[:16, :16], rho_ref[dep_INJ])
- assert_allclose(DV_rho_test[16:, 16:], rho_ref[dep_INJ])
- assert_allclose(DV_rho_test[:16, 16:], np.zeros((16, 16)))
- assert_allclose(DV_rho_test[16:, :16], np.zeros((16, 16)))
-
- # ---------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # -------------------------------------------- check damage calculation
-
- # COL
- # there shall be no collapses
- assert A._COL.describe().T['mean'].values == 0
-
- # DMG
- DMG_check = A._DMG
-
- # Fragilities are not tested here, so we only do a few simple checks
- assert np.min(DMG_check.describe().loc['mean'].values) > 0.
- assert np.min(DMG_check.describe().loc['std'].values) > 0.
-
- # ---------------------------------------------------------------------
-
- A.calculate_losses()
-
- # ---------------------------------------------- check loss calculation
-
- # COST and TIME and INJ
- DV_COST = A._DV_dict['rec_cost'] / DMG_check
- DV_TIME = A._DV_dict['rec_time'] / DMG_check
- DV_INJ_dict = deepcopy(A._DV_dict['injuries'])
- DV_INJ0 = DV_INJ_dict[0] / DMG_check
- DV_INJ1 = DV_INJ_dict[1] / DMG_check
-
- for dv_i, (DV, DV_tag) in enumerate(
- zip([DV_COST, DV_TIME, DV_INJ0, DV_INJ1],
- ['cost', 'time', 'inj0', 'inj1'])):
-
- DV_desc = DV.describe().T
- DV_desc_log = np.log(DV).describe().T
-
- if DV_tag == 'cost':
-
- # cost consequences in DS1 are lognormal
- mu_ds1_ref = np.exp(np.log(10.) + 0.31 ** 2. / 2.)
- sig_ds1_ref = np.sqrt(
- np.exp(2 * np.log(10.) + 0.31 ** 2.) * (
- np.exp(0.31 ** 2.) - 1.))
- assert_allclose(DV_desc['mean'].values[::2], mu_ds1_ref,
- rtol=0.02)
- assert_allclose(DV_desc['std'].values[::2], sig_ds1_ref,
- rtol=0.10)
- assert_allclose(DV_desc_log['mean'].values[::2],
- np.log(10.), atol=0.02)
- assert_allclose(DV_desc_log['std'].values[::2], 0.31,
- rtol=0.10)
-
- # cost consequences in DS2 are (truncated) normal
- mu_ds2_ref, var_ds2_ref = tnorm.stats(-1. / 0.71, 1000.,
- loc=1000., scale=710.,
- moments='mv')
- sig_ds2_ref = np.sqrt(var_ds2_ref)
- assert_allclose(DV_desc['mean'].values[1::2], mu_ds2_ref,
- rtol=0.05)
- assert_allclose(DV_desc['std'].values[1::2], sig_ds2_ref,
- rtol=0.10)
-
- # make sure that all damages correspond to positive
- # reconstruction costs
- assert np.all(np.min(DV) > 0.)
-
- elif DV_tag == 'time':
-
- # cost consequences in DS1 are (truncated) normal for FG1 and
- # lognormal for FG2
- # DS1 - FG1
- mu_ds1_ref, var_ds1_ref = tnorm.stats(-1. / 0.32, 1000.,
- loc=0.01,
- scale=0.0032,
- moments='mv')
- sig_ds1_ref = np.sqrt(var_ds1_ref)
- assert_allclose(DV_desc['mean'].values[::2][:4], mu_ds1_ref,
- rtol=0.02)
- assert_allclose(DV_desc['std'].values[::2][:4], sig_ds1_ref,
- rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[::2][:4]) == pytest.approx(
- sig_ds1_ref, rel=0.1)
-
- # DS1 - FG2
- mu_ds1_ref = np.exp(np.log(0.01) + 0.32 ** 2. / 2.)
- sig_ds1_ref = np.sqrt(
- np.exp(2 * np.log(0.01) + 0.32 ** 2.) * (
- np.exp(0.32 ** 2.) - 1.))
- assert_allclose(DV_desc['mean'].values[::2][4:], mu_ds1_ref,
- rtol=0.02)
- assert_allclose(DV_desc['std'].values[::2][4:], sig_ds1_ref,
- rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[::2][4:]) == pytest.approx(
- sig_ds1_ref, rel=0.1)
- assert_allclose(DV_desc_log['mean'].values[::2][4:],
- np.log(0.01), atol=0.02)
- assert_allclose(DV_desc_log['std'].values[::2][4:], 0.32,
- rtol=0.20)
- assert np.mean(
- DV_desc_log['std'].values[::2][4:]) == pytest.approx(
- 0.32, rel=0.1)
-
- # cost consequences in DS2 are lognormal for FG1 and
- # (truncated) normal for FG2
- # DS2 - FG1
- mu_ds2_ref = np.exp(np.log(1.) + 0.72 ** 2. / 2.)
- sig_ds2_ref = np.sqrt(
- np.exp(2 * np.log(1.) + 0.72 ** 2.) * (
- np.exp(0.72 ** 2.) - 1.))
- assert_allclose(DV_desc['mean'].values[1::2][:4],
- mu_ds2_ref, rtol=0.05)
- assert_allclose(DV_desc['std'].values[1::2][:4],
- sig_ds2_ref, rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[1::2][:4]) == pytest.approx(
- sig_ds2_ref, rel=0.1)
-
- assert_allclose(DV_desc_log['mean'].values[1::2][:4],
- np.log(1.), atol=0.05)
- assert_allclose(DV_desc_log['std'].values[1::2][:4], 0.72,
- rtol=0.20)
- assert np.mean(
- DV_desc_log['std'].values[1::2][:4]) == pytest.approx(
- 0.72, rel=0.1)
-
- # DS2 - FG2
- mu_ds2_ref, var_ds2_ref = tnorm.stats(-1. / 0.72, 1000.,
- loc=1., scale=0.72,
- moments='mv')
- sig_ds2_ref = np.sqrt(var_ds2_ref)
- assert_allclose(DV_desc['mean'].values[1::2][4:],
- mu_ds2_ref, rtol=0.05)
- assert_allclose(DV_desc['std'].values[1::2][4:],
- sig_ds2_ref, rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[1::2][4:]) == pytest.approx(
- sig_ds2_ref, rel=0.1)
-
- # make sure that all damages correspond to positive
- # reconstruction time
- assert np.all(np.min(DV) > 0.)
-
- elif DV_tag in ['inj0', 'inj1']:
-
- # Injuries follow a truncated normal distribution in all cases
- # The beta values provided are coefficients of variation of the
- # non-truncated distribution. These provide the reference mean
- # and standard deviation values for the truncated case.
- mu_ds1, mu_ds2 = {'inj0': [0.5, 0.6], 'inj1': [0.1, 0.2]}[
- DV_tag]
- beta_ds1, beta_ds2 = \
- {'inj0': [0.34, 0.74], 'inj1': [0.35, 0.75]}[DV_tag]
-
- # DS1
- # The affected population in DS1 per unit quantity (identical
- # for all FGs and injury levels)
- p_aff = 0.05
-
- mu_ref, var_ref = tnorm.stats(-1. / beta_ds1, (
- 1. - mu_ds1) / mu_ds1 / beta_ds1, loc=mu_ds1,
- scale=mu_ds1 * beta_ds1,
- moments='mv')
- sig_ref = np.sqrt(var_ref)
- assert_allclose(DV_desc['mean'].values[::2], mu_ref * p_aff,
- rtol=beta_ds1 / 10.)
- assert_allclose(DV_desc['std'].values[::2], sig_ref * p_aff,
- rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[::2]) == pytest.approx(
- sig_ref * p_aff, rel=0.1)
-
- # DS2
- # the affected population in DS1 per unit quantity (identical
- # for all FGs and injury levels)
- p_aff = 0.1
- mu_ref, var_ref = tnorm.stats(-1. / beta_ds2, (
- 1. - mu_ds2) / mu_ds2 / beta_ds2, loc=mu_ds2,
- scale=mu_ds2 * beta_ds2,
- moments='mv')
- sig_ref = np.sqrt(var_ref)
- assert_allclose(DV_desc['mean'].values[1::2],
- mu_ref * p_aff, rtol=beta_ds2 / 10.)
- assert_allclose(DV_desc['std'].values[1::2],
- sig_ref * p_aff, rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[1::2]) == pytest.approx(
- sig_ref * p_aff, rel=0.1)
-
- # red tags have to be treated separately
- DV_RED = A._DV_dict['red_tag']
-
- DMG_norm = DMG_check / 25.
-
- for i in range(16):
-
- is_dam = pd.DataFrame(np.zeros((len(DMG_norm.index), 5)),
- columns=range(5))
- is_dam[0] = (DMG_norm.iloc[:, i] < 0.01)
- is_dam[1] = (DMG_norm.iloc[:, i] > 0.01) & (
- DMG_norm.iloc[:, i] < 0.275)
- is_dam[2] = (DMG_norm.iloc[:, i] > 0.275) & (
- DMG_norm.iloc[:, i] < 0.525)
- is_dam[3] = (DMG_norm.iloc[:, i] > 0.525) & (
- DMG_norm.iloc[:, i] < 0.775)
- is_dam[4] = (DMG_norm.iloc[:, i] > 0.775)
-
- mu_red = ([0.87, 0.23185] * 4 + [0.50, 0.23185] * 4)[i]
- beta_red = ([0.33, 0.73] * 8)[i]
- mu_ref = np.zeros(5)
- mu_ref[1] = tnorm.cdf(0.25, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
- mu_ref[2] = tnorm.cdf(0.50, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
- mu_ref[3] = tnorm.cdf(0.75, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
- mu_ref[4] = tnorm.cdf(1.00, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
-
- sample_count = np.array(
- [(DV_RED.iloc[:, i])[is_dam[c]].describe().loc['count'] for
- c in range(5)])
- mu_test = np.array(
- [(DV_RED.iloc[:, i])[is_dam[c]].describe().loc['mean'] for c
- in range(5)])
-
- assert mu_test[0] == 0.
- for step in range(1, 5):
- if sample_count[step] > 0:
- assert mu_test[step] == pytest.approx(
- mu_ref[step],
- abs=5 * 0.4 / np.sqrt(sample_count[step]))
-
- # CORRELATIONS
-
- # repair and injury correlations
- DV_REP = pd.concat([DV_COST, DV_TIME], axis=1)
- DV_INJ = pd.concat([DV_INJ0, DV_INJ1], axis=1)
-
- for DV, RV, dv_tag in zip([DV_REP, DV_INJ, DV_RED],
- [RV_REP, RV_INJ, RV_RED],
- ['rep', 'inj', 'red']):
-
- if dv_tag == 'rep':
- # transform the lognormal variables to log scale
- log_flags = ([True, False] * 8 +
- [False, True] * 4 +
- [True, False] * 4)
- for c, is_log in enumerate(log_flags):
- if is_log:
- DV.iloc[:, c] = np.log(DV.iloc[:, c])
-
- elif dv_tag == 'red':
- DV_RED_n = pd.DataFrame(np.ones(DV.shape) * np.nan,
- index=DV.index, columns=DV.columns)
- DMG_filter = pd.concat(
- [(DMG_check.iloc[:, [0, 2, 4, 6]] / 25.0 > 0.525) & (
- DMG_check.iloc[:, [0, 2, 4, 6]] / 25.0 < 0.775),
- (DMG_check.iloc[:, [1, 3, 5, 7]] / 25.0 > 0.025) & (
- DMG_check.iloc[:, [1, 3, 5, 7]] / 25.0 < 0.275),
- (DMG_check.iloc[:, [8, 10, 12, 14]] / 25.0 > 0.275) & (
- DMG_check.iloc[:, [8, 10, 12, 14]] / 25.0 < 0.525),
- (DMG_check.iloc[:, [9, 11, 13, 15]] / 25.0 > 0.025) & (
- DMG_check.iloc[:,
- [9, 11, 13, 15]] / 25.0 < 0.275)], axis=1)
-
- DV_RED_n[DMG_filter] = DV_RED[DMG_filter]
- DV = DV_RED_n
-
- DV_corr = DV.corr()
-
- # use the correlations specified for the random variable as
- # reference (that we already verified earlier)
- # COV_ref = RV.COV
- # sig_ref = np.sqrt(np.diagonal(COV_ref))
- # rho_ref = COV_ref / np.outer(sig_ref, sig_ref)
-
- rho_ref = RV[0].RV_set.Rho([rv.name for rv in RV])
-
- # perform the tests
- for i in range(len(DV_corr.index)):
- for j in range(len(DV_corr.columns)):
- ref_i = rho_ref[i, j]
- if ref_i != 0.0:
- if ref_i > 0.0:
- assert DV_corr.iloc[i, j] > 0.97 * ref_i
- else:
- assert DV_corr.iloc[i, j] < 0.0
- else:
- assert DV_corr.iloc[i, j] == pytest.approx(ref_i,
- abs=0.15)
-
- # ---------------------------------------------------------------------
-
- A.aggregate_results()
-
- # -------------------------------------------- check result aggregation
-
- # Aggregate results are checked in detail by other tests.
- # Here we only focus on some simple checks to make sure the results
- # make sense.
-
- S = A._SUMMARY
- SD = S.describe().T
-
- assert SD.loc[('inhabitants', ''), 'mean'] == 20.0
- assert SD.loc[('inhabitants', ''), 'std'] == 0.0
-
- assert SD.loc[('collapses', 'collapsed'), 'mean'] == 0.0
- assert SD.loc[('collapses', 'collapsed'), 'std'] == 0.0
-
- assert_allclose(A._DV_dict['rec_cost'].sum(axis=1),
- S.loc[:, ('reconstruction', 'cost')])
- assert_allclose(A._DV_dict['rec_time'].sum(axis=1),
- S.loc[:, ('reconstruction', 'time-sequential')])
- assert_allclose(A._DV_dict['rec_time'].max(axis=1),
- S.loc[:, ('reconstruction', 'time-parallel')])
- assert_allclose(A._DV_dict['injuries'][0].sum(axis=1),
- S.loc[:, ('injuries', 'sev1')])
- assert_allclose(A._DV_dict['injuries'][1].sum(axis=1),
- S.loc[:, ('injuries', 'sev2')])
-
- #print()
-
-[docs]def test_FEMA_P58_Assessment_DV_uncertainty_dependencies_with_partial_DV_data():
- """
- Perform loss assessment with customized inputs that focus on testing the
- propagation of uncertainty in consequence functions and decision variables
- when not every component has injury and red tag consequences assigned to it.
- Dispersions in other calculation parameters are reduced to negligible
- levels. This allows us to test the results against pre-defined reference
- values in spite of the randomness involved in the calculations.
- """
-
- base_input_path = 'resources/'
-
- DL_input = base_input_path + 'input data/' + "DL_input_test_11.json"
- EDP_input = base_input_path + 'EDP data/' + "EDP_table_test_11.out"
-
- dep_list = ['IND', 'FG', 'PG', 'DIR', 'LOC', 'DS']
-
- for d in range(7):
-
- if d > 0:
- dep_COST = dep_list[[0, 1, 2, 3, 4, 5][d - 1]]
- dep_TIME = dep_list[[1, 2, 3, 4, 5, 0][d - 1]]
- dep_RED = dep_list[[2, 3, 4, 5, 0, 1][d - 1]]
- dep_INJ = dep_list[[3, 4, 5, 0, 1, 2][d - 1]]
- else:
- dep_COST = np.random.choice(dep_list)
- dep_TIME = np.random.choice(dep_list)
- dep_RED = np.random.choice(dep_list)
- dep_INJ = np.random.choice(dep_list)
- dep_CT = np.random.choice([True, False])
- dep_ILVL = np.random.choice([True, False])
-
- # print([dep_COST, dep_TIME, dep_RED, dep_INJ, dep_CT, dep_ILVL], end=' ')
-
- A = FEMA_P58_Assessment()
-
- A.read_inputs(DL_input, EDP_input, verbose=False)
-
- # set the dependencies
- A._AIM_in['dependencies']['rec_costs'] = dep_COST
- A._AIM_in['dependencies']['rec_times'] = dep_TIME
- A._AIM_in['dependencies']['red_tags'] = dep_RED
- A._AIM_in['dependencies']['injuries'] = dep_INJ
-
- A._AIM_in['dependencies']['cost_and_time'] = dep_CT
- A._AIM_in['dependencies']['injury_lvls'] = dep_ILVL
-
- A.define_random_variables()
-
- # ---------------------------------------------- check random variables
-
- rho_ref = dict(
- IND=np.zeros((16, 16)),
- FG=np.ones((16, 16)),
- PG=np.array([
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1.],
- ]),
- LOC=np.array([
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 1.],
- ]),
- DIR=np.array([
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1.],
- ]),
- DS=np.array([
- [1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.],
- [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1.],
- ])
- )
- np.fill_diagonal(rho_ref['IND'], 1.0)
-
- # RV_REP = deepcopy(A._RV_dict['DV_REP'])
- # RV_RED = deepcopy(A._RV_dict['DV_RED'])
- # RV_INJ = deepcopy(A._RV_dict['DV_INJ'])
-
- RV_REP = list(A._DV_REP_dict.values())
- RV_RED = list(A._DV_RED_dict.values())
- RV_INJ = list(A._DV_INJ_dict.values())
-
- for r, (RV_DV, RV_tag) in enumerate(
- zip([RV_REP, RV_RED, RV_INJ], ['rep', 'red', 'inj'])):
-
- # assert len(RV_DV._dimension_tags) == [32, 8, 16][r]
- assert len(RV_DV) == [32, 8, 16][r]
-
- DV_theta_test, DV_beta_test = np.array([rv.theta for rv in RV_DV]).T
- DV_rho_test = RV_DV[0].RV_set.Rho([rv.name for rv in RV_DV])
-
- # COV_test = RV_DV.COV
- # sig_test = np.sqrt(np.diagonal(COV_test))
- # rho_test = COV_test / np.outer(sig_test, sig_test)
-
- if RV_tag == 'rep':
-
- assert_allclose(DV_theta_test, np.ones(32))
- assert_allclose(DV_beta_test, np.array(
- [0.31, 0.71] * 8 + [0.32, 0.72] * 8))
-
- if dep_CT == True:
- if (((dep_COST == 'LOC') and (dep_TIME == 'DIR')) or
- ((dep_COST == 'DIR') and (dep_TIME == 'LOC'))):
- rho_ref_CT = rho_ref['PG']
- else:
- rho_ref_CT = np.maximum(rho_ref[dep_COST],
- rho_ref[dep_TIME])
-
- assert_allclose(DV_rho_test[:16, :16], rho_ref_CT)
- assert_allclose(DV_rho_test[16:, 16:], rho_ref_CT)
- assert_allclose(DV_rho_test[:16, 16:], rho_ref_CT)
- assert_allclose(DV_rho_test[16:, :16], rho_ref_CT)
-
- else:
- assert_allclose(DV_rho_test[:16, :16], rho_ref[dep_COST])
- assert_allclose(DV_rho_test[16:, 16:], rho_ref[dep_TIME])
- assert_allclose(DV_rho_test[:16, 16:], np.zeros((16, 16)))
- assert_allclose(DV_rho_test[16:, :16], np.zeros((16, 16)))
-
- elif RV_tag == 'red':
-
- assert_allclose(DV_theta_test, np.ones(8))
- assert_allclose(DV_beta_test, np.array([0.33, 0.73] * 4))
-
- assert_allclose(DV_rho_test, rho_ref[dep_RED][:8,:8])
-
- elif RV_tag == 'inj':
-
- assert_allclose(DV_theta_test, np.ones(16))
- assert_allclose(DV_beta_test, np.array(
- [0.34, 0.74] * 4 + [0.35, 0.75] * 4))
-
- if dep_ILVL == True:
- assert_allclose(DV_rho_test[:8, :8], rho_ref[dep_INJ][:8,:8])
- assert_allclose(DV_rho_test[8:, 8:], rho_ref[dep_INJ][:8,:8])
- assert_allclose(DV_rho_test[:8, 8:], rho_ref[dep_INJ][:8,:8])
- assert_allclose(DV_rho_test[8:, :8], rho_ref[dep_INJ][:8,:8])
- else:
- assert_allclose(DV_rho_test[:8, :8], rho_ref[dep_INJ][:8,:8])
- assert_allclose(DV_rho_test[8:, 8:], rho_ref[dep_INJ][:8,:8])
- assert_allclose(DV_rho_test[:8, 8:], np.zeros((8, 8)))
- assert_allclose(DV_rho_test[8:, :8], np.zeros((8, 8)))
-
- # ---------------------------------------------------------------------
-
- A.define_loss_model()
-
- A.calculate_damage()
-
- # -------------------------------------------- check damage calculation
-
- # COL
- # there shall be no collapses
- assert A._COL.describe().T['mean'].values == 0
-
- # DMG
- DMG_check = A._DMG
-
- # Fragilities are not tested here, so we only do a few simple checks
- assert np.min(DMG_check.describe().loc['mean'].values) > 0.
- assert np.min(DMG_check.describe().loc['std'].values) > 0.
-
- # ---------------------------------------------------------------------
-
- A.calculate_losses()
-
- # ---------------------------------------------- check loss calculation
-
- # COST and TIME and INJ
- DV_COST = A._DV_dict['rec_cost'] / DMG_check
- DV_TIME = A._DV_dict['rec_time'] / DMG_check
- DV_INJ_dict = deepcopy(A._DV_dict['injuries'])
- DV_INJ0 = DV_INJ_dict[0] / DMG_check
- DV_INJ1 = DV_INJ_dict[1] / DMG_check
-
- for dv_i, (DV, DV_tag) in enumerate(
- zip([DV_COST, DV_TIME, DV_INJ0, DV_INJ1],
- ['cost', 'time', 'inj0', 'inj1'])):
-
- DV_desc = DV.describe().T
- DV_desc_log = np.log(DV).describe().T
-
- if DV_tag == 'cost':
-
- # cost consequences in DS1 are lognormal
- mu_ds1_ref = np.exp(np.log(10.) + 0.31 ** 2. / 2.)
- sig_ds1_ref = np.sqrt(
- np.exp(2 * np.log(10.) + 0.31 ** 2.) * (
- np.exp(0.31 ** 2.) - 1.))
- assert_allclose(DV_desc['mean'].values[::2], mu_ds1_ref,
- rtol=0.02)
- assert_allclose(DV_desc['std'].values[::2], sig_ds1_ref,
- rtol=0.10)
- assert_allclose(DV_desc_log['mean'].values[::2],
- np.log(10.), atol=0.02)
- assert_allclose(DV_desc_log['std'].values[::2], 0.31,
- rtol=0.10)
-
- # cost consequences in DS2 are (truncated) normal
- mu_ds2_ref, var_ds2_ref = tnorm.stats(-1. / 0.71, 1000.,
- loc=1000., scale=710.,
- moments='mv')
- sig_ds2_ref = np.sqrt(var_ds2_ref)
- assert_allclose(DV_desc['mean'].values[1::2], mu_ds2_ref,
- rtol=0.05)
- assert_allclose(DV_desc['std'].values[1::2], sig_ds2_ref,
- rtol=0.10)
-
- # make sure that all damages correspond to positive
- # reconstruction costs
- assert np.all(np.min(DV) > 0.)
-
- elif DV_tag == 'time':
-
- # cost consequences in DS1 are (truncated) normal for FG1 and
- # lognormal for FG2
- # DS1 - FG1
- mu_ds1_ref, var_ds1_ref = tnorm.stats(-1. / 0.32, 1000.,
- loc=0.01,
- scale=0.0032,
- moments='mv')
- sig_ds1_ref = np.sqrt(var_ds1_ref)
- assert_allclose(DV_desc['mean'].values[::2][:4], mu_ds1_ref,
- rtol=0.02)
- assert_allclose(DV_desc['std'].values[::2][:4], sig_ds1_ref,
- rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[::2][:4]) == pytest.approx(
- sig_ds1_ref, rel=0.1)
-
- # DS1 - FG2
- mu_ds1_ref = np.exp(np.log(0.01) + 0.32 ** 2. / 2.)
- sig_ds1_ref = np.sqrt(
- np.exp(2 * np.log(0.01) + 0.32 ** 2.) * (
- np.exp(0.32 ** 2.) - 1.))
- assert_allclose(DV_desc['mean'].values[::2][4:], mu_ds1_ref,
- rtol=0.02)
- assert_allclose(DV_desc['std'].values[::2][4:], sig_ds1_ref,
- rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[::2][4:]) == pytest.approx(
- sig_ds1_ref, rel=0.1)
- assert_allclose(DV_desc_log['mean'].values[::2][4:],
- np.log(0.01), atol=0.02)
- assert_allclose(DV_desc_log['std'].values[::2][4:], 0.32,
- rtol=0.20)
- assert np.mean(
- DV_desc_log['std'].values[::2][4:]) == pytest.approx(
- 0.32, rel=0.1)
-
- # cost consequences in DS2 are lognormal for FG1 and
- # (truncated) normal for FG2
- # DS2 - FG1
- mu_ds2_ref = np.exp(np.log(1.) + 0.72 ** 2. / 2.)
- sig_ds2_ref = np.sqrt(
- np.exp(2 * np.log(1.) + 0.72 ** 2.) * (
- np.exp(0.72 ** 2.) - 1.))
- assert_allclose(DV_desc['mean'].values[1::2][:4],
- mu_ds2_ref, rtol=0.05)
- assert_allclose(DV_desc['std'].values[1::2][:4],
- sig_ds2_ref, rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[1::2][:4]) == pytest.approx(
- sig_ds2_ref, rel=0.1)
-
- assert_allclose(DV_desc_log['mean'].values[1::2][:4],
- np.log(1.), atol=0.05)
- assert_allclose(DV_desc_log['std'].values[1::2][:4], 0.72,
- rtol=0.20)
- assert np.mean(
- DV_desc_log['std'].values[1::2][:4]) == pytest.approx(
- 0.72, rel=0.1)
-
- # DS2 - FG2
- mu_ds2_ref, var_ds2_ref = tnorm.stats(-1. / 0.72, 1000.,
- loc=1., scale=0.72,
- moments='mv')
- sig_ds2_ref = np.sqrt(var_ds2_ref)
- assert_allclose(DV_desc['mean'].values[1::2][4:],
- mu_ds2_ref, rtol=0.05)
- assert_allclose(DV_desc['std'].values[1::2][4:],
- sig_ds2_ref, rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[1::2][4:]) == pytest.approx(
- sig_ds2_ref, rel=0.1)
-
- # make sure that all damages correspond to positive
- # reconstruction time
- assert np.all(np.min(DV) > 0.)
-
- elif DV_tag in ['inj0', 'inj1']:
-
- # Injuries follow a truncated normal distribution in all cases
- # The beta values provided are coefficients of variation of the
- # non-truncated distribution. These provide the reference mean
- # and standard deviation values for the truncated case.
- mu_ds1, mu_ds2 = {'inj0': [0.5, 0.6],
- 'inj1': [0.1, 0.2]}[DV_tag]
- beta_ds1, beta_ds2 = {'inj0': [0.34, 0.74],
- 'inj1': [0.35, 0.75]}[DV_tag]
-
- # DS1
- # The affected population in DS1 per unit quantity (identical
- # for all FGs and injury levels)
- p_aff = 0.05
-
- mu_ref, var_ref = tnorm.stats(
- -1. / beta_ds1, (1. - mu_ds1) / mu_ds1 / beta_ds1,
- loc=mu_ds1,
- scale=mu_ds1 * beta_ds1,
- moments='mv')
- sig_ref = np.sqrt(var_ref)
- mu_ref = mu_ref * p_aff
- sig_ref = sig_ref * p_aff
- assert_allclose(DV_desc['mean'].values[::2],
- [np.nan]*4 + [mu_ref]*4,
- rtol=beta_ds1 / 10.)
- assert_allclose(DV_desc['std'].values[::2],
- [np.nan] * 4 + [sig_ref] * 4,
- rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[::2][4:]) == pytest.approx(
- sig_ref, rel=0.1)
-
- # DS2
- # the affected population in DS1 per unit quantity (identical
- # for all FGs and injury levels)
- p_aff = 0.1
- mu_ref, var_ref = tnorm.stats(-1. / beta_ds2, (
- 1. - mu_ds2) / mu_ds2 / beta_ds2, loc=mu_ds2,
- scale=mu_ds2 * beta_ds2,
- moments='mv')
- sig_ref = np.sqrt(var_ref)
- mu_ref = mu_ref * p_aff
- sig_ref = sig_ref * p_aff
- assert_allclose(DV_desc['mean'].values[1::2],
- [np.nan] * 4 + [mu_ref] * 4,
- rtol=beta_ds2 / 10.)
- assert_allclose(DV_desc['std'].values[1::2],
- [np.nan] * 4 + [sig_ref] * 4,
- rtol=0.20)
- assert np.mean(
- DV_desc['std'].values[1::2][4:]) == pytest.approx(
- sig_ref, rel=0.1)
-
- # red tags have to be treated separately
- DV_RED = A._DV_dict['red_tag']
-
- DMG_norm = DMG_check / 25.
-
- assert len(DV_RED.columns) == 8
-
- for i in range(8):
-
- dmg_i = i+8
- is_dam = pd.DataFrame(np.zeros((len(DMG_norm.index), 5)),
- columns=range(5))
- is_dam[0] = (DMG_norm.iloc[:, dmg_i] < 0.01)
- is_dam[1] = (DMG_norm.iloc[:, dmg_i] > 0.01) & (
- DMG_norm.iloc[:, dmg_i] < 0.275)
- is_dam[2] = (DMG_norm.iloc[:, dmg_i] > 0.275) & (
- DMG_norm.iloc[:, dmg_i] < 0.525)
- is_dam[3] = (DMG_norm.iloc[:, dmg_i] > 0.525) & (
- DMG_norm.iloc[:, dmg_i] < 0.775)
- is_dam[4] = (DMG_norm.iloc[:, dmg_i] > 0.775)
-
- mu_red = ([0.50, 0.23185] * 4)[i]
- beta_red = ([0.33, 0.73] * 4)[i]
- mu_ref = np.zeros(5)
- mu_ref[1] = tnorm.cdf(0.25, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
- mu_ref[2] = tnorm.cdf(0.50, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
- mu_ref[3] = tnorm.cdf(0.75, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
- mu_ref[4] = tnorm.cdf(1.00, -1. / beta_red,
- (1. - mu_red) / mu_red / beta_red,
- loc=mu_red, scale=mu_red * beta_red)
-
- sample_count = np.array(
- [(DV_RED.iloc[:, i])[is_dam[c]].describe().loc['count'] for
- c in range(5)])
- mu_test = np.array(
- [(DV_RED.iloc[:, i])[is_dam[c]].describe().loc['mean'] for c
- in range(5)])
-
- assert mu_test[0] == 0.
- for step in range(1, 5):
- if sample_count[step] > 0:
- assert mu_test[step] == pytest.approx(
- mu_ref[step],
- abs=5 * 0.4 / np.sqrt(sample_count[step]))
-
- # CORRELATIONS
-
- # repair and injury correlations
- DV_REP = pd.concat([DV_COST, DV_TIME], axis=1)
- DV_INJ = pd.concat([DV_INJ0, DV_INJ1], axis=1)
-
- for DV, RV, dv_tag in zip([DV_REP, DV_INJ, DV_RED],
- [RV_REP, RV_INJ, RV_RED],
- ['rep', 'inj', 'red']):
-
- if dv_tag == 'rep':
- # transform the lognormal variables to log scale
- log_flags = ([True, False] * 8 +
- [False, True] * 4 +
- [True, False] * 4)
- for c, is_log in enumerate(log_flags):
- if is_log:
- DV.iloc[:, c] = np.log(DV.iloc[:, c])
-
- if dv_tag == 'inj':
- # remove the columns with only nan values from DV
- DV = pd.concat([DV.iloc[:,8:16], DV.iloc[:,24:32]], axis=1)
-
- elif dv_tag == 'red':
- DV_RED_n = pd.DataFrame(np.ones(DV.shape) * np.nan,
- index=DV.index, columns=DV.columns)
- DMG_filter = pd.concat(
- [(DMG_check.iloc[:, [8, 10, 12, 14]] / 25.0 > 0.275) & (
- DMG_check.iloc[:, [8, 10, 12, 14]] / 25.0 < 0.525),
- (DMG_check.iloc[:, [9, 11, 13, 15]] / 25.0 > 0.025) & (
- DMG_check.iloc[:,
- [9, 11, 13, 15]] / 25.0 < 0.275)], axis=1)
-
- DV_RED_n[DMG_filter] = DV_RED[DMG_filter]
- DV = DV_RED_n
-
- DV_corr = DV.corr()
-
- # use the correlations specified for the random variable as
- # reference (that we already verified earlier)
- # COV_ref = RV.COV
- # sig_ref = np.sqrt(np.diagonal(COV_ref))
- # rho_ref = COV_ref / np.outer(sig_ref, sig_ref)
-
- rho_ref = RV[0].RV_set.Rho([rv.name for rv in RV])
-
- # perform the tests
- for i in range(len(DV_corr.index)):
- for j in range(len(DV_corr.columns)):
- ref_i = rho_ref[i, j]
- if ref_i != 0.0:
- if ref_i > 0.0:
- assert DV_corr.iloc[i, j] > 0.97 * ref_i
- else:
- assert DV_corr.iloc[i, j] < 0.0
- else:
- assert DV_corr.iloc[i, j] == pytest.approx(ref_i,
- abs=0.15)
-
- # ---------------------------------------------------------------------
-
- A.aggregate_results()
-
- # -------------------------------------------- check result aggregation
-
- # Aggregate results are checked in detail by other tests.
- # Here we only focus on some simple checks to make sure the results
- # make sense.
-
- S = A._SUMMARY
- SD = S.describe().T
-
- assert SD.loc[('inhabitants', ''), 'mean'] == 20.0
- assert SD.loc[('inhabitants', ''), 'std'] == 0.0
-
- assert SD.loc[('collapses', 'collapsed'), 'mean'] == 0.0
- assert SD.loc[('collapses', 'collapsed'), 'std'] == 0.0
-
- assert_allclose(A._DV_dict['rec_cost'].sum(axis=1),
- S.loc[:, ('reconstruction', 'cost')])
- assert_allclose(A._DV_dict['rec_time'].sum(axis=1),
- S.loc[:, ('reconstruction', 'time-sequential')])
- assert_allclose(A._DV_dict['rec_time'].max(axis=1),
- S.loc[:, ('reconstruction', 'time-parallel')])
- assert_allclose(A._DV_dict['injuries'][0].sum(axis=1),
- S.loc[:, ('injuries', 'sev1')])
- assert_allclose(A._DV_dict['injuries'][1].sum(axis=1),
- S.loc[:, ('injuries', 'sev2')])
-
- # print()
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-These are unit and integration tests on the db module of pelicun.
-
-"""
-
-import pytest
-
-import os, sys, inspect
-from pathlib import Path
-current_dir = os.path.dirname(
- os.path.abspath(inspect.getfile(inspect.currentframe())))
-parent_dir = os.path.dirname(current_dir)
-sys.path.insert(0,os.path.dirname(parent_dir))
-
-from pelicun.db import *
-
-# -----------------------------------------------------------------------------
-# convert P58 data to JSON
-# -----------------------------------------------------------------------------
-
-[docs]def test_convert_P58_data_to_json():
- """
- Test if the damage and loss data from the FEMA P58 project is properly
- converted into the SimCenter JSON format using xml and xlsx files avialable
- from ATC (the test uses a subset of the files).
- """
-
- data_dir = Path('resources/io testing/P58 converter/source/').resolve()
- ref_dir = Path('resources/io testing/P58 converter/ref/').resolve()
- test_dir = Path('test/').resolve()
-
- try:
- # convert the files in the data folder
- os.mkdir(test_dir)
- convert_P58_data_to_json(data_dir, test_dir)
-
- # collect the prepared reference files
- ref_files = sorted(os.listdir(ref_dir))
-
- # collect the converted files
- test_files = sorted(os.listdir(test_dir / 'DL json'))
-
- # compare the reference files to the converted ones
- for test, ref in zip(test_files, ref_files):
- with open((test_dir / 'DL json') / test,'r') as f_test:
- with open(ref_dir / ref,'r') as f_ref:
- #print(test, ref)
- assert json.load(f_test) == json.load(f_ref)
-
- finally:
- #pass
- shutil.rmtree(test_dir)
-
-# -----------------------------------------------------------------------------
-# create HAZUS JSON files
-# -----------------------------------------------------------------------------
-
-[docs]def test_create_HAZUS_json_files():
- """
- Test if the damage and loss data from HAZUS is properly converted into the
- SimCenter JSON format the prepared raw HAZUS JSON files (the test uses a
- subset of the HAZUS input data).
- """
-
- data_dir = Path('resources/io testing/HAZUS creator/source/').resolve()
- ref_dir = Path('resources/io testing/HAZUS creator/ref/').resolve()
- test_dir = Path('test/').resolve()
-
- print(test_dir)
-
- try:
- # convert the files in the data folder
- os.mkdir(test_dir)
- create_HAZUS_EQ_json_files(data_dir, test_dir)
-
- # collect the prepared reference files
- ref_files = sorted(os.listdir(ref_dir / 'DL json'))
-
- # collect the converted files
- test_files = sorted(os.listdir(test_dir / 'DL json/'))
-
- # compare the reference files to the converted ones
- for test, ref in zip(test_files, ref_files):
- with open((test_dir / 'DL json') / test, 'r') as f_test:
- with open((ref_dir / 'DL json') / ref, 'r') as f_ref:
- #print(test, ref)
- assert json.load(f_test) == json.load(f_ref)
-
- # compare the population files
- with open(test_dir / 'population.json', 'r') as f_test:
- with open(ref_dir / 'population.json', 'r') as f_ref:
- #print(test, ref)
- assert json.load(f_test) == json.load(f_ref)
-
- finally:
- #pass
- shutil.rmtree(test_dir)
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-These are unit and integration tests on the file_io module of pelicun.
-
-"""
-
-import pytest
-
-import os, sys, inspect, shutil
-current_dir = os.path.dirname(
- os.path.abspath(inspect.getfile(inspect.currentframe())))
-parent_dir = os.path.dirname(current_dir)
-sys.path.insert(0,os.path.dirname(parent_dir))
-
-from pelicun.file_io import *
-
-# -----------------------------------------------------------------------------
-# read_SimCenter_DL_input
-# -----------------------------------------------------------------------------
-
-[docs]def test_read_SimCenter_DL_input_minimum_input():
- """
- Test if the minimum input is read without producing any errors and also
- check if some warnings are shown that draw attention to the lack of
- potentially important information in the input file.
- """
-
- # load the reference results
- with open('resources/io testing/ref/ref_DL_input_min.json') as f:
- ref_DL = json.load(f)
-
- # read the input file and check for at least one warning
- with pytest.warns(UserWarning) as e_info:
- test_DL = read_SimCenter_DL_input('resources/io testing/test/'
- 'test_DL_input_min.json',
- verbose=False)
-
- # make sure the paths under data sources point to the right locations
- assert test_DL['data_sources']['path_CMP_data'] == \
- pelicun_path + '/resources/FEMA_P58_2nd_ed.hdf'
- test_DL.pop('data_sources', None)
-
- # check if the returned dictionary is appropriate
- for key in set(list(ref_DL.keys())+list(test_DL.keys())):
- assert ref_DL[key] == test_DL[key]
-
-
-[docs]def test_read_SimCenter_DL_input_full_input():
- """
- Test if the full input (i.e. all possible fields populated and all supported
- decision variables turned on) is read without producing any errors or
- warnings.
- """
-
- # load the reference results
- with open('resources/io testing/ref/ref_DL_input_full.json') as f:
- ref_DL = json.load(f)
-
- # read the input file
- test_DL = read_SimCenter_DL_input('resources/io testing/test/'
- 'test_DL_input_full.json', verbose=False)
-
- # make sure the paths under data sources point to the right locations
- assert test_DL['data_sources']['path_CMP_data'] == \
- pelicun_path + '/resources/FEMA_P58_2nd_ed.hdf'
- assert test_DL['data_sources']['path_POP_data'] == \
- pelicun_path + '/resources/FEMA_P58_2nd_ed.hdf'
- test_DL.pop('data_sources', None)
-
- # check if the returned dictionary is appropriate
- for key in set(list(ref_DL.keys()) + list(test_DL.keys())):
- assert ref_DL[key] == test_DL[key]
-
-[docs]def test_read_SimCenter_DL_input_non_standard_units():
- """
- Test if the inputs are properly converted when non-standard units are used.
- """
-
- # load the reference results
- with open('resources/io testing/ref/ref_DL_input_ns_units.json') as f:
- ref_DL = json.load(f)
-
- # read the input file and check for at least one warning
- with pytest.warns(UserWarning) as e_info:
- test_DL = read_SimCenter_DL_input('resources/io testing/test/'
- 'test_DL_input_ns_units.json',
- verbose=False)
-
- # make sure the path under data sources points to the right location
- assert test_DL['data_sources']['path_CMP_data'] == \
- pelicun_path + '/resources/FEMA_P58_2nd_ed.hdf'
- test_DL.pop('data_sources', None)
-
- # check if the returned dictionary is appropriate
- for key in set(list(ref_DL.keys()) + list(test_DL.keys())):
- assert ref_DL[key] == test_DL[key]
-
-[docs]def test_read_SimCenter_DL_input_unknown_unit():
- """
- Test if a warning is shown if the input file contains an unknown unit type.
- """
-
- with pytest.warns(UserWarning) as e_info:
- test_DL = read_SimCenter_DL_input('resources/io testing/test/'
- 'test_DL_input_unknown_unit.json',
- verbose=False)
-
-[docs]def test_read_SimCenter_DL_input_injuries_only():
- """
- Test if the inputs are read properly if the user is only interested in
- calculating injuries.
- """
-
- # read the input file and check for at least one error because the plan
- # area is not specified in the file
- with pytest.raises(ValueError) as e_info:
- test_DL = read_SimCenter_DL_input('resources/io testing/test/'
- 'test_DL_input_injuries_only.json',
- verbose=False)
-
- # now test if warnings are shown if the plan area is in the file, but the
- # population is only specified for the first two stories
-
- # load the reference results
- with open('resources/io testing/ref/'
- 'ref_DL_input_injuries_missing_pop.json') as f:
- ref_DL = json.load(f)
-
- with pytest.warns(UserWarning) as e_info:
- test_DL = read_SimCenter_DL_input(
- 'resources/io testing/test/test_DL_input_injuries_missing_pop.json',
- verbose=False)
-
- # remove the data_sources entry (it has already been tested)
- test_DL.pop('data_sources', None)
-
- # check if the returned dictionary is appropriate
- for key in set(list(ref_DL.keys()) + list(test_DL.keys())):
- assert ref_DL[key] == test_DL[key]
-
- # now test if an error is shown if other pieces of data are missing
-
- # load the reference results
- with open('resources/io testing/ref/'
- 'ref_DL_input_injuries_missing_data.json') as f:
- ref_DL = json.load(f)
-
- with pytest.raises(ValueError) as e_info:
- test_DL = read_SimCenter_DL_input(
- 'resources/io testing/test/test_DL_input_injuries_missing_data.json',
- verbose=False)
-
-
-[docs]def test_read_SimCenter_DL_input_unknown_component_unit():
- """
- Test if an error is shown if the input file contains an unknown unit type
- for one of the components.
- """
-
- with pytest.raises(ValueError) as e_info:
- test_DL = read_SimCenter_DL_input('resources/io testing/test/'
- 'test_DL_input_unknown_comp_unit.json',
- verbose=False)
-
-[docs]def test_read_SimCenter_DL_input_no_realizations():
- """
- Test if an error is shown if the input file contains no information about
- the number of realizations to run.
- """
-
- with pytest.raises(ValueError) as e_info:
- test_DL = read_SimCenter_DL_input('resources/io testing/test/'
- 'test_DL_input_no_realizations.json',
- verbose=False)
-
-# -----------------------------------------------------------------------------
-# read_SimCenter_EDP_input
-# -----------------------------------------------------------------------------
-
-[docs]def test_read_SimCenter_EDP_input():
- """
- Test if the function can read the provided set of EDPs from a file and
- return them and the corresponding additional information in a properly
- structured format.
- """
- # load the reference results
- with open('resources/io testing/ref/ref_EDP_input.json') as f:
- ref_EDP = json.load(f)
-
- # read the input file
- test_EDP = read_SimCenter_EDP_input(
- 'resources/io testing/test/test_EDP_input.out',
- EDP_kinds=('PID', 'PFA', 'RD', 'PRD'),
- units = dict(PID=1., PFA=9.81, RD=1., PRD=0.2),
- verbose=False)
-
- # check if the returned dictionary is appropriate
- assert ref_EDP == test_EDP
-
-# -----------------------------------------------------------------------------
-# read_population_distribution
-# -----------------------------------------------------------------------------
-
-[docs]def test_read_population_distribution():
- """
- Test if the population distribution is read according to the specified
- occupancy type and if the peak population is properly converted from
- 1/1000*ft2 to 1/m2.
- """
-
- # load the reference results
- with open('resources/io testing/ref/ref_POP_data.json') as f:
- ref_POP = json.load(f)
-
- # read the input file
- test_POP = read_population_distribution(
- 'resources/io testing/test/test_POP_data.json',
- occupancy='Commercial Office', verbose=False)
-
- # check if the returned dictionary is appropriate
- assert ref_POP == test_POP
-
-# -----------------------------------------------------------------------------
-# read_component_DL_data
-# -----------------------------------------------------------------------------
-
-[docs]def test_read_component_DL_data():
- """
- Test if the component data is properly read from the resource xml files.
- Use a series of tests to see if certain component features trigger proper
- warnings or errors.
- """
-
- # basic case with a typical component
- comp_info = {
- "B1071.011": {
- "locations": [2, 2, 3, 3],
- "directions": [1, 2, 1, 2],
- "quantities": [50.0, 50.0, 50.0, 50.0],
- "csg_weights" : [
- [1.0,],
- [1.0,],
- [1.0,],
- [1.0,]],
- "cov" : ["0.1", "0.1", "0.1", "0.1"],
- "distribution": ["normal", "normal", "normal", "normal"],
- "unit" : "ft2"
- }
- }
-
- # read the component data
- test_CMP = read_component_DL_data('../resources/FEMA_P58_1st_ed.hdf',
- comp_info)
-
- # load the reference results
- with open('resources/io testing/ref/ref_CMP_B1071.011.json',
- 'r') as f:
- ref_CMP = json.load(f)
-
- # check if the returned dictionary is appropriate
- for key in set(list(ref_CMP.keys())+list(test_CMP.keys())):
- assert ref_CMP[key] == test_CMP[key]
-
- # acceleration-sensitive component with injuries
- comp_info = {
- "C3032.001a": {
- "locations" : [1, 1],
- "directions" : [1, 2],
- "quantities" : [125.0, 125.0],
- "csg_weights" : [
- [0.2, 0.8],
- [0.4, 0.6],
- ],
- "cov" : ["1.0", "1.0"],
- "distribution": ["normal", "normal"],
- "unit" : "ft2"
- }
- }
-
- # read the component data
- test_CMP = read_component_DL_data('../resources/FEMA_P58_1st_ed.hdf',
- comp_info)
-
- # load the reference results
- with open('resources/io testing/ref/ref_CMP_C3032.001a.json',
- 'r') as f:
- ref_CMP = json.load(f)
-
- # check if the returned dictionary is appropriate
- for key in set(list(ref_CMP.keys()) + list(test_CMP.keys())):
- assert ref_CMP[key] == test_CMP[key]
-
- # component with simultaneous damage states
- comp_info = {
- "D1014.011": {
- "locations" : [1, 1],
- "directions" : [1, 2],
- "quantities" : [3.0, 7.0],
- "csg_weights": [
- [2./3., 1./3.],
- [1./7., 6./7.],
- ],
- "cov" : ["1.0", "1.0"],
- "distribution": ["normal", "normal"],
- "unit" : "ea"
- }
- }
-
- # read the component data
- test_CMP = read_component_DL_data('../resources/FEMA_P58_1st_ed.hdf',
- comp_info)
-
- # load the reference results
- with open('resources/io testing/ref/ref_CMP_D1014.011.json',
- 'r') as f:
- ref_CMP = json.load(f)
-
- # check if the returned dictionary is appropriate
- assert ref_CMP["D1014.011"] == test_CMP["D1014.011"]
-
- # component with mutually exclusive damage states
- comp_info = {
- "B1035.051": {
- "locations" : [1, 1],
- "directions" : [1, 2],
- "quantities" : [3.0, 7.0],
- "csg_weights": [
- [2./3., 1./3.],
- [1./7., 6./7.],
- ],
- "cov" : ["1.0", "1.0"],
- "distribution": ["normal", "normal"],
- "unit" : "ea"
- }
- }
-
- # read the component data
- test_CMP = read_component_DL_data('../resources/FEMA_P58_1st_ed.hdf',
- comp_info)
-
- # load the reference results
- with open('resources/io testing/ref/ref_CMP_B1035.051.json',
- 'r') as f:
- ref_CMP = json.load(f)
-
- # check if the returned dictionary is appropriate
- assert test_CMP == ref_CMP
-
- # an incomplete component shall not get parsed and shall produce a warning
- comp_info = {
- "E2022.023": {
- "locations" : [1, 1],
- "directions" : [1, 2],
- "quantities" : [3.0, 7.0],
- "csg_weights": [
- [2./3., 1./3.],
- [1./7., 6./7.],
- ],
- "cov" : ["1.0", "1.0"],
- "distribution": ["normal", "normal"],
- "unit" : "ea"
- }
- }
-
- # read the component data
- with pytest.warns(UserWarning) as e_info:
- test_CMP = read_component_DL_data('../resources/FEMA_P58_1st_ed.hdf',
- comp_info)
- assert test_CMP == {}
-
- # a component with unknown EDP shall not get parsed and shall produce a warning
- comp_info = {
- "B1042.001a": {
- "locations" : [1, 1],
- "directions" : [1, 2],
- "quantities" : [3.0, 7.0],
- "csg_weights": [
- [2./3., 1./3.],
- [1./7., 6./7.],
- ],
- "cov" : ["1.0", "1.0"],
- "distribution": ["normal", "normal"],
- "unit" : "ea"
- }
- }
-
- # read the component data
- with pytest.warns(UserWarning) as e_info:
- test_CMP = read_component_DL_data('../resources/FEMA_P58_1st_ed.hdf',
- comp_info)
-
- assert test_CMP == {}
-
-# -----------------------------------------------------------------------------
-# write_SimCenter_DL_output
-# -----------------------------------------------------------------------------
-
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-These are unit and integration tests on the model module of pelicun.
-
-"""
-import pytest
-import numpy as np
-from numpy.testing import assert_allclose
-from scipy.stats import norm, truncnorm
-
-import os, sys, inspect
-current_dir = os.path.dirname(
- os.path.abspath(inspect.getfile(inspect.currentframe())))
-parent_dir = os.path.dirname(current_dir)
-sys.path.insert(0,os.path.dirname(parent_dir))
-
-from pelicun.model import *
-from pelicun.tests.test_reference_data import standard_normal_table
-#from pelicun.uq import RandomVariable, RandomVariableSubset
-from pelicun.uq import *
-
-# -------------------------------------------------------------------------------
-# Fragility_Function
-# ------------------------------------------------------------------------------
-
-[docs]def test_FragilityFunction_Pexc_lognormal_unit_mean_unit_std():
- """
- Given a lognormal fragility function with theta=1.0 and beta=1.0, test if
- the calculated exceedance probabilities are sufficiently accurate.
- The reference results are based on a standard normal table. This limits the
- accuracy of testing to an absolute probability difference of 1e-5.
- """
- # prepare the inputs
- EDP = np.exp(np.concatenate([-standard_normal_table[0][::-1],
- standard_normal_table[0]]))
- reference_P_exc = np.concatenate([0.5 - standard_normal_table[1][::-1],
- 0.5 + standard_normal_table[1]])
-
- # create the fragility function
- RV_A = RandomVariable(name='A', distribution='lognormal', theta=[1.0, 1.0])
-
- FF_set = RandomVariableSet('A_set', [RV_A,], [1.0])
-
- fragility_function = FragilityFunction(EDP_limit=[RV_A,])
-
- # calculate the exceedance probabilities
- test_P_exc = fragility_function.P_exc(EDP, DSG_ID=1)
-
- assert_allclose(test_P_exc, reference_P_exc, atol=1e-5)
-
-[docs]def test_FragilityFunction_Pexc_lognormal_non_trivial_case():
- """
- Given a lognormal fragility function with theta=0.5 and beta=0.2, test if
- the calculated exceedance probabilities are sufficiently accurate.
- The reference results are based on a standard normal table. This limits the
- accuracy of testing to an absolute probability difference of 1e-5.
- """
- # prepare the inputs
- target_theta = 0.5
- target_beta = 0.2
- EDP = np.concatenate([-standard_normal_table[0][::-1],
- standard_normal_table[0]])
- EDP = np.exp(EDP * target_beta + np.log(target_theta))
- reference_P_exc = np.concatenate([0.5 - standard_normal_table[1][::-1],
- 0.5 + standard_normal_table[1]])
-
- # create the fragility function
- RV_A = RandomVariable(name='A', distribution='lognormal',
- theta=[target_theta, target_beta])
-
- FF_set = RandomVariableSet('A_set', [RV_A, ], [1.0])
-
- fragility_function = FragilityFunction(EDP_limit=[RV_A,])
-
- # calculate the exceedance probabilities
- test_P_exc = fragility_function.P_exc(EDP, DSG_ID=1)
-
- assert_allclose(test_P_exc, reference_P_exc, atol=1e-5)
-
-[docs]def test_FragilityFunction_Pexc_lognormal_zero_input():
- """
- Given a zero EDP input to a lognormal fragility function, the result shall
- be 0 exceedance probability, even though zero input in log space shall
- correspond to -infinity. This slight modification makes our lives much
- easier when real inputs are fed to the fragility functions.
- """
- # create the fragility function
- RV_A = RandomVariable(name='A', distribution='lognormal',
- theta=[1.0, 1.0])
-
- FF_set = RandomVariableSet('A_set', [RV_A, ], [1.0])
-
- fragility_function = FragilityFunction(EDP_limit=[RV_A,])
-
- # calculate the exceedance probability
- test_P_exc = fragility_function.P_exc(0., DSG_ID=1)
-
- assert test_P_exc == 0.
-
-[docs]def test_FragilityFunction_Pexc_lognormal_nonzero_scalar_input():
- """
- Given a nonzero scalar EDP input, the fragility function should return a
- nonzero scalar output.
- """
- # create the fragility function
- RV_A = RandomVariable(name='A', distribution='lognormal',
- theta=[1.0, 1.0])
-
- FF_set = RandomVariableSet('A_set', [RV_A, ], [1.0])
- fragility_function = FragilityFunction(EDP_limit=[RV_A,])
-
- # calculate the exceedance probabilities
- test_P_exc = fragility_function.P_exc(standard_normal_table[0][0], DSG_ID=1)
-
- assert test_P_exc == pytest.approx(standard_normal_table[1][0], abs=1e-5)
-
-[docs]def test_FragilityFunction_Pexc_multiple_damage_states_with_correlation():
- """
- Test if the fragility function returns an appropriate list of exceedance
- probabilities for various scenarios with multiple damage states that have
- potentially correlated fragilities.
-
- """
- # P_exc is requested for a list of EDPs
- EDP = np.exp(np.linspace(-2., 2., num=11))
-
- # 3 damage state groups, perfectly correlated
- # the DSGs are unordered in the RV only to make the test more general
- dims = 3
- ref_mean = np.exp([2.0, 0., 0.5])
- ref_std = [1.5, 0.5, 1.0]
- ref_rho = np.ones((dims, dims)) * 1.
- np.fill_diagonal(ref_rho, 1.0)
-
- RV_reg = RandomVariableRegistry()
-
- for i, (name, theta, beta) in enumerate(zip(['C', 'A', 'B'],
- ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=name, distribution='lognormal',
- theta=[theta, beta]))
-
- # a single DSG fragility
- FF_set = RandomVariableSet('A_set', [RV_reg.RV['A'], ], [1.0])
- fragility_function = FragilityFunction(EDP_limit=[RV_reg.RV['A'],])
-
- test_res = fragility_function.P_exc(EDP, 1)
- ref_res = norm.cdf(np.log(EDP),
- loc=np.log(ref_mean[1]), scale=ref_std[1])
- assert_allclose(test_res, ref_res)
-
- # three DSGs in proper order, P_exc for A is requested considering all
- # three
- FF_set = RandomVariableSet('FF_0',
- [RV_reg.RV[rv] for rv in ['A', 'B', 'C']], ref_rho)
- fragility_function = FragilityFunction(
- EDP_limit=[RV_reg.RV[rv] for rv in ['A', 'B', 'C']])
-
- test_res = fragility_function.P_exc(EDP, 1)
- ref_res = [norm.cdf(np.log(EDP),
- loc=np.log(ref_mean[i]), scale=ref_std[i])
- for i in range(3)]
- ref_res = np.max(np.asarray(ref_res), axis=0)
- assert_allclose(test_res, ref_res)
-
- # change the covariance matrix - uncorrelated fragilities
- ref_rho = np.ones((dims, dims)) * 0.
- np.fill_diagonal(ref_rho, 1.0)
-
- FF_set = RandomVariableSet('FF_0',
- [RV_reg.RV[rv] for rv in ['A', 'B', 'C']],
- ref_rho)
- fragility_function = FragilityFunction(
- EDP_limit=[RV_reg.RV[rv] for rv in ['A', 'B', 'C']])
-
- # three DSGs, still interested in P_exc for A considering all three
- test_res = fragility_function.P_exc(EDP, 1)
- ref_res = [norm.cdf(np.log(EDP),
- loc=np.log(ref_mean[i]), scale=ref_std[i])
- for i in range(3)]
- ref_res[1] = ref_res[1] * (1. - ref_res[0])
- ref_res[2] = ref_res[2] * (1. - np.sum(np.asarray(ref_res[:2]), axis=0))
- ref_res = np.sum(np.asarray(ref_res), axis=0)
- assert_allclose(ref_res, test_res)
-
-[docs]def test_FragilityFunction_DSG_ID_given_EDP_general():
- """
- Test if the DSG_IDs returned by the function are appropriate using
- exceedance probabilities from the already tested P_exc function.
-
- """
- # 3 damage state groups, perfectly correlated
- # the DSGs are unordered in the RV only to make the test more general
- dims = 3
- ref_mean = np.exp([2.0, 0., 0.5])
- ref_std = [1.5, 0.5, 1.0]
- ref_rho = np.ones((dims, dims)) * 1.
- np.fill_diagonal(ref_rho, 1.0)
-
- RV_reg = RandomVariableRegistry()
-
- for i, (name, theta, beta) in enumerate(zip(['C', 'A', 'B'],
- ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=name, distribution='lognormal',
- theta=[theta, beta]))
-
- FF_set = RandomVariableSet('FF_0',
- [RV_reg.RV[rv] for rv in ['A', 'B', 'C']],
- ref_rho)
- fragility_function = FragilityFunction(
- EDP_limit=[RV_reg.RV[rv] for rv in ['A', 'B', 'C']])
-
- RV_reg.add_RV_set(FF_set)
- RV_reg.generate_samples(sample_size=10000)
-
- # same EDP 10^5 times to allow for P_exc-based testing
- for target_EDP in [0., 0.75, 2.0]:
-
- # create the EDP vector
- EDP = np.ones(10000) * np.exp(target_EDP)
-
- # get the DSG_IDs
- DSG_ID = fragility_function.DSG_given_EDP(EDP)
-
- # calculate the DSG_ID probabilities
- P_DS_test = np.histogram(DSG_ID.values,
- bins=np.arange(5) - 0.5, density=True)[0]
-
- # use the P_exc function to arrive at the reference DSG_ID probabilities
- P_exc = np.asarray(list(map(
- lambda x: fragility_function.P_exc(np.exp(target_EDP), x),
- [0, 1, 2, 3])))
- P_DS_ref = np.concatenate([P_exc[:-1] - P_exc[1:], [P_exc[-1], ]])
-
- # compare
- assert_allclose(P_DS_test, P_DS_ref, atol=0.02)
-
- # random set of EDPs uniformly distributed over the several different
- # domains
- for a, b in [[-1., -0.9], [1., 1.1], [-1., 1.]]:
- EDP = np.exp(np.random.uniform(a, b, 10000))
-
- # get a DSG_ID sample for each EDP
- DSG_ID = fragility_function.DSG_given_EDP(EDP)
-
- # get the test DSG_ID probabilities
- P_DS_test = \
- np.histogram(DSG_ID.values, bins=np.arange(5) - 0.5,
- density=True)[0]
-
- # get the EDP-P_exc functions - basically the fragility functions
- EDP = np.exp(np.linspace(a, b, num=100))
- P_exc_f = np.asarray(
- list(map(lambda x: fragility_function.P_exc(EDP, x), [0, 1, 2, 3])))
-
- # Calculate the area enclosed by the two functions that define each DS
- # it should be the same as the P_DS from the test
- CDF = [p - np.max(P_exc_f[i + 1:], axis=0)
- for i, p in enumerate(P_exc_f[:3])]
- CDF.append(P_exc_f[-1])
- CDF = np.asarray(CDF)
- P_DS_ref = np.asarray(
- list(map(lambda x: np.trapz(CDF[x], np.log(EDP)),
- [0, 1, 2, 3]))) / (b - a)
-
- assert_allclose(P_DS_test, P_DS_ref, atol=0.02)
-
-'''
-def test_FragilityFunction_DSG_given_EDP_insufficient_samples():
- """
- Test if the function raises an error message if the number of EDP values
- provided is greater than the number of available samples from the RVS.
-
- """
- # create a simple random variable
- RV = RandomVariable(ID=1, dimension_tags=['A'],
- distribution_kind='lognormal',
- theta=1.0, COV=1.0)
-
- # assign it to the fragility function
- RVS = RandomVariableSubset(RV=RV, tags=['A'])
- FF = FragilityFunction(EDP_limit=RVS)
-
- # sample 10 realizations
- RVS.sample_distribution(10)
-
- # create 100 EDP values
- EDP = np.ones(100)
-
- # try to get the DSG_IDs... and expect an error
- with pytest.raises(ValueError) as e_info:
- FF.DSG_given_EDP(EDP)
-'''
-
-# ------------------------------------------------------------------------------
-# Consequence_Function
-# ------------------------------------------------------------------------------
-
-[docs]def test_ConsequenceFunction_fixed_median_value():
- """
- Test if the function returns the prescribed median.
- """
- for dist in ['normal', 'lognormal']:
- RV = RandomVariable(name='A', distribution=dist,
- theta=[1.0, 1.0])
-
- conseq_function = ConsequenceFunction(
- DV_median=prep_constant_median_DV(1.0),
- DV_distribution=RV
- )
-
- assert conseq_function.median() == 1.0
- assert conseq_function.median(1.0) == 1.0
- assert conseq_function.median([1., 1.]) == 1.0
-
-[docs]def test_ConsequenceFunction_bounded_linear_median_value():
- """
- Test if the function returns an appropriate output for single quantities
- and for quantity arrays, and if it raises an error if the quantity is not
- specified for a quantity-dependent median.
- """
- test_quants = [0.5, 1.0, 1.5, 2.0, 2.5]
- ref_vals = [2.0, 2.0, 1.5, 1.0, 1.0]
-
- for dist in ['normal', 'lognormal']:
- RV = RandomVariable(name='A', distribution=dist,
- theta=[1.0, 1.0])
-
- f_median = prep_bounded_linear_median_DV(
- median_max=2.0, median_min=1.0,
- quantity_lower=1.0, quantity_upper=2.0
- )
- conseq_function = ConsequenceFunction(
- DV_median=f_median,
- DV_distribution=RV
- )
-
- # should raise an error if the quantity is not specified
- with pytest.raises(ValueError) as e_info:
- conseq_function.median()
-
- # single quantities
- for quant, ref_val in zip(test_quants, ref_vals):
- assert conseq_function.median(quantity=quant) == ref_val
-
- # quantity array
- test_medians = conseq_function.median(quantity=test_quants)
- assert_allclose(test_medians, ref_vals, rtol=1e-10)
-
-
-[docs]def test_ConsequenceFunction_sample_unit_DV():
- """
- Test if the function samples the DV distribution properly. Note that we
- have already tested the sampling algorithm in the uq module, so we will not
- do a thorough verification of the samples here, but rather check for errors
- in the inputs that would typically lead to significant mistakes in the
- results.
- """
- test_quants = [0.5, 1.0, 1.5, 2.0, 2.5]
-
- # create a Random Variable with 3 correlated decision variables
- dims = 3
- ref_mean = [1., 1., 0.]
- ref_std = [0.4, 0.3, 0.2]
- ref_rho = np.ones((dims, dims)) * 0.8
- np.fill_diagonal(ref_rho, 1.0)
-
- ref_mean[2] = np.exp(ref_mean[2])
-
- # prepare lower truncation limits at 0 for all...
- tr_lower = np.zeros(dims).tolist()
- # and an upper limit at 2 sigma for the second
- tr_upper = [np.inf, 1.6, np.inf]
-
- RV_reg = RandomVariableRegistry()
-
- for i, (name, dist, theta, beta) in enumerate(
- zip(['A', 'B', 'C'], ['normal', 'normal', 'lognormal'],
- ref_mean, ref_std)):
- RV_reg.add_RV(
- RandomVariable(name=name, distribution=dist, theta=[theta, beta],
- truncation_limits=[tr_lower[i], tr_upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('set_A', [RV_reg.RV[rv] for rv in ['A', 'B', 'C']],
- ref_rho))
- RV_reg.generate_samples(sample_size=1000)
-
- # first test sampling for each decision variable
- for r_i, tag in enumerate(['A', 'B', 'C']):
-
- # use fixed value for 'B' and bounded linear for the other two
- if tag == 'B':
- f_median = prep_constant_median_DV(10.)
- else:
- f_median = prep_bounded_linear_median_DV(
- median_max=20.0, median_min=2.0,
- quantity_lower=1.0, quantity_upper=2.0
- )
-
- # create the consequence function
- conseq_function = ConsequenceFunction(
- DV_median=f_median,
- DV_distribution=RV_reg.RV[tag]
- )
-
- for qnt in test_quants:
- samples = conseq_function.sample_unit_DV(quantity=qnt,
- sample_size=1000)
-
- # transform the results to log space for 'C' to facilitate testing
- if tag == 'C':
- samples = np.log(samples)
- ref_mu = np.log(f_median(qnt))
- ref_min = np.log(max(np.nextafter(0, 1), tr_lower[r_i]))
- ref_max = np.log(max(np.nextafter(0, 1), tr_upper[r_i]))
- a = (ref_min - np.log(ref_mean[r_i])) / ref_std[r_i]
- b = (ref_max - np.log(ref_mean[r_i])) / ref_std[r_i]
- ref_max = ref_mu * b
- else:
- ref_mu = f_median(qnt)
- ref_min = tr_lower[r_i]
- a = (ref_min - ref_mean[r_i]) / ref_std[r_i]
- b = (tr_upper[r_i] - ref_mean[r_i]) / ref_std[r_i]
- ref_max = ref_mu * b
-
- trNorm = truncnorm(a=a, b=b, loc=ref_mu,
- scale=ref_std[r_i] if tag == 'C'
- else ref_std[r_i] * ref_mu)
- ref_samples = trNorm.rvs(size=1000)
-
- # test the means and coefficients of variation
- assert np.mean(samples) == pytest.approx(np.mean(ref_samples),
- rel=0.1)
- assert np.std(samples) == pytest.approx(np.std(ref_samples),
- rel=0.15)
-
- # test the limits
- assert np.min(samples) > ref_min
- assert np.max(samples) < ref_max
-
-'''
-def test_ConsequenceFunction_sample_unit_DV_insufficient_samples():
- """
- Test if the function raises an error message if the number of samples
- requested is greater than the number of available samples from the RVS.
-
- """
- # create a simple random variable
- RV = RandomVariable(ID=1, dimension_tags=['A'],
- distribution_kind='lognormal',
- theta=1.0, COV=1.0)
-
- # assign it to the fragility function
- RVS = RandomVariableSubset(RV=RV, tags=['A'])
- CF = ConsequenceFunction(DV_median=prep_constant_median_DV(1.0),
- DV_distribution = RVS)
-
- # sample 10 realizations
- RVS.sample_distribution(10)
-
- # try to get the DSG_IDs... and expect an error
- with pytest.raises(ValueError) as e_info:
- CF.sample_unit_DV(sample_size=100)
-'''
-
-# ------------------------------------------------------------------------------
-# Damage State
-# ------------------------------------------------------------------------------
-[docs]def test_DamageState_weight():
- """
- Test if the damage state returns the assigned weight value.
- """
- DS = DamageState(ID=1, weight=0.4)
-
- assert DS.weight == 0.4
-
-
-[docs]def test_DamageState_description():
- """
- Test if the damage state returns the assigned description.
- """
- ref_str = 'Test description.'
- DS = DamageState(ID=1, description=ref_str)
-
- assert DS.description == ref_str
-
-
-[docs]def test_DamageState_repair_cost_sampling():
- """
- Test if the repair cost consequence function is properly linked to the
- damage state and if it returns the requested samples.
- """
-
- # create a consequence function (the small standard deviation facilitates
- # the assertion of the returned samples)
- f_median = prep_bounded_linear_median_DV(
- median_max=2.0, median_min=1.0,
- quantity_lower=10.0, quantity_upper=20.0
- )
-
- RV = RandomVariable(name='A', distribution='lognormal',
- theta=[1.0, 1e-10])
- RV_reg = RandomVariableRegistry()
- RV_reg.add_RV(RV)
- RV_reg.generate_samples(sample_size=5)
-
- CF = ConsequenceFunction(DV_median=f_median,
- DV_distribution=RV)
-
- # create a damage state and assign the CF to it
- DS = DamageState(ID=1, repair_cost_CF=CF)
-
- # sample the repair cost distribution
- test_vals = DS.unit_repair_cost(quantity=[5.0, 10.0, 15.0, 20.0, 25.0],
- sample_size=4)
-
- assert test_vals.size == 5
-
- ref_medians = np.asarray([2.0, 2.0, 1.5, 1.0, 1.0])
-
- assert_allclose(test_vals, ref_medians, rtol=1e-4)
-
-
-[docs]def test_DamageState_reconstruction_time_sampling():
- """
- Test if the reconstruction time consequence function is properly linked to
- the damage state and if it returns the requested samples.
- """
-
- # create a consequence function (the small standard deviation facilitates
- # the assertion of the returned samples)
- f_median = prep_bounded_linear_median_DV(
- median_max=2.0, median_min=1.0,
- quantity_lower=10.0, quantity_upper=20.0
- )
-
- RV = RandomVariable(name='A', distribution='lognormal',
- theta=[1.0, 1e-10])
- RV_reg = RandomVariableRegistry()
- RV_reg.add_RV(RV)
- RV_reg.generate_samples(sample_size=5)
-
- CF = ConsequenceFunction(DV_median=f_median,
- DV_distribution=RV)
-
- # create a damage state and assign the CF to it
- DS = DamageState(ID=1, reconstruction_time_CF=CF)
-
- # sample the repair cost distribution
- test_vals = DS.unit_reconstruction_time(
- quantity=[5.0, 10.0, 15.0, 20.0, 25.0],
- sample_size=4)
-
- assert test_vals.size == 5
-
- ref_medians = np.asarray([2.0, 2.0, 1.5, 1.0, 1.0])
-
- assert_allclose(test_vals, ref_medians, rtol=1e-4)
-
-[docs]def test_DamageState_red_tag_sampling():
- """
- Test if the red tag consequence function is properly linked to the damage
- state and if it returns the requested samples.
- """
-
- # create a consequence function
- f_median = prep_constant_median_DV(0.25)
-
- RV = RandomVariable(name='A', distribution='normal',
- theta=[1.0, 1.0],
- truncation_limits=[0., 4.])
- RV_reg = RandomVariableRegistry()
- RV_reg.add_RV(RV)
- RV_reg.generate_samples(sample_size=1000)
-
- CF = ConsequenceFunction(DV_median=f_median,
- DV_distribution=RV)
-
- # create a damage state and assign the CF to it
- DS = DamageState(ID=1, red_tag_CF=CF)
-
- # sample the repair cost distribution
- test_vals = DS.red_tag_dmg_limit(sample_size=1000)
-
- assert test_vals.size == 1000
-
- # sample the reference truncated normal distribution and use the samples for testing
- ref_samples = truncnorm.rvs(a=-1., b=3., loc=0.25, scale=0.25, size=1000)
-
- assert np.mean(test_vals) == pytest.approx(np.mean(ref_samples), rel=0.1)
- assert np.std(test_vals) == pytest.approx(np.std(ref_samples), rel=0.1)
-
- assert np.min(test_vals) > 0.
- assert np.max(test_vals) < 1.
-
-
-[docs]def test_DamageState_injury_sampling():
- """
- Test if the set of injury consequence functions is properly linked to the
- damage state and if it returns the requested samples.
- """
-
- # create two consequence functions that are correlated
- dims = 2
- ref_median = [0.5, 0.4]
- ref_sig = [0.5, 0.6]
- ref_rho = np.ones((dims, dims)) * 0.8
- np.fill_diagonal(ref_rho, 1.0)
-
- tr_lower = np.zeros(2)
- tr_upper = 1. + ((np.ones(2) - ref_median) / ref_median)
-
- f_median_0 = prep_constant_median_DV(ref_median[0])
- f_median_1 = prep_constant_median_DV(ref_median[1])
-
- RV_reg = RandomVariableRegistry()
-
- for i, (name, std) in enumerate(zip(['A', 'B'], ref_sig)):
- RV_reg.add_RV(
- RandomVariable(name=name, distribution='normal', theta=[1.0, std],
- truncation_limits=[tr_lower[i], tr_upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('set_A', [RV_reg.RV[rv] for rv in ['A', 'B']],
- ref_rho))
- RV_reg.generate_samples(sample_size=10000)
-
- CF_0 = ConsequenceFunction(DV_median=f_median_0,
- DV_distribution=RV_reg.RV['A'])
- CF_1 = ConsequenceFunction(DV_median=f_median_1,
- DV_distribution=RV_reg.RV['B'])
-
- # create a damage state and assign the CF list to it
- DS = DamageState(ID=1, injuries_CF_set=[CF_0, CF_1])
-
- # sample the two types of injuries and check if the values are appropriate
- for s_i in [0, 1]:
- samples = DS.unit_injuries(severity_level=s_i, sample_size=10000)
- assert samples.size == 10000
-
- # sample the reference truncated normal distribution and use the
- # samples for testing
- ref_samples = truncnorm.rvs(a=(-1.) / ref_sig[s_i],
- b=(tr_upper[s_i] - 1.) / ref_sig[s_i],
- loc=ref_median[s_i],
- scale=ref_sig[s_i] * ref_median[s_i],
- size=1000)
-
- assert np.mean(samples) == pytest.approx(np.mean(ref_samples), rel=0.1)
- assert np.std(samples) == pytest.approx(np.std(ref_samples), abs=0.02)
- assert np.min(samples) == pytest.approx(0., abs=0.05)
- assert np.max(samples) == pytest.approx(1., abs=0.05)
-
- # finally, check the correlation between A and B
- test_corr = np.corrcoef(pd.DataFrame(RV_reg.RV_set['set_A'].samples).values,
- rowvar=False)[0, 1]
- assert test_corr == pytest.approx(0.8, rel=0.1)
-
-# ------------------------------------------------------------------------------
-# Damage State Group
-# ------------------------------------------------------------------------------
-[docs]def test_DamageStateGroup_kind():
- """
- Test if the damage state group returns the assigned group type.
- """
- ref_kind = 'single'
- DSG = DamageStateGroup(ID=1, DS_set=None, DS_set_kind=ref_kind)
-
- assert DSG._DS_set_kind == ref_kind
-
-# ------------------------------------------------------------------------------
-# Performance Group
-# ------------------------------------------------------------------------------
-
-[docs]def test_PerformanceGroup_Pexc():
- """
- Test if the performance group returns exceedance probabilities from the
- assigned fragility function for a given damage state group appropriately.
- """
-
- RV_reg = RandomVariableRegistry()
-
- # create the fragility function
- for i, (name, theta, beta) in enumerate(zip(['A', 'B'],
- [0.5, 0.7],
- [0.4, 0.4])):
- RV_reg.add_RV(RandomVariable(name=name, distribution='lognormal',
- theta=[theta, beta]))
-
- FF_set = RandomVariableSet('A_set', [RV_reg.RV['A'], RV_reg.RV['B']],
- np.ones((2,2)))
- fragility_function = FragilityFunction(
- EDP_limit=[RV_reg.RV['A'], RV_reg.RV['B']])
-
- RV_reg.add_RV_set(FF_set)
-
- # create two damage state groups
- DSG_0 = DamageStateGroup(ID=1, DS_set=None, DS_set_kind='single')
- DSG_1 = DamageStateGroup(ID=2, DS_set=None, DS_set_kind='single')
-
- # create a random quantity variable
- QNT = RandomVariable(name='Q_A', distribution='normal',
- theta=[100., np.sqrt(100)])
- RV_reg.add_RV(QNT)
-
- # create the performance group
- PG = PerformanceGroup(ID=1, location=1, quantity=QNT,
- fragility_functions=fragility_function,
- DSG_set=[DSG_0, DSG_1])
-
- EDP = np.linspace(0.1, 0.9, 9)
-
- for edp in EDP:
- assert fragility_function.P_exc(edp, DSG_ID=1) == PG.P_exc(edp, DSG_ID=1)
- assert fragility_function.P_exc(edp, DSG_ID=2) == PG.P_exc(edp, DSG_ID=2)
-
- assert_allclose(PG.P_exc(EDP, DSG_ID=1), PG.P_exc(EDP, DSG_ID=1),
- rtol=1e-10)
- assert_allclose(PG.P_exc(EDP, DSG_ID=2), PG.P_exc(EDP, DSG_ID=2),
- rtol=1e-10)
-
-# ------------------------------------------------------------------------------
-# Fragility Group
-# ------------------------------------------------------------------------------
-[docs]def test_FragilityGroup_description_and_name():
- """
- Test if the fragility group returns the assigned description.
- """
- ref_desc = 'Test long description.'
- ref_name = 'Test short description.'
-
- # create a dummy performance group
-
- # create the fragility function
- RV = RandomVariable(name='A', distribution='lognormal', theta=[0.5, 0.4])
- FF = FragilityFunction(EDP_limit=[RV,])
-
- # some of the inputs below do not make sense, but since the subject of the
- # test is not the performance group, they will work fine
- PG = PerformanceGroup(ID=1, location=1,
- quantity=RV,
- fragility_functions=FF,
- DSG_set=None)
-
- FG = FragilityGroup(ID=1, demand_type='PID',
- performance_groups = [PG, ],
- name=ref_name, description=ref_desc)
-
- assert FG.name == ref_name
- assert FG.description == ref_desc
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-These are unit and integration tests on the uq module of pelicun.
-
-"""
-import pytest
-import numpy as np
-from scipy.stats import norm
-from numpy.testing import assert_allclose
-from copy import deepcopy
-
-import os, sys, inspect
-current_dir = os.path.dirname(
- os.path.abspath(inspect.getfile(inspect.currentframe())))
-parent_dir = os.path.dirname(current_dir)
-sys.path.insert(0,os.path.dirname(parent_dir))
-
-from pelicun.tests.test_pelicun import assert_normal_distribution
-from pelicun.uq import *
-
-# ------------------------------------------------------------------------------
-# ORTHOTOPE DENSITY
-# ------------------------------------------------------------------------------
-[docs]def test_MVN_CDF_univariate():
- """
- Test if the MVN CDF function provides accurate results for the special
- univariate case.
-
- """
-
- # Testing is based on the CDF results from scipy's norm function.
- ref_mean = 0.5
- ref_std = 0.25
- ref_var = ref_std ** 2.
- lower = np.arange(13)/5.-1.5
- upper = np.arange(13)/3.
- for l,u in zip(lower,upper):
- ref_l, ref_u = norm.cdf([l,u],loc=ref_mean, scale=ref_std)
- ref_res = ref_u-ref_l
- test_res, __ = mvn_orthotope_density(ref_mean, ref_var,
- lower=l, upper=u)
-
- assert ref_res == pytest.approx(test_res)
-
- # test if the function works properly with infinite boundaries
- test_res, __ = mvn_orthotope_density(ref_mean, ref_var)
- assert test_res == 1.
-
-[docs]def test_MVN_CDF_multivariate():
- """
- Test if the MVN CDF function provides accurate results for multivariate
- cases.
-
- """
- # Testing is based on univariate results compared to estimates of MVN CDF
- # with special correlation structure
-
- # First, assume perfect correlation. Results should be identical for all
- # dimensions
-
- for dims in range(2, 6):
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.arange(1, dims + 1, dtype=np.float64)
- ref_rho = np.ones((dims, dims)) * 1.0
- ref_COV = np.outer(ref_std, ref_std) * ref_rho
- np.fill_diagonal(ref_COV, ref_std ** 2.)
-
- lower = ref_mean - ref_std * 2.5
- upper = ref_mean + ref_std * 1.5
-
- test_res, __ = mvn_orthotope_density(ref_mean, ref_COV,
- lower=lower, upper=upper)
-
- ref_l, ref_u = norm.cdf([-2.5, 1.5], loc=0., scale=1.)
- ref_res = ref_u - ref_l
-
- assert ref_res == pytest.approx(test_res)
-
- # Second, assume independence. Results should be equal to the univariate
- # result on the power of the number of dimensions.
- for dims in range(2, 6):
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.arange(1, dims + 1, dtype=np.float64)
- ref_rho = np.ones((dims, dims)) * 0.0
- ref_COV = np.outer(ref_std, ref_std) * ref_rho
- np.fill_diagonal(ref_COV, ref_std ** 2.)
-
- lower = ref_mean - ref_std * 2.5
- upper = ref_mean + ref_std * 1.5
-
- test_res, __ = mvn_orthotope_density(ref_mean, ref_COV,
- lower=lower, upper=upper)
-
- ref_l, ref_u = norm.cdf([-2.5, 1.5], loc=0., scale=1.)
- ref_res = ref_u - ref_l
-
- assert ref_res**dims == pytest.approx(test_res)
-
-# ------------------------------------------------------------------------------
-# Random_Variable objects
-# ------------------------------------------------------------------------------
-[docs]def test_RandomVariable_incorrect_none_defined():
- """
- Test if the random variable object raises an error when the distribution
- is not empirical and no parameters are provided.
- """
-
- with pytest.raises(ValueError) as e_info:
- RandomVariable(name='A', distribution='normal')
-
-[docs]def test_RandomVariable_incorrect_multinomial_definition():
- """
- Test if the random variable object raises an error when a multinomial
- distribution is defined with incorrect parameters, and test
- that it does not raise an error when the right parameters are provided.
- """
-
- p_values = [0.5, 0.2, 0.1, 0.2]
- # correct parameters
- RandomVariable(name='A', distribution='multinomial',
- theta=[0.5, 0.2, 0.1, 0.2])
-
- # sum(p) less than 1.0 -> should be automatically corrected
- RandomVariable(name='A', distribution='multinomial',
- theta=[0.5, 0.2, 0.1, 0.1])
-
- # sum(p) more than 1.0 -> should raise an error
- with pytest.raises(ValueError) as e_info:
- RandomVariable(name='A', distribution='multinomial',
- theta=[0.5, 0.2, 0.1, 0.3])
-
-# ------------------------------------------------------------------------------
-# SAMPLING
-# ------------------------------------------------------------------------------
-[docs]def test_sampling_tr_alpha_error():
- """
- Test if the function raises an error when the probability density that the
- truncation limits define is not sufficiently accurate for further use.
-
- """
-
- RV_reg = RandomVariableRegistry()
-
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[0.5, 0.25],
- truncation_limits = [-10.0, -9.9]))
-
- with pytest.raises(ValueError) as e_info:
- RV_reg.generate_samples(sample_size=10, seed=1)
-
-[docs]def test_sampling_non_truncated():
- """
- Test if the sampling method returns appropriate samples for a non-truncated
- univariate and multivariate normal distribution.
-
- """
-
- # univariate case
- ref_mean = 0.5
- ref_std = 0.25
-
- RV_reg = RandomVariableRegistry()
-
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std]))
-
- def sampling_function(sample_size):
-
- RV_reg.generate_samples(sample_size=sample_size)
-
- return RV_reg.RV_samples['A']
-
- assert assert_normal_distribution(sampling_function, ref_mean, ref_std**2.)
-
- # bivariate case, various correlation coefficients
- rho_list = [-1., -0.5, 0., 0.5, 1.]
- for rho in rho_list:
- ref_mean = [0.5, 1.5]
- ref_std = [0.25, 1.0]
- ref_rho = np.asarray([[1.0, rho],[rho, 1.0]])
- ref_COV = np.outer(ref_std, ref_std) * ref_rho
-
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(len(ref_mean))],
- ref_rho))
-
- def sampling_function(sample_size):
-
- RV_reg.generate_samples(sample_size=sample_size)
-
- return pd.DataFrame(RV_reg.RV_samples).values
-
- assert_normal_distribution(sampling_function, ref_mean, ref_COV)
-
- # multi-dimensional case
- dims = 5
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.arange(1,dims+1, dtype=np.float64)
- ref_rho = np.ones((dims,dims))*0.3
- np.fill_diagonal(ref_rho, 1.0)
- ref_COV = np.outer(ref_std,ref_std) * ref_rho
-
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(len(ref_mean))],
- ref_rho))
-
- def sampling_function(sample_size):
-
- RV_reg.generate_samples(sample_size=sample_size)
-
- return pd.DataFrame(RV_reg.RV_samples).values
-
- assert_normal_distribution(sampling_function, ref_mean, ref_COV)
-
-[docs]def test_sampling_truncated_wide_limits():
- """
- Test if the sampling method returns appropriate samples for a truncated
- univariate and multivariate normal distribution when the truncation limits
- are sufficiently wide to consider the result a normal distribution.
-
- """
-
- # assign a non-symmetric, but very wide set of limits for all cases
-
- # univariate case
- ref_mean = 0.5
- ref_std = 0.25
- ref_var = ref_std ** 2.
- lower = -1e10
- upper = 1e9
-
- # truncation both ways
- RV_reg = RandomVariableRegistry()
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std],
- truncation_limits = [lower, upper]))
-
- def sampling_function(sample_size):
- RV_reg.generate_samples(sample_size=sample_size)
-
- return RV_reg.RV_samples['A']
-
- assert assert_normal_distribution(sampling_function, ref_mean,
- ref_std ** 2.)
- # upper truncation
- RV_reg = RandomVariableRegistry()
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std],
- truncation_limits=[None, upper]))
-
- def sampling_function(sample_size):
- RV_reg.generate_samples(sample_size=sample_size)
-
- return RV_reg.RV_samples['A']
-
- assert assert_normal_distribution(sampling_function, ref_mean,
- ref_std ** 2.)
-
- # lower truncation
- RV_reg = RandomVariableRegistry()
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std],
- truncation_limits=[lower, None]))
-
- def sampling_function(sample_size):
- RV_reg.generate_samples(sample_size=sample_size)
-
- return RV_reg.RV_samples['A']
-
- assert assert_normal_distribution(sampling_function, ref_mean,
- ref_std ** 2.)
-
- # bivariate case, various correlation coefficients
- rho_list = [-1., -0.5, 0., 0.5, 1.]
- lower = np.ones(2) * lower
- upper = np.ones(2) * upper
- for rho in rho_list:
- ref_mean = [0.5, 1.5]
- ref_std = [0.25, 1.0]
- ref_rho = np.asarray([[1.0, rho], [rho, 1.0]])
- ref_COV = np.outer(ref_std, ref_std) * ref_rho
-
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(
- RandomVariable(name=i, distribution='normal',
- theta=[mu, std],
- truncation_limits = [lower[i], upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(len(ref_mean))],
- ref_rho))
-
- def sampling_function(sample_size):
-
- RV_reg.generate_samples(sample_size=sample_size)
-
- return pd.DataFrame(RV_reg.RV_samples).values
-
- assert_normal_distribution(sampling_function, ref_mean, ref_COV)
-
- # multi-dimensional case
- dims = 3
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.ones(dims) * 0.25
- ref_rho = np.ones((dims, dims)) * 0.3
- np.fill_diagonal(ref_rho, 1.0)
- ref_COV = np.outer(ref_std, ref_std) * ref_rho
-
- lower = np.ones(dims) * lower[0]
- upper = np.ones(dims) * upper[0]
-
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std],
- truncation_limits = [lower[i], upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(len(ref_mean))],
- ref_rho))
-
- def sampling_function(sample_size):
-
- RV_reg.generate_samples(sample_size=sample_size)
-
- return pd.DataFrame(RV_reg.RV_samples).values
-
- assert_normal_distribution(sampling_function, ref_mean, ref_COV)
-
-[docs]def test_sampling_truncated_narrow_limits():
- """
- Test if the sampling method returns appropriate samples for a truncated
- univariate and multivariate normal distribution when the truncation limits
- are narrow.
-
- """
-
- # here we focus on testing if the returned samples are between the
- # pre-defined limits in all dimensions
-
- # univariate case
- ref_mean = 0.5
- ref_std = 0.25
- ref_var = ref_std ** 2.
- lower = 0.1
- upper = 0.6
-
- RV_reg = RandomVariableRegistry()
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std],
- truncation_limits=[lower, upper]))
-
- RV_reg.generate_samples(sample_size=1000)
-
- samples = RV_reg.RV_samples['A']
-
- sample_min = np.min(samples)
- sample_max = np.max(samples)
-
- assert sample_min > lower
- assert sample_max < upper
-
- assert sample_min == pytest.approx(lower, abs=0.01)
- assert sample_max == pytest.approx(upper, abs=0.01)
-
- # multi-dimensional case
- dims = 2
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.ones(dims) * 0.25
- ref_rho = np.ones((dims, dims)) * 0.3
- np.fill_diagonal(ref_rho, 1.0)
-
- lower = ref_mean - ref_std * 2.5
- upper = ref_mean + ref_std * 1.5
-
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std],
- truncation_limits=[lower[i], upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(len(ref_mean))],
- ref_rho))
-
- RV_reg.generate_samples(sample_size=1000)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values
-
- sample_min = np.amin(samples, axis=0)
- sample_max = np.amax(samples, axis=0)
-
- assert np.all(sample_min > lower)
- assert np.all(sample_max < upper)
-
- assert_allclose(sample_min, lower, atol=0.1)
- assert_allclose(sample_max, upper, atol=0.1)
-
-[docs]def test_RandomVariable_sample_distribution_mixed_normal():
- """
- Test if the distribution is sampled appropriately for a correlated mixture
- of normal and lognormal variables. Note that we already tested the sampling
- algorithm itself earlier, so we will not do a thorough verification of
- the samples, but rather check for errors in the inputs that would
- typically lead to significant mistakes in the results.
-
- """
-
- dims = 3
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.ones(dims) * 1.00
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
-
- # prepare the truncation limits - note that these are wide limits and
- # barely affect the distribution
- lower = (ref_mean - ref_std * 10.).tolist()
- upper = ref_mean + ref_std * 10.
-
- # variable 1 is assumed to have lognormal distribution with no lower
- # truncation
- rv_mean = ref_mean.copy()
- rv_mean[1] = np.exp(rv_mean[1])
- lower[1] = None
- upper[1] = np.exp(upper[1])
-
- dist_list = ['normal', 'lognormal', 'normal']
-
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std, dist) in enumerate(zip(rv_mean, ref_std, dist_list)):
- RV_reg.add_RV(RandomVariable(name=i, distribution=dist,
- theta=[mu, std],
- truncation_limits=[lower[i],
- upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(len(ref_mean))],
- ref_rho))
-
- RV_reg.generate_samples(sample_size=1000)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values.T
-
- # make sure that retrieving samples of an individual RV works as intended
- assert_allclose(samples[1], RV_reg.RV_samples[1])
-
- # convert the lognormal samples to log space for the checks
- samples[1] = np.log(samples[1])
-
- assert_allclose(np.mean(samples, axis=1), ref_mean, atol=0.01)
-
- ref_COV = np.outer(ref_std, ref_std) * ref_rho
- assert_allclose(np.cov(samples), ref_COV, atol=0.10)
-
-[docs]def test_RandomVariable_sample_distribution_multinomial():
- """
- Test if the distribution is sampled appropriately for a multinomial
- variable. Also test that getting values for an individual RV that is part
- of a correlated RV_set works appropriately."
-
- """
- # first test with an incomplete p_ref
- p_ref = [0.1, 0.3, 0.5]
- dims = 3
-
- RV_reg= RandomVariableRegistry()
-
- for i in range(dims):
- RV_reg.add_RV(RandomVariable(name=i, distribution='multinomial',
- theta=p_ref))
-
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(dims)], ref_rho))
-
- RV_reg.generate_samples(sample_size=10000)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values.T
-
- p_ref[-1] = 1. - np.sum(p_ref[:-1])
-
- h_bins = np.arange(len(p_ref) + 1) - 0.5
- p_test = np.histogram(samples[1], bins=h_bins, density=True)[0]
-
- assert_allclose(p_test, p_ref, atol=0.05)
-
- # also make sure that individual RV's samples are returned appropriately
- p_test_RV = np.histogram(RV_reg.RV_samples[1], bins=h_bins, density=True)[0]
- assert_allclose(p_test_RV, p_ref, atol=0.05)
-
- assert_allclose(samples[1], RV_reg.RV_samples[1])
-
- # and the prescribed correlation is applied
- # note that the correlation between the samples is not going to be identical
- # to the correlation used to generate the underlying uniform distribution
- rho_target = [[1.0, 0.383, 0.383], [0.383, 1.0, 0.383], [0.383, 0.383, 1.0]]
- assert_allclose(np.corrcoef(samples), rho_target, atol=0.05)
-
- # finally, check the original sampling with the complete p_ref
- RV_reg = RandomVariableRegistry()
-
- for i in range(dims):
- RV_reg.add_RV(RandomVariable(name=i, distribution='multinomial',
- theta=p_ref))
-
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(dims)], ref_rho))
-
- RV_reg.generate_samples(sample_size=10000)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values.T
-
- p_test = np.histogram(samples[1], bins=h_bins, density=True)[0]
-
- assert_allclose(p_test, p_ref, atol=0.05)
-
-# ------------------------------------------------------------------------------
-# FITTING
-# ------------------------------------------------------------------------------
-[docs]def test_fitting_baseline():
- """
- Test if the max. likelihood estimates of a (multivariate) normal
- distribution are sufficiently accurate in the baseline case with no
- truncation and no censoring.
-
- """
- # univariate case
- ref_mean = 0.5
- ref_std = 0.25
-
- # generate samples
- RV_reg = RandomVariableRegistry()
-
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std]))
-
- RV_reg.generate_samples(sample_size=100)
-
- samples = RV_reg.RV_samples['A']
-
- # estimate the parameters of the distribution
- mu, std = fit_distribution(samples, 'normal')[0][0]
-
- assert ref_mean == pytest.approx(mu, abs=0.01)
- assert ref_std == pytest.approx(std, rel=0.05)
-
- # multi-dimensional case
- dims = 3
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.ones(dims) * 0.5
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
-
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(dims)],
- ref_rho))
-
- RV_reg.generate_samples(sample_size=100)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values.T
-
- # estimate the parameters of the distribution
- test_theta, test_rho = fit_distribution(samples, 'normal')
-
- test_mu, test_std = test_theta.T
-
- assert_allclose(test_mu, ref_mean, atol=0.01)
- assert_allclose(test_std, ref_std, rtol=0.1)
- assert_allclose(test_rho, ref_rho, atol=0.3)
-
-[docs]def test_fitting_censored():
- """
- Test if the max. likelihood estimates of a multivariate normal distribution
- are sufficiently accurate in cases with censored data.
-
- """
- # univariate case
- ref_mean = 0.5
- ref_std = 0.25
- c_lower = 0.35
- c_upper = 1.25
- sample_count = 100
-
- # generate censored samples
- RV_reg = RandomVariableRegistry()
-
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std]))
-
- RV_reg.generate_samples(sample_size=sample_count)
-
- samples = RV_reg.RV_samples['A']
-
- # censor the samples
- good_ones = np.all([samples>c_lower, samples<c_upper],axis=0)
- c_samples = samples[good_ones]
- c_count = sample_count - sum(good_ones)
-
- # estimate the parameters of the distribution
- test_theta, __ = fit_distribution(c_samples, 'normal',
- censored_count=c_count,
- detection_limits=[c_lower, c_upper])
-
- test_mu, test_std = test_theta[0]
-
- assert ref_mean == pytest.approx(test_mu, abs=0.05)
- assert ref_std == pytest.approx(test_std, rel=0.05)
-
- # multi-dimensional case
-
- dims = 3
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.ones(dims) * 0.25
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
-
- c_lower = ref_mean - 1.0 * ref_std
- c_upper = ref_mean + 8.5 * ref_std
-
- c_lower[2] = -np.inf
- c_upper[0] = np.inf
-
- sample_count = 1000
-
- # generate samples
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(dims)],
- ref_rho))
-
- RV_reg.generate_samples(sample_size=sample_count)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values
-
- # censor the samples
- good_ones = np.all([samples > c_lower, samples < c_upper], axis=0)
- good_ones = np.all(good_ones, axis=1)
- c_samples = samples[good_ones]
- c_count = sample_count - sum(good_ones)
- det_lims = np.array([c_lower, c_upper]).T
-
- test_theta, test_rho = fit_distribution(c_samples.T, 'normal',
- censored_count=c_count,
- detection_limits=det_lims)
-
- test_mu, test_std = test_theta.T
-
- assert_allclose(test_mu, ref_mean, atol=0.1)
- assert_allclose(test_std , ref_std, rtol=0.25)
- assert_allclose(test_rho, ref_rho, atol=0.3)
-
-[docs]def test_fitting_truncated():
- """
- Test if the max. likelihood estimates of a multivariate normal distribution
- are sufficiently accurate in cases with truncation and uncensored data.
-
- """
- # univariate case
- ref_mean = 0.5
- ref_std = 0.25
- tr_lower = 0.35
- tr_upper = 1.25
-
- # generate samples of a TMVN distribution
- RV_reg = RandomVariableRegistry()
-
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std],
- truncation_limits=[tr_lower, tr_upper]))
-
- RV_reg.generate_samples(sample_size=100)
-
- samples = RV_reg.RV_samples['A']
-
- # estimate the parameters of the distribution
- test_theta, __ = fit_distribution(samples, 'normal',
- truncation_limits=[tr_lower, tr_upper])
-
- test_mu, test_std = test_theta[0]
-
- assert ref_mean == pytest.approx(test_mu, abs=0.05)
- assert ref_std == pytest.approx(test_std, rel=0.05)
-
- # multi-dimensional case
- dims = 3
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.ones(dims) * 0.25
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
-
- tr_lower = ref_mean - 0.25 * ref_std
- tr_upper = ref_mean + 2.0 * ref_std
-
- tr_lower[2] = -np.inf
- tr_upper[0] = np.inf
-
- # generate samples
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std],
- truncation_limits=[tr_lower[i],
- tr_upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(dims)],
- ref_rho))
-
- RV_reg.generate_samples(sample_size=1000)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values.T
-
- # estimate the parameters of the distribution
- tr_lims = np.array([tr_lower, tr_upper]).T
- test_theta, test_rho = fit_distribution(samples, 'normal',
- truncation_limits=tr_lims)
-
- test_mu, test_std = test_theta.T
-
- #print(max(abs(test_mu-ref_mean)), max(abs(test_std-ref_std)))
-
- assert_allclose(test_mu, ref_mean, atol=0.1)
- assert_allclose(test_std, ref_std, atol=0.05)
- assert_allclose(test_rho, ref_rho, atol=0.2)
-
-[docs]def test_fitting_truncated_and_censored():
- """
- Test if the max. likelihood estimates of a multivariate normal distribution
- are sufficiently accurate in cases with truncation and censored data.
-
- """
- # univariate case
- ref_mean = 0.5
- ref_std = 0.25
- tr_lower = 0.35
- tr_upper = 2.5
- det_upper = 1.25
- det_lower = tr_lower
-
- # generate samples of a TMVN distribution
- RV_reg = RandomVariableRegistry()
-
- RV_reg.add_RV(RandomVariable(name='A', distribution='normal',
- theta=[ref_mean, ref_std],
- truncation_limits=[tr_lower, tr_upper]))
-
- RV_reg.generate_samples(sample_size=100)
-
- samples = RV_reg.RV_samples['A']
-
- # censor the samples
- good_ones = samples < det_upper
- c_samples = samples[good_ones]
- c_count = 100 - sum(good_ones)
-
- # estimate the parameters of the distribution
- test_theta, __ = fit_distribution(c_samples, 'normal',
- censored_count=c_count,
- detection_limits=[det_lower, det_upper],
- truncation_limits=[tr_lower, tr_upper])
-
- test_mu, test_std = test_theta[0]
-
- assert ref_mean == pytest.approx(test_mu, abs=0.05)
- assert ref_std == pytest.approx(test_std, rel=0.05)
-
- # # multi-dimensional case
- dims = 3
- ref_mean = np.arange(dims, dtype=np.float64)
- ref_std = np.ones(dims) * 0.25
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
-
- tr_lower = ref_mean - 4.5 * ref_std
- tr_upper = ref_mean + 2.5 * ref_std
- tr_lower[2] = -np.inf
- tr_upper[0] = np.inf
-
- det_lower = ref_mean - 1. * ref_std
- det_lower[2] = -np.inf
- det_upper = tr_upper
-
- # generate samples
- RV_reg = RandomVariableRegistry()
-
- for i, (mu, std) in enumerate(zip(ref_mean, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='normal',
- theta=[mu, std],
- truncation_limits=[tr_lower[i],
- tr_upper[i]]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(dims)],
- ref_rho))
-
- RV_reg.generate_samples(sample_size=1000)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values
-
- # censor the samples
- good_ones = np.all([samples > det_lower, samples < det_upper], axis=0)
- good_ones = np.all(good_ones, axis=1)
- c_samples = samples[good_ones]
- c_count = 1000 - sum(good_ones)
-
- # estimate the parameters of the distribution
- det_lims = np.array([det_lower, det_upper]).T
- tr_lims = np.array([tr_lower, tr_upper]).T
-
- test_theta, test_rho = fit_distribution(
- c_samples.T, 'normal', censored_count=c_count,
- detection_limits=det_lims, truncation_limits=tr_lims)
-
- test_mu, test_std = test_theta.T
-
- assert_allclose(test_mu, ref_mean, atol=0.1)
- assert_allclose(test_std, ref_std, rtol=0.25)
- assert_allclose(test_rho, ref_rho, atol=0.3)
-
-[docs]def test_fitting_lognormal():
- """
- Test if the max. likelihood estimates of a multivariate lognormal
- distribution are sufficiently accurate
- """
- # univariate case
- # generate raw data
- ref_median = 0.5
- ref_std = 0.25
-
- # generate samples
- RV_reg = RandomVariableRegistry()
-
- RV_reg.add_RV(RandomVariable(name='A', distribution='lognormal',
- theta=[ref_median, ref_std]))
-
- RV_reg.generate_samples(sample_size=100)
-
- samples = RV_reg.RV_samples['A']
-
- # estimate the parameters of the distribution
- median, std = fit_distribution(samples, 'lognormal')[0][0]
-
- assert ref_median == pytest.approx(median, abs=0.01)
- assert ref_std == pytest.approx(std, rel=0.05)
-
- # multivariate case
- # generate raw data
- dims = 6
- ref_median = np.exp(np.arange(dims, dtype=np.float64))
- ref_std = np.ones(dims) * 0.5
- ref_rho = np.ones((dims, dims)) * 0.5
- np.fill_diagonal(ref_rho, 1.0)
-
- RV_reg = RandomVariableRegistry()
-
- for i, (median, std) in enumerate(zip(ref_median, ref_std)):
- RV_reg.add_RV(RandomVariable(name=i, distribution='lognormal',
- theta=[median, std]))
-
- RV_reg.add_RV_set(
- RandomVariableSet('A',
- [RV_reg.RV[rv] for rv in range(dims)],
- ref_rho))
-
- RV_reg.generate_samples(sample_size=100)
-
- samples = pd.DataFrame(RV_reg.RV_samples).values.T
-
- # estimate the parameters of the distribution
- test_theta, test_rho = fit_distribution(samples, 'lognormal')
-
- test_median, test_std = test_theta.T
-
- assert_allclose(np.log(test_median), np.log(ref_median), atol=0.01)
- assert_allclose(test_std, ref_std, rtol=0.2)
- assert_allclose(test_rho, ref_rho, atol=0.3)
-
-
-
-
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2018 Leland Stanford Junior University
-# Copyright (c) 2018 The Regents of the University of California
-#
-# This file is part of pelicun.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# 3. Neither the name of the copyright holder nor the names of its contributors
-# may be used to endorse or promote products derived from this software without
-# specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
-# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-# POSSIBILITY OF SUCH DAMAGE.
-#
-# You should have received a copy of the BSD 3-Clause License along with
-# pelicun. If not, see <http://www.opensource.org/licenses/>.
-#
-# Contributors:
-# Adam Zsarnóczay
-
-"""
-This module defines constants, classes and methods for uncertainty
-quantification in pelicun.
-
-.. rubric:: Contents
-
-.. autosummary::
-
- mvn_orthotope_density
- fit_distribution
- RandomVariable
- RandomVariableSet
- RandomVariableRegistry
-
-
-"""
-
-from .base import *
-
-from scipy.stats import uniform, norm
-from scipy.stats import multivariate_normal as mvn
-from scipy.stats.mvn import mvndst
-from scipy.linalg import cholesky, svd
-from scipy.optimize import minimize
-
-import warnings
-
-
-[docs]def mvn_orthotope_density(mu, COV, lower=None, upper=None):
- """
- Estimate the probability density within a hyperrectangle for an MVN distr.
-
- Use the method of Alan Genz (1992) to estimate the probability density
- of a multivariate normal distribution within an n-orthotope (i.e.,
- hyperrectangle) defined by its lower and upper bounds. Limits can be
- relaxed in any direction by assigning infinite bounds (i.e. numpy.inf).
-
- Parameters
- ----------
- mu: float scalar or ndarray
- Mean(s) of the non-truncated distribution.
- COV: float ndarray
- Covariance matrix of the non-truncated distribution
- lower: float vector, optional, default: None
- Lower bound(s) for the truncated distributions. A scalar value can be
- used for a univariate case, while a list of bounds is expected in
- multivariate cases. If the distribution is non-truncated from below
- in a subset of the dimensions, use either `None` or assign an infinite
- value (i.e. -numpy.inf) to those dimensions.
- upper: float vector, optional, default: None
- Upper bound(s) for the truncated distributions. A scalar value can be
- used for a univariate case, while a list of bounds is expected in
- multivariate cases. If the distribution is non-truncated from above
- in a subset of the dimensions, use either `None` or assign an infinite
- value (i.e. numpy.inf) to those dimensions.
- Returns
- -------
- alpha: float
- Estimate of the probability density within the hyperrectangle
- eps_alpha: float
- Estimate of the error in alpha.
-
- """
-
- # process the inputs and get the number of dimensions
- mu = np.atleast_1d(mu)
- COV = np.atleast_2d(COV)
-
- if mu.shape == ():
- mu = np.asarray([mu])
- COV = np.asarray([COV])
- else:
- COV = np.asarray(COV)
-
- sig = np.sqrt(np.diag(COV))
- corr = COV / np.outer(sig, sig)
-
- ndim = mu.size
-
- if lower is None:
- lower = -np.ones(ndim) * np.inf
- else:
- lower = np.atleast_1d(lower)
-
- if upper is None:
- upper = np.ones(ndim) * np.inf
- else:
- upper = np.atleast_1d(upper)
-
- # replace None with np.inf
- lower[np.where(lower == None)[0]] = -np.inf
- lower = lower.astype(np.float64)
- upper[np.where(upper == None)[0]] = np.inf
- upper = upper.astype(np.float64)
-
- # standardize the truncation limits
- lower = (lower - mu) / sig
- upper = (upper - mu) / sig
-
- # prepare the flags for infinite bounds (these are needed for the mvndst
- # function)
- lowinf = np.isneginf(lower)
- uppinf = np.isposinf(upper)
- infin = 2.0 * np.ones(ndim)
-
- np.putmask(infin, lowinf, 0)
- np.putmask(infin, uppinf, 1)
- np.putmask(infin, lowinf * uppinf, -1)
-
- # prepare the correlation coefficients
- if ndim == 1:
- correl = 0
- else:
- correl = corr[np.tril_indices(ndim, -1)]
-
- # estimate the density
- eps_alpha, alpha, __ = mvndst(lower, upper, infin, correl)
-
- return alpha, eps_alpha
-
-
-def get_theta(params, inits, distribution):
-
- theta = np.zeros(inits.shape)
-
- for i, (params_i, inits_i) in enumerate(zip(params, inits)):
-
- if distribution in ['normal', 'lognormal']:
- sig = np.exp(np.log(inits_i[1]) + params_i[1])
- mu = inits_i[0] + params_i[0] * sig
- theta[i, 0] = mu
- theta[i, 1] = sig
-
- return theta
-
-
-def _get_limit_probs(limits, distribution, theta):
-
- if distribution in ['normal', 'lognormal']:
-
- a, b = limits
- mu = theta[0]
- sig = theta[1]
-
- if a is not None:
- p_a = norm.cdf((a - mu) / sig)
- else:
- p_a = 0.0
-
- if b is not None:
- p_b = norm.cdf((b - mu) / sig)
- else:
- p_b = 1.0
-
- return p_a, p_b
-
-
-def _get_std_samples(samples, theta, tr_limits, distribution):
-
- ndims = samples.shape[0]
-
- std_samples = np.zeros(samples.shape)
-
- for i, (samples_i, theta_i, tr_lim_i) in enumerate(
- zip(samples, theta, tr_limits)):
- if distribution in ['normal', 'lognormal']:
- # consider truncation if needed
- p_a, p_b = _get_limit_probs(tr_lim_i, distribution, theta_i)
- uni_samples = (norm.cdf(samples_i, loc=theta_i[0],
- scale=theta_i[1]) - p_a) / (p_b - p_a)
- std_samples[i] = norm.ppf(uni_samples, loc=0., scale=1.)
-
- return std_samples
-
-
-def _get_std_corr_matrix(std_samples):
-
- n_dims, n_samples = std_samples.shape
-
- rho_hat = np.zeros((n_dims, n_dims))
- np.fill_diagonal(rho_hat, 1.0)
- for dim_i in range(n_dims):
- for dim_j in np.arange(dim_i + 1, n_dims):
- rho_hat[dim_i, dim_j] = np.sum(
- std_samples[dim_i] * std_samples[dim_j]) / n_samples
- rho_hat[dim_j, dim_i] = rho_hat[dim_i, dim_j]
-
- # make sure rho_hat is positive semidefinite
- try:
- L = cholesky(rho_hat, lower=True) # if this works, we're good
-
- except: # otherwise, we can try to fix the matrix using SVD
-
- try:
- U, s, V = svd(rho_hat, )
- except:
- return None
-
- S = np.diagflat(s)
-
- rho_hat = U @ S @ U.T
- np.fill_diagonal(rho_hat, 1.0)
-
- if ((np.max(rho_hat) > 1.0) or (np.min(rho_hat) < -1.0)):
- return None
-
- return rho_hat
-
-
-def _mvn_scale(x, rho):
-
- x = np.atleast_2d(x)
- n_dims = x.shape[1]
-
- rho_0 = np.zeros((n_dims, n_dims))
- np.fill_diagonal(rho_0, 1)
- a = mvn.pdf(x, mean=np.zeros(n_dims), cov=rho_0)
-
- b = mvn.pdf(x, mean=np.zeros(n_dims), cov=rho)
-
- return b / a
-
-
-def _neg_log_likelihood(params, inits, bnd_lower, bnd_upper, samples,
- distribution, tr_limits, det_limits, censored_count,
- enforce_bounds=False):
-
- # First, check if the parameters are within the pre-defined bounds
- if enforce_bounds:
- if ((params > bnd_lower) & (params < bnd_upper)).all(0) == False:
- # if they are not, then return a large value to discourage the
- # optimization algorithm from going in that direction
- return 1e10
-
- # If there is nan in params, return a large value
- if np.isnan(np.sum(params)):
- return 1e10
-
- params = np.reshape(params, inits.shape)
- n_dims, n_samples = samples.shape
-
- theta = get_theta(params, inits, distribution)
-
- likelihoods = np.zeros(samples.shape)
-
- # calculate the marginal likelihoods
- for i, (theta_i, samples_i, tr_lim_i) in enumerate(
- zip(theta, samples, tr_limits)):
-
- # consider truncation if needed
- p_a, p_b = _get_limit_probs(tr_lim_i, distribution, theta_i)
- tr_alpha = p_b - p_a # this is the probability mass within the
- # truncation limits
-
- # calculate the likelihood for each available sample
- if distribution in ['normal', 'lognormal']:
- likelihoods[i] = norm.pdf(samples_i, loc=theta_i[0],
- scale=theta_i[1]) / tr_alpha
-
- # transform every sample into standard normal space and get the correlation
- # matrix
- std_samples = _get_std_samples(samples, theta, tr_limits, distribution)
- rho_hat = _get_std_corr_matrix(std_samples)
- if rho_hat is None:
- return 1e10
-
- # likelihoods related to censoring need to be handled together
- if censored_count > 0:
-
- det_lower = np.zeros(n_dims)
- det_upper = np.zeros(n_dims)
-
- for i, (theta_i, tr_lim_i, det_lim_i) in enumerate(
- zip(theta, tr_limits, det_limits)):
- # also prepare the standardized detection limits
- p_a, p_b = _get_limit_probs(tr_lim_i, distribution, theta_i)
- p_l, p_u = _get_limit_probs(det_lim_i, distribution, theta_i)
-
- # rescale to consider truncation
- p_l, p_u = [np.min([np.max([lim, p_a]), p_b]) for lim in [p_l, p_u]]
- p_l, p_u = [(lim - p_a) / (p_b - p_a) for lim in [p_l, p_u]]
-
- det_lower[i], det_upper[i] = norm.ppf([p_l, p_u], loc=0., scale=1.)
-
- # get the likelihood of censoring a sample
- det_alpha, eps_alpha = mvn_orthotope_density(np.zeros(n_dims), rho_hat,
- det_lower, det_upper)
-
- # Make sure that det_alpha is estimated with sufficient accuracy
- if det_alpha <= 100. * eps_alpha:
- return 1e10
-
- # make sure that the likelihood is a positive number
- cen_likelihood = max(1.0 - det_alpha, np.nextafter(0, 1))
-
- else:
- # If the data is not censored, use 1.0 for cen_likelihood to get a
- # zero log-likelihood later. Note that although this is
- # theoretically not correct, it does not alter the solution and
- # it is numerically much more convenient than working around the
- # log of zero likelihood.
- cen_likelihood = 1.0
-
- # flatten the likelihoods calculated in each dimension
- try:
- scale = _mvn_scale(std_samples.T, rho_hat)
- except:
- return 1e10
- likelihoods = np.prod(likelihoods, axis=0) * scale
-
- # Zeros are a result of limited floating point precision. Replace them
- # with the smallest possible positive floating point number to
- # improve convergence.
- likelihoods = np.clip(likelihoods, a_min=np.nextafter(0, 1), a_max=None)
-
- # calculate the total negative log likelihood
- NLL = -(np.sum(np.log(likelihoods)) # from samples
- + censored_count * np.log(cen_likelihood)) # censoring influence
-
- # normalize the NLL with the sample count
- NLL = NLL / samples.size
-
- # print(theta[0], NLL)
-
- return NLL
-
-[docs]def fit_distribution(raw_samples, distribution, truncation_limits=[None, None],
- censored_count=0, detection_limits=[None, None],
- multi_fit=False, alpha_lim=1e-4):
- """
- Fit a distribution to samples using maximum likelihood estimation.
-
- The number of dimensions of the distribution are inferred from the
- shape of the sample data. Censoring is automatically considered if the
- number of censored samples and the corresponding detection limits are
- provided. Infinite or unspecified truncation limits lead to fitting a
- non-truncated distribution in that dimension.
-
- Parameters
- ----------
- raw_samples: float ndarray
- Raw data that serves as the basis of estimation. The number of samples
- equals the number of columns and each row introduces a new feature. In
- other words: a list of sample lists is expected where each sample list
- is a collection of samples of one variable.
- distribution: {'normal', 'lognormal'}
- Defines the target probability distribution type. Different types of
- distributions can be mixed by providing a list rather than a single
- value. Each element of the list corresponds to one of the features in
- the raw_samples.
- truncation_limits: float ndarray, optional, default: [None, None]
- Lower and/or upper truncation limits for the specified distributions.
- A two-element vector can be used for a univariate case, while two lists
- of limits are expected in multivariate cases. If the distribution is
- non-truncated from one side in a subset of the dimensions, use either
- `None` or assign an infinite value (i.e. numpy.inf) to those dimensions.
- censored_count: int, optional, default: None
- The number of censored samples that are beyond the detection limits.
- All samples outside the detection limits are aggregated into one set.
- This works the same way in one and in multiple dimensions. Prescription
- of specific censored sample counts for sub-regions of the input space
- outside the detection limits is not supported.
- detection_limits: float ndarray, optional, default: [None, None]
- Lower and/or upper detection limits for the provided samples. A
- two-element vector can be used for a univariate case, while two lists
- of limits are expected in multivariate cases. If the data is not
- censored from one side in a subset of the dimensions, use either `None`
- or assign an infinite value (i.e. numpy.inf) to those dimensions.
- multi_fit: bool, optional, default: False
- If True, we attempt to fit a multivariate distribution to the samples.
- Otherwise, we fit each marginal univariate distribution independently
- and estimate the correlation matrix in the end based on the fitted
- marginals. Using multi_fit can be advantageous with censored data and
- if the correlation in the data is not Gaussian. It leads to
- substantially longer calculation time and does not always produce
- better results, especially when the number of dimensions is large.
- alpha_lim: float, optional, default:None
- Introduces a lower limit to the probability density within the
- n-orthotope defined by the truncation limits. Assigning a reasonable
- minimum (such as 1e-4) can be useful when the mean of the distribution
- is several standard deviations from the truncation limits and the
- sample size is small. Such cases without a limit often converge to
- distant means with inflated variances. Besides being incorrect
- estimates, those solutions only offer negligible reduction in the
- negative log likelihood, while making subsequent sampling of the
- truncated normal distribution very challenging.
-
- Returns
- -------
- theta: float ndarray
- Estimates of the parameters of the fitted probability distribution in
- each dimension. The following parameters are returned for the supported
- distributions:
- normal - mean, standard deviation;
- lognormal - median, log standard deviation;
- Rho: float 2D ndarray, optional
- In the multivariate case, returns the estimate of the correlation
- matrix.
- """
- samples = np.atleast_2d(raw_samples)
- tr_limits = np.atleast_2d(truncation_limits)
- det_limits = np.atleast_2d(detection_limits)
- n_dims, n_samples = samples.shape
-
- if (tr_limits.shape[0] == 1) and (samples.shape[0] != 1):
- tr_limits = np.tile(tr_limits[0], samples.shape[0]
- ).reshape([samples.shape[0], 2])
-
- if (det_limits.shape[0] == 1) and (samples.shape[0] != 1):
- det_limits = np.tile(det_limits[0], samples.shape[0]
- ).reshape([samples.shape[0], 2])
-
- # Convert samples to log space if the distribution is lognormal
- if distribution == 'lognormal':
- samples = np.log(samples)
- for dim in range(tr_limits.shape[0]):
- for var in range(tr_limits.shape[1]):
- if tr_limits[dim][var] is not None:
- tr_limits[dim][var] = np.log(tr_limits[dim][var])
- for dim in range(det_limits.shape[0]):
- for var in range(det_limits.shape[1]):
- if det_limits[dim][var] is not None:
- det_limits[dim][var] = np.log(det_limits[dim][var])
-
-
- # Define initial values of distribution parameters
- if distribution in ['normal', 'lognormal']:
- # use the first two moments for normal distribution
- mu_init = np.mean(samples, axis=1)
-
- # replace zero standard dev with negligible standard dev
- sig_init = np.std(samples, axis=1)
- sig_zero_id = np.where(sig_init == 0.0)[0]
- sig_init[sig_zero_id] = 1e-6 * np.abs(mu_init[sig_zero_id])
-
- # prepare a vector of initial values
- # Note: The actual optimization uses zeros as initial parameters to
- # avoid bias from different scales. These initial values are sent to
- # the likelihood function and considered in there.
- inits = np.transpose([mu_init, sig_init])
-
- # Define the bounds for each input (assuming standardized initials)
- # These bounds help avoid unrealistic results and improve the
- # convergence rate
- bnd_lower = np.array([[-10.0, -5.0] for t in range(n_dims)])
- bnd_upper = np.array([[10.0, 5.0] for t in range(n_dims)])
-
- bnd_lower = bnd_lower.flatten()
- bnd_upper = bnd_upper.flatten()
-
- #inits_0 = np.copy(inits)
-
- # There is nothing to gain from a time-consuming optimization if..
- # the number of samples is too small
- if ((n_samples < 3) or
- # there are no truncation or detection limits involved
- (np.all(tr_limits == None) and np.all(det_limits == None))):
-
- # In this case, it is typically hard to improve on the method of
- # moments estimates for the parameters of the marginal distributions
- theta = inits
-
- # Otherwise, we run the optimization that aims to find the parameters that
- # maximize the likelihood of observing the samples
- else:
-
- # First, optimize for each marginal independently
- for dim in range(n_dims):
-
- inits_i = inits[dim:dim + 1]
-
- tr_limits_i = [None, None]
- for lim in range(2):
- if ((tr_limits[dim][lim] is None) and
- (det_limits[dim][lim] is not None)):
- tr_limits_i[lim] = det_limits[dim][lim]
- elif det_limits[dim][lim] is not None:
- tr_limits_i[lim] = np.max([tr_limits[dim][lim],
- det_limits[dim][lim]])
- else:
- tr_limits_i[lim] = tr_limits[dim][lim]
-
- out_m_i = minimize(_neg_log_likelihood,
- np.zeros(inits[dim].size),
- args=(inits_i,
- bnd_lower[dim:dim + 1],
- bnd_upper[dim:dim + 1],
- samples[dim:dim + 1],
- distribution,
- [tr_limits_i, ],
- [None, None],
- 0, True,),
- method='BFGS',
- options=dict(maxiter=50)
- )
-
- out = out_m_i.x.reshape(inits_i.shape)
- theta = get_theta(out, inits_i, distribution)
- inits[dim] = theta[0]
-
- # Second, if requested, we attempt the multivariate fitting using the
- # marginal results as initial parameters.
- if multi_fit or (censored_count > 0):
-
- out_m = minimize(_neg_log_likelihood,
- np.zeros(inits.size),
- args=(inits, bnd_lower, bnd_upper, samples,
- distribution, tr_limits, det_limits,
- censored_count, True,),
- method='BFGS',
- options=dict(maxiter=50)
- )
-
- out = out_m.x.reshape(inits.shape)
- theta = get_theta(out, inits, distribution)
-
- else:
- theta = inits
-
- # Calculate rho in the standard normal space because we will generate new
- # samples using that type of correlation (i.e., Gaussian copula)
- std_samples = _get_std_samples(samples, theta, tr_limits, distribution)
- rho_hat = _get_std_corr_matrix(std_samples)
- if rho_hat is None:
- # If there is not enough data to produce a valid correlation matrix
- # estimate, we assume independence
- # TODO: provide a warning for the user
- rho_hat = np.zeros((n_dims, n_dims))
- np.fill_diagonal(rho_hat, 1.0)
-
- # Convert mean back to linear space if the distribution is lognormal
- if distribution == 'lognormal':
- theta_mod = theta.T.copy()
- theta_mod[0] = np.exp(theta_mod[0])
- theta = theta_mod.T
-
- #for val in list(zip(inits_0, theta)):
- # print(val)
-
- return theta, rho_hat
-
-[docs]class RandomVariable(object):
- """
- Description
-
- Parameters
- ----------
- name: string
- A unique string that identifies the random variable.
- distribution: {'normal', 'lognormal', 'multinomial', 'custom', 'empirical',
- 'coupled_empirical', 'uniform'}, optional
- Defines the type of probability distribution for the random variable.
- theta: float scalar or ndarray, optional
- Set of parameters that define the cumulative distribution function of
- the variable given its distribution type. The following parameters are
- expected currently for the supported distribution types:
- normal - mean, standard deviation;
- lognormal - median, log standard deviation;
- uniform - a, b, the lower and upper bounds of the distribution;
- multinomial - likelihood of each unique event (the last event's
- likelihood is adjusted automatically to ensure the likelihoods sum up
- to one);
- custom - according to the custom expression provided;
- empirical and coupled_empirical - N/A.
- truncation_limits: float ndarray, optional
- Defines the [a,b] truncation limits for the distribution. Use None to
- assign no limit in one direction.
- bounded: float ndarray, optional
- Defines the [P_a, P_b] probability bounds for the distribution. Use None
- to assign no lower or upper bound.
- custom_expr: string, optional
- Provide an expression that is a Python syntax for a custom CDF. The
- controlling variable shall be "x" and the parameters shall be "p1",
- "p2", etc.
- anchor: RandomVariable, optional
- Anchors this to another variable. If the anchor is not None, this
- variable will be perfectly correlated with its anchor. Note that
- the attributes of this variable and its anchor do not have to be
- identical.
- """
-
- def __init__(self, name, distribution, theta=None, truncation_limits=None,
- bounds=None, custom_expr=None, raw_samples=None, anchor=None):
-
- self.name = name
-
- if ((distribution not in ['empirical', 'coupled_empirical']) and
- (theta is None)):
- raise ValueError(
- f"A random variable that follows a {distribution} distribution "
- f"is characterized by a set of parameters (theta). The "
- f"parameters need to be provided when the RV is created."
- )
-
- if distribution == 'multinomial':
- if np.sum(theta) > 1:
- raise ValueError(
- f"The set of p values provided for a multinomial "
- f"distribution shall sum up to less than or equal to 1.0. "
- f"The provided values sum up to {np.sum(theta)}. p = "
- f"{theta} ."
- )
-
- # save the other parameters internally
- self._distribution = distribution
- self._theta = theta
- self._truncation_limits = truncation_limits
- self._bounds = bounds
- self._custom_expr = custom_expr
- self._raw_samples = np.atleast_1d(raw_samples)
- self._uni_samples = None
- self._RV_set = None
- if anchor == None:
- self._anchor = self
- else:
- self._anchor = anchor
-
- @property
- def distribution(self):
- """
- Return the assigned probability distribution type.
- """
- return self._distribution
-
- @property
- def theta(self):
- """
- Return the assigned probability distribution parameters.
- """
- return self._theta
-
- @theta.setter
- def theta(self, value):
- """
- Assign an anchor to the random variable
- """
- self._theta = value
-
- @property
- def truncation_limits(self):
- """
- Return the assigned truncation limits.
- """
- return self._truncation_limits
-
- @property
- def bounds(self):
- """
- Return the assigned probability bounds.
- """
- return self._bounds
-
- @property
- def custom_expr(self):
- """
- Return the assigned custom expression for CDF.
- """
- return self._custom_expr
-
- @property
- def RV_set(self):
- """
- Return the RV_set this RV is a member of
- """
- return self._RV_set
-
- @RV_set.setter
- def RV_set(self, value):
- """
- Assign an RV_set to this RV
- """
- self._RV_set = value
-
- @property
- def samples(self):
- """
- Return the empirical or generated samples.
- """
- return self._samples
-
- @property
- def samples_DF(self):
- """
- Return the empirical or generated samples in a pandas Series.
- """
- return self._samples_DF
-
- @samples.setter
- def samples(self, value):
- """
- Assign samples to the random variable
- """
- self._samples = value
- self._samples_DF = pd.Series(value)
-
- @property
- def uni_samples(self):
- """
- Return the samples from the controlling uniform distribution.
- """
- return self._anchor._uni_samples
-
- @uni_samples.setter
- def uni_samples(self, value):
- """
- Assign the controlling samples to the random variable
-
- Parameters
- ----------
- value: float ndarray
- An array of floating point values in the [0, 1] domain.
- """
- self._uni_samples = value
-
- @property
- def anchor(self):
- """
- Return the anchor of the variable (if any).
- """
- return self._anchor
-
- @anchor.setter
- def anchor(self, value):
- """
- Assign an anchor to the random variable
- """
- self._anchor = value
-
-[docs] def cdf(self, values):
- """
- Returns the cdf at the given values
- """
- result = None
-
- if self.distribution == 'normal':
- mu, sig = self.theta
-
- if self.truncation_limits is not None:
- a, b = self.truncation_limits
-
- if a is None:
- a = -np.inf
- if b is None:
- b = np.inf
-
- p_a, p_b = [norm.cdf((lim - mu) / sig) for lim in [a, b]]
-
- # cap the values at the truncation limits
- values = np.minimum(np.maximum(values, a), b)
-
- # get the cdf from a non-truncated normal
- p_vals = norm.cdf(values, loc=mu, scale=sig)
-
- # adjust for truncation
- result = (p_vals - p_a) / (p_b - p_a)
-
- else:
- result = norm.cdf(values, loc=mu, scale=sig)
-
- elif self.distribution == 'lognormal':
- theta, beta = self.theta
-
- if self.truncation_limits is not None:
- a, b = self.truncation_limits
-
- if a is None:
- a = np.nextafter(0, 1)
- if b is None:
- b = np.inf
-
- p_a, p_b = [norm.cdf((np.log(lim) - np.log(theta)) / beta)
- for lim in [a, b]]
-
- # cap the values at the truncation limits
- values = np.minimum(np.maximum(values, a), b)
-
- # get the cdf from a non-truncated lognormal
- p_vals = norm.cdf(np.log(values), loc=np.log(theta), scale=beta)
-
- # adjust for truncation
- result = (p_vals - p_a) / (p_b - p_a)
-
- else:
- values = np.maximum(values, np.nextafter(0, 1))
-
- result = norm.cdf(np.log(values), loc=np.log(theta), scale=beta)
-
- elif self.distribution == 'uniform':
- a, b = self.theta
-
- if a is None:
- a = -np.inf
- if b is None:
- b = np.inf
-
- if self.truncation_limits is not None:
- a, b = self.truncation_limits
-
- result = uniform.cdf(values, loc=a, scale=b-a)
-
- return result
-
-
-[docs] def inverse_transform(self, values):
- """
- Uses inverse probability integral transformation on the provided values.
- """
- result = None
-
- if self.distribution == 'normal':
- mu, sig = self.theta
-
- if self.truncation_limits is not None:
- a, b = self.truncation_limits
-
- if a is None:
- a = -np.inf
- if b is None:
- b = np.inf
-
- p_a, p_b = [norm.cdf((lim-mu)/sig) for lim in [a, b]]
-
- if p_b - p_a == 0:
- raise ValueError(
- "The probability mass within the truncation limits is "
- "too small and the truncated distribution cannot be "
- "sampled with sufficiently high accuracy. This is most "
- "probably due to incorrect truncation limits set for "
- "the distribution."
- )
-
- result = norm.ppf(values * (p_b - p_a) + p_a,
- loc=mu, scale=sig)
-
- else:
- result = norm.ppf(values, loc=mu, scale=sig)
-
- elif self.distribution == 'lognormal':
- theta, beta = self.theta
-
- if self.truncation_limits is not None:
- a, b = self.truncation_limits
-
- if a is None:
- a = np.nextafter(0, 1)
- else:
- a = np.maximum(np.nextafter(0, 1), a)
-
- if b is None:
- b = np.inf
-
- p_a, p_b = [norm.cdf((np.log(lim) - np.log(theta)) / beta)
- for lim in [a, b]]
-
- result = np.exp(
- norm.ppf(values * (p_b - p_a) + p_a,
- loc=np.log(theta), scale=beta))
-
- else:
- result = np.exp(norm.ppf(values, loc=np.log(theta), scale=beta))
-
- elif self.distribution == 'uniform':
- a, b = self.theta
-
- if a is None:
- a = -np.inf
- if b is None:
- b = np.inf
-
- if self.truncation_limits is not None:
- a, b = self.truncation_limits
-
- result = uniform.ppf(values, loc=a, scale=b-a)
-
- elif self.distribution == 'empirical':
-
- s_ids = (values * len(self._raw_samples)).astype(int)
- result = self._raw_samples[s_ids]
-
- elif self.distribution == 'coupled_empirical':
-
- raw_sample_count = len(self._raw_samples)
- new_sample_count = len(values)
- new_samples = np.tile(self._raw_samples,
- int(new_sample_count/raw_sample_count)+1)
- result = new_samples[:new_sample_count]
-
- elif self.distribution == 'multinomial':
-
- p_cum = np.cumsum(self.theta)[:-1]
-
- samples = values
-
- for i, p_i in enumerate(p_cum):
- samples[samples < p_i] = 10 + i
- samples[samples <= 1.0] = 10 + len(p_cum)
-
- result = samples - 10
-
- return result
-
-[docs] def inverse_transform_sampling(self):
- """
- Creates samples using inverse probability integral transformation.
- """
-
- self.samples = self.inverse_transform(self.uni_samples)
-
-[docs]class RandomVariableSet(object):
- """
- Description
-
- Parameters
- ----------
- name: string
- A unique string that identifies the set of random variables.
- RV_list: list of RandomVariable
- Defines the random variables in the set
- Rho: float 2D ndarray
- Defines the correlation matrix that describes the correlation between
- the random variables in the set. Currently, only the Gaussian copula
- is supported.
- """
-
- def __init__(self, name, RV_list, Rho):
-
- self.name = name
-
- if len(RV_list) > 1:
-
- # put the RVs in a dictionary for more efficient access
- reorder = np.argsort([RV.name for RV in RV_list])
- self._variables = dict([(RV_list[i].name, RV_list[i]) for i in reorder])
-
- # reorder the entries in the correlation matrix to correspond to the
- # sorted list of RVs
- self._Rho = np.asarray(Rho[(reorder)].T[(reorder)].T)
-
- else: # if there is only one variable (for testing, probably)
- self._variables = dict([(rv.name, rv) for rv in RV_list])
- self._Rho = np.asarray(Rho)
-
- # assign this RV_set to the variables
- for __, var in self._variables.items():
- var.RV_set = self
-
- @property
- def RV(self):
- """
- Return the random variable(s) assigned to the set
- """
- return self._variables
-
- @property
- def size(self):
- """
- Return the size (i.e., number of variables in the) RV set
- """
- return len(self._variables)
-
- @property
- def samples(self):
- """
- Return the samples of the variables in the set
- """
- return dict([(name, rv.samples) for name, rv
- in self._variables.items()])
-
-[docs] def Rho(self, var_subset=None):
- """
- Return the (subset of the) correlation matrix.
- """
- if var_subset is None:
- return self._Rho
- else:
- var_ids = [list(self._variables.keys()).index(var_i)
- for var_i in var_subset]
- return (self._Rho[var_ids]).T[var_ids]
-
-[docs] def apply_correlation(self):
- """
- Apply correlation to n dimensional uniform samples.
-
- Currently, correlation is applied using a Gaussian copula. First, we
- try using Cholesky transformation. If the correlation matrix is not
- positive semidefinite and Cholesky fails, use SVD to apply the
- correlations while preserving as much as possible from the correlation
- matrix.
- """
-
- U_RV = np.array([RV.uni_samples for RV_name, RV in self.RV.items()])
-
- # First try doing the Cholesky transformation
- try:
- N_RV = norm.ppf(U_RV)
-
- L = cholesky(self._Rho, lower=True)
-
- NC_RV = L @ N_RV
-
- UC_RV = norm.cdf(NC_RV)
-
- except:
-
- # if the Cholesky doesn't work, we need to use the more
- # time-consuming but more robust approach based on SVD
- N_RV = norm.ppf(U_RV)
-
- U, s, __ = svd(self._Rho, )
- S = np.diagflat(np.sqrt(s))
-
- NC_RV = (N_RV.T @ S @ U.T).T
-
- UC_RV = norm.cdf(NC_RV)
-
- for (RV_name, RV), uc_RV in zip(self.RV.items(), UC_RV):
- RV.uni_samples = uc_RV
-
-[docs] def orthotope_density(self, lower=None, upper=None, var_subset=None):
- """
- Estimate the probability density within an orthotope for the RV set.
-
- Use the mvn_orthotope_density function in this module for the
- calculation. The distribution of individual RVs is not limited to the
- normal family. The provided limits are converted to the standard normal
- space that is the basis of all RVs in pelicun. Truncation limits and
- correlation (using Gaussian copula) are automatically taken into
- consideration.
-
- Parameters
- ----------
- lower: float ndarray, optional, default: None
- Lower bound(s) of the orthotope. A scalar value can be used for a
- univariate RV; a list of bounds is expected in multivariate cases.
- If the orthotope is not bounded from below in a dimension, use
- 'None' to that dimension.
- upper: float ndarray, optional, default: None
- Upper bound(s) of the orthotope. A scalar value can be used for a
- univariate RV; a list of bounds is expected in multivariate cases.
- If the orthotope is not bounded from above in a dimension, use
- 'None' to that dimension.
- var_subset: list of strings, optional, default: None
- If provided, allows for selecting only a subset of the variables in
- the RV_set for the density calculation.
-
- Returns
- -------
- alpha: float
- Estimate of the probability density within the orthotope.
- eps_alpha: float
- Estimate of the error in alpha.
-
- """
-
- if lower is not None:
- target_shape = lower.shape
- elif upper is not None:
- target_shape = upper.shape
- else:
- return 1.0
-
- lower_std = np.full(target_shape, None)
- upper_std = np.full(target_shape, None)
-
- # collect the variables involved
- if var_subset is None:
- vars = list(self._variables.keys())
- else:
- vars = var_subset
-
- # first, convert limits to standard normal values
- for var_i, var_name in enumerate(vars):
-
- var = self._variables[var_name]
-
- if (lower is not None) and (lower[var_i] is not None):
- lower_std[var_i] = norm.ppf(var.cdf(lower[var_i]), loc=0, scale=1)
-
- if (upper is not None) and (upper[var_i] is not None):
- upper_std[var_i] = norm.ppf(var.cdf(upper[var_i]), loc=0, scale=1)
-
- # then calculate the orthotope results in std normal space
- lower_std = lower_std.T
- upper_std = upper_std.T
-
- OD = [mvn_orthotope_density(mu=np.zeros(len(vars)),
- COV=self.Rho(var_subset),
- lower=l_i, upper=u_i)[0]
- for l_i, u_i in zip(lower_std, upper_std)]
-
- return np.asarray(OD)
-
-[docs]class RandomVariableRegistry(object):
- """
- Description
-
- Parameters
- ----------
-
- """
-
- def __init__(self):
-
- self._variables = {}
- self._sets = {}
-
- @property
- def RV(self):
- """
- Return all random variable(s) in the registry
- """
- return self._variables
-
-[docs] def RVs(self, keys):
- """
- Return a subset of the random variables in the registry
- """
- return {name:self._variables[name] for name in keys}
-
-[docs] def add_RV(self, RV):
- """
- Add a new random variable to the registry.
- """
- self._variables.update({RV.name: RV})
-
- @property
- def RV_set(self):
- """
- Return the random variable set(s) in the registry.
- """
- return self._sets
-
-[docs] def add_RV_set(self, RV_set):
- """
- Add a new set of random variables to the registry
- """
- self._sets.update({RV_set.name: RV_set})
-
- @property
- def RV_samples(self):
- """
- Return the samples for every random variable in the registry
- """
- return dict([(name, rv.samples) for name,rv in self.RV.items()])
-
-
-[docs] def generate_samples(self, sample_size, method='LHS_midpoint', seed=None):
- """
- Generates samples for all variables in the registry.
-
- Parameters
- ----------
-
- sample_size: int
- The number of samples requested per variable.
- method: {'random', 'LHS', 'LHS_midpoint'}, optional
- The sample generation method to use. 'random' stands for
- conventional random sampling; 'LHS' is Latin HyperCube Sampling
- with random sample location within each bin of the hypercube;
- 'LHS_midpoint' is like LHS, but the samples are assigned to the
- midpoints of the hypercube bins.
- seed: int, optional
- Random seed used for sampling.
- """
- # Initialize the random number generator
- rng = np.random.default_rng(seed)
-
- # Generate a dictionary with IDs of the free (non-anchored) variables
- RV_list = [RV_name for RV_name, RV in self.RV.items() if
- RV.anchor == RV]
- RV_ID = dict([(RV_name, ID) for ID, RV_name in enumerate(RV_list)])
- RV_count = len(RV_ID)
-
- # Generate controlling samples from a uniform distribution for free RVs
- if 'LHS' in method:
- bin_low = np.array([rng.permutation(sample_size)
- for i in range(RV_count)])
-
- if method == 'LHS_midpoint':
- U_RV = np.ones([RV_count, sample_size]) * 0.5
- U_RV = (bin_low + U_RV) / sample_size
-
- elif method == 'LHS':
- U_RV = rng.random(size=[RV_count, sample_size])
- U_RV = (bin_low + U_RV) / sample_size
-
- elif method == 'random':
- U_RV = rng.random(size=[RV_count, sample_size])
-
- # Assign the controlling samples to the RVs
- for RV_name, RV_id in RV_ID.items():
- self.RV[RV_name].uni_samples = U_RV[RV_id]
-
- # Apply correlations for the pre-defined RV sets
- for RV_set_name, RV_set in self.RV_set.items():
- # prepare the correlated uniform distribution for the set
- RV_set.apply_correlation()
-
- # Convert from uniform to the target distribution for every RV
- for RV_name, RV in self.RV.items():
- RV.inverse_transform_sampling()
-