Commit 70f93e62 authored by Graeme Weatherill's avatar Graeme Weatherill
Browse files

Updates to installation and documentation, Fix of Vs30 bug in Point Workflow

parent 038c91d2
......@@ -143,3 +143,5 @@ dmypy.json
# OSX
.DS_Store
# Don't track site data files
site_data/
No preview for this file type
......@@ -12,25 +12,32 @@ License (document): Creative Commons CC-By-4.0
---
```
The exposure to site tool is a short(ish) Python 3 module for constructing the SERA site amplification files for a given exposure model configuration. It supports several different workflows, depending on whether the user wishes to define the resulting site model from a regularly spaced grid, an existing exposure model or simply a shapefile of administative regions.
The exposure to site tool is a short(ish) Python 3 module for constructing the ESRM20 site amplification files for a given exposure model configuration. It supports several different workflows, depending on whether the user wishes to define the resulting site model from a regularly spaced grid, an existing exposure model or simply a shapefile of administative regions.
### Before you start
The tools require additional data files not included with the main code. These files contain the regional scale European site input data. More details can be found in the description below. The data files are distributed with a Createive Commons CC-By-4.0 license and are available for download from:
<https://nextcloud.gfz-potsdam.de/s/93ZR4ky8D4mDXb9>
### Installation
The tool requires as standard set of Python libraries that can be installed via most common Python package managers. A `setup.py` file is included to facilitate installation. We recommend using a Python virtual environment for installing and running the tools (<https://docs.python.org/3.6/tutorial/venv.html>).
The tool requires as standard set of Python libraries that can be installed via most common Python package managers. A `setup.py` file is included to facilitate installation. We recommend using a Python virtual environment for installing and running the tools (<https://docs.python.org/3.8/tutorial/venv.html>).
The required dependencies are:
(Required)
* Python >=3.
* h5py >=2.10, <2.11
* Numpy >=1.18, <1.20
* Scipy >=1.3, <1.7
* Pandas >=0.25 <1.3
* Shapely >=1.7, < 1.8
* Pyproj >=1.9
* matplotlib >=3.0
* geopandas >=0.9.0
* Python >=3.7
* h5py >=2.10
* Numpy >=1.18
* Scipy >=1.3
* Pandas >=0.25
* Shapely >=1.7, < 2.0
* Pyproj >= 1.9
* matplotlib >= 3.0
* geopandas >= 0.9.0
(Additional)
......@@ -68,9 +75,9 @@ pip install --update pip
```
Then run:
```bash
cd esrm20_sitemodel/exposure_site_tool/
pip install -e .
pip install -e esrm20_site_model/
```
This should install the dependencies and the `exposure2site` package. Additional packages to help run the code can be installed via
......@@ -80,6 +87,12 @@ pip install ipython
pip install jupyter
```
<i><b>Note</b>: some installations may also require `pygeos`. If an error is raised indicating that it is not availablr then it can be installed via:</i>
```bash
pip install pygeos
```
##### Windows
......@@ -90,7 +103,7 @@ For other users we suggest installing and running the toolkit using Anaconda. De
To create the virtual environment via `conda` run:
```bash
conda create --name exp2site python=3.7
conda create --name exp2site python=3.8
conda activate exp2site
```
......@@ -109,10 +122,10 @@ pip install --update pip
Navigate to the `exposure_site_tool` and run the installation
```bash
cd esrm20_sitemodel\exposure_site_tool\
pip install -e .
pip install -e esrm20_site_model/
```
This should install the dependencies and the `exposure2site` package. Additional packages to help run the code can be installed via
```bash
......@@ -125,7 +138,11 @@ pip install jupyter
## Required Datasets
In addition to the tools themselves, you will need several files containing pre-processed site data, which can be made available on request from the author (<graeme.weatherill@gfz-potsdam.de>). These should be stored in a subdirectory named `data`, and placed in the folder where the `exposure_to_site_tools.py` code is found.
In addition to the tools themselves, you will need several files containing pre-processed site data.
The zipped file containing all of the necessary data is available for download from here: <https://nextcloud.gfz-potsdam.de/s/93ZR4ky8D4mDXb9>
The downloaded data should be unzipped and stored in a directory named `site_data`, this directory should then be placed into the main code within the folder `exposure2site`.
The required datasets comprise 5 files:
......@@ -173,10 +190,10 @@ Longitudes and latitudes in all 30 arc-second data sets refer to the locations o
# General Usage
The Ex2Site tool is mostly intended for command line usage (although Jupyter Notebooks illustrating how to call the functions directly are also provided). With the `exposure_to_site_tools.py` file in the current working directory, and the data files in the subdirectory `./data` the tool is run by:
The Ex2Site tool is mostly intended for command line usage (although Jupyter Notebooks illustrating how to call the functions directly are also provided). With the `exposure_to_site_tools.py` file in the current working directory, and the data files in the subdirectory `./site_data` the tool is run by:
```bash
python exposure_to_site_tools.py --run=choice_of_workflow --output-file=path/to/output_site_model.csv
exposure2site --run=choice_of_workflow --output-file=path/to/output_site_model.csv
```
......@@ -269,7 +286,7 @@ The weighting approach described previously may be applicable for the case that
The following command would produce a 120 arc-second grid between 22˚E and 30˚E and 45˚N to 51.5˚N, clipping only to onshore sites, applying building density weighing and taking the resulting property as the maximum numerical value and modal geological category. The result will be exported to `example_site_model.csv`
```bash
python exposure_to_site_tools.py --run grid --output-file example_site_model.csv --bbox 22/45/30/51.5 --weighting True --averaging max --geological-weighting mode --onshore-only True
exposure2site --run grid --output-file example_site_model.csv --bbox 22/45/30/51.5 --weighting True --averaging max --geological-weighting mode --onshore-only True
```
***
......@@ -297,7 +314,7 @@ All of the optional arguments specified for the `grid` option can be used here,
The following command will read the exposure model in the file `Exposure_Res_Dummy.csv`, whose geometry can be found in the file `Adm2_Dummy.shp` in the directory `Country_Shapefiles`. Weighting based on built density will be applied, this time with the value of the site property at the maximum weighted location. Geology will be determined from the modal geological category within the polygon. The result will be written to the output xml file `Dummy_Site_Model.xml`
```bash
python exposure_to_site_tools.py --run exposure --output-file Dummy_Site_Model.xml --input-exposure ./Exposure_Res_Dummy.csv --shapefile-dir ./Country_Shapefiles --weighting True --averaging max --geological-weighting mode
exposure2site --run exposure --output-file Dummy_Site_Model.xml --input-exposure ./Exposure_Res_Dummy.csv --shapefile-dir ./Country_Shapefiles --weighting True --averaging max --geological-weighting mode
```
***
......@@ -357,7 +374,7 @@ The required input file can be any comma-separated text file, but the target sit
The following command will define the site model from a simple csv file of longitudes and latitudes (`./site_locations.csv`) and export the resulting model to `./example_point_site_model.csv`, clipping only to onshore sites
```bash
python exposure_to_site_tools.py --run point --output-file ./example_point_site_model.csv --input-file ./site_locations.csv --onshore-only True
exposure2site --run point --output-file ./example_point_site_model.csv --input-file ./site_locations.csv --onshore-only True
```
## `join`
......@@ -377,7 +394,7 @@ One of either:
The following command joins together the site model in the file `site_model_countryA.csv` and the site model in the file `site_model_countryB.xml`, exporting them to `site_model_countriesAB.csv`:
```bash
python exposure_to_site_tools.py --run join --output-file site_model_countriesAB.csv --files site_model_countryA.csv+site_model_countryB.xml
exposure2site --run join --output-file site_model_countriesAB.csv --files site_model_countryA.csv+site_model_countryB.xml
```
......
from exposure2site.exposure_to_site_tools import SiteManager
from exposure2site.exposure_to_site_tools import (
SiteManager, get_site_set_from_exposure, get_maximum_admin_level)
from exposure2site.node_handler import Node, nrml_read, nrml_write
__all__ = [
"SiteManager",
"get_site_set_from_exposure",
"get_maximum_admin_level",
"Node",
"nrml_read",
"nrml_write",
......
......@@ -51,7 +51,7 @@ GEOL_DICT_NAME_TO_KEY = {"UNKNOWN": 0,
# Data files
DATA_PATH = os.path.join(os.path.dirname(__file__), "data")
DATA_PATH = os.path.join(os.path.dirname(__file__), "site_data")
# Geology shapefiles
GEOLOGY_FILE = os.path.join(DATA_PATH, "GEOL_V8_ERA2.shp")
# File containing 30 arc second grid of slope, geology, elevation and
......@@ -849,7 +849,7 @@ def get_slope_geology_vs30_at_location(site_lons, site_lats, spc=(1. / 120.),
print("---- Retrieving data within bounding box")
lons, lats, slope = slice_wgs84_datafile(SITE_HDF5_FILE, bbox, "slope")
geology = slice_wgs84_datafile(SITE_HDF5_FILE, bbox, "geology")[2]
vs30 = slice_wgs84_datafile(VS30_HDF5_FILE, bbox, "vs30")[2]
lons_vs, lats_vs, vs30 = slice_wgs84_datafile(VS30_HDF5_FILE, bbox, "vs30")
if onshore_only:
elevation = slice_wgs84_datafile(SITE_HDF5_FILE, bbox, "elevation")[2]
else:
......@@ -858,6 +858,8 @@ def get_slope_geology_vs30_at_location(site_lons, site_lats, spc=(1. / 120.),
# Integer number of multiples of spc indicates the x and y locations
dx = ((site_lons - (lons[0] - spc / 2.)) / spc).astype(int)
dy = (np.fabs((site_lats - (lats[0] + spc / 2.)) / spc)).astype(int)
dx_vs = ((site_lons - (lons_vs[0] - spc / 2.)) / spc).astype(int)
dy_vs = (np.fabs((site_lats - (lats_vs[0] + spc / 2.)) / spc)).astype(int)
# Get backarc distance
print("---- Getting volcanic distance")
xvf = interpolate_xvf_grid(site_lons, site_lats)
......@@ -867,7 +869,7 @@ def get_slope_geology_vs30_at_location(site_lons, site_lats, spc=(1. / 120.),
print("---- Building dataframe")
elevation = elevation[dy, dx]
idx = elevation >= -5.0
print(site_lons, site_lats)
#print(site_lons, site_lats)
return gpd.GeoDataFrame({
"geometry": gpd.GeoSeries([
Point(lon, lat)
......@@ -875,7 +877,7 @@ def get_slope_geology_vs30_at_location(site_lons, site_lats, spc=(1. / 120.),
"lon": site_lons[idx],
"lat": site_lats[idx],
"slope": slope[dy, dx][idx],
"vs30": vs30[dy, dx][idx],
"vs30": vs30[dy_vs, dx_vs][idx],
"geology": [GEOL_DICT_KEY_TO_NAME[geol]
for geol in geology[dy, dx][idx]],
"xvf": xvf[idx]})
......@@ -1707,7 +1709,8 @@ def set_up_arg_parser():
return parser
if __name__ == "__main__":
#if __name__ == "__main__":
def main():
parser = set_up_arg_parser()
# Argparser - arguments
args = parser.parse_args()
......@@ -1866,3 +1869,7 @@ if __name__ == "__main__":
else:
# Export to csv
site_manager.to_csv(args.output_file)
if __name__ == "__main__":
main()
......@@ -8,18 +8,19 @@ setup(
version="1.0",
description="Tool for building and calibrating OpenQuake site files from exposure models",
license="GPLv3",
extras_require = {"OQ": ['openquake.engine',]},
extras_require = {"OQ": ['openquake.engine>=3.13',]},
install_requires= [
'h5py >=2.10, <2.11',
'numpy >=1.18, <1.20',
'scipy >=1.3, <1.7',
'pandas >=0.25, <1.3',
'shapely >=1.7, <1.8',
'numpy >=1.18',
'scipy >=1.3',
'pandas >=0.25',
'shapely >=1.7, <2.0',
'pyproj >=1.9',
'matplotlib',
'geopandas',
'ipython'],
packages=find_packages(),
python_requires=">=3.6",
python_requires=">=3.8",
entry_points={"console_scripts": ["exposure2site = exposure2site.exposure_to_site_tools:main"]}
)
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment