From e2d93949dadbaabbd43fb542aa824443714392a8 Mon Sep 17 00:00:00 2001
From: Marcus Herrmann <marcus.herrmann@sed.ethz.ch>
Date: Wed, 8 May 2019 21:49:46 +0200
Subject: [PATCH] Update README.md (add info, rewrite, reformat)
---
README.md | 113 ++++++++++++++++++++++++++++++++++--------------------
1 file changed, 72 insertions(+), 41 deletions(-)
diff --git a/README.md b/README.md
index b2d595d..feff178 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
# Template Matching App (aka xcorr[^1])
>>>
-*offical name suggestions* :question::thought_balloon:
+*offical name suggestions*:
* temapy, SED-MaFiApy,
* **seis**scanpy, **seis**matchpy, **seis**TMpy, **seis**Xcorrpy,
@@ -11,6 +11,7 @@
>>>
+
## Setup
1. Put all the code of this project somewhere into a folder called `code_TM_app` (a suggestion).
@@ -19,12 +20,10 @@
git clone git@gitlab.seismo.ethz.ch:microEQ/TM.git somewhere/code_TM_app
```
- and install the required dependencies
+ and install the required dependencies:
```
- pip3 install --user obspy
- pip3 install --user mpi4py
- pip3 install --user sortedcontainers
+ pip3 install --user obspy mpi4py pyFFTW sortedcontainers pillow psycopg2 tqdm pandas joblib
```
@@ -32,7 +31,7 @@
(a suggestion).
The `_sequences` directory will contain all data and results of different scans
- (e.g., different regions, different stations, different setups, etc.).
+ (e.g., *different regions, different stations, different setups*, etc.).
This is a suggested layout:
@@ -40,17 +39,18 @@
.
└── _sequences
├── data <- (e.g., .sql script, .csv catalog)
- │ ├── SED-events_region1_{station}_mseed <- Created automatically to contain
- │ ├── SED-events_region2_{station}_mseed catalog event waveforms and their plots
+ │ ├── catalog-events_region1_{station}_mseed <- Created automatically to contain
+ │ ├── catalog-events_region2_{station1}_mseed catalog event waveforms and their plots
+ │ ├── catalog-events_region2_{station2}_mseed
│ └── ...
│
├── results_region1_{station}_{startdate}_{enddate} <- The results of a particular run on your sequence
- ├── results_region2_{station}_{startdate}_{enddate} (with station name and date span)
+ ├── results_region2_{station1}_{startdate}_{enddate} (with station name and date span)
+ ├── results_region2_{station2}_{startdate}_{enddate}
├── ...
- └── config_region1.ini
- └── config_region2.ini
-
-
+ ├── config_region1.ini
+ ├── config_region2_station1.ini
+ └── config_region2_station2.ini <- ... scan at another station
```
If you want to separate different regions, you can also create a 'sequence'
@@ -63,20 +63,24 @@
.
├── _sequence_region1
│ ├── data
- │ │ └── SED-events_region1_{station}_mseed
+ │ │ └── catalog-events_region1_{station}_mseed
│ |
- │ ├── results_{station}_{startdate}_{enddate} <- with config.set_dir(regionName=False)
- │ └── config_region1.ini
+ │ ├── results_{station}_{startdate}_{enddate}
+ │ └── config_region1.ini <- set OUTDIR_REGION_NAME = NO (to omit region name in results folder name)
│
└── _sequence_region2
├── data
- │ └── SED-events_region2_{station}_mseed
+ │ ├── catalog-events_region2_{station1}_mseed
+ │ └── catalog-events_region2_{station2}_mseed
|
- ├── results_{station}_{startdate}_{enddate} <- with config.set_dir(regionName=False)
- └── config_region2.ini
+ ├── results_{station}_{startdate}_{enddate}
+ ├── config_region2_station1.ini <- set OUTDIR_REGION_NAME = NO
+ └── config_region2_station2.ini (also here)
```
-3. Edit the file config.ini and run the tool.
+3. Edit the file config.ini to your needs and run the tool.
+
+
---
@@ -84,64 +88,91 @@
1. Provide a reference catalog in csv-Format
* Option 1: provide the file in appropriate format
- * Option 2: query the SED-DB with a sql-file in the `data` directory; an example can be found in `Example/data/seddb_SQL_query.sql`. At the end of the example file there are the parameters you need to change: Station, Lat , Lon , Radius, max. depth. Once the settings have been updated you can run the following command, which will use the file just edited, to query the events from the SED database.
+ * Option 2: query the SED-DB with a sql-file in the `data` directory; an example can be found in `Example/data/seddb_SQL_query.sql`.
+ At the end of the example file there are the parameters you need to change:
+ *Station*, *Lat*, *Lon*, *Radius*, and *max. depth*.
+ Once the settings have been set, you can execute the following command to query the events from the SED database.
```
python3 xcorr.py config.ini --get-from-DB
```
-
+
2. Preload your catalog events and place them in the data folder, so that you can inspect them:
```
python3 xcorr.py config.ini --preload-events
```
+
+ Now go through the waveform plots in the `data/catalog-\*` directory and review your new waveforms.
+ Comment out events you don't want to use as templates (e.g., low quality events) in the `seddb-\*.csv` file (with a `#` at the start of a line).
- Now go throug the waveform plots in the data/catalog-\* directory and comment-out all low quality events that you not want to use as templates in the seddb-\*.csv-file (# at start of line)
-
- Important:
-
- After you have commented-out events, delete the "data/catalog-\*"-directories and rerun the preload-events command.
-
- Important:
+ **!! Important:** If you commented out events in the `seddb-\*.csv` file, delete the `data/catalog-\*` directories and rerun the preload-events command.
- If you update the seddb-\*.csv file, make sure to keep the seddb-\*.csv.old-part of your catalog unchanged. Only add new events from the seddb-\*.csv file that are junger than
- the oldest event in the seddb-\*.csv.old-file to the end of the seddb-\*.csv.old-file. Rename this combined file to seddb-\*.csv and run the preload-events command. Review your new waveforms, commet-out events you not want to use as templates in the seddb-\*csv file, delete the data/catalog-\* directories and rerun the preload-events command.
+ **!! Important (for updating the scan later on):** If you update the `seddb-\*.csv` file (with the command in step 1),
+ make sure to keep the `seddb-\*.csv.old`-part of your catalog unchanged.
+ Only append new events from updated new `seddb-\*.csv` file that are junger than the
+ oldest event in the `seddb-\*.csv.old` file to the end of the `seddb-\*.csv.old` file.
+ Rename this `seddb-\*.csv.old` file to `seddb-\*.csv`.
+ Now you are ready to execute the preload-events command.
- !!If you not follow this steps your magnitude regression and event assoziation might go wrong!!
+ **If you do not follow these steps, your magnitude regression and event association might go wrong!!**
-3. Run the cross correlation
+3. Run the cross correlation:
-
```
python3 xcorr.py config.ini
```
- or faster:
+ or in parallel (recommended):
```
mpiexec -n CORES python3 xcorr.py config.ini --mpi
```
+
+ (*CORES*: the number of processor cores *+ 1*; the additional '*+1*' will be reserved for the master process, which usually does not use much CPU)
-4. Run the post processing (plots)
+4. Run the post processing:
```
python3 xcorr.py config.ini --post-processing
```
-**Note**
+ This performs:
+ 1) merging the detections of individual templates ('unique'),
+ 2) determining the magnitudes ('regress-mags'), and
+ 3) plotting the magnitude vs. time plots of the detections ('plot-timeline').
+
+ These and more post-processing steps can be executed individually with:
+ ```
+ python3 xcorr.py config.ini --post-processing {basic, unique, regress-mags, plot-timeline, plot-detections, plot-helicorder}
+ ```
+
+
+---
+
+### Notes
+
+The paths for executing the tool need to be specified according to your current working directory in the terminal:
-you might be either in
- * the sequency directory
+ * from the sequency directory:
* then you have to give the path to `xcorr.py` (can be relative):
```
python3 ../../xcorr.py config.ini --post-processing
```
- * anywhere
- * then you have to prepend the path of _both_ `xcorr.py` and `config.ini`:
+ * from anywhere else:
+ * then you have to prepend the path to _both_ `xcorr.py` and `config.ini`:
```
python3 ../xcorr.py region1/config.ini --post-processing
```
+
+To see all possible command line arguments and a brief help, execute:
+
+ ```
+ python3 xcorr.py config.ini --help
+ ```
+
+
---
[^1]: xcorr: cross-correlation
--
GitLab