Skip to content
Snippets Groups Projects
Commit e2d93949 authored by Marcus Herrmann's avatar Marcus Herrmann
Browse files

Update README.md (add info, rewrite, reformat)

parent 5e7d9f93
No related branches found
Tags v0.7.5
No related merge requests found
# Template Matching App (aka xcorr[^1])
>>>
*offical name suggestions* :question::thought_balloon:
*offical name suggestions*:
* temapy, SED-MaFiApy,
* **seis**scanpy, **seis**matchpy, **seis**TMpy, **seis**Xcorrpy,
......@@ -11,6 +11,7 @@
>>>
## Setup
1. Put all the code of this project somewhere into a folder called `code_TM_app` (a suggestion).
......@@ -19,12 +20,10 @@
git clone git@gitlab.seismo.ethz.ch:microEQ/TM.git somewhere/code_TM_app
```
and install the required dependencies
and install the required dependencies:
```
pip3 install --user obspy
pip3 install --user mpi4py
pip3 install --user sortedcontainers
pip3 install --user obspy mpi4py pyFFTW sortedcontainers pillow psycopg2 tqdm pandas joblib
```
......@@ -32,7 +31,7 @@
(a suggestion).
The `_sequences` directory will contain all data and results of different scans
(e.g., different regions, different stations, different setups, etc.).
(e.g., *different regions, different stations, different setups*, etc.).
This is a suggested layout:
......@@ -40,17 +39,18 @@
.
└── _sequences
├── data <- (e.g., .sql script, .csv catalog)
│ ├── SED-events_region1_{station}_mseed <- Created automatically to contain
│ ├── SED-events_region2_{station}_mseed catalog event waveforms and their plots
│ ├── catalog-events_region1_{station}_mseed <- Created automatically to contain
│ ├── catalog-events_region2_{station1}_mseed catalog event waveforms and their plots
│ ├── catalog-events_region2_{station2}_mseed
│ └── ...
├── results_region1_{station}_{startdate}_{enddate} <- The results of a particular run on your sequence
├── results_region2_{station}_{startdate}_{enddate} (with station name and date span)
├── results_region2_{station1}_{startdate}_{enddate} (with station name and date span)
├── results_region2_{station2}_{startdate}_{enddate}
├── ...
└── config_region1.ini
└── config_region2.ini
├── config_region1.ini
├── config_region2_station1.ini
└── config_region2_station2.ini <- ... scan at another station
```
If you want to separate different regions, you can also create a 'sequence'
......@@ -63,20 +63,24 @@
.
├── _sequence_region1
│ ├── data
│ │ └── SED-events_region1_{station}_mseed
│ │ └── catalog-events_region1_{station}_mseed
│ |
│ ├── results_{station}_{startdate}_{enddate} <- with config.set_dir(regionName=False)
│ └── config_region1.ini
│ ├── results_{station}_{startdate}_{enddate}
│ └── config_region1.ini <- set OUTDIR_REGION_NAME = NO (to omit region name in results folder name)
└── _sequence_region2
├── data
│ └── SED-events_region2_{station}_mseed
│ ├── catalog-events_region2_{station1}_mseed
│ └── catalog-events_region2_{station2}_mseed
|
├── results_{station}_{startdate}_{enddate} <- with config.set_dir(regionName=False)
└── config_region2.ini
├── results_{station}_{startdate}_{enddate}
├── config_region2_station1.ini <- set OUTDIR_REGION_NAME = NO
└── config_region2_station2.ini (also here)
```
3. Edit the file config.ini and run the tool.
3. Edit the file config.ini to your needs and run the tool.
---
......@@ -84,64 +88,91 @@
1. Provide a reference catalog in csv-Format
* Option 1: provide the file in appropriate format
* Option 2: query the SED-DB with a sql-file in the `data` directory; an example can be found in `Example/data/seddb_SQL_query.sql`. At the end of the example file there are the parameters you need to change: Station, Lat , Lon , Radius, max. depth. Once the settings have been updated you can run the following command, which will use the file just edited, to query the events from the SED database.
* Option 2: query the SED-DB with a sql-file in the `data` directory; an example can be found in `Example/data/seddb_SQL_query.sql`.
At the end of the example file there are the parameters you need to change:
*Station*, *Lat*, *Lon*, *Radius*, and *max. depth*.
Once the settings have been set, you can execute the following command to query the events from the SED database.
```
python3 xcorr.py config.ini --get-from-DB
```
2. Preload your catalog events and place them in the data folder, so that you can inspect them:
```
python3 xcorr.py config.ini --preload-events
```
Now go through the waveform plots in the `data/catalog-\*` directory and review your new waveforms.
Comment out events you don't want to use as templates (e.g., low quality events) in the `seddb-\*.csv` file (with a `#` at the start of a line).
Now go throug the waveform plots in the data/catalog-\* directory and comment-out all low quality events that you not want to use as templates in the seddb-\*.csv-file (# at start of line)
Important:
After you have commented-out events, delete the "data/catalog-\*"-directories and rerun the preload-events command.
Important:
**!! Important:** If you commented out events in the `seddb-\*.csv` file, delete the `data/catalog-\*` directories and rerun the preload-events command.
If you update the seddb-\*.csv file, make sure to keep the seddb-\*.csv.old-part of your catalog unchanged. Only add new events from the seddb-\*.csv file that are junger than
the oldest event in the seddb-\*.csv.old-file to the end of the seddb-\*.csv.old-file. Rename this combined file to seddb-\*.csv and run the preload-events command. Review your new waveforms, commet-out events you not want to use as templates in the seddb-\*csv file, delete the data/catalog-\* directories and rerun the preload-events command.
**!! Important (for updating the scan later on):** If you update the `seddb-\*.csv` file (with the command in step 1),
make sure to keep the `seddb-\*.csv.old`-part of your catalog unchanged.
Only append new events from updated new `seddb-\*.csv` file that are junger than the
oldest event in the `seddb-\*.csv.old` file to the end of the `seddb-\*.csv.old` file.
Rename this `seddb-\*.csv.old` file to `seddb-\*.csv`.
Now you are ready to execute the preload-events command.
!!If you not follow this steps your magnitude regression and event assoziation might go wrong!!
**If you do not follow these steps, your magnitude regression and event association might go wrong!!**
3. Run the cross correlation
3. Run the cross correlation:
```
python3 xcorr.py config.ini
```
or faster:
or in parallel (recommended):
```
mpiexec -n CORES python3 xcorr.py config.ini --mpi
```
(*CORES*: the number of processor cores *+ 1*; the additional '*+1*' will be reserved for the master process, which usually does not use much CPU)
4. Run the post processing (plots)
4. Run the post processing:
```
python3 xcorr.py config.ini --post-processing
```
**Note**
This performs:
1) merging the detections of individual templates ('unique'),
2) determining the magnitudes ('regress-mags'), and
3) plotting the magnitude vs. time plots of the detections ('plot-timeline').
These and more post-processing steps can be executed individually with:
```
python3 xcorr.py config.ini --post-processing {basic, unique, regress-mags, plot-timeline, plot-detections, plot-helicorder}
```
---
### Notes
The paths for executing the tool need to be specified according to your current working directory in the terminal:
you might be either in
* the sequency directory
* from the sequency directory:
* then you have to give the path to `xcorr.py` (can be relative):
```
python3 ../../xcorr.py config.ini --post-processing
```
* anywhere
* then you have to prepend the path of _both_ `xcorr.py` and `config.ini`:
* from anywhere else:
* then you have to prepend the path to _both_ `xcorr.py` and `config.ini`:
```
python3 ../xcorr.py region1/config.ini --post-processing
```
To see all possible command line arguments and a brief help, execute:
```
python3 xcorr.py config.ini --help
```
---
[^1]: xcorr: cross-correlation
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment