Compare commits
No commits in common. "master" and "code_trimming" have entirely different histories.
master
...
code_trimm
@ -1,3 +1,2 @@
|
|||||||
include nuwavdet/pixpos/*
|
include nuwavsource/pixpos/*
|
||||||
include nuwavdet/badpix_headers/*
|
include nuwavsource/badpix_headers/*
|
||||||
|
|
101
README.md
101
README.md
@ -1,67 +1,10 @@
|
|||||||
# nuwavdet
|
# nuwavsource
|
||||||
|
|
||||||
This pacakge is used to generate region masks separating any focused X-ray flux from background signal in NuSTAR observations.
|
This package is supposed to be used to detect the sources in NuStar observations and generate a mask excluding the signal from the sources of any kind.
|
||||||
|
|
||||||
## Installation
|
Additionaly, it generates a table containing:
|
||||||
This package is to be used with Python 3.x.x
|
|
||||||
```bash
|
|
||||||
pip install git+http://heagit.cosmos.ru/nustar/nuwavdet.git
|
|
||||||
```
|
|
||||||
|
|
||||||
To update the package to the current version one should delete the previous version
|
Useful data about the observation:
|
||||||
```bash
|
|
||||||
pip uninstall nuwavdet
|
|
||||||
```
|
|
||||||
|
|
||||||
And simply repeat the intallation procedure again from the repository.
|
|
||||||
|
|
||||||
## Installation verification
|
|
||||||
If the installation was successful the package can be used with the following import:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from nuwavdet import nuwavdet as nw
|
|
||||||
```
|
|
||||||
|
|
||||||
To verify the installation we suggest running a simple script:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from nuwavdet import nuwavdet as nw
|
|
||||||
|
|
||||||
print(nw.binary_array(2))
|
|
||||||
```
|
|
||||||
|
|
||||||
The output of the script should be
|
|
||||||
|
|
||||||
```bash
|
|
||||||
[[False False]
|
|
||||||
[False True]
|
|
||||||
[ True False]
|
|
||||||
[ True True]]
|
|
||||||
```
|
|
||||||
|
|
||||||
## Main use
|
|
||||||
|
|
||||||
The main functionality of the package is presented with a single function
|
|
||||||
```python
|
|
||||||
nw.process(obs_path, thresh)
|
|
||||||
```
|
|
||||||
|
|
||||||
Inputs are string with path to the _cl.evt file to use and a tuple of thresholds, e.g.
|
|
||||||
```python
|
|
||||||
nw.process('D:\\Data\\obs_cl.evt', (3, 2))
|
|
||||||
```
|
|
||||||
|
|
||||||
The detailed script description of the data extraction with the script is presented in the examples folder of the repository.
|
|
||||||
|
|
||||||
The function nw.process returns severl python objects:
|
|
||||||
1. python-dictionary with some metadata and properties of the observation after mask generation procedure.
|
|
||||||
2. region array with mask in DET1 coordinate frame. Note that this mask is for numpy mask application so True (1) corresponds to masked pixel and False (0) otherwise.
|
|
||||||
3. custom bad pixel table with flagged pixels in RAW coordinates. It can be exported as fits file for further application to the nupipeline as fpma_userbpfile or fpmb_userbpfile.
|
|
||||||
4. array with the sum of wavelet planes for potential alternative applications.
|
|
||||||
|
|
||||||
Metadata about the observation returned by the nw.process is:
|
|
||||||
|
|
||||||
Observation metadata:
|
|
||||||
|
|
||||||
1. OBS_ID
|
1. OBS_ID
|
||||||
2. Detector
|
2. Detector
|
||||||
@ -71,14 +14,36 @@ Observation metadata:
|
|||||||
|
|
||||||
Useful algorythm-related data:
|
Useful algorythm-related data:
|
||||||
|
|
||||||
6. Average count rate of unmasked area
|
6. Average count rate on unmasked area
|
||||||
7. Fraction of unmasked area
|
7. Portion of unmasked area
|
||||||
8. Modified Cash-statistic per bin before and after masking the detected sources
|
8. Specific statistical metric[1] before and after masking the detected sources
|
||||||
|
9. Root-mean-square of counts in unmasked area
|
||||||
|
|
||||||
## Other uses
|
## Installation
|
||||||
|
This package is to be used with Python 3.x.x
|
||||||
|
|
||||||
Other possbile usecases are shown in the examples folder.
|
To install tha package write
|
||||||
|
|
||||||
## Contact information
|
```bash
|
||||||
|
pip install nuwavsource
|
||||||
|
```
|
||||||
|
|
||||||
If you have any questions or issues with the code, feel free to contact Andrey Mukhin: amukhin@cosmos.ru
|
## Usage
|
||||||
|
|
||||||
|
To use the package in your project, import it in by writing
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nuwavsource import nuwavsource
|
||||||
|
```
|
||||||
|
|
||||||
|
You can process the cl.evt file by creating an Observation class object:
|
||||||
|
|
||||||
|
```python
|
||||||
|
obs = nuwavsource.Observation(path_to_evt_file)
|
||||||
|
```
|
||||||
|
|
||||||
|
Additionally, the energy band in KeV to get events from can be passed as an argument. The default value is [3,20].
|
||||||
|
|
||||||
|
```python
|
||||||
|
obs = nuwavsource.Observation(path_to_evt_file,E_borders=[E_min,E_max])
|
||||||
|
```
|
||||||
|
@ -1,54 +0,0 @@
|
|||||||
from nuwavdet import nuwavdet as nw
|
|
||||||
|
|
||||||
|
|
||||||
OBS_PATH = r'.//path_to_obs//nu<obsid><DET>01_cl.evt'
|
|
||||||
THRESH = (3, 2)
|
|
||||||
|
|
||||||
SAVE_BADPIX_PATH = r'.//out//badpix.fits'
|
|
||||||
SAVE_REGION_PATH = r'.//out//region.fits'
|
|
||||||
SAVE_WAVSUM_PATH = r'.//out//wavsum.fits'
|
|
||||||
|
|
||||||
METADATA_PATH = r'.//out//metadata.csv'
|
|
||||||
METADATA_FITS_PATH = r'.//out//metadata.fits'
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
# PROCESS THE OBSERVATION WITH GIVEN THRESHOLD
|
|
||||||
result, region, region_raw, wav_sum = nw.process(OBS_PATH, thresh=THRESH)
|
|
||||||
|
|
||||||
# SAVE THE REGION BAD PIXEL FILES TO THE FITS FILE WITH NUPIPELINE
|
|
||||||
# COMPATIBLE FORMAT AND HEADERS.
|
|
||||||
region_raw.writeto(SAVE_BADPIX_PATH)
|
|
||||||
|
|
||||||
# SAVE REGION MASK AS A FITS IMAGE
|
|
||||||
nw.save_region(region, SAVE_REGION_PATH, overwrite=False)
|
|
||||||
# Note that the Python script uses numpy masked array with
|
|
||||||
# True (1) as as masked and False (0) as unmasked pixel.
|
|
||||||
# nw.save_region transfers the numpy masked array to
|
|
||||||
# conventional format with 1 for unmasked and 0 for masked pixel.
|
|
||||||
# However, if mask is used in the Python you need to transfer it back with
|
|
||||||
# numpy.logical_not(mask).
|
|
||||||
|
|
||||||
# SAVE WAVSUM ARRAY AS A FITS IMAGE
|
|
||||||
nw.fits.writeto(SAVE_WAVSUM_PATH, wav_sum, overwrite=False)
|
|
||||||
|
|
||||||
# SAVE METADATA
|
|
||||||
# WE SUGGEST SAVING ALL THE METADATA FOR SEVERAL OBSERVATIONS
|
|
||||||
# IN ONE FILE.
|
|
||||||
|
|
||||||
# CREATE CSV FILE TO SAVE DATA
|
|
||||||
# IF FILE ALREADY EXISTS YOU SHOULD REMOVE THIS BLOCK FROM YOUR CODE
|
|
||||||
table = {
|
|
||||||
'obs_id': [], 'detector': [], 'ra': [], 'dec': [],
|
|
||||||
'lon': [], 'lat': [], 't_start': [], 'exposure': [],
|
|
||||||
'count_rate': [], 'remaining_area': [], 'cash_stat': [],
|
|
||||||
'cash_stat_full': []
|
|
||||||
}
|
|
||||||
out_table = nw.DataFrame(table)
|
|
||||||
out_table.to_csv(METADATA_PATH)
|
|
||||||
|
|
||||||
# SAVE DATA TO CREATED CSV
|
|
||||||
nw.DataFrame(result, index=[0]).to_csv(METADATA_PATH, mode='a', header=False)
|
|
||||||
|
|
||||||
# TRANSFORM THE CSV TO FITS-TABLE
|
|
||||||
nw.csv_to_table(METADATA_PATH, METADATA_FITS_PATH)
|
|
@ -1,33 +0,0 @@
|
|||||||
from nuwavdet import nuwavdet as nw
|
|
||||||
|
|
||||||
INPUT_FOLDER = r'path_to_directory'
|
|
||||||
OUTPUT_FOLDER = r'.//Output'
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
# BEGIN PROCESSING ALL THE OBSERVATIONS INSIDE THE FOLDER
|
|
||||||
nw.process_folder(input_folder=INPUT_FOLDER,
|
|
||||||
start_new_file='y',
|
|
||||||
fits_folder=OUTPUT_FOLDER,
|
|
||||||
thresh=(3, 2),
|
|
||||||
cpu_num=10
|
|
||||||
)
|
|
||||||
|
|
||||||
# IF THE PROCESSING WAS INTERRUPTED YOU CAN CONTINUE IT WITH THE SAME CODE
|
|
||||||
# BY CHANGING THE start_new_file TO 'n'.
|
|
||||||
|
|
||||||
# THE STRUCTURE OF THE FOLDER IS
|
|
||||||
# OUTPUT_FOLDER
|
|
||||||
# __overview.csv csv-table with observations metadata
|
|
||||||
# __overvies.fits fits-table with the same metadata
|
|
||||||
# __overview_skipped.csv csv-table with the skipped observations
|
|
||||||
# __Region folder for region mask images
|
|
||||||
# ____<obsid><DET>_region.fits
|
|
||||||
# __Region_raw folder for region masks in RAW coordinates
|
|
||||||
# ____<obsid><DET>_reg_raw.fits
|
|
||||||
# __Wav_sum folder for sum of wavelet layers
|
|
||||||
# ____<obsid><DET>_wav_sum.fits
|
|
||||||
|
|
||||||
# Note nw.process_folder uses multiprocessing with cpu_num cores.
|
|
||||||
# The number of cores can be manually chosen or automatically
|
|
||||||
# detected if cpu_num = 0.
|
|
@ -1,23 +0,0 @@
|
|||||||
from nuwavdet import nuwavdet as nw
|
|
||||||
|
|
||||||
OBS_PATH = r'.//path_to_obs//nu<obsid><DET>01_cl.evt'
|
|
||||||
THRESH = (3, 2)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
# CREATE THE OBSERVATION CLASS OBJECT
|
|
||||||
obs = nw.Observation(OBS_PATH)
|
|
||||||
|
|
||||||
# CALCULATE THE WAVLET LAYERS WITH GIVEN THRESHOLD
|
|
||||||
wav_layers = obs.wavdecomp(mode='atrous', occ_coeff=True, thresh=THRESH)
|
|
||||||
|
|
||||||
# ALL THE LAYERS CAN BE ACCESSED AS AN ELEMENT OF wav_layers VARIABLE
|
|
||||||
# wav_layers[0] for the 1st wavelet layer
|
|
||||||
# wav_layers[4] for 5th wavelet layer
|
|
||||||
# wav_layers[-1] for the last wavelet layer
|
|
||||||
# wav_layers[2:5] for the list of the layers from 3 to 5
|
|
||||||
# wav_layers[[1, 3, 5]] for the list of layers 2, 4 and 6
|
|
||||||
|
|
||||||
# To calculate the sum of wavelet layers one should use sum() method
|
|
||||||
# wav_layers[2:7].sum(0) returns a sum of layers from 3 to 7
|
|
||||||
# wav_layers[[1, 3, 5]].sum(0) returns a sum of layers 2, 4 and 6.
|
|
@ -1,23 +0,0 @@
|
|||||||
from nuwavdet import nuwavdet as nw
|
|
||||||
import numpy as np
|
|
||||||
|
|
||||||
|
|
||||||
OBS_PATH = r'.//path_to_obs//nu<obsid><DET>01_cl.evt'
|
|
||||||
MASK_PATH = r'.//path_to_mask//<obsid><DET>.fits'
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
# CREATE THE OBSERVATION CLASS OBJECT
|
|
||||||
obs = nw.Observation(OBS_PATH)
|
|
||||||
|
|
||||||
# READ THE REGION MASK FILE
|
|
||||||
region = nw.fits.getdata(MASK_PATH)
|
|
||||||
|
|
||||||
# TRANSFORM REGION MASK DATA TO NUMPY MASK DATA (SEE 1_save_results.py).
|
|
||||||
region = np.logical_not(region.astype(bool))
|
|
||||||
|
|
||||||
# CREATE MASKED ARRAY CLASS OBJECT
|
|
||||||
masked_data = np.ma.masked_array(obs, mask=region)
|
|
||||||
|
|
||||||
# CALCULATE THE CSTAT ON THE MASKED DATA
|
|
||||||
print(nw.сstat(masked_data.mean(), masked_data))
|
|
@ -1 +0,0 @@
|
|||||||
name = 'nuwavdet'
|
|
1
nuwavsource/__init__.py
Normal file
1
nuwavsource/__init__.py
Normal file
@ -0,0 +1 @@
|
|||||||
|
name = 'nuwavsource'
|
BIN
nuwavsource/__pycache__/__init__.cpython-39.pyc
Normal file
BIN
nuwavsource/__pycache__/__init__.cpython-39.pyc
Normal file
Binary file not shown.
BIN
nuwavsource/__pycache__/nuwavsource.cpython-39.pyc
Normal file
BIN
nuwavsource/__pycache__/nuwavsource.cpython-39.pyc
Normal file
Binary file not shown.
File diff suppressed because it is too large
Load Diff
10
setup.py
10
setup.py
@ -4,14 +4,14 @@ with open("README.md", "r") as fh:
|
|||||||
long_description = fh.read()
|
long_description = fh.read()
|
||||||
|
|
||||||
setuptools.setup(
|
setuptools.setup(
|
||||||
name="nuwavdet",
|
name="nuwavsource",
|
||||||
version="0.1.1",
|
version="0.0.8",
|
||||||
author="Andrey Mukhin",
|
author="Andrey Mukhin",
|
||||||
author_email="amukhin@cosmos.ru",
|
author_email="amukhin@phystech.edu",
|
||||||
description="A package for source exclusion in NuSTAR observation data using wavelet decomposition",
|
description="A package for source exclusion in NuStar observation data using wavelet decomposition",
|
||||||
long_description=long_description,
|
long_description=long_description,
|
||||||
long_description_content_type="text/markdown",
|
long_description_content_type="text/markdown",
|
||||||
url="https://github.com/andrey-rrousan/nuwavdet",
|
url="https://github.com/Andreyousan/nuwavsource",
|
||||||
packages=setuptools.find_packages(),
|
packages=setuptools.find_packages(),
|
||||||
include_package_data=True,
|
include_package_data=True,
|
||||||
classifiers=(
|
classifiers=(
|
||||||
|
Loading…
x
Reference in New Issue
Block a user