Skip to content

Commit 1b0eadb

Browse files
pjhartzellgadomskiTom Augspurger
authored
Update Sentinel-5P (#167)
--------- Co-authored-by: Pete Gadomski <[email protected]> Co-authored-by: Tom Augspurger <[email protected]>
1 parent bbc8cf9 commit 1b0eadb

File tree

12 files changed

+666
-216
lines changed

12 files changed

+666
-216
lines changed

‎datasets/io-land-cover/dataset.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ id: io_lulc
22
image: ${{ args.registry }}/pctasks-task-base:latest
33

44
args:
5-
- registry
5+
- registry
66

77
code:
88
src: ${{ local.path(./io_lulc.py) }}

‎datasets/ms-buildings/Dockerfile‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
FROM ubuntu:20.04
2+
3+
# Setup timezone info
4+
ENV TZ=UTC
5+
6+
ENV LC_ALL=C.UTF-8
7+
ENV LANG=C.UTF-8
8+
9+
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
10+
11+
RUN apt-get update && apt-get install -y software-properties-common
12+
13+
RUN add-apt-repository ppa:ubuntugis/ppa && \
14+
apt-get update && \
15+
apt-get install -y build-essential python3-dev python3-pip \
16+
jq unzip ca-certificates wget curl git && \
17+
apt-get autoremove && apt-get autoclean && apt-get clean
18+
19+
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10
20+
21+
# See https://github.com/mapbox/rasterio/issues/1289
22+
ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
23+
24+
# Install Python 3.11
25+
RUN curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh" \
26+
&& bash "Mambaforge-$(uname)-$(uname -m).sh" -b -p /opt/conda \
27+
&& rm -rf "Mambaforge-$(uname)-$(uname -m).sh"
28+
29+
ENV PATH /opt/conda/bin:$PATH
30+
ENV LD_LIBRARY_PATH /opt/conda/lib/:$LD_LIBRARY_PATH
31+
32+
RUN mamba install -y -c conda-forge python=3.11 gdal pip setuptools cython numpy
33+
34+
RUN python -m pip install --upgrade pip
35+
36+
# Install common packages
37+
COPY requirements-task-base.txt /tmp/requirements.txt
38+
RUN python -m pip install --no-build-isolation -r /tmp/requirements.txt
39+
40+
#
41+
# Copy and install packages
42+
#
43+
44+
COPY pctasks/core /opt/src/pctasks/core
45+
RUN cd /opt/src/pctasks/core && \
46+
pip install .
47+
48+
COPY pctasks/cli /opt/src/pctasks/cli
49+
RUN cd /opt/src/pctasks/cli && \
50+
pip install .
51+
52+
COPY pctasks/task /opt/src/pctasks/task
53+
RUN cd /opt/src/pctasks/task && \
54+
pip install .
55+
56+
COPY pctasks/client /opt/src/pctasks/client
57+
RUN cd /opt/src/pctasks/client && \
58+
pip install .
59+
60+
COPY pctasks/ingest /opt/src/pctasks/ingest
61+
RUN cd /opt/src/pctasks/ingest && \
62+
pip install .
63+
64+
COPY pctasks/dataset /opt/src/pctasks/dataset
65+
RUN cd /opt/src/pctasks/dataset && \
66+
pip install .
67+
68+
COPY ./datasets/ms-buildings/requirements.txt /opt/src/datasets/ms-buildings/requirements.txt
69+
RUN python3 -m pip install -r /opt/src/datasets/ms-buildings/requirements.txt
70+
71+
# Setup Python Path to allow import of test modules
72+
ENV PYTHONPATH=/opt/src:$PYTHONPATH
73+
74+
WORKDIR /opt/src

‎datasets/sentinel-5p/Dockerfile‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
FROM ubuntu:20.04
2+
3+
# Setup timezone info
4+
ENV TZ=UTC
5+
6+
ENV LC_ALL=C.UTF-8
7+
ENV LANG=C.UTF-8
8+
9+
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
10+
11+
RUN apt-get update && apt-get install -y software-properties-common
12+
13+
RUN add-apt-repository ppa:ubuntugis/ppa && \
14+
apt-get update && \
15+
apt-get install -y build-essential python3-dev python3-pip \
16+
jq unzip ca-certificates wget curl git && \
17+
apt-get autoremove && apt-get autoclean && apt-get clean
18+
19+
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10
20+
21+
# See https://github.com/mapbox/rasterio/issues/1289
22+
ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
23+
24+
# Install Python 3.8
25+
RUN curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh" \
26+
&& bash "Mambaforge-$(uname)-$(uname -m).sh" -b -p /opt/conda \
27+
&& rm -rf "Mambaforge-$(uname)-$(uname -m).sh"
28+
29+
ENV PATH /opt/conda/bin:$PATH
30+
ENV LD_LIBRARY_PATH /opt/conda/lib/:$LD_LIBRARY_PATH
31+
32+
RUN mamba install -y -c conda-forge python=3.8 gdal=3.3.3 pip setuptools cython numpy==1.21.5
33+
34+
RUN python -m pip install --upgrade pip
35+
36+
# Install common packages
37+
COPY requirements-task-base.txt /tmp/requirements.txt
38+
RUN python -m pip install --no-build-isolation -r /tmp/requirements.txt
39+
40+
#
41+
# Copy and install packages
42+
#
43+
44+
COPY pctasks/core /opt/src/pctasks/core
45+
RUN cd /opt/src/pctasks/core && \
46+
pip install .
47+
48+
COPY pctasks/cli /opt/src/pctasks/cli
49+
RUN cd /opt/src/pctasks/cli && \
50+
pip install .
51+
52+
COPY pctasks/task /opt/src/pctasks/task
53+
RUN cd /opt/src/pctasks/task && \
54+
pip install .
55+
56+
COPY pctasks/client /opt/src/pctasks/client
57+
RUN cd /opt/src/pctasks/client && \
58+
pip install .
59+
60+
COPY pctasks/ingest /opt/src/pctasks/ingest
61+
RUN cd /opt/src/pctasks/ingest && \
62+
pip install .
63+
64+
COPY pctasks/dataset /opt/src/pctasks/dataset
65+
RUN cd /opt/src/pctasks/dataset && \
66+
pip install .
67+
68+
COPY ./datasets/sentinel-5p/requirements.txt /opt/src/datasets/sentinel-5p/requirements.txt
69+
RUN python3 -m pip install -r /opt/src/datasets/sentinel-5p/requirements.txt
70+
71+
# Setup Python Path to allow import of test modules
72+
ENV PYTHONPATH=/opt/src:$PYTHONPATH
73+
74+
WORKDIR /opt/src

‎datasets/sentinel-5p/README.md‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,11 @@
1-
# Work in progress - in a broken state
1+
# planetary-computer-tasks dataset: sentinel-5p
2+
3+
Sentinel 5 Precursor
4+
5+
## Building the Docker image
6+
7+
To build and push a custom docker image to our container registry:
8+
9+
```shell
10+
az acr build -r {the registry} --subscription {the subscription} -t pctasks-sentinel-5p:latest -f datasets/sentinel-5p/Dockerfile .
11+
```

‎datasets/sentinel-5p/collection/description.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
The Copernicus [Sentinel-5 Precursor](https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-5p) mission provides high spatio-temporal resolution measurements of the Earth's atmosphere. The mission consists of one satellite carrying the [TROPOspheric Monitoring Instrument](http://www.tropomi.eu/) (TROPOMI). The satellite flies in loose formation with NASA's [Suomi NPP](https://www.nasa.gov/mission_pages/NPP/main/index.html) spacecraft, allowing utilization of co-located cloud mask data provided by the [Visible Infrared Imaging Radiometer Suite](https://www.nesdis.noaa.gov/current-satellite-missions/currently-flying/joint-polar-satellite-system/visible-infrared-imaging) (VIIRS) instrument onboard Suomi NPP during processing of the TROPOMI methane product.
2+
3+
The Sentinel-5 Precursor mission aims to reduce the global atmospheric data gap between the retired [ENVISAT](https://earth.esa.int/eogateway/missions/envisat) and [AURA](https://www.nasa.gov/mission_pages/aura/main/index.html) missions and the future [Sentinel-5](https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-5) mission. Sentinel-5 Precursor [Level 2 data](http://www.tropomi.eu/data-products/level-2-products) provide total columns of ozone, sulfur dioxide, nitrogen dioxide, carbon monoxide and formaldehyde, tropospheric columns of ozone, vertical profiles of ozone and cloud & aerosol information. These measurements are used for improving air quality forecasts and monitoring the concentrations of atmospheric constituents.
4+
5+
This STAC Collection provides Sentinel-5 Precursor Level 2 data, in NetCDF format, since April 2018 for the following products:
6+
7+
* [`L2__AER_AI`](http://www.tropomi.eu/data-products/uv-aerosol-index): Ultraviolet aerosol index
8+
* [`L2__AER_LH`](http://www.tropomi.eu/data-products/aerosol-layer-height): Aerosol layer height
9+
* [`L2__CH4___`](http://www.tropomi.eu/data-products/methane): Methane (CH<sub>4</sub>) total column
10+
* [`L2__CLOUD_`](http://www.tropomi.eu/data-products/cloud): Cloud fraction, albedo, and top pressure
11+
* [`L2__CO____`](http://www.tropomi.eu/data-products/carbon-monoxide): Carbon monoxide (CO) total column
12+
* [`L2__HCHO__`](http://www.tropomi.eu/data-products/formaldehyde): Formaldehyde (HCHO) total column
13+
* [`L2__NO2___`](http://www.tropomi.eu/data-products/nitrogen-dioxide): Nitrogen dioxide (NO<sub>2</sub>) total column
14+
* [`L2__O3____`](http://www.tropomi.eu/data-products/total-ozone-column): Ozone (O<sub>3</sub>) total column
15+
* [`L2__O3_TCL`](http://www.tropomi.eu/data-products/tropospheric-ozone-column): Ozone (O<sub>3</sub>) tropospheric column
16+
* [`L2__SO2___`](http://www.tropomi.eu/data-products/sulphur-dioxide): Sulfur dioxide (SO<sub>2</sub>) total column
17+
* [`L2__NP_BD3`](http://www.tropomi.eu/data-products/auxiliary): Cloud from the Suomi NPP mission, band 3
18+
* [`L2__NP_BD6`](http://www.tropomi.eu/data-products/auxiliary): Cloud from the Suomi NPP mission, band 6
19+
* [`L2__NP_BD7`](http://www.tropomi.eu/data-products/auxiliary): Cloud from the Suomi NPP mission, band 7

0 commit comments

Comments
 (0)