D4Science
/
d4science_copernicus_notebooks
15
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: D4Science:fb573ed673e59cc10599a87029128bec262e8abb
Branches Tags
D4Science:main
D4Science:shared_download
...
compare: D4Science:7cf23cd1cc9c6f714394ce7b644be07d57cc331d
Branches Tags
D4Science:shared_download
D4Science:main

2 Commits
fb573ed673 ... 7cf23cd1cc

Author SHA1 Message Date
Alfredo Oliviero 7cf23cd1cc requirements_tutorials.txt 2024-12-02 14:50:45 +01:00
Alfredo Oliviero c9d037d07e configuration notebook 2024-12-02 14:49:01 +01:00
2 changed files with 221 additions and 71 deletions
Show Stats Expand all files Collapse all files

291
config_auth_cds.ipynb
View File

@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### d4science_copernicus_cds Library - Setup and Example\n",
"### d4science_copernicus_cds Library Setup and Example\n",
"\n",
"This Jupyter notebook will guide you through setting up all dependencies and configuring the environment to use the `d4science_copernicus_cds` library. It also provides a comprehensive example of the library's features and capabilities, helping you to manage Climate Data Store (CDS) API authentication and make programmatic requests from the CDS.\n",
"\n",
@ -14,8 +14,8 @@
"\n",
"To begin, you’ll need your CDS API credentials. Follow these steps to obtain them:\n",
"\n",
"1. Register or log in to the CDS at [https://cds.climate.copernicus.eu](https://cds.climate.copernicus.eu).\n",
"2. Visit [https://cds.climate.copernicus.eu/how-to-api](https://cds-beta.climate.copernicus.eu/how-to-api) and copy the API key provided.\n",
"1. Register or log in to the CDS at [https://cds-beta.climate.copernicus.eu](https://cds-beta.climate.copernicus.eu).\n",
"2. Visit [https://cds-beta.climate.copernicus.eu/how-to-api](https://cds-beta.climate.copernicus.eu/how-to-api) and copy the API key provided.\n",
"\n",
"The library will prompt you to enter:\n",
"- **URL**: The URL field is prefilled; simply press Enter to accept the default.\n",
@ -44,54 +44,57 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting cdsapi\n",
" Downloading cdsapi-0.7.5-py2.py3-none-any.whl (12 kB)\n",
"Collecting datapi (from cdsapi)\n",
" Downloading datapi-0.1.1-py3-none-any.whl (26 kB)\n",
"Requirement already satisfied: requests>=2.5.0 in /opt/conda/lib/python3.10/site-packages (from cdsapi) (2.31.0)\n",
"Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from cdsapi) (4.65.0)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi) (3.1.0)\n",
"Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi) (3.4)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi) (2.0.2)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi) (2024.7.4)\n",
"Requirement already satisfied: attrs in /opt/conda/lib/python3.10/site-packages (from datapi->cdsapi) (23.1.0)\n",
"Collecting multiurl>=0.3.2 (from datapi->cdsapi)\n",
" Using cached multiurl-0.3.3-py3-none-any.whl\n",
"Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from datapi->cdsapi) (4.6.2)\n",
"Requirement already satisfied: pytz in /opt/conda/lib/python3.10/site-packages (from multiurl>=0.3.2->datapi->cdsapi) (2023.3)\n",
"Requirement already satisfied: python-dateutil in /opt/conda/lib/python3.10/site-packages (from multiurl>=0.3.2->datapi->cdsapi) (2.8.2)\n",
"Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil->multiurl>=0.3.2->datapi->cdsapi) (1.16.0)\n",
"Installing collected packages: multiurl, datapi, cdsapi\n",
"Successfully installed cdsapi-0.7.5 datapi-0.1.1 multiurl-0.3.3\n",
"Requirement already satisfied: attrs in /opt/conda/lib/python3.10/site-packages (23.1.0)\n",
"Collecting attrs\n",
" Using cached attrs-24.2.0-py3-none-any.whl (63 kB)\n",
"Installing collected packages: attrs\n",
" Attempting uninstall: attrs\n",
" Found existing installation: attrs 23.1.0\n",
" Uninstalling attrs-23.1.0:\n",
" Successfully uninstalled attrs-23.1.0\n",
"Successfully installed attrs-24.2.0\n",
"Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.10/site-packages (4.6.2)\n",
"Collecting typing_extensions\n",
" Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)\n",
"Installing collected packages: typing_extensions\n",
" Attempting uninstall: typing_extensions\n",
" Found existing installation: typing_extensions 4.6.2\n",
" Uninstalling typing_extensions-4.6.2:\n",
" Successfully uninstalled typing_extensions-4.6.2\n",
"Successfully installed typing_extensions-4.12.2\n"
]
}
],
"source": [
"!pip install \"cdsapi>=0.7.4\"\n",
"!pip install \"attrs>=24.0.0\"\n",
"!pip install \"typing_extensions>=4.0.0\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install numpy matplotlib cartopy xarray netCDF4 cdsapi"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install \"numpy>=1.16.5,<1.23.0\"\n",
"!pip install pandas xarray \n",
"!pip install cdsapi\n",
"!pip install matplotlib cartopy\n",
"!pip install cfgrib\n",
"!pip install xskillscore"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"pip install pandas"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install zarr dask fsspec"
"!pip install -U cdsapi\n",
"!pip install -U attrs\n",
"!pip install -U typing_extensions"
]
},
{
@ -107,9 +110,43 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting git+https://code-repo.d4science.org/D4Science/d4science_copernicus_cds.git\n",
" Cloning https://code-repo.d4science.org/D4Science/d4science_copernicus_cds.git to /tmp/pip-req-build-ezp3t9w8\n",
" Running command git clone --filter=blob:none --quiet https://code-repo.d4science.org/D4Science/d4science_copernicus_cds.git /tmp/pip-req-build-ezp3t9w8\n",
" warning: filtering not recognized by server, ignoring\n",
" Resolved https://code-repo.d4science.org/D4Science/d4science_copernicus_cds.git to commit 22ec9dd7e72830d0057adc7454aa7e864a86a23c\n",
" Preparing metadata (setup.py) ... \u001b[?25ldone\n",
"\u001b[?25hRequirement already satisfied: cdsapi>=0.7.2 in /opt/conda/lib/python3.10/site-packages (from d4science-copernicus-cds==1.0.0) (0.7.5)\n",
"Requirement already satisfied: attrs in /opt/conda/lib/python3.10/site-packages (from d4science-copernicus-cds==1.0.0) (24.2.0)\n",
"Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.10/site-packages (from d4science-copernicus-cds==1.0.0) (4.12.2)\n",
"Requirement already satisfied: datapi in /opt/conda/lib/python3.10/site-packages (from cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (0.1.1)\n",
"Requirement already satisfied: requests>=2.5.0 in /opt/conda/lib/python3.10/site-packages (from cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (2.31.0)\n",
"Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (4.65.0)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (3.1.0)\n",
"Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (3.4)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (2.0.2)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests>=2.5.0->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (2024.7.4)\n",
"Requirement already satisfied: multiurl>=0.3.2 in /opt/conda/lib/python3.10/site-packages (from datapi->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (0.3.3)\n",
"Requirement already satisfied: pytz in /opt/conda/lib/python3.10/site-packages (from multiurl>=0.3.2->datapi->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (2023.3)\n",
"Requirement already satisfied: python-dateutil in /opt/conda/lib/python3.10/site-packages (from multiurl>=0.3.2->datapi->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (2.8.2)\n",
"Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil->multiurl>=0.3.2->datapi->cdsapi>=0.7.2->d4science-copernicus-cds==1.0.0) (1.16.0)\n",
"Building wheels for collected packages: d4science-copernicus-cds\n",
" Building wheel for d4science-copernicus-cds (setup.py) ... \u001b[?25ldone\n",
"\u001b[?25h Created wheel for d4science-copernicus-cds: filename=d4science_copernicus_cds-1.0.0-py3-none-any.whl size=12131 sha256=6ab25037c1292ce4574176dcbf903fc89ef4fefb1f35e2c4b05305a21b04b886\n",
" Stored in directory: /tmp/pip-ephem-wheel-cache-bqdnt94j/wheels/fc/f5/c5/7e21e683ac76148a67a1198259550e1ea8370aa32dad6e4d0e\n",
"Successfully built d4science-copernicus-cds\n",
"Installing collected packages: d4science-copernicus-cds\n",
"Successfully installed d4science-copernicus-cds-1.0.0\n"
]
}
],
"source": [
"!pip install -U git+https://code-repo.d4science.org/D4Science/d4science_copernicus_cds.git"
]
@ -125,7 +162,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
@ -163,9 +200,21 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 4,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ENV - None None\n",
"env is not configured\n",
"Configuration from file /home/jovyan/.cdsapirc: {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"saving config to env\n",
"Set environment variables CDSAPI_URL, CDSAPI_KEY\n"
]
}
],
"source": [
"client = cds_authenticate()"
]
@ -188,9 +237,34 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 5,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ENV - https://cds.climate.copernicus.eu/api db1f2085-6b8b-42e6-b832-625dfaf831a4\n",
"Configuration from environment {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"Configuration from file /home/jovyan/.cdsapirc: {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"Configuration from environment: {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"Configuration from file: {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n"
]
},
{
"data": {
"text/plain": [
"({'url': 'https://cds.climate.copernicus.eu/api',\n",
" 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'},\n",
" {'url': 'https://cds.climate.copernicus.eu/api',\n",
" 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'})"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cds_show_conf()"
]
@ -211,9 +285,18 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"URL https://cds.climate.copernicus.eu/api\n",
"KEY db1f2085-6b8b-42e6-b832-625dfaf831a4\n"
]
}
],
"source": [
"URL, KEY = cds_get_credentials()\n",
"print(\"URL\", URL)\n",
@ -237,9 +320,19 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ENV - https://cds.climate.copernicus.eu/api db1f2085-6b8b-42e6-b832-625dfaf831a4\n",
"Configuration from environment {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"Saved Configuration file /home/jovyan/.cdsapirc\n"
]
}
],
"source": [
"cds_save_conf()"
]
@ -259,7 +352,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
@ -281,7 +374,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
@ -301,9 +394,34 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 10,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ENV - https://cds.climate.copernicus.eu/api db1f2085-6b8b-42e6-b832-625dfaf831a4\n",
"Configuration from environment {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"Configuration from file /home/jovyan/.cdsapirc: {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"Configuration from environment: {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n",
"Configuration from file: {'url': 'https://cds.climate.copernicus.eu/api', 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'}\n"
]
},
{
"data": {
"text/plain": [
"({'url': 'https://cds.climate.copernicus.eu/api',\n",
" 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'},\n",
" {'url': 'https://cds.climate.copernicus.eu/api',\n",
" 'key': 'db1f2085-6b8b-42e6-b832-625dfaf831a4'})"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cds_show_conf()"
]
@ -333,13 +451,38 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 11,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"datadir: %s /home/jovyan/cds_dataDir/out_2024_12_02_13_38_29_example/\n"
]
}
],
"source": [
"datadir = cds_datadir(\"example\")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"datadir: %s ./out/out_2024_12_02_13_38_29_current_example/\n"
]
}
],
"source": [
"datadir_current = cds_datadir(\"current_example\", basepath=\"./out\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
@ -367,9 +510,17 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 13,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"datadir: %s ./out/out_2024_12_02_13_38_29_current_example/\n"
]
}
],
"source": [
"datadir_current = cds_datadir(\"current_example\", basepath=\"./out\")\n"
]
@ -384,7 +535,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
@ -398,7 +549,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
"version": "3.10.11"
}
},
"nbformat": 4,

1
requirements_tutorials.txt
View File

@ -13,4 +13,3 @@ cfgrib
zarr
dask
fsspec
git+https://code-repo.d4science.org/D4Science/d4science_copernicus_cds.git