{ "cells": [ { "cell_type": "markdown", "id": "8fff8f33", "metadata": {}, "source": [ "# Advanced GNNs functionalities: truncated graphs and long range interactions\n", "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/luigibonati/mlcolvar/blob/main/docs/notebooks/tutorials/adv_gnn_advanced.ipynb)\n", "\n", "\n", "Prerequisites: \n", "- GNN-based CVs tutorial\n", "\n", "Reference papers: \n", "- _Zhang, Bonati, Trizio, Zhang, Kang, Hou, and Parrinello, [JCTC](https://doi.org/10.1021/acs.jctc.4c01197) (2025), [ArXiv](https://arxiv.org/abs/2409.07339)_\n", "- _Kang, Zhang, Trizio, Hou, and Parrinello, [JCTC](https://doi.org/10.1021/acs.jctc.5c01848) (2026), [ArXiv](https://arxiv.org/abs/2510.18018)_\n", "\n", "\n", "Author: Enrico Trizio\n", "\n", "---" ] }, { "cell_type": "markdown", "id": "9183590b", "metadata": {}, "source": [ "### Colab setup" ] }, { "cell_type": "code", "execution_count": 1, "id": "9b6f0a32", "metadata": {}, "outputs": [], "source": [ "# Colab setup\n", "import os\n", "\n", "if os.getenv(\"COLAB_RELEASE_TAG\"):\n", " import subprocess\n", " subprocess.run('wget https://raw.githubusercontent.com/luigibonati/mlcolvar/main/colab_setup.sh', shell=True)\n", " cmd = subprocess.run('bash colab_setup.sh TUTORIAL', shell=True, stdout=subprocess.PIPE)\n", " print(cmd.stdout.decode('utf-8'))" ] }, { "cell_type": "markdown", "id": "15bcfb7d", "metadata": {}, "source": [ "## Overview\n", "GNN models are much more complex than the descriptor-based counterparts as they directly process atomic coordinates.\n", "This higher complexity also results in a wide array of possibilities in the graph definition with different schemes that are better suited for different scenarios to improve the performance while keeping the computational cost as low as reasonably possible.\n", "\n", "In this tutorial, we will explain two advanced approaches for GNN-based MLCVs:\n", "- **Truncated graphs**, in which the input graph is built only within a cutoff from a subset of atoms (the `system` atoms), which are still allowed to interact with the `environment` atoms that are within the cutoff but not beyond that. This approach is very convenient, for example, when describing interactions of molecules with solvents or surfaces, where only the atoms close to the molecule are relevant whereas the others can be neglected with potenitally huge computational savings.\n", "\n", "- **Long range interactions**, in which a `long_range_cutoff`, larger than the standard one, is defined for a `subsystem` to keep connectivity at larger distances so that information can still be propagated between such nodes. This approach is convenient, for example, when large conformational changes can occur in the system, for instance, in binding/unbinding and association/dissociation processes.\n", "\n", "
\n", "\n", "More details about the two approaches are provided in the correpsonding sections of the tutorial.\n", "\n", "---" ] }, { "cell_type": "markdown", "id": "26044d02", "metadata": {}, "source": [ "## Truncated graphs\n", "\n", "In many cases, including all the atoms in the simulation cell into the input graph is not the best choice, computationally and conceptually.\n", "For instance, the interaction of a molecule with an environment (e.g., a solvent or a surface) is rather local, with the molecule interacting only with a reduced neighborhood of atoms rather than with the whole environment.\n", "It follows that, in such cases, not all the atoms of the environment need to be included in the graph, thus providing significant computational savings thanks to simpler and smaller input graphs.\n", "\n", "- **Selection**\n", "\n", " In `mlcolvar`, this scenario is implemented as a *truncated graph* based on the definition of two atomic subsets from the whole simulation box:\n", " - `system_selection`: the *reactive* part of the system, e.g., the molecule in the example above. Information from the `system_atoms` is used both to update the GNN through the layers and for the final readout.\n", " - `environment_selection`: the *non-reactive* part of the system, e.g., the surface/solvent in the example above. Information from the `environment_atoms` is used to update the GNN through the layers but **not** for the final readout.\n", "\n", "\n", "- **Graph construction**\n", "\n", " Based on these subsets, the graph is built following a two-step procedure:\n", " 1. **Pre-selection of nodes**: non relevant atoms are filtered out from the whole original simulation cell, thus reducing the nodes on which the graph will be built. \n", " To this aim, `system_atoms` are always included as nodes, whereas `enivronment_atoms` are included as nodes only if they are within a `cutoff+buffer` from one of the `system_atoms`. `cutoff` is the same cutoff used for drawing edges later, while the additional `buffer` is applied to have a more stable behaviour of the graph when `environment_atoms` are close to the border of the selected region.\n", " 2. **Edge placement**: edges are drawn between nodes that are within a `cutoff` from each other.\n", "\n", "\n", "- **Readout**\n", "\n", " The readout, both graph-level following a pooling or node-level, is limited to the `system_atoms`.\n", "\n", "\n", "**NOTE**\n", "\n", "To create/handle these advanced graphs it is **highly recommended** to use the util function `create_dataset_from_trajectories`" ] }, { "cell_type": "markdown", "id": "e8778c82", "metadata": {}, "source": [ "### Load data\n", "To create/handle these advanced graphs it is **highly recommended** to use the util function `create_dataset_from_trajectories`.\n", "There, the keys `system_selection` and `environment_selection` can be used to conveniently select the atoms corresponing to the system and the environment using the `MDtraj` selection syntax and the `buffer` key can be used to define the buffer region around the cutoff used for the preselection of the nodes." ] }, { "cell_type": "code", "execution_count": 2, "id": "83b16f88", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/etrizio@iit.local/Bin/dev/mlcolvar/mlcolvar/data/graph/utils.py:86: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n", " graph_labels = torch.tensor( config.graph_labels, dtype=torch.get_default_dtype() ) if config.graph_labels is not None else None\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Dataset info:\n", " DictDataset( \"data_list\": 1000,\n", "\t metadata={\"atomic_numbers\": [8, 11, 17],\n", "\t\t \"cutoff\": 4.0,\n", "\t\t \"buffer\": 3.0,\n", "\t\t \"long_range_cutoff\": -1.0,\n", "\t\t \"used_idx\": tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,\n", " 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,\n", " 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41,\n", " 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,\n", " 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69,\n", " 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83,\n", " 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,\n", " 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,\n", " 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,\n", " 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,\n", " 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,\n", " 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,\n", " 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n", " 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,\n", " 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,\n", " 210, 211, 212, 213, 214, 215, 216, 217]),\n", "\t\t \"used_names\": [MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-Na, MOL1-Cl],\n", "\t\t \"data_type\": graphs } )\n", "\n", "Datamodule info:\n", " DictModule(dataset -> DictDataset( \"data_list\": 1000,\n", "\t metadata={\"atomic_numbers\": [8, 11, 17],\n", "\t\t \"cutoff\": 4.0,\n", "\t\t \"buffer\": 3.0,\n", "\t\t \"long_range_cutoff\": -1.0,\n", "\t\t \"used_idx\": tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,\n", " 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,\n", " 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41,\n", " 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,\n", " 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69,\n", " 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83,\n", " 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,\n", " 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,\n", " 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,\n", " 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,\n", " 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,\n", " 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,\n", " 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n", " 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,\n", " 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,\n", " 210, 211, 212, 213, 214, 215, 216, 217]),\n", "\t\t \"used_names\": [MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-Na, MOL1-Cl],\n", "\t\t \"data_type\": graphs } ),\n", "\t train_loader -> DictLoader(length=0.8, batch_size=1000, shuffle=True),\n", "\t valid_loader -> DictLoader(length=0.2, batch_size=1000, shuffle=True))\n" ] } ], "source": [ "from mlcolvar.data import DictModule\n", "from mlcolvar.io import create_dataset_from_trajectories\n", "\n", "# loading arguments \n", "# same as to laod_dataframe\n", "load_args = [{'start' : 0, 'stop' : 500, 'stride' : 1},\n", " {'start' : 0, 'stop' : 500, 'stride' : 1}\n", " ]\n", "\n", "# create dataset\n", "dataset = create_dataset_from_trajectories(\n", " trajectories=[\"https://github.com/EnricoTrizio/nacl_gnn_data/raw/refs/heads/main/UNBOUND/traj.xyz\", \n", " \"https://github.com/EnricoTrizio/nacl_gnn_data/raw/refs/heads/main/BOUND/traj.xyz\"],\n", " topologies=None, \n", " cutoff=4.0, # Angstrom \n", " buffer=3.0, # Angstrom \n", " system_selection='type Na or type Cl',\n", " environment_selection='type O',\n", " load_args=load_args,\n", " )\n", "print('Dataset info:\\n', dataset, end=\"\\n\\n\")\n", "\n", "# load dataset into a DictModule\n", "datamodule = DictModule(dataset=dataset)\n", "print('Datamodule info:\\n', datamodule)" ] }, { "cell_type": "markdown", "id": "bddb9654", "metadata": {}, "source": [ "### Visualize the truncated graph\n", "The generated dataset can be saved as an `extxyz` using the `save_dataset_configurations_as_extxyz` utils.\n", "This way, one can visualize which atoms are included in the graph, for example by comparing the selected atoms with the original complete system with a visualization software that supports dynamics extxyz files." ] }, { "cell_type": "code", "execution_count": 3, "id": "0675485d", "metadata": {}, "outputs": [], "source": [ "from mlcolvar.data.utils import save_dataset_configurations_as_extyz\n", "save_dataset_configurations_as_extyz(dataset, 'test.xyz')" ] }, { "cell_type": "markdown", "id": "3f86b729", "metadata": {}, "source": [ "### GNN-model initialization\n", "\n", "The GNN model can be initialized from the generated dataset using the `dataset_for_initialization` to make it simpler to inherit all the parameters used in the construction of the graphs." ] }, { "cell_type": "code", "execution_count": 4, "id": "dc879724", "metadata": {}, "outputs": [], "source": [ "from mlcolvar.core.nn.graph.schnet import SchNetModel\n", "\n", "gnn_model = SchNetModel(n_out=1,\n", " dataset_for_initialization=dataset,\n", " pooling_operation=\"mean\",\n", " n_bases=16,\n", " n_layers=2,\n", " n_filters=16,\n", " n_hidden_channels=16,\n", " w_out_after_pool=True,\n", " aggr='mean'\n", " )" ] }, { "cell_type": "markdown", "id": "c32aa4e6", "metadata": {}, "source": [ "### CV model initialization" ] }, { "cell_type": "code", "execution_count": 5, "id": "5b9419b4", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/etrizio@iit.local/Bin/miniconda3/envs/graph_mlcolvar_test_2.5/lib/python3.9/site-packages/lightning/pytorch/utilities/parsing.py:198: Attribute 'model' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['model'])`.\n" ] } ], "source": [ "import torch\n", "from mlcolvar.cvs import DeepTDA\n", "\n", "# we can still set the options for the optimizer the usual way\n", "# options for the BLOCKS of the cv are disabled when passing an external model\n", "options = {'optimizer' : {'lr' : 1e-3},\n", " 'lr_scheduler': {\n", " 'scheduler': torch.optim.lr_scheduler.ExponentialLR,\n", " 'gamma': 0.9999}\n", " }\n", "\n", "model = DeepTDA(n_states=2,\n", " n_cvs=1,\n", " target_centers=[-7, 7],\n", " target_sigmas=[0.2, 0.2],\n", " model=gnn_model)" ] }, { "cell_type": "markdown", "id": "96ea8118", "metadata": {}, "source": [ "### Model training" ] }, { "cell_type": "code", "execution_count": null, "id": "de7bea91", "metadata": {}, "outputs": [], "source": [ "from lightning import Trainer\n", "from mlcolvar.utils.trainer import MetricsCallback\n", "from mlcolvar.utils.plot import plot_metrics\n", "import matplotlib.pyplot as plt\n", "\n", "# define callbacks\n", "metrics = MetricsCallback()\n", "\n", "# here the number of epochs is low for testing, you should increase it for applications\n", "trainer = Trainer(\n", " callbacks=[metrics],\n", " logger=False,\n", " enable_checkpointing=False,\n", " max_epochs=5,\n", " enable_model_summary=False\n", ")\n", "\n", "trainer.fit(model, datamodule)" ] }, { "cell_type": "markdown", "id": "6e6a8121", "metadata": {}, "source": [ "#### Check training metrics" ] }, { "cell_type": "code", "execution_count": 7, "id": "49a38958", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAE8CAYAAAAmDQ2PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABJuUlEQVR4nO3dd3gU5drH8e9syaZveoOQhN47hFCUEpqCVAXlKKAHRIpifeV4KIqKYkPhiMcGFgRBQbGASEcMoXcILRBaCoR00nbn/SOwx5UWQpJJsvfnuuZyMy2/WeLeO/PMPI+iqqqKEEIIh6TTOoAQQgjtSBEQQggHJkVACCEcmBQBIYRwYFIEhBDCgUkREEIIByZFQAghHJgUASGEcGBSBIQQwoFJERCiGMLDwxkxYoTWMYQodVIERLmZP38+iqKwfft2raMIIa4waB1AiMogLi4OnU6+M4mqR/6qhcMpLCwkPz//trYxmUwYjcYySqSt7OxsrSMIDUkREBXO2bNnefTRRwkMDMRkMtGoUSM+//xzu3Xy8/OZMmUKrVq1wmw24+bmRqdOnVi3bp3deidPnkRRFN5++21mzZpFrVq1MJlMHDx4kGnTpqEoCseOHWPEiBF4eXlhNpsZOXIkOTk5dvv5e5vA1Utbmzdv5plnnsHf3x83NzcGDBhASkqK3bZWq5Vp06YREhKCq6srXbp04eDBg8VuZ7Barbz//vs0adIEZ2dn/P396dWrl+2y2tVjnD9//jXbKorCtGnTbD9fPeaDBw/y0EMP4e3tTceOHXn77bdRFIVTp05ds49Jkybh5OTEpUuXbPNiY2Pp1asXZrMZV1dX7r77bjZv3my3XWZmJhMnTiQ8PByTyURAQADdu3dn586dtzxmUX7kcpCoUJKSkmjXrh2KojB+/Hj8/f1ZsWIFjz32GBkZGUycOBGAjIwMPv30Ux588EFGjRpFZmYmn332GT179mTr1q00b97cbr/z5s0jNzeX0aNHYzKZ8PHxsS174IEHiIiIYMaMGezcuZNPP/2UgIAA3nzzzVvmnTBhAt7e3kydOpWTJ08ya9Ysxo8fz7fffmtbZ9KkScycOZO+ffvSs2dP9uzZQ8+ePcnNzS3We/LYY48xf/58evfuzT//+U8KCwvZtGkTW7ZsoXXr1sXax9/df//91KlTh9dffx1VVenTpw8vvPACixcv5vnnn7dbd/HixfTo0QNvb28A1q5dS+/evWnVqhVTp05Fp9Mxb948unbtyqZNm2jbti0AY8aM4bvvvmP8+PE0bNiQixcv8scff3Do0CFatmxZotyiDKhClJN58+apgLpt27YbrvPYY4+pwcHB6oULF+zmDx06VDWbzWpOTo6qqqpaWFio5uXl2a1z6dIlNTAwUH300Udt8+Lj41VA9fT0VJOTk+3Wnzp1qgrYra+qqjpgwADV19fXbl5YWJg6fPjwa44lOjpatVqttvlPP/20qtfr1bS0NFVVVTUxMVE1GAxq//797fY3bdo0FbDb5/WsXbtWBdQnn3zymmVXf+/VY5w3b9416wDq1KlTrznmBx988Jp1o6Ki1FatWtnN27p1qwqoX375pe131qlTR+3Zs6fdcefk5KgRERFq9+7dbfPMZrM6bty4mx6f0J5cDhIVhqqqfP/99/Tt2xdVVblw4YJt6tmzJ+np6bZLCXq9HicnJ6DocklqaiqFhYW0bt36upcbBg0ahL+//3V/75gxY+x+7tSpExcvXiQjI+OWmUePHo2iKHbbWiwW22WVNWvWUFhYyNixY+22mzBhwi33DfD999+jKApTp069Ztlff+/t+vsxAwwZMoQdO3Zw/Phx27xvv/0Wk8lEv379ANi9ezdHjx7loYce4uLFi7Z/n+zsbLp168bGjRuxWq0AeHl5ERsby7lz50qcU5Q9KQKiwkhJSSEtLY2PP/4Yf39/u2nkyJEAJCcn29b/4osvaNq0Kc7Ozvj6+uLv788vv/xCenr6NfuOiIi44e+tUaOG3c9XL3v89Rp4Sbe9Wgxq165tt56Pj49t3Zs5fvw4ISEhdpevSsP13o/7778fnU5nu5SlqipLliyhd+/eeHp6AnD06FEAhg8ffs2/0aeffkpeXp7t/Z85cyb79+8nNDSUtm3bMm3aNE6cOFGqxyHunLQJiArj6jfIf/zjHwwfPvy66zRt2hSAr7/+mhEjRtC/f3+ef/55AgIC0Ov1zJgxw+6b7FUuLi43/L16vf6689VijLx6J9uWlhudEVgslhtuc733IyQkhE6dOrF48WL+9a9/sWXLFhISEuzaRq7+G7311lvXtLtc5e7uDhS1tXTq1Illy5axatUq3nrrLd58802WLl1K7969i3t4ooxJERAVhr+/Px4eHlgsFqKjo2+67nfffUfNmjVZunSp3Yfg9S6baCksLAyAY8eO2X37vnjxYrHONGrVqsVvv/1GamrqDc8Grp5RpKWl2c2/3p0+tzJkyBDGjh1LXFwc3377La6urvTt29cuD4Cnp+ct/40AgoODGTt2LGPHjiU5OZmWLVvy2muvSRGoQORykKgw9Ho9gwYN4vvvv2f//v3XLP/rrZdXv4H/9Rt3bGwsMTExZR/0NnTr1g2DwcDcuXPt5s+ZM6dY2w8aNAhVVXn55ZevWXb12D09PfHz82Pjxo12yz/88MPbzjto0CD0ej0LFy5kyZIl9OnTBzc3N9vyVq1aUatWLd5++22ysrKu2f7qv5HFYrnmslxAQAAhISHk5eXddi5RduRMQJS7zz//nJUrV14z/6mnnuKNN95g3bp1REZGMmrUKBo2bEhqaio7d+5k9erVpKamAtCnTx+WLl3KgAEDuPfee4mPj+ejjz6iYcOG1/1w0kpgYCBPPfUU77zzDvfddx+9evViz549rFixAj8/v1s27nbp0oWHH36YDz74gKNHj9KrVy+sViubNm2iS5cujB8/HoB//vOfvPHGG/zzn/+kdevWbNy4kSNHjtx23oCAALp06cK7775LZmYmQ4YMsVuu0+n49NNP6d27N40aNWLkyJFUq1aNs2fPsm7dOjw9Pfnpp5/IzMykevXqDB48mGbNmuHu7s7q1avZtm0b77zzzm3nEmVHioAod3//VnzViBEjqF69Olu3buWVV15h6dKlfPjhh/j6+tKoUSO7a9MjRowgMTGR//73v/z22280bNiQr7/+miVLlrB+/fpyOpLiefPNN3F1deWTTz5h9erVREVFsWrVKjp27Iizs/Mtt583bx5Nmzbls88+4/nnn8dsNtO6dWvat29vW2fKlCmkpKTw3XffsXjxYnr37s2KFSsICAi47bxDhgxh9erVeHh4cM8991yzvHPnzsTExDB9+nTmzJlDVlYWQUFBREZG8vjjjwPg6urK2LFjWbVqFUuXLsVqtVK7dm0+/PBDnnjiidvOJMqOopZnC5YQAii6fu/t7c2rr77KSy+9pHUc4cCkTUCIMnb58uVr5s2aNQso+lYthJbkcpAQZezbb79l/vz53HPPPbi7u/PHH3+wcOFCevToQYcOHbSOJxycFAEhyljTpk0xGAzMnDmTjIwMW2Pxq6++qnU0IaRNQAghHJm0CQghhAOTIiCEEA5M2gQo6g/l3LlzeHh43FHPjEIIUVGoqkpmZiYhISE3HRpVigBw7tw5QkNDtY4hhBCl7vTp01SvXv2Gy6UIAB4eHkDRm3W1y1whhKjMMjIyCA0NtX2+3YgUAf7XFa+np6cUASFElXKrS9zSMCyEEA5MioAQQjgwKQJCCOHApAgIIYQDkyIghBAOTIqAEEI4MLlFtITi4rag7ppPrs5Ens6FfL0LFqMrVqMbOLmjM3lgcPbE6GLG5OqFq6snHs4umE0mXIxGeTJZCFEhSBEoobzMZJrqisa7xXplKrjx+lYVMlQDF6xG0lUjOYoTuYozeXoXLAZXLEY3jM6euLp74+buh9kvjEDfINydnMrjcIQQDkqKQAmF1WrLUaMTltx0LHmZkJ+FLj8LXWEOxsLLmKyXcbHm4qrm4axY0CngpRTipSsE/jbSVOGV6TJw6X+zL1qMxOPBJaMfOR41cA5sQJ3wJoSYzXImIYQoFTKeAEWPV5vNZtLT08vmiWFLAeRno+Znkp+TRk72JXJz0im4nI4lNwM1v6iIKPnZGApzcLPm4KPkXndXmVY9B60+nHevjWtoG1rXa4mPq2vpZxZCVGrF/VyTIkA5FIGSKLhMVuopLiYeJffCcVwyThJYkIQJi91qRwrc2OVUB7faXenSqCVucvlIiBsKDw9n4sSJTJw48Y73tX79erp06cKlS5fw8vK64/2VtuJ+rsnloIrK6IJ7YH3cA+v/b57VQu7FE6SciEGXuIfA3DPUNWZTV92N5chuNhwI4IR/J6Lb9iDc21u77EKUos6dO9O8eXNmzZp1x/vatm0bbm5udx6qCpEiUJno9Dj71yHUv07Rz3lZpB9bR97xtQTknqWrUzJd079n6y9rWOnbmXui7qFGBfyGIkRpUlUVi8WCwXDrjzN/f/9ySFS5yHMClZnJHXOjvgTc9x5qj7e44N8aCwptndIYk/kDe3+czMcbf+dywU1uWxIOS1VVLhcUaDIV9yr0iBEj2LBhA++//z6KoqAoCvPnz0dRFFasWEGrVq0wmUz88ccfHD9+nH79+hEYGIi7uztt2rRh9erVdvsLDw+3O6NQFIVPP/2UAQMG4OrqSp06dVi+fHmJ39Pvv/+eRo0aYTKZCA8P55133rFb/uGHH1KnTh2cnZ0JDAxk8ODBtmXfffcdTZo0wcXFBV9fX6Kjo8nOzi5xluKSM4EqQvEKw6/LC5B9gUvbv8A7KZY+Lklkn/ucj77dRmSnYbQPC9M6pqhAcgsLifroY01+d8yY0bgYjbdc7/333+fIkSM0btyYV155BYADBw4A8OKLL/L2229Ts2ZNvL29OX36NPfccw+vvfYaJpOJL7/8kr59+xIXF0eNGjVu+DtefvllZs6cyVtvvcXs2bMZNmwYp06dwsfH57aOaceOHTzwwANMmzaNIUOG8OeffzJ27Fh8fX0ZMWIE27dv58knn+Srr76iffv2pKamsmnTJgDOnz/Pgw8+yMyZMxkwYACZmZls2rSp2MXyTmh6JrBx40b69u1LSEgIiqLwww8/2C1XVZUpU6YQHByMi4sL0dHRHD161G6d1NRUhg0bhqenJ15eXjz22GNkZWWV41FUMG5+eN/9LHR7nQy3UNx0Fp522k3mhpn8Z+MaCq1WrRMKUWxmsxknJydcXV0JCgoiKCgIvV4PwCuvvEL37t2pVasWPj4+NGvWjMcff5zGjRtTp04dpk+fTq1atW75zX7EiBE8+OCD1K5dm9dff52srCy2bt1621nfffddunXrxuTJk6lbty4jRoxg/PjxvPXWWwAkJCTg5uZGnz59CAsLo0WLFjz55JNAUREoLCxk4MCBhIeH06RJE8aOHYu7u/tt57hdmp4JZGdn06xZMx599FEGDhx4zfKZM2fywQcf8MUXXxAREcHkyZPp2bMnBw8exNnZGYBhw4Zx/vx5fv/9dwoKChg5ciSjR4/mm2++Ke/DqVh8a+PZ+y3yD3yP/tD39HROocnZ+byy9BQTeg/BXxrHHJ6zwUDMmNGa/e471bp1a7ufs7KymDZtGr/88ovtQ/Xy5cskJCTcdD9Nmza1vXZzc8PT05Pk5OTbznPo0CH69etnN69Dhw7MmjULi8VC9+7dCQsLo2bNmvTq1YtevXrZLkM1a9aMbt260aRJE3r27EmPHj0YPHgw3uVwg4emZwK9e/fm1VdfZcCAAdcsU1WVWbNm8e9//5t+/frRtGlTvvzyS86dO2c7Yzh06BArV67k008/JTIyko4dOzJ79mwWLVrEuXPnyvloKiCdDqcm96Pv9io5Jh9C9Hm8UPg77373H+JSLmidTmhMURRcjEZNptJ42PHvd/k899xzLFu2jNdff51Nmzaxe/dumjRpQn5+/k33Y/zbZSlFUbCWwRmzh4cHO3fuZOHChQQHBzNlyhSaNWtGWloaer2e33//nRUrVtCwYUNmz55NvXr1iI+PL/Ucf1dhG4bj4+NJTEwkOjraNs9sNhMZGUlMTAwAMTExeHl52X0jiI6ORqfTERsbe8N95+XlkZGRYTdVab61ce31NrnedXHXWXjVeTvLf/6AbWfOap1MiFtycnLCYrHccr3NmzczYsQIBgwYQJMmTQgKCuLkyZNlH/CKBg0asHnz5msy1a1b13YJy2AwEB0dzcyZM9m7dy8nT55k7dq1QFHx6dChAy+//DK7du3CycmJZcuWlXnuClsEEhMTAQgMDLSbHxgYaFuWmJhIQECA3XKDwYCPj49tneuZMWMGZrPZNoWGhpZy+grI5I5z12nkh3ZEr8DzbofZvGoOa44d1zqZEDcVHh5ObGwsJ0+e5MKFCzf8ll6nTh2WLl3K7t272bNnDw899FCZfKO/kWeffZY1a9Ywffp0jhw5whdffMGcOXN47rnnAPj555/54IMP2L17N6dOneLLL7/EarVSr149YmNjef3119m+fTsJCQksXbqUlJQUGjRoUOa5K2wRKEuTJk0iPT3dNp0+fVrrSOVDb8Cp3QQK6xVdt5zofpyDGz9mtRQCUYE999xz6PV6GjZsiL+//w2v8b/77rt4e3vTvn17+vbtS8+ePWnZsmW55WzZsiWLFy9m0aJFNG7cmClTpvDKK68wYsQIALy8vFi6dCldu3alQYMGfPTRRyxcuJBGjRrh6enJxo0bueeee6hbty7//ve/eeedd+jdu3eZ564w3UYoisKyZcvo378/ACdOnKBWrVrs2rWL5s2b29a7++67ad68Oe+//z6ff/45zz77LJcu/a/XtcLCQpydnVmyZMl12xqup0J2G1HGrPuXoDu4BIAPsyNocPfjdKlZU+NUQojSUtzPtQp7JhAREUFQUBBr1qyxzcvIyCA2NpaoqCgAoqKiSEtLY8eOHbZ11q5di9VqJTIystwzVya6xvdjbTQEgLFu8exb/zEb409qG0oIUe40LQJZWVns3r2b3bt3A0WNwbt37yYhIQFFUZg4cSKvvvoqy5cvZ9++fTzyyCOEhITYzhYaNGhAr169GDVqFFu3bmXz5s2MHz+eoUOHEhISot2BVRK6RoOwXCkET7qfYPu6j/nz1M1vpxPCUYwZMwZ3d/frTmPGjNE6XqnR9HLQ1V74/m748OHMnz8fVVWZOnUqH3/8MWlpaXTs2JEPP/yQunXr2tZNTU1l/Pjx/PTTT+h0OgYNGsQHH3xwWw9ZOOLloL+y7FuE/tBSAGZm1eWu6MdpV8MBGsuFuInk5OQb3jno6el5zU0pFY10JX0bHL0IoKpY9i5AH1f0ZOXrWfXp3uNx2lSvpnEwIURJVfo2AVGOFAV902FYat8DwItuh/n1t0+IdZS7poRwYFIERBFFQd9iOJaa3dEp8G/3g6xZ9V9+jTuidTIhRBmSIiD+R1HQt3oMS3hX9Ar8y+MIaTFzeXPdWnJu8ei9EKJykjYBpE3gGqqK9dBSdPu/BeBkoQufFjahfpNo7mvYAE+TSeOAQohbkYbh2yBF4AbObiN/60c4FWQCkFDowoq8QM571qdGjaa0Dw+jrp8fulLoDEwIUbqkYVjcuWptcLr3Awpr9aJQMVLDcJnH3U4yzbKS3sfeZ88vr/Hc/LeZ/NvvrD1+goJidPIlRHm73mhifx+75K9OnjyJoii255duZv369SiKQlpa2h3n1IqMLCZuzskNQ6tHoelDcHYLl+M3Y7xwgGB9HkNczzGEc5xN3cdPGwL5dH0Yreo2566IcJoGBWEqhT7jhSht58+fL5d++isL+b9UFI/RGcI74xLeGQrzIXkf1jNbsZ6OoRq5jHE/xRhOcfz0dvac8GSLxZU8kxe4+mP0DMbLK4hQLy+qmz0JNZuLNbSgEGUhKChI6wgVilwOErfP4AQhrdC1fQJDv0+h3ZNYA5qgolDLkMNAl0QmuJ/gOeNOniv4jacuzmfosTepEfsKcb+8zAdfTuX/Fszh3yt/Zf7OnWxJOM2ly5e1PirHo6pQmKvNVMymyI8//piQkJBruoTu168fjz76aLEGl/+7v18O2rp1Ky1atMDZ2ZnWrVuza9eu234r/6qyDTYvZwLizhicoEZHdDU6Ql4GXDiCmnqMy+nnKcxKRn/5Ii4F6ZgUldqGHGobcuhLEnCMwvSNHL/oxuFCd9YXeJBk9MfgE0GEfzD1/f2p5+9HiIdHqYxCJa7DkgdLH9Hmdw/8EgzOt1zt/vvvZ8KECaxbt45u3boBRV3FrFy5kl9//ZWsrKwSDS5/VVZWFn369KF79+58/fXXxMfH89RTT5X4sCrjYPNSBETpMXlCtdYo1Vrj+tf5VgvkXISMM5B6jLyUIyiXjuNUmE09Y9HUz6WoMFhzYjh1woXDcR4sLnTnlOKNag4nNKBaUWHw8yPCxxuDTk5iHYG3tze9e/fmm2++sRWB7777Dj8/P7p06YJOp6NZs2a29adPn86yZctYvnw548ePv+X+v/nmG6xWK5999hnOzs40atSIM2fO8MQTT5Qo718HmweoW7cuBw8e5K233mLEiBF2g817eHjYBpwH+8Hmw8LCAGjSpEmJctwOKQKi7On04B5QNIW0xARFlwMup8KleLh0gsLU41hTT+CUn06E4TIRhsv05spg35YdnDtjYm+8Jz8UeHLQ4oXiFUHz6qG0rl6NFsHB0sZQEnpT0TdyrX53MQ0bNoxRo0bx4YcfYjKZWLBgAUOHDkWn05V4cPmrDh06RNOmTXF2/t9ZydWu6kuiMg42L0VAaENRwNW3aKrW+n9/iLlpcOkkpMVjTT1B4cXjOOVeIESfR4g+hV7OKQDkqzs5dMKdnXFezC/wpdC7DpFh4dwdEU49Pz+5hFQcilKsSzJa69u3L6qq8ssvv9CmTRs2bdrEe++9BxSNOvb777/z9ttvU7t2bVxcXBg8ePAtB5fXytXB5tevX8+qVauYMmUK06ZNY9u2bXh5efH777/z559/smrVKmbPns1LL71EbGwsERERZZZJioCoWJy9ILg5BDdHBzgB5GfDpeNw8RjqxSNYLxzBqSCLZk6ZNHPKZCSnybTsY8thb77Y48dhQxhtImrTOSKCNtWrYbwyyLeonJydnRk4cCALFizg2LFj1KtXzzZs5F8Hl4eia/y3M7h8gwYN+Oqrr8jNzbWdDWzZsqXEWW9nsPno6GimTp2Kl5cXa9euZeDAgbbB5jt06MCUKVMICwtj2bJlPPPMMyXOdCtSBETF5+QGgU0hsCkKoFdVyE4qaoRO2oP13C48CrLo7nyB7s4XyFWPsClhB8uOBPCKEkLnuvW5t15dGgcGyhlCJTVs2DD69OnDgQMH+Mc//mGbf3Vw+b59+6IoCpMnT76tweUfeughXnrpJUaNGsWkSZM4efIkb7/9dolzPvvss7Rp04bp06czZMgQYmJimDNnDh9++CFQNNj8iRMnuOuuu/D29ubXX3+1G2x+zZo19OjRg4CAAGJjY8tlsHkpAqLyURRwDwL3IJTwu9BbrXDpBJzbjvX0nzhnJdoKQrY1jpXxB5hxMJhs91AeaNKYfg0b4O7kpPVRiNvQtWtXfHx8iIuL46GHHrLNf/fdd3n00Udp3749fn5+/N///d8NB4K5Hnd3d3766SfGjBlDixYtaNiwIW+++SaDBg0qUc6rg81PmTKF6dOnExwcfN3B5qdNm0Zubi516tSxDTZ/6NAhNm7cyKxZs8jIyCAsLKxcBpuXvoOQvoOqFFWFtHhI+BP19J8oORdsiw4WuPP95WDWFFanV4NGPNisKWFeXtplFaIMSQdyt0GKQBWlqpByCE6sRj2zBcVaCMBFi5FvcqrxXW41OtVtxOg2rakhxUBUMdKBnBCKAgENod2TKH3/C80eQXX1w1dfwASPk/zsuwW/hN8Z+vVXvLpuvTy1LK7hCIPNy5kAcibgUKyFcDoGDv8A6UXDZyZaTLyfFcGfhDK+XRQDGzVELw+jCSr3YPNyOeg2SBFwQKoVTv0B+xcWPc0MbMzz4dWMuoQEhvNa92iqmeVvQVRecjlIiJtRdBB+F/R+HxoPQdUZuMuUyjK/bYRc2s0DCxex/NDhMu+3RQitSREQjk3vBA0HoXSfCb51cFMsvG4+zDjnQ7yy+nemrVlLvgyWI6owKQJCAJirQ5fp0LDo/vCHXM/yifceNsXtZdSyH0iVRmNRRUkREOIqnQ4aD4GOL4DRhRZOGXzhs5uUpHge+34ZiZmZWicUotRJERDi70JaQ/QMcPMnVH+Zr3z3QMYZRn6/lLPpxX8aVYjKQIqAENfjEQJdp4M5FF9dHp/67kefk8LjP/xIclbZjvQkRHmSIiDEjbj4QOep4BmKr5LLZz77yMtKYcyPP5KRl6d1OiFKhRQBIW7G5Al3TwaPYAJ1l/mPz0HOpV7gxZWrKLyN3iqFqKgqdBGwWCxMnjyZiIgIXFxcqFWrFtOnT7e7d1tVVaZMmUJwcDAuLi5ER0dz9OhRDVOLKsfFCzr9C0we1NVnMMMrjpiEU7y3+U+tkwlxxyp0EXjzzTeZO3cuc+bM4dChQ7z55pvMnDmT2bNn29aZOXMmH3zwAR999BGxsbG4ubnRs2dPcnNzNUwuqhz3QGj/POgMdDGl8LDrGRbs3sPa4ye0TibEHanQ3Ub06dOHwMBAPvvsM9u8QYMG4eLiwtdff42qqoSEhPDss8/y3HPPAZCenk5gYCDz589n6NChxfo90m2EKLbjv8OOT7CgY0RqMxL0/ix+cCiB7u5aJxPCTpXoNqJ9+/asWbOGI0eOALBnzx7++OMP2yAL8fHxJCYmEh0dbdvGbDYTGRlJTEzMDfebl5dHRkaG3SREsdSMhtAo9Fh5xzuOgrxsJv++WrqXEJVWhS4CL774IkOHDqV+/foYjUZatGjBxIkTGTZsGACJiYkABAYG2m0XGBhoW3Y9M2bMwGw226bQ0NCyOwhRtSgKtHoc3PwJUHKY6HGKrWfO8uOhw1onE6JEKnQRWLx4MQsWLOCbb75h586dfPHFF7z99tt88cUXd7TfSZMmkZ6ebptOnz5dSomFQ3ByLSoEwP0uZ2hmTOe9zZulawlRKVXoIvD888/bzgaaNGnCww8/zNNPP82MGTMACAoKAiApKcluu6SkJNuy6zGZTHh6etpNQtyWoKYQ3hkFeNX7ONm5l3n3j81apxLitlXoIpCTk4Pub4N76PV6rFfuz46IiCAoKIg1a9bYlmdkZBAbG0tUVFS5ZhUOqPkjYPIgVMlkoMt5fj4cx6HkFK1TCXFbKnQR6Nu3L6+99hq//PILJ0+eZNmyZbz77rsMGDAAAEVRmDhxIq+++irLly9n3759PPLII4SEhNC/f39tw4uqz8kdGg0B4CnzGTyUQt7bvFkaiUWlYtA6wM3Mnj2byZMnM3bsWJKTkwkJCeHxxx9nypQptnVeeOEFsrOzGT16NGlpaXTs2JGVK1fi7OysYXLhMGp2g2MrcMs4y2j3BN45Y+CPU6foFB6udTIhiqVCPydQXuQ5AXFHzu+ETW9QiJ6eKW3x9w1h4ZAHUBRF62TCgVWJ5wSEqBSCWoBvHQxYeNT9LIdTLvDHqVNapxKiWKQICHGnFAUaDgbgAddzeCv5fLJtu7QNiEpBioAQpSGoOXjXwqgWMtz9HHsTk9h65ozWqYS4JSkCQpQGRYGGAwEY6paIMxa+3Llb20xCFIMUASFKS0grcA/EWc2jr0sSmxMSiE+9pHUqIW5KioAQpUXRQe2izg3/aU5GQWXh3r0ahxLi5qQICFGaIjqDwYVANZ12Tpf46fBhGYpSVGhSBIQoTUbXokIAjDRf4HJBIT8ePKRpJCFuRoqAEKWtZncAWusS8dPlsezgQbldVFRYUgSEKG3m6uBbDx0qA1xTOJF6ib2JSbfeTggNSBEQoizU7ArAUI8UQGXpgQPa5hHiBqQICFEWqkeBwQVfawatjen8dvQYWfn5WqcS4hpSBIQoC0ZnqNEegAfNl8gtLGTllbGyhahIpAgIUVbC7gKgkzERExaWHpC7hETFI0VAiLLiVw9c/XCy5tPZ+RIHk5M5nCIjj4mKRYqAEGVF0UGNjgA84psJIM8MiApHioAQZSmsEwANLGfxVAr4Je4IeYWFGocS4n+kCAhRlsyh4BWOTrUw2JxBRl4e60/Ea51KCBspAkKUtSuXhAZ7FvUoukwuCYkKRIqAEGWtRkdAISTvLCG6XGJPn+ZcRobWqYQApAgIUfZcfSCgEQCjAnNRgeWHDmubSYgrpAgIUR6uNBBHG88BKj8eOoxVOpUTFYAUASHKQ7VI0BnxyEuhhUse5zMz2XpaxiAW2pMiIER5cHKFaq0BGBOYC8APh6SBWGhPioAQ5eVKNxKtCk+gR2Xt8RNcunxZ41DC0UkREKK8BDUDkyeGgkyGBFjIt1j4fr90MS20JUVAiPKiM9ieGRjmVXSL6OJ9+ymwWLRMJRycFAEhytOVS0IhmYep4WogOTubtcdPaBxKODIpAkKUJ+8I8KyOYi3g6TADAAv27JExiIVmKnwROHv2LP/4xz/w9fXFxcWFJk2asH37dttyVVWZMmUKwcHBuLi4EB0dzdGjRzVMLMRNKIrtbKAjJ3HS69mbmMTWM3K7qNBGhS4Cly5dokOHDhiNRlasWMHBgwd555138Pb2tq0zc+ZMPvjgAz766CNiY2Nxc3OjZ8+e5ObmaphciJsI6wQoGFPj+Ge9EAA+3rr95tsIUUYMWge4mTfffJPQ0FDmzZtnmxcREWF7raoqs2bN4t///jf9+vUD4MsvvyQwMJAffviBoUOHXne/eXl55OXl2X7OkH5cRHly9YXg5nB+F8M8k/lEp2PHuXNsO3OWNtWraZ1OOJgKfSawfPlyWrduzf33309AQAAtWrTgk08+sS2Pj48nMTGR6Oho2zyz2UxkZCQxMTE33O+MGTMwm822KTQ0tEyPQ4hr1OoJgNvZzdzfsC4AH26JlbYBUe4qdBE4ceIEc+fOpU6dOvz222888cQTPPnkk3zxxRcAJCYmAhAYGGi3XWBgoG3Z9UyaNIn09HTbdPr06bI7CCGuJ6g5uPpDfjaPBxfibDCw6/x5uVNIlLsKXQSsVistW7bk9ddfp0WLFowePZpRo0bx0Ucf3dF+TSYTnp6edpMQ5Uqng1pFZ7DmM+t5pHkzAN7b/Cf58tyAKEclKgKnT5/mzF/uZti6dSsTJ07k448/LrVgAMHBwTRs2NBuXoMGDUhISAAgKCgIgKSkJLt1kpKSbMuEqLAiuoLOCKnHebSGM/5urpzJyGDhnr1aJxMOpERF4KGHHmLdunVA0SWZ7t27s3XrVl566SVeeeWVUgvXoUMH4uLi7OYdOXKEsLAwoKiROCgoiDVr1tiWZ2RkEBsbS1RUVKnlEKJMOJshokvRy2O/MK5dOwA+2badVOlTSJSTEhWB/fv307ZtWwAWL15M48aN+fPPP1mwYAHz588vtXBPP/00W7Zs4fXXX+fYsWN88803fPzxx4wbNw4ARVGYOHEir776KsuXL2ffvn088sgjhISE0L9//1LLIUSZqXcfKDpI2sN9QU7U9/cjKz+fubFbtU4mHESJikBBQQEmkwmA1atXc9999wFQv359zp8/X2rh2rRpw7Jly1i4cCGNGzdm+vTpzJo1i2HDhtnWeeGFF5gwYQKjR4+mTZs2ZGVlsXLlSpydnUsthxBlxj0AanQAQHdoGc91Kupb6Pv9Bzh64aKWyYSDUNQS3JMWGRlJly5duPfee+nRowdbtmyhWbNmbNmyhcGDB9u1F1QGGRkZmM1m0tPTpZFYlL/0M7DqWVBV6Poqz207wepjx2lTvRof9++HoihaJxSVUHE/10p0JvDmm2/y3//+l86dO/Pggw/SrFnRnQ3Lly+3XSYSQhSTuTqEdy56vecrnm4fhZNez7YzZ1l3Il7TaKLqK9GZAIDFYiEjI8OuC4eTJ0/i6upKQEBAqQUsD3ImIDSXkworngRLPrR/hjmn4dPtO6ju6cn3wx7EZKjQD/eLCqhMzwQuX75MXl6erQCcOnWKWbNmERcXV+kKgBAVgqsP1O1T9HrPAh5t3hh/NzfOZGSwYPcebbOJKq1ERaBfv358+eWXAKSlpREZGck777xD//79mTt3bqkGFMJh1O8PLt6QnYRr/G9MbF90m/Mn27eTkp2tbTZRZZWoCOzcuZNOnToB8N133xEYGMipU6f48ssv+eCDD0o1oBAOw+gMTR8uen1oGb1DfWgSGMjlgkJmx2zRNpuoskpUBHJycvDw8ABg1apVDBw4EJ1OR7t27Th16lSpBhTCodToAH4NwJKPbs9XvHBX0S2jyw8dZv/fnowXojSUqAjUrl2bH374gdOnT/Pbb7/Ro0cPAJKTk6VhVYg7oSjQcmTRf89soYlygT716gEwc+Mf0suoKHUlKgJTpkzhueeeIzw8nLZt29q6aFi1ahUtWrQo1YBCOByvcKjZvej1rnlMaNcaF6OBvYmJ/Hb0mKbRRNVToiIwePBgEhIS2L59O7/99pttfrdu3XjvvfdKLZwQDqvxUHDygIzTBCb+yciWLQGYHbNFehkVparEXUkHBQXRokULzp07Z3tCuG3bttSvX7/UwgnhsEzu0OTKyHgHFvNwwwj83Vw5m5HB4n37tc0mqpQSFQGr1corr7yC2WwmLCyMsLAwvLy8mD59OlartbQzCuGYIrqBdwQU5OByaAlPREYC8Mm2bWT8ZXhUIe5EiYrASy+9xJw5c3jjjTfYtWsXu3bt4vXXX2f27NlMnjy5tDMK4Zh0OmjxaNHr+HXcF2igpo836bl5fL59h7bZRJVRom4jQkJC+Oijj2y9h171448/MnbsWM6ePVtqAcuDdBshKrTYOXBqI/jUZmPNf/LkLytw0uv58eFhBF+5VVuIvyvTbiNSU1Ove+2/fv36pKamlmSXQogbaToMDC6QeoxOnKRVtRDyLRb+syVW62SiCihREWjWrBlz5sy5Zv6cOXNo2rTpHYcSQvyFizc0HASAsm8hz0YW3Yb9y+E4jl2UMQfEnSlR14QzZ87k3nvvZfXq1bZnBGJiYjh9+jS//vprqQYUQgB17oH4tZB5jobJ64muVYvVx4/zUew23r6nl9bpRCVWojOBu+++myNHjjBgwADS0tJIS0tj4MCBHDhwgK+++qq0Mwoh9AZoMbLo9bGVTGgcigKsPn6cwykpmkYTlVuJxxO4nj179tCyZUsslexhFmkYFpXGH2/CuR1QrS0vZjZl5dGjdI4IZ1afe7VOJiqYMm0YFkJopMlDRf0Knd3Kk/X80CkK6+NPciApWetkopKSIiBEZWIOhbC7AQg5+RP31KkDwIexcqeQKBkpAkJUNo0fAJ0RUg7wZC0XdIrC5lMJHEqWtgFx+27r7qCBAwfedHlaWtqdZBFCFIerH9TuCUd+JuDEj/SqHc2vR4/x+Y4dvNVb7hQSt+e2ioDZbL7l8kceeeSOAgkhiqHBAIhfA2mnGN/EyK9HYfWx45y6lEaYt5fW6UQlUqp3B1VWcneQqJT2fgOHfwCf2jyZ056Np04xoGEDpnbrqnUyUQHI3UFCVHV17wW9E6Qe48na7gD8dDiOpKwsjYOJykSKgBCVlbMZanYDoHbSBlpVC6HQauWrXbu1zSUqFSkCQlRm9e4DnR5SDvBUXR8Alh04SKaMNyCKSYqAEJWZq6/tuYEml2Ko6eNNdkEBPxw8pHEwUVlUqiLwxhtvoCgKEydOtM3Lzc1l3Lhx+Pr64u7uzqBBg0hKStIupBDlrX5/QEE5v5MnGlQD4Js9eymUUf5EMVSaIrBt2zb++9//XtNV9dNPP81PP/3EkiVL2LBhA+fOnbvl8wxCVCkeQVCtDQBdCg/i7ezM+cxM1h4/oXEwURlUiiKQlZXFsGHD+OSTT/D29rbNT09P57PPPuPdd9+la9eutGrVinnz5vHnn3+yZcsWDRMLUc7q9QHAkPAHjzSMAODr3Xu0TCQqiUpRBMaNG8e9995LdHS03fwdO3ZQUFBgN79+/frUqFGDmJiYG+4vLy+PjIwMu0mISs23HvjUAWsBQ1zPYdTp2JuYyN7ERK2TiQquwheBRYsWsXPnTmbMmHHNssTERJycnPDy8rKbHxgYSOJN/vhnzJiB2Wy2TaGhoaUdW4jypSi2swHXU+u4r+6Vs4FdcjYgbq5CF4HTp0/z1FNPsWDBApydnUttv5MmTSI9Pd02nT59utT2LYRmqrUFN3/Iz2R0QA4Aa44f55yc6YqbqNBFYMeOHSQnJ9OyZUsMBgMGg4ENGzbwwQcfYDAYCAwMJD8//5qO65KSkggKCrrhfk0mE56ennaTEJWeTg91igaXCTy3gXbVqmFRVRbt3adxMFGRVegi0K1bN/bt28fu3bttU+vWrRk2bJjttdFoZM2aNbZt4uLiSEhIsI19LIRDiegKRjfIPM+4cCMAPx46RG5hocbBREVVooHmy4uHhweNGze2m+fm5oavr69t/mOPPcYzzzyDj48Pnp6eTJgwgaioKNq1a6dFZCG0ZXSGWtFw+Ecap28n2COC85mZrDxylP4NG2idTlRAFfpMoDjee+89+vTpw6BBg7jrrrsICgpi6dKlWscSQju1e4KioKQcYHS9QAC+3bsP6TBYXI90JY10JS2qoM1vw9mt5IZ15a4dkG+x8OX9g2h6k7YyUbVIV9JCOLI6RSOMOZ/dzH21awBFZwNC/J0UASGqIv9G4BkKhXk85psJwKqjx0jNydE4mKhopAgIURUpSlHbABCc9CdNAv0psFpZeuCgxsFERSNFQIiqKuwuMLpA1nnGhbsA8N3+A9K7qLAjRUCIqsroDOFdAGiTux9vZ2cSs7LYdPKktrlEhSJFQIiq7MolIX3ibh6uGwwgl4SEHSkCQlRlHsEQ1BxQGWw6C8DmUwkkZmZqGktUHFIEhKjqahfdLup5/k/ahwRiVVUZflLYSBEQoqoLbg5ugVCQzRMhRQPQ/3DwEBZpIBZIERCi6lN0traBRhm7MDs7kZiVxZ8JCRoHExWBFAEhHEFEZ9A7oUtPYEzNoi4EpIFYgBQBIRyDkzvU6AhAH2PRGcDG+JOkZGdrmUpUAFIEhHAUtboD4JG8i45BPlhUlR+lgdjhSREQwlH41ALvCLAW8kRgUR9CSw8exCodCTs0KQJCOJKaRWcDDbL24O5k5FxGJjvOntU4lNCSFAEhHEmNjmBwQZeVyJhwVwB+PHRY41BCS1IEhHAkRmcIu9JA7HQagNXHjpOdn69lKqEhKQJCOJorDcTmC3tp5uVMbmEhvx87rnEooRUpAkI4Gq9w8KmDolqYEJwLwE9ySchhSREQwhHVigagee5B9ArsOHeO0+npGocSWpAiIIQjCm0PRlcMOSmMqGYA5GzAUUkREMIRGUxFI48BD7ieB+Cnw4flmQEHJEVACEd1pYE4IOMwYSYr5zOz2H5GnhlwNFIEhHBU5lDwq4+iWnn6ShfTy+WSkMORIiCEI7vSQNzechQdKquPHydLnhlwKFIEhHBk1duBkztOeZcY6JNf9MzA0WNapxLlSIqAEI5M7wThdwPwiPkCIJeEHI0UASEc3ZVO5UJzjhOsz2PX+fMkpKVpm0mUGykCQjg6zxDwb4SCyoQrXUzL2YDjqNBFYMaMGbRp0wYPDw8CAgLo378/cXFxduvk5uYybtw4fH19cXd3Z9CgQSQlJWmUWIhK6srtol118Riw8tPhOBmI3kFU6CKwYcMGxo0bx5YtW/j9998pKCigR48eZP9lSLynn36an376iSVLlrBhwwbOnTvHwIEDNUwtRCVUrS2YPHEuyKSXewZJWVnEnj6jdSpRDhRVrTyPCKakpBAQEMCGDRu46667SE9Px9/fn2+++YbBgwcDcPjwYRo0aEBMTAzt2rUr1n4zMjIwm82kp6fj6elZlocgRMW19xs4/AMnnULpf7om3WvX4q3evbROJUqouJ9rFfpM4O/Sr3Rw5ePjA8COHTsoKCggOjratk79+vWpUaMGMTExN9xPXl4eGRkZdpMQDq9WNKAQnn+aGvoc1p2I59Lly1qnEmWs0hQBq9XKxIkT6dChA40bNwYgMTERJycnvLy87NYNDAwkMTHxhvuaMWMGZrPZNoWGhpZldCEqB7cACG4OwOO+aRRarfwad0TbTKLMVZoiMG7cOPbv38+iRYvueF+TJk0iPT3dNp0+fboUEgpRBdTqAUC0/gwmLPxw8BCV6IqxKIFKUQTGjx/Pzz//zLp166hevbptflBQEPn5+aT97Z7mpKQkgoKCbrg/k8mEp6en3SSEAIJagKs/Jutlerte5OjFixxMTtE6lShDFboIqKrK+PHjWbZsGWvXriUiIsJueatWrTAajaxZs8Y2Ly4ujoSEBKKioso7rhCVn04HNbsBMNLrIgA/HDqkZSJRxip0ERg3bhxff/0133zzDR4eHiQmJpKYmMjlK41VZrOZxx57jGeeeYZ169axY8cORo4cSVRUVLHvDBJC/E1EV1D0hFmSqWfIYkXcEXILC7VOJcpIhS4Cc+fOJT09nc6dOxMcHGybvv32W9s67733Hn369GHQoEHcddddBAUFsXTpUg1TC1HJuXhB9bYADPdMISs/nzXHZSD6qqpSPSdQVuQ5ASH+JvkArH+ZAsVI56S2NKoWxscD+mudStyGKvmcgBCinPg3BM9qGNUC+jonsfXMWc7IQPRVkhQBIcS1FMV2u+gIzyQUVJYeOKhxKFEWpAgIIa4vvDMYXQgikw5OqSw7cJA8aSCucqQICCGuz+gCEVduF/VI5FJuLr8fkwbiqkaKgBDixmr3AkWhleECtfTZLNq7V+tEopRJERBC3Jh7AIS0AWCY2zn2JyWzX8brqFKkCAghbq7uPQD0cUnCrBTw7d79GgcSpUmKgBDi5vwagFcETlgY4nqW344eJVW6mK4ypAgIIW5OUaD+fQA87HYevTWfxXv3aRxKlBYpAkKIW6seBe7BeCj53O9yjkV793G5oEDrVKIUSBEQQtyaTgf1+wEwwv0sObk5/HhQehetCqQICCGKJ+wucPXFR8mjv8t5vty9mwKLRetU4g5JERBCFI/eAPWKzgYedT9DckY6Px+O0ziUuFNSBIQQxRfRFZzNBOlyGeRyno+3bSNfzgYqNSkCQojiMzhBw/sBGOOeQEZWGt/vP6BxKHEnpAgIIW5Pza7gEYy3Lp/hrqf5ZNt2MvPytE4lSkiKgBDi9ugM0GQYAMPdzuKcl8rc2K0ahxIlJUVACHH7qrUB/0aYFAsveR5l0d69xKVc0DqVKAEpAkKI26co0HoU6Ix0MF2ilymJV9etl1tGKyEpAkKIkvEIgYaDAHjR4xgXU07JZaFKSIqAEKLk6t0HPrXx1BXyttcBvt6xnT9PJWidStwGKQJCiJLTGyDqGXDyoKExiymecfzfypUcSk7ROpkoJikCQog74+YH7Z5CRaGPSzLPOO9n3I8/cjw1VetkohikCAgh7lxQU5R2E1BRGOCSyAumXYxZ8i1/JsiloYpOioAQonTU6FhUCBQdvZxT+K9HLP/5dSH/2RJLXmGh1unEDUgREEKUnhodUTpPQ3X2pqYhh6+8dxB8eAHjFs5n7fETqKqqdULxN4oq/ypkZGRgNptJT0/H09NT6zhCVH656ai756MkbAagQFVYmRvAVqcGtGp6Nz3r1sXFaNQ4ZNVW3M81KQJIERCizFw4TOGebzBcPGyblVDowpYCX/Ldq2H0CsUrtAU96tRGURQNg1Y9DlcE/vOf//DWW2+RmJhIs2bNmD17Nm3bti3WtlIEhChjF4+RH/crytlYjKr9sJT7CjzIUFw5r/fhslddMIdSLySCegH+eJpMUhxKyKGKwLfffssjjzzCRx99RGRkJLNmzWLJkiXExcUREBBwy+2lCAhRTgouY03cQ2bCNvJST+F3+TQ6rv0Isqiwq8DMKZ0fVmdvLM7eOLn742kOxNvVBaN7IIEeHrg7OeHm5KTBgVR8DlUEIiMjadOmDXPmzAHAarUSGhrKhAkTePHFF2+5vRQBITSSeZ7s8wdJz0jGlLwHslPwVTNvuVmuqiPLqidLNXAZI7mKkXydiUKdM1aDM4U6ZywGE6rBBcXgjN5oQm90Rqd3QjE4odM7oTcY0eudiiaDEYPBiEHvhNFgwGgwYDA4YTQYMeoNGPV6jDodOkWpNGcmxf1cM5RjpjKRn5/Pjh07mDRpkm2eTqcjOjqamJiY626Tl5dH3l/6P8/IyCjznEKI6/AIxs0jGDcAHiyal5dJ4eU0LOd3cTHlFGpOKoa8NJwL0nCx5qKoVpwVK856K34UAJf/tz8VuHq1qZSGOLCqYEGhAAWrCioKVq7/X/U66xRNVyhF8+wptujKX9dFubqIZK/GtOk5sXQO6G8qfRG4cOECFouFwMBAu/mBgYEcPnz4utvMmDGDl19+uTziCSFul8kDg8kDg1coIQ2us9xSCJdToSCHzOxLXM7JICcnHUt+DgW5mVgLclAKL6MvzEVvyUWx5KO3FqBTCzGohRhUC3qs6LGgV60YsGLEil65/kURnQI6VIyoXPP5XU4u5F4qs31X+iJQEpMmTeKZZ56x/ZyRkUFoaKiGiYQQxaY3gHtRW5+HdzgepbVfVQXVCqoFrFZQrVithRQUFlBoKaCwsAAVFdVqRVUtRativfKzFRUrWK22dVDVovm2CUDFqqqAWvTrVPXKsxMq1ivLr85TAVW1AhBgDrxR6jtW6YuAn58fer2epKQku/lJSUkEBQVddxuTyYTJZCqPeEKIykJRQNEDetAXzdIBpitTVVXpnxh2cnKiVatWrFmzxjbParWyZs0aoqKiNEwmhBAVX6U/EwB45plnGD58OK1bt6Zt27bMmjWL7OxsRo4cqXU0IYSo0KpEERgyZAgpKSlMmTKFxMREmjdvzsqVK69pLBZCCGGvSjwncKfkOQEhRFVT3M+1St8mIIQQouSkCAghhAOTIiCEEA6sSjQM36mrzSLSfYQQoqq4+nl2q2ZfKQJAZmZRh1Xy1LAQoqrJzMzEbDbfcLncHUTRw2Xnzp3Dw8PjtnoIvNrdxOnTpyvdXUWSXRuSXRuVOTuULL+qqmRmZhISEoJOd+Mr/3ImQFGvo9WrVy/x9p6enpXyDwsku1YkuzYqc3a4/fw3OwO4ShqGhRDCgUkREEIIByZF4A6YTCamTp1aKXsklezakOzaqMzZoWzzS8OwEEI4MDkTEEIIByZFQAghHJgUASGEcGBSBIQQwoFJESih//znP4SHh+Ps7ExkZCRbt27VOtI1pk2bhqIodlP9+vVty3Nzcxk3bhy+vr64u7szaNCga8ZqLi8bN26kb9++hISEoCgKP/zwg91yVVWZMmUKwcHBuLi4EB0dzdGjR+3WSU1NZdiwYXh6euLl5cVjjz1GVlaW5tlHjBhxzb9Dr169KkT2GTNm0KZNGzw8PAgICKB///7ExcXZrVOcv5OEhATuvfdeXF1dCQgI4Pnnn6ewsFDz7J07d77mvR8zZozm2efOnUvTpk1tD39FRUWxYsUK2/Jyfc9VcdsWLVqkOjk5qZ9//rl64MABddSoUaqXl5ealJSkdTQ7U6dOVRs1aqSeP3/eNqWkpNiWjxkzRg0NDVXXrFmjbt++XW3Xrp3avn17TbL++uuv6ksvvaQuXbpUBdRly5bZLX/jjTdUs9ms/vDDD+qePXvU++67T42IiFAvX75sW6dXr15qs2bN1C1btqibNm1Sa9eurT744IOaZx8+fLjaq1cvu3+H1NRUu3W0yt6zZ0913rx56v79+9Xdu3er99xzj1qjRg01KyvLts6t/k4KCwvVxo0bq9HR0equXbvUX3/9VfXz81MnTZqkefa7775bHTVqlN17n56ernn25cuXq7/88ot65MgRNS4uTv3Xv/6lGo1Gdf/+/aqqlu97LkWgBNq2bauOGzfO9rPFYlFDQkLUGTNmaJjqWlOnTlWbNWt23WVpaWmq0WhUlyxZYpt36NAhFVBjYmLKKeH1/f2D1Gq1qkFBQepbb71lm5eWlqaaTCZ14cKFqqqq6sGDB1VA3bZtm22dFStWqIqiqGfPntUsu6oWFYF+/frdcJuKkl1VVTU5OVkF1A0bNqiqWry/k19//VXV6XRqYmKibZ25c+eqnp6eal5enmbZVbWoCDz11FM33KaiZFdVVfX29lY//fTTcn/P5XLQbcrPz2fHjh1ER0fb5ul0OqKjo4mJidEw2fUdPXqUkJAQatasybBhw0hISABgx44dFBQU2B1H/fr1qVGjRoU7jvj4eBITE+2yms1mIiMjbVljYmLw8vKidevWtnWio6PR6XTExsaWe+a/W79+PQEBAdSrV48nnniCixcv2pZVpOzp6ekA+Pj4AMX7O4mJiaFJkyZ2Y3r37NmTjIwMDhw4oFn2qxYsWICfnx+NGzdm0qRJ5OTk2JZVhOwWi4VFixaRnZ1NVFRUub/n0oHcbbpw4QIWi+WaQewDAwM5fPiwRqmuLzIykvnz51OvXj3Onz/Pyy+/TKdOndi/fz+JiYk4OTnh5eVlt01gYCCJiYnaBL6Bq3mu955fXZaYmEhAQIDdcoPBgI+Pj+bH06tXLwYOHEhERATHjx/nX//6F7179yYmJga9Xl9hslutViZOnEiHDh1o3LgxQLH+ThITE6/7b3N1WXm4XnaAhx56iLCwMEJCQti7dy//93//R1xcHEuXLtU8+759+4iKiiI3Nxd3d3eWLVtGw4YN2b17d7m+51IEqrDevXvbXjdt2pTIyEjCwsJYvHgxLi4uGiZzLEOHDrW9btKkCU2bNqVWrVqsX7+ebt26aZjM3rhx49i/fz9//PGH1lFu242yjx492va6SZMmBAcH061bN44fP06tWrXKO6adevXqsXv3btLT0/nuu+8YPnw4GzZsKPcccjnoNvn5+aHX669pqU9KSiIoKEijVMXj5eVF3bp1OXbsGEFBQeTn55OWlma3TkU8jqt5bvaeBwUFkZycbLe8sLCQ1NTUCnc8NWvWxM/Pj2PHjgEVI/v48eP5+eefWbdunV236sX5OwkKCrruv83VZWXtRtmvJzIyEsDuvdcqu5OTE7Vr16ZVq1bMmDGDZs2a8f7775f7ey5F4DY5OTnRqlUr1qxZY5tntVpZs2YNUVFRGia7taysLI4fP05wcDCtWrXCaDTaHUdcXBwJCQkV7jgiIiIICgqyy5qRkUFsbKwta1RUFGlpaezYscO2ztq1a7Farbb/8SuKM2fOcPHiRYKDgwFts6uqyvjx41m2bBlr164lIiLCbnlx/k6ioqLYt2+fXSH7/fff8fT0pGHDhpplv57du3cD2L33WmS/HqvVSl5eXvm/56XRqu1oFi1apJpMJnX+/PnqwYMH1dGjR6teXl52LfUVwbPPPquuX79ejY+PVzdv3qxGR0erfn5+anJysqqqRbeh1ahRQ127dq26fft2NSoqSo2KitIka2Zmprpr1y51165dKqC+++676q5du9RTp06pqlp0i6iXl5f6448/qnv37lX79et33VtEW7RoocbGxqp//PGHWqdOnXK5zfJm2TMzM9XnnntOjYmJUePj49XVq1erLVu2VOvUqaPm5uZqnv2JJ55QzWazun79ervbKHNycmzr3Orv5Ortij169FB3796trly5UvX39y/z2yxvlf3YsWPqK6+8om7fvl2Nj49Xf/zxR7VmzZrqXXfdpXn2F198Ud2wYYMaHx+v7t27V33xxRdVRVHUVatWqapavu+5FIESmj17tlqjRg3VyclJbdu2rbplyxatI11jyJAhanBwsOrk5KRWq1ZNHTJkiHrs2DHb8suXL6tjx45Vvb29VVdXV3XAgAHq+fPnNcm6bt06FbhmGj58uKqqRbeJTp48WQ0MDFRNJpParVs3NS4uzm4fFy9eVB988EHV3d1d9fT0VEeOHKlmZmZqmj0nJ0ft0aOH6u/vrxqNRjUsLEwdNWrUNV8YtMp+vdyAOm/ePNs6xfk7OXnypNq7d2/VxcVF9fPzU5999lm1oKBA0+wJCQnqXXfdpfr4+Kgmk0mtXbu2+vzzz9s9J6BV9kcffVQNCwtTnZycVH9/f7Vbt262AqCq5fueS1fSQgjhwKRNQAghHJgUASGEcGBSBIQQwoFJERBCCAcmRUAIIRyYFAEhhHBgUgSEEMKBSREQQggHJkVAiErgesNWClEapAgIcQvXGyP4euMEC1EZyXgCQhRDr169mDdvnt08k8mkURohSo+cCQhRDCaTiaCgILvJ29sbKLpUM3fuXHr37o2Liws1a9bku+++s9t+3759dO3aFRcXF3x9fRk9ejRZWVl263z++ec0atQIk8lEcHAw48ePt1t+4cIFBgwYgKurK3Xq1GH58uVle9DCIUgREKIUTJ48mUGDBrFnzx6GDRvG0KFDOXToEADZ2dn07NkTb29vtm3bxpIlS1i9erXdh/zcuXMZN24co0ePZt++fSxfvpzatWvb/Y6XX36ZBx54gL1793LPPfcwbNgwUlNTy/U4RRV0hz2iClHlDR8+XNXr9aqbm5vd9Nprr6mqWtSl8ZgxY+y2iYyMVJ944glVVVX1448/Vr29vdWsrCzb8l9++UXV6XS2LqVDQkLUl1566YYZAPXf//637eesrCwVUFesWFFqxykck7QJCFEMXbp0Ye7cuXbzfHx8bK//PhpbVFSUbRSrQ4cO0axZM9zc3GzLO3TogNVqJS4uDkVROHfu3C3HG27atKnttZubG56entcMSynE7ZIiIEQxuLm5XXN5prS4uLgUaz2j0Wj3s6IoWK3WsogkHIi0CQhRCrZs2XLNzw0aNACgQYMG7Nmzh+zsbNvyzZs3o9PpqFevHh4eHoSHh9uNKStEeZEzASGKIS8vj8TERLt5BoMBPz8/AJYsWULr1q3p2LEjCxYsYOvWrXz22WcADBs2jKlTpzJ8+HCmTZtGSkoKEyZM4OGHHyYwMBCAadOmMWbMGAICAujduzeZmZls3ryZCRMmlO+BCocjRUCIYli5ciXBwcF28+rVq8fhw4eBojt3Fi1axNixYwkODmbhwoU0bNgQAFdXV3777Teeeuop2rRpg6urK4MGDeLdd9+17Wv48OHk5uby3nvv8dxzz+Hn58fgwYPL7wCFw5IxhoW4Q4qisGzZMvr37691FCFum7QJCCGEA5MiIIQQDkzaBIS4Q3JFVVRmciYghBAOTIqAEEI4MCkCQgjhwKQICCGEA5MiIIQQDkyKgBBCODApAkII4cCkCAghhAP7f6chL5sKfT9fAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots(1,1,figsize=(4,3))\n", "plot_metrics(metrics.metrics,\n", " keys=['train_loss', 'valid_loss'],\n", " colors=['fessa1', 'fessa5'],\n", " yscale='linear',\n", " ax = ax)" ] }, { "cell_type": "markdown", "id": "48ff9913", "metadata": {}, "source": [ "#### Check training outcome in CV space" ] }, { "cell_type": "code", "execution_count": 8, "id": "c1eac1b3", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz8AAAE8CAYAAADuXg/EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAz1ElEQVR4nO3deVxUZf//8fewo8iQCyAqi0uu5ZpEm95GkVq3pr9MswLl1iwsTbP0Ua7lWpm33pp5V1gPNcvWOy1byKVc07LFFLM0tQIzBVwSFa7fH36ZHFkEnYGB83o+HvNo5jrXOedz5uD59JnrzDU2Y4wRAAAAAFRxXhUdAAAAAACUB4ofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAFRKe/fulc1m08KFCy9qfZvNpgkTJrg0Jng2ih94rIULF8pmsxX5GD16dEWHV6JzY/Xx8VHNmjXVvn17DRs2TD/88MNFb/fEiROaMGGCVq9e7bpgL8H69es1YcIEZWVlVXQoAFAickphnpZTgPLgU9EBABcyadIkxcTEOLW1atWqgqIpvZtuukn33nuvjDHKzs7WN998o1deeUXz5s3T9OnTNWLEiDJv88SJE5o4caIkqXPnzi6OuOzWr1+viRMnKikpSSEhIRUdDgBcEDnlb56WU4DyQPEDj9e1a1d16NChVH1PnjwpPz8/eXlV/KDm5Zdfrrvvvtupbdq0abrttts0cuRINWvWTN26daug6ADAmsgpgLVV/L9m4CKtXr1aNptNS5cu1RNPPKF69eqpWrVqysnJkSQtW7ZM7du3V2BgoGrXrq27775bv/76q9M2kpKSFBQUpH379unWW29VUFCQ6tWrp7lz50qSvvvuO3Xp0kXVq1dXVFSUlixZckkx16pVS0uXLpWPj48mT57saD916pTGjRun9u3by263q3r16rr++uu1atUqR5+9e/eqTp06kqSJEyc6boEouFf522+/VVJSkho2bKiAgACFh4dr4MCB+vPPP51iOHr0qIYPH67o6Gj5+/srNDRUN910k7766iunfps2bdItt9wiu92uatWqqVOnTlq3bp1j+YQJEzRq1ChJUkxMjCOevXv3XtJ7BAAVgZxycTllwoQJstls2rVrl+6++27Z7XbVqVNHY8eOlTFG+/fvV48ePRQcHKzw8HA9++yzhY7j4MGDSk5OVlhYmAICAtS6dWu98sorhfplZWUpKSlJdrtdISEhSkxMLPK2686dOxc5kpWUlKTo6OgLvq+//vqrBg4cqLCwMPn7+6tly5Z6+eWXL7geKgdGfuDxsrOzdejQIae22rVrO54/+eST8vPz0yOPPKLc3Fz5+flp4cKFGjBggK666ipNnTpVmZmZ+ve//61169bp66+/drpFKy8vT127dtUNN9ygGTNmaPHixRo6dKiqV6+uxx9/XP3791evXr00f/583XvvvYqLiyt0y0RZREZGqlOnTlq1apVycnIUHBysnJwcvfjii+rXr58GDRqko0eP6qWXXlJCQoI2b96sNm3aqE6dOnr++ed1//336/bbb1evXr0kSVdeeaUk6ZNPPtHPP/+sAQMGKDw8XNu3b9eCBQu0fft2bdy4UTabTZI0ZMgQvfnmmxo6dKhatGihP//8U1988YV27Nihdu3aSZI+++wzde3aVe3bt9f48ePl5eWl1NRUdenSRZ9//rk6duyoXr16adeuXXrttdf03HPPOc5JQTIFAE9ETnFtTilw5513qnnz5po2bZpWrFihp556SjVr1tQLL7ygLl26aPr06Vq8eLEeeeQRXXXVVbrhhhskSX/99Zc6d+6s3bt3a+jQoYqJidGyZcuUlJSkrKwsDRs2TJJkjFGPHj30xRdfaMiQIWrevLneeecdJSYmXvR7V5TMzExdffXVstlsGjp0qOrUqaMPP/xQycnJysnJ0fDhw126P1QAA3io1NRUI6nIhzHGrFq1ykgyDRs2NCdOnHCsd+rUKRMaGmpatWpl/vrrL0f78uXLjSQzbtw4R1tiYqKRZKZMmeJoO3LkiAkMDDQ2m80sXbrU0b5z504jyYwfP/6CsUsyKSkpxS4fNmyYkWS++eYbY4wxZ86cMbm5uU59jhw5YsLCwszAgQMdbX/88UexMZz7HhR47bXXjCSzdu1aR5vdbi8xtvz8fNOkSROTkJBg8vPznbYfExNjbrrpJkfb008/bSSZPXv2FLs9APAE5BT35JTx48cbSWbw4MGOtjNnzpj69esbm81mpk2b5hRDYGCgSUxMdLTNmjXLSDKLFi1ytJ06dcrExcWZoKAgk5OTY4wx5t133zWSzIwZM5z2c/311xtJJjU11dHeqVMn06lTp0LxJyYmmqioKKe2848/OTnZ1K1b1xw6dMipX9++fY3dbi/yfUHlwm1v8Hhz587VJ5984vQ4V2JiogIDAx2vt2zZooMHD+qBBx5QQECAo7179+5q1qyZVqxYUWgf//rXvxzPQ0JC1LRpU1WvXl19+vRxtDdt2lQhISH6+eefL/mYgoKCJJ29BU2SvL295efnJ0nKz8/X4cOHdebMGXXo0KHQ7WjFOfc9OHnypA4dOqSrr75akpy2ERISok2bNum3334rcjvbtm3Tjz/+qLvuukt//vmnDh06pEOHDun48eO68cYbtXbtWuXn55f9oAHAA5BTXJtTCpx7zN7e3urQoYOMMUpOTna0F7wX5x7zBx98oPDwcPXr18/R5uvrq4ceekjHjh3TmjVrHP18fHx0//33O+3nwQcfLNXxlIYxRm+99ZZuu+02GWMc+e/QoUNKSEhQdnZ2qd8/eC5ue4PH69ixY4lfTj3/doFffvlF0tnEcr5mzZrpiy++cGoLCAgodKuW3W5X/fr1Cw3r2+12HTlypEzxF+XYsWOSpBo1ajjaXnnlFT377LPauXOnTp8+7Wgv7e0Qhw8f1sSJE7V06VIdPHjQaVl2drbj+YwZM5SYmKgGDRqoffv26tatm+699141bNhQkvTjjz9KUom3EmRnZ+uyyy4rVVwA4EnIKa7NKQUiIyOdXtvtdgUEBDjdUljQfu73hn755Rc1adKk0KQSzZs3dywv+G/dunUdhV6Bos7Lxfrjjz+UlZWlBQsWaMGCBUX2Of+9QOVD8YNK79xPpy6Gt7d3mdqNMZe0P0n6/vvv5e3t7UhCixYtUlJSknr27KlRo0YpNDRU3t7emjp1qn766adSbbNPnz5av369Ro0apTZt2igoKEj5+fm65ZZbnEZq+vTpo+uvv17vvPOOPv74Yz399NOaPn263n77bXXt2tXR9+mnn1abNm2K3Nf5yQcAqgpyylmlzSkFijo+dx5zSWw2W5H7yMvLK3G9guO6++67i/0AsOA7Uai8KH5Q5URFRUmS0tPT1aVLF6dl6enpjuUVZd++fVqzZo3i4uIcn9K9+eabatiwod5++22nTwbHjx/vtO75nxoWOHLkiNLS0jRx4kSNGzfO0V4winO+unXr6oEHHtADDzyggwcPql27dpo8ebK6du2qRo0aSZKCg4MVHx9f4rEUFw8AVBXklAvnlEsRFRWlb7/9Vvn5+U6jPzt37nQsL/hvWlqajh075vQBXHp6eqFtXnbZZUXeTlgwilScOnXqqEaNGsrLy7tg/kPlxXd+UOV06NBBoaGhmj9/vnJzcx3tH374oXbs2KHu3btXWGyHDx9Wv379lJeXp8cff9zRXvDp2LmfVG3atEkbNmxwWr9atWqSVGhqz6LWl6RZs2Y5vc7Lyyt0u0JoaKgiIiIc71X79u3VqFEjPfPMM45bKc71xx9/OJ5Xr169yHgAoKogp/zt/JziCt26dVNGRoZef/11R9uZM2c0Z84cBQUFqVOnTo5+Z86c0fPPP+/ol5eXpzlz5hTaZqNGjbRz506nfPXNN984/VxDUby9vdW7d2+99dZb+v777wstP3d7qLwY+UGV4+vrq+nTp2vAgAHq1KmT+vXr55iWNDo6Wg8//HC5xLFr1y4tWrRIxhjl5OTom2++0bJly3Ts2DHNnDlTt9xyi6Pvrbfeqrffflu33367unfvrj179mj+/Plq0aKFUwESGBioFi1a6PXXX9fll1+umjVrqlWrVmrVqpVjWtXTp0+rXr16+vjjj7Vnzx6nmI4ePar69evr//2//6fWrVsrKChIn376qb788kvHby94eXnpxRdfVNeuXdWyZUsNGDBA9erV06+//qpVq1YpODhY77//vqSzhZIkPf744+rbt698fX112223OYoiAKjsyCnF5xRXGDx4sF544QUlJSVp69atio6O1ptvvql169Zp1qxZjtGs2267Tddee61Gjx6tvXv3qkWLFnr77beL/P7RwIEDNXPmTCUkJCg5OVkHDx7U/Pnz1bJlS8fvNhVn2rRpWrVqlWJjYzVo0CC1aNFChw8f1ldffaVPP/1Uhw8fdvl7gHJWIXPMAaVQMC3pl19+WeTygmlJly1bVuTy119/3bRt29b4+/ubmjVrmv79+5sDBw449UlMTDTVq1cvtG6nTp1My5YtC7VHRUWZ7t27XzB2nTOFqpeXlwkJCTFt27Y1w4YNM9u3by/UPz8/30yZMsVERUUZf39/07ZtW7N8+fIip+Vcv369ad++vfHz83OaovPAgQPm9ttvNyEhIcZut5s77rjD/Pbbb059cnNzzahRo0zr1q1NjRo1TPXq1U3r1q3NvHnzCsX09ddfm169eplatWoZf39/ExUVZfr06WPS0tKc+j355JOmXr16xsvLi2mvAXgscorrc4oxf091/ccff1z0e5GZmWkGDBhgateubfz8/MwVV1zhNHV1gT///NPcc889Jjg42NjtdnPPPfeYr7/+utBU18YYs2jRItOwYUPj5+dn2rRpYz766KNSTXVdEE9KSopp0KCB8fX1NeHh4ebGG280CxYsKBQTKh+bMW7+1hkAAAAAeAC+8wMAAADAEih+AAAAAFgCxQ8AAAAASyhz8bN27VrddtttioiIkM1m07vvvuu03BijcePGqW7dugoMDFR8fHyheeEPHz6s/v37Kzg4WCEhIUpOTi5ySl0AAC6EvAQAKK0yFz/Hjx9X69atNXfu3CKXz5gxQ7Nnz9b8+fO1adMmVa9eXQkJCTp58qSjT//+/bV9+3Z98sknWr58udauXavBgwdf/FEAACyLvAQAKK1Lmu3NZrPpnXfeUc+ePSWd/XQtIiJCI0eO1COPPCJJys7OVlhYmBYuXKi+fftqx44datGihb788kt16NBBkrRy5Up169ZNBw4cUERExKUfFQDAkshLAICSuPRHTvfs2aOMjAzFx8c72ux2u2JjY7Vhwwb17dtXGzZsUEhIiCPBSFJ8fLy8vLy0adMm3X777YW2m5ub6/Sryvn5+Tp8+LBq1aolm83mykMAAJTAGKOjR48qIiJCXl6e/7VRd+UlidwEAJ6iLLnJpcVPRkaGJCksLMypPSwszLEsIyNDoaGhzkH4+KhmzZqOPuebOnWqJk6c6MpQAQCXYP/+/apfv35Fh3FB7spLErkJADxNaXKTS4sfdxkzZoxGjBjheJ2dna3IyEjt379fwcHBbt13q/EfFdn+/cQEt+4XAM5V3LVIKt/rUU5Ojho0aKAaNWqU2z49VUXlJk/5WwAAT/n/5LLkJpcWP+Hh4ZKkzMxM1a1b19GemZmpNm3aOPocPHjQab0zZ87o8OHDjvXP5+/vL39//0LtwcHBbi9+vPyrFdnu7v0CQPToFY7nxV2LpIq5HlWW27rclZekistNnva3AMC6PO3/k0uTm1x6w3ZMTIzCw8OVlpbmaMvJydGmTZsUFxcnSYqLi1NWVpa2bt3q6PPZZ58pPz9fsbGxrgwHAGBx5CUAwLnKPPJz7Ngx7d692/F6z5492rZtm2rWrKnIyEgNHz5cTz31lJo0aaKYmBiNHTtWERERjpl3mjdvrltuuUWDBg3S/Pnzdfr0aQ0dOlR9+/ZlRh0AQJmRlwAApVXm4mfLli36xz/+4XhdcL9zYmKiFi5cqEcffVTHjx/X4MGDlZWVpeuuu04rV65UQECAY53Fixdr6NChuvHGG+Xl5aXevXtr9uzZLjgcAIDVkJcAAKV1Sb/zU1FycnJkt9uVnZ3t9nsKz73n/lx7p3V3634BoLjrz/nK83pUntffyqa83puS/i7ITQDKk6f8f3JZrr+e/yMNAAAAAOACFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBJ8KjqAyip69ArH873TuldgJAAAAABKg5EfAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS3B58ZOXl6exY8cqJiZGgYGBatSokZ588kkZYxx9jDEaN26c6tatq8DAQMXHx+vHH390dSgAAJCXAAAOLi9+pk+frueff17/+c9/tGPHDk2fPl0zZszQnDlzHH1mzJih2bNna/78+dq0aZOqV6+uhIQEnTx50tXhAAAsjrwEACjg8qmu169frx49eqh797PTP0dHR+u1117T5s2bJZ39dG3WrFl64okn1KNHD0nSq6++qrCwML377rvq27evq0MCAFgYeQkAUMDlIz/XXHON0tLStGvXLknSN998oy+++EJdu3aVJO3Zs0cZGRmKj493rGO32xUbG6sNGzYUuc3c3Fzl5OQ4PQAAKA135CWJ3AQAlZHLR35Gjx6tnJwcNWvWTN7e3srLy9PkyZPVv39/SVJGRoYkKSwszGm9sLAwx7LzTZ06VRMnTnR1qAAAC3BHXpLITQBQGbl85OeNN97Q4sWLtWTJEn311Vd65ZVX9Mwzz+iVV1656G2OGTNG2dnZjsf+/ftdGDEAoCpzR16SyE0AUBm5fORn1KhRGj16tOMe6SuuuEK//PKLpk6dqsTERIWHh0uSMjMzVbduXcd6mZmZatOmTZHb9Pf3l7+/v6tDBQBYgDvykkRuAoDKyOUjPydOnJCXl/Nmvb29lZ+fL0mKiYlReHi40tLSHMtzcnK0adMmxcXFuTocAIDFkZcAAAVcPvJz2223afLkyYqMjFTLli319ddfa+bMmRo4cKAkyWazafjw4XrqqafUpEkTxcTEaOzYsYqIiFDPnj1dHQ4AwOLISwCAAi4vfubMmaOxY8fqgQce0MGDBxUREaH77rtP48aNc/R59NFHdfz4cQ0ePFhZWVm67rrrtHLlSgUEBLg6HACAxZGXAAAFbObcn7iuJHJycmS325Wdna3g4GC37it69IoL9tk7rbtbYwBgTaW5/kjlew0qz+tvZVNe701JfxfkIwDlqbjrUXlfi8py/XX5d34AAAAAwBNR/AAAAACwBJd/56cqKO2tJgAAAAAqD0Z+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASmugYAAABQKpX9J2EY+QEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCX4VHQAAADANaJHr3A83zutewVGAgCeiZEfAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCW4pfn799VfdfffdqlWrlgIDA3XFFVdoy5YtjuXGGI0bN05169ZVYGCg4uPj9eOPP7ojFAAAyEsAAEluKH6OHDmia6+9Vr6+vvrwww/1ww8/6Nlnn9Vll13m6DNjxgzNnj1b8+fP16ZNm1S9enUlJCTo5MmTrg4HAGBx5CUAQAEfV29w+vTpatCggVJTUx1tMTExjufGGM2aNUtPPPGEevToIUl69dVXFRYWpnfffVd9+/Z1dUgAAAsjLwEACrh85Od///ufOnTooDvuuEOhoaFq27at/vvf/zqW79mzRxkZGYqPj3e02e12xcbGasOGDUVuMzc3Vzk5OU4PAABKwx15SSI3AUBl5PLi5+eff9bzzz+vJk2a6KOPPtL999+vhx56SK+88ookKSMjQ5IUFhbmtF5YWJhj2fmmTp0qu93ueDRo0MDVYQMAqih35CWpfHNT9OgVjgcA4OK5vPjJz89Xu3btNGXKFLVt21aDBw/WoEGDNH/+/Ive5pgxY5Sdne147N+/34URAwCqMnfkJYncBACVkcuLn7p166pFixZObc2bN9e+ffskSeHh4ZKkzMxMpz6ZmZmOZefz9/dXcHCw0wMAgNJwR16SyE0AUBm5vPi59tprlZ6e7tS2a9cuRUVFSTr7JdPw8HClpaU5lufk5GjTpk2Ki4tzdTgAAIsjLwEACrh8treHH35Y11xzjaZMmaI+ffpo8+bNWrBggRYsWCBJstlsGj58uJ566ik1adJEMTExGjt2rCIiItSzZ09XhwMAsDjyEgCggMuLn6uuukrvvPOOxowZo0mTJikmJkazZs1S//79HX0effRRHT9+XIMHD1ZWVpauu+46rVy5UgEBAa4OBwBgceQlAEABlxc/knTrrbfq1ltvLXa5zWbTpEmTNGnSJHfsHgAAJ+QlAIDkpuIHAFB+zp3+eO+07hUYCQAAns3lEx4AAAAAgCei+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJPhUdAAAAAICqI3r0CsfzvdO6V2AkhTHyAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsAS3Fz/Tpk2TzWbT8OHDHW0nT55USkqKatWqpaCgIPXu3VuZmZnuDgUAAPISAFiYW4ufL7/8Ui+88IKuvPJKp/aHH35Y77//vpYtW6Y1a9bot99+U69evdwZCgAA5CUAsDi3FT/Hjh1T//799d///leXXXaZoz07O1svvfSSZs6cqS5duqh9+/ZKTU3V+vXrtXHjRneFAwCwOPISAMBtxU9KSoq6d++u+Ph4p/atW7fq9OnTTu3NmjVTZGSkNmzYUOS2cnNzlZOT4/QAAKAsXJmXJHITAFRGPu7Y6NKlS/XVV1/pyy+/LLQsIyNDfn5+CgkJcWoPCwtTRkZGkdubOnWqJk6c6I5QAQAW4Oq8JJGbAKAycvnIz/79+zVs2DAtXrxYAQEBLtnmmDFjlJ2d7Xjs37/fJdsFAFR97shLErkJACojlxc/W7du1cGDB9WuXTv5+PjIx8dHa9as0ezZs+Xj46OwsDCdOnVKWVlZTutlZmYqPDy8yG36+/srODjY6QEAQGm4Iy9J5CYAqIxcftvbjTfeqO+++86pbcCAAWrWrJkee+wxNWjQQL6+vkpLS1Pv3r0lSenp6dq3b5/i4uJcHQ4AwOLISwCAAi4vfmrUqKFWrVo5tVWvXl21atVytCcnJ2vEiBGqWbOmgoOD9eCDDyouLk5XX321q8MBAFgceQkAUMAtEx5cyHPPPScvLy/17t1bubm5SkhI0Lx58yoiFAAAyEsAYBHlUvysXr3a6XVAQIDmzp2ruXPnlsfuAQBwQl4CAGty2+/8AAAAAIAnofgBAAAAYAkUPwAAAAAsoUImPAAAABdvb8BdTq+jTy6poEgAoHJh5AcAAACAJVD8AAAAALAEih8AAAAAlsB3fgDAg0SPXlHRIQAAUGUx8gMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlMNU1AAAAgCJVtZ9gYOQHAAAAgCUw8uMC51fEe6d1r6BIAAAAABSHkR8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLYMIDAAAAAKWyN+CuYpdFn1xSjpFcHEZ+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCT4VHQAAAHC96NErHM/3TutegZEAgOdg5AcAAACAJVD8AAAAALAElxc/U6dO1VVXXaUaNWooNDRUPXv2VHp6ulOfkydPKiUlRbVq1VJQUJB69+6tzMxMV4cCAIAkchMA4CyXFz9r1qxRSkqKNm7cqE8++USnT5/WzTffrOPHjzv6PPzww3r//fe1bNkyrVmzRr/99pt69erl6lBKLXr0CqcHAKBqqYy5qSz2BtzleAAAiufyCQ9Wrlzp9HrhwoUKDQ3V1q1bdcMNNyg7O1svvfSSlixZoi5dukiSUlNT1bx5c23cuFFXX321q0MCAFgcuQkAIJXDd36ys7MlSTVr1pQkbd26VadPn1Z8fLyjT7NmzRQZGakNGzYUuY3c3Fzl5OQ4PQAAuFjkJgCwJrcWP/n5+Ro+fLiuvfZatWrVSpKUkZEhPz8/hYSEOPUNCwtTRkZGkduZOnWq7Ha749GgQQN3hg0AqMLITQBgXW4tflJSUvT9999r6dKll7SdMWPGKDs72/HYv3+/iyIEAFgNuQkArMttP3I6dOhQLV++XGvXrlX9+vUd7eHh4Tp16pSysrKcPmHLzMxUeHh4kdvy9/eXv7+/u0IFAFhEZc5NTGYAAJfO5SM/xhgNHTpU77zzjj777DPFxMQ4LW/fvr18fX2VlpbmaEtPT9e+ffsUFxfn6nAAACA3AQAkuWHkJyUlRUuWLNF7772nGjVqOO6VttvtCgwMlN1uV3JyskaMGKGaNWsqODhYDz74oOLi4phNBwDgFuQmAHC/c0eoo08uqcBIiufy4uf555+XJHXu3NmpPTU1VUlJSZKk5557Tl5eXurdu7dyc3OVkJCgefPmuToUAAAkkZsAAGe5vPgxxlywT0BAgObOnau5c+e6evcAABRCbgIASOXwOz8AAAAA4AkofgAAAABYgtumugYAABcvevSKig4BAKocRn4AAAAAWALFDwAAAABLoPgBAAAAYAl85wcAqpBzvyeyd1r3CowEAADPw8gPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJTDhAQAAVcjegLscz6NPLqnASADA8zDyAwAAAMASGPkBAAAA4Bbn/gSDVPE/w8DIDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVYdsKD8798BQAAAKCwc6fQv5h1PGnafUZ+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALMGyEx4AAOBpmIwHANyLkR8AAAAAlsDIDwAAHupippcFABSPkR8AAAAAlsDIDwBUMHd9z+Pc7e6d1t0t+4Bnc4wcTZA0IbsiQwFQiVTl7x8y8gMAAADAEhj5AYByVpU/UQMAoCQVfVcCIz8AAAAALIHiBwAAAIAlcNubG1T0cB4AoPJy1/TWBbmJvASgvJ17XYs+uaQCI2HkBwAAAIBFMPIDAAAAWJxVJuOp0JGfuXPnKjo6WgEBAYqNjdXmzZsrMhwAgMWRlwCgaquwkZ/XX39dI0aM0Pz58xUbG6tZs2YpISFB6enpCg0NraiwAKBK4ruIF0ZeAmBJE+ySpL0B5b/rishNFTbyM3PmTA0aNEgDBgxQixYtNH/+fFWrVk0vv/xyRYUEALAw8hIAVH0VMvJz6tQpbd26VWPGjHG0eXl5KT4+Xhs2bCjUPzc3V7m5uY7X2dnZkqScnJyLjiE/98RFr1sWlxIjgKqpvK4/xbmU61LBusYYV4XjEcqalyT35aYcm3ve24K/O/ISACe55Xs9Ly4HllduqpDi59ChQ8rLy1NYWJhTe1hYmHbu3Fmo/9SpUzVx4sRC7Q0aNHBbjK5in1XREQCAM1dcl44ePSq73X7pG/IQZc1Lkvtyk/ve1T5ntz/LbTsAgFLoU2RreeWmSjHb25gxYzRixAjH6/z8fB0+fFi1atWSzWa76O3m5OSoQYMG2r9/v4KDg10RaoWqascjVb1j4ng8G8dzYcYYHT16VBERES7ZXmVGbiodjsfzVbVj4ng8W0XnpgopfmrXri1vb29lZmY6tWdmZio8PLxQf39/f/n7+zu1hYSEuCye4ODgKvHHVKCqHY9U9Y6J4/FsHE/JqtKIT4Gy5iWJ3FRWHI/nq2rHxPF4torKTRUy4YGfn5/at2+vtLQ0R1t+fr7S0tIUFxdXESEBACyMvAQA1lBht72NGDFCiYmJ6tChgzp27KhZs2bp+PHjGjBgQEWFBACwMPISAFR9FVb83Hnnnfrjjz80btw4ZWRkqE2bNlq5cmWhL5u6k7+/v8aPH1/otoXKqqodj1T1jonj8Wwcj7V5Ql6Sqt5543g8X1U7Jo7Hs1X08dhMVZuvFAAAAACKUGE/cgoAAAAA5YniBwAAAIAlUPwAAAAAsASKHwAAAACWYKniZ/Xq1bLZbEU+vvzyy2LX69y5c6H+Q4YMKcfIixcdHV0otmnTppW4zsmTJ5WSkqJatWopKChIvXv3LvTDfhVh7969Sk5OVkxMjAIDA9WoUSONHz9ep06dKnE9Tzs/c+fOVXR0tAICAhQbG6vNmzeX2H/ZsmVq1qyZAgICdMUVV+iDDz4op0hLNnXqVF111VWqUaOGQkND1bNnT6Wnp5e4zsKFCwudi4CAgHKKuGQTJkwoFFuzZs1KXMdTz02Bov7922w2paSkFNnfk8+PlZGbziI3uRe5yTOvfVUtN1WKvGQsJDc31/z+++9Oj3/9618mJibG5OfnF7tep06dzKBBg5zWy87OLsfIixcVFWUmTZrkFNuxY8dKXGfIkCGmQYMGJi0tzWzZssVcffXV5pprrimniIv34YcfmqSkJPPRRx+Zn376ybz33nsmNDTUjBw5ssT1POn8LF261Pj5+ZmXX37ZbN++3QwaNMiEhISYzMzMIvuvW7fOeHt7mxkzZpgffvjBPPHEE8bX19d899135Rx5YQkJCSY1NdV8//33Ztu2baZbt24mMjKyxL+v1NRUExwc7HQuMjIyyjHq4o0fP960bNnSKbY//vij2P6efG4KHDx40Ol4PvnkEyPJrFq1qsj+nnx+rIzcdBa5yX3ITZ577atquaky5CVLFT/nO3XqlKlTp46ZNGlSif06depkhg0bVj5BlVFUVJR57rnnSt0/KyvL+Pr6mmXLljnaduzYYSSZDRs2uCHCSzNjxgwTExNTYh9POj8dO3Y0KSkpjtd5eXkmIiLCTJ06tcj+ffr0Md27d3dqi42NNffdd59b47wYBw8eNJLMmjVriu2Tmppq7HZ7+QVVBuPHjzetW7cudf/KdG4KDBs2zDRq1KjY/2H25PODv5GbziI3uQ65yXOvfVU9N3liXrLUbW/n+9///qc///yzVL/evXjxYtWuXVutWrXSmDFjdOLEiXKIsHSmTZumWrVqqW3btnr66ad15syZYvtu3bpVp0+fVnx8vKOtWbNmioyM1IYNG8oj3DLJzs5WzZo1L9jPE87PqVOntHXrVqf31svLS/Hx8cW+txs2bHDqL0kJCQkeey4kXfB8HDt2TFFRUWrQoIF69Oih7du3l0d4pfLjjz8qIiJCDRs2VP/+/bVv375i+1amcyOd/ftbtGiRBg4cKJvNVmw/Tz4/OIvcdBa5yTXITWd58rWvquYmT81LPm7duod76aWXlJCQoPr165fY76677lJUVJQiIiL07bff6rHHHlN6errefvvtcoq0eA899JDatWunmjVrav369RozZox+//13zZw5s8j+GRkZ8vPzU0hIiFN7WFiYMjIyyiHi0tu9e7fmzJmjZ555psR+nnJ+Dh06pLy8vEK/Bh8WFqadO3cWuU5GRkaR/T3tXOTn52v48OG69tpr1apVq2L7NW3aVC+//LKuvPJKZWdn65lnntE111yj7du3X/DfmbvFxsZq4cKFatq0qX7//XdNnDhR119/vb7//nvVqFGjUP/Kcm4KvPvuu8rKylJSUlKxfTz5/OBv5Ka/eeK/OXKT5yA3neWJ50by4LxUruNMbvLYY48ZSSU+duzY4bTO/v37jZeXl3nzzTfLvL+0tDQjyezevdtVh+DkYo6nwEsvvWR8fHzMyZMni1y+ePFi4+fnV6j9qquuMo8++qhLj6PAxRzPgQMHTKNGjUxycnKZ9+fu81OcX3/91Ugy69evd2ofNWqU6dixY5Hr+Pr6miVLlji1zZ0714SGhrotzosxZMgQExUVZfbv31+m9U6dOmUaNWpknnjiCTdFdvGOHDligoODzYsvvljk8spybgrcfPPN5tZbby3TOp58fqoCctPfyE3kJncgN53liefGGM/NS1Vi5GfkyJElVpWS1LBhQ6fXqampqlWrlv75z3+WeX+xsbGSzn7606hRozKvfyEXczwFYmNjdebMGe3du1dNmzYttDw8PFynTp1SVlaW0ydsmZmZCg8Pv5Swi1XW4/ntt9/0j3/8Q9dcc40WLFhQ5v25+/wUp3bt2vL29i40O1FJ7214eHiZ+leEoUOHavny5Vq7dm2ZP4Xx9fVV27ZttXv3bjdFd/FCQkJ0+eWXFxtbZTg3BX755Rd9+umnZf5E2ZPPT1VAbvobuYnc5Grkpr952rmRPDwvua2s8mD5+fkmJibmgjO1FOeLL74wksw333zj4sgu3aJFi4yXl5c5fPhwkcsLvlR67qeKO3fu9JgvlR44cMA0adLE9O3b15w5c+aitlGR56djx45m6NChjtd5eXmmXr16JX6p9PxPReLi4jzii4v5+fkmJSXFREREmF27dl3UNs6cOWOaNm1qHn74YRdHd+mOHj1qLrvsMvPvf/+7yOWefG7ON378eBMeHm5Onz5dpvU8+fxYEbmJ3OQu5CZnnnztqyq5yZPzkiWLn08//bTY4fkDBw6Ypk2bmk2bNhljjNm9e7eZNGmS2bJli9mzZ4957733TMOGDc0NN9xQ3mEXsn79evPcc8+Zbdu2mZ9++sksWrTI1KlTx9x7772OPucfjzFnh4kjIyPNZ599ZrZs2WLi4uJMXFxcRRyCkwMHDpjGjRubG2+80Rw4cMBp2sNz+3jy+Vm6dKnx9/c3CxcuND/88IMZPHiwCQkJcUzbeM8995jRo0c7+q9bt874+PiYZ555xuzYscOMHz/eY6asvP/++43dbjerV692OhcnTpxw9Dn/eCZOnOiYDnbr1q2mb9++JiAgwGzfvr0iDsHJyJEjzerVq82ePXvMunXrTHx8vKldu7Y5ePCgMaZynZtz5eXlmcjISPPYY48VWlaZzg/ITeQm9yE3ee61ryrmJk/PS5Ysfvr161fsbwfs2bPHaT7yffv2mRtuuMHUrFnT+Pv7m8aNG5tRo0Z5xG8pbN261cTGxhq73W4CAgJM8+bNzZQpU5zuqT7/eIwx5q+//jIPPPCAueyyy0y1atXM7bff7nQRryipqanF3nddoDKcnzlz5pjIyEjj5+dnOnbsaDZu3OhY1qlTJ5OYmOjU/4033jCXX3658fPzMy1btjQrVqwo54iLVty5SE1NdfQ5/3iGDx/uOPawsDDTrVs389VXX5V/8EW48847Td26dY2fn5+pV6+eufPOO53uva9M5+ZcH330kZFk0tPTCy2rTOcH5CZyk3uRmzzz2lcVc5On5yWbMca476Y6AAAAAPAMlv6dHwAAAADWQfEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxA5wnIyNDw4YNU+PGjRUQEKCwsDBde+21ev7553XixAlHv+joaNlsNm3cuNFp/eHDh6tz586O1xMmTJDNZtOQIUOc+m3btk02m0179+4tMZ7du3drwIABql+/vvz9/RUTE6N+/fppy5YtyszMlK+vr5YuXVrkusnJyWrXrl3Z3gAAgEchLwGuQ/EDnOPnn39W27Zt9fHHH2vKlCn6+uuvtWHDBj366KNavny5Pv30U6f+AQEBeuyxxy643YCAAL300kv68ccfyxTPli1b1L59e+3atUsvvPCCfvjhB73zzjtq1qyZRo4cqbCwMHXv3l0vv/xyoXWPHz+uN954Q8nJyWXaJwDAc5CXANfyqegAAE/ywAMPyMfHR1u2bFH16tUd7Q0bNlSPHj1kjHHqP3jwYM2fP18ffPCBunXrVux2mzZtqtDQUD3++ON64403ShWLMUZJSUlq0qSJPv/8c3l5/f1ZRZs2bTRs2DBJZz9F69mzp/bt26fIyEhHn2XLlunMmTPq379/qfYHAPA85CXAtRj5Af7Pn3/+qY8//lgpKSlOCeZcNpvN6XVMTIyGDBmiMWPGKD8/v8TtT5s2TW+99Za2bNlSqni2bdum7du3a+TIkU4JpkBISIgkqVu3bgoLC9PChQudlqempqpXr16OfgCAyoW8BLgexQ/wf3bv3i1jjJo2berUXrt2bQUFBSkoKKjIWwmeeOIJ7dmzR4sXLy5x++3atVOfPn1KdTuCJMetCM2aNSuxn7e3txITE7Vw4ULHJ4A//fSTPv/8cw0cOLBU+wIAeB7yEuB6FD/ABWzevFnbtm1Ty5YtlZubW2h5nTp19Mgjj2jcuHE6depUidt66qmn9Pnnn+vjjz++4H7Pv5WhJAMHDtSePXu0atUqSWc/XYuOjlaXLl1KvQ0AQOVAXgIuHsUP8H8aN24sm82m9PR0p/aGDRuqcePGCgwMLHbdESNG6K+//tK8efNK3EejRo00aNAgjR49+oJJ5PLLL5ck7dy584KxN2nSRNdff71SU1OVn5+vV199VQMGDCh0OwQAoPIgLwGuR/ED/J9atWrppptu0n/+8x8dP368TOsGBQVp7Nixmjx5so4ePVpi33HjxmnXrl3FTgNaoE2bNmrRooWeffbZIu/bzsrKcnqdnJyst956S2+99ZZ+/fVXJSUllekYAACehbwEuB7FD3COefPm6cyZM+rQoYNef/117dixQ+np6Vq0aJF27twpb2/vYtcdPHiw7Ha7lixZUuI+wsLCNGLECM2ePbvEfjabTampqdq1a5euv/56ffDBB/r555/17bffavLkyerRo4dT/zvuuEO+vr667777dPPNN6tBgwalP3AAgEciLwGuRfEDnKNRo0b6+uuvFR8frzFjxqh169bq0KGD5syZo0ceeURPPvlksev6+vrqySef1MmTJy+4n0ceeURBQUEX7NexY0dt2bJFjRs31qBBg9S8eXP985//1Pbt2zVr1iynvtWqVVPfvn115MgRvlAKAFUEeQlwLZspy7fXAAAAAKCSYuQHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACW8P8BKOtdXbm0f8wAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, axs = plt.subplots(1,2, figsize=(10,3))\n", "\n", "ax = axs[0]\n", "out_graph = model(dataset.get_graph_inputs())\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "ax.set_title('From Dataset')\n", "ax.set_xlabel('GNN CV')\n", "ax.set_ylim(0,100)\n", "\n", "ax = axs[1]\n", "out_graph = model(datamodule.get_graph_inputs(\"train\"))\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "out_graph = model(datamodule.get_graph_inputs(\"valid\"))\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "\n", "ax.set_title('From Datamodule')\n", "ax.set_xlabel('GNN CV')\n", "ax.set_ylim(0,100)\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "648ac69a", "metadata": {}, "source": [ "### Export model for inference\n", "\n", "The exported module can then be used in PLUMED using the GNN-based C++ interfaces provided in the `plumed_interfaces` folder of the library." ] }, { "cell_type": "code", "execution_count": 19, "id": "03766824", "metadata": {}, "outputs": [], "source": [ "traced_model = model.to_torchscript('model.pt', method='trace')" ] }, { "cell_type": "markdown", "id": "9d317b3e", "metadata": {}, "source": [ "## Long-range interactions\n", "\n", "When the *truncated_graph* strategy is applied, for certain process, it may happen that, rather having a single and continuous graph in which all the `system_atoms` are included, one has separated graphs built around subsets of the `system_atoms`.\n", "For instance, if one considers the dissociation of a ion pair in a solvent, in the bound state the ions are close to each other and will thus be part of the same graphs, whereas in the unbound state they are likely to be far from each beyond the cutoff+buffer radius, thus being confined to disconnected graphs\n", "\n", "To overcome this possible limitation, it is useful to allow `system_atoms` to interact with each other according to a `long_range_cutoff`, larger than the normal cutoff, which is used exclusively used to draw edges between the `subsystem_atoms`, a subset of the `system_atoms`." ] }, { "cell_type": "markdown", "id": "25f9109f", "metadata": {}, "source": [ "### Load data\n", "To create/handle these advanced graphs it is **highly recommended** to use the util function `create_dataset_from_trajectories`.\n", "There, besides the keys `system_selection`, `environment_selection` and `buffer` used for the truncated graph definition, the `long_range_cutoff` key can be used to define the long range cutoff taht will be applied to draw the edges between the atoms selected with the `subsystem_selection` keyword. \n", "\n", "Note that `subsystem_selection` must be a subset of the `system_selection`." ] }, { "cell_type": "code", "execution_count": 10, "id": "b6eac48e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Dataset info:\n", " DictDataset( \"data_list\": 1000,\n", "\t metadata={\"atomic_numbers\": [8, 11, 17],\n", "\t\t \"cutoff\": 4.0,\n", "\t\t \"buffer\": 3.0,\n", "\t\t \"long_range_cutoff\": 10.0,\n", "\t\t \"used_idx\": tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,\n", " 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,\n", " 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41,\n", " 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,\n", " 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69,\n", " 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83,\n", " 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,\n", " 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,\n", " 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,\n", " 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,\n", " 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,\n", " 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,\n", " 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n", " 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,\n", " 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,\n", " 210, 211, 212, 213, 214, 215, 216, 217]),\n", "\t\t \"used_names\": [MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-Na, MOL1-Cl],\n", "\t\t \"data_type\": graphs } )\n", "\n", "Datamodule info:\n", " DictModule(dataset -> DictDataset( \"data_list\": 1000,\n", "\t metadata={\"atomic_numbers\": [8, 11, 17],\n", "\t\t \"cutoff\": 4.0,\n", "\t\t \"buffer\": 3.0,\n", "\t\t \"long_range_cutoff\": 10.0,\n", "\t\t \"used_idx\": tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,\n", " 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,\n", " 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41,\n", " 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,\n", " 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69,\n", " 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83,\n", " 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,\n", " 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,\n", " 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,\n", " 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,\n", " 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,\n", " 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,\n", " 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n", " 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,\n", " 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,\n", " 210, 211, 212, 213, 214, 215, 216, 217]),\n", "\t\t \"used_names\": [MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-O, MOL1-Na, MOL1-Cl],\n", "\t\t \"data_type\": graphs } ),\n", "\t train_loader -> DictLoader(length=0.8, batch_size=1000, shuffle=True),\n", "\t valid_loader -> DictLoader(length=0.2, batch_size=1000, shuffle=True))\n" ] } ], "source": [ "from mlcolvar.data import DictModule\n", "from mlcolvar.io import create_dataset_from_trajectories\n", "\n", "# loading arguments \n", "# same as to laod_dataframe\n", "load_args = [{'start' : 0, 'stop' : 500, 'stride' : 1},\n", " {'start' : 0, 'stop' : 500, 'stride' : 1}\n", " ]\n", "\n", "# create dataset\n", "dataset = create_dataset_from_trajectories(\n", " trajectories=[\"https://github.com/EnricoTrizio/nacl_gnn_data/raw/refs/heads/main/UNBOUND/traj.xyz\", \n", " \"https://github.com/EnricoTrizio/nacl_gnn_data/raw/refs/heads/main/BOUND/traj.xyz\"],\n", " topologies=None, \n", " cutoff=4.0, # Angstrom \n", " buffer=3.0, # Angstrom\n", " system_selection='type Na or type Cl',\n", " subsystem_selection='type Na or type Cl',\n", " long_range_cutoff=10.0, # Angstrom \n", " environment_selection='type O',\n", " load_args=load_args,\n", " )\n", "print('Dataset info:\\n', dataset, end=\"\\n\\n\")\n", "\n", "# load dataset into a DictModule\n", "datamodule = DictModule(dataset=dataset)\n", "print('Datamodule info:\\n', datamodule)" ] }, { "cell_type": "markdown", "id": "391c8022", "metadata": {}, "source": [ "### Visualize the truncated graph\n", "The generated dataset can be saved as an `extxyz` using the `save_dataset_configurations_as_extxyz` utils.\n", "This way, one can visualize which atoms are included in the graph, for example by comparing the selected atoms with the original complete system with a visualization software that supports dynamics extxyz files." ] }, { "cell_type": "code", "execution_count": 11, "id": "e58613ac", "metadata": {}, "outputs": [], "source": [ "from mlcolvar.data.utils import save_dataset_configurations_as_extyz\n", "save_dataset_configurations_as_extyz(dataset, 'test.xyz')" ] }, { "cell_type": "markdown", "id": "fff2e8fe", "metadata": {}, "source": [ "### GNN-model initialization\n", "\n", "The GNN model can be initialized from the generated dataset using the `dataset_for_initialization` to make it simpler to inherit all the parameters used in the construction of the graphs." ] }, { "cell_type": "code", "execution_count": 12, "id": "44fd91ec", "metadata": {}, "outputs": [], "source": [ "from mlcolvar.core.nn.graph.schnet import SchNetModel\n", "\n", "gnn_model = SchNetModel(n_out=1,\n", " dataset_for_initialization=dataset,\n", " pooling_operation=\"mean\",\n", " n_bases=16,\n", " n_layers=2,\n", " n_filters=16,\n", " n_hidden_channels=16,\n", " w_out_after_pool=True,\n", " aggr='mean'\n", " )" ] }, { "cell_type": "markdown", "id": "2f21503a", "metadata": {}, "source": [ "### CV model initialization" ] }, { "cell_type": "code", "execution_count": 13, "id": "f5c7ad8c", "metadata": {}, "outputs": [], "source": [ "import torch\n", "from mlcolvar.cvs import DeepTDA\n", "\n", "# we can still set the options for the optimizer the usual way\n", "# options for the BLOCKS of the cv are disabled when passing an external model\n", "options = {'optimizer' : {'lr' : 1e-3},\n", " 'lr_scheduler': {\n", " 'scheduler': torch.optim.lr_scheduler.ExponentialLR,\n", " 'gamma': 0.9999}\n", " }\n", "\n", "model = DeepTDA(n_states=2,\n", " n_cvs=1,\n", " target_centers=[-7, 7],\n", " target_sigmas=[0.2, 0.2],\n", " model=gnn_model)" ] }, { "cell_type": "markdown", "id": "e6772954", "metadata": {}, "source": [ "### Model training" ] }, { "cell_type": "code", "execution_count": null, "id": "3ebfb141", "metadata": {}, "outputs": [], "source": [ "from lightning import Trainer\n", "from mlcolvar.utils.trainer import MetricsCallback\n", "from mlcolvar.utils.plot import plot_metrics\n", "import matplotlib.pyplot as plt\n", "\n", "# define callbacks\n", "metrics = MetricsCallback()\n", "\n", "# here the number of epochs is low for testing, you should increase it for applications\n", "trainer = Trainer(\n", " callbacks=[metrics],\n", " logger=False,\n", " enable_checkpointing=False,\n", " max_epochs=5,\n", " enable_model_summary=False\n", ")\n", "\n", "trainer.fit(model, datamodule)" ] }, { "cell_type": "markdown", "id": "0ac6fe36", "metadata": {}, "source": [ "#### Check training metrics" ] }, { "cell_type": "code", "execution_count": 15, "id": "20fd6e98", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAE8CAYAAAAmDQ2PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIlklEQVR4nO3deVxVdf7H8de5F7jsu2yKgKi4kyuhZpooaplrY+lMWo2OuUy2zS+nUsvKsmXMnGzK0popTU3Lyi33JcQV93BDwQVckF3We35/XLh1FRUQOBfv5/l4nAdwNt7nivdzz/me8/0qqqqqCCGEsEk6rQMIIYTQjhQBIYSwYVIEhBDChkkREEIIGyZFQAghbJgUASGEsGFSBIQQwoZJERBCCBsmRUAIIWyYFAEhKiA0NJRRo0ZpHUOIaidFQNSaBQsWoCgKu3fv1jqKEKKUndYBhKgLEhMT0enkM5O4+8hftbA5xcXFFBYWVmobg8GAvb19DSXSVm5urtYRhIakCAirc+7cOZ588kn8/f0xGAy0bNmSL774wmKdwsJCpkyZQvv27fHw8MDFxYX77ruPjRs3Wqx3+vRpFEXhvffeY9asWYSHh2MwGDhy5AjTpk1DURROnDjBqFGj8PT0xMPDgyeeeIK8vDyL/VzfJlB2aWv79u0899xz1KtXDxcXFwYNGsSlS5cstjUajUybNo2goCCcnZ3p0aMHR44cqXA7g9Fo5MMPP6R169Y4OjpSr149+vTpY76sVnaMCxYsuGFbRVGYNm2a+eeyYz5y5AjDhw/Hy8uLrl278t5776EoCmfOnLlhH5MnT8bBwYGrV6+a58XHx9OnTx88PDxwdnbm/vvvZ/v27RbbZWdnM2nSJEJDQzEYDPj5+dGrVy/27t1722MWtUcuBwmrkpaWxr333ouiKEyYMIF69eqxatUqnnrqKbKyspg0aRIAWVlZzJs3j8cee4zRo0eTnZ3N559/TmxsLDt37uSee+6x2O/8+fPJz89nzJgxGAwGvL29zcv+9Kc/ERYWxowZM9i7dy/z5s3Dz8+Pd95557Z5J06ciJeXF1OnTuX06dPMmjWLCRMm8O2335rXmTx5MjNnzqR///7Exsayf/9+YmNjyc/Pr9Br8tRTT7FgwQL69u3LX//6V4qLi9m6dSs7duygQ4cOFdrH9R555BGaNGnCW2+9haqqPPTQQ/zjH/9g8eLFvPjiixbrLl68mN69e+Pl5QXAhg0b6Nu3L+3bt2fq1KnodDrmz5/PAw88wNatW+nUqRMAY8eOZenSpUyYMIEWLVpw5coVtm3bxtGjR2nXrl2VcosaoApRS+bPn68C6q5du266zlNPPaUGBgaqly9ftpj/6KOPqh4eHmpeXp6qqqpaXFysFhQUWKxz9epV1d/fX33yySfN85KSklRAdXd3Vy9evGix/tSpU1XAYn1VVdVBgwapPj4+FvNCQkLUkSNH3nAsMTExqtFoNM9/9tlnVb1er2ZkZKiqqqqpqamqnZ2dOnDgQIv9TZs2TQUs9lmeDRs2qID697///YZlZb+37Bjnz59/wzqAOnXq1BuO+bHHHrth3ejoaLV9+/YW83bu3KkC6ldffWX+nU2aNFFjY2MtjjsvL08NCwtTe/XqZZ7n4eGhjh8//pbHJ7Qnl4OE1VBVle+++47+/fujqiqXL182T7GxsWRmZpovJej1ehwcHADT5ZL09HSKi4vp0KFDuZcbhgwZQr169cr9vWPHjrX4+b777uPKlStkZWXdNvOYMWNQFMVi25KSEvNllfXr11NcXMy4ceMstps4ceJt9w3w3XffoSgKU6dOvWHZH39vZV1/zADDhg1jz549nDx50jzv22+/xWAwMGDAAAASEhI4fvw4w4cP58qVK+Z/n9zcXHr27MmWLVswGo0AeHp6Eh8fz/nz56ucU9Q8KQLCaly6dImMjAw+/fRT6tWrZzE98cQTAFy8eNG8/pdffkmbNm1wdHTEx8eHevXq8fPPP5OZmXnDvsPCwm76exs2bGjxc9lljz9eA6/qtmXFoHHjxhbreXt7m9e9lZMnTxIUFGRx+ao6lPd6PPLII+h0OvOlLFVVWbJkCX379sXd3R2A48ePAzBy5Mgb/o3mzZtHQUGB+fWfOXMmhw4dIjg4mE6dOjFt2jROnTpVrcch7py0CQirUfYJ8s9//jMjR44sd502bdoA8L///Y9Ro0YxcOBAXnzxRfz8/NDr9cyYMcPik2wZJyenm/5evV5f7ny1AiOv3sm21eVmZwQlJSU33aa81yMoKIj77ruPxYsX889//pMdO3aQnJxs0TZS9m/07rvv3tDuUsbV1RUwtbXcd999LF++nLVr1/Luu+/yzjvvsGzZMvr27VvRwxM1TIqAsBr16tXDzc2NkpISYmJibrnu0qVLadSoEcuWLbN4EyzvsomWQkJCADhx4oTFp+8rV65U6EwjPDycNWvWkJ6eftOzgbIzioyMDIv55d3pczvDhg1j3LhxJCYm8u233+Ls7Ez//v0t8gC4u7vf9t8IIDAwkHHjxjFu3DguXrxIu3btePPNN6UIWBG5HCSshl6vZ8iQIXz33XccOnTohuV/vPWy7BP4Hz9xx8fHExcXV/NBK6Fnz57Y2dkxd+5ci/lz5syp0PZDhgxBVVVee+21G5aVHbu7uzu+vr5s2bLFYvnHH39c6bxDhgxBr9ezcOFClixZwkMPPYSLi4t5efv27QkPD+e9994jJyfnhu3L/o1KSkpuuCzn5+dHUFAQBQUFlc4lao6cCYha98UXX7B69eob5j/zzDO8/fbbbNy4kaioKEaPHk2LFi1IT09n7969rFu3jvT0dAAeeughli1bxqBBg3jwwQdJSkrik08+oUWLFuW+OWnF39+fZ555hvfff5+HH36YPn36sH//flatWoWvr+9tG3d79OjBX/7yF2bPns3x48fp06cPRqORrVu30qNHDyZMmADAX//6V95++23++te/0qFDB7Zs2cKxY8cqndfPz48ePXrwwQcfkJ2dzbBhwyyW63Q65s2bR9++fWnZsiVPPPEE9evX59y5c2zcuBF3d3d+/PFHsrOzadCgAUOHDiUyMhJXV1fWrVvHrl27eP/99yudS9QcKQKi1l3/qbjMqFGjaNCgATt37uT1119n2bJlfPzxx/j4+NCyZUuLa9OjRo0iNTWV//znP6xZs4YWLVrwv//9jyVLlrBp06ZaOpKKeeedd3B2duazzz5j3bp1REdHs3btWrp27Yqjo+Ntt58/fz5t2rTh888/58UXX8TDw4MOHTrQuXNn8zpTpkzh0qVLLF26lMWLF9O3b19WrVqFn59fpfMOGzaMdevW4ebmRr9+/W5Y3r17d+Li4pg+fTpz5swhJyeHgIAAoqKi+Nvf/gaAs7Mz48aNY+3atSxbtgyj0Ujjxo35+OOPefrppyudSdQcRa3NFiwhBGC6fu/l5cUbb7zByy+/rHUcYcOkTUCIGnbt2rUb5s2aNQswfaoWQktyOUiIGvbtt9+yYMEC+vXrh6urK9u2bWPhwoX07t2bLl26aB1P2DgpAkLUsDZt2mBnZ8fMmTPJysoyNxa/8cYbWkcTQtoEhBDClkmbgBBC2DApAkIIYcOkTQBTfyjnz5/Hzc3tjnpmFEIIa6GqKtnZ2QQFBd1yaFQpAsD58+cJDg7WOoYQQlS7lJQUGjRocNPlUgQANzc3wPRilXWZK4QQdVlWVhbBwcHm97ebkSLA713xuru7SxEQQtxVbneJWxqGhRDChkkREEIIGyZFQAghbJgUASGEsGFSBIQQwoZJERBCCBsmt4hWUeLRbTgcWEARdhQrdpTo7ClR7CnR2aPq7FH19qh6A+gdwM4BRe+Izs4Bnb0jejtH9PaO2Nk74mBwws7eCQdHd9xcvXAyOMpTy0KIWiNFoIoK8q4SoWT9PsNY+rXkzvabbbQjGwfyFAP5OkcK9M7kO3iiOPticPfH1SsIb99QfN29pFgIIe6YFIEqahjeiUQHJ0qK8ykpKqCkKB+1pABjUQGUFEJJIUpJIYqxEJ2xyDzZqcXYqcXYl04OlGBQSnBVigFw0xXjRjGQBypQXDrlAZdNv9uownmjExf1nuQ5+mP0aoR3cFuaBjfGXq/X5gURQtRJMp4ApserPTw8yMzM1OyJYbWkmGt5GWRnp3Mt9yrX8tIpysvEeC0DXf4V7POv4lKchZeag1tpwbje+RJHku38yfFqTkDTbrQIDkcnZwtC2KSKvq9JEcA6ikBlFOVd5eKFRDLTjmPMSMYzN4VAYzr6P7zfG1U4XOJJskcbglv3pXVwI7l8JGxeaGgokyZNYtKkSXe8r02bNtGjRw+uXr2Kp6fnHe+vulX0fU0uB9VB9s5e1A+/l/rh95rnqYV5XDiTQGbKHtzTjxBkvEJruwxa526hIG4rW7cHkRXSi+4dYnB1cNAwvRCV0717d+655x5mzZp1x/vatWsXLi4udx7qLiJF4C6hODgT2KQzgU06A1CUfZFzR9bieHYbASXpdNOfg7ML2JW0gnP1Y+ge9RCeTo4apxbizqmqSklJCXZ2t387q1evXi0kqlvkOYG7lL2bH6FRfyZg8FyudZ/OGY82lKgKHe3TGXhxMQe+e54VuzZSbDTefmfirqSqKteKijSZKnoVetSoUWzevJkPP/wQRVFQFIUFCxagKAqrVq2iffv2GAwGtm3bxsmTJxkwYAD+/v64urrSsWNH1q1bZ7G/0NBQizMKRVGYN28egwYNwtnZmSZNmrBixYoqv6bfffcdLVu2xGAwEBoayvvvv2+x/OOPP6ZJkyY4Ojri7+/P0KFDzcuWLl1K69atcXJywsfHh5iYGHJzc6ucpaLkTOBupyg4+UUQEvsKJdmppOz6mqBLO+nmcImiU5/w/YkNhHcdTdvghlonFbUsv7iY6E8+1eR3x40dg5O9/W3X+/DDDzl27BitWrXi9ddfB+Dw4cMAvPTSS7z33ns0atQILy8vUlJS6NevH2+++SYGg4GvvvqK/v37k5iYSMOGN//7fu2115g5cybvvvsuH330ESNGjODMmTN4e3tX6pj27NnDn/70J6ZNm8awYcP49ddfGTduHD4+PowaNYrdu3fz97//nf/+97907tyZ9PR0tm7dCsCFCxd47LHHmDlzJoMGDSI7O5utW7dWuFjeCU3PBLZs2UL//v0JCgpCURS+//57i+WqqjJlyhQCAwNxcnIiJiaG48ePW6yTnp7OiBEjcHd3x9PTk6eeeoqcnJxaPIq6Q+8WQPADz0Psu6S6hGOvqAzVJ+K4ZRr/2/IzRSV3+JCDENXMw8MDBwcHnJ2dCQgIICAgAH3pbdCvv/46vXr1Ijw8HG9vbyIjI/nb3/5Gq1ataNKkCdOnTyc8PPy2n+xHjRrFY489RuPGjXnrrbfIyclh586dlc76wQcf0LNnT1599VWaNm3KqFGjmDBhAu+++y4AycnJuLi48NBDDxESEkLbtm35+9//DpiKQHFxMYMHDyY0NJTWrVszbtw4XF1dK52jsjQ9E8jNzSUyMpInn3ySwYMH37B85syZzJ49my+//JKwsDBeffVVYmNjOXLkCI6OpuvZI0aM4MKFC/zyyy8UFRXxxBNPMGbMGL755pvaPpw6Q+/ZkIB+b5GbtA1lz6c0t88h7MJ/WbD0EH37jKWBh4fWEUUtcLSzI27sGM1+953q0KGDxc85OTlMmzaNn3/+2fymeu3aNZKTk2+5nzZt2pi/d3Fxwd3dnYsXL1Y6z9GjRxkwYIDFvC5dujBr1ixKSkro1asXISEhNGrUiD59+tCnTx/zZajIyEh69uxJ69atiY2NpXfv3gwdOhQvL69K56gsTc8E+vbtyxtvvMGgQYNuWKaqKrNmzeKVV15hwIABtGnThq+++orz58+bzxiOHj3K6tWrmTdvHlFRUXTt2pWPPvqIRYsWcf78+Vo+mjpGUXBpdB/OD33IZbcmOCpGRit7iFvxBnvPntU6nagFiqLgZG+vyVQdtytff5fPCy+8wPLly3nrrbfYunUrCQkJtG7dmsLCwlvux/66y1KKomCsgbYyNzc39u7dy8KFCwkMDGTKlClERkaSkZGBXq/nl19+YdWqVbRo0YKPPvqIiIgIkpKSqj3H9ay2YTgpKYnU1FRiYmLM8zw8PIiKiiIuLg6AuLg4PD09LT4RxMTEoNPpiI+Pv+m+CwoKyMrKsphslpM3vn3eILvJQAAeMZzh2sY3WH14v7a5hCjl4OBASQUuVW7fvp1Ro0YxaNAgWrduTUBAAKdPn675gKWaN2/O9u3bb8jUtGlT8yUsOzs7YmJimDlzJgcOHOD06dNs2LABMBWfLl268Nprr7Fv3z4cHBxYvnx5jee22obh1NRUAPz9/S3m+/v7m5elpqbi5+dnsdzOzg5vb2/zOuWZMWMGr732WjUnrsMUBbe2wynwaoiy8990MaRzYN9svsl7iuEdO2udTti40NBQ4uPjOX36NK6urjf9lN6kSROWLVtG//79URSFV199tUY+0d/M888/T8eOHZk+fTrDhg0jLi6OOXPm8PHHHwPw008/cerUKbp164aXlxcrV67EaDQSERFBfHw869evp3fv3vj5+REfH8+lS5do3rx5jee22jOBmjR58mQyMzPNU0pKitaRrIIhtCt2PV/nmuJIG4dsIo99ztc7f9U6lrBxL7zwAnq9nhYtWlCvXr2bXuP/4IMP8PLyonPnzvTv35/Y2FjatWtXaznbtWvH4sWLWbRoEa1atWLKlCm8/vrrjBo1CgBPT0+WLVvGAw88QPPmzfnkk09YuHAhLVu2xN3dnS1bttCvXz+aNm3KK6+8wvvvv0/fvn1rPLfVdBuhKArLly9n4MCBAJw6dYrw8HD27dvHPffcY17v/vvv55577uHDDz/kiy++4Pnnn+fq1avm5cXFxTg6OrJkyZJy2xrKU9e6jahxGae5tm4qTsZrHC1yZXfjp/hLVBetUwkhKqGi72tWeyYQFhZGQEAA69evN8/LysoiPj6e6OhoAKKjo8nIyGDPnj3mdTZs2IDRaCQqKqrWM981PENx6jWdazpnmtvn0Pz4Ahbv26t1KiFEDdC0COTk5JCQkEBCQgJgagxOSEggOTkZRVGYNGkSb7zxBitWrODgwYM8/vjjBAUFmc8WmjdvTp8+fRg9ejQ7d+5k+/btTJgwgUcffZSgoCDtDuxu4NEQp5jXKFQc6OCQicfBz9lw4oTWqYSoNWPHjsXV1bXcaezYsVrHqzaaXg4q64XveiNHjmTBggWoqsrUqVP59NNPycjIoGvXrnz88cc0bdrUvG56ejoTJkzgxx9/RKfTMWTIEGbPnl2phyzkctDNqakHMG55Cz1G/pvXkDaxLxAZGKB1LCFq3MWLF29656C7u/sNN6VYG+lKuhKkCNxaSdIm9LtMdzi8mdeaJwY/Q5C8TkJYtTrfJiCshz6sO0URpkb255wO8+GqxRQUlz+wjRCibpEiICrEvvUw8n1a4qQYGVeylQ82rr/9RkIIqydFQFSMTodj12cpcPAkxO4aHS78wLJDh7VOJYS4Q1IERMUZ3DHc9w9K0NHL8TKH4hdzKj1d61RCiDsgRUBUjk9jlNbDAHjW5RgfrF0hXVALUYdJERCVposYQJFnOG66Ev5S9CufxFe+73Uhakt5o4ldP3bJH50+fRpFUczPL93Kpk2bUBSFjIyMO86pFSkCovJ0Ouyj/06JYk+UIYO8oz+y7/wFrVMJUSEXLlyolT556gopAqJq3ALR3/MXACa4JDF7/U9y26ioEwICAjAYDFrHsBpSBETVNe5NiXcTXHQlDFf38cUe6V+oTlFVKM7XZqrgM6qffvopQUFBN3QJPWDAAJ588skKDS5/vesvB+3cuZO2bdvi6OhIhw4d2LdvX6Vfyj+qa4PNW+14AqIOUHToO4zBuPb/6OV4mZ8OrOVM06aEeHlqnUxUREkBLHtcm989+Cuwc7ztao888ggTJ05k48aN9OzZEzB1FbN69WpWrlxJTk5OlQaXL5OTk8NDDz1Er169+N///kdSUhLPPPNMlQ+rLg42L0VA3BnPEJSmD8KxH3nO5Thvb1zPx4MGV8vwgUJ4eXnRt29fvvnmG3MRWLp0Kb6+vvTo0QOdTkdkZKR5/enTp7N8+XJWrFjBhAkTbrv/b775BqPRyOeff46joyMtW7bk7NmzPP3001XK+8fB5gGaNm3KkSNHePfddxk1apTFYPNubm7mAefBcrD5kJAQAFq3bl2lHJUhRUDcMaXlUEpObyKEbELTd7PyWCsejIjQOpa4Hb3B9Ilcq99dQSNGjGD06NF8/PHHGAwGvv76ax599FF0Ol2VB5cvc/ToUdq0aYOj4+9nJWVd1VdFXRxsXtoExJ2zd0Lf+lEA/uZ6hvm/biVfGomtn6KYLsloMVXiTLF///6oqsrPP/9MSkoKW7duZcSIEUDVB5fXijUONi9FQFSPsAcwutXHU1fMQxzhmwQZqF5UD0dHRwYPHszXX3/NwoULiYiIMA8beaeDyzdv3pwDBw6Qn59vnrdjx44qZ62Lg81LERDVQ6dHF2m6ZXSY83mW740j/do1jUOJu8WIESP4+eef+eKLL8xnAfD74PIJCQns37+f4cOHV2pw+eHDh6MoCqNHj+bIkSOsXLmS9957r8o5n3/+edavX8/06dM5duwYX375JXPmzOGFF14ATIPNz549m4SEBM6cOcNXX31lMdj8W2+9xe7du0lOTmbZsmW1Mti8FAFRfQLbono3xkkx8oj9Kebv3nP7bYSogAceeABvb28SExMZPny4ef6dDi7v6urKjz/+yMGDB2nbti0vv/wy77zzTpVz1sXB5mVQGWRQmWqVmgBb3iJf1THkame+evyv+Dg7a51KCJsjg8oIbfhHovo0xVExMsxwmi/33tmDN0KImiVFQFQvRUFpMQSAIU4XWHloH1fy8jQOJUTV2MJg83I5CLkcVO1UFXXNCyhZKXyYHYauxSAmRt+rdSohKq0uDzZf0fc1eVhMVD9FQWk2AHbOYbjzOR47uJ+/dmiPk7291smEqBQ/Pz+rfqOvDnI5SNSMhp1Rnbyppy+ki3KWH44c1TqREKIcUgREzdDZoTTuA8Ajzuf5b8J+Sipx/7YQonZIERA1J+wBVJ0dreyz8cw7y+ak01onEkJcR4qAqDmO7ijBps64HnG+wOKDhzQOJIS4nhQBUbPCewPQx/EiR86e4kwdHotViLuRFAFRs3yagmcIjoqRh51SWSpnA0JYFSkComYpCoTHAvAnpwusOHpUupkWwopIERA1r2FXVDsnGtpdI0JNY+OpU1onEkKUsuoiUFJSwquvvkpYWBhOTk6Eh4czffp0izE3VVVlypQpBAYG4uTkRExMDMePH9cwtbiBvSNKwy4A9HdMY8XR3zQOJIQoY9VF4J133mHu3LnMmTOHo0eP8s477zBz5kw++ugj8zozZ85k9uzZfPLJJ8THx+Pi4kJsbKzFIBHCCoR1ByDG8RIHU5JIzc7WNI4QwsSqi8Cvv/7KgAEDePDBBwkNDWXo0KH07t2bnTt3AqazgFmzZvHKK68wYMAA2rRpw1dffcX58+f5/vvvtQ0vLHk3AbcgnBQjMYZL/PRbotaJhBBYeRHo3Lkz69ev59ixYwDs37+fbdu2mQdZSEpKIjU1lZiYGPM2Hh4eREVFERcXd9P9FhQUkJWVZTGJGqYoENodgIedUlnx229I34VCaM+qi8BLL73Eo48+SrNmzbC3t6dt27ZMmjTJPLxcamoqAP7+/hbb+fv7m5eVZ8aMGXh4eJin4ODgmjsI8buQbqgotHPIQs1OZf8t/o2EELXDqovA4sWL+frrr/nmm2/Yu3cvX375Je+99x5ffvnlHe138uTJZGZmmqeUlJRqSixuydkbJSASgP6OqdJALIQVsOoi8OKLL5rPBlq3bs1f/vIXnn32WWbMmAFAQEAAAGlpaRbbpaWlmZeVx2Aw4O7ubjGJWlJ6Sai/Uxq/HD9OgTwzIISmrLoI5OXlodNZRtTr9RhLe6MMCwsjICCA9evXm5dnZWURHx9PdHR0rWYVFVS/A6q9M4H6AiLUi2w7fUbrRELYNKsuAv379+fNN9/k559/5vTp0yxfvpwPPviAQYMGAaAoCpMmTeKNN95gxYoVHDx4kMcff5ygoCAGDhyobXhRPr2DuVO5/k5prJJnOoTQlFWPLPbRRx/x6quvMm7cOC5evEhQUBB/+9vfmDJlinmdf/zjH+Tm5jJmzBgyMjLo2rUrq1evxtHRUcPk4pZC7odT64kxXOb9pJPkFBbi6uCgdSohbJKMMYyMMVzrVBV15USU3ItMzmxG1/tG8GCzCK1TCXFXqej7mlVfDhJ3KUVBCekGwEOOaaw6JpeEhNCKFAGhjdIicK/DVY6fO0HGNenmQwgtSBEQ2nALAJ8I9Ar0dkhl3ckTWicSwiZJERDaCf39ktDqY1IEhNCCFAGhneBoVMWOCPtcstMSScvJ0TqREDZHioDQjoMrSv32APRzTGPtcTkbEKK2SREQ2gq5H4B+jhdZe0y6lxaitkkRENoKuAejvSv19IW4ZRwjOSND60RC2BQpAkJbejt0IV0B6O+Uyhq5JCRErZIiILRX2rNoT8NltiUelMFmhKhFUgSE9rwbUeIRhoOiElmQyLHLV7ROJITNkCIgrIK+cS8AhjhdYHXiMY3TCGE7pAgI69CwK8U6AyF217hwKg6jXBISolZIERDWwd4RShuIe3GS/Rdk/GEhaoMUAWE17Jo+CEAPw2V2HtmpcRohbIMUAWE9PBqQ7tkCnQJBqZspLh1GVAhRc6QICKviHvkIAL3tz7Hv5BGN0whx95MiIKyKnV8Lztr5Y1BUrh1ernUcIe56UgSEdVEUipsNAiCq4DA5Gec1DiTE3U2KgLA6Ic26c8jog0ExcmnnAq3jCHFXkyIgrI6i03GmoelOoZCM/ZCZrHEiIe5eUgSEVYqK7MG6fF90qFyL/wRUuVNIiJogRUBYJV8XF7Z53k+eUYdTxglI2qh1JCHuSlIEhNXq1rIjH+eGAaDu/x/kZ2gbSIi7kBQBYbXuCw1hlbERR4tcUYpyIeErrSMJcdeRIiCslr1eT79mzZme1RQjCiRvg9QDWscS4q4iRUBYtUEtW3Ck2I1v84JMM/Z+BsWF2oYS4i4iRUBYtUbe3rQLCmROTii5elfISYOjy7SOJcRdQ4qAsHqDW7YkV7XjX3kRphmJP0DWWW1DCXGXsPoicO7cOf785z/j4+ODk5MTrVu3Zvfu3eblqqoyZcoUAgMDcXJyIiYmhuPHj2uYWFS3mMbhuBkMLM1wId2zORhLYM88kIFnhLhjVl0Erl69SpcuXbC3t2fVqlUcOXKE999/Hy8vL/M6M2fOZPbs2XzyySfEx8fj4uJCbGws+fn5GiYX1cnRzo6HIpoCCnMLW4HeAS4dgXO7tI4mRJ2nqKr1fpx66aWX2L59O1u3bi13uaqqBAUF8fzzz/PCCy8AkJmZib+/PwsWLODRRx8td7uCggIKCgrMP2dlZREcHExmZibu7u7VfyDijh2/fIVHFi7CTqdjS2dXnI//CG6BEPs+6Oy0jieE1cnKysLDw+O272tWfSawYsUKOnTowCOPPIKfnx9t27bls88+My9PSkoiNTWVmJgY8zwPDw+ioqKIi4u76X5nzJiBh4eHeQoODq7R4xB3romvD20C/Ck2GllW3BgM7pB9AU6t1zqaEHWaVReBU6dOMXfuXJo0acKaNWt4+umn+fvf/86XX34JQGqqaRxaf39/i+38/f3Ny8ozefJkMjMzzVNKSkrNHYSoNoNbtgTg26MnMLYYapp5eCkUF9xiKyHErVh1ETAajbRr14633nqLtm3bMmbMGEaPHs0nn3xyR/s1GAy4u7tbTML6xTZpjKuDAymZWexxaA4u9aAgU84GhLgDVSoCKSkpnD37+y16O3fuZNKkSXz66afVFgwgMDCQFi1aWMxr3rw5ycmmroUDAgIASEtLs1gnLS3NvEzcPZzs7ekb0RSApUd/g9LBZ/jtByiRB8iEqIoqFYHhw4ezcaOpV8fU1FR69erFzp07efnll3n99derLVyXLl1ITEy0mHfs2DFCQkIACAsLIyAggPXrf/8kmJWVRXx8PNHR0dWWQ1iPIS1NHwo2nDxFun8ncPaB/KvSy6gQVVSlInDo0CE6deoEwOLFi2nVqhW//vorX3/9NQsWLKi2cM8++yw7duzgrbfe4sSJE3zzzTd8+umnjB8/HgBFUZg0aRJvvPEGK1as4ODBgzz++OMEBQUxcODAasshrEezevVo4edHkdHIT8dOQsQA04LfVpieHxBCVEqVikBRUREGgwGAdevW8fDDDwPQrFkzLly4UG3hOnbsyPLly1m4cCGtWrVi+vTpzJo1ixEjRpjX+cc//sHEiRMZM2YMHTt2JCcnh9WrV+Po6FhtOYR1GVx6NrDs8BHUsB5gcIO8S3Bup8bJhKh7qvScQFRUFD169ODBBx+kd+/e7Nixg8jISHbs2MHQoUMt2gvqgoreTyusQ25hITFfzOdaUTGfDx5E+6vb4MhS8GkCPd/UOp4QVqFGnxN45513+M9//kP37t157LHHiIyMBEz39ZddJhKiprg4ONCniamBeNnhw9C4t+mBsSvH4fIxjdMJUbdUqQh0796dy5cvc/nyZb744gvz/DFjxtzx7ZtCVMSQVqZLQr+cOEkmjtCwq2nB8Z81TCVE3VOlInDt2jUKCgrMfficOXOGWbNmkZiYiJ+fX7UGFKI8Lf38aOrrQ2FJCT/9lghN+pkWnN0pw1AKUQlVKgIDBgzgq69MQ/1lZGQQFRXF+++/z8CBA5k7d261BhSiPIqiMKT0CeJlh4+geoaAdxNQSyBpk7bhhKhDqlQE9u7dy3333QfA0qVL8ff358yZM3z11VfMnj27WgMKcTP9IpriaGfHyfR09qemQnhpH1Kn1oFq1DacEHVElYpAXl4ebm5uAKxdu5bBgwej0+m49957OXPmTLUGFOJm3AwGejdpDJjOBgjuDPbOkHsR0mQsYiEqokpFoHHjxnz//fekpKSwZs0aevfuDcDFixflFktRq8qeGVh7/ARZJUBIN9OCk+u0CyVEHVKlIjBlyhReeOEFQkND6dSpk7mLhrVr19K2bdtqDSjErUQGBBDu7U1+cTGrEo9BeC/TgvO74Vq6tuGEqAOqVASGDh1KcnIyu3fvZs2aNeb5PXv25F//+le1hRPidhRFMZ8NfH/kKHgEg28zU5uA9CckxG1VuSvpgIAA2rZty/nz581PCHfq1IlmzZpVWzghKqJvRFPsdDqOXrrEqfR0aPSAacHpLTIOsRC3UaUiYDQaef311/Hw8CAkJISQkBA8PT2ZPn06RqPclSFql7eTE50bNgRgZeIxqH8v2Bkg5wJckSeIhbiVKhWBl19+mTlz5vD222+zb98+9u3bx1tvvcVHH33Eq6++Wt0ZhbitB0vHGVh57BiqnQHqR5kWnN6sYSohrF+VRuj+8ssvmTdvnrn3UIA2bdpQv359xo0bx5tvSideonZ1CwvFxd6e81nZJFxIpW1odzizBVJ+hXtGgZ2D1hGFsEpVOhNIT08v99p/s2bNSE+XOzJE7XOyt+eB8HAAViYmgl8LcPaFojzTnUJCiHJVqQhERkYyZ86cG+bPmTOHNm3a3HEoIaqi7JLQmuMnKDKqvz8zIJeEhLipKl0OmjlzJg8++CDr1q0zPyMQFxdHSkoKK1eurNaAQlRUxwb1qefizKXcPLadOUOP0Pvh6DJIS4BrV8HJS+uIQlidKp0J3H///Rw7doxBgwaRkZFBRkYGgwcP5vDhw/z3v/+t7oxCVIhep6Nv09IG4sRj4BYIPhGm20TPbNU4nRDWqUoji93M/v37adeuHSUldWusVxlZ7O7x26VLPLpoMQ56PeufegK3s1thz6fgHgyx74GiaB1RiFpRoyOLCWGtInx9Cff2prCkhPUnT0JwNOjsISsFriZpHU8IqyNFQNxVFEWhb9MmgKmBGAcXqN/RtPDMFg2TCWGdpAiIu06v0u6ld6acJeNaPoSW3iWUvA2MxRomE8L6VOruoMGDB99yeUZGxp1kEaJahHh60tTXh2OXr7Ap6RQDm0WCwQMKMiF1PwS11zqiEFajUmcCHh4et5xCQkJ4/PHHayqrEBXWq7HpbOCX4ydBp4eQ0oHo5ZkBISxU6kxg/vz5NZVDiGrVu3Fj/r0jnvizZ8nMz8cj5H449jOc3wOFOeDgqnVEIayCtAmIu1KIlydNfHwoNhrZeCoJPEPAoyEYiyBlh9bxhLAaUgTEXatXY1NfQr+cOGF6PqCsG4kzcklIiDJSBMRdq6xdID7lLFn5+aZ2AUWBy4mQk6pxOiGsgxQBcdcK8/aisY83xUYjm5JOg5M3+Jd2cHhanhkQAupYEXj77bdRFIVJkyaZ5+Xn5zN+/Hh8fHxwdXVlyJAhpKWlaRdSWBXzXUInTphmhNxv+npGhp4UAupQEdi1axf/+c9/buiq+tlnn+XHH39kyZIlbN68mfPnz9/2eQZhO2JK2wXiklPIKigwPT1s5wi5F02XhYSwcXWiCOTk5DBixAg+++wzvLx+7w44MzOTzz//nA8++IAHHniA9u3bM3/+fH799Vd27JA7QASEe3vTyNuLYqORrUmnTWMPN7jXtFAaiIWoG0Vg/PjxPPjgg8TExFjM37NnD0VFRRbzmzVrRsOGDYmLi7vp/goKCsjKyrKYxN3rgUaNANiYVNqBXGjpJaGUOCgu1CiVENbB6ovAokWL2Lt3LzNmzLhhWWpqKg4ODnh6elrM9/f3JzX15nd/zJgxw+JJ5+Dg4OqOLaxIj9IisP3MGQqKi6Fe89+HnrwgQ08K22bVRSAlJYVnnnmGr7/+GkdHx2rb7+TJk8nMzDRPKSkp1bZvYX1a+NXDz8WFa0XF7Dx7FhQdhNxnWih3CQkbZ9VFYM+ePVy8eJF27dphZ2eHnZ0dmzdvZvbs2djZ2eHv709hYeENHdelpaUREBBw0/0aDAbc3d0tJnH3UhSF7o3CAExPD8PvD46lJkB+hia5hLAGVl0EevbsycGDB0lISDBPHTp0YMSIEebv7e3tWb9+vXmbxMREkpOTzWMfCwGYi8DmpCSMqgru9cG7MahGSN6ucTohtFOlgeZri5ubG61atbKY5+Ligo+Pj3n+U089xXPPPYe3tzfu7u5MnDiR6Oho7r33Xi0iCyvVsX59XB0cuJJ3jYOpaUQGBpjOBtJPwOlN0KSfDD0pbJJVnwlUxL/+9S8eeughhgwZQrdu3QgICGDZsmVaxxJWxl6vp2tICACbyi4JNexiGnoy4wxcPalhOiG0U60DzddVMtC8bVhz7Dj/t2YtoZ6efP+XEaaZ8R/Bma0Q1gM6Pq1tQCGqkQw0L8R1uoSGYKfTcTojg6T0q6aZ4b1MX5O3Q2GuduGE0IgUAWEzXB0c6NSgAQCbyh4c84kAj2AoKTSdEQhhY6QICJvSw3yr6CnTDEWBRqVnAyd/kU7lhM2RIiBsyv1hpiJwMDWNS7mll39CuoHeAFkp0qmcsDlSBIRN8XN1oZW/HyqwOem0aaaDs+lOIYCTa7SKJoQmpAgIm9O99GzAfKsoQOPepq8pcZB7WYNUQmhDioCwOT3CTR3KxaekkFtY2ouoVyPwa2V6gvj4Sg3TCVG7pAgIm9PIy4uGnh4UGY38mpz8+4KI/qavp9bJ7aLCZkgREDZHURRz99Ib/3hJKOAe0+2ixfmmO4WEsAFSBIRNKrtVdEvSaYpKSkwzFQUiHjZ9n/gjFF3TKJ0QtUeKgLBJrf398XF2IqewkD3nzv++oGFXcAuEwmxpGxA2QYqAsEl6nY5uodeNMQCg00PLR0zfJ/4IhTkapBOi9kgREDbrj08PW/SjGNzZ1DZQlAdHv9cmnBC1RIqAsFlRwQ1wsrfjYm4uRy5e+n2BooPWw03fH18J2Tcfr1qIuk6KgLBZBjs7upSOMWDuS6hMYDsIiARjMez/SoN0QtQOKQLCpvW4fuzhMooC94w0nRWc3w2pBzRIJ0TNkyIgbNp9IaYxBk6mp5OckWG50L0BNO5j+j5hARhLajueEDVOioCwae6OjrSvHwRc15dQmZZDwcENss7C8VW1nE6ImidFQNi8cp8eLuPgCm1KG4kPL4a89FpMJkTNkyIgbF73sFAAEi5cID0v78YVwnqATxNTdxLSSCzuMlIEhM0LcHOjhd91Ywz8kaKDdqNNjcUpv0ojsbirSBEQgt/vElp/8lT5K3iFQuO+pu/3zjONSSzEXUCKgBBAr8bhAOxISSHjWn75K7X6Ezh6QU4q/LaiFtMJUXOkCAgBhHp5EeHrS7HRyPqTJ8tfyd4Z7nnc9P3R5aZiIEQdJ0VAiFJ9mjYBYPXx4zdfKbgz+LUGYxHsmw9/7HNIiDpIioAQpXo3aQzA7rPnuJR7k5HFFAXaPwU6O7iwD87tqsWEQlQ/KQJClKrv7k6bgABUYO3xEzdf0S3o98Fn9s2Hopu0IQhRB0gREOIP+pZeEvrh6FHL7qWv13wwuPjBtStwZEktpROi+kkREOIP+kU0xUGv59jlKxy9dOnmK9o5QNsnTd8f+wku/1Y7AYWoZlZdBGbMmEHHjh1xc3PDz8+PgQMHkpiYaLFOfn4+48ePx8fHB1dXV4YMGUJaWppGiUVd5+HoyAOl3UgsP3L01isHtYOQbqbG4fiPTIPQCFHHWHUR2Lx5M+PHj2fHjh388ssvFBUV0bt3b3L/0Gj37LPP8uOPP7JkyRI2b97M+fPnGTx4sIapRV03qGVzAFYlHuNaUdGtV273JLjUg9xLsOczuVtI1DmKessLn9bl0qVL+Pn5sXnzZrp160ZmZib16tXjm2++YejQoQD89ttvNG/enLi4OO69994K7TcrKwsPDw8yMzNxd3evyUMQdYBRVen/1f84l5XFazEPMKB581tvcPk32DgNVCO0GQHNBtRKTiFupaLva1Z9JnC9zMxMALy9vQHYs2cPRUVFxMTEmNdp1qwZDRs2JC4u7qb7KSgoICsry2ISooxOURjcsgUAXyfsv3UDMYBvM7hnlOn7A9/IbaOiTqkzRcBoNDJp0iS6dOlCq1atAEhNTcXBwQFPT0+Ldf39/UlNvfnTnDNmzMDDw8M8BQcH12R0UQcNbdUSRzs7jl2+QnzK2dtv0DgWGsUAKsT9Cy4k1HREIapFnSkC48eP59ChQyxatOiO9zV58mQyMzPNU0pKSjUkFHcTD0dHBrUwXQb6al/C7TdQFFP7QP1OpnGJt78L53bWbEghqkGdKAITJkzgp59+YuPGjTRo0MA8PyAggMLCQjKuGxYwLS2NgICAm+7PYDDg7u5uMQlxvT/fcw86ReHX5GQSL12+/QY6O4ieVFoIimD7e3B4qamtQAgrZdVFQFVVJkyYwPLly9mwYQNhYWEWy9u3b4+9vT3r1683z0tMTCQ5OZno6OjajivuMvU93M29i86Nr+Cn+rJCUDY28eHFpkbjzApcUhJCA1Z9d9C4ceP45ptv+OGHH4iIiDDP9/DwwMnJCYCnn36alStXsmDBAtzd3Zk4cSIAv/76a4V/j9wdJG4mKf0qQ75ZiFFV+XLoECIDb36GeYNTGyBhPhQXgKKH4Gho0g+8G5kGqrE2qmqaFMU0iTqtou9rVl0ElJv8Ic6fP59Ro0YBpofFnn/+eRYuXEhBQQGxsbF8/PHHt7wcdD0pAuJWXlu/geVHjtI+KIh5gwfe9O+yXLmXYN8XcH7P7/McPcC3ObgFgpM32DmBvRPoDaYnkfV/nAy/f6/TVy54SSEUZJVO2VCQCdcyIL+cqTjf1JYBpt9j7wz2LqZnIFz8TVm9wsCrkWmZsHp3RRGoLVIExK2kZmfz8H+/prCkhFkP9qN7o7Dbb3S99FOm7iXO7zKdGVSFoi8tBnY3TqrR1A5hLIKSYlMBKKni77kdt0DwCgefpqbJs6Epg7AqUgQqQYqAuJ0Pt8cxf+9egtzdWDZiOI52VXzTKymGK7/B1dOQcwHyM02fwouvQXHh72/eJYW/T3dCpwcHdzCUTk6e4Fg2ef3+vb2TaV1Fb/r9RXlQkAO5F02D52SdhatJkFdOf0p6h9Ki0AR8SwuDo+ed5RZ3TIpAJUgRELeTV1jIoK8XkpaTw5iOHRh3b1Tt/GJVhZKiPxSHItNlG4upyNTGoLcHnX3pVwdwcDW9uVfn9f38LLh6EtJPwJXjpqmonLEXXOqBdxPwaAjuDcA9CFwCQC9nDLVFikAlSBEQFfHLiRO8uGoNDno93z46jDBvL60jaU81Qvb50oJwzDRlngXKe1tRTO0hTl6mthCDR2nbg9Pv7SI6e1NBK29SjTdOlH41lrNMNZpymH9WS3++xdey23nL/Xqbbcv7ev3x34xFoS5nPb+WENbj5tuXo6Lva1KWhaigmPBwujRsyPbkZKau38D8IYPQ66zwLp/apOhKP+k3+P1Nqiiv9EzhhKlAZJ+DrHOmy15lDdFXk7RMXffo7StdBCpKioAQFaQoCq8+0J2h3yziQGoq/0vYz8h2bbWOZX3sncG/jWkqo6qldyelw7Wrpq8FWVB0zdQeUnTNVDyMJaWf2kuw/FRfduvqTc4SbpjKWRfl91tzy/uqKKXrVPbrzfZZutz0Ali+Fjd1k/U8G1bgha8aKQJCVEKAmxvPd+3Caxs28u8d8UQFN6BZvXpax7J+ivJ7I7RcRbMqNn4uK0TlDWzRnPvDQiksKeH/Vq8lt/AO7+ARQkNSBISoJEVReK1nT/xdXTmTkcH0jZtu3920EFZKioAQVeDp5Mjbsb3RKwqrjx3n8917br+REFZIioAQVdQ2KJCX7u8GwJwd8aw5flzjREJUnhQBIe7AI61bMTzSdBfMy2vXsf3MGY0TCVE5UgSEuEPPd+1Cr8bhFBuNPPfzKuJlkCJRh0gREOIO6XU63urdi/vDQikoKWHCip/YcPKU1rGEqBApAkJUA3u9npl9YunRKIwio5EXVq3m+yNHtY4lxG1JERCimhjs7Hi3bx8GtmiOUVWZtn4Dn+3aLbePCqsmRUCIamSn0zH1gR6MKu1O4t874nnll3UUlpRonEyI8kkREKKaKYrCpC6deaXH/egVhZ8TjzF6+fek5+VpHU2IG0gREKKGDG3Vin8/3B9XBwf2X0jlz0uWcuLKFa1jCWFBioAQNejehsH895GhBHu4cz4rm5FLvuOXEye0jiWEmRQBIWpYmLcX/33kEdrXDyK3qIgXV63hnc1bpJ1AWAUpAkLUAk8nRz4Z8DBPtGsHwMIDBxm19Du5PCQ0J0VAiFpir9fzTJdoZj/0IB6OBo5cvMSjixbz2a7dFMlZgdCIFAEhalm3sFCWPPYY3cNCKTYa+feOeB77djE7kqW7CVH7ZKB5ZKB5oQ1VVVl9/DjvbN5KRn4+APeHhfJcly6EeHlqG07UeRV9X5MigBQBoa2s/Hw+2bmLxQcPUWw0YqfTMahFc0a1b0d9+XsUVSRFoBKkCAhrkJR+lQ+2b2fraVN31HY6Hf0imvJU+/ZyZiAqTYpAJUgRENZkz7nzfLZrNztKu6RWgM4hDRnaqiX3hYZip5OmPHF7UgQqQYqAsEYHU1OZt3sPm5NOm+f5ubgwoEVzYps0prGPj3bhhNWzuSLw73//m3fffZfU1FQiIyP56KOP6NSpU4W2lSIgrFlyRgbfHT7CiiNHuVragAzQyNuLXo0b06NRGE19fdEpioYphbWxqSLw7bff8vjjj/PJJ58QFRXFrFmzWLJkCYmJifj5+d12eykCoi4oLClhw8lTrEw8RlxyMkVGo3mZl6MjHRrUp1ODBrQO8Cfc2xt7vV7DtEJrNlUEoqKi6NixI3PmzAHAaDQSHBzMxIkTeemll267vRQBUddkFxSwOek0v5w4wa6z58grKrJYbqfT0cjbi6Y+vtT3cCfQzY0gNzf8XF1wNxhwMxikSNzlKvq+ZleLmWpEYWEhe/bsYfLkyeZ5Op2OmJgY4uLiyt2moKCAgoIC889ZWVk1nlOI6uRmMPBQswgeahZBUUkJhy9eZGfKWXafO8fRS5fJLijg2OUrHLt8824pHO3scLKzw06vQ6/osNPp0OtMX+XKknXpGR7O01EVu7xdWXW+CFy+fJmSkhL8/f0t5vv7+/Pbb7+Vu82MGTN47bXXaiOeEDXOXq/nnsBA7gkMZAwdUVWVC9nZJF6+zIkr6VzIzuZCdjap2Tlcys0lp7AQgPziYvKLizVOLyqiTUBAje27zheBqpg8eTLPPfec+eesrCyCg4M1TCRE9VEUhSB3d4Lc3enRqNENy0uMRnILC8kqKCC/uJgSo0qx0UiJaqS4xEjxH9oa6jqVOn+1G4B6Li41tu86XwR8fX3R6/WkpaVZzE9LSyPgJtXTYDBgMBhqI54QVkev0+Hu6Ii7o6PWUYQVqPNPnTg4ONC+fXvWr19vnmc0Glm/fj3R0dEaJhNCCOtX588EAJ577jlGjhxJhw4d6NSpE7NmzSI3N5cnnnhC62hCCGHV7ooiMGzYMC5dusSUKVNITU3lnnvuYfXq1Tc0FgshhLB0VzwncKfkOQEhxN2mou9rdb5NQAghRNVJERBCCBsmRUAIIWzYXdEwfKfKmkWk+wghxN2i7P3sds2+UgSA7OxsAHlqWAhx18nOzsbDw+Omy+XuIEwPl50/fx43NzeUSvScVdbdREpKSp27q0iya0Oya6MuZ4eq5VdVlezsbIKCgtDdYjQ6ORPA1OtogwYNqry9u7t7nfzDAsmuFcmujbqcHSqf/1ZnAGWkYVgIIWyYFAEhhLBhUgTugMFgYOrUqXWyR1LJrg3Jro26nB1qNr80DAshhA2TMwEhhLBhUgSEEMKGSREQQggbJkVACCFsmBSBKvr3v/9NaGgojo6OREVFsXPnTq0j3WDatGkoimIxNWvWzLw8Pz+f8ePH4+Pjg6urK0OGDLlhrObasmXLFvr3709QUBCKovD9999bLFdVlSlTphAYGIiTkxMxMTEcP37cYp309HRGjBiBu7s7np6ePPXUU+Tk5GiefdSoUTf8O/Tp08cqss+YMYOOHTvi5uaGn58fAwcOJDEx0WKdivydJCcn8+CDD+Ls7Iyfnx8vvvgixcXFmmfv3r37Da/92LFjNc8+d+5c2rRpY374Kzo6mlWrVpmX1+prropKW7Rokerg4KB+8cUX6uHDh9XRo0ernp6ealpamtbRLEydOlVt2bKleuHCBfN06dIl8/KxY8eqwcHB6vr169Xdu3er9957r9q5c2dNsq5cuVJ9+eWX1WXLlqmAunz5covlb7/9turh4aF+//336v79+9WHH35YDQsLU69du2Zep0+fPmpkZKS6Y8cOdevWrWrjxo3Vxx57TPPsI0eOVPv06WPx75Cenm6xjlbZY2Nj1fnz56uHDh1SExIS1H79+qkNGzZUc3JyzOvc7u+kuLhYbdWqlRoTE6Pu27dPXblyperr66tOnjxZ8+z333+/Onr0aIvXPjMzU/PsK1asUH/++Wf12LFjamJiovrPf/5Ttbe3Vw8dOqSqau2+5lIEqqBTp07q+PHjzT+XlJSoQUFB6owZMzRMdaOpU6eqkZGR5S7LyMhQ7e3t1SVLlpjnHT16VAXUuLi4WkpYvuvfSI1GoxoQEKC+++675nkZGRmqwWBQFy5cqKqqqh45ckQF1F27dpnXWbVqlaooinru3DnNsquqqQgMGDDgpttYS3ZVVdWLFy+qgLp582ZVVSv2d7Jy5UpVp9Opqamp5nXmzp2ruru7qwUFBZplV1VTEXjmmWduuo21ZFdVVfXy8lLnzZtX66+5XA6qpMLCQvbs2UNMTIx5nk6nIyYmhri4OA2Tle/48eMEBQXRqFEjRowYQXJyMgB79uyhqKjI4jiaNWtGw4YNre44kpKSSE1Ntcjq4eFBVFSUOWtcXByenp506NDBvE5MTAw6nY74+Phaz3y9TZs24efnR0REBE8//TRXrlwxL7Om7JmZmQB4e3sDFfs7iYuLo3Xr1hZjesfGxpKVlcXhw4c1y17m66+/xtfXl1atWjF58mTy8vLMy6whe0lJCYsWLSI3N5fo6Ohaf82lA7lKunz5MiUlJTcMYu/v789vv/2mUaryRUVFsWDBAiIiIrhw4QKvvfYa9913H4cOHSI1NRUHBwc8PT0ttvH39yc1NVWbwDdRlqe817xsWWpqKn5+fhbL7ezs8Pb21vx4+vTpw+DBgwkLC+PkyZP885//pG/fvsTFxaHX660mu9FoZNKkSXTp0oVWrVoBVOjvJDU1tdx/m7JltaG87ADDhw8nJCSEoKAgDhw4wP/93/+RmJjIsmXLNM9+8OBBoqOjyc/Px9XVleXLl9OiRQsSEhJq9TWXInAX69u3r/n7Nm3aEBUVRUhICIsXL8bJyUnDZLbl0UcfNX/funVr2rRpQ3h4OJs2baJnz54aJrM0fvx4Dh06xLZt27SOUmk3yz5mzBjz961btyYwMJCePXty8uRJwsPDazumhYiICBISEsjMzGTp0qWMHDmSzZs313oOuRxUSb6+vuj1+hta6tPS0ggICNAoVcV4enrStGlTTpw4QUBAAIWFhWRkZFisY43HUZbnVq95QEAAFy9etFheXFxMenq61R1Po0aN8PX15cSJE4B1ZJ8wYQI//fQTGzdutOhWvSJ/JwEBAeX+25Qtq2k3y16eqKgoAIvXXqvsDg4ONG7cmPbt2zNjxgwiIyP58MMPa/01lyJQSQ4ODrRv357169eb5xmNRtavX090dLSGyW4vJyeHkydPEhgYSPv27bG3t7c4jsTERJKTk63uOMLCwggICLDImpWVRXx8vDlrdHQ0GRkZ7Nmzx7zOhg0bMBqN5v/41uLs2bNcuXKFwMBAQNvsqqoyYcIEli9fzoYNGwgLC7NYXpG/k+joaA4ePGhRyH755Rfc3d1p0aKFZtnLk5CQAGDx2muRvTxGo5GCgoLaf82ro1Xb1ixatEg1GAzqggUL1CNHjqhjxoxRPT09LVrqrcHzzz+vbtq0SU1KSlK3b9+uxsTEqL6+vurFixdVVTXdhtawYUN1w4YN6u7du9Xo6Gg1Ojpak6zZ2dnqvn371H379qmA+sEHH6j79u1Tz5w5o6qq6RZRT09P9YcfflAPHDigDhgwoNxbRNu2bavGx8er27ZtU5s0aVIrt1neKnt2drb6wgsvqHFxcWpSUpK6bt06tV27dmqTJk3U/Px8zbM//fTTqoeHh7pp0yaL2yjz8vLM69zu76TsdsXevXurCQkJ6urVq9V69erV+G2Wt8t+4sQJ9fXXX1d3796tJiUlqT/88IPaqFEjtVu3bppnf+mll9TNmzerSUlJ6oEDB9SXXnpJVRRFXbt2raqqtfuaSxGooo8++kht2LCh6uDgoHbq1EndsWOH1pFuMGzYMDUwMFB1cHBQ69evrw4bNkw9ceKEefm1a9fUcePGqV5eXqqzs7M6aNAg9cKFC5pk3bhxowrcMI0cOVJVVdNtoq+++qrq7++vGgwGtWfPnmpiYqLFPq5cuaI+9thjqqurq+ru7q4+8cQTanZ2tqbZ8/Ly1N69e6v16tVT7e3t1ZCQEHX06NE3fGDQKnt5uQF1/vz55nUq8ndy+vRptW/fvqqTk5Pq6+urPv/882pRUZGm2ZOTk9Vu3bqp3t7eqsFgUBs3bqy++OKLFs8JaJX9ySefVENCQlQHBwe1Xr16as+ePc0FQFVr9zWXrqSFEMKGSZuAEELYMCkCQghhw6QICCGEDZMiIIQQNkyKgBBC2DApAkIIYcOkCAghhA2TIiCEEDZMioAQdUB5w1YKUR2kCAhxG+WNEVzeOMFC1EUynoAQFdCnTx/mz59vMc9gMGiURojqI2cCQlSAwWAgICDAYvLy8gJMl2rmzp1L3759cXJyolGjRixdutRi+4MHD/LAAw/g5OSEj48PY8aMIScnx2KdL774gpYtW2IwGAgMDGTChAkWyy9fvsygQYNwdnamSZMmrFixomYPWtgEKQJCVINXX32VIUOGsH//fkaMGMGjjz7K0aNHAcjNzSU2NhYvLy927drFkiVLWLduncWb/Ny5cxk/fjxjxozh4MGDrFixgsaNG1v8jtdee40//elPHDhwgH79+jFixAjS09Nr9TjFXegOe0QV4q43cuRIVa/Xqy4uLhbTm2++qaqqqUvjsWPHWmwTFRWlPv3006qqquqnn36qenl5qTk5OeblP//8s6rT6cxdSgcFBakvv/zyTTMA6iuvvGL+OScnRwXUVatWVdtxCtskbQJCVECPHj2YO3euxTxvb2/z99ePxhYdHW0exero0aNERkbi4uJiXt6lSxeMRiOJiYkoisL58+dvO95wmzZtzN+7uLjg7u5+w7CUQlSWFAEhKsDFxeWGyzPVxcnJqULr2dvbW/ysKApGo7EmIgkbIm0CQlSDHTt23PBz8+bNAWjevDn79+8nNzfXvHz79u3odDoiIiJwc3MjNDTUYkxZIWqLnAkIUQEFBQWkpqZazLOzs8PX1xeAJUuW0KFDB7p27crXX3/Nzp07+fzzzwEYMWIEU6dOZeTIkUybNo1Lly4xceJE/vKXv+Dv7w/AtGnTGDt2LH5+fvTt25fs7Gy2b9/OxIkTa/dAhc2RIiBEBaxevZrAwECLeREREfz222+A6c6dRYsWMW7cOAIDA1m4cCEtWrQAwNnZmTVr1vDMM8/QsWNHnJ2dGTJkCB988IF5XyNHjiQ/P59//etfvPDCC/j6+jJ06NDaO0Bhs2SMYSHukKIoLF++nIEDB2odRYhKkzYBIYSwYVIEhBDChkmbgBB3SK6oirpMzgSEEMKGSREQQggbJkVACCFsmBQBIYSwYVIEhBDChkkREEIIGyZFQAghbJgUASGEsGH/D12tz2uK38mSAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots(1,1,figsize=(4,3))\n", "plot_metrics(metrics.metrics,\n", " keys=['train_loss', 'valid_loss'],\n", " colors=['fessa1', 'fessa5'],\n", " yscale='linear',\n", " ax = ax)" ] }, { "cell_type": "markdown", "id": "93415c28", "metadata": {}, "source": [ "#### Check training outcome in CV space" ] }, { "cell_type": "code", "execution_count": 16, "id": "162c8a13", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz8AAAE8CAYAAADuXg/EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAzzUlEQVR4nO3deVxU9f7H8fewo8jgBojKoplruUvkLbtGkZpX01+mWYFyNQtL0yx95N7i0ubNa5E3Q3uoWVbWTcsWcim30rLFFLM00QI1BVwSF76/P7xMjiyCDjBwXs/H4zxkvud7zny+c+B8/Mz3zBmbMcYIAAAAAKo4j4oOAAAAAADKA8UPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAACqlPXv2yGazaf78+Ze0vc1m0+TJk10aE9wbxQ/c1vz582Wz2Qpdxo4dW9HhFev8WL28vFSrVi21b99eI0aM0I8//njJ+z1x4oQmT56s1atXuy7Yy7B+/XpNnjxZWVlZFR0KABSLnFKQu+UUoDx4VXQAwMVMnTpVUVFRTm2tWrWqoGhK7qabbtI999wjY4yys7P17bffasGCBXrxxRc1Y8YMjRo1qtT7PHHihKZMmSJJuuGGG1wccemtX79eU6ZMUUJCgoKCgio6HAC4KHLKX9wtpwDlgeIHbq9bt27q0KFDifqePHlSPj4+8vCo+EnNK6+8UnfddZdT2/Tp09WzZ0+NHj1azZo1U/fu3SsoOgCwJnIKYG0V/9cMXKLVq1fLZrNpyZIlGj9+vOrXr69q1aopJydHkrR06VK1b99e/v7+qlOnju666y7t37/faR8JCQkKCAjQ3r17deuttyogIED169fXnDlzJEnff/+9unbtqurVqysiIkKLFy++rJhr166tJUuWyMvLS08++aSj/dSpU5o4caLat28vu92u6tWr67rrrtOqVascffbs2aO6detKkqZMmeK4BCL/WuXvvvtOCQkJatSokfz8/BQaGqrBgwfrjz/+cIrh6NGjGjlypCIjI+Xr66vg4GDddNNN+vrrr536bdq0SbfccovsdruqVaumLl26aN26dY71kydP1pgxYyRJUVFRjnj27NlzWa8RAFQEcsql5ZTJkyfLZrNp586duuuuu2S321W3bl1NmDBBxhilp6erV69eCgwMVGhoqJ599tkC4zhw4IASExMVEhIiPz8/tW7dWgsWLCjQLysrSwkJCbLb7QoKClJ8fHyhl13fcMMNhc5kJSQkKDIy8qKv6/79+zV48GCFhITI19dXLVu21KuvvnrR7VA5MPMDt5edna1Dhw45tdWpU8fx8+OPPy4fHx89/PDDys3NlY+Pj+bPn69BgwapY8eOmjZtmjIzM/Wvf/1L69at0zfffON0idbZs2fVrVs3XX/99Zo5c6YWLVqk4cOHq3r16nrsscc0cOBA9enTR8nJybrnnnsUExNT4JKJ0ggPD1eXLl20atUq5eTkKDAwUDk5OXrllVc0YMAADRkyREePHtW8efMUFxenL7/8Um3atFHdunX10ksv6b777tNtt92mPn36SJKuvvpqSdInn3yiX375RYMGDVJoaKi2bdumuXPnatu2bdq4caNsNpskadiwYXrrrbc0fPhwtWjRQn/88Ye++OILbd++Xe3atZMkffbZZ+rWrZvat2+vSZMmycPDQykpKeratas+//xzderUSX369NHOnTv1+uuv6/nnn3cck/xkCgDuiJzi2pyS74477lDz5s01ffp0rVixQk888YRq1aqll19+WV27dtWMGTO0aNEiPfzww+rYsaOuv/56SdKff/6pG264Qbt27dLw4cMVFRWlpUuXKiEhQVlZWRoxYoQkyRijXr166YsvvtCwYcPUvHlzLVu2TPHx8Zf82hUmMzNT11xzjWw2m4YPH666devqww8/VGJionJycjRy5EiXPh8qgAHcVEpKipFU6GKMMatWrTKSTKNGjcyJEycc2506dcoEBwebVq1amT///NPRvnz5ciPJTJw40dEWHx9vJJmnnnrK0XbkyBHj7+9vbDabWbJkiaN9x44dRpKZNGnSRWOXZJKSkopcP2LECCPJfPvtt8YYY86cOWNyc3Od+hw5csSEhISYwYMHO9oOHjxYZAznvwb5Xn/9dSPJrF271tFmt9uLjS0vL880adLExMXFmby8PKf9R0VFmZtuusnR9vTTTxtJZvfu3UXuDwDcATmlbHLKpEmTjCQzdOhQR9uZM2dMgwYNjM1mM9OnT3eKwd/f38THxzvaZs2aZSSZhQsXOtpOnTplYmJiTEBAgMnJyTHGGPPuu+8aSWbmzJlOz3PdddcZSSYlJcXR3qVLF9OlS5cC8cfHx5uIiAintgvHn5iYaOrVq2cOHTrk1K9///7GbrcX+rqgcuGyN7i9OXPm6JNPPnFazhcfHy9/f3/H482bN+vAgQO6//775efn52jv0aOHmjVrphUrVhR4jn/+85+On4OCgtS0aVNVr15d/fr1c7Q3bdpUQUFB+uWXXy57TAEBAZLOXYImSZ6envLx8ZEk5eXl6fDhwzpz5ow6dOhQ4HK0opz/Gpw8eVKHDh3SNddcI0lO+wgKCtKmTZv022+/FbqfrVu36qefftKdd96pP/74Q4cOHdKhQ4d0/Phx3XjjjVq7dq3y8vJKP2gAcAPkFNfmlHznj9nT01MdOnSQMUaJiYmO9vzX4vwxf/DBBwoNDdWAAQMcbd7e3nrwwQd17NgxrVmzxtHPy8tL9913n9PzPPDAAyUaT0kYY/T222+rZ8+eMsY48t+hQ4cUFxen7OzsEr9+cF9c9ga316lTp2I/nHrh5QK//vqrpHOJ5ULNmjXTF1984dTm5+dX4FItu92uBg0aFJjWt9vtOnLkSKniL8yxY8ckSTVq1HC0LViwQM8++6x27Nih06dPO9pLejnE4cOHNWXKFC1ZskQHDhxwWpedne34eebMmYqPj1fDhg3Vvn17de/eXffcc48aNWokSfrpp58kqdhLCbKzs1WzZs0SxQUA7oSc4tqcki88PNzpsd1ul5+fn9Mlhfnt539u6Ndff1WTJk0K3FSiefPmjvX5/9arV89R6OUr7LhcqoMHDyorK0tz587V3LlzC+1z4WuByofiB5Xe+e9OXQpPT89StRtjLuv5JOmHH36Qp6enIwktXLhQCQkJ6t27t8aMGaPg4GB5enpq2rRp+vnnn0u0z379+mn9+vUaM2aM2rRpo4CAAOXl5emWW25xmqnp16+frrvuOi1btkwff/yxnn76ac2YMUPvvPOOunXr5uj79NNPq02bNoU+14XJBwCqCnLKOSXNKfkKG19Zjrk4Nput0Oc4e/Zssdvlj+uuu+4q8g3A/M9EofKi+EGVExERIUlKS0tT165dndalpaU51leUvXv3as2aNYqJiXG8S/fWW2+pUaNGeuedd5zeGZw0aZLTthe+a5jvyJEjSk1N1ZQpUzRx4kRHe/4szoXq1aun+++/X/fff78OHDigdu3a6cknn1S3bt3UuHFjSVJgYKBiY2OLHUtR8QBAVUFOuXhOuRwRERH67rvvlJeX5zT7s2PHDsf6/H9TU1N17Ngxpzfg0tLSCuyzZs2ahV5OmD+LVJS6deuqRo0aOnv27EXzHyovPvODKqdDhw4KDg5WcnKycnNzHe0ffvihtm/frh49elRYbIcPH9aAAQN09uxZPfbYY472/HfHzn+natOmTdqwYYPT9tWqVZOkArf2LGx7SZo1a5bT47Nnzxa4XCE4OFhhYWGO16p9+/Zq3LixnnnmGcelFOc7ePCg4+fq1asXGg8AVBXklL9cmFNcoXv37srIyNAbb7zhaDtz5oxmz56tgIAAdenSxdHvzJkzeumllxz9zp49q9mzZxfYZ+PGjbVjxw6nfPXtt986fV1DYTw9PdW3b1+9/fbb+uGHHwqsP39/qLyY+UGV4+3trRkzZmjQoEHq0qWLBgwY4LgtaWRkpB566KFyiWPnzp1auHChjDHKycnRt99+q6VLl+rYsWN67rnndMsttzj63nrrrXrnnXd02223qUePHtq9e7eSk5PVokULpwLE399fLVq00BtvvKErr7xStWrVUqtWrdSqVSvHbVVPnz6t+vXr6+OPP9bu3budYjp69KgaNGig//u//1Pr1q0VEBCgTz/9VF999ZXjuxc8PDz0yiuvqFu3bmrZsqUGDRqk+vXra//+/Vq1apUCAwP1/vvvSzpXKEnSY489pv79+8vb21s9e/Z0FEUAUNmRU4rOKa4wdOhQvfzyy0pISNCWLVsUGRmpt956S+vWrdOsWbMcs1k9e/ZU586dNXbsWO3Zs0ctWrTQO++8U+jnjwYPHqznnntOcXFxSkxM1IEDB5ScnKyWLVs6vrepKNOnT9eqVasUHR2tIUOGqEWLFjp8+LC+/vprffrppzp8+LDLXwOUswq5xxxQAvm3Jf3qq68KXZ9/W9KlS5cWuv6NN94wbdu2Nb6+vqZWrVpm4MCBZt++fU594uPjTfXq1Qts26VLF9OyZcsC7REREaZHjx4XjV3n3ULVw8PDBAUFmbZt25oRI0aYbdu2Feifl5dnnnrqKRMREWF8fX1N27ZtzfLlywu9Lef69etN+/btjY+Pj9MtOvft22duu+02ExQUZOx2u7n99tvNb7/95tQnNzfXjBkzxrRu3drUqFHDVK9e3bRu3dq8+OKLBWL65ptvTJ8+fUzt2rWNr6+viYiIMP369TOpqalO/R5//HFTv3594+HhwW2vAbgtcorrc4oxf93q+uDBg5f8WmRmZppBgwaZOnXqGB8fH3PVVVc53bo63x9//GHuvvtuExgYaOx2u7n77rvNN998U+BW18YYs3DhQtOoUSPj4+Nj2rRpYz766KMS3eo6P56kpCTTsGFD4+3tbUJDQ82NN95o5s6dWyAmVD42Y8r4U2cAAAAA4Ab4zA8AAAAAS6D4AQAAAGAJFD8AAAAALKHUxc/atWvVs2dPhYWFyWaz6d1333Vab4zRxIkTVa9ePfn7+ys2NrbAfeEPHz6sgQMHKjAwUEFBQUpMTCz0lroAAFwMeQkAUFKlLn6OHz+u1q1ba86cOYWunzlzpl544QUlJydr06ZNql69uuLi4nTy5ElHn4EDB2rbtm365JNPtHz5cq1du1ZDhw699FEAACyLvAQAKKnLutubzWbTsmXL1Lt3b0nn3l0LCwvT6NGj9fDDD0uSsrOzFRISovnz56t///7avn27WrRooa+++kodOnSQJK1cuVLdu3fXvn37FBYWdvmjAgBYEnkJAFAcl37J6e7du5WRkaHY2FhHm91uV3R0tDZs2KD+/ftrw4YNCgoKciQYSYqNjZWHh4c2bdqk2267rcB+c3Nznb5VOS8vT4cPH1bt2rVls9lcOQQAQDGMMTp69KjCwsLk4eH+Hxstq7wkkZsAwF2UJje5tPjJyMiQJIWEhDi1h4SEONZlZGQoODjYOQgvL9WqVcvR50LTpk3TlClTXBkqAOAypKenq0GDBhUdxkWVVV6SyE0A4G5KkptcWvyUlXHjxmnUqFGOx9nZ2QoPD1d6eroCAwPLLY5Wkz4qtP2HKXHlFgMAFHUuulBZnJtycnLUsGFD1ahRw+X7rmzITQBQtPPPTWV9PipNbnJp8RMaGipJyszMVL169RztmZmZatOmjaPPgQMHnLY7c+aMDh8+7Nj+Qr6+vvL19S3QHhgYWK4JxsO3WqHt5RkDABR1LrpQWZ6bKstlXWWVlyRyEwBcKHLsCsfP55+byut8VJLc5NILtqOiohQaGqrU1FRHW05OjjZt2qSYmBhJUkxMjLKysrRlyxZHn88++0x5eXmKjo52ZTgAAIsjLwEAzlfqmZ9jx45p165djse7d+/W1q1bVatWLYWHh2vkyJF64okn1KRJE0VFRWnChAkKCwtz3HmnefPmuuWWWzRkyBAlJyfr9OnTGj58uPr3788ddQAApUZeAgCUVKmLn82bN+vvf/+743H+9c7x8fGaP3++HnnkER0/flxDhw5VVlaW/va3v2nlypXy8/NzbLNo0SINHz5cN954ozw8PNS3b1+98MILLhgOAMBqyEsAgJK6rO/5qSg5OTmy2+3Kzs4u12uaz7+O8Xx7pvcotxgAoKhz0YXK4txUUeffyoDcBMDqKup8VJrzr/t/SQMAAAAAuADFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAluBV0QEAAC4ucuyKig4BAIBKj5kfAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYgsuLn7Nnz2rChAmKioqSv7+/GjdurMcff1zGGEcfY4wmTpyoevXqyd/fX7Gxsfrpp59cHQoAAOQlAICDy4ufGTNm6KWXXtK///1vbd++XTNmzNDMmTM1e/ZsR5+ZM2fqhRdeUHJysjZt2qTq1asrLi5OJ0+edHU4AACLIy8BAPJ5uXqH69evV69evdSjRw9JUmRkpF5//XV9+eWXks69uzZr1iyNHz9evXr1kiS99tprCgkJ0bvvvqv+/fu7OiQAgIWRlwAA+Vw+83PttdcqNTVVO3fulCR9++23+uKLL9StWzdJ0u7du5WRkaHY2FjHNna7XdHR0dqwYUOh+8zNzVVOTo7TAgBASZRFXpLITQBQGbl85mfs2LHKyclRs2bN5OnpqbNnz+rJJ5/UwIEDJUkZGRmSpJCQEKftQkJCHOsuNG3aNE2ZMsXVoQIALKAs8pJEbgKAysjlMz9vvvmmFi1apMWLF+vrr7/WggUL9Mwzz2jBggWXvM9x48YpOzvbsaSnp7swYgBAVVYWeUkiNwFAZeTymZ8xY8Zo7Nixjmukr7rqKv3666+aNm2a4uPjFRoaKknKzMxUvXr1HNtlZmaqTZs2he7T19dXvr6+rg4VAGABZZGXJHITAFRGLp/5OXHihDw8nHfr6empvLw8SVJUVJRCQ0OVmprqWJ+Tk6NNmzYpJibG1eEAACyOvAQAyOfymZ+ePXvqySefVHh4uFq2bKlvvvlGzz33nAYPHixJstlsGjlypJ544gk1adJEUVFRmjBhgsLCwtS7d29XhwMAsDjyEgAgn8uLn9mzZ2vChAm6//77deDAAYWFhenee+/VxIkTHX0eeeQRHT9+XEOHDlVWVpb+9re/aeXKlfLz83N1OAAAiyMvAQDy2cz5X3FdSeTk5Mhutys7O1uBgYHl9ryRY1cU2r5neo9yiwGANRV1/ilOWZybKur8WxmQmwBYXUWdj0pz/nX5Z34AAAAAwB1R/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWUCbFz/79+3XXXXepdu3a8vf311VXXaXNmzc71htjNHHiRNWrV0/+/v6KjY3VTz/9VBahAABAXgKAMhI5doVjqQxcXvwcOXJEnTt3lre3tz788EP9+OOPevbZZ1WzZk1Hn5kzZ+qFF15QcnKyNm3apOrVqysuLk4nT550dTgAAIsjLwEA8nm5eoczZsxQw4YNlZKS4miLiopy/GyM0axZszR+/Hj16tVLkvTaa68pJCRE7777rvr37+/qkAAAFkZeAgDkc/nMz3//+1916NBBt99+u4KDg9W2bVv95z//cazfvXu3MjIyFBsb62iz2+2Kjo7Whg0bCt1nbm6ucnJynBYAAEqiLPKSRG4CgMrI5cXPL7/8opdeeklNmjTRRx99pPvuu08PPvigFixYIEnKyMiQJIWEhDhtFxIS4lh3oWnTpslutzuWhg0bujpsAEAVVRZ5SSI3AUBl5PLiJy8vT+3atdNTTz2ltm3baujQoRoyZIiSk5MveZ/jxo1Tdna2Y0lPT3dhxACAqqws8pJEbgKAysjlxU+9evXUokULp7bmzZtr7969kqTQ0FBJUmZmplOfzMxMx7oL+fr6KjAw0GkBAKAkyiIvSeQmAKiMXF78dO7cWWlpaU5tO3fuVEREhKRzHzINDQ1VamqqY31OTo42bdqkmJgYV4cDALA48hIAIJ/L7/b20EMP6dprr9VTTz2lfv366csvv9TcuXM1d+5cSZLNZtPIkSP1xBNPqEmTJoqKitKECRMUFham3r17uzocAIDFkZcAAPlcXvx07NhRy5Yt07hx4zR16lRFRUVp1qxZGjhwoKPPI488ouPHj2vo0KHKysrS3/72N61cuVJ+fn6uDgcAYHHkJQBAPpcXP5J066236tZbby1yvc1m09SpUzV16tSyeHoAAJyQlwAAUhl85gcAAAAA3BHFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFiCV0UHAAAALt0evzudHkeeXFxBkQCA+2PmBwAAAIAlUPwAAAAAsAQuewMAAABQZiLHrnB6vGd6jwqKhJkfAAAAABZB8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAluBV0QG4u8ixKyo6BAAAAAAuwMwPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJZR58TN9+nTZbDaNHDnS0Xby5EklJSWpdu3aCggIUN++fZWZmVnWoQAAQF4CAAsr0+Lnq6++0ssvv6yrr77aqf2hhx7S+++/r6VLl2rNmjX67bff1KdPn7IMBQAA8hIAWFyZFT/Hjh3TwIED9Z///Ec1a9Z0tGdnZ2vevHl67rnn1LVrV7Vv314pKSlav369Nm7cWFbhAAAsjrwEACiz4icpKUk9evRQbGysU/uWLVt0+vRpp/ZmzZopPDxcGzZsKHRfubm5ysnJcVoAACgNV+YlidwEAJVRmXzJ6ZIlS/T111/rq6++KrAuIyNDPj4+CgoKcmoPCQlRRkZGofubNm2apkyZUhahAgAswNV5SSI3AUBl5PKZn/T0dI0YMUKLFi2Sn5+fS/Y5btw4ZWdnO5b09HSX7BcAUPWVRV6SyE0AUBm5vPjZsmWLDhw4oHbt2snLy0teXl5as2aNXnjhBXl5eSkkJESnTp1SVlaW03aZmZkKDQ0tdJ++vr4KDAx0WgAAKImyyEsSuQkAKiOXX/Z244036vvvv3dqGzRokJo1a6ZHH31UDRs2lLe3t1JTU9W3b19JUlpamvbu3auYmBhXhwMAsDjyEgAgn8uLnxo1aqhVq1ZObdWrV1ft2rUd7YmJiRo1apRq1aqlwMBAPfDAA4qJidE111zj6nAAABZHXgIA5CuTGx5czPPPPy8PDw/17dtXubm5iouL04svvlgRoQAAQF4CAIsol+Jn9erVTo/9/Pw0Z84czZkzpzyeHgAAJ+QlALCmMvueHwAAAABwJxQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS/Cq6AAAAEDJRY5d4fR4j18FBQIAlRAzPwAAAAAsgeIHAAAAgCVw2ZsLFLgEYXqPCooEAIBzzs9N5CUAOIeZHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEvuQUAAAAQOlNtmuP37kfI08urthYSoiZHwAAAACWwMwPAAAAgBKLHLtCkhyzPpUJMz8AAAAALIHiBwAAAIAlcNkbAAAAgMuyx+9Ox8/ufPMDZn4AAAAAWAIzPwDgpvI/UAoAQGVy/iyQ5F4zQS6f+Zk2bZo6duyoGjVqKDg4WL1791ZaWppTn5MnTyopKUm1a9dWQECA+vbtq8zMTFeHAgCAJHITAOAclxc/a9asUVJSkjZu3KhPPvlEp0+f1s0336zjx487+jz00EN6//33tXTpUq1Zs0a//fab+vTp4+pQAACQZK3ctMfvTscCAHDm8sveVq5c6fR4/vz5Cg4O1pYtW3T99dcrOztb8+bN0+LFi9W1a1dJUkpKipo3b66NGzfqmmuucXVIAACLIzcBAKRyuOFBdna2JKlWrVqSpC1btuj06dOKjY119GnWrJnCw8O1YcOGQveRm5urnJwcpwUAgEtFbgIAayrT4icvL08jR45U586d1apVK0lSRkaGfHx8FBQU5NQ3JCREGRkZhe5n2rRpstvtjqVhw4ZlGTYAoAojNwGAdZVp8ZOUlKQffvhBS5Ysuaz9jBs3TtnZ2Y4lPT3dRRECAKyG3AQA1lVmt7oePny4li9frrVr16pBgwaO9tDQUJ06dUpZWVlO77BlZmYqNDS00H35+vrK19e3rEIFAFgEuQkAyp87fQGqy2d+jDEaPny4li1bps8++0xRUVFO69u3by9vb2+lpqY62tLS0rR3717FxMS4OhwAAMhNAABJZTDzk5SUpMWLF+u9995TjRo1HNdK2+12+fv7y263KzExUaNGjVKtWrUUGBioBx54QDExMdxNBwBQJshNAACpDIqfl156SZJ0ww03OLWnpKQoISFBkvT888/Lw8NDffv2VW5uruLi4vTiiy+6OhQAACSRmwAA57i8+DHGXLSPn5+f5syZozlz5rj66QEA/xM5doXj5z3Te1RgJBWP3AQAkMrhe34AAAAAwB1Q/AAAAACwBIofAAAAAJZA8QMAAADAEsrsS04BAEDZOP8LAwEAJcfMDwAAAABLYOYHAAAAQLmpyK9iYOYHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACW4FXRAbijyLErKjoEAHCp889re6b3qMBIAACVTVX6vzEzPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJbADQ8AAHBzVenDxgAqtz1+d1Z0CJeFmR8AAAAAlsDMDwC4Ed7hBwCg7DDzAwAAAMASmPkBAAAAUC7O/8xQ5MnF5f78zPwAAAAAsARmfsQ19gAA90NuAlDV7fG7U5r8vweTs8vlOZn5AQAAAGAJFD8AAAAALMGyl71xOQEAAABgLcz8AAAAALAEy878lKXzZ5X2TO9RgZEAqAyYiUa+4n4Xzr897OXsl7wEoCSqam6q0JmfOXPmKDIyUn5+foqOjtaXX35ZkeEAACyOvAQAVVuFzfy88cYbGjVqlJKTkxUdHa1Zs2YpLi5OaWlpCg4OrqiwXK6yv9tW2eMvSlUdF4BL5y556XJmeEqisp//Knv8xZpsv+Bx+dz6F7iYsj4vlacKm/l57rnnNGTIEA0aNEgtWrRQcnKyqlWrpldffbWiQgIAWBh5CQCqvgqZ+Tl16pS2bNmicePGOdo8PDwUGxurDRs2FOifm5ur3Nxcx+Ps7HPvhOTk5FxyDHm5Jy5520t1OfFWlPNfp8oYf1Gq6rhQOZX3+ehyfufztzXGuCoct1DavCSVXW7Ksbnutb3Y71ZlPP9V6fN37gXHvqqND5WK09+aC89LRSqv3GQqwP79+40ks379eqf2MWPGmE6dOhXoP2nSJCOJhYWFhcVNlvT09PJKGeWitHnJGHITCwsLi7stJclNleJub+PGjdOoUaMcj/Py8nT48GHVrl1bNpvNZc+Tk5Ojhg0bKj09XYGBgS7bb0VhPO6N8bg3xlM4Y4yOHj2qsLAwF0ZXOZVHbuL30P1VtTExHvfGeApXmtxUIcVPnTp15OnpqczMTKf2zMxMhYaGFujv6+srX19fp7agoKAyiy8wMLBK/ELlYzzujfG4N8ZTkN1ud1E07qO0eUkq39zE76H7q2pjYjzujfEUVNLcVCE3PPDx8VH79u2VmprqaMvLy1NqaqpiYmIqIiQAgIWRlwDAGirssrdRo0YpPj5eHTp0UKdOnTRr1iwdP35cgwYNqqiQAAAWRl4CgKqvwoqfO+64QwcPHtTEiROVkZGhNm3aaOXKlQoJCamokOTr66tJkyYVuIyhsmI87o3xuDfGYz3kpbJX1cYjVb0xMR73xngun82YKna/UgAAAAAoRIV9ySkAAAAAlCeKHwAAAACWQPEDAAAAwBIofgAAAABYgmWLn9WrV8tmsxW6fPXVV0Vud8MNNxToP2zYsHKMvGiRkZEFYps+fXqx25w8eVJJSUmqXbu2AgIC1Ldv3wJf8ldR9uzZo8TEREVFRcnf31+NGzfWpEmTdOrUqWK3c6djNGfOHEVGRsrPz0/R0dH68ssvi+2/dOlSNWvWTH5+frrqqqv0wQcflFOkxZs2bZo6duyoGjVqKDg4WL1791ZaWlqx28yfP7/AcfDz8yuniIs3efLkArE1a9as2G3c9djkK+zv32azKSkpqdD+7nx8rIzcdI675qaqkJckcpO7nvuqWm5y27xkLCo3N9f8/vvvTss///lPExUVZfLy8orcrkuXLmbIkCFO22VnZ5dj5EWLiIgwU6dOdYrt2LFjxW4zbNgw07BhQ5Oammo2b95srrnmGnPttdeWU8TF+/DDD01CQoL56KOPzM8//2zee+89ExwcbEaPHl3sdu5yjJYsWWJ8fHzMq6++arZt22aGDBligoKCTGZmZqH9161bZzw9Pc3MmTPNjz/+aMaPH2+8vb3N999/X86RFxQXF2dSUlLMDz/8YLZu3Wq6d+9uwsPDi/39SklJMYGBgU7HISMjoxyjLtqkSZNMy5YtnWI7ePBgkf3d+djkO3DggNN4PvnkEyPJrFq1qtD+7nx8rIzcdI675qbKnpeMITe587mvquUmd81Lli1+LnTq1ClTt25dM3Xq1GL7denSxYwYMaJ8giqliIgI8/zzz5e4f1ZWlvH29jZLly51tG3fvt1IMhs2bCiDCC/fzJkzTVRUVLF93OUYderUySQlJTkenz171oSFhZlp06YV2r9fv36mR48eTm3R0dHm3nvvLdM4L8WBAweMJLNmzZoi+6SkpBi73V5+QZXCpEmTTOvWrUvcvzIdm3wjRowwjRs3LvI/zO58fPAXctM57pybKlNeMobc5M7nvqqem9wlL1n2srcL/fe//9Uff/xRom/yXrRokerUqaNWrVpp3LhxOnHiRDlEWDLTp09X7dq11bZtWz399NM6c+ZMkX23bNmi06dPKzY21tHWrFkzhYeHa8OGDeURbqllZ2erVq1aF+1X0cfo1KlT2rJli9Nr6+HhodjY2CJf2w0bNjj1l6S4uDi3PBbZ2dmSdNFjcezYMUVERKhhw4bq1auXtm3bVh7hlchPP/2ksLAwNWrUSAMHDtTevXuL7FuZjo107vdv4cKFGjx4sGw2W5H93Pn44Bxy0znunJsqS16SyE353PncV1VzkzvlJS+X77GSmjdvnuLi4tSgQYNi+915552KiIhQWFiYvvvuOz366KNKS0vTO++8U06RFu3BBx9Uu3btVKtWLa1fv17jxo3T77//rueee67Q/hkZGfLx8VFQUJBTe0hIiDIyMsoh4tLZtWuXZs+erWeeeabYfu5wjA4dOqSzZ88W+Gb4kJAQ7dixo9BtMjIyCu3vbsciLy9PI0eOVOfOndWqVasi+zVt2lSvvvqqrr76amVnZ+uZZ57Rtddeq23btl3076ysRUdHa/78+WratKl+//13TZkyRdddd51++OEH1ahRo0D/ynJs8r377rvKyspSQkJCkX3c+fjgL+Smv7jj31xlyksSuUly73NfVc5NbpWXynxuqZw9+uijRlKxy/bt2522SU9PNx4eHuatt94q9fOlpqYaSWbXrl2uGoKTSxlPvnnz5hkvLy9z8uTJQtcvWrTI+Pj4FGjv2LGjeeSRR1w6jvNdypj27dtnGjdubBITE0v9fGV9jAqzf/9+I8msX7/eqX3MmDGmU6dOhW7j7e1tFi9e7NQ2Z84cExwcXGZxXophw4aZiIgIk56eXqrtTp06ZRo3bmzGjx9fRpFduiNHjpjAwEDzyiuvFLq+shybfDfffLO59dZbS7WNOx+fqoDc9Bd3zE1WyEvGkJsK487nvqqUm9wpL1W5mZ/Ro0cXW1VKUqNGjZwep6SkqHbt2vrHP/5R6ueLjo6WdO7dn8aNG5d6+4u5lPHki46O1pkzZ7Rnzx41bdq0wPrQ0FCdOnVKWVlZTu+wZWZmKjQ09HLCLlZpx/Tbb7/p73//u6699lrNnTu31M9X1seoMHXq1JGnp2eBuxMV99qGhoaWqn9FGD58uJYvX661a9eW+l0Yb29vtW3bVrt27Sqj6C5dUFCQrrzyyiJjqwzHJt+vv/6qTz/9tNTvKLvz8akKyE1/ccfcZIW8JJGbCuPO576qkpvcLi+5tJSqhPLy8kxUVNRF79RSlC+++MJIMt9++62LI7t8CxcuNB4eHubw4cOFrs//UOn57yru2LHDrT5Uum/fPtOkSRPTv39/c+bMmUvaR0Udo06dOpnhw4c7Hp89e9bUr1+/2A+VXviuSExMjFt8cDEvL88kJSWZsLAws3Pnzkvax5kzZ0zTpk3NQw895OLoLt/Ro0dNzZo1zb/+9a9C17vzsbnQpEmTTGhoqDl9+nSptnPn42NF5Cb3zU2VOS8ZQ266kDuf+6pKbnK3vGT54ufTTz8tcnp+3759pmnTpmbTpk3GGGN27dplpk6dajZv3mx2795t3nvvPdOoUSNz/fXXl3fYBaxfv948//zzZuvWrebnn382CxcuNHXr1jX33HOPo8+F4zHm3DRxeHi4+eyzz8zmzZtNTEyMiYmJqYghFLBv3z5zxRVXmBtvvNHs27fP6daH5/dx12O0ZMkS4+vra+bPn29+/PFHM3ToUBMUFOS4bePdd99txo4d6+i/bt064+XlZZ555hmzfft2M2nSJLe5ZeV9991n7Ha7Wb16tdNxOHHihKPPheOZMmWK43awW7ZsMf379zd+fn5m27ZtFTEEJ6NHjzarV682u3fvNuvWrTOxsbGmTp065sCBA8aYynVsznf27FkTHh5uHn300QLrKtPxAbnJXXNTZc9LxpCb3PncVxVzkzvmJcsXPwMGDCjyuwN2797tdD/yvXv3muuvv97UqlXL+Pr6miuuuMKMGTPGLb5LYcuWLSY6OtrY7Xbj5+dnmjdvbp566imna6ovHI8xxvz555/m/vvvNzVr1jTVqlUzt912m9NJvCKlpKQUee11Pnc/RrNnzzbh4eHGx8fHdOrUyWzcuNGxrkuXLiY+Pt6p/5tvvmmuvPJK4+PjY1q2bGlWrFhRzhEXrqjjkJKS4uhz4XhGjhzpGHtISIjp3r27+frrr8s/+ELccccdpl69esbHx8fUr1/f3HHHHU7X3lemY3O+jz76yEgyaWlpBdZVpuMDcpO75qaqkJeMITe567mvKuYmd8xLNmOMce2FdAAAAADgfvieHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxA1wgIyNDI0aM0BVXXCE/Pz+FhISoc+fOeumll3TixAlHv8jISNlsNm3cuNFp+5EjR+qGG25wPJ48ebJsNpuGDRvm1G/r1q2y2Wzas2dPsfHs2rVLgwYNUoMGDeTr66uoqCgNGDBAmzdvVmZmpry9vbVkyZJCt01MTFS7du1K9wIAANwKeQlwHYof4Dy//PKL2rZtq48//lhPPfWUvvnmG23YsEGPPPKIli9frk8//dSpv5+fnx599NGL7tfPz0/z5s3TTz/9VKp4Nm/erPbt22vnzp16+eWX9eOPP2rZsmVq1qyZRo8erZCQEPXo0UOvvvpqgW2PHz+uN998U4mJiaV6TgCA+yAvAa7lVdEBAO7k/vvvl5eXlzZv3qzq1as72hs1aqRevXrJGOPUf+jQoUpOTtYHH3yg7t27F7nfpk2bKjg4WI899pjefPPNEsVijFFCQoKaNGmizz//XB4ef71X0aZNG40YMULSuXfRevfurb179yo8PNzRZ+nSpTpz5owGDhxYoucDALgf8hLgWsz8AP/zxx9/6OOPP1ZSUpJTgjmfzWZzehwVFaVhw4Zp3LhxysvLK3b/06dP19tvv63NmzeXKJ6tW7dq27ZtGj16tFOCyRcUFCRJ6t69u0JCQjR//nyn9SkpKerTp4+jHwCgciEvAa5H8QP8z65du2SMUdOmTZ3a69Spo4CAAAUEBBR6KcH48eO1e/duLVq0qNj9t2vXTv369SvR5QiSHJciNGvWrNh+np6eio+P1/z58x3vAP7888/6/PPPNXjw4BI9FwDA/ZCXANej+AEu4ssvv9TWrVvVsmVL5ebmFlhft25dPfzww5o4caJOnTpV7L6eeOIJff755/r4448v+rwXXspQnMGDB2v37t1atWqVpHPvrkVGRqpr164l3gcAoHIgLwGXjuIH+J8rrrhCNptNaWlpTu2NGjXSFVdcIX9//yK3HTVqlP7880+9+OKLxT5H48aNNWTIEI0dO/aiSeTKK6+UJO3YseOisTdp0kTXXXedUlJSlJeXp9dee02DBg0qcDkEAKDyIC8BrkfxA/xP7dq1ddNNN+nf//63jh8/XqptAwICNGHCBD355JM6evRosX0nTpyonTt3Fnkb0Hxt2rRRixYt9OyzzxZ63XZWVpbT48TERL399tt6++23tX//fiUkJJRqDAAA90JeAlyP4gc4z4svvqgzZ86oQ4cOeuONN7R9+3alpaVp4cKF2rFjhzw9PYvcdujQobLb7Vq8eHGxzxESEqJRo0bphRdeKLafzWZTSkqKdu7cqeuuu04ffPCBfvnlF3333Xd68skn1atXL6f+t99+u7y9vXXvvffq5ptvVsOGDUs+cACAWyIvAa5F8QOcp3Hjxvrmm28UGxurcePGqXXr1urQoYNmz56thx9+WI8//niR23p7e+vxxx/XyZMnL/o8Dz/8sAICAi7ar1OnTtq8ebOuuOIKDRkyRM2bN9c//vEPbdu2TbNmzXLqW61aNfXv319HjhzhA6UAUEWQlwDXspnSfHoNAAAAACopZn4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJ/w/t9tar05k+kgAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, axs = plt.subplots(1,2, figsize=(10,3))\n", "\n", "ax = axs[0]\n", "out_graph = model(dataset.get_graph_inputs())\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "ax.set_title('From Dataset')\n", "ax.set_xlabel('GNN CV')\n", "ax.set_ylim(0,100)\n", "\n", "ax = axs[1]\n", "out_graph = model(datamodule.get_graph_inputs(\"train\"))\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "out_graph = model(datamodule.get_graph_inputs(\"valid\"))\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "\n", "ax.set_title('From Datamodule')\n", "ax.set_xlabel('GNN CV')\n", "ax.set_ylim(0,100)\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "70b96b5f", "metadata": {}, "source": [ "### Export model for inference\n", "\n", "The exported module can then be used in PLUMED using the GNN-based C++ interfaces provided in the `plumed_interfaces` folder of the library." ] }, { "cell_type": "code", "execution_count": 18, "id": "d03f2fff", "metadata": {}, "outputs": [], "source": [ "traced_model = model.to_torchscript('model.pt', method='trace')" ] }, { "cell_type": "code", "execution_count": null, "id": "5c207733", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "graph_mlcolvar_test_2.5", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.18" } }, "nbformat": 4, "nbformat_minor": 5 }