{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Using `mlcolvar` with graph neural networks (GNNs)\n", "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/luigibonati/mlcolvar/blob/main/docs/notebooks/tutorials/adv_gnn_based_cvs.ipynb)\n", "\n", "Prerequisites:\n", "- Most of the workings of the library are the same using standard feed-forward-nn-based machine-learning CVs or GNN-based ones.\n", "Thus, it is recommended to first go through the basic tutorials for the standard scenario before moving to this tutorial.\n", "\n", "Reference papers: \n", "- _Zhang, Bonati, Trizio, Zhang, Kang, Hou, and Parrinello, [JCTC](https://doi.org/10.1021/acs.jctc.4c01197) (2025), [ArXiv](https://arxiv.org/abs/2409.07339)_\n", "\n", "Author: Enrico Trizio\n", "\n", "**NOTE**: For a more advanced usage of GNN models, allowing for better performance and lower computational costs in some scenarios, also check the tutorial on truncated GNNs.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Colab setup" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Colab setup\n", "import os\n", "\n", "if os.getenv(\"COLAB_RELEASE_TAG\"):\n", " import subprocess\n", " subprocess.run('wget https://raw.githubusercontent.com/luigibonati/mlcolvar/main/colab_setup.sh', shell=True)\n", " cmd = subprocess.run('bash colab_setup.sh TUTORIAL', shell=True, stdout=subprocess.PIPE)\n", " print(cmd.stdout.decode('utf-8'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overview\n", "\n", "### Feed-Forward-based CVs vs GNN-based CVs\n", "\n", "The default setting of `mlcolvar` is to represent the CVs as the output nodes of Feed-Forward Neural Networks (FFNNs or NNs, for simplicity) which take as input a set of physical descriptors (e.g., distances, angles, etc.).\n", "The code is thus designed to reflect this choice, with the default values of the classes set to intilialize the CV model in this framework, which is the most diffused for the time being in the field of machine-learning CVs and suits the needs of most users.\n", "\n", "However, recently a different approach have been proposed, in which the CVs are represented as Graph Neural Networks (GNNs) which directly take as input the Cartesian coordinates of the atoms in the studied system and return the CV space after a node-pooling operation on the output layer.\n", "This approach is thus descriptor-free and goes in the direcion of a more automated way of desgining CVs.\n", "Unfortunately, it typically comes at a higher computational cost (i.e., slower training and evaluation fo the CV) and the underlying codebase is more complex (i.e., more complex models and data format.)\n", "\n", "In this tutorial, we show how GNN models can be used within `mlcolvar` to build CVs using the implemented CV methods.\n", "\n", "**NOTE**: the GNN-based require a specific interface for PLUMED, in which the graph is built in PLUMED on-the-fly. Updated source files for such interfaces and more info are available in the `mlcovlar/plumed_interfaces` folder.\n", "\n", "### Outline\n", "Typically, the process of constructing a GNN-based CV requires the following ingredients;\n", "1. A **dataset** of attributed connected graphs (nodes and edges), which are constructed from the atomic positions. The parameters of the dataset can also be used to initialize the model.\n", "2. A **GNN-model** to represent the CV. Different architectures can be used in this regard.\n", "3. A **CV method** and the associated **loss function**. These are all the methods implemented for *standard* machine-learning CVs, except for those based on autoencoders. \n", "\n", "---\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load data\n", "#### The inputs of GNNs CVs\n", "The input of GNN models are attributed and connected graphs, in which nodes (representing the atoms, in our case) are connected by edges (the lines of the graph).\n", "Nodes and edges are then assigned with scalar and, eventually, vector features that are then processed through the layers of the GNN.\n", "\n", "In the context of GNN-CVs, such graphs most likely are created directly from the atomic coordinates from a trajectory file and the connectivity between the nodes is determined according to a radial `cutoff`.\n", "\n", "#### Truncated graphs\n", "In some cases, graphs can be built focusing the attention on a subset of the whole system, e.g., a molecule on a surface, but still keeping into account the interaction with the environment, e.g., the surface.\n", "In this case, only the nodes from the `system_selection` will be used for the final pooling, whereas the nodes from the `enviroment_selection` will be used only to update the information through the layers.\n", "Moreover, to reduce the computational costs, only the atoms closer to the `system_selection` atoms will be included in the graphs, according to the set `cutoff` and a `buffer` value to ensure stability and continuity. \n", "For example, this setup is useful when treating solvent or surface interactions.\n", "\n", "#### Create dataset from trajectory files\n", "To make this process easier, in `mlcolvar` there is an util function to do this under-the-hood: `create_dataset_from_trajectories`, which is analogous to the `create_dataset_from_files` used with descriptors.\n", "\n", "The loading process is built on the external library [`MDTraj`](https://www.mdtraj.org/), which can natively load most common trajectory+topology format used in biophysics. One advantage of MDTraj, is that it comes with a simple and user friendly syntax for atom selection, which can be used also here.\n", "\n", "For applications not related to biological system (e.g., solids, surfaces, molecules), we support the use of the `.xyz` file format. In this case, a topology pdb file is created using `ase` so that the convenient MDTraj selection syntax can be used. The generated pdb file can also be used as a topolgy file when running the simualtions in PLUMED.\n", "\n", "Here, as an example, we load some data about the state A and B of Alanine Dipeptide." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/etrizio@iit.local/Bin/dev/mlcolvar/mlcolvar/data/graph/utils.py:64: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n", " graph_labels = torch.tensor( config.graph_labels, dtype=torch.get_default_dtype() ) if config.graph_labels is not None else None\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Dataset info:\n", " DictDataset( \"data_list\": 4000,\n", "\t metadata={\"atomic_numbers\": [6, 7, 8],\n", "\t\t \"cutoff\": 10.0,\n", "\t\t \"buffer\": 0.0,\n", "\t\t \"used_idx\": tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),\n", "\t\t \"used_names\": [ACE1-CH3, ACE1-C, ACE1-O, ALA2-N, ALA2-CA, ALA2-CB, ALA2-C, ALA2-O, NME3-N, NME3-C],\n", "\t\t \"data_type\": graphs } )\n", "\n", "Datamodule info:\n", " DictModule(dataset -> DictDataset( \"data_list\": 4000,\n", "\t metadata={\"atomic_numbers\": [6, 7, 8],\n", "\t\t \"cutoff\": 10.0,\n", "\t\t \"buffer\": 0.0,\n", "\t\t \"used_idx\": tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),\n", "\t\t \"used_names\": [ACE1-CH3, ACE1-C, ACE1-O, ALA2-N, ALA2-CA, ALA2-CB, ALA2-C, ALA2-O, NME3-N, NME3-C],\n", "\t\t \"data_type\": graphs } ),\n", "\t train_loader -> DictLoader(length=0.8, batch_size=4000, shuffle=True),\n", "\t valid_loader -> DictLoader(length=0.2, batch_size=4000, shuffle=True))\n" ] } ], "source": [ "from mlcolvar.data import DictModule\n", "from mlcolvar.io import create_dataset_from_trajectories\n", "\n", "# loading arguments \n", "# same as to laod_dataframe\n", "load_args = [{'start' : 0, 'stop' : 10000, 'stride' : 5},\n", " {'start' : 0, 'stop' : 10000, 'stride' : 5}]\n", "\n", "# create dataset\n", "dataset = create_dataset_from_trajectories(\n", " trajectories=[\"alad_A.trr\", \n", " \"alad_B.trr\"],\n", " topologies=\"alad.gro\", \n", " folder=\"data/alanine_gnn\", \n", " cutoff=10.0, # Angstrom \n", " system_selection='all and not type H',\n", " show_progress=False,\n", " load_args=load_args,\n", " lengths_conversion=10.0, # MDTraj uses nm by default, we use Angstroms\n", " )\n", "print('Dataset info:\\n', dataset, end=\"\\n\\n\")\n", "\n", "# load dataset into a DictModule\n", "datamodule = DictModule(dataset=dataset)\n", "print('Datamodule info:\\n', datamodule)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Accessing graph data\n", "The built graphs are then stored as `torch_geometric.Data` objects into the usual `DictDataset` with the information about each graph entry (e.g., nodes positons, edges, weights, elabels etc.) under the key `data_list` and the common information for all the graphs (e.g., map from types to chemical species, cutoff) in the `metadata` attribute dictionary." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Example of a graph entry:\n", " Data(edge_index=[2, 90], shifts=[90, 3], unit_shifts=[90, 3], positions=[10, 3], cell=[3, 3], node_attrs=[10, 3], graph_labels=[1, 1], n_system=[1, 1], n_env=[1, 1], system_masks=[10, 1], weight=1.0, names_idx=[10])\n", "\n", "Dataset metadata:\n", " {'atomic_numbers': [6, 7, 8], 'cutoff': 10.0, 'buffer': 0.0, 'used_idx': tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), 'used_names': [ACE1-CH3, ACE1-C, ACE1-O, ALA2-N, ALA2-CA, ALA2-CB, ALA2-C, ALA2-O, NME3-N, NME3-C], 'data_type': 'graphs'}\n" ] } ], "source": [ "print('Example of a graph entry:\\n', dataset['data_list'][0], end='\\n\\n')\n", "print('Dataset metadata:\\n', dataset.metadata)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Initializing the GNN model\n", "At variance with the procedure with FFNNs, here the model is initialized **outside** the CV class, to which is then passed only later as an input.\n", "GNN architectures are indeed much more complex than FFNNs and have many parameters that can be set.\n", "In addition, when introducing GNN models into the code, we maintained the standard CVs as the default, which still covers most of the users.\n", "\n", "Here, for example, we initialize a `SchNetModel`.\n", "Many other architectures are available in [`pytorch_geometric`](https://pytorch-geometric.readthedocs.io/en/latest/) and can be readily adapted to this library.\n", "\n", "#### NOTE\n", "As the input graph are built with the dataset and then processed in the GNN-model, we recommend initializing the model passing a dataset to the `dataset_for_initialization` keyword.\n", "This way, the values stored in the `dataset.metadata` (e.g., `cutoff`, `atomic_numbers`, `buffer`) will be infered from the dataset and registered in the model.\n", "This avoids possible mismatches and errors between the graphs in the training dataset, the model architecture and the graphs built in PLUMED during the simulations (see the `mlcovlar/plumed_interfaces` folder).\n", "\n", "Nonetheless, the `cutoff`, `atomic_numbers` and `buffer` variables can also be set manually setting `dataset_for_initialization=None` *at you own risk*." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from mlcolvar.core.nn.graph.schnet import SchNetModel\n", "\n", "gnn_model = SchNetModel(n_out=1,\n", " dataset_for_initialization=dataset,\n", " pooling_operation=\"mean\",\n", " n_bases=16,\n", " n_layers=2,\n", " n_filters=16,\n", " n_hidden_channels=16,\n", " w_out_after_pool=True,\n", " aggr='mean'\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Initializing CV class\n", "The initalization of the CV class is almost identical to the standard case, with the only difference that we provide the initialized GNN object as model.\n", "\n", "Here, for example, we use the `DeepTDA` CV." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/etrizio@iit.local/Bin/miniconda3/envs/graph_mlcolvar_test_2.5/lib/python3.9/site-packages/lightning/pytorch/utilities/parsing.py:198: Attribute 'model' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['model'])`.\n" ] } ], "source": [ "import torch\n", "from mlcolvar.cvs import DeepTDA\n", "\n", "# we can still set the options for the optimizer the usual way\n", "# options for the BLOCKS of the cv are disabled when passing an external model\n", "options = {'optimizer' : {'lr' : 1e-3},\n", " 'lr_scheduler': {\n", " 'scheduler': torch.optim.lr_scheduler.ExponentialLR,\n", " 'gamma': 0.9999}\n", " }\n", "\n", "model = DeepTDA(n_states=2,\n", " n_cvs=1,\n", " target_centers=[-7, 7],\n", " target_sigmas=[0.2, 0.2],\n", " model=gnn_model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Training the CV\n", "Here, everything works the same!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from lightning import Trainer\n", "from mlcolvar.utils.trainer import MetricsCallback\n", "from mlcolvar.utils.plot import plot_metrics\n", "import matplotlib.pyplot as plt\n", "\n", "# define callbacks\n", "metrics = MetricsCallback()\n", "\n", "# here the number of epochs is low for testing, you should increase it for applications\n", "trainer = Trainer(\n", " callbacks=[metrics],\n", " logger=False,\n", " enable_checkpointing=False,\n", " max_epochs=5,\n", " enable_model_summary=False\n", ")\n", "\n", "trainer.fit(model, datamodule)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot training metrics" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAE8CAYAAAAmDQ2PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABNjklEQVR4nO3deVxU9f7H8dcZlmHflFVRcBcFXFAktVJRXMut3LqpdTPXtLT7yyy1siwtr1lXy0ptc09Kc0nF3RD3XXFXEgEV2Xfm/P4YnZrUBAUGmM/z8ZiHnO85c/h8oebN2b5fRVVVFSGEEGZJY+oChBBCmI6EgBBCmDEJASGEMGMSAkIIYcYkBIQQwoxJCAghhBmTEBBCCDMmISCEEGZMQkAIIcyYhIAQReDn58eQIUNMXYYQJU5CQJSZRYsWoSgK+/fvN3UpQojbLE1dgBAVQWxsLBqN/M0kKh/5r1qYnYKCAvLy8or1Hq1Wi5WVVSlVZFqZmZmmLkGYkISAKHeuXr3KCy+8gKenJ1qtlkaNGrFgwQKjbfLy8pg8eTLNmzfH2dkZe3t72rZty9atW422u3TpEoqi8PHHHzN79mxq166NVqvl5MmTTJ06FUVROHfuHEOGDMHFxQVnZ2eGDh1KVlaW0X7+fk3gzqmt3bt389prr+Hu7o69vT29evXi+vXrRu/V6XRMnToVHx8f7OzsaNeuHSdPnizydQadTsenn35KYGAgNjY2uLu707lzZ8NptTt9XLRo0V3vVRSFqVOnGpbv9PnkyZMMHDgQV1dX2rRpw8cff4yiKFy+fPmufUycOBFra2tu3bplaIuJiaFz5844OztjZ2fHE088we7du43el56ezrhx4/Dz80Or1eLh4UHHjh05ePDgA/ssyo6cDhLlSmJiIq1atUJRFEaPHo27uzvr16/nxRdfJC0tjXHjxgGQlpbG119/zYABA3jppZdIT0/nm2++ISIigr1799KkSROj/S5cuJCcnByGDRuGVqvFzc3NsO7ZZ5/F39+f6dOnc/DgQb7++ms8PDz46KOPHljvmDFjcHV1ZcqUKVy6dInZs2czevRoli1bZthm4sSJzJgxgx49ehAREcGRI0eIiIggJyenSD+TF198kUWLFtGlSxf+/e9/U1BQwM6dO9mzZw8hISFF2sffPfPMM9StW5cPPvgAVVXp3r07//nPf1i+fDmvv/660bbLly+nU6dOuLq6ArBlyxa6dOlC8+bNmTJlChqNhoULF9K+fXt27txJy5YtARg+fDgrV65k9OjRBAQEcPPmTXbt2sWpU6do1qzZQ9UtSoEqRBlZuHChCqj79u277zYvvvii6u3trd64ccOovX///qqzs7OalZWlqqqqFhQUqLm5uUbb3Lp1S/X09FRfeOEFQ9vFixdVQHVyclKTkpKMtp8yZYoKGG2vqqraq1cvtUqVKkZtNWvWVAcPHnxXX8LDw1WdTmdof/XVV1ULCws1JSVFVVVVTUhIUC0tLdWePXsa7W/q1KkqYLTPe9myZYsKqK+88spd6+583zt9XLhw4V3bAOqUKVPu6vOAAQPu2jYsLExt3ry5UdvevXtVQP3uu+8M37Nu3bpqRESEUb+zsrJUf39/tWPHjoY2Z2dnddSoUf/YP2F6cjpIlBuqqvLTTz/Ro0cPVFXlxo0bhldERASpqamGUwkWFhZYW1sD+tMlycnJFBQUEBIScs/TDX369MHd3f2e33f48OFGy23btuXmzZukpaU9sOZhw4ahKIrRewsLCw2nVaKioigoKGDkyJFG7xszZswD9w3w008/oSgKU6ZMuWvdX79vcf29zwD9+vXjwIEDnD9/3tC2bNkytFotTz/9NACHDx/m7NmzDBw4kJs3bxp+P5mZmXTo0IEdO3ag0+kAcHFxISYmhvj4+IeuU5Q+CQFRbly/fp2UlBTmz5+Pu7u70Wvo0KEAJCUlGbb/9ttvCQoKwsbGhipVquDu7s7atWtJTU29a9/+/v73/b41atQwWr5z2uOv58Af9r13wqBOnTpG27m5uRm2/Sfnz5/Hx8fH6PRVSbjXz+OZZ55Bo9EYTmWpqsqKFSvo0qULTk5OAJw9exaAwYMH3/U7+vrrr8nNzTX8/GfMmMHx48fx9fWlZcuWTJ06lQsXLpRoP8Sjk2sCoty48xfkc889x+DBg++5TVBQEAA//PADQ4YMoWfPnrz++ut4eHhgYWHB9OnTjf6SvcPW1va+39fCwuKe7WoRZl59lPeWlPsdERQWFt73Pff6efj4+NC2bVuWL1/Om2++yZ49e7hy5YrRtZE7v6OZM2fedd3lDgcHB0B/raVt27ZERkayceNGZs6cyUcffcSqVavo0qVLUbsnSpmEgCg33N3dcXR0pLCwkPDw8H/cduXKldSqVYtVq1YZfQje67SJKdWsWROAc+fOGf31ffPmzSIdadSuXZvffvuN5OTk+x4N3DmiSElJMWq/150+D9KvXz9GjhxJbGwsy5Ytw87Ojh49ehjVA+Dk5PTA3xGAt7c3I0eOZOTIkSQlJdGsWTPef/99CYFyRE4HiXLDwsKCPn368NNPP3H8+PG71v/11ss7f4H/9S/umJgYoqOjS7/QYujQoQOWlpbMmzfPqP3zzz8v0vv79OmDqqq88847d62703cnJyeqVq3Kjh07jNbPnTu32PX26dMHCwsLlixZwooVK+jevTv29vaG9c2bN6d27dp8/PHHZGRk3PX+O7+jwsLCu07LeXh44OPjQ25ubrHrEqVHjgREmVuwYAEbNmy4q33s2LF8+OGHbN26ldDQUF566SUCAgJITk7m4MGDbN68meTkZAC6d+/OqlWr6NWrF926dePixYt88cUXBAQE3PPDyVQ8PT0ZO3Ysn3zyCU899RSdO3fmyJEjrF+/nqpVqz7w4m67du3417/+xZw5czh79iydO3dGp9Oxc+dO2rVrx+jRowH497//zYcffsi///1vQkJC2LFjB2fOnCl2vR4eHrRr145Zs2aRnp5Ov379jNZrNBq+/vprunTpQqNGjRg6dCjVqlXj6tWrbN26FScnJ9asWUN6ejrVq1enb9++BAcH4+DgwObNm9m3bx+ffPJJsesSpUdCQJS5v/9VfMeQIUOoXr06e/fu5d1332XVqlXMnTuXKlWq0KhRI6Nz00OGDCEhIYEvv/yS3377jYCAAH744QdWrFjBtm3byqgnRfPRRx9hZ2fHV199xebNmwkLC2Pjxo20adMGGxubB75/4cKFBAUF8c033/D666/j7OxMSEgIjz32mGGbyZMnc/36dVauXMny5cvp0qUL69evx8PDo9j19uvXj82bN+Po6EjXrl3vWv/kk08SHR3Ne++9x+eff05GRgZeXl6Ehoby8ssvA2BnZ8fIkSPZuHEjq1atQqfTUadOHebOncuIESOKXZMoPYpallewhBCA/vy9q6sr06ZNY9KkSaYuR5gxuSYgRCnLzs6+q2327NmA/q9qIUxJTgcJUcqWLVvGokWL6Nq1Kw4ODuzatYslS5bQqVMnWrduberyhJmTEBCilAUFBWFpacmMGTNIS0szXCyeNm2aqUsTQq4JCCGEOZNrAkIIYcYkBIQQwozJNQH046HEx8fj6Oj4SCMzCiFEeaGqKunp6fj4+Pzj1KgSAkB8fDy+vr6mLkMIIUpcXFwc1atXv+96CQHA0dER0P+w7gyZK4QQFVlaWhq+vr6Gz7f7kRDgz6F4nZycJASEEJXKg05xy4VhIYQwYxICQghhxiQEhBDCjEkICCGEGZMQEEIIMyYhIIQQZkxuEX1Isad2Y3V0EblYkadYkaexJl+xpkBjTaGFlkILLYqVLYqVHZbWdlhY22GtdUBr44CjgzNVnKriaO8kTygLIUxKQuAh5WbepL7yl4m0dbf/LQTyi7aPVJ0lSdiTbulMjp0XGtdauFVrRO1qdQwTqQshRGmSEHhINeqEEqu1RZefjZqXjVqQjZqfjaYwF6UgB01hDpqCHCx0uVgW5mKl5mOt5qNV87GhAEtFxVlTgDOpoEuFjCuQsRfiIL7QlgvWvhR6NycwuCNu9g6m7q4QopKS+QTQP17t7OxMampq2TwxrKrk5qSTcjOO9OQ/yLwVhyYtDteca3jpUrFU/vyVXC+05rC2Pu5NnyHYr76cPhJCFElRP9ckBDBBCPyDwtxM4s/HkBG3l+opx3FU8gDIVjVs1dSndtuXqO91/8GghBD35+fnx7hx4xg3btwj72vbtm20a9eOW7du4eLi8sj7K2lF/VyT00HljIXWHt+A9hDQHgoLSDyzjYJTv1CtIJGu6imubZ3ISo9uPP3kM1jJdQNhBp588kmaNGnC7NmzH3lf+/btw97e/tGLqkTkFtHyzMISz4bhVOs1h1vNxpCsOOBtkUvvG6uIXPk+SekZpq5QCJNTVZWCgoIibevu7o6dnV0pV1SxSAhUBIqCa522uPWaR3yVEDQKPKsc58CaqVxOTjZ1daKCUlWV7Px8k7yKehZ6yJAhbN++nU8//RRFUVAUhUWLFqEoCuvXr6d58+ZotVp27drF+fPnefrpp/H09MTBwYEWLVqwefNmo/35+fkZHVEoisLXX39Nr169sLOzo27duqxevfqhf6Y//fQTjRo1QqvV4ufnxyeffGK0fu7cudStWxcbGxs8PT3p27evYd3KlSsJDAzE1taWKlWqEB4eTmZm5kPXUlRyOqgisdTi0+E/3Dq+BqcT39PF8gob172LpvsUfF1cTV2dqGByCgoI+2K+Sb539PBh2FpZPXC7Tz/9lDNnztC4cWPeffddAE6cOAHAG2+8wccff0ytWrVwdXUlLi6Orl278v7776PVavnuu+/o0aMHsbGx1KhR477f45133mHGjBnMnDmTzz77jEGDBnH58mXc3NyK1acDBw7w7LPPMnXqVPr168fvv//OyJEjqVKlCkOGDGH//v288sorfP/99zz22GMkJyezc+dOAK5du8aAAQOYMWMGvXr1Ij09nZ07dxY5LB+FSY8EduzYQY8ePfDx8UFRFH7++Wej9aqqMnnyZLy9vbG1tSU8PJyzZ88abZOcnMygQYNwcnLCxcWFF198kYyMyn2axLVxD7Kbj6BAVehkHc/udZ+QnJVl6rKEKHHOzs5YW1tjZ2eHl5cXXl5ehmdo3n33XTp27Ejt2rVxc3MjODiYl19+mcaNG1O3bl3ee+89ateu/cC/7IcMGcKAAQOoU6cOH3zwARkZGezdu7fYtc6aNYsOHTrw9ttvU69ePYYMGcLo0aOZOXMmAFeuXMHe3p7u3btTs2ZNmjZtyiuvvALoQ6CgoIDevXvj5+dHYGAgI0eOxMGh9G8PN+mRQGZmJsHBwbzwwgv07t37rvUzZsxgzpw5fPvtt/j7+/P2228TERHByZMnsbGxAWDQoEFcu3aNTZs2kZ+fz9ChQxk2bBiLFy8u6+6UKYc67UgryMXp6AL6W55h/toveaHPWCz/YS5RIf7KxtKS6OHDTPa9H1VISIjRckZGBlOnTmXt2rWGD9Xs7GyuXLnyj/sJCgoyfG1vb4+TkxNJSUnFrufUqVM8/fTTRm2tW7dm9uzZFBYW0rFjR2rWrEmtWrXo3LkznTt3NpyGCg4OpkOHDgQGBhIREUGnTp3o27cvrq6lf4Rv0k+MLl26MG3aNHr16nXXOlVVmT17Nm+99RZPP/00QUFBfPfdd8THxxuOGE6dOsWGDRv4+uuvCQ0NpU2bNnz22WcsXbqU+Pj4Mu5N2XNq0JkUv84ADCzcy9LfNz/gHUL8SVEUbK2sTPIqiedd/n6Xz4QJE4iMjOSDDz5g586dHD58mMDAQPLy8v5xP1Z/Oy2lKAo6ne4+Wz88R0dHDh48yJIlS/D29mby5MkEBweTkpKChYUFmzZtYv369QQEBPDZZ59Rv359Ll68WOJ1/F25/bPx4sWLJCQkEB4ebmhzdnYmNDSU6OhoAKKjo3FxcTH6iyA8PByNRkNMTMx9952bm0taWprRq6JyCRlCsn1NHDSFBFxawaGrV01dkhAlytramsLCwgdut3v3boYMGUKvXr0IDAzEy8uLS5culX6BtzVs2JDdu3ffVVO9evUMp7AsLS0JDw9nxowZHD16lEuXLrFlyxZAHz6tW7fmnXfe4dChQ1hbWxMZGVnqdZfbEEhISADA09PTqN3T09OwLiEhAQ8PD6P1lpaWuLm5Gba5l+nTp+Ps7Gx4+fr6lnD1ZUijwe2J18nFimbWqRzYsYj8IvwPI0RF4efnR0xMDJcuXeLGjRv3/Su9bt26rFq1isOHD3PkyBEGDhxYKn/R38/48eOJiorivffe48yZM3z77bd8/vnnTJgwAYBff/2VOXPmcPjwYS5fvsx3332HTqejfv36xMTE8MEHH7B//36uXLnCqlWruH79Og0bNiz1usttCJSmiRMnkpqaanjFxcWZuqRH4+BBYeAAAPoqJ/jp4P2PgoSoaCZMmICFhQUBAQG4u7vf9xz/rFmzcHV15bHHHqNHjx5ERETQrFmzMquzWbNmLF++nKVLl9K4cWMmT57Mu+++y5AhQwBwcXFh1apVtG/fnoYNG/LFF1+wZMkSGjVqhJOTEzt27KBr167Uq1ePt956i08++YQuXbqUet3lZtgIRVGIjIykZ8+eAFy4cIHatWtz6NAhmjRpYtjuiSeeoEmTJnz66acsWLCA8ePHc+vWLcP6goICbGxsWLFixT2vNdxLeRo24qHpCkldMxbn3CSW51QnvO803OShGCHMVlE/18rtkYC/vz9eXl5ERUUZ2tLS0oiJiSEsLAyAsLAwUlJSOHDggGGbLVu2oNPpCA0NLfOaTUpjgWOrlwHorf2Dn2K2mrggIURFYNIQyMjI4PDhwxw+fBjQXww+fPgwV65cQVEUxo0bx7Rp01i9ejXHjh3j+eefx8fHx3C00LBhQzp37sxLL73E3r172b17N6NHj6Z///74+PiYrmMmovEM5JZzfSwVqBq3icRK/ryEEKVp+PDhODg43PM1fPhwU5dXYkx6OujOKHx/N3jwYBYtWoSqqkyZMoX58+eTkpJCmzZtmDt3LvXq1TNsm5yczOjRo1mzZg0ajYY+ffowZ86cYj1kUSlOB92mXj+FsnUK+arCvKr/4pUO3U1dkhAVUlJS0n3vHHRycrrrppTyRoaSLobKFAIAaRvexCntHMuyq9Px2fdxs7U1dUlCiDJW4a8JiIfn2KQfAN211/jlyIEHbC2EMGcSApWQ4hlEhtYde00h6bGbyC3iMLtCCPMjIVAZKQq2DboC0M3qMhvOnDFxQUKI8kpCoJKy8G9HvmJFbcssTp7YZupyhBDllIRAZWVtR4FvawACs05y7uZNExckhCiPJAQqMdu6+sH3OtjcYO3xIyauRgjTuNdsYn+fu+SvLl26hKIohueX/sm2bdtQFIWUlJRHrtNUZGaxysytLlnaqtjl3iDz4i7y2jyBtUxOL8zctWvXymSc/opCjgQqM0XBpk57ANpb/MHW8xdMXJAQpufl5YVWqzV1GeWGhEAlp/F7HICW1ilsO7HfxNWIckVVoSDHNK8iPqM6f/58fHx87hoS+umnn+aFF14o0uTyf/f300F79+6ladOm2NjYEBISwqFDh4r9o/yrijbZvJwOquzsPch1rYv21lnck49wLb073o6Opq5KlAeFubDqedN8797fgaXNAzd75plnGDNmDFu3bqVDhw6AfqiYDRs2sG7dOjIyMh5qcvk7MjIy6N69Ox07duSHH37g4sWLjB079qG7VREnm5cQMANa/yfg1lk62yTx6+lYXmoR8uA3CVEOuLq60qVLFxYvXmwIgZUrV1K1alXatWuHRqMhODjYsP17771HZGQkq1evZvTo0Q/c/+LFi9HpdHzzzTfY2NjQqFEj/vjjD0aMGPFQ9f51snmAevXqcfLkSWbOnMmQIUOMJpt3dHQ0TDgPxpPN16xZE4DAwMCHqqM4JATMgW8rdAcXEGCVwZzT+/h3SPMSmeNVVHAWWv1f5Kb63kU0aNAgXnrpJebOnYtWq+XHH3+kf//+aDSah55c/o5Tp04RFBSEjc2fRyV3hqp/GBVxsnm5JmAOtE6onkEANMm/wMH4ayYuSJQLiqI/JWOKVzH+COnRoweqqrJ27Vri4uLYuXMngwYNAh5+cnlTKY+TzUsImAkLv7YAdLFJ4peTJ01cjRBFZ2NjQ+/evfnxxx9ZsmQJ9evXN0wb+aiTyzds2JCjR4+Sk5NjaNuzZ89D11oRJ5uX00HmwqcFOo0VNS2zibt0iKy8x7GztjZ1VUIUyaBBg+jevTsnTpzgueeeM7TfmVy+R48eKIrC22+/XazJ5QcOHMikSZN46aWXmDhxIpcuXeLjjz9+6DrHjx9PixYteO+99+jXrx/R0dF8/vnnzJ07F9BPNn/hwgUef/xxXF1dWbdundFk81FRUXTq1AkPDw9iYmLKZLJ5ORIwF1Y2KNX0F4TbWV5j0/nzJi5IiKJr3749bm5uxMbGMnDgQEP7o04u7+DgwJo1azh27BhNmzZl0qRJfPTRRw9dZ0WcbF4mlaHyTSpzX1f3w+4ZJBVa86ZVL77u28fUFQkhSolMKiPu5tUEnZU9HhZ5cOOUDConhJAQMCsWlmiqhwLQxeY63x48bNp6hCjnzGGyeTkdhBmdDgJIOg7b3iVVZ0nnm61Z+dy/qOZcyfssxEOqyJPNF/VzTe4OMjdVA8DWFefsW7S0usFHO3byafeu8vCYEPfg4eFRrj/oS4KcDjI3Gg34PgZAL9tEdly6ROTJUyYuSghhKhIC5qhWOKDwpPYGtS0yeX/rNqLOyS2jQpgjCQFz5FQNqrcEYKp3MoWqyusbfuOXU3JEIIS5kRAwVw17g6IQmH+B12vbolNVpmzewo+HZRpKIcyJhIC5cvWHet0BGJi3m1ca68den7lzF/Ni9pb6GOZCiPJBQsCcNeoHrv4oeekMzVjLW81qAfDl3n18sH0HhcUYg0UIUTFJCJgzS2toOwmcqqNkJ9M3aSn/DfFFAVYcO86wn3/hSkqKqasUQpQiCQFzZ+ME7d4F94aQn027P5awvKkN9lYWHLgaT68fFvNO1Baupt77gRkhRMVWrkOgsLCQt99+G39/f2xtbalduzbvvfee0flqVVWZPHky3t7e2NraEh4eztmzZ01YdQWkdYDH34IabUAtpG78b2zxv8jw6go6VUfkyVM8/cOPvBO1lcSMDFNXK4QoQeU6BD766CPmzZvH559/zqlTp/joo4+YMWMGn332mWGbGTNmMGfOHL744gtiYmKwt7cnIiLCaJIIUQQWVhA6BlqMAEst2tSLDM/fxp6aJ5nik4ajmkPkyZP0/mExy48dRycXjoWoFMr12EHdu3fH09OTb775xtDWp08fbG1t+eGHH1BVFR8fH8aPH8+ECRMASE1NxdPTk0WLFtG/f/8ifR+zGjuoKLJvwdl1cH4T5GcBoFMs2KDW5r0kD7JVC5r7+PB+p3C8HB1NXKwQ4l4qxVDSjz32GFFRUZw5cwaAI0eOsGvXLsMkCxcvXiQhIYHw8HDDe5ydnQkNDSU6Ovq++83NzSUtLc3oJf7C1hWCBkGPLyBkOLjVQaMW0pUzRPkcIdQ2gwPx8Ty7ZBlbzl8wdbVCiEdQrkPgjTfeoH///jRo0AArKyuaNm3KuHHjDJNMJyQkAODp6Wn0Pk9PT8O6e5k+fTrOzs6Gl6+vb+l1oiKztIFa7SH8A2j7Bth7YleQzhfOh3jNI5W03BxeW7ee97dtJ6egwNTVCiEeQrkOgeXLl/Pjjz+yePFiDh48yLfffsvHH3/Mt99++0j7nThxIqmpqYZXXFxcCVVciXk3g04zwfcxFFXH88phFvtdxwodK44d57nlK2SSGiEqoHIdAq+//rrhaCAwMJB//etfvPrqq0yfPh0ALy8vABITE43el5iYaFh3L1qtFicnJ6OXKAIrG2g1FpoMAUVDQPYpour8gb+9BeduJvPc8pWsiz1j6iqFEMVQrkMgKysLjca4RAsLC3S3n2T19/fHy8uLqKgow/q0tDRiYmIICwsr01rNhqJAva7QdiJY2eKUfpGVHsfoVd2JnIIC3ty4iRk7dpJfWGjqSoUQRVCuQ6BHjx68//77rF27lkuXLhEZGcmsWbPo1asXAIqiMG7cOKZNm8bq1as5duwYzz//PD4+PvTs2dO0xVd2XsHQ/n2w98Qi6zqTieKdxlUBWHzkKC///As3MjNNXKQQ4kHK9S2i6enpvP3220RGRpKUlISPjw8DBgxg8uTJWFtbA/qHxaZMmcL8+fNJSUmhTZs2zJ07l3r16hX5+8gtoo8gNw12fww3ToOiIdb3KV44nEFmfj7u9vbM7BJBE29vU1cphNkp6udauQ6BsiIh8IgK82H/l3B5BwBpNdrzwnlHziWnYKnRMLVDe7o3qG/iIoUwL5XiOQFRQVhYQctREDgAAKcrW1hW/RLdavlSoNPx1qbNfB69R54yFqIckhAQJUNRoGEvCHsNLKyxSDzCNIvtvBJcB4Cv9x/gjQ0b5XkCIcoZCQFRsnxbQbupYOOCkhbHC6mrmNWmEZYaDRvPnWPU6jVk5uWZukohxG0SAqLkudXRP2Xs5AvZt2h/aRE/PBmAvZUVB67G8/LPv5AqA/wJUS5ICIjSYVcV2r8DVRtAfhYNTn3JkjY1cbGx4XhiEi+uiiQ5K8vUVQph9iQEROmxvj1PQbWWoCugxqmFLGnti7u9HeduJjP859VyRCCEiUkIiNJlaa2/WOz3BKg6vE8s4MfHalLVzo4zN28y4pfVpOfmmrpKIcyWhIAofRoNhIyAmm1B1eFx7Cu+b10TVxsbTiZdZ/SaX8mSi8VCmISEgCgbGg20GAU1WoNaiPfxr/j28bo4arUcuZbAq+vWy3hDQpiAhIAoOxoNtBwN1VuBrpAaJ75iYbsgbK0siYn7g3e3bEMeYBeibEkIiLKlsdDPZewZCAW51Dn5JZ890QyNorDm9Gm+2LvP1BUKYVYkBETZs7CCxyaAay3ITSfk/CLebd0MgC/37uPnkydNXKAQ5kNCQJiGla1+TgIHL8i6TveknxjetBEA07Zu5+DVeBMXKIR5kBAQpmPjDI9PAhsXSL3My+oeImrXokCnY8L6DSRmZJi6QiEqPQkBYVoOntDmP6CxQrl2kGk+N6lbpQrJ2dlMWLeBPLljSIhSJSEgTM+tDrQYAYDVmdV80cwNJ62WY4mJfLh9h4mLE6JykxAQ5UPNNtCgJwBVTnzLnLaBKMCqEydZefyESUsTojKTEBDlR+P+4NUECvNpcnkp40L1dwx9tH0HxxISTFubEJWUhIAoP+48TGbjCmlXeZ7DtK9Vi3ydjvHrNnBTRh0VosRJCIjyxcYJWo0BFJRLW/mggQ3+rq4kZWbynw2/yYViIUqYhIAofzwaQ0AfAGyOLGJOu5aGCWnGr1svcxULUYIkBET5FNAXqtSHgmx8z/zIrK4R2FhasvPSZT6P3iNjDAlRQiQERPmk0UDLUWChhesnCM05zhtPPA7AggMH+WD7DgkCIUqAhIAovxy9IPhf+q+PLqanWx6vtX4MBVhx7DgfbN9BoU5n0hKFqOgkBET5VrsjVA8FXT7EfMbz9bwNRwQrjh1n0qbNZOfnm7hIISouCQFRvikKhI4Fp+qQmwbb36dfQH0+6NQRBdhw5iwv//wLNzIzTV2pEBWShIAo/yws9QPN2bpCZiLsm0vXurWY3+tpnLRajiYk0mfxEvb98YepKxWiwpEQEBWDXRX9hWLFAuJ+h90f08LHi0V9e9PAvSqpObkM/3k1C/YfkFtIhSgGCQFRcXgGQesJYGEN1w7C3rnUctCyqG8futWvR6GqMid6D6NWryFZni4WokjKfQhcvXqV5557jipVqmBra0tgYCD79+83rFdVlcmTJ+Pt7Y2trS3h4eGcPXvWhBWLUuXTHFqN1V8ruLILtr2DjaJjWsdwprRvh42lJdFX4nh2yTL2XIkzdbVClHvlOgRu3bpF69atsbKyYv369Zw8eZJPPvkEV1dXwzYzZsxgzpw5fPHFF8TExGBvb09ERAQ5OTkmrFyUqmotoO0ksLKHlEuw4TWUW+fp1SiAH57tS203N25kZTHil9XM+T2afBlqQoj7UtRy/MTNG2+8we7du9m5c+c916uqio+PD+PHj2fChAkApKam4unpyaJFi+jfv3+Rvk9aWhrOzs6kpqbi5ORUYvWLUnb9JOyaAflZ+tnJmr0A1VuRnZ/PJ7t2G4agDvT05MOITlRzlt+tMB9F/Vwr10cCq1evJiQkhGeeeQYPDw+aNm3KV199ZVh/8eJFEhISCA8PN7Q5OzsTGhpKdHT0ffebm5tLWlqa0UtUQO4B0OVTcPKFnBT4fRacXIWthYa32j3JzC4RON6enKbf0mX8dkZOEwrxd+U6BC5cuMC8efOoW7cuv/32GyNGjOCVV17h22+/BSDh9hjznp6eRu/z9PQ0rLuX6dOn4+zsbHj5+vqWXidE6bJxhvbvQa3bfwgcXwpRkyAnlY516rCs/7MEe3uRkZfH//22kbc3RZGWm2vamoUoR8r16SBra2tCQkL4/fffDW2vvPIK+/btIzo6mt9//53WrVsTHx+Pt7e3YZtnn30WRVFYtmzZPfebm5tL7l8+CNLS0vD19ZXTQRWZqsK53/QhkJ+lD4eQl8EnhAKdji/37uPrfftRAU8HB6Z2aE9YDQl/UXmV6umguLg4/vjLgzl79+5l3LhxzJ8//2F2d1/e3t4EBAQYtTVs2JArV64A4OXlBUBiYqLRNomJiYZ196LVanFycjJ6iQpOUaBuZwifrp+8PicVds+Eo4uxVAsZ1SqUBX164+vsTGJGBiN+Wc37W7eRlZdn6sqFMKmHCoGBAweydetWQH9KpmPHjuzdu5dJkybx7rvvllhxrVu3JjY21qjtzJkz1KxZEwB/f3+8vLyIiooyrE9LSyMmJoawsLASq0NUII7eEDEL/Nvrjw5O/ww73oeMRJr6eLN8QD/6BwUCsOL4CfouWcq+P66atmYhTOihQuD48eO0bNkSgOXLl9O4cWN+//13fvzxRxYtWlRixb366qvs2bOHDz74gHPnzrF48WLmz5/PqFGjAFAUhXHjxjFt2jRWr17NsWPHeP755/Hx8aFnz54lVoeoYCys9KeCWo0FSxv9XUSb/gOJR7G1suKNJx7ny55P4e3oQHxaOi9F/sxH23fIQHTCLD1UCOTn56PVagHYvHkzTz31FAANGjTg2rVrJVZcixYtiIyMZMmSJTRu3Jj33nuP2bNnM2jQIMM2//nPfxgzZgzDhg2jRYsWZGRksGHDBmxsbEqsDlEBKQrUaA3tp4FrbcjPhu3T4OACKMgl1NeXFQMH0KeR/nTjkqPHeHbJUg5ejTdx4UKUrYe6MBwaGkq7du3o1q0bnTp1Ys+ePQQHB7Nnzx769u1rdL2gIpDnBCq5glw48j2c36hfdvCGkJf001gCv1++wjtbtpKYkYECDGwSzOhWodhaWZmuZiEeUVE/1x4qBLZt20avXr1IS0tj8ODBLFiwAIA333yT06dPs2rVqoev3AQkBMxEwhHYNw+yk/XLtTpA8GCwsiE9N5dPdu3m55OnAPB1dubd8A409fH+hx0KUX6VaggAFBYWkpaWZjSEw6VLl7Czs8PDw+NhdmkyEgJmJC8Dji2B85sBVT86achw8AoGYNely7y7ZStJmZkowIDgIIa1aIGLrZxeFBVLqYZAdnY2qqpiZ2cHwOXLl4mMjKRhw4ZEREQ8fNUmIiFghhKOwoH5kJmkX/YN0x8V2LmRlpvLxzt3sfrUaQAcrK15rkkwA5sE43T7WpgQ5V2phkCnTp3o3bs3w4cPJyUlhQYNGmBlZcWNGzeYNWsWI0aMeKTiy5qEgJkqyIGji+HcBv2yxgoa9oL6PcBSy++XrzD79985c+MmAFXt7HivY7g8ZCYqhFJ9WOzgwYO0bdsWgJUrV+Lp6cnly5f57rvvmDNnzsNVLERZs7TRDzoX/iE419TPY3xiOWx+E26e4bGaNVjavx8zOkfg5+JiGJn0lTVrORQvdxGJyuGhQiArKwtHR0cANm7cSO/evdFoNLRq1YrLly+XaIFClDq3WtBpBjR9AazsIC0Otk6Bw9+hyU2nU906LOn/LM80bgTAjkuXGPpTJFOjtnA1VQYfFBXbQ4VAnTp1+Pnnn4mLi+O3336jU6dOACQlJcnpFFEx3Rl2IuJjqNYSdIVw5ldYNxoOfIVt+hUmtXuSyEED6XV7KJOfT56i+3ff89ra9VxMvmXa+oV4SA91TWDlypUMHDiQwsJC2rdvz6ZNmwD96Jw7duxg/fr1JV5oaZJrAsKIqupvJz2+FG5d0Lcpin6k0qBBYGXHofhrfLl3H3vi9LOXKUDrmjXp27gRbf1qYqEp1wP0CjNQ6reIJiQkcO3aNYKDg9Hc/g9+7969ODk50aBBg4er2kQkBMQ9qSrE79dfOE48pm9TFP2RQkAfcPHjfHIyn/0ezbaLlwxvq+7kRP/gILrWr4ebra1pahdmr9RD4I47TwdXr179UXZjUhIC4oGuHYRDCyHjLyPWVm2gn/PY70ku56isOnGSn0+eJDVHP0y5pUZD65o16BUQQBu/mljK0YEoQ6UaAjqdjmnTpvHJJ5+QkZEBgKOjI+PHj2fSpEmGI4OKQkJAFImqQmocnFoFcdHA7f91LKzBuxl4NyW7aiC/Xkog8uQpTiYlGd7q6eBA9wb16Vq/HrXd3ExTvzArpRoCEydO5JtvvuGdd96hdevWAOzatYupU6fy0ksv8f777z985SYgISCKLTMJrh2Ci9vg1vm/rFDAvQF4NyfOzp8Vl1NZfTqWlJwcwxb1q1ala/16dK5XF08HhzIvXZiHUg0BHx8fvvjiC8PooXf88ssvjBw5kqtXK9b47BIC4qGpKiSf158uunbob4EA2Fah0DOII3izJAG2xl2jQKcDQKMohNXwpVdAAI/7+2FtYWGCDojKqlRDwMbGhqNHj1KvXj2j9tjYWJo0aUJ2dnbxKzYhCQFRYjJvwNW9kHAYrp+Awr/MUaCxoMClDrEaLxbftGNtQpZhlaNWS6c6teneoD5NvL1RFKXsaxeVSqmGQGhoKKGhoXc9HTxmzBj27t1LTExM8Ss2IQkBUSoK8vQT2iQc0h8lZCQYrc5xrUe0Wp3//aFwLrPA0O7n4kKfxo3o3qA+rnJ3kXhIpRoC27dvp1u3btSoUcMwjWN0dDRxcXGsW7fOMKRERSEhIMpEegIkHYM/9kLiUe5cWFYVC9Id/dhaUI3//qEhJV/fbqXR0KFObfo0CiCkWjU5OhDFUuq3iMbHx/O///2P06f1Iy02bNiQYcOGMW3atBKfcL60SQiIMpd5XX+HUdzvfz6QBqgWWs47BTHvpitR13MN7b7OzvRuFMBTDRtQ5fbovUL8kzJ7TuCvjhw5QrNmzSgsLCypXZYJCQFhUhkJcHkXXNwCWTcMzZku9dha4M2cOEjK07dZaTT0bBTAC82b4X17/C4h7kVCoBgkBES5oKr600VnN8C1A/pl9KeLrjnU5ft0D5YkAihYajR0qluH/kGBBHp6yqkicRcJgWKQEBDlTuYNuLwdLm79c+IbIN3Rn/lZ/nwf/+f/YwEeHgxrEcIT/n4SBsJAQqAYJAREuXb9JJzbCH/sAVX/jEG2gy8bdX78N04hpUD/wR/g4c6I0Ja0qVlTwkCUTgj07t37H9enpKSwfft2CQEhSkN6gn5464tR+qGuAZ2VPXtsmzDpkhW3bt9VFOjpyYjQloTV8JUwMGOlEgJDhw4t0nYLFy4s6i7LBQkBUaHkpMD5TXB+I+SkAlBoW4UNNi15/1wuWQX6gKjh4kzPhg3pG9hY5kY2QyY5HVRRSQiICqkgDy5sgtOrIUc/qU2BY3XWWgQy60I2qbePDLwdHfigUyea+nibslpRxiQEikFCQFRo+VlwZh2cWQv5mYbmZNtqfJxag3W3rLFQFAYGB9EvKJDqzs4mLFaUFQmBYpAQEJVCbgac/ll/dJD/5/hdK22fYNol/dfWFhaMCWvFoCbBaOR6QaUmIVAMEgKiUsnLgtjV+nkPbkt0a8qi1CosidcPaNfcx4exrcMI8vIyVZWilEkIFIOEgKiUCnLh8LdwYbOh6bxrCM+fdyIzX3/x+JnAxkxo0xqtpaWpqhSlpKifaxVrCjAhRNFZaiFkGDz2Gti5A1D71n42N0hhYP0aAKw4dpznV/zE6evXTVmpMCE5EkCOBIQZUFWIXQPHfjQMR3HLuT79r/iSmJOPhaLwfNOmvNSiOXbW1iYuVpSESnkk8OGHH6IoCuPGjTO05eTkMGrUKKpUqYKDgwN9+vQhMTHx/jsRwhwpCjR4CjpMB9daALimxrLO+wi9a3lRqKosPHiQp39YzG9nzpq4WFGWKkwI7Nu3jy+//JKgoCCj9ldffZU1a9awYsUKtm/fTnx8/AOfbBbCbLnVgvDp0GIkWFhjkXWdyXmr+SnEgRpODlzPzOT/ftvI9G07DNNgisqtQoRARkYGgwYN4quvvsLV1dXQnpqayjfffMOsWbNo3749zZs3Z+HChfz+++/s2bPnvvvLzc0lLS3N6CWE2VAU8H8SOs3UHxXkZ1M7bi2rbdcyM0A/V8GyY8cYv2492fn5/7grUfFViBAYNWoU3bp1Izw83Kj9wIED5OfnG7U3aNCAGjVqEB0dfd/9TZ8+HWdnZ8PL19e31GoXotxy9IbwDyCgr6Gp4831bKlzFTsLhe0XLzH0p1Wcu3nThEWK0lbuQ2Dp0qUcPHiQ6dOn37UuISEBa2trXFxcjNo9PT1JSEi4a/s7Jk6cSGpqquEVFxdX0mULUTEoGmj8rP4U0W1u6edYX+8G7jbWnL5+gwFLl/P+1m1cSUkxXZ2i1JTrEIiLi2Ps2LH8+OOP2NjYlNh+tVotTk5ORi8hzJpbbej9PVSpB4Bz8nHWBGQS7ledfJ2OFcdP0PvHJSw6eNDEhYqSVq5D4MCBAyQlJdGsWTMsLS2xtLRk+/btzJkzB0tLSzw9PcnLyyPlb3+hJCYm4iVPQgpRPJZaaP8uNO4PgM3V35lZ8DM/talBpxoeFOh0fLo7mqP/cJQtKp5yHQIdOnTg2LFjHD582PAKCQlh0KBBhq+trKyIiooyvCc2NpYrV64QFhZmwsqFqKAUDQT0hrZvgNYRJT+T2me/Z0buChb5XsUCHW9viuLg1XjkEaPKoVw/K+7o6Ejjxo2N2uzt7alSpYqh/cUXX+S1117Dzc0NJycnxowZQ1hYGK1atTJFyUJUDt7N4PG3YdN/DE1N8s7xjLMdS1I0vLAqkl4BDZncvp1MXFPBlesjgaL473//S/fu3enTpw+PP/44Xl5erFq16sFvFEL8M1c/6LkQ3AMMTa+63+KpenWxUBQiT55i2bHjpqtPlAgZNgIZNkKIB7q8A2I+NyyedQyi3zkXFEXDiNCWDG7WFCsLCxMWKP6uUg4bIYQwkZqPQ4sRhsW66UcZV9uBQlXl8z0xDFm5isy8PBMWKB6WhIAQomj820HL0YbF5zPWMevxZjjbaDmRlMRbmzajkxMLFY6EgBCi6Pweh44fGhbbx/6XFYEKVhoNWy9c5KPtO+WuoQpGQkAIUTyutSBiFmj01wA8rvzGskYFKOjHHBr76zqSs7JMW6MoMgkBIUTxOVeHNv9nWKyVtJNPWgdgbWHBjkuX6P3jEmJkOJYKQUJACPFwvJpAxxmGxfbJm1jSuzv1qlQhJSeH8es2yOBzFYCEgBDi4bn6wZOT9U8aJ5+j9o7XWNzMnjBvDzLy8vi/DRvJLyw0dZXiH0gICCEejUdjaD/NsGh59Hv+Z7WRKrZazicn88XefSYsTjyIhIAQ4tFVqWP0HIEmN5Vl1c4DKt/sP8Dn0XvILSgwXX3iviQEhBAlw78dhLxsWKyafp4pQdUA+Hr/AXr+sJgDV+NNVZ24DwkBIUTJqdVBP3/xbb0sz/N+x3Dc7e25lp7OuLXruCyT05QrEgJCiJLl9wQ0eFr/9dW9dMvazZrenQjy9CQ9N5c3f9skD5SVIxICQoiSpSjQsA9Uvz2c+7kN2Gwcx0K7rbhYaTiRlMSmc+dNW6MwkBAQQpQ8KxsIGwcu/oYmi6zrfFYzBYCPd+7iaEKCHBGUAxICQojSoWig9QSjpsD0Q3RxLSQpM5PnV/zEq2vXkZaba6ICBUgICCFKk707dJph1DQpuBpd69fDUqNh28VLTNyw0UTFCZAQEEKUNhc/6PCBYdHh8mY+aNuC73p0xFKjYfeVKxyKl1tHTUVCQAhR+qrU+fPUUNZNWDOMgD1vMsVff03gs+g9MheBiUgICCHKhnczqNfdqKlHxjaaarM4GH+NMat/5XxysomKM18SAkKIsqGxhCbPg72nUfMrQTWxUBR2X7nCwKXL2XL+gokKNE8SAkKIsvXEJKPFpvkXWDagH4/VqEFuYSHvbtlKWk6OiYozPxICQoiy5eAFnWb+uXztIHVuxjCnfQtqu7mRkpPDPBl5tMxICAghyp5LTWg78c/lw4uwXP8KX3hfQUFl2dFj7PvjD9PVZ0YkBIQQpuHVBJoMNmpyv3GA4bVd0akqI35ZI0NQlwEJASGEaSgK1Ov25xhDtw1t5Ed4ndoU6HR8vf8Ab26UAedKk4SAEMK0/jIZDYB1zKd8/ERLZnaJwFKjIer8BVadOGmi4io/CQEhhGlZ2d41tASXttKxdm3GhOmPEmbu3MUFeYagVEgICCFMz8UPHn/rz+Xjy2BFP55PXkFYdR9yCgqYsH4DWXl5JiuxspIQEEKUD15BEPaaUZOSfI6Z9Sxwt7fjQvIt/h35C4kZGSYqsHIq1yEwffp0WrRogaOjIx4eHvTs2ZPY2FijbXJychg1ahRVqlTBwcGBPn36kJiYaKKKhRCPxLcVNPu3UZPDiR+Y+3hjXGxsOJmUxKBlKzhyLcFEBVY+5ToEtm/fzqhRo9izZw+bNm0iPz+fTp06kZmZadjm1VdfZc2aNaxYsYLt27cTHx9P7969TVi1EOKR1OkE9h5GTXXVG/zwbF/qVHHjRlYWY39dy7X0dBMVWLkoagW69+r69et4eHiwfft2Hn/8cVJTU3F3d2fx4sX07dsXgNOnT9OwYUOio6Np1arVPfeTm5tL7l8mskhLS8PX15fU1FScnJzKpC9CiH+QHg/rx/25bOcO3T4ju6CQF36K5NT16wR6erKgTy+sLCxMVmZ5lpaWhrOz8wM/18r1kcDfpaamAuDm5gbAgQMHyM/PJzw83LBNgwYNqFGjBtHR0ffdz/Tp03F2dja8fH19S7dwIUTxOPrAE2//uZx1HXZ+hO3Nk3wc0QEnrZZjiYl8smu36WqsJCpMCOh0OsaNG0fr1q1p3LgxAAkJCVhbW+Pi4mK0raenJwkJ9z9nOHHiRFJTUw2vuLi40ixdCPEwPAOhzf/9uZxwCHa8T7WoMUxv1xqApUePESnPEDySChMCo0aN4vjx4yxduvSR96XVanFycjJ6CSHKIZ/m0O5d4zZdAa2VK/w7pDkA727ZysrjJ0xQXOVQIUJg9OjR/Prrr2zdupXq1asb2r28vMjLyyMlJcVo+8TERLy8vMq4SiFEqXBvAE1fMG47+A2jGnozICgQFZi2dRubzp0zSXkVXbkOAVVVGT16NJGRkWzZsgV/f3+j9c2bN8fKyoqoqChDW2xsLFeuXCEsLKysyxVClJa6ncGpmlGTcu43/vN4W/oHBQIweXMUZ2/cNEV1FVq5DoFRo0bxww8/sHjxYhwdHUlISCAhIYHs7GwAnJ2defHFF3nttdfYunUrBw4cYOjQoYSFhd33ziAhRAX12Hjj5at7UXSFTGjbhtDq1cnOL2Dc2nWkZMuENMVRrm8RVRTlnu0LFy5kyJAhgP5hsfHjx7NkyRJyc3OJiIhg7ty5xTodVNRbqYQQJnbjDGx5y7gt6DlSanbiueUr+CMtjRbVqzH3qR5mf+toUT/XynUIlBUJASEqkOunYOsU47amQznn1pLnV/xEVn4+3erXY1rH8Pv+IWkOKuVzAkIIgXtD6P29fj6COw4tpI56nZldIrBQFNbGnuGz6D2mq7ECkRAQQlQ8ltq7rxFcO0zrmjWZ3KEdAAsOHLzvraOLjxxh3K/rOJ6YaPYT1kgICCEqpmotoXanP5dPrYIbsTzdsCEjQlsC8OH2HXfNVbw2NpYZO3ax7eJFnlu+krG/rivLqssdCQEhRMXVuD/wl9NCR3+A/CyGtQihc726FOh0TFi/gSu3nyXSqSozduw02sWOS5f43IxPHUkICCEqLq0D9FoEtq765RuxEDkEpTCXqR3a09jTg9ScXMb+uo603FyupadTkJvJCPtL1Lb4czTir/cf4MZfRic2JxICQoiKzcoWenwJNdr82bbqeWwKs/hvt654Ojhw8dYtJqxbT2zSdT5yPsXLDpf5qep+PnA6hbOSD0CGmc5aJiEghKgcWow0Xj69Gnd7ez7t3hU7Kyv2/nGV49vn00b751zFXW2TmOB4HoDYGzfKstpyQ0JACFE5WFhCy9F/LseuhqTjNHB359Pu3dBaWPCK48W73lbfUj9d5f9t2MjBq/FlVW25ISEghKg8araFmo//uXxwIagqLapXY1HfPvd8i72m0PD1C6siOXztWmlXWa5ICAghKg9F0c9RbGmrX06Lg9O/ANDQw/2eb3FQCqhhkYUG/fMC3x48VCallhcSAkKIysXKBp7+Cpxr6pePL4OEw3Cfh8KcNQWsrrqP/3PUD0W99cJF0v4y/WxlJyEghKh8LKyh0wxwDwC1EHZ8ACv6/eNb+tn9eT3gk527SrvCckNCQAhROSkKhI5+8Hb38Mup02SZyS2jEgJCiMrLriq0HPng7W5b4HqIVxwuANBp4bdm8QCZhIAQonKr+Tg4FG1+kWbWabxgHwfoHx4LX7CIq6lppVmdyUkICCEqN0UDoWOK9ZbFLapgpxQA0O277zlQiZ8fkBAQQlR+VepC43++MPxXAVdW8Uv1Pyeuf3FVJO9t2VoprxNICAghzINPi2Jt7p6XyDMN6xiWfzpxkse+/IpjCQklXZlJSQgIIcyDsy/U6Vyst0yy2s+PbfyN2v614qf7TlZTEckcw8gcw0KYlcwkyM8Bew9IOAQHvoK8jH98y7G2s/nXykijtkYeHnzduye2VlalWe1DkzmGhRDiXuw9wKWG/sli3zB4+huoUv8f3xKoJrBjSH+ebtjA0HYiKYmwL+az+tTpCj1FpRwJIEcCQpi9o4vh9M//vI2NC3SezfV8lZcjf+HCrVuGVc18vHm5ZQtCfX1LtcziKOrnmoQAEgJCmL2CHDj8nX6Moax/mFfAKxiyb0Gb/xBfaMP8fftZc+o0hbc/RkOrV2dAcBBP+PuhKMr991MGJASKQUJACAFAVjIcWwxJx/Qf9vfjHqC/0Fy7I1dx4YcjR1h57Dj5Oh0A1Zyc6B8USM+AhjhqtWVUvDEJgWKQEBBCGCnI0Q9BbWmrn7z+fjwa6Z8/cPEnPiuXHw4fYdWJk+QU6B80s9Jo6Fi3Dt0b1KdFtWpYWViUUQckBIpFQkAIcU+F+fDbBMjPgNz0+2/nVA3824NPc9Ksq/DziZOsOH6CuNRUwyZV7exo6+dH+9r+hPr6Yl3KgSAhUAwSAkKI+9Lp9COSRk2C5HMP3t7FH6q1QNewNwevXWPt6Vg2n79A+l/mKLC3sqJVDV9aVKtGSPVq1HJzQ1PC1xAkBIpBQkAI8UDZKZBzS3+K6Nx6sPfUnyrSFdz/PW51oeUo8vNziEnTsOPyZbZeuMj1v41O6myjJdjLm0AvTxp5eNDQwx1XW9tHKldCoBgkBIQQD6UgR/+g2bmN+n8vbQVd4b23dfAErROqZxDXcuFopoaYG9mcvnGd2FwtWkVHtvrnKSJ3e3tqu7lRp4obLapX4wl//3vv9z7MLgT+97//MXPmTBISEggODuazzz6jZcuWRXqvhIAQokQU5ulHLT26GFKvQOLRIr1NVTQoqo5bigOFugJ0Oh15qgYHTQHpOksuOjTk8acmFquUon6uWRZrr+XUsmXLeO211/jiiy8IDQ1l9uzZREREEBsbi4eHh6nLE0KYCwtr/b9Nntf/m50CmYn6U0Z/xIDGAm6c1s93nB5/e95jFaVQPzqpq5oBCvCXa8YumgJwKL3bTCvFkUBoaCgtWrTg888/B0Cn0+Hr68uYMWN444037to+NzeX3L9cpElLS8PX11eOBIQQZef2MwXo8iE3FSxsIP0qWFjpjyYK88DKHvIzwdpRfwdSMZjNkUBeXh4HDhxg4sQ/D5U0Gg3h4eFER0ff8z3Tp0/nnXfeKasShRDibprbQ7dptGB5+4yFTdn/EVrhB5C7ceMGhYWFeHp6GrV7enqScJ9xvydOnEhqaqrhFRcXVxalCiFEuVPhjwQehlarRWuiR7mFEKI8qfBHAlWrVsXCwoLExESj9sTERLy8ija5tBBCmKsKHwLW1tY0b96cqKgoQ5tOpyMqKoqwsDATViaEEOVfpTgd9NprrzF48GBCQkJo2bIls2fPJjMzk6FDh5q6NCGEKNcqRQj069eP69evM3nyZBISEmjSpAkbNmy462KxEEIIY5XiOYFHJU8MCyEqG7N5TqAk3MnBtLQ0E1cihBAl487n2YP+zpcQANLT9eOE+5aj+UGFEKIkpKen4+zsfN/1cjoI/d1E8fHxODo6Fmte0DvDTcTFxVXK00iVvX9Q+ftY2fsHlb+PD9s/VVVJT0/Hx8cHjeb+N4LKkQD6YSaqV6/+0O93cnKqlP/x3VHZ+weVv4+VvX9Q+fv4MP37pyOAOyr8cwJCCCEenoSAEEKYMQmBR6DVapkyZUqlHYeosvcPKn8fK3v/oPL3sbT7JxeGhRDCjMmRgBBCmDEJASGEMGMSAkIIYcYkBIQQwoxJCDyk//3vf/j5+WFjY0NoaCh79+41dUlFtmPHDnr06IGPjw+KovDzzz8brVdVlcmTJ+Pt7Y2trS3h4eGcPXvWaJvk5GQGDRqEk5MTLi4uvPjii2RkZJRhL+5v+vTptGjRAkdHRzw8POjZsyexsbFG2+Tk5DBq1CiqVKmCg4MDffr0uWtioitXrtCtWzfs7Ozw8PDg9ddfp6CgoCy7ck/z5s0jKCjI8PBQWFgY69evN6yvyH27lw8//BBFURg3bpyhraL3cerUqSiKYvRq0KCBYX2Z9k8VxbZ06VLV2tpaXbBggXrixAn1pZdeUl1cXNTExERTl1Yk69atUydNmqSuWrVKBdTIyEij9R9++KHq7Oys/vzzz+qRI0fUp556SvX391ezs7MN23Tu3FkNDg5W9+zZo+7cuVOtU6eOOmDAgDLuyb1FRESoCxcuVI8fP64ePnxY7dq1q1qjRg01IyPDsM3w4cNVX19fNSoqSt2/f7/aqlUr9bHHHjOsLygoUBs3bqyGh4erhw4dUtetW6dWrVpVnThxoim6ZGT16tXq2rVr1TNnzqixsbHqm2++qVpZWanHjx9XVbVi9+3v9u7dq/r5+alBQUHq2LFjDe0VvY9TpkxRGzVqpF67ds3wun79umF9WfZPQuAhtGzZUh01apRhubCwUPXx8VGnT59uwqoezt9DQKfTqV5eXurMmTMNbSkpKapWq1WXLFmiqqqqnjx5UgXUffv2GbZZv369qiiKevXq1TKrvaiSkpJUQN2+fbuqqvr+WFlZqStWrDBsc+rUKRVQo6OjVVXVB6VGo1ETEhIM28ybN091cnJSc3Nzy7YDReDq6qp+/fXXlapv6enpat26ddVNmzapTzzxhCEEKkMfp0yZogYHB99zXVn3T04HFVNeXh4HDhwgPDzc0KbRaAgPDyc6OtqElZWMixcvkpCQYNQ/Z2dnQkNDDf2Ljo7GxcWFkJAQwzbh4eFoNBpiYmLKvOYHSU1NBcDNzQ2AAwcOkJ+fb9THBg0aUKNGDaM+BgYGGk1MFBERQVpaGidOnCjD6v9ZYWEhS5cuJTMzk7CwsErVt1GjRtGtWzejvkDl+f2dPXsWHx8fatWqxaBBg7hy5QpQ9v2TAeSK6caNGxQWFt41a5mnpyenT582UVUlJyEhAeCe/buzLiEhAQ8PD6P1lpaWuLm5GbYpL3Q6HePGjaN169Y0btwY0NdvbW2Ni4uL0bZ/7+O9fgZ31pnasWPHCAsLIycnBwcHByIjIwkICODw4cMVvm8AS5cu5eDBg+zbt++udZXh9xcaGsqiRYuoX78+165d45133qFt27YcP368zPsnISAqtVGjRnH8+HF27dpl6lJKVP369Tl8+DCpqamsXLmSwYMHs337dlOXVSLi4uIYO3YsmzZtwsbGxtTllIouXboYvg4KCiI0NJSaNWuyfPlybG1ty7QWOR1UTFWrVsXCwuKuK/WJiYl4eXmZqKqSc6cP/9Q/Ly8vkpKSjNYXFBSQnJxcrn4Go0eP5tdff2Xr1q1GQ4V7eXmRl5dHSkqK0fZ/7+O9fgZ31pmatbU1derUoXnz5kyfPp3g4GA+/fTTStG3AwcOkJSURLNmzbC0tMTS0pLt27czZ84cLC0t8fT0rPB9/DsXFxfq1avHuXPnyvx3KCFQTNbW1jRv3pyoqChDm06nIyoqirCwMBNWVjL8/f3x8vIy6l9aWhoxMTGG/oWFhZGSksKBAwcM22zZsgWdTkdoaGiZ1/x3qqoyevRoIiMj2bJlC/7+/kbrmzdvjpWVlVEfY2NjuXLlilEfjx07ZhR2mzZtwsnJiYCAgLLpSDHodDpyc3MrRd86dOjAsWPHOHz4sOEVEhLCoEGDDF9X9D7+XUZGBufPn8fb27vsf4fFvqwt1KVLl6parVZdtGiRevLkSXXYsGGqi4uL0ZX68iw9PV09dOiQeujQIRVQZ82apR46dEi9fPmyqqr6W0RdXFzUX375RT169Kj69NNP3/MW0aZNm6oxMTHqrl271Lp165abW0RHjBihOjs7q9u2bTO6BS8rK8uwzfDhw9UaNWqoW7ZsUffv36+GhYWpYWFhhvV3bsHr1KmTevjwYXXDhg2qu7t7ubjF8I033lC3b9+uXrx4UT169Kj6xhtvqIqiqBs3blRVtWL37X7+eneQqlb8Po4fP17dtm2bevHiRXX37t1qeHi4WrVqVTUpKUlV1bLtn4TAQ/rss8/UGjVqqNbW1mrLli3VPXv2mLqkItu6dasK3PUaPHiwqqr620Tffvtt1dPTU9VqtWqHDh3U2NhYo33cvHlTHTBggOrg4KA6OTmpQ4cOVdPT003Qm7vdq2+AunDhQsM22dnZ6siRI1VXV1fVzs5O7dWrl3rt2jWj/Vy6dEnt0qWLamtrq1atWlUdP368mp+fX8a9udsLL7yg1qxZU7W2tlbd3d3VDh06GAJAVSt23+7n7yFQ0fvYr18/1dvbW7W2tlarVaum9uvXTz137pxhfVn2T4aSFkIIMybXBIQQwoxJCAghhBmTEBBCCDMmISCEEGZMQkAIIcyYhIAQQpgxCQEhhDBjEgJCCGHGJASEqADuNQ2oECVBQkCIBxgyZMhd88EqikLnzp1NXZoQj0zmExCiCDp37szChQuN2rRarYmqEaLkyJGAEEWg1Wrx8vIyerm6ugL6UzXz5s2jS5cu2NraUqtWLVauXGn0/mPHjtG+fXtsbW2pUqUKw4YNIyMjw2ibBQsW0KhRI7RaLd7e3owePdpo/Y0bN+jVqxd2dnbUrVuX1atXl26nhVmQEBCiBLz99tv06dOHI0eOMGjQIPr378+pU6cAyMzMJCIiAldXV/bt28eKFSvYvHmz0Yf8vHnzGDVqFMOGDePYsWOsXr2aOnXqGH2Pd955h2effZajR4/StWtXBg0aRHJycpn2U1RCjzgiqhCV3uDBg1ULCwvV3t7e6PX++++rqqofunr48OFG7wkNDVVHjBihqqqqzp8/X3V1dVUzMjIM69euXatqNBrDHBQ+Pj7qpEmT7lsDoL711luG5YyMDBVQ169fX2L9FOZJrgkIUQTt2rVj3rx5Rm1ubm6Gr/8+q1xYWBiHDx8G4NSpUwQHB2Nvb29Y37p1a3Q6HbGxsSiKQnx8PB06dPjHGoKCggxf29vb4+TkdNc0n0IUl4SAEEVgb29/1+mZklLUicWtrKyMlhVFQafTlUZJwozINQEhSsCePXvuWm7YsCEADRs25MiRI2RmZhrW7969G41GQ/369XF0dMTPz89oTlkhyoocCQhRBLm5uSQkJBi1WVpaUrVqVQBWrFhBSEgIbdq04ccff2Tv3r188803AAwaNIgpU6YwePBgpk6dyvXr1xkzZgz/+te/8PT0BGDq1KkMHz4cDw8PunTpQnp6Ort372bMmDFl21FhdiQEhCiCDRs24O3tbdRWv359Tp8+Dejv3Fm6dCkjR47E29ubJUuWEBAQAICdnR2//fYbY8eOpUWLFtjZ2dGnTx9mzZpl2NfgwYPJycnhv//9LxMmTKBq1ar07du37DoozJbMMSzEI1IUhcjISHr27GnqUoQoNrkmIIQQZkxCQAghzJhcExDiEckZVVGRyZGAEEKYMQkBIYQwYxICQghhxiQEhBDCjEkICCGEGZMQEEIIMyYhIIQQZkxCQAghzNj/A6492kuZZIzxAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots(1,1,figsize=(4,3))\n", "plot_metrics(metrics.metrics,\n", " keys=['train_loss', 'valid_loss'],\n", " colors=['fessa1', 'fessa5'],\n", " yscale='linear',\n", " ax = ax)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Testing the model\n", "As the graph data are stored as `torch_geometric.Data` they need to be loaded using a loader object.\n", "For convenience, we implemented both in `DictDataset ` and `DictModule` a method `.get_graph_data` to do it so that one can simply evaluate the model calling either:\n", "- `model(dataset.get_graph_data())` --> Returns the **whole dataset**\n", "- `model(datamodule.get_graph_data())` --> Returns either the **train or valid dataset**" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz8AAAE8CAYAAADuXg/EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA0y0lEQVR4nO3deViVdf7/8ddhR5BDLoC4AJqpaJNbGjVlX6NIqan0N6ZjBcroWDijuZRelWul2ebkWOaMgd9LHcuWWbRFM7NSMrVsQcUszW2ATAGXBIHP74++nOHIIkeRczj383Fd9yXnc3/u+35/7hvO2/e573PfNmOMEQAAAAB4OR93BwAAAAAADYHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACN0v79+2Wz2ZSZmXlBy9tsNs2YMaNeY4Jno/iBx8rMzJTNZqt2mjJlirvDq1XlWP38/NSsWTP16tVL48aN086dOy94vadPn9aMGTP04Ycf1l+wF2Hz5s2aMWOGCgoK3B0KANSKnFKVp+UUoCH4uTsA4HxmzZqluLg4p7Zu3bq5KZq6u/nmm3XffffJGKPCwkJ9+eWXWrp0qV588UU99dRTmjBhgsvrPH36tGbOnClJuvHGG+s5Ytdt3rxZM2fOVGpqqsLDw90dDgCcFznlvzwtpwANgeIHHm/AgAHq3bt3nfqeOXNGAQEB8vFx/0nNK664Qvfcc49T29y5c3X77bdr4sSJ6ty5swYOHOim6ADAmsgpgLW5/68ZuEAffvihbDabVq5cqUcffVStW7dWkyZNVFRUJElatWqVevXqpeDgYLVo0UL33HOPDh8+7LSO1NRUhYaG6sCBA7rtttsUGhqq1q1ba+HChZKkr7/+Wv3791dISIhiYmK0YsWKi4q5efPmWrlypfz8/PTEE0842ktKSjRt2jT16tVLdrtdISEhuv7667VhwwZHn/3796tly5aSpJkzZzougai4Vvmrr75Samqq2rdvr6CgIEVFRWnkyJH66aefnGI4ceKExo8fr9jYWAUGBioiIkI333yzPv/8c6d+W7Zs0a233iq73a4mTZqoX79+2rRpk2P+jBkzNHnyZElSXFycI579+/df1D4CAHcgp1xYTpkxY4ZsNpv27Nmje+65R3a7XS1bttRjjz0mY4wOHjyoO+64Q2FhYYqKitKzzz5bZRz5+flKS0tTZGSkgoKCdNVVV2np0qVV+hUUFCg1NVV2u13h4eFKSUmp9rLrG2+8sdozWampqYqNjT3vfj18+LBGjhypyMhIBQYGqmvXrnrllVfOuxwaB878wOMVFhbq6NGjTm0tWrRw/Dx79mwFBARo0qRJKi4uVkBAgDIzMzVixAhdffXVmjNnjvLy8vTnP/9ZmzZt0hdffOF0iVZZWZkGDBigG264QfPmzdPy5cs1duxYhYSE6JFHHtHw4cM1aNAgLVq0SPfdd58SEhKqXDLhinbt2qlfv37asGGDioqKFBYWpqKiIv3tb3/TsGHDNGrUKJ04cUJLlixRUlKSPvvsM3Xv3l0tW7bUSy+9pPvvv1933XWXBg0aJEn61a9+JUlat26dvv/+e40YMUJRUVHKzs7W4sWLlZ2drU8//VQ2m02SNGbMGL3++usaO3as4uPj9dNPP+mTTz7Rrl271LNnT0nSBx98oAEDBqhXr16aPn26fHx8lJGRof79++vjjz9Wnz59NGjQIO3Zs0d///vf9fzzzzuOSUUyBQBPRE6p35xS4e6771aXLl00d+5crVmzRo8//riaNWuml19+Wf3799dTTz2l5cuXa9KkSbr66qt1ww03SJJ+/vln3Xjjjdq7d6/Gjh2ruLg4rVq1SqmpqSooKNC4ceMkScYY3XHHHfrkk080ZswYdenSRW+99ZZSUlIueN9VJy8vT9dcc41sNpvGjh2rli1b6p133lFaWpqKioo0fvz4et0e3MAAHiojI8NIqnYyxpgNGzYYSaZ9+/bm9OnTjuVKSkpMRESE6datm/n5558d7atXrzaSzLRp0xxtKSkpRpJ58sknHW3Hjx83wcHBxmazmZUrVzrad+/ebSSZ6dOnnzd2SSY9Pb3G+ePGjTOSzJdffmmMMaa0tNQUFxc79Tl+/LiJjIw0I0eOdLT9+OOPNcZQeR9U+Pvf/24kmY8++sjRZrfba42tvLzcdOzY0SQlJZny8nKn9cfFxZmbb77Z0fb0008bSWbfvn01rg8APAE55dLklOnTpxtJZvTo0Y620tJS06ZNG2Oz2czcuXOdYggODjYpKSmOtvnz5xtJZtmyZY62kpISk5CQYEJDQ01RUZExxph//OMfRpKZN2+e03auv/56I8lkZGQ42vv162f69etXJf6UlBQTExPj1Hbu+NPS0kyrVq3M0aNHnfoNHTrU2O32avcLGhcue4PHW7hwodatW+c0VZaSkqLg4GDH623btik/P18PPPCAgoKCHO3Jycnq3Lmz1qxZU2Ubv//97x0/h4eHq1OnTgoJCdGQIUMc7Z06dVJ4eLi+//77ix5TaGiopF8uQZMkX19fBQQESJLKy8t17NgxlZaWqnfv3lUuR6tJ5X1w5swZHT16VNdcc40kOa0jPDxcW7Zs0ZEjR6pdz44dO/Ttt9/qd7/7nX766ScdPXpUR48e1alTp3TTTTfpo48+Unl5ueuDBgAPQE6p35xSofKYfX191bt3bxljlJaW5miv2BeVx/z2228rKipKw4YNc7T5+/vrT3/6k06ePKmNGzc6+vn5+en+++932s4f//jHOo2nLowxeuONN3T77bfLGOPIf0ePHlVSUpIKCwvrvP/gubjsDR6vT58+tX459dzLBX744QdJvySWc3Xu3FmffPKJU1tQUFCVS7XsdrvatGlT5bS+3W7X8ePHXYq/OidPnpQkNW3a1NG2dOlSPfvss9q9e7fOnj3raK/r5RDHjh3TzJkztXLlSuXn5zvNKywsdPw8b948paSkqG3bturVq5cGDhyo++67T+3bt5ckffvtt5JU66UEhYWFuuyyy+oUFwB4EnJK/eaUCu3atXN6bbfbFRQU5HRJYUV75e8N/fDDD+rYsWOVm0p06dLFMb/i31atWjkKvQrVHZcL9eOPP6qgoECLFy/W4sWLq+1z7r5A40Pxg0av8qdTF8LX19eldmPMRW1Pkr755hv5+vo6ktCyZcuUmpqqO++8U5MnT1ZERIR8fX01Z84cfffdd3Va55AhQ7R582ZNnjxZ3bt3V2hoqMrLy3Xrrbc6nakZMmSIrr/+er311ltau3atnn76aT311FN68803NWDAAEffp59+Wt27d692W+cmHwDwFuSUX9Q1p1SobnyXcsy1sdls1W6jrKys1uUqxnXPPffU+AFgxXei0HhR/MDrxMTESJJycnLUv39/p3k5OTmO+e5y4MABbdy4UQkJCY5P6V5//XW1b99eb775ptMng9OnT3da9txPDSscP35c69ev18yZMzVt2jRHe8VZnHO1atVKDzzwgB544AHl5+erZ8+eeuKJJzRgwAB16NBBkhQWFqbExMRax1JTPADgLcgp588pFyMmJkZfffWVysvLnc7+7N692zG/4t/169fr5MmTTh/A5eTkVFnnZZddVu3lhBVnkWrSsmVLNW3aVGVlZefNf2i8+M4PvE7v3r0VERGhRYsWqbi42NH+zjvvaNeuXUpOTnZbbMeOHdOwYcNUVlamRx55xNFe8elY5U+qtmzZoqysLKflmzRpIklVbu1Z3fKSNH/+fKfXZWVlVS5XiIiIUHR0tGNf9erVSx06dNAzzzzjuJSish9//NHxc0hISLXxAIC3IKf817k5pT4MHDhQubm5evXVVx1tpaWlWrBggUJDQ9WvXz9Hv9LSUr300kuOfmVlZVqwYEGVdXbo0EG7d+92yldffvml0+MaquPr66vBgwfrjTfe0DfffFNlfuX1ofHizA+8jr+/v5566imNGDFC/fr107Bhwxy3JY2NjdWDDz7YIHHs2bNHy5YtkzFGRUVF+vLLL7Vq1SqdPHlSzz33nG699VZH39tuu01vvvmm7rrrLiUnJ2vfvn1atGiR4uPjnQqQ4OBgxcfH69VXX9UVV1yhZs2aqVu3burWrZvjtqpnz55V69attXbtWu3bt88pphMnTqhNmzb6f//v/+mqq65SaGio3n//fW3dutXx7AUfHx/97W9/04ABA9S1a1eNGDFCrVu31uHDh7VhwwaFhYXp3//+t6RfCiVJeuSRRzR06FD5+/vr9ttvdxRFANDYkVNqzin1YfTo0Xr55ZeVmpqq7du3KzY2Vq+//ro2bdqk+fPnO85m3X777bruuus0ZcoU7d+/X/Hx8XrzzTer/f7RyJEj9dxzzykpKUlpaWnKz8/XokWL1LVrV8dzm2oyd+5cbdiwQX379tWoUaMUHx+vY8eO6fPPP9f777+vY8eO1fs+QANzyz3mgDqouC3p1q1bq51fcVvSVatWVTv/1VdfNT169DCBgYGmWbNmZvjw4ebQoUNOfVJSUkxISEiVZfv162e6du1apT0mJsYkJyefN3ZVuoWqj4+PCQ8PNz169DDjxo0z2dnZVfqXl5ebJ5980sTExJjAwEDTo0cPs3r16mpvy7l582bTq1cvExAQ4HSLzkOHDpm77rrLhIeHG7vdbn7729+aI0eOOPUpLi42kydPNldddZVp2rSpCQkJMVdddZV58cUXq8T0xRdfmEGDBpnmzZubwMBAExMTY4YMGWLWr1/v1G/27NmmdevWxsfHh9teA/BY5JT6zynG/PdW1z/++OMF74u8vDwzYsQI06JFCxMQEGCuvPJKp1tXV/jpp5/Mvffea8LCwozdbjf33nuv+eKLL6rc6toYY5YtW2bat29vAgICTPfu3c17771Xp1tdV8STnp5u2rZta/z9/U1UVJS56aabzOLFi6vEhMbHZswl/tYZAAAAAHgAvvMDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJTTKh5yWl5fryJEjatq0qWw2m7vDAQDLMMboxIkTio6Olo8Pn59VRm4CAPdwJTc1yuLnyJEjatu2rbvDAADLOnjwoNq0aePuMDwKuQkA3KsuualRFj9NmzaV9MsAw8LC3BwNAFSv2/T3HD9/MzPJjZHUn6KiIrVt29bxPoz/IjcBaAysnpsaZfFTcTlBWFgYCQaAx/IJbOL42dveq7isqypyE4DGwOq5iQu2AQAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASXip+ysjI99thjiouLU3BwsDp06KDZs2fLGOPoY4zRtGnT1KpVKwUHBysxMVHffvut03qOHTum4cOHKywsTOHh4UpLS9PJkyfrZ0QAAEshNwEA6sql4uepp57SSy+9pL/85S/atWuXnnrqKc2bN08LFixw9Jk3b55eeOEFLVq0SFu2bFFISIiSkpJ05swZR5/hw4crOztb69at0+rVq/XRRx9p9OjR9TcqAIBlkJsAAHVlM5U/GjuP2267TZGRkVqyZImjbfDgwQoODtayZctkjFF0dLQmTpyoSZMmSZIKCwsVGRmpzMxMDR06VLt27VJ8fLy2bt2q3r17S5LeffddDRw4UIcOHVJ0dPR54ygqKpLdbldhYaHXPZwJgPeInbLG8fP+uclujKT+eOL7L7kJAOrO6rnJpTM/1157rdavX689e/ZIkr788kt98sknGjBggCRp3759ys3NVWJiomMZu92uvn37KisrS5KUlZWl8PBwR3KRpMTERPn4+GjLli3Vbre4uFhFRUVOEwAAErkJAFB3fq50njJlioqKitS5c2f5+vqqrKxMTzzxhIYPHy5Jys3NlSRFRkY6LRcZGemYl5ubq4iICOcg/PzUrFkzR59zzZkzRzNnznQlVACARZCbAAB15dKZn9dee03Lly/XihUr9Pnnn2vp0qV65plntHTp0ksVnyRp6tSpKiwsdEwHDx68pNsDADQe5CYAQF25dOZn8uTJmjJlioYOHSpJuvLKK/XDDz9ozpw5SklJUVRUlCQpLy9PrVq1ciyXl5en7t27S5KioqKUn5/vtN7S0lIdO3bMsfy5AgMDFRgY6EqoAACLIDcBAOrKpTM/p0+flo+P8yK+vr4qLy+XJMXFxSkqKkrr1693zC8qKtKWLVuUkJAgSUpISFBBQYG2b9/u6PPBBx+ovLxcffv2veCBAACsidwEAKgrl8783H777XriiSfUrl07de3aVV988YWee+45jRw5UpJks9k0fvx4Pf744+rYsaPi4uL02GOPKTo6WnfeeackqUuXLrr11ls1atQoLVq0SGfPntXYsWM1dOjQOt1NBwCAyshNAIC6cqn4WbBggR577DE98MADys/PV3R0tP7whz9o2rRpjj4PPfSQTp06pdGjR6ugoEC//vWv9e677yooKMjRZ/ny5Ro7dqxuuukm+fj4aPDgwXrhhRfqb1QAAMsgNwEA6sql5/x4Cp6lAKAxsPqzFKyGfQOgMbB6bnLpOz8AAAAA0FhR/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASXC5+Dh8+rHvuuUfNmzdXcHCwrrzySm3bts0x3xijadOmqVWrVgoODlZiYqK+/fZbp3UcO3ZMw4cPV1hYmMLDw5WWlqaTJ09e/GgAAJZEbgIA1IVLxc/x48d13XXXyd/fX++884527typZ599Vpdddpmjz7x58/TCCy9o0aJF2rJli0JCQpSUlKQzZ844+gwfPlzZ2dlat26dVq9erY8++kijR4+uv1EBACyD3AQAqCubMcbUtfOUKVO0adMmffzxx9XON8YoOjpaEydO1KRJkyRJhYWFioyMVGZmpoYOHapdu3YpPj5eW7duVe/evSVJ7777rgYOHKhDhw4pOjr6vHEUFRXJbrersLBQYWFhdQ0fABpU7JQ1jp/3z012YyT1xxPff8lNAFB3Vs9NLp35+de//qXevXvrt7/9rSIiItSjRw/99a9/dczft2+fcnNzlZiY6Giz2+3q27evsrKyJElZWVkKDw93JBdJSkxMlI+Pj7Zs2VLtdouLi1VUVOQ0AQAgkZsAAHXnUvHz/fff66WXXlLHjh313nvv6f7779ef/vQnLV26VJKUm5srSYqMjHRaLjIy0jEvNzdXERERTvP9/PzUrFkzR59zzZkzR3a73TG1bdvWlbABAF6M3AQAqCuXip/y8nL17NlTTz75pHr06KHRo0dr1KhRWrRo0aWKT5I0depUFRYWOqaDBw9e0u0BABoPchMAoK5cKn5atWql+Ph4p7YuXbrowIEDkqSoqChJUl5enlOfvLw8x7yoqCjl5+c7zS8tLdWxY8ccfc4VGBiosLAwpwkAAIncBACoO5eKn+uuu045OTlObXv27FFMTIwkKS4uTlFRUVq/fr1jflFRkbZs2aKEhARJUkJCggoKCrR9+3ZHnw8++EDl5eXq27fvBQ/EnWKnrHGaAAANh9wEAKgrP1c6P/jgg7r22mv15JNPasiQIfrss8+0ePFiLV68WJJks9k0fvx4Pf744+rYsaPi4uL02GOPKTo6WnfeeaekXz6Nu/XWWx2XJJw9e1Zjx47V0KFD63Q3HQAAKiM3AQDqyqXi5+qrr9Zbb72lqVOnatasWYqLi9P8+fM1fPhwR5+HHnpIp06d0ujRo1VQUKBf//rXevfddxUUFOTos3z5co0dO1Y33XSTfHx8NHjwYL3wwgv1NyoAgGWQmwAAdeXSc348hac9S+HcS9285Z7pAC6O1Z+lYDXsGwCNgdVzk0vf+QEAAACAxoriBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACzBz90BeCNvfHIuAAAA0Nhx5gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASuOEBADQAboQCAID7ceYHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJfi5OwAAAFD/Yqescfy8f26yGyMBAM9B8QMAgJejEAKAX3DZGwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS7io4mfu3Lmy2WwaP368o+3MmTNKT09X8+bNFRoaqsGDBysvL89puQMHDig5OVlNmjRRRESEJk+erNLS0osJBQAASeQmAEDNLrj42bp1q15++WX96le/cmp/8MEH9e9//1urVq3Sxo0bdeTIEQ0aNMgxv6ysTMnJySopKdHmzZu1dOlSZWZmatq0aRc+CgAARG4CAFfETlnjmKzigoqfkydPavjw4frrX/+qyy67zNFeWFioJUuW6LnnnlP//v3Vq1cvZWRkaPPmzfr0008lSWvXrtXOnTu1bNkyde/eXQMGDNDs2bO1cOFClZSU1M+oAACWQ24CAJzPBRU/6enpSk5OVmJiolP79u3bdfbsWaf2zp07q127dsrKypIkZWVl6corr1RkZKSjT1JSkoqKipSdnV3t9oqLi1VUVOQ0AQBQGbkJAHA+fq4usHLlSn3++efaunVrlXm5ubkKCAhQeHi4U3tkZKRyc3MdfSonl4r5FfOqM2fOHM2cOdPVUAEAFkFuAgDUhUtnfg4ePKhx48Zp+fLlCgoKulQxVTF16lQVFhY6poMHDzbYtgEAno3cBACoK5eKn+3btys/P189e/aUn5+f/Pz8tHHjRr3wwgvy8/NTZGSkSkpKVFBQ4LRcXl6eoqKiJElRUVFV7rBT8bqiz7kCAwMVFhbmNAEAIJGbAAB151Lxc9NNN+nrr7/Wjh07HFPv3r01fPhwx8/+/v5av369Y5mcnBwdOHBACQkJkqSEhAR9/fXXys/Pd/RZt26dwsLCFB8fX0/DAgBYBbkJAFBXLn3np2nTpurWrZtTW0hIiJo3b+5oT0tL04QJE9SsWTOFhYXpj3/8oxISEnTNNddIkm655RbFx8fr3nvv1bx585Sbm6tHH31U6enpCgwMrKdhAQCsgtwEAKgrl294cD7PP/+8fHx8NHjwYBUXFyspKUkvvviiY76vr69Wr16t+++/XwkJCQoJCVFKSopmzZpV36EAACCJ3AQA+MVFFz8ffvih0+ugoCAtXLhQCxcurHGZmJgYvf322xe7aQAAqkVuAgBU54Ke8wMAAAAAjQ3FDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsAQ/dwfQWMVOWePuEAAAAAC4gDM/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS/BzdwAAAKB+xE5Z4+4QAMCjceYHAAAAgCVQ/AAAAACwBIofAAAAAJZA8QMAAADAEih+AAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIHiBwAAAIAlUPwAAAAAsAQ/dwfg7WKnrHH8vH9ushsjAQAAAKyNMz8AAAAALMGl4mfOnDm6+uqr1bRpU0VEROjOO+9UTk6OU58zZ84oPT1dzZs3V2hoqAYPHqy8vDynPgcOHFBycrKaNGmiiIgITZ48WaWlpRc/GgCA5ZCbAAB15VLxs3HjRqWnp+vTTz/VunXrdPbsWd1yyy06deqUo8+DDz6of//731q1apU2btyoI0eOaNCgQY75ZWVlSk5OVklJiTZv3qylS5cqMzNT06ZNq79RAQAsg9wEAK7ZH/Q7x2Q1Ln3n591333V6nZmZqYiICG3fvl033HCDCgsLtWTJEq1YsUL9+/eXJGVkZKhLly769NNPdc0112jt2rXauXOn3n//fUVGRqp79+6aPXu2Hn74Yc2YMUMBAQH1NzoAgNcjNwEA6uqivvNTWFgoSWrWrJkkafv27Tp79qwSExMdfTp37qx27dopKytLkpSVlaUrr7xSkZGRjj5JSUkqKipSdnZ2tdspLi5WUVGR0wQAQHXITQCAmlxw8VNeXq7x48fruuuuU7du3SRJubm5CggIUHh4uFPfyMhI5ebmOvpUTi4V8yvmVWfOnDmy2+2OqW3bthcaNgDAi5GbAAC1ueDiJz09Xd98841WrlxZn/FUa+rUqSosLHRMBw8evOTbBAA0PuQmAEBtLug5P2PHjtXq1av10UcfqU2bNo72qKgolZSUqKCgwOkTtry8PEVFRTn6fPbZZ07rq7jjTkWfcwUGBiowMPBCQgUAWAS5CQBwPi6d+THGaOzYsXrrrbf0wQcfKC4uzml+r1695O/vr/Xr1zvacnJydODAASUkJEiSEhIS9PXXXys/P9/RZ926dQoLC1N8fPzFjAUAYEHkJgBAXbl05ic9PV0rVqzQP//5TzVt2tRxHbTdbldwcLDsdrvS0tI0YcIENWvWTGFhYfrjH/+ohIQEXXPNNZKkW265RfHx8br33ns1b9485ebm6tFHH1V6ejqfoAEAXEZuAgDUlUvFz0svvSRJuvHGG53aMzIylJqaKkl6/vnn5ePjo8GDB6u4uFhJSUl68cUXHX19fX21evVq3X///UpISFBISIhSUlI0a9asixsJAMCSyE0AgLpyqfgxxpy3T1BQkBYuXKiFCxfW2CcmJkZvv/22K5sGAKBa5CbXxE5Z4/R6/9xkN0UCAA3vgm54AACo3rn/sQQAAJ7joh5yCgAAAACNBcUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlsBzfgAAAAAvxfPnnHHmBwAAAIAlUPwAAAAAsASKHwAAAACWwHd+XMA1kwAAAEDjxZkfAAAAAJbAmR8AAADAy+0P+p27Q/AInPkBAAAAYAkUPwAAAAAsgcveAADwQpUvcYk9s8KNkQCA56D4AYAGdu6dI/fPTXZTJPAG3IkUAOqOy94AAAAAWALFDwAAAABLoPgBAAAAYAkUPwAAAAAsgeIHAAAAgCVwtzcAAADA4irfOdKb70JK8dOArPJLBQDwLBXP/OF5PwCsjsveAAAAAFgCxQ8AAAAAS6D4AQAAAGAJFD8AAAAALIEbHgAAYGHcjAeAlXDmBwAAAIAlcOYHAACLqLjltcRtrwH8wmrvC5z5AQAAAGAJnPkBAMBLVP4EFwBQFcXPeVT+IigAAACAxovL3gAAAABYAsUPAAAAAEvgsjcAABoRLscGgAvHmR8AAAAAlsCZHwAALMhqz/YAAIkzPwAAAAAsgjM/AAA0YjzbBwDqjuLHTWr7wur+uckNGAkAd6v8fsDfPzwFv5cAvJFbL3tbuHChYmNjFRQUpL59++qzzz5zZzgAAIuzel6KnbLGMQGwnv1Bv/vlbPIMu9e+F7jtzM+rr76qCRMmaNGiRerbt6/mz5+vpKQk5eTkKCIiwl1hSXL/bUT5tA0AGp6V8xI3PwBgFW478/Pcc89p1KhRGjFihOLj47Vo0SI1adJEr7zyirtCAgC345N392lMeani09mG+r4Pv5cAvIVbzvyUlJRo+/btmjp1qqPNx8dHiYmJysrKqtK/uLhYxcXFjteFhYWSpKKioguOodv09xw/fzMzyWleefHpC15vfWv34Kp6W9e546xJbfsGv6htH7H/Lo0L2a81LVO53ZX11bT8pVLT3/+F/F7V1+9lxfuuMeaC1+GJXM1LUsPmpnPzUpHt0u3/r2zDap1fNFXqdmZJlfZvgtKqX2DqoVr/Zury+8j7at1cqvc81OxS5aaLeZ8/92/xK9sv/xYVn7tE7SreC4r++7ZY7d9+hUaTm4wbHD582EgymzdvdmqfPHmy6dOnT5X+06dPN5KYmJiYmDxkOnjwYEOljAbhal4yhtzExMTE5GlTXXJTo7jb29SpUzVhwgTH6/Lych07dkzNmzeXzWa7qHUXFRWpbdu2OnjwoMLCwi42VLfxhnF4wxgk7xiHN4xB8o5xeNoYjDE6ceKEoqOj3R2K25Gbzs8bxuENY5C8YxzeMAbJO8bhaWNwJTe5pfhp0aKFfH19lZeX59Sel5enqKioKv0DAwMVGBjo1BYeHl6vMYWFhXnEwbtY3jAObxiD5B3j8IYxSN4xDk8ag91ud3cI9c7VvCSRm1zhDePwhjFI3jEObxiD5B3j8KQx1DU3ueWGBwEBAerVq5fWr1/vaCsvL9f69euVkJDgjpAAABZGXgIAa3DbZW8TJkxQSkqKevfurT59+mj+/Pk6deqURowY4a6QAAAWRl4CAO/ntuLn7rvv1o8//qhp06YpNzdX3bt317vvvqvIyMgGjSMwMFDTp0+vculCY+MN4/CGMUjeMQ5vGIPkHePwhjE0Fp6SlyTvOe7eMA5vGIPkHePwhjFI3jGOxjwGmzFedr9SAAAAAKiG2x5yCgAAAAANieIHAAAAgCVQ/AAAAACwBIofAAAAAJZgueLnww8/lM1mq3baunVrjcvdeOONVfqPGTOmASOvKjY2tkpMc+fOrXWZM2fOKD09Xc2bN1doaKgGDx5c5aF+DWX//v1KS0tTXFycgoOD1aFDB02fPl0lJSW1LucJx2LhwoWKjY1VUFCQ+vbtq88++6zW/qtWrVLnzp0VFBSkK6+8Um+//XYDRVrVnDlzdPXVV6tp06aKiIjQnXfeqZycnFqXyczMrLLPg4KCGiji6s2YMaNKTJ07d651GU86DhWq+zu22WxKT0+vtr8nHgtcPG/JTY09L0nkJnchN3nGcZAskJeMxRQXF5v//Oc/TtPvf/97ExcXZ8rLy2tcrl+/fmbUqFFOyxUWFjZg5FXFxMSYWbNmOcV08uTJWpcZM2aMadu2rVm/fr3Ztm2bueaaa8y1117bQBE7e+edd0xqaqp57733zHfffWf++c9/moiICDNx4sRal3P3sVi5cqUJCAgwr7zyisnOzjajRo0y4eHhJi8vr9r+mzZtMr6+vmbevHlm586d5tFHHzX+/v7m66+/brCYK0tKSjIZGRnmm2++MTt27DADBw407dq1q/V3JyMjw4SFhTnt89zc3AaMuqrp06ebrl27OsX0448/1tjf045Dhfz8fKcxrFu3zkgyGzZsqLa/Jx4LXDxvyU2NPS8ZQ24iN10cb8hN3p6XLFf8nKukpMS0bNnSzJo1q9Z+/fr1M+PGjWuYoOooJibGPP/883XuX1BQYPz9/c2qVascbbt27TKSTFZW1iWI0HXz5s0zcXFxtfZx97Ho06ePSU9Pd7wuKysz0dHRZs6cOdX2HzJkiElOTnZq69u3r/nDH/5wSeOsq/z8fCPJbNy4scY+GRkZxm63N1xQdTB9+nRz1VVX1bm/px+HCuPGjTMdOnSo8T+8nngsUP8aa27yxrxkDLnJHchNnnEcjPG+vGS5y97O9a9//Us//fRTnZ7gvXz5crVo0ULdunXT1KlTdfr06QaIsHZz585V8+bN1aNHDz399NMqLS2tse/27dt19uxZJSYmOto6d+6sdu3aKSsrqyHCPa/CwkI1a9bsvP3cdSxKSkq0fft2p33o4+OjxMTEGvdhVlaWU39JSkpK8qh9Lum8+/3kyZOKiYlR27Ztdccddyg7O7shwqvVt99+q+joaLVv317Dhw/XgQMHauzr6cdB+uX3a9myZRo5cqRsNluN/TzxWKB+Nebc5G15SSI3uQO5yTOOgzfmJT93B+BuS5YsUVJSktq0aVNrv9/97neKiYlRdHS0vvrqKz388MPKycnRm2++2UCRVvWnP/1JPXv2VLNmzbR582ZNnTpV//nPf/Tcc89V2z83N1cBAQEKDw93ao+MjFRubm4DRFy7vXv3asGCBXrmmWdq7efOY3H06FGVlZVVeeJ7ZGSkdu/eXe0yubm51fb3hH1eXl6u8ePH67rrrlO3bt1q7NepUye98sor+tWvfqXCwkI988wzuvbaa5WdnX3ev51LpW/fvsrMzFSnTp30n//8RzNnztT111+vb775Rk2bNq3S35OPQ4V//OMfKigoUGpqao19PPFYoP411tzkbXlJIje5A7nJM46D5KV5yd2nnurLww8/bCTVOu3atctpmYMHDxofHx/z+uuvu7y99evXG0lm79699TUEY8yFjaPCkiVLjJ+fnzlz5ky185cvX24CAgKqtF999dXmoYcecusYDh06ZDp06GDS0tJc3t6lOhbVOXz4sJFkNm/e7NQ+efJk06dPn2qX8ff3NytWrHBqW7hwoYmIiLhkcdbVmDFjTExMjDl48KBLy5WUlJgOHTqYRx999BJF5rrjx4+bsLAw87e//a3a+Z58HCrccsst5rbbbnNpGU88Fvgvb8hN3pCXLnQc5Cb3IDd5xnEwxjvzktec+Zk4cWKtVakktW/f3ul1RkaGmjdvrt/85jcub69v376SfvlEqEOHDi4vX5MLGUflmEpLS7V//3516tSpyvyoqCiVlJSooKDA6VO2vLw8RUVFXUzYTlwdw5EjR/Q///M/uvbaa7V48WKXt3epjkV1WrRoIV9f3yp3IqptH0ZFRbnUv6GMHTtWq1ev1kcffeTyJzP+/v7q0aOH9u7de4mic114eLiuuOKKGmPy1ONQ4YcfftD777/v8qfEnngs8F/ekJu8IS9J5KZzeep7IrnJM46D5MV5yd3Vl7uUl5ebuLi48969pSaffPKJkWS+/PLLeo7swi1btsz4+PiYY8eOVTu/4oullT9N3L17t1u/WHro0CHTsWNHM3ToUFNaWnpB62joY9GnTx8zduxYx+uysjLTunXrWr9Ueu6nJgkJCW77MmN5eblJT0830dHRZs+ePRe0jtLSUtOpUyfz4IMP1nN0F+7EiRPmsssuM3/+85+rne9px+Fc06dPN1FRUebs2bMuLeeJxwIXzttyU2PMS8aQm9yB3PRfnpKbvDUvWbb4ef/992s8VX/o0CHTqVMns2XLFmOMMXv37jWzZs0y27ZtM/v27TP//Oc/Tfv27c0NN9zQ0GE7bN682Tz//PNmx44d5rvvvjPLli0zLVu2NPfdd5+jz7njMOaXU8nt2rUzH3zwgdm2bZtJSEgwCQkJ7hiCOXTokLn88svNTTfdZA4dOuR0i8TKfTztWKxcudIEBgaazMxMs3PnTjN69GgTHh7uuK3jvffea6ZMmeLov2nTJuPn52eeeeYZs2vXLjN9+nS33sby/vvvN3a73Xz44YdO+/z06dOOPueOYebMmY7bvm7fvt0MHTrUBAUFmezsbHcMwRhjzMSJE82HH35o9u3bZzZt2mQSExNNixYtTH5+vjHG849DZWVlZaZdu3bm4YcfrjKvMRwL1J/GnJu8IS9VxEhuanjkJs84DhW8OS9ZtvgZNmxYjc8R2Ldvn9P9zA8cOGBuuOEG06xZMxMYGGguv/xyM3nyZLc+S2H79u2mb9++xm63m6CgINOlSxfz5JNPOl1Xfe44jDHm559/Ng888IC57LLLTJMmTcxdd93l9IbekDIyMmq87rqCpx6LBQsWmHbt2pmAgADTp08f8+mnnzrm9evXz6SkpDj1f+2118wVV1xhAgICTNeuXc2aNWsaNN7KatrnGRkZjj7njmH8+PGO8UZGRpqBAweazz//vOGDr+Tuu+82rVq1MgEBAaZ169bm7rvvdrq23tOPQ2XvvfeekWRycnKqzGsMxwL1pzHnJm/IS8aQm9yF3OQZx6GCN+clmzHGXLqL6gAAAADAM1j+OT8AAAAArIHiBwAAAIAlUPwAAAAAsASKHwAAAACWQPEDAAAAwBIofgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gc4R25ursaNG6fLL79cQUFBioyM1HXXXaeXXnpJp0+fdvSLjY2VzWbTp59+6rT8+PHjdeONNzpez5gxQzabTWPGjHHqt2PHDtlsNu3fv7/WePbu3asRI0aoTZs2CgwMVFxcnIYNG6Zt27YpLy9P/v7+WrlyZbXLpqWlqWfPnq7tAACARyEvAfWH4geo5Pvvv1ePHj20du1aPfnkk/riiy+UlZWlhx56SKtXr9b777/v1D8oKEgPP/zwedcbFBSkJUuW6Ntvv3Upnm3btqlXr17as2ePXn75Ze3cuVNvvfWWOnfurIkTJyoyMlLJycl65ZVXqix76tQpvfbaa0pLS3NpmwAAz0FeAuqXn7sDADzJAw88ID8/P23btk0hISGO9vbt2+uOO+6QMcap/+jRo7Vo0SK9/fbbGjhwYI3r7dSpkyIiIvTII4/otddeq1MsxhilpqaqY8eO+vjjj+Xj89/PKrp3765x48ZJ+uVTtDvvvFMHDhxQu3btHH1WrVql0tJSDR8+vE7bAwB4HvISUL848wP8n59++klr165Venq6U4KpzGazOb2Oi4vTmDFjNHXqVJWXl9e6/rlz5+qNN97Qtm3b6hTPjh07lJ2drYkTJzolmArh4eGSpIEDByoyMlKZmZlO8zMyMjRo0CBHPwBA40JeAuofxQ/wf/bu3StjjDp16uTU3qJFC4WGhio0NLTaSwkeffRR7du3T8uXL691/T179tSQIUPqdDmCJMelCJ07d661n6+vr1JSUpSZmen4BPC7777Txx9/rJEjR9ZpWwAAz0NeAuofxQ9wHp999pl27Nihrl27qri4uMr8li1batKkSZo2bZpKSkpqXdfjjz+ujz/+WGvXrj3vds+9lKE2I0eO1L59+7RhwwZJv3y6Fhsbq/79+9d5HQCAxoG8BFw4ih/g/1x++eWy2WzKyclxam/fvr0uv/xyBQcH17jshAkT9PPPP+vFF1+sdRsdOnTQqFGjNGXKlPMmkSuuuEKStHv37vPG3rFjR11//fXKyMhQeXm5/vd//1cjRoyocjkEAKDxIC8B9Y/iB/g/zZs3180336y//OUvOnXqlEvLhoaG6rHHHtMTTzyhEydO1Np32rRp2rNnT423Aa3QvXt3xcfH69lnn632uu2CggKn12lpaXrjjTf0xhtv6PDhw0pNTXVpDAAAz0JeAuofxQ9QyYsvvqjS0lL17t1br776qnbt2qWcnBwtW7ZMu3fvlq+vb43Ljh49Wna7XStWrKh1G5GRkZowYYJeeOGFWvvZbDZlZGRoz549uv766/X222/r+++/11dffaUnnnhCd9xxh1P/3/72t/L399cf/vAH3XLLLWrbtm3dBw4A8EjkJaB+UfwAlXTo0EFffPGFEhMTNXXqVF111VXq3bu3FixYoEmTJmn27Nk1Luvv76/Zs2frzJkz593OpEmTFBoaet5+ffr00bZt23T55Zdr1KhR6tKli37zm98oOztb8+fPd+rbpEkTDR06VMePH+cLpQDgJchLQP2yGVe+vQYAAAAAjRRnfgAAAABYAsUPAAAAAEug+AEAAABgCRQ/AAAAACyB4gcAAACAJVD8AAAAALAEih8AAAAAlkDxAwAAAMASKH4AAAAAWALFDwAAAABLoPgBAAAAYAn/H3nKL8Us4c5SAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, axs = plt.subplots(1,2, figsize=(10,3))\n", "\n", "ax = axs[0]\n", "out_graph = model(dataset.get_graph_inputs())\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "ax.set_title('From Dataset')\n", "ax.set_xlabel('GNN CV')\n", "ax.set_ylim(0,850)\n", "\n", "ax = axs[1]\n", "out_graph = model(datamodule.get_graph_inputs(\"train\"))\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "out_graph = model(datamodule.get_graph_inputs(\"valid\"))\n", "ax.hist(out_graph.detach().squeeze(), bins=100)\n", "\n", "ax.set_title('From Datamodule')\n", "ax.set_xlabel('GNN CV')\n", "ax.set_ylim(0,850)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Save the model to TorchScript\n", "As for normal CVs, the frozen model can be saved to TorchScript suing the `Lightning` util `to_torchscript` using `method=trace`." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/etrizio@iit.local/Bin/dev/mlcolvar/mlcolvar/data/datamodule.py:341: UserWarning: Length of split at index 1 is 0. This might result in an empty dataset.\n", " warnings.warn(\n" ] }, { "data": { "text/plain": [ "True" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "traced_model = model.to_torchscript('gnn_model.pt', method='trace')\n", "\n", "# we can also check the outputs coincide\n", "torch.allclose(model(dataset.get_graph_inputs()), traced_model(dataset.get_graph_inputs()))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "graph_mlcolvar_test_2.5", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.18" } }, "nbformat": 4, "nbformat_minor": 2 }