examples/quickstart/obd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quickstart Example with Open Bandit Dataset\n",
    "---\n",
    "This notebook demonstrates an example of conducting OPE of Bernoulli Thompson Sampling (BernoulliTS) as an evaluation policy. We use some OPE estimators and logged bandit data generated by running the Random policy (behavior policy) on the ZOZOTOWN platform. We also evaluate and compare the OPE performance (accuracy) of several estimators.\n",
    "\n",
    "The example consists of the following four major steps:\n",
    "- (1) Data Loading and Preprocessing\n",
    "- (2) Replicating Production Policy\n",
    "- (3) Off-Policy Evaluation (OPE)\n",
    "- (4) Evaluation of OPE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# needed when using Google Colab\n",
    "# !pip install obp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# import open bandit pipeline (obp)\n",
    "import obp\n",
    "from obp.dataset import OpenBanditDataset\n",
    "from obp.policy import BernoulliTS\n",
    "from obp.ope import (\n",
    "    OffPolicyEvaluation, \n",
    "    RegressionModel,\n",
    "    DirectMethod,\n",
    "    InverseProbabilityWeighting,\n",
    "    DoublyRobust\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5.4\n"
     ]
    }
   ],
   "source": [
    "# obp version\n",
    "print(obp.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (1) Data Loading and Preprocessing\n",
    "\n",
    "`obp.dataset.OpenBanditDataset` is an easy-to-use data loader for Open Bandit Dataset. \n",
    "\n",
    "It takes behavior policy ('bts' or 'random') and campaign ('all', 'men', or 'women') as inputs and provides dataset preprocessing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:obp.dataset.real:When `data_path` is not given, this class downloads the small-sized version of Open Bandit Dataset.\n"
     ]
    }
   ],
   "source": [
    "# load and preprocess raw data in \"All\" campaign collected by the Random policy (behavior policy here)\n",
    "# When `data_path` is not given, this class downloads the small-sized version of the Open Bandit Dataset.\n",
    "dataset = OpenBanditDataset(behavior_policy='random', campaign='all')\n",
    "\n",
    "# obtain logged bandit feedback generated by behavior policy\n",
    "bandit_feedback = dataset.obtain_batch_bandit_feedback()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the logged bandit dataset is collected by the behavior policy as follows.\n",
    "\n",
    "$ \\mathcal{D}_b := \\{(x_i,a_i,r_i)\\}$  where $(x,a,r) \\sim p(x)\\pi_b(a | x)p(r | x,a) $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['n_rounds', 'n_actions', 'action', 'position', 'reward', 'pscore', 'context', 'action_context'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# `bandit_feedback` is a dictionary storing logged bandit feedback\n",
    "bandit_feedback.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### let's see some properties of the dataset class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'obd'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# name of the dataset is 'obd' (open bandit dataset)\n",
    "dataset.dataset_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# number of actions of the \"All\" campaign is 80\n",
    "dataset.n_actions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10000"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# small sample example data has 10,000 samples (or rounds)\n",
    "dataset.n_rounds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# default context (feature) engineering creates context vector with 20 dimensions\n",
    "dataset.dim_context"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# ZOZOTOWN recommendation interface has three positions\n",
    "# (please see https://github.com/st-tech/zr-obp/blob/master/images/recommended_fashion_items.png)\n",
    "dataset.len_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (2) Replicating Production Policy\n",
    "\n",
    "After preparing the dataset, we now replicate the BernoulliTS policy implemented on the ZOZOTOWN recommendation interface during the data collection period.\n",
    "\n",
    "Here, we use `obp.policy.BernoulliTS` as an evaluation policy. \n",
    "By activating its `is_zozotown_prior` argument, we can replicate (the policy parameters of) BernoulliTS used in the ZOZOTOWN production.\n",
    "\n",
    "(When `is_zozotown_prior=False`, non-informative prior distribution is used.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# define BernoulliTS as an evaluation policy\n",
    "evaluation_policy = BernoulliTS(\n",
    "    n_actions=dataset.n_actions, \n",
    "    len_list=dataset.len_list, \n",
    "    is_zozotown_prior=True, # replicate the BernoulliTS policy in the ZOZOTOWN production\n",
    "    campaign=\"all\",\n",
    "    random_state=12345,\n",
    ")\n",
    "\n",
    "# compute the action choice probabilities of the evaluation policy via Monte Carlo simulation\n",
    "action_dist = evaluation_policy.compute_batch_action_dist(\n",
    "    n_sim=100000, n_rounds=bandit_feedback[\"n_rounds\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[0.01078, 0.00931, 0.00917],\n",
       "        [0.00167, 0.00077, 0.00076],\n",
       "        [0.0058 , 0.00614, 0.00631],\n",
       "        ...,\n",
       "        [0.0008 , 0.00087, 0.00071],\n",
       "        [0.00689, 0.00724, 0.00755],\n",
       "        [0.0582 , 0.07603, 0.07998]],\n",
       "\n",
       "       [[0.01078, 0.00931, 0.00917],\n",
       "        [0.00167, 0.00077, 0.00076],\n",
       "        [0.0058 , 0.00614, 0.00631],\n",
       "        ...,\n",
       "        [0.0008 , 0.00087, 0.00071],\n",
       "        [0.00689, 0.00724, 0.00755],\n",
       "        [0.0582 , 0.07603, 0.07998]],\n",
       "\n",
       "       [[0.01078, 0.00931, 0.00917],\n",
       "        [0.00167, 0.00077, 0.00076],\n",
       "        [0.0058 , 0.00614, 0.00631],\n",
       "        ...,\n",
       "        [0.0008 , 0.00087, 0.00071],\n",
       "        [0.00689, 0.00724, 0.00755],\n",
       "        [0.0582 , 0.07603, 0.07998]],\n",
       "\n",
       "       ...,\n",
       "\n",
       "       [[0.01078, 0.00931, 0.00917],\n",
       "        [0.00167, 0.00077, 0.00076],\n",
       "        [0.0058 , 0.00614, 0.00631],\n",
       "        ...,\n",
       "        [0.0008 , 0.00087, 0.00071],\n",
       "        [0.00689, 0.00724, 0.00755],\n",
       "        [0.0582 , 0.07603, 0.07998]],\n",
       "\n",
       "       [[0.01078, 0.00931, 0.00917],\n",
       "        [0.00167, 0.00077, 0.00076],\n",
       "        [0.0058 , 0.00614, 0.00631],\n",
       "        ...,\n",
       "        [0.0008 , 0.00087, 0.00071],\n",
       "        [0.00689, 0.00724, 0.00755],\n",
       "        [0.0582 , 0.07603, 0.07998]],\n",
       "\n",
       "       [[0.01078, 0.00931, 0.00917],\n",
       "        [0.00167, 0.00077, 0.00076],\n",
       "        [0.0058 , 0.00614, 0.00631],\n",
       "        ...,\n",
       "        [0.0008 , 0.00087, 0.00071],\n",
       "        [0.00689, 0.00724, 0.00755],\n",
       "        [0.0582 , 0.07603, 0.07998]]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# `action_dist` is an array of shape (n_rounds, n_actions, len_list) \n",
    "# representing the distribution over actions by the evaluation policy\n",
    "action_dist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (3) Off-Policy Evaluation (OPE)\n",
    "Our next step is OPE, which aims to estimate the performance of evaluation policies using logged bandit data and OPE estimators.\n",
    "\n",
    "Here, we use \n",
    "- `obp.ope.InverseProbabilityWeighting` (IPW)\n",
    "- `obp.ope.DirectMethod` (DM)\n",
    "- `obp.ope.DoublyRobust` (DR)\n",
    "\n",
    "as estimators and visualize the OPE results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (3-1) Obtaining a reward estimator\n",
    "A reward estimator $\\hat{q}(x,a)$ is needed for model dependent estimators such as DM or DR.\n",
    "\n",
    "$\\hat{q}(x,a) \\approx \\mathbb{E} [r \\mid x,a]$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# estimate the expected rewards by using an ML model (Logistic Regression here)\n",
    "# the estimated rewards are used by model-dependent estimators such as DM and DR\n",
    "regression_model = RegressionModel(\n",
    "    n_actions=dataset.n_actions,\n",
    "    len_list=dataset.len_list,\n",
    "    action_context=dataset.action_context,\n",
    "    base_model=LogisticRegression(max_iter=1000, random_state=12345),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimated_rewards_by_reg_model = regression_model.fit_predict(\n",
    "    context=bandit_feedback[\"context\"],\n",
    "    action=bandit_feedback[\"action\"],\n",
    "    reward=bandit_feedback[\"reward\"],\n",
    "    position=bandit_feedback[\"position\"],\n",
    "    pscore=bandit_feedback[\"pscore\"],\n",
    "    n_folds=3, # use 3-fold cross-fitting\n",
    "    random_state=12345,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "please refer to https://arxiv.org/abs/2002.08536 about the details of the cross-fitting procedure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (3-2) Off-Policy Evaluation\n",
    "$V(\\pi_e) \\approx \\hat{V} (\\pi_e; \\mathcal{D}_b, \\theta)$ using DM, IPW, and DR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# estimate the policy value of BernoulliTS based on its action choice probabilities\n",
    "# it is possible to set multiple OPE estimators to the `ope_estimators` argument\n",
    "ope = OffPolicyEvaluation(\n",
    "    bandit_feedback=bandit_feedback,\n",
    "    ope_estimators=[InverseProbabilityWeighting(), DirectMethod(), DoublyRobust()]\n",
    ")\n",
    "\n",
    "# `summarize_off_policy_estimates` returns pandas dataframes including the OPE results\n",
    "estimated_policy_value, estimated_interval = ope.summarize_off_policy_estimates(\n",
    "    action_dist=action_dist, \n",
    "    estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
    "    n_bootstrap_samples=10000, # number of resampling performed in bootstrap sampling.\n",
    "    random_state=12345,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>estimated_policy_value</th>\n",
       "      <th>relative_estimated_policy_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ipw</th>\n",
       "      <td>0.004553</td>\n",
       "      <td>1.198126</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dm</th>\n",
       "      <td>0.003545</td>\n",
       "      <td>0.932797</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dr</th>\n",
       "      <td>0.004727</td>\n",
       "      <td>1.244027</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     estimated_policy_value  relative_estimated_policy_value\n",
       "ipw                0.004553                         1.198126\n",
       "dm                 0.003545                         0.932797\n",
       "dr                 0.004727                         1.244027"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the estimated policy value of the evaluation policy (the BernoulliTS policy)\n",
    "# relative_estimated_policy_value is the policy value of the evaluation policy \n",
    "# relative to the ground-truth policy value of the behavior policy (the Random policy here)\n",
    "estimated_policy_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>95.0% CI (lower)</th>\n",
       "      <th>95.0% CI (upper)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ipw</th>\n",
       "      <td>0.004544</td>\n",
       "      <td>0.001531</td>\n",
       "      <td>0.009254</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dm</th>\n",
       "      <td>0.003545</td>\n",
       "      <td>0.003501</td>\n",
       "      <td>0.003589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dr</th>\n",
       "      <td>0.004717</td>\n",
       "      <td>0.001702</td>\n",
       "      <td>0.009416</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         mean  95.0% CI (lower)  95.0% CI (upper)\n",
       "ipw  0.004544          0.001531          0.009254\n",
       "dm   0.003545          0.003501          0.003589\n",
       "dr   0.004717          0.001702          0.009416"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# confidence intervals of policy value of BernoulliTS estimated by OPE estimators\n",
    "# (`mean` values in this dataframe is also estimated via the non-parametric bootstrap procedure \n",
    "# and is a bit different from the above values in `estimated_policy_value`)\n",
    "estimated_interval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhUAAAGSCAYAAABUuTODAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABSWklEQVR4nO3deVzU1f4/8NewDMuwi6MIGsoiAgLiDi6EuFw1tVzrZoXVVTP9mprd9Jaa3Xs1NfOq6a2r1+VaImK5dE3FhVRSQQdJFnEDhYlNQfZ9fn/w43MdgfEzMCgjr+fj0SPnM+eceQ9NzMvP53zOkahUKhWIiIiImsngWRdAREREzweGCiIiItIJhgoiIiLSCYYKIiIi0gmGCiIiItIJhgoiIiLSCYYKIiIi0gmGCiIiItIJhgoiIiLSCSOxDZVKJX777TckJSUhNzcXhYWFkEqlsLKygrOzM7y8vODt7Q2pVNqS9RIREVErJXnSMt3nz5/H8ePHkZyc/MTBZDIZgoKCMGrUKMjlclEFpKenY/v27UhJSYFMJkNwcDAmT54MAwPNJ1FKSkqwY8cOxMTEoKamBr1790ZoaCgsLS2FNvHx8Th16hRu3LiBnJwcTJo0CVOmTGnSWERERKRZo6Hi2rVr2LVrF9LS0mBubo6+ffvCw8MDLi4usLGxgYWFBSoqKlBYWAilUomUlBTEx8fjxo0bMDIywh/+8Ae88sorMDc3b/TFi4qKsHDhQjg5OWH8+PHIzMzE7t27MWbMGEybNk1j4X/961+hVCoxffp0GBgYYM+ePbC2tsZnn30mtNm1axfi4uLg5uaG6OhovPTSSw2GCjFjERERkWaNXv5YuXIlunbtivnz56NPnz4wNjau18bMzAxmZmaQy+Xw8/PDlClT8Pvvv+PEiRM4duwYTE1NMWnSpEZf/MSJE6ioqMDChQthbm4OHx8flJaWIjw8HOPGjWs0kKSkpODq1atYvnw5PD09AQB2dnZYsmQJ4uPj4ePjAwB4/fXX8cYbbwAAYmNjmzUWERERadboNYaFCxdi1apVGDhwYIOBojEODg544403sHHjxid+IcfFxcHX11ctPAQGBqKiogKJiYmN9lMoFLC2thZCAAC4urpCLpcjLi5OOPakSyjajEVERESaNXqmol+/fs0a2MbGBjY2NhrbZGRkwMvLS+2Yvb09TExMoFQqNfZzdHSsd9zR0REZGRla1anLsYiIiNoy0Xd/tITi4mLIZLJ6x2UyGYqKijT2a+jSiEwmQ3Z2ttY1aDtWZGQkIiMjAQCrVq1CRUWFVq9JRESkrzTd5flMQ4W+CgkJQUhIiPA4Nzf3GVZDRET09HTq1KnR5zSGivfff1/rF5NIJNi4caOotjKZDCUlJfWOFxcXw8LCQmO/wsLCBvs1dObjSTXoaiwiIqK2TGOoyMnJadEXb2jeQm5uLsrLyzUmIUdHR5w8ebLecaVSib59+2pdg67GIiIiass0hopNmza16Iv7+fnh0KFDKC0thZmZGQAgOjoaUqlU7W6Mx/Xq1QsRERFITk6Gh4cHAODWrVvIysqCn5+fVjXociwiIqK2TGOoaN++fYu++PDhw3H06FGsXbsW48ePR3Z2NsLDwzF27Fi1yZNz586Fp6cnZs+eDQBwd3eHr68vNm3ahOnTp0MikWDPnj3w8PBQu401JycHt27dAgBUVVUhPT0dFy5cgImJCXr16qXVWERERKTZE5fpbmnp6enYtm2b2jLdU6ZMUVtjYs6cOfD09MScOXOEY8XFxdi5cycuXboElUoFf39/hIaGwsrKSmhz5swZfP311/Ves3379ti8ebNWY2mi6fZXIiKi54mm6QkaQ0VNTQ2++uorSCQSzJ07F0ZGDZ/YqKqqwsaNGyGRSDB//vxmF6xvGCqIiKit0BQqNC45efHiRVy8eBF9+vRpNFAAgJGREfr27Ytff/0VFy5caHqlREREpLc0hopff/0VdnZ2GDRo0BMHCgwMhJ2dHc6dO6ez4oiIiEh/aAwVt27dgpeXFyQSyRMHkkgk8Pb2xu3bt3VWHBEREekPjaEiPz8f7dq1Ez2YnZ0dHj582OyiiIiISP9oDBVGRkaorKwUPVhlZaXGuRdERET0/NKYAGxtbZGWliZ6sLS0NNja2ja7KCIioqdp9erVyM3Nhb29PT766KNnXY7e0nimonv37khMTERmZuYTB8rMzERiYqKwKiUREZG+yM3NRWZmJjeIbCaNoWL48OGoqanBl19+qXGuREFBAdavX4+amhq13TuJiIio7dB4+cPV1RUhISGIjIzEggULMHz4cHh7e8POzg4A8ODBA1y7dg2RkZEoLCzE8OHD4erq+lQKJyIiotblibMqZ8yYgZqaGpw6dQo//PADfvjhhwbbDRs2DDNmzNB5gURERKQfnhgqDA0NMXPmTAQFBeHEiRO4fv068vPzAQA2Njbw8PBASEgIunfv3tK1kp7iBCgiorZB9P2f3bt3Z3CgJqmbAEVERM83jRM1iYiIiMRiqCAiIiKdYKggIiIinWCoICIiIp1gqCAiIiKdYKggIiIinWCoICIiIp1o1j7l2dnZSE9PBwA4OTlBLpfrpCgiIiLSP00KFaWlpdi6dSsuXLigdnzgwIGYNWsWTE1NdVIcERER6Y8mhYpt27YhPj4eU6ZMQbdu3VBZWYnY2FhERUXBxMQEs2fP1nWdRERE1MppDBXl5eUwMTGpdzwmJgbvvPMOBg8eLBzr168fysvLcenSJYYKIiKiNkjjRM1Fixbh2rVr9Y5XV1fDzMys3nEzMzPU1NTorjoiIiLSGxrPVLi5uWHlypUYNmwYpk+fLgQJb29vbNu2DWVlZejatSsqKytx+fJlREVFoXfv3k+lcCIiImpdNIaKefPmYdCgQfj222+hUCjwpz/9Cb169cI777yDNWvWYOPGjWrtu3XrhhkzZrRowURERNQ6PXGipr+/P9atW4ddu3Zh1apVGDx4MN566y2sXr0a8fHxyMjIAFB7S2nPnj1bvGAiIiJqnUTd/WFubo5Zs2YhICAA33zzDRYuXIi3334b/fr1g4+PT0vXSERERHpAqxU1fXx8sHbtWvTr1w/r1q3D+vXrUVBQ0FK1ERERkR4RFSoKCgpw+/ZtFBQUwNTUFG+//TaWL1+O1NRUfPDBBzh37lxL10lEREStnMbLH2VlZdiyZYvaypn9+/fHe++9hx49emDNmjXYu3cvNm/ejOjoaPzpT3+CjY1NS9dMRERErZDGMxXfffcdLly4gKFDh+Ltt99GUFAQLl68iD179gAApFIp3njjDaxcuRKZmZn44IMPcPr06adSOBEREbUuGs9UxMTECGcm6pSWliI2NhZvv/22cMzV1RVffPEF9u/fj2+//RYvvvhiy1VMRERErdITl+lu166d2rF27do1uMqmkZERpk2bhgEDBui2QiIiItILGi9/uLm54ZdffkFycjKqqqqQkpKCs2fPws3NrdE+zs7Ouq6RiIiI9IDGMxWhoaFYsWIFli1bJhyzs7PDW2+91dJ1ERERkZ7RGCo6duyIr776CpcvX0Zubi7s7e3h7+8PU1PTp1UfERER6YknrqhpYmKCgICAp1ELERER6TGtVtQkIiIiaoyovT8aEhsbi6SkJJSXl0MulyMgIAD29va6rI2IiIj0iMZQ8d1338HHxwfe3t7CseLiYnzxxRdITk5WaxsWFoaZM2diyJAhLVMpERERtWoaQ8XBgwchlUrVQsU///lPJCcnQy6XIzAwEFZWVkhJScGvv/6KrVu3wtnZGV26dGnxwomIiKh10eryR2ZmJi5evIiuXbti2bJlMDMzAwCMHj0a/v7+2Lx5M/773/9i1qxZLVIsERERtV5aTdRMSkoCALz66qtCoKgzZMgQuLq6IjExUXfVERERkd7QKlTk5+cDAFxcXBp83sXFBQ8ePGh2UURERKR/tAoVdWcnjI2NG3ze2NgYEomk+VURERGR3nninIqEhAThz5mZmQCAnJwcODk51Wt7//59WFpa6rA8IiIi0hdPDBWJiYn15klcuXKlwVBx+/ZtODo66q46IiIi0hsaQ8WjG4k9ysrKqt6x27dvo7q6Gj179tRNZURERKRXNIYKT09P0QN169YNmzdvbnZBREREpJ+49wcRERHphFaLX1VXVyMrKwvFxcWQSCSwtrZG+/btW6o2IiIi0iOiQsWlS5dw7NgxJCUlobq6Wu05KysrBAYGYsKECbCxsdG6gPT0dGzfvh0pKSmQyWQIDg7G5MmTYWCg+SRKSUkJduzYgZiYGNTU1KB3794IDQ2td/dJTEwM9u7di8zMTMjlckyePLneVu63bt3C999/j1u3bgGovZQzbdo0uLm5af1+iIiI2iqN39wqlQqbN2/GunXrcO3aNbVAYW9vDycnJ5SUlODo0aNYuHBhvU3GnqSoqAgrV66ERCLB4sWLMXHiRBw5cgT79u17Yt/169cjISEBM2fOxJw5c3Dr1i2sWbNGrU1ycjLWrVsHLy8vfPzxx/D398eGDRtw9epVoU1ubi5WrlyJ6upqzJ07F3PnzkV1dTU+//xz5OTkaPV+iIiI2jKNZyoiIyPxyy+/wN/fH1OnTkWHDh2QlZWFffv24fr161i6dCnat2+P8+fPY/fu3Vi9ejXWrVsHOzs7US9+4sQJVFRUYOHChTA3N4ePjw9KS0sRHh6OcePGwdzcvMF+KSkpuHr1KpYvXy5MJrWzs8OSJUsQHx8PHx8fAEBERAR69OiBGTNmAAC8vb2Rnp6O/fv3w9fXF0Dt7bGlpaX48MMPhddzd3fH22+/DYVCgREjRoh6L0RERG2dxjMVp06dgpOTExYtWgRnZ2eYmZnB2dkZCxcuhI2NDb777jsYGxsjKCgIn3zyCcrKyvDjjz+KfvG4uDj4+vqqhYfAwEBUVFRo3ENEoVDA2tpa7e4UV1dXyOVyxMXFAQAqKytx7do1DBw4UK1vQEAAUlJSUFJSAqB2noihoSFMTEyENqampjA0NIRKpRL9XoiIiNo6jaEiPT0dPXv2hKGhodpxQ0ND9OzZU221TWdnZ/j7+0OhUIh+8YyMDHTq1EntmL29PUxMTKBUKjX2a2iRLUdHR2RkZAAAsrKyUF1dXa+do6MjVCqVMH7//v1hYmKCXbt24eHDh3j48CF27twJmUxWL5AQERFR4zRe/pBIJKioqGjwuYqKClRWVqodc3R0FM4UiFFcXAyZTFbvuEwmQ1FRkcZ+DV0akclkyM7OBgCh/+PjW1hYCGMAtZdNli1bhlWrVuHo0aMAAFtbWyxdurTBRb6A2stCkZGRAIBVq1bB3t5e4/ts6+pCqaGhIX9WRNQq8feUbmgMFZ07d0ZsbCxee+014csYqP3Cjo2NhYODg1r7srIySKXSlqm0heTl5eHLL79Et27dMGvWLADAzz//jFWrVuHzzz9v8MMVEhKCkJAQ4XFubu5Tq1cf1U3wra6u5s+KiFol/p4S7/ErDI/SePnjxRdfxMOHD7FkyRIcP34ccXFxOH78OJYuXYqHDx9i8ODBau3v3buHjh07ii5MJpMJcxseVVxcrBZiGupXWlraYL+6MxN1/R8f//EzGIcOHUJ1dTUWLFgAPz8/+Pn5YdGiRTAwMMChQ4dEvxciIqK2TuOZipCQECQmJuL8+fPYtm2b2nN+fn4YO3as8Li0tBQVFRX11oDQ5NE5EHVyc3NRXl6uMQk5Ojri5MmT9Y4rlUr07dsXANChQwcYGhoiIyNDbUKnUqmERCIRxlcqlXBycoKR0f9+FEZGRujcuTOysrJEvxciIqK27omLX82bNw8DBgzApUuX8PDhQ1haWsLf3x8BAQFqC1SZmZnhr3/9q1Yv7ufnh0OHDqG0tBRmZmYAgOjoaEilUo37jvTq1QsRERFITk6Gh4cHgNoFrLKysuDn5wcAMDY2hre3Ny5cuIDhw4cLfaOjo+Hu7i7MybC3t4dCoUBVVZUQLCorK3H37l307t1bq/dDRETUlolaUbNfv37o16+fzl98+PDhOHr0KNauXYvx48cjOzsb4eHhGDt2rNpEzLlz58LT0xOzZ88GULuOhK+vLzZt2oTp06dDIpFgz5498PDwENaoAICJEydi+fLl2LFjB/r27QuFQgGFQoElS5YIbYYNG4ZTp05hzZo1GDlyJFQqFY4dO4b8/Hy1eRNERESkmVZ7f+iahYUFPv30U2zbtg2rV6+GTCbDmDFjMGXKFLV2NTU1qKmpUTs2f/587Ny5E1u2bIFKpYK/vz9CQ0PV2nh4eGDBggUICwvD8ePHIZfLMW/ePGHhK6B2Se4lS5Zg//792LhxIwCgS5cu+Mtf/gJnZ+eWeeNERETPIYmKKzw1m6Y1NQj48MMPkZmZiY4dO9ZbSp2IqDXg7ynxmnz3BxEREZFYDBVERESkEwwVREREpBMMFURERKQTDBVERESkEwwVREREpBNah4rExETs379f6+eIiIjo+aZ1qEhISEB4eLjWzxEREdHzjZc/iIiISCcYKoiIiEgnGCqIiIhIJ0RtKJabmyv8ubi4uN4xoHYLcSIiImq7RIWKOXPmaDwmkUiwd+9e3VVFREREekdUqJg4cSIkEgmA2ttGExMTMWnSpBYtjIiIiPSLqFAxZcoU4c/h4eFITEzE5MmTW6woIiIi0j+cqElEREQ6wVBBREREOsFQQURERDqhdahQqVRNeo6IiIieb6Imaj5qypQpahM3xT5HREREzzde/iAiIiKdYKggIiIinWg0VFRUVDR7cF2MQURERPqh0VAxZ84c/Pe//0VlZaXWg6ampuKLL77AoUOHmlUcERER6Y9GJ2r6+vpi586dCA8PR0BAAAYOHAh3d3dIpdIG22dlZeHq1auIiorCzZs3YW9vj3HjxrVY4URERNS6NBoq3n//fYwaNQp79+5FZGQkIiMjYWBgACcnJ9jY2EAmk6GyshJFRUVQKpUoKCgAAFhZWeHVV1/FmDFjYGxs/NTeCBERET1bGm8pdXV1xV/+8hf8/vvvOHXqFK5du4bU1FTcvXtXrZ2VlRX69+8v/GNkpPWdqkRERKTnRH37Ozg44I9//CMAoLy8HA8ePEBhYSGkUimsra1ha2vbokUSERFR66f1KQUTExM4ODjAwcGhJeohIiIiPcV1KoiIiEgnGCqIiIhIJxgqiIiISCcYKoiIiEgnGCqIiIhIJxgqiIiISCcYKoiIiEgntF6noqqqCteuXUN6ejrKysowadIkALU7kpaWlsLS0hIGBswqREREbY1WoSIuLg5btmxBfn6+cKwuVKSmpuKTTz7B3LlzMWjQIJ0WSURERK2f6FMKt27dwpo1ayCRSPDmm28iMDBQ7Xl3d3fI5XJcunRJ50USERFR6yc6VEREREAqlWLVqlUYPXp0g8t0u7i4IC0tTacFEhERkX4QHSquX7+Ovn37wsbGptE29vb2apdGiIiIqO0QHSrKyspgZWWlsU15eTlqamqaXRQRERHpH9Ghws7ODvfu3dPYJjU1FR06dGh2UURERKR/RIcKPz8/XL16FcnJyQ0+r1AokJKSAn9/f50VR0RERPpD9C2lL7/8MqKjo/H5559j1KhRyMnJAQBcuXIFiYmJOHbsGGxsbDB27NgWK5aIiIhaL9Ghws7ODkuXLsX69etx+PBh4fjq1asBAB06dMCiRYueOO+CiIiInk9aLX7VrVs3bNiwAVeuXEFKSgoKCwthbm4ONzc39O3bF4aGhi1VJxEREbVyWi/TbWBggD59+qBPnz4tUQ8RERHpKW7SQURERDoh+kxFVFSU6EGHDh3apGKIiIhIf4kOFV9//bXoQRkqiIiI2h7RoWL27NkNHi8pKcHNmzcRHR2Nfv36cZ0KIiKiNkp0qAgKCtL4/IsvvihsNqaN9PR0bN++HSkpKZDJZAgODsbkyZNhYKB5ukdJSQl27NiBmJgY1NTUoHfv3ggNDYWlpaVau5iYGOzduxeZmZmQy+WYPHkyAgIC6o138eJF/Pjjj7h79y5MTEzg4uKChQsXwtTUVKv3Q0RE1FbpbKJmz5494evri7CwMNF9ioqKsHLlSkgkEixevBgTJ07EkSNHsG/fvif2Xb9+PRISEjBz5kzMmTNH2Jr9UcnJyVi3bh28vLzw8ccfw9/fHxs2bMDVq1fV2p08eRL/+Mc/4OfnhyVLlmDWrFlwcHDgPiZERERa0PqWUk06deqEEydOiG5/4sQJVFRUYOHChTA3N4ePjw9KS0sRHh6OcePGwdzcvMF+KSkpuHr1KpYvXw5PT08AtYtzLVmyBPHx8fDx8QFQu117jx49MGPGDACAt7c30tPTsX//fvj6+gIACgoKsHPnToSGhiIkJER4jX79+jXpZ0BERNRW6fSW0vT0dK3ax8XFwdfXVy08BAYGoqKiAomJiY32UygUsLa2FgIFALi6ukIulyMuLg4AUFlZiWvXrmHgwIFqfQMCApCSkoKSkhIAwK+//grgyZd3iIiISLNmn6moqanB/fv3cfLkSSgUCvTq1Ut034yMDHh5eakds7e3h4mJCZRKpcZ+jo6O9Y47OjoiIyMDAJCVlYXq6up67RwdHaFSqaBUKuHq6oobN26gU6dOOHXqFA4cOICHDx+ia9euePPNN9G9e3fR74WIiKitEx0qpk6d+sQ2FhYWeP3110W/eHFxMWQyWb3jMpkMRUVFGvs1dGlEJpMhOzsbAIT+j49vYWEhjAEADx8+hFKpREREBF5//XVYWlri4MGD+Nvf/oYNGzbAxsam3utERkYiMjISALBq1SrY29uLeLdtV93y7YaGhvxZEVGrxN9TuiE6VPTo0QMSiaTecYlEAplMBldXV7z44ot6t6GYSqVCWVkZFixYAD8/PwCAu7s75syZg59//hnTpk2r1yckJERt/kVubu7TKlcvVVdXC//mz4qIWiP+nhKvU6dOjT4nOlQsX75cF7WokclkwtyGRxUXFwtnFBrrV1hY2GC/ujMTdf0fH//xMxgymQwSiURtfoa5uTm6deum9RwRIiKituyZ7v3x6ByIOrm5uSgvL9eYhBrqBwBKpVKYQ9GhQwcYGhrWa6dUKiGRSITx6+ZYPE6lUj1xrQwiIiL6n2f6renn54erV6+itLRUOBYdHQ2pVKp25uBxvXr1Qn5+PpKTk4Vjt27dQlZWlnAJw9jYGN7e3rhw4YJa3+joaLi7uwtzMnr37g0AuHbtmtCmpKQEt2/fxgsvvNDs90hERNRWNHr5Q5u9Ph4lkUgaXdL7ccOHD8fRo0exdu1ajB8/HtnZ2QgPD8fYsWPVJmLOnTsXnp6ewrju7u7w9fXFpk2bMH36dEgkEuzZswceHh7CGhUAMHHiRCxfvhw7duxA3759oVAooFAosGTJEqGNi4sL+vTpg61bt+K1116DlZUVDh48CENDQ4wcObJJPwMiIqK2qNFQoc2upI8TGyosLCzw6aefYtu2bVi9ejVkMhnGjBmDKVOmqLWrqampt7rl/PnzsXPnTmzZsgUqlQr+/v4IDQ1Va+Ph4YEFCxYgLCwMx48fh1wux7x584SFr+rMmzcPu3fvxq5du1BeXg4PDw8sW7ZM47wOIiIiUidRNTShAEBOTk6TB23fvn2T++ojTWtqEPDhhx8iMzMTHTt2rLeUOhFRa8DfU+I16e6PthYMiIiIqHl4ewMRERHpRJOW6a6pqUFBQQGqqqoafJ6rkREREbU9WoWKu3fvYs+ePUhISEBlZWWDbSQSCfbu3auT4oiIiEh/iA4V6enp+Mtf/gIA8PHxweXLl/HCCy/A2toad+7cQWFhIby8vHiWgoiIqI0SHSoOHDiA6upq/P3vf0eXLl0wdepU9OvXD5MmTUJZWRn+/e9/Q6FQ4L333mvJeomIiKiVEj1RMyEhAf7+/ujSpYtwrO5uVFNTU/zpT3+CTCZDWFiY7qskIiKiVk90qCgsLISDg8P/OhoYoLy8XHhsaGgILy8vxMfH67ZCIiIi0guiQ4WFhQXKysqEx1ZWVvW2hzUyMmpw11EiIiJ6/okOFR06dEB2drbwuGvXrvjtt9/w8OFDAEBZWRliY2Mhl8t1XyURERG1eqInavr6+uLgwYMoKyuDqakpRowYAYVCgcWLF6N79+64ffs2cnJy8MYbb7RkvURERNRKiQ4Vw4YNQ6dOnVBRUQFTU1P4+/vjzTffRHh4OC5evAipVIrx48fjD3/4Q0vWS0RERK2UxlCxePFihISEYPDgwbC1tUVAQIDa86NHj8aoUaNQUFAAa2trSCSSFi2WiIiIWi+NcyrS0tKwbds2zJw5E1u3bsWNGzfqD2BgABsbGwYKIiKiNk7jmYqVK1ciMjISFy5cwOnTp3H69Gl06dIFw4YNw5AhQ2Bubv606iQiIqJWTmOocHd3h7u7O0JDQ3H27FmcOnUKd+7cwb///W/s2bMHAwYMwLBhw+Dh4fG06iUiIqJWStRETTMzM4wYMQIjRoxAamoqIiMjcf78efzyyy/45Zdf4OTkJJy9sLCwaOmaiYiIqBUSvU5FHWdnZ7zzzjv45z//iffeew/du3dHeno6du7ciVmzZmHjxo0tUScRERG1clqHijpSqRRDhw7FZ599hvXr18PDwwOVlZU4d+6cLusjIiIiPSF6nYqGFBUVISoqCqdOnUJ6ejoAcPImERFRG9WkUHHt2jVERkYiJiYGVVVVAAA3NzeEhITUW8uCiIiI2gbRoSI/Px+nT5/GqVOnhD1AZDIZQkJCEBISgs6dO7dYkURERNT6aQwVKpUKV65cwcmTJ6FQKFBTUwMA8PDwwLBhwzBgwABIpdKnUigRERG1bhpDxXvvvYcHDx4AqN36fMiQIQgJCYGjo+NTKY6IiIj0h8ZQ8eDBA3h6egpnJYyMmjWvk4iIiJ5jGlPCV199BQcHh6dVCxEREekxjetUMFAQERGRWE1e/IqIiIjoUQwVREREpBMMFURERKQTDBVERESkEwwVREREpBOiQ8XFixeFFTWJiIiIHid6Nasvv/wStra2ePHFFzFs2DDY29u3ZF1ERNQKHDv0+7Mu4akoLqoS/t1W3vPIcbpfNkL0mYqRI0eivLwcBw4cwNy5c7Fq1SpcvnwZKpVK50URERGR/hF9pmLGjBl4/fXXER0djRMnTkChUEChUMDOzg7Dhg1DcHAw7OzsWrJWIiIiasW02sxDKpUiKCgIQUFBuHv3LiIjI3H27FmEh4cjIiIC/v7+GD58OPz8/FqoXCIiImqtmrxDWJcuXdTOXoSFhSE2NhaxsbGwt7fHyJEjMWLECJiamuqyXiIiImqlmnVLaVlZGX755Rf8/PPPwhbpzs7OKCoqwp49e/DBBx8gNTVVF3USERFRK9ekMxV37tzBiRMncP78eZSVlUEqlSI4OBgjR46Es7MzysrKcOzYMezbtw///ve/sWLFCl3X/Vz4/cN3nnUJT0VVTsX//3dWm3nPDmv+9axLICJ66kSHivLycpw/fx4nTpzA7du3AQCOjo4YPnw4hg4dCnNzc6Gtqakpxo8fj/v37+PUqVO6r5qIiIhaHdGhYubMmSgtLYWBgQH69++PkSNHwsvLS2MfOzs7VFZWNrtIIiIiav1EhwozMzOMHTsWISEhsLGxEdVnxIgRCAwMbGptREREpEdEh4rNmzfDwEC7eZ3m5uZql0WIiIjo+SU6JWgbKIiIiKhtEZ0UIiIi8Oqrrwq3jj7uwYMHePXVV/Hjjz/qqjYiIiLSI6JDxeXLl+Hp6dnoUtx2dnbw9vZGTEyMzoojIiIi/SE6VGRmZsLJyUljG0dHR2RmZja7KCIiItI/okNFRUUFTExMNLaRSqUoKytrdlFERESkf0SHinbt2uHGjRsa29y4cYM7lRIREbVRokOFr68vEhMTER0d3eDz58+fR2JiIncoJSIiaqNEr1MxYcIEnDt3Dhs2bEB0dDT8/PxgZ2eHBw8eQKFQIDY2FhYWFpgwYUILlktEREStlehQYWdnh6VLl+LLL79ETExMvbs82rdvjwULFqBdu3ZaFZCeno7t27cjJSUFMpkMwcHBmDx58hPXxSgpKcGOHTsQExODmpoa9O7dG6GhobC0tFRrFxMTg7179yIzMxNyuRyTJ09GQEBAg2PW1NRgyZIluH37Nj766CP07t1bq/dCRETUlmm1S6mLiws2bNiAy5cv48aNGyguLoZMJoObmxt69+4NIyPtNj0tKirCypUr4eTkhMWLFyMzMxO7d++GSqXCtGnTNPZdv349lEolZs6cCQMDA+zZswdr1qzBZ599JrRJTk7GunXrMGLECISGhkKhUGDDhg2QyWTw9fWtN+apU6dw//59rd4DERER1dJ663MjIyP0798f/fv3b/aLnzhxAhUVFVi4cCHMzc3h4+OD0tJShIeHY9y4cY0u8Z2SkoKrV69i+fLl8PT0BFB7JmXJkiWIj4+Hj48PgNoFu3r06IEZM2YAALy9vZGeno79+/fXCxVFRUX4/vvv8cc//hFbt25t9nsjIiJqa57p2ttxcXHw9fVVCw+BgYGoqKhAYmJio/0UCgWsra2FQAEArq6ukMvliIuLAwBUVlbi2rVrGDhwoFrfgIAApKSkoKSkRO14WFgYunfvDm9vbx28MyIioran0TMVUVFRAIB+/frBzMxMeCzG0KFDRbXLyMiot326vb09TExMoFQqNfZzdHSsd9zR0REZGRkAgKysLFRXV9dr5+joCJVKBaVSCVdXVwBAWloaTp8+jbVr14qqm4iIiOprNFR8/fXXAAA3NzeYmZkJj8UQGyrq5mQ8TiaToaioSGO/hi6NyGQyZGdnA4DQ//HxLSwshDHqbN++HaNGjULHjh2F/ppERkYiMjISALBq1SrY29s/sU9Dfm9SL9IHTf1MELU+/E31vGqJ31ONhorZs2cDAGxtbdUeP2/Onz8PpVKJjz76SHSfkJAQhISECI9zc3NbojTSY/xMEFFr19TfU506dWr0uUZDRVBQkMbHuiCTyerNbQBqzyLUnVForF9hYWGD/erOTNT1f3z8R89gVFVV4T//+Q/Gjx8PlUqF4uJilJaWAgDKy8tRWloKMzOzpr05IiKiNkbruz906dE5EHVyc3NRXl6uMQk5Ojri5MmT9Y4rlUr07dsXANChQwcYGhoiIyNDbUKnUqmERCJBp06dUF5ejvv372PXrl3YtWuX2lhfffUVOnTogI0bNzbnLRIREbUZzzRU+Pn54dChQ2pnBKKjoyGVStWCwON69eqFiIgIJCcnw8PDAwBw69YtZGVlCcuEGxsbw9vbGxcuXMDw4cOFvtHR0XB3d4e5uTmqq6uxbNkytbHz8/OxYcMGvPrqq7wThIiISAuNhor333+/SQNKJBLRf7sfPnw4jh49irVr12L8+PHIzs5GeHg4xo4dqzYRc+7cufD09BTmdbi7u8PX1xebNm3C9OnTIZFIsGfPHnh4eAhrVADAxIkTsXz5cuzYsQN9+/aFQqGAQqHAkiVLAACGhob17j6pm6jZpUsXuLm5NelnQERE1BY1GipUKlWTBtSmn4WFBT799FNs27YNq1evhkwmw5gxYzBlyhS1djU1NaipqVE7Nn/+fOzcuRNbtmyBSqWCv78/QkND1dp4eHhgwYIFCAsLw/HjxyGXyzFv3rwGV9MkIiKi5pGompoeSKBpTQ1Nfv/wHR1X0jp9kVOB3GoV7A0lWNxe+qzLeSoc1vzrWZdApBPHDrWNW0oPHv0ChUW5sLSwx/g/LH7W5TwVI8c5NKmfpjmPz3RFTSIiInp+NDlUlJaWIjc3t8FbQomIiKjt0eruj+rqahw+fBgnT55UW3lSLpdj2LBheOmll2BoaKjzIomIiKj1Ex0qqqqq8Ne//hWJiYmQSCSwt7eHjY0N8vPzkZOTg++//x5xcXH4y1/+ovUW6ERERKT/RH/7HzlyBImJifD398cbb7wBB4f/TfDIzMzErl27cPnyZRw5cgQTJkxoiVqJiAAAq1evRm5uLuzt7bVaYp+IWpboORXnzp1D586d8eGHH6oFCgDo2LEjFi1ahM6dO+Ps2bM6L5KI6FG5ubnIzMzkHitErYzoUJGZmQk/Pz8YGDTcxcDAAH5+fsjKytJZcURERKQ/RF/+MDIyQllZmcY25eXlnKhJ9Ay9tfPXZ13CU1FdULvxX2ZBaZt5zzveHPisSyB6ItFnKl544QVcvHgRBQUFDT5fUFCACxcuwNnZWVe1ERERkR4RHSpGjhyJgoICfPzxxzh16hSysrJQUVGB7OxsnD59GkuXLkVBQQFGjhzZkvUSEQGmloC5de2/iajVEH35IyAgAKmpqTh48CD++c9/Nthm3LhxCAgI0FlxREQNMfQf96xLIKIGaLWgxGuvvYY+ffrg1KlTSE1NRUlJCczNzeHs7Izg4GC4u7u3VJ1ERETUyokOFYWFhZBIJHB3d2d4ICIionqeGCpiYmKwa9cuYVnujh07Yvr06ejTp0+LF0dERET6Q+NEzZSUFKxbt05tn4/MzEysW7cOKSkpLV4cERER6Q+NoeLIkSNQqVSYOHEivv32W3zzzTd45ZVXUFNTgyNHjjytGomIiEgPaLz8cePGDXh4eGDKlCnCsalTpyIxMZFnKoiIiEiNxjMVDx8+hJubW73jbm5ujS6CRURERG2TxlBRXV0NU1PTesdNTExQXV3dYkURERGR/hG9oiYRERGRJk+8pfTMmTNISEhQO5aTkwMAWLFiRb32EokEn376qY7KIyIiIn3xxFCRk5MjhIjHJSYm6rwgIiIi0k8aQ8WyZcueVh1ERESk5zSGCk9Pz6dVBxEREek5TtQkIiIinWCoICIiIp1gqCAiIiKdYKggIiIinWCoICIiIp1gqCAiIiKdYKggIiIinWCoICIiIp1odPGr/fv3N3nQSZMmNbkvERER6adGQ0V4eHiTB2WoICIiansaDRUN7ftx5MgRKBQKDB48GJ6enrCxsUF+fj4SEhJw7tw5+Pv7Y8yYMS1aMBEREbVOjYaKx/f9iIqKwm+//Ya//vWv6Natm9pzQUFBGDVqFJYtW4b+/fu3TKVERETUqomeqPnTTz9h4MCB9QJFHRcXFwwcOBA//fSTzoojIiIi/SE6VCiVStja2mpsY2trC6VS2eyiiIiISP+IDhVmZma4fv26xjbXr1+Hqalps4siIiIi/SM6VPj7+yMpKQm7du1CaWmp2nOlpaXYtWsXkpOT0bt3b50XSURERK1foxM1H/faa68hMTERP/30E06dOgVnZ2dYW1vj4cOHSE1NRWlpKeRyOV599dWWrJeIiIhaKdGhwtraGn/729/w3Xff4dy5c0hKShKek0qlGDZsGF599VVYWlq2SKFERETUuokOFQBgaWmJmTNn4p133kFGRgZKSkpgbm4OR0dHGBoatlSNREREpAe0ChV1DA0N0aVLF13XQkRERHpM61BRVVWFa9euIT09HWVlZcKS3BUVFSgtLYWlpSUMDLhPGRERUVujVaiIi4vDli1bkJ+fLxyrCxWpqan45JNPMHfuXAwaNEinRRIREVHrJ/qUwq1bt7BmzRpIJBK8+eabCAwMVHve3d0dcrkcly5d0nmRRERE1PqJDhURERGQSqVYtWoVRo8eDQcHh3ptXFxckJaWptMCiYiISD+IDhXXr19H3759YWNj02gbe3t7tUsjRERE1HaIDhVlZWWwsrLS2Ka8vBw1NTXNLoqIiIj0j+hQYWdnh3v37mlsk5qaig4dOjS7KCIiItI/okOFn58frl69iuTk5AafVygUSElJgb+/v86KIyIiIv0h+pbSl19+GdHR0fj8888xatQo5OTkAACuXLmCxMREHDt2DDY2Nhg7dqxWBaSnp2P79u1ISUmBTCZDcHAwJk+e/MS1LkpKSrBjxw7ExMSgpqYGvXv3RmhoaL1lwmNiYrB3715kZmZCLpdj8uTJCAgIEJ6/efMmjh8/jqSkJOTl5aFdu3YYNGgQxo8fD6lUqtV7ISIiastEhwo7OzssXboU69evx+HDh4Xjq1evBgB06NABixYteuK8i0cVFRVh5cqVcHJywuLFi5GZmYndu3dDpVJh2rRpGvuuX78eSqUSM2fOhIGBAfbs2YM1a9bgs88+E9okJydj3bp1GDFiBEJDQ6FQKLBhwwbIZDL4+voCAKKjo5GVlYXx48fDwcEBaWlpCAsLQ1paGhYtWiT6vRAREbV1Wi1+1a1bN2zYsAFXrlxBSkoKCgsLYW5uDjc3N/Tt21fr/T9OnDiBiooKLFy4EObm5vDx8UFpaSnCw8Mxbtw4mJubN9gvJSUFV69exfLly+Hp6QmgNvQsWbIE8fHx8PHxAVB7G2yPHj0wY8YMAIC3tzfS09Oxf/9+IVRMmDBBLQh5eXlBKpXim2++QU5ODtq3b6/VeyIiImqrtF5P28DAAH369MFrr72GmTNnYvr06RgwYECTNhSLi4uDr6+vWngIDAxERUUFEhMTG+2nUChgbW0tBAoAcHV1hVwuR1xcHACgsrIS165dw8CBA9X6BgQEICUlBSUlJQDQ4JkVZ2dnAEBeXp7W74mIiKitEh0qVqxYgaioKI1tfvnlF6xYsUL0i2dkZKBTp05qx+zt7WFiYgKlUqmxn6OjY73jjo6OyMjIAABkZWWhurq6XjtHR0eoVCqN46ekpEAikfBOFiIiIi2IvvyRmJgILy8vjW1yc3M1nmF4XHFxMWQyWb3jMpkMRUVFGvs1dGlEJpMhOzsbAIT+j49vYWEhjNGQ/Px8HDhwAEOGDIG1tXWDbSIjIxEZGQkAWLVqFezt7RutVZPfm9SL9EFTPxNEjXl2nyn+pnpetcRnqklbnzemoqKiSZdBWouqqiqsX78epqamePPNNxttFxISgpCQEOFxbm7u0yiP9Ag/E6Rr/EyRrjX1M/X4FYZH6WSPcpVKhZycHCgUCrRr1050P5lMJsxteFRxcbFwRqGxfqWlpQ32qzszUdf/8fEbO4OhUqmwadMm3Lt3Dx9//LHG1yciIqL6NJ6pmDp1qtrj8PBwhIeHaxzw5ZdfFv3ij86BqJObm4vy8nKNScjR0REnT56sd1ypVKJv374Aam9xNTQ0REZGhtqETqVSCYlEUm/8ujUvPvnkkwbnaxAREZFmGkNFjx49IJFIANTOqbC3t4dcLq/XzsDAABYWFujZsyeCg4NFv7ifnx8OHTqE0tJSmJmZAahdN0IqlaoFgcf16tULERERSE5OhoeHB4DardmzsrLg5+cHADA2Noa3tzcuXLiA4cOHC32jo6Ph7u6uNifjhx9+wM8//4wPPvhAGI+IiIi0ozFULF++XPjz1KlT8eKLL2LSpEk6e/Hhw4fj6NGjWLt2LcaPH4/s7GyEh4dj7Nixal/6c+fOhaenJ2bPng0AcHd3h6+vLzZt2oTp06dDIpFgz5498PDwENaoAICJEydi+fLl2LFjB/r27QuFQgGFQoElS5YIbc6dO4fvv/8eQUFBsLOzQ0pKivBcx44dtVrMi4iIqC0TPVFz06ZNDd6p0RwWFhb49NNPsW3bNqxevRoymQxjxozBlClT1NrV1NTU2/10/vz52LlzJ7Zs2QKVSgV/f3+EhoaqtfHw8MCCBQsQFhaG48ePQy6XY968ecLCVwBw9epVAMCZM2dw5swZtf7vvfcegoKCdPeGiYiInmMSlUqletZF6DtNa15o8vuH7+i4ktbpi5wK5FarYG8oweL2bWM/FYc1/3omr/vWzl+fyetSy9vx5sAnN2oBxw61jVtKDx79AoVFubC0sMf4Pyx+1uU8FSPHOTSpn6Y5j1rfUpqXl4fffvsNDx48QFVVVYNtdHmJhIiIiPSDVqFi3759+PHHH1FdXa2xHUMFERHpEwuZrdq/qWlEh4qzZ88iIiIC3t7eGDlyJNatW4ehQ4fC19cXCQkJOH36NAYMGKB2pwUREZE+GDbk3WddwnNBdKg4fvy4sBNo3aqZcrkcgYGBCAwMRL9+/bBq1SoEBga2WLFERETUeoleUfPu3bvo1auX2jLcj96R4efnB19fXxw+fFi3FRIREZFeEB0qqqurYWlpKTyWSqX1lsDu3LkzUlNTdVYcERER6Q/RocLW1hZ5eXnCY3t7e6Slpam1ycvL0+sNxYiIiKjpRIcKZ2dn3Lt3T3js5eWF5ORk/PLLLygrK8OVK1dw4cIFdO3atUUKJf1lawjYG0pgy7xJRPRcEz1Rs3fv3vjXv/6F7OxsyOVyTJgwAb/++is2b96MzZs31w5mZFRvEzKid+3axoJXRERtnehQERQUpLZktb29Pf7+97/j8OHDyMrKQvv27TFy5Eh06dKlJeokIiKiVk7rFTUfJZfL8fbbb+uqFiIiItJjoudUEBEREWmi9ZmKmpoaPHjwQOPeH56ens0ujIiIiPSLVqHi0KFDOHz4MAoKCjS2CwsLa1ZRREREpH9Eh4p9+/YhIiICFhYWGDp0KOzs7LgmBREREQlEh4rTp09DLpdj9erVMDc3b8maiIiISA+JnqhZWFiIPn36MFAQERFRg0SHio4dO6K4uLglayEiIiI9JjpUjBgxApcvX0Z+fn4LlkNERET6SvScihEjRuD333/HJ598gokTJ6Jbt26NXgqxt7fXWYFERESkH7S6pfSFF17AmTNnsGXLlkbbSCQS7N27t9mFERERkX4RHSpOnjyJb775BoaGhvDy8oKtrS1vKSUiIiKB6FBx+PBhWFtb4/PPP4dcLm/JmoiIiEgPiZ6omZOTgwEDBjBQEBERUYNEhwo7O7tG9/ogIiIiEh0qhg4dCoVCgdLS0pash4iIiPSU6FDx8ssvw9XVFStXrkRCQgLDBREREakRPVHztddeE/782WefNdqOt5QSERG1TaJDRY8ePSCRSFqyFiIiItJjokPF8uXLW7AMIiIi0nei51QQERERacJQQURERDrR6OWP/fv3AwBGjRoFCwsL4bEYkyZNan5lREREpFcaDRXh4eEAgICAAFhYWAiPxWCoICIiansaDRXLli0D8L9tzOseExERETWk0VDh6emp8TERERHRo0RP1IyKikJaWprGNnfv3kVUVFSziyIiIiL9IzpUfP3114iJidHYJjY2Fl9//XWziyIiIiL9o9NbSmtqarjqJhERURul01ChVCohk8l0OSQRERHpCY3LdD9+KSMmJgbZ2dn12tXU1OD+/ftISkqCv7+/biskIiIivaAxVDw+6TI1NRWpqamNtndzc8Obb76pk8KIiIhIv2gMFZs2bQIAqFQqzJ07F6NHj8bo0aPrtTMwMIBMJoOpqWnLVElEREStnsZQ0b59e+HPkyZNgpeXl9oxIiIiojqitz6fPHlyS9ZBREREek50qLhz5w5SUlIwePBgmJubAwDKysrwr3/9C7GxsTAxMcH48eMbvDxCREREzz/Rt5QePHgQBw4cEAIFAHz33Xc4e/YsVCoVCgsLsXPnTly9erVFCiUiIqLWTXSouHXrFry8vITHVVVViIqKgqurK7799lts2rQJVlZWOHr0aIsUSkRERK2b6FBRUFCAdu3aCY9v376NsrIyhISEQCqVws7ODn369Hni/iBERET0fNJqRc3q6mrhz8nJyQDUdy+1srJCQUGBjkojIiIifSI6VNjb2+PGjRvC45iYGLRr1w4dOnQQjuXl5cHCwkK3FRIREZFeEH33x8CBAxEeHo5169bB2NgYKSkpGDNmjFqbjIwMtZBBREREbYfoUDF27FhcvXoVly5dAgA4Oztj0qRJwvPZ2dm4efMmXn75Za0KSE9Px/bt25GSkgKZTIbg4GBMnjwZBgaaT6KUlJRgx44diImJQU1NDXr37o3Q0FBYWlqqtYuJicHevXuRmZkJuVyOyZMnIyAgoEljERERUeNEhwpTU1OsXLkSd+/eBQA4OTnV++JftGgRXFxcRL94UVERVq5cCScnJyxevBiZmZnYvXs3VCoVpk2bprHv+vXroVQqMXPmTBgYGGDPnj1Ys2YNPvvsM6FNcnIy1q1bhxEjRiA0NBQKhQIbNmyATCaDr6+vVmMRERGRZqJDRZ0uXbo0eFwul0Mul2s11okTJ1BRUYGFCxfC3NwcPj4+KC0tRXh4OMaNG6e2JsajUlJScPXqVSxfvlyYKGpnZ4clS5YgPj4ePj4+AICIiAj06NEDM2bMAAB4e3sjPT0d+/fvF0KF2LGIiIhIM43XGBITE5Gbmyt6sLS0tHo7m2oSFxcHX19ftfAQGBiIiooKJCYmNtpPoVDA2tpa7c4TV1dXyOVyxMXFAQAqKytx7do1DBw4UK1vQEAAUlJSUFJSInosIiIiejKNoWLFihU4c+aM2rEff/xR+Jv/4y5duoSvv/5a9ItnZGSgU6dOasfs7e1hYmICpVKpsZ+jo2O9446OjsjIyAAAZGVlobq6ul47R0dHqFQqYXwxYxEREdGTaX35o7KyEsXFxTp58eLiYshksnrHZTIZioqKNPZr6NKITCZDdnY2AAj9Hx+/7pbXuvcgZqzHRUZGIjIyEgCwatWqesFIrE57/tukfkSNOf7xxGddAj1nQmc17fcbtU1aLX5FtUJCQrBq1SqsWrXqWZeiN/785z8/6xLoOcPPFOkaP1PN90xDhUwmE+Y2PKq4uFjjIloymQylpaUN9qs7M1HX//HxHz+DIWYsIiIierJnGioamreQm5uL8vJyjZcUGpvvoFQqhfkRHTp0gKGhYb12SqUSEolEGF/MWERERPRkzzRU+Pn54erVq2pnCqKjoyGVStXuxnhcr169kJ+fL+w/AtTuopqVlQU/Pz8AgLGxMby9vXHhwgW1vtHR0XB3dxfmUYgZi5ovJCTkWZdAzxl+pkjX+JlqvmcaKoYPHw5jY2OsXbsW8fHxiIyMRHh4OMaOHas2eXLu3LnYsmWL8Njd3R2+vr7YtGkTLl68iEuXLuEf//gHPDw81NaVmDhxIhISErBjxw4kJCTgP//5DxQKhdpKoGLHoubh/6yka/xMka7xM9V8EpVKpWrsyalTpzZp0LCwMNFt09PTsW3bNrVluqdMmaK2WuecOXPg6emJOXPmCMeKi4uxc+dOXLp0CSqVCv7+/ggNDYWVlZXa+JcuXUJYWBh+//13YZnuwMBAtTZixyIiIqLGPfNQQURERM8HjaGCiIiISCytF7+itik7Oxvvv/8++vfvj4ULFwIANm/erLYsu0QigampKbp06YKgoCAEBwdDIpEgISEBK1asQEBAAObPn19v7KVLl+LGjRsYNWpUg6u1zps3D9nZ2di+fXuj+8GQfqv7fD3KxMQEFhYW6Ny5M7y9vREUFFTvkuS+ffuwf/9+AMBrr72GCRMmNDh+3WcMANauXdvoHkb0fNHF5+rRfg4ODhgwYADGjh0LqVTa4vXrI4YKarYRI0bAysoKNTU1yMnJwcWLF3H9+nXcuXMH77zzDtzc3GBsbIykpKR6fcvKynD79m1IJJIGn3/w4AEyMzPRrVs3Boo2wNHRUdivp6KiAnl5eUhOTkZcXBwiIiLwzjvvYPDgwfX6GRoaIioqqsFQkZ6ejhs3bsDQ0BDV1dUt/RaoFWrq5yowMBAODg4AgLy8PMTExGDv3r1ISEjAJ5988lTfg75gqKBmGzFihNrf/CZMmICPP/4YJ06cwEsvvYQOHTrA1dUVSUlJyMzMRMeOHYW2KSkpqK6uRt++fREbG4uioiK1hc/qNpbTdIsxPT+cnJwwZcoUtWMqlQrnzp3Dt99+i02bNkEmk8Hf31+tja+vL65cuYKbN2/C1dVV7bkzZ87A0NAQPXv25CaBbVRTP1eDBg1C7969hcd//OMfsWjRIvz222+4du0avL29n0r9+oTLdJPOde7cGV5eXlCpVLh9+zYAwMvLCwDq7T6bmJgIY2NjjBs3DiqVqt7Zirr2df2p7ZFIJBg8eDDeffddqFQq7N69G49PBQsICICxsXG9DRBrampw9uxZ+Pr6wtra+ilWTa2dmM/V4ywsLNCnTx8AEH63kTqGCmpREokEwP/ONDQUKlxdXeHm5gYzM7N6zyclJUEikaBHjx5Pp2BqtQYNGgS5XI6MjAykpaWpPSeTydCnTx9ER0ejqqpKOH716lXk5eUhKCjoKVdL+kLT50oTQ0PDFqxKfzFUkM6lp6cjMTEREokE3bp1A1C7yJixsbFaaKioqMDNmzfRo0cPGBgYoHv37mrP5+fnIyMjA87OzpxPQZBIJPDw8ADQ8N8Sg4KCUFRUhNjYWOHYmTNn1P52SfS4J32uHvXo58vd3b3Fa9NHnFNBzXb8+HFYWVlBpVIJEzXLy8sxatQoyOVyAIBUKhXmVWRnZ0MulyMlJQVVVVXCWQgPDw+EhYWhpKQE5ubmwqUQXvqgOra2tgCAwsLCes/5+vrC1tYWUVFRGDBgAIqLixEbG4vg4GAYGfFXHTWusc/VuXPncOvWLQD/m6hZUFCA4cOHw83N7anXqQ/4fxo12/HjxwH875ZSZ2dnvPjii3jxxRfV2nl6eiIpKQmJiYmQy+VITEyEoaEhunfvLjyvUqmQnJwMf39/TtIkrRgYGGDw4MH46aef8PDhQ1y8eBGVlZW89EFNdv78+XrHQkJC8O677z6DavQDQwU1m9j7/j09PREREYHExEQEBQUhKSkJXbt2hampKQDAxcVFuERSFyo4n4IelZeXBwCNLqEfFBSEQ4cO4ezZs4iOjkbnzp3h4uLyNEskPdTY5+qjjz5C7969UVVVhXv37mH79u2IjIzECy+8gJEjRz6LUls9zqmgp6Z79+4wMjJCUlISKisrkZKSohYYjI2N4erqisTERBQWFiI9PR3Ozs6QyWTPsGpqLerOYgEQ5uo8zsnJCS4uLjh8+DBu3ryJoUOHPs0SSQ+J+VwZGRmha9eu+POf/wxra2vs2rUL9+/ff5pl6g2GCnpq6uZVZGVlCaemHz8L0aNHD9y5cwcKhQIqlYqXPkhw/vx5ZGdnw9HRUeOZsaCgIOTl5cHAwABDhgx5ihWSPhL7uQJq7zKaPHkyKisrERER8ZQq1C8MFfRU1YWEH374QW3WdZ0ePXqguroaBw8eVGtPbVfdIkXffPMNJBIJ3njjDeFW5YYMGTIEixYtwtKlS2FjY/P0CiW9ou3nqk5wcDDatWuH06dPIzc39ylUql84p4KeKk9PTxw4cAD37t3DCy+8oLZ6JlB7icTAwAD37t3jfIo2KD09Hfv27QMAVFZWIi8vD0lJScjJyYGZmRnef/999OrVS+MYZmZm6Nev39Mol/SELj5XdYyMjDBhwgRs27YNBw4cwJ/+9KeWLF3vMFTQU1U3r+LRW0kfZWpqiq5du+LWrVsNhg56vmVkZAgbOT268dPIkSMb3PiJSAxdf66Cg4Pxww8/4MyZM3jllVdgb2/fEmXrJW59TkRERDrBORVERESkEwwVREREpBMMFURERKQTDBVERESkEwwVREREpBMMFURERKQTDBVERESkEwwVRKT3EhISMGXKFEyZMuVZl0LUpnFFTSIdqqioQFRUFC5fvoy0tDQUFBTAyMgIdnZ28PDwQGBgILy9vTWOMWfOHOTk5NQ7bmpqivbt26NHjx4YNWoUnJyc6rVZvnw5EhMTRdXq6emJ5cuXi2r7pNoaMnToUMyZM0er8R9XXFyMn376CQAwZsyY53LH2jNnziA7OxteXl7w8vJ61uUQNQtDBZGOxMfHY8uWLWpbIpuZmaGqqgoZGRnIyMjAyZMn0atXL7z//vuwtLTUOJ6xsTHMzc0B1G5+VFhYiHv37uHevXs4efIk3n33XQQHBzfY19DQ8IlLnDdnCfRHa2vMk54Xo7i4WFheOSgoqNFQYWJigk6dOjX79Z6FM2fOCEGQoYL0HUMFkQ5ER0dj48aNqK6uhp2dHaZMmYJ+/foJX9wZGRk4ceIEjh07BoVCgaVLl2LlypWwtrZudMyAgAC1v+lXVFTg8uXL2L59Ox4+fIhvvvkGLi4ueOGFF+r17d69u9ZnIbTxeG3PmqurK7766qtnXQZRm8c5FUTNlJ6eji1btqC6uhpdunTBF198geDgYLUzAY6Ojnjrrbfw4YcfwsjICJmZmfjHP/6h1etIpVIMHDgQc+fOBQDU1NTg+PHjOn0vRETNwTMVRM20d+9elJeXw9jYGAsWLNC446G/vz9eeeUV7Nu3D7/99huuXLkCf39/rV7Px8cHtra2yMvLw61bt5pb/lN1//59HD58GPHx8cjJyUF1dTUsLS1hY2ODHj16YNCgQXB1dQVQf37I+++/rzbWo3NCEhISsGLFCgAQtriuc+bMGXz99ddo3749Nm/ejKSkJBw8eBA3b95EeXk5HBwcMGrUKLVLSVeuXMFPP/2E1NRUlJeXo3PnznjppZcQEBDQ4PvKzs5GdHQ0EhISkJ2djQcPHgAA7O3t4evri7Fjx9bbybKurjr79+8XLvXU2bRpE+RyufC4pqYGZ86cwdmzZ3H37l2UlpbC0tIS3bt3x8iRIxu9fFL3s5w0aRJeeeUVHD16FOfPn0dmZiZKSkqwbNkyoW9GRgaOHDmCxMRE3L9/HyqVClZWVrCzs4OXlxeGDh0KR0fHBl+HiKGCqBny8vIQExMDAAgMDBR1XX/s2LE4fPgwSktLcezYMa1DBQDY2dkhLy8PpaWlWvd9VlJTU7FixQoUFxcDAAwMDGBmZob8/Hzk5eXhzp07KC4uFkKFhYUFLC0tUVhYCACwtLSEgcH/Tq42ZU7IyZMn8c033wCone9SXl6O1NRUbN26FZmZmXjttdewb98+7N+/HxKJBGZmZqioqMCtW7fw1VdfoaioCCNGjKg37tdffy0EICMjI5iZmaGoqEiYS3PmzBn8+c9/hoeHh9BHKpXC2toaRUVFqK6uhomJCUxNTdXGffT9lpSUYM2aNUhISKj387tw4QIuXLiAl156CdOnT2/0/VdWVmLFihW4fv06DA0NYWpqColEIjwfHx+P1atXo7KyEgCENvfv38f9+/dx48YNGBkZ8S4bahRDBVEzJCQkQKVSAQD69+8vqo+pqSl8fHxw8eJFJCUlobq6GoaGhlq9bt0dGM2ZbPm07d69G8XFxejatSvefvttuLm5QSKRoKqqCjk5OYiNjRV+lgCwaNEiZGdnC2co/v73v6v9rV1bBQUF2LZtG0aNGoWJEyfCysoKRUVF2LlzJ6KionDw4EHIZDIcOHAA06ZNw6hRo2Bubo68vDxs2bIFcXFx2L17NwYNGlRvEqqzszMGDhwIHx8fdOjQAQYGBqiursadO3ewb98+xMXFYf369di4cSOkUimA2nkpAQEBwlmEl156SeOX9ZYtW5CQkAAjIyNMnz4dwcHBMDExQX5+Pr7//nucPn0ahw8fRocOHRoMPgBw7NgxAMB7772HgIAASKVSFBYWCsHi22+/RWVlJXx9fTF9+nR06dIFQO18nqysLFy8eLHeGReiRzFUEDVDenq68OeuXbuK7ufs7IyLFy+irKwMOTk56Nixo+i+Fy5cQEFBAQDAzc2twTbXr1/Hu+++q3Gc0NDQRk/nP0l0dDTi4uI0tlm0aBG6d++uVhMAvP3223B3dxeOGxkZwcHBAS+99FKTahGrvLwcwcHBCA0NFY5ZWFhg9uzZSEpKQnZ2Nvbs2YNp06bhlVdeEdrY2tpi/vz5mDlzJsrLyxEbG4shQ4aojf3WW2/Vez1DQ0O4urriz3/+Mz766COkpaXhwoUL9fqKcePGDVy8eBEAMGPGDISEhAjP2djYYPbs2SgpKcHFixcRFhaGoKAgIbw8qqysDIsXL0afPn2EY3V3IT18+BBZWVkAakOHra2t0EYqlaJz587o3Lmz1rVT28KJmkTNUHdqHtDurMGjt5MWFRU9sb1KpUJOTg6OHj2KLVu2AKj9Mh45cmSD7aurq/Hw4UON/1RUVIiu93GVlZVPHL+qqkqtT93toHl5eU1+3eaaMGFCvWMGBgbC2iHGxsYYPXp0vTbm5uZCELp7965Wr2lgYABfX18AQHJyspYV14qOjgYAtGvXrtHbiKdOnQqg9jMZHx/fYJvOnTurBYpHmZmZCWcsnuV/I9JvPFNB1EpFRUUhKiqqwedMTU0xZ84cODg4NPh8Uxa20kZTFrby9/fHyZMnsXnzZly/fh19+vSBi4sLTExMWqhKdRYWFo2eEbKxsQEAODk51ZvXUKfu9t/GQmBSUhJOnTqFGzdu4P79+ygvL6/Xpm4Cp7Zu374NoHYdi0fnWTzKyckJdnZ2ePDgAW7fvt1geHj0zNHjpFIpevbsifj4ePztb3/D8OHD4e/vj65du8LIiF8VJA4/KUTN8PgZBzs7O1H9xJzheHSBKYlEAhMTE9jb26NHjx4YNmwY2rVr14zKn77XX38dmZmZSEhIwJEjR3DkyBEYGBjA2dkZ/v7+CAkJEf3zawozM7NGn6v7otbUpm7eS3V1db3n/vOf/+DQoUNq48lkMuHLuKysDOXl5Q0GDTEePnwIAE/8+bRr1w4PHjwQ2j9O051JADBr1iysXr0aaWlpiIiIQEREBIyMjODi4oK+ffvWu1Wa6HEMFUTN8OhS2bdv3xb9pXjnzh0A/1t6uyGtbYGp5pLJZFi2bBmSk5MRGxuL69ev4/bt28I/hw4dwqxZszBo0KBnXapW4uPjhUAxYsQIjBgxAk5OTmpnFPbu3YsDBw6oTUR9Fho7y1HH3t4eq1evRnx8PBQKBa5fv460tDRcv34d169fxw8//ICFCxc+cal5arsYKoiawcvLCxKJBCqVChcvXmz0evWjysrK8NtvvwEAevToofWdH/rOw8NDuLWyoqIC8fHx2Lt3L+7evYstW7bA29tbuByhD86fPw8A8PX1xTvvvNNgm/z8/Ga9hrW1NZRKpdoS8A2pe17TSq1PYmBgAD8/P/j5+QEASktLcfnyZXz33XfIzc3Fhg0bsGXLFl4SoQZxoiZRM9ja2qJv374AaifTKZXKJ/Y5cuSIsL5EY7f+tRVSqRR9+vTBokWLANROAH10MuOT/mbdGtR9kTd2949KpRLWlmjIo+tENKZbt24Aam9hrqmpabBNRkaGMGfDxcXliWOKZWZmhkGDBmHWrFkAai/FaDtZldqO1v9/LFErN3XqVEilUlRWVuLLL78UbvdsiEKhwIEDBwDUnuVoysJX+qi6urrRL0MAarc/PhokHp3jULdoVmtTN+8lLS2twedPnDgh3KrZkLr3qOn9BQYGAqid6Hnq1KkG24SFhQGonefTs2fPJxf+mMfv1nnco/+NxAQhapsYKoiaqXPnzpg1axYMDAxw9+5dfPTRRzh16pTal4RSqcTOnTvxxRdfoKqqCh06dMD//d//tZlfzvfv38f//d//ISIiAnfu3FGb7JiWloaNGzcCqN1t1NPTU3hOJpMJ81ROnz7d4CTJZ63uMoFCocD+/ftRVlYGoDYkHDhwANu3b9e4I23dAlMKhaLRu0NcXV2FxdW2b9+On3/+WZj0mZ+fj61bt+LChQsA/hdytXX9+nUsWrQIR44cQXp6uhACVSoVrl+/jn/9618AaieDNrSJHRHAORVEOjFo0CBYWFgIW59v3boVW7duhbm5OSorK4Vlj4Haa+9z58594kz85hCz+BVQu4JiU4hZ/Mre3h5///vfhcdZWVkICwtDWFgYDAwMYG5ujrKyMuFvyEZGRpgzZ069uwuGDx+OsLAw/Pzzzzh58iSsrKxgYGAANzc3zJ8/v0n169KQIUMQFRWFpKQk7Nu3D+Hh4TA3N0dJSQlUKhX8/f3h7OwsnKF63NChQ3H48GFkZmZi9uzZsLKyEkLBZ599JtzlM3v2bBQWFiIxMRHbt2/Hzp07YWpqKrwOALz00kvNuqR29+5d7Nq1C7t27YKhoaHwPurCnJmZGebNm6cXl6Xo2WCoINIRPz8/bNy4EWfOnMHly5eRlpaGwsJCGBkZCbeCBgYGNunUtLbqFr9qKXWLX2ny6N+W7ezssHjxYiQkJCAlJUW47dHQ0BAdO3aEl5cXRo8e3eC6Gy+//DLMzMxw9uxZYd6ASqVq9K6Zp83IyAhLly7Fjz/+iPPnzwtLqLu6umLo0KEICQmpt1HYoxwcHLBs2TL8+OOPuHHjhrAXCKB++6q5uTk+/fRTYUOx1NRUlJWVwcbGBu7u7hg1alSjG4qJ4eLigg8++AAJCQm4efMm8vLyUFBQAGNjY3Tu3Bk+Pj4YPXp0i972S/pPonrW9zgRERHRc4HnsIiIiEgnGCqIiIhIJxgqiIiISCcYKoiIiEgnGCqIiIhIJxgqiIiISCcYKoiIiEgnGCqIiIhIJxgqiIiISCcYKoiIiEgn/h/0YVDVlf7ZZQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualize the estimated policy values of BernoulliTS and their 95% confidence intervals (estimated by bootstrap)\n",
    "ope.visualize_off_policy_estimates(\n",
    "    action_dist=action_dist,\n",
    "    estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
    "    n_bootstrap_samples=10000, # number of resampling performed in bootstrap sampling\n",
    "    random_state=12345,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgIAAAGSCAYAAACRy6kSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABG+UlEQVR4nO3de1xUdf4/8NcMMFzl5jiKoIJcBRVEvIEKIl4yNTPTbNfMtNTUtjbTNrfUrBUztfLWllra11Yl3by0rYkXvJAIiqKAoCIoEAICch0uw/z+8MesIzCcgRkQ5/V8PHroOedzznkPjcxrzvmcz0ekVCqVICIiIoMkbusCiIiIqO0wCBARERkwBgEiIiIDxiBARERkwBgEiIiIDBiDABERkQFjECAiIjJgDAJEREQGjEGAiIjIgBkLbZidnY2rV68iOTkZ+fn5KCkpgUQigbW1NZydneHj44PevXtDIpHos14iIiLSIVFTQwyfO3cOv/32G65fv97kwSwtLRESEoKxY8dCJpPprEgiIiLSj0aDwLVr17Br1y5kZGTAwsICAwYMgJeXF1xdXWFrawsrKytUVVWhpKQE2dnZSE1NRUJCAm7cuAFjY2M888wzmDx5MiwsLFr7NREREZFAjQaBadOmwcXFBc899xwCAgJgYmIi6IB//PEHjh07hmPHjuG5557DlClTdFowERER6U6jQeDChQsYOHBgsw9cVFSE3NxceHh4NPsYREREpF9N9hEgIiKip5fgpwaeNtnZ2W1dAhERUavo2rVro9s4jgAREZEB03hFYOHChVofUCQSYePGjc0uiIiIiFqPxiCQl5fXWnUQERFRG9DYWbC5QaBTp07NLqi1sI8AEREZCk19BDReEWgPH+hERETUfOwsSEREZMA0XhGora3FF198AZFIhEWLFsHYuOHmNTU12LhxI0QiEd5++2191ElERER6oPGKQExMDGJiYhAQENBoCAAAY2NjDBgwAL///jvOnz+v8yKJiIhIPzQGgd9//x329vYYOnRokwcKCgqCvb09zp49q7PiiIiISL80BoFbt27Bx8cHIpGoyQOJRCL07t0baWlpOiuOiIiI9EtjH4GioiJ07NhR8MHs7e3x4MEDwe1///13nD59GmlpaSgvL0fXrl0xYcKEJq9ATJ06td46d3d3fPrpp4LPTURERE0EAWNjY1RXVws+WHV1tca+BI87cuQIZDIZZs6cCWtra1y6dAlfffUVSkpK8Mwzz2jcd/z48Rg8eLBq2dzcXPB5iYiI6CGNn9p2dnbIyMgQfLCMjAzY2dkJbr906VJYW1urlnv37o3CwkIcOXKkySAgk8k4xTERkQFbs2YN8vPzIZVKsXTp0rYup93S2EfA09MTSUlJyMnJafJAOTk5SEpKgpeXl+CTPxoC6ri4uKCwsFDwMYiIyDDl5+cjJycH+fn5bV1Ku6YxCIwaNQq1tbVYv369xnv/xcXF2LBhA2praxEWFtaiglJTU+Hg4NBku4iICLz00kuYPXs2tmzZgtLS0hadl4iIyBBpvDXg5uaGsLAwREZG4q9//StGjRqF3r17w97eHgBQUFCAa9euITIyEiUlJRg1ahTc3NyaXczVq1cRGxuL+fPna2wXHByM/v37w9raGmlpadi/fz8yMjKwevVqiMUNZ5vIyEhERkYCAMLDwyGVSptdJxERtT0jIyPVn/yd3nwaJx0CAIVCgW3btuHEiRMaDzRy5EjMmTOn0Q/ipuTm5mLZsmXw8PDAe++9p9W+8fHxWL16NRYvXoyBAwcK2oeTDhERtW/vvfcecnJy0KVLF6xdu7aty3miNXvSIeBh0po7dy5CQkJw7NgxpKSkoKioCABga2sLLy8vhIWFwdPTs9kFlpaWYvXq1ZBKpXjrrbe03t/Pzw9mZma4ffu24CBAmrETDhGRYRD8rJ+np2eLPuwbU1lZifDwcNTU1OD999+Hqamp1seoG/BIyMBHJExdJxwiInq6tensgwqFAuvXr8cff/yBDz74ADY2Ns06zuXLlyGXy9GzZ08dV0hERPR0Ez76jx5s27YN8fHxePXVV1FSUoKSkhLVNhcXF5iYmODjjz8GAHz00UcAHnb6u3XrFvr06aPqLHjgwAG4ubnB39+/TV4HERFRe9WmQSAhIQEA8P3339fbtmnTJshkMtTW1qqt79y5M6KiohATE4OKigrY2tpi+PDheOmll5rdUZGIiMhQtWkQ2Lx5c5NtVqxYobbcp08f9OnTR08VERERGRZ+hSYiIjJgDAJEREQGjEGAiIjIgLWoj0Bubi4yMzMBAE5OTpDJZDopioiIiFpHs4JARUUFvv76a5w/f15t/ZAhQzBv3jyYmZnppDgiIiLSr2YFge3btyMhIQFTp05Fz549UV1djbi4OERFRcHU1LTJSYOIiIjoyaAxCFRWVjY45G9sbCzmzJmDYcOGqdYNHDgQlZWVuHDhAoMAERFRO6Gxs+DixYtx7dq1eusVCgXMzc3rrTc3N683ABARERE9uTReEXB3d8eqVaswcuRIzJgxQ/Xh37t3b2zfvh1yuRwuLi6orq7GxYsXERUVhf79+7dK4URERNRyGoPAW2+9haFDh+Lbb79FfHw83njjDfTr1w9z5szB2rVrsXHjRrX2PXv2xGuvvabXgomIiEh3muws6O/vj3Xr1mHXrl0IDw/HsGHD8Oqrr2LNmjVISEhAVlYWgIePD3LoXyIiovZF0FMDFhYWmDdvHgIDA/HNN9/g3XffxezZszFw4ED07dtX3zUSERGRnmg1smDfvn3x+eefY+DAgVi3bh02bNiA4uJifdVGREREeiYoCBQXFyMtLQ3FxcUwMzPD7NmzsWLFCqSnp+Odd97B2bNn9V0nERER6YHGWwNyuRxbt25VG0Fw0KBBePPNN9GrVy+sXbsWe/bswebNmxEdHY033ngDtra2+q6ZiIiIdETjFYEff/wR58+fR3BwMGbPno2QkBDExMRg9+7dAACJRIJXXnkFq1atQk5ODt555x2cPHmyVQonIiKiltN4RSA2NlZ1BaBORUUF4uLiMHv2bNU6Nzc3fPbZZ/jpp5/w7bffYsSIEfqrmIiIiHSmySGGO3bsqLauY8eODY42aGxsjJdeegmDBw/WbYVERESkNxpvDbi7u+P06dO4fv06ampqkJqaijNnzsDd3b3RfZydnXVdIxEREemJxisCs2bNwsqVK7F8+XLVOnt7e7z66qv6rouIiIhagcYg0KVLF3zxxRe4ePEi8vPzIZVK4e/vDzMzs9aqj4iIiPSoyZEFTU1NERgY2Bq1EBERUSvTamRBIiIieroImmugIXFxcUhOTkZlZSVkMhkCAwMhlUp1WRsRERHpmcYg8OOPP6Jv377o3bu3al1ZWRk+++wzXL9+Xa3t3r17MXfuXAwfPlw/lRIREZHOaQwCBw8ehEQiUQsC//znP3H9+nXIZDIEBQXB2toaqamp+P333/H111/D2dkZ3bt313vhRERE1HJa3RrIyclBTEwMXFxcsHz5cpibmwMAxo0bB39/f2zevBn/+c9/MG/ePL0US0RERLqlVWfB5ORkAMD06dNVIaDO8OHD4ebmhqSkJN1VR0RERHqlVRAoKioCALi6uja43dXVFQUFBS0uioiIiFqHVkGg7iqAiYlJg9tNTEwgEolaXhURERG1iib7CCQmJqr+npOTAwDIy8uDk5NTvbb3799Hhw4ddFgeERER6VOTQSApKaneff9Lly41GATS0tLg6Oiou+qIiIhIrzQGgUcnG3qUtbV1vXVpaWlQKBTo06ePbiojIiIivdMYBLy9vQUfqGfPnti8eXOLCyIiIqLWw7kGiIiIDJhWAwopFArcu3cPZWVlEIlEsLGxQadOnfRVGxEREemZoCBw4cIFHD16FMnJyVAoFGrbrK2tERQUhEmTJsHW1lYfNRIREZGeaAwCSqUSW7ZswenTp+ttk0qlMDMzQ05ODn799VecOXMG7733Hry8vPRWLBEREemWxiAQGRmJ06dPw9/fH9OmTUPnzp1x79497Nu3DykpKVi2bBk6deqEc+fO4YcffsCaNWuwbt062Nvbt1b9RERE1AIaOwueOHECTk5OWLx4MZydnWFubg5nZ2e8++67sLW1xY8//ggTExOEhITgww8/hFwux88//9xKpRMREVFLaQwCmZmZ6NOnD4yMjNTWGxkZoU+fPmqjDjo7O8Pf3x/x8fH6qZSIiIh0TmMQEIlEqKqqanBbVVUVqqur1dY5Ojpy0iEiIqJ2RGMQ6NatG+Li4lBaWqq2vrS0FHFxcXBwcFBbL5fLIZFIdF8lERER6YXGzoIjRozAt99+iw8++ADjx4+HTCZDbm4ufvnlFzx48ADjx49Xa3/37l106dJFrwUTERGR7mgMAmFhYUhKSsK5c+ewfft2tW1+fn5qQaCiogJVVVUIDAzUT6VERESkc00OKPTWW29h8ODBuHDhAh48eIAOHTrA398fgYGBEIv/d2fB3Nwcn376qV6LJSIiIt0SNLLgwIEDMXDgQH3XQkRERK2Mkw4REREZMAYBIiIiA6bV7IO69vvvv+P06dNIS0tDeXk5unbtigkTJmDo0KEa96uursa//vUvnD59GpWVlfD29sbs2bMhk8laqXIiIqKnQ5teEThy5AjMzMwwc+ZMLF26FD4+Pvjqq6/w66+/atzvu+++w6lTpzBjxgz89a9/RUlJCT755JNGBz8iIiKihrXpFYGlS5fC2tpatdy7d28UFhbiyJEjeOaZZxrc5/79+zhx4gTmz5+P4OBgAECPHj2wYMECnDlzBiNHjmyV2omIiJ4GbXpF4NEQUMfFxQWFhYWN7nPlyhUAwKBBg1Tr7O3t4eXlxXkOiIiItPTEdRZMTU2tN3Txo7Kzs9GxY0eYmZmprXd0dER2dra+yyMiInqqtOmtgcddvXoVsbGxmD9/fqNtSktLYWFhUW+9lZUVysrKGt0vMjISkZGRAIDw8HBIpdKWF/wUq5tx0sjIiD8rInoi8feUbmgdBJKSkpCUlIQpU6Zota0pubm5+OqrrxAQEICQkBCt929KWFgYwsLCVMv5+fk6P8fTRKFQqP7kz4qInkT8PSVc165dG92m9a2BxMREREREaL1Nk9LSUqxevRpSqRRvvfWWxrZWVlYoLy9v8BiWlpZan5uIiMiQtXkfgcrKSoSHh6Ompgbvv/8+TE1NNbbv2rUr7t+/D7lcrrY+OztbY+IhIiKi+to0CCgUCqxfvx5//PEHPvjgA9jY2DS5j6+vLwDgwoULqnUFBQVITk5Gv3799FYrERHR06hNOwtu27YN8fHxePXVV1FSUoKSkhLVNhcXF5iYmODjjz8GAHz00UcAgI4dOyI0NBQ7d+4E8PARxIiICHTq1AnDhg1r/RdBRETUjgkKAo92wqjrmf94x4zm9NhMSEgAAHz//ff1tm3atAkymQy1tbX1ts2aNQumpqbYuXMnqqqq4O3tjb/85S+QSCRa10BERGTIBAWBBQsWaFwnEomwZ88erU++efPmJtusWLGi3joTExPMnDkTM2fO1PqcRERE9D+CgsALL7wAkUgEoGWPCBIREdGTRVAQmDp1qurvERERSEpKwosvvqi3ooiIiKh1tPnjg0RERNR2GASIiIgMGIMAERGRAdM6CCiVymZtIyIioieP1gMKTZ06Va3zoNBtRERE9OThrQEiIiIDxiBARERkwBoNAlVVVS0+uC6OQURERPrTaBBYsGAB/vOf/6C6ulrrg6anp+Ozzz7DoUOHWlQcERER6VejnQV9fX2xc+dOREREIDAwEEOGDIGHh0ejE/vcu3cPV65cQVRUFG7evAmpVIqJEyfqrXAiIiJquUaDwMKFCzF27Fjs2bMHkZGRiIyMhFgshpOTE2xtbWFpaYnq6mqUlpYiOzsbxcXFAB5OCzx9+nQ8++yzMDExabUXQkRERNrT+Pigm5sb/v73v+OPP/7AiRMncO3aNaSnp+POnTtq7aytrTFo0CDVf8bGWj+VSERERG1A0Ce2g4MD/vSnPwEAKisrUVBQgJKSEkgkEtjY2MDOzk6vRRIREZF+aP3V3dTUFA4ODnBwcNBHPURERNSKOI4AERGRAWMQICIiMmAMAkRERAaMQYCIiMiAMQgQEREZMAYBIiIiA8YgQEREZMC0HkegpqYG165dQ2ZmJuRyOaZMmQLg4UyDFRUV6NChA8Ri5gsiIqL2QKsgcPnyZWzduhVFRUWqdXVBID09HR9++CEWLVqEoUOH6rRIIiIi0g/BX91v3bqFtWvXQiQSYebMmQgKClLb7uHhAZlMhgsXLui8SCIiItIPwUFg//79kEgkCA8Px7hx4xocYtjV1RUZGRk6LZCIiIj0R3AQSElJwYABA2Bra9toG6lUqnbbgIiIiJ5sgoOAXC6HtbW1xjaVlZWora1tcVFERETUOgQHAXt7e9y9e1djm/T0dHTu3LnFRREREVHrEBwE/Pz8cOXKFVy/fr3B7fHx8UhNTYW/v7/OiiMiIiL9Evz44PPPP4/o6Gh88sknGDt2LPLy8gAAly5dQlJSEo4ePQpbW1uMHz9eb8USERGRbgkOAvb29li2bBk2bNiAw4cPq9avWbMGANC5c2csXry4yX4ERERE9OTQakChnj174ssvv8SlS5eQmpqKkpISWFhYwN3dHQMGDICRkZG+6iQiIiI90HqIYbFYjICAAAQEBOijHiIiImpFnBSAiIjIgAm+IhAVFSX4oMHBwc0qhoiIiFqX4CCwZcsWwQdlECAiImofBAeB+fPnN7i+vLwcN2/eRHR0NAYOHMhxBIiIiNoRwUEgJCRE4/YRI0aoJiQiIiKi9kFnnQX79OkDX19f7N27V1eHJCIiIj3T6VMDXbt2RVpami4PSURERHqk0yCQmZmpy8MRERGRnmk9oNDjamtrcf/+fRw/fhzx8fHo16+fLuoiIiKiViA4CEybNq3JNlZWVvjzn//cooKIiIio9QgOAr169YJIJKq3XiQSwdLSEm5ubhgxYgQnHSIiImpHBAeBFStW6LEMIiIiaguca4CIiMiAtbizYEvl5OTg0KFDSE1Nxd27d9GrV68mrz7k5uZi4cKF9dYHBgbi7bff1k+hRERET6FGg4A2cws8SiQSNToccUPu3r2L+Ph4uLu7Q6FQaHWuGTNmwNPTU7XM/glERETaaTQIaDPb4OO0CQL9+/fHgAEDAADr1q1DSUmJ4H27du0KDw8PresjIiKihxoNAps2bWqVAsRidlMgIiJqK40GgU6dOrVmHc2yZcsWlJaWwsbGBkFBQZg+fTokEklbl0VERNRutHlnweYwMTHBmDFj4OvrC3NzcyQmJuLgwYO4d+8elixZ0tblERERtRvNCgK1tbUoLi5GTU1Ng9ulUmmLimqKnZ0dZs+erVr28fGBra0ttm3bhvT0dDg7O9fbJzIyEpGRkQCA8PBwvdfY3hkZGan+5M+KiJ5E/D2lG1oFgTt37mD37t1ITExEdXV1g21EIhH27Nmjk+K0MXjwYGzbtg1paWkNBoGwsDCEhYWplvPz81uxuvan7gkOhULBnxURPZH4e0q4rl27NrpNcBDIzMzE3//+dwBA3759cfHiRfTo0QM2Nja4ffs2SkpK4OPj0+aprKFhkImIiKhhgoPAgQMHoFAosHr1anTv3h3Tpk3DwIEDMWXKFMjlcnz33XeIj4/Hm2++qc96G3X+/HkAQM+ePdvk/ERERO2R4CCQmJgIf39/dO/eXbVOqVQCAMzMzPDGG29g8eLF2Lt3LxYsWCC4gMrKSsTHxwMACgoKUFFRofpQ79evH0xNTbFo0SJ4e3urxifYt28f5HI5PD09YW5ujuTkZBw6dAgDBw5Ejx49BJ+biIjI0AkOAiUlJXBwcFAti8ViVFZWqpaNjIzg4+OD2NhYrQp48OAB1q9fr7aubnnTpk2QyWSora1FbW2tarujoyMOHz6M48ePo6qqClKpFBMnTsTkyZO1OjcREZGhExwErKysIJfLVcvW1tb1OmcYGxujvLxcqwJkMhn27dunsc3mzZvVloOCghAUFKTVeYiIiKg+wcP6de7cGbm5uaplFxcXXL16FQ8ePAAAyOVyxMXFQSaT6b5KIiIi0gvBVwR8fX1x8OBByOVymJmZYfTo0YiPj8eSJUvg6emJtLQ05OXl4ZVXXtFnvURERKRDgoPAyJEj0bVrV1RVVcHMzAz+/v6YOXMmIiIiEBMTA4lEgueeew7PPPOMPuslIiIiHdIYBJYsWYKwsDAMGzYMdnZ2CAwMVNs+btw4jB07FsXFxbCxseEz/ERERO2Mxj4CGRkZ2L59O+bOnYuvv/4aN27cqH8AsRi2trYMAURERO2QxisCq1atQmRkJM6fP4+TJ0/i5MmT6N69O0aOHInhw4fDwsKiteokIiIiPdAYBDw8PODh4YFZs2bhzJkzOHHiBG7fvo3vvvsOu3fvxuDBgzFy5Eh4eXm1Vr1ERESkQ4I6C5qbm2P06NEYPXo00tPTERkZiXPnzuH06dM4ffo0nJycVFcJrKys9F0zERER6YjgcQTqODs7Y86cOfjnP/+JN998E56ensjMzMTOnTsxb948bNy4UR91EhERkR5oHQTqSCQSBAcH4+OPP8aGDRvg5eWF6upqnD17Vpf1ERERkR4JHkegIaWlpYiKisKJEyeQmZkJAOxASERE1I40Kwhcu3YNkZGRiI2NRU1NDQDA3d0dYWFh9cYaICIioieX4CBQVFSEkydP4sSJE6o5BywtLREWFoawsDB069ZNb0USERGRfmgMAkqlEpcuXcLx48cRHx+vmgrYy8sLI0eOxODBgyGRSFqlUCIiItI9jUHgzTffREFBAYCH0xAPHz4cYWFhcHR0bJXiiIiISL80BoGCggJ4e3urvv0bG7eobyERERE9YTR+sn/xxRdwcHBorVqIiIiolWkcR4AhgIiI6OnW7AGFiIiIqP1jECAiIjJgDAJEREQGjEGAiIjIgDEIEBERGTDBQSAmJkY1siARERE9HQSPELR+/XrY2dlhxIgRGDlyJKRSqT7rIiKiZjp66I+2LqFVlJXWqP40lNc8ZqLuH+sXfEVgzJgxqKysxIEDB7Bo0SKEh4fj4sWLUCqVOi+KiIiIWofgKwKvvfYa/vznPyM6OhrHjh1DfHw84uPjYW9vj5EjRyI0NBT29vb6rJWIiIh0TKvJAyQSCUJCQhASEoI7d+4gMjISZ86cQUREBPbv3w9/f3+MGjUKfn5+eiqXiIiIdKnZswh1795d7SrB3r17ERcXh7i4OEilUowZMwajR4+GmZmZLuslIiIiHWrR44NyuRynT5/Gf//7X9V0xc7OzigtLcXu3bvxzjvvID09XRd1EhERkR4064rA7du3cezYMZw7dw5yuRwSiQShoaEYM2YMnJ2dIZfLcfToUezbtw/fffcdVq5cqeu628wf781p6xJaRU1e1f//857BvGaHtdvaugQiolYnOAhUVlbi3LlzOHbsGNLS0gAAjo6OGDVqFIKDg2FhYaFqa2Zmhueeew7379/HiRMndF81ERER6YTgIDB37lxUVFRALBZj0KBBGDNmDHx8fDTuY29vj+rq6hYXSURERPohOAiYm5tj/PjxCAsLg62traB9Ro8ejaCgoObWRkRERHomOAhs3rwZYrF2fQstLCzUbhkQERHRk0XwJ7u2IYCIiIiefII/3ffv34/p06erHhN8XEFBAaZPn46ff/5ZV7URERGRngkOAhcvXoS3t3ejwwjb29ujd+/eiI2N1VlxREREpF+Cg0BOTg6cnJw0tnF0dEROTk6LiyIiIqLWITgIVFVVwdTUVGMbiUQCuVze4qKIiIiodQgOAh07dsSNGzc0trlx4wZnICQiImpHBAcBX19fJCUlITo6usHt586dQ1JSEmceJCIiakcEjyMwadIknD17Fl9++SWio6Ph5+cHe3t7FBQUID4+HnFxcbCyssKkSZP0WC4RERHpkuAgYG9vj2XLlmH9+vWIjY2t93RAp06d8Ne//hUdO3bUeZFERESkH1rNPujq6oovv/wSFy9exI0bN1BWVgZLS0u4u7ujf//+MDZu1mSGRERE1Ea0/uQ2NjbGoEGDMGjQIH3UQ0RERK2I4wYTEREZsEavCERFRQEABg4cCHNzc9WyEMHBwS2vjIiIiPSu0SCwZcsWAIC7uzvMzc1Vy0IwCBAREbUPjQaB+fPnAwDs7OzUlnUtJycHhw4dQmpqKu7evYtevXphxYoVTe5XXl6O77//HrGxsaitrUX//v0xa9YsdOjQQS91EhERPY0aDQIhISEal3Xl7t27iI+Ph7u7OxQKheD9NmzYgOzsbMydOxdisRi7d+/G2rVr8fHHH+ulTiIioqdRmz/v179/fwwYMAAAsG7dOpSUlDS5T2pqKq5cuYIVK1bA29sbwMNxDj744AMkJCSgb9++eq2ZiIjoadHmTw2IxdqXEB8fDxsbG1UIAAA3NzfIZDJcvnxZh9URERE93Rq9IrBw4cJmHVAkEmHjxo3NLkiIrKwsODo61lvv6OiIrKwsvZ6biIjoadJoEFAqlc06YHP300ZZWRksLCzqrbe0tERubm6D+0RGRiIyMhIAEB4eDqlU2qxz/9Gsvag9aO57gujJw99UTyt9/J5qNAhs3rxZ5ydrS2FhYQgLC1Mt5+fnt2E19CTie4KInnTN/T3VtWvXRre1eR+B5rC0tERFRUW99XVzHxAREZEwzQ4CFRUVyM/PR3l5uS7rEaSxvgDZ2dkN9h0gIiKihmn1+KBCocDhw4dx/PhxtXvxMpkMI0eOxIQJE2BkZKTzIh/Xr18/7N+/H9evX4eXlxcA4NatW7h37x78/Pz0fn4iIqKnheAgUFNTg08//RRJSUkQiUSQSqWwtbVFUVER8vLy8K9//QuXL1/G3//+d62mI66srER8fDwAoKCgABUVFTh//jyAhx/4pqamWLRoEby9vVWjG3p4eMDX1xebNm3CjBkzIBKJsHv3bnh5eXEMASIiIi0I/sQ+cuQIkpKS4O/vj1deeQUODg6qbTk5Odi1axcuXryII0eOYNKkSYILePDgAdavX6+2rm5506ZNkMlkqK2tRW1trVqbt99+Gzt37sTWrVuhVCrh7++PWbNmCT4vEbWuNWvWID8/H1KpFEuXLm3rcojo/xMcBM6ePYtu3brhvffeqzcIUJcuXbB48WIsWbIEZ86c0SoIyGQy7Nu3T2Obhp5gsLS0xJtvvok333xT8LmIqO3k5+cjJyenrcsgoscI7iyYk5MDPz+/RkcCFIvF8PPzw71793RWHBEREemX4CsCxsbGkMvlGttUVla2SmdBoqfJqzt/b+sSWoWi+OEjvznFFQbzmr+fOaStSyBqkuArAj169EBMTAyKi4sb3F5cXIzz58/D2dlZV7URERGRngkOAmPGjEFxcTH+9re/4cSJE7h37x6qqqqQm5uLkydPYtmyZSguLsaYMWP0WS8RtVdmHQALm4d/EtETQ/CtgcDAQKSnp+PgwYP45z//2WCbiRMnIjAwUGfFEdHTw8h/YluXQEQN0GpAoZdffhkBAQE4ceIE0tPTUV5eDgsLCzg7OyM0NBQeHh76qpOIiIj0QHAQKCkpgUgkgoeHBz/wiYiInhJNBoHY2Fjs2rVLNaRwly5dMGPGDAQEBOi9OCIiItIvjZ0FU1NTsW7dOrV5BXJycrBu3TqkpqbqvTgiIiLSL41B4MiRI1AqlXjhhRfw7bff4ptvvsHkyZNRW1uLI0eOtFaNREREpCcabw3cuHEDXl5emDp1qmrdtGnTkJSUxCsCRERETwGNVwQePHgAd3f3euvd3d0bHViIiIiI2g+NQUChUMDMzKzeelNTUygUCr0VRURERK1D8MiCRERE9PRp8vHBU6dOITExUW1dXl4eAGDlypX12otEInz00Uc6Ko+IiIj0qckgkJeXp/rgf1xSUpLOCyIiIqLWozEILF++vLXqICIiojagMQh4e3u3Vh1ERETUBthZkIiIyIAxCBARERkwBgEiIiIDxiBARERkwBgEiIiIDBiDABERkQFjECAiIjJgDAJEREQGrNEBhX766admH3TKlCnN3peIiIhaT6NBICIiotkHZRAgIiJqHxoNAg3NM3DkyBHEx8dj2LBh8Pb2hq2tLYqKipCYmIizZ8/C398fzz77rF4LJiIiIt1pNAg8Ps9AVFQUrl69ik8//RQ9e/ZU2xYSEoKxY8di+fLlGDRokH4qJSIiIp0T3Fnwl19+wZAhQ+qFgDqurq4YMmQIfvnlF50VR0RERPolOAhkZ2fDzs5OYxs7OztkZ2e3uCgiIiJqHYKDgLm5OVJSUjS2SUlJgZmZWYuLIiIiotYhOAj4+/sjOTkZu3btQkVFhdq2iooK7Nq1C9evX0f//v11XiQRERHpR6OdBR/38ssvIykpCb/88gtOnDgBZ2dn2NjY4MGDB0hPT0dFRQVkMhmmT5+uz3qJiIhIhwQHARsbG/zjH//Ajz/+iLNnzyI5OVm1TSKRYOTIkZg+fTo6dOigl0KJiIhI9wQHAQDo0KED5s6dizlz5iArKwvl5eWwsLCAo6MjjIyM9FUjERER6YlWQaCOkZERunfvrutaiIiIqJVpHQRqampw7do1ZGZmQi6Xq4YTrqqqQkVFBTp06ACxmHMZERERtQdaBYHLly9j69atKCoqUq2rCwLp6en48MMPsWjRIgwdOlSnRRIREZF+CP7qfuvWLaxduxYikQgzZ85EUFCQ2nYPDw/IZDJcuHBB50USERGRfggOAvv374dEIkF4eDjGjRsHBweHem1cXV2RkZGh0wKJiIhIfwQHgZSUFAwYMAC2traNtpFKpWq3DYiIiOjJJjgIyOVyWFtba2xTWVmJ2traFhdFRERErUNwELC3t8fdu3c1tklPT0fnzp1bXBQRERG1DsFBwM/PD1euXMH169cb3B4fH4/U1FT4+/vrrDgiIiLSL8GPDz7//POIjo7GJ598grFjxyIvLw8AcOnSJSQlJeHo0aOwtbXF+PHj9VYsERER6ZbgIGBvb49ly5Zhw4YNOHz4sGr9mjVrAACdO3fG4sWLm+xHQERERE8OrQYU6tmzJ7788ktcunQJqampKCkpgYWFBdzd3TFgwADON0BERNTOaD3EsFgsRkBAAAICAnRSQGZmJnbs2IHU1FRYWloiNDQUL774osZhinNzc7Fw4cJ66wMDA/H222/rpC4iIiJDIDgIrFy5EiEhIQgODm60zenTp3Hy5EksX75c0DFLS0uxatUqODk5YcmSJcjJycEPP/wApVKJl156qcn9Z8yYAU9PT9Uyb0sQERFpR3AQSEpKgo+Pj8Y2+fn5SEpKEnzyY8eOoaqqCu+++y4sLCzQt29fVFRUICIiAhMnToSFhYXG/bt27QoPDw/B5yMiIiJ1Op0msKqqSqt+ApcvX4avr6/aB35QUBCqqqq0ChRERETUPFr3EWiIUqlEfn4+4uPj0bFjR8H7ZWVl1bvKIJVKYWpqiuzs7Cb337JlC0pLS2FjY4OgoCBMnz4dEolE6/qJiIgMlcYgMG3aNLXliIgIREREaDzg888/L/jkZWVlsLS0rLfe0tISpaWlje5nYmKCMWPGwNfXF+bm5khMTMTBgwdx7949LFmypMF9IiMjERkZCQAIDw+HVCoVXOej/mjWXtQeNPc9QdSYtntP8TfV00of7ymNQaBXr14QiUQAHvYRkEqlkMlk9dqJxWJYWVmhT58+CA0N1XmRj7Ozs8Ps2bNVyz4+PrC1tcW2bduQnp4OZ2fnevuEhYUhLCxMtZyfn6/3Oql94XuCdI3vKdK15r6nunbt2ug2jUFgxYoVqr9PmzYNI0aMwJQpU5pVREMsLS1RXl5eb31ZWRmsrKy0OtbgwYOxbds2pKWlNRgEiIiIqD7BfQQ2bdrU4GX8lnB0dERWVpbauvz8fFRWVmpML5rUXcEgIiKipgl+aqBTp05NPs6nrbqJjCoqKlTroqOjIZFI4O3trdWxzp8/D+Dh6IdEREQkjNZPDRQWFuLq1asoKChATU1Ng22E3j4YNWoUfv31V3z++ed47rnnkJubi4iICIwfP14tdCxatAje3t6YP38+AGDfvn2Qy+Xw9PSEubk5kpOTcejQIQwcOBA9evTQ9iUREREZLK2CwL59+/Dzzz9DoVBobCc0CFhZWeGjjz7C9u3bsWbNGlhaWuLZZ5/F1KlT1drV1taitrZWtezo6IjDhw/j+PHjqKqqglQqxcSJEzF58mRtXg4REbVjVpZ2an9S8wgOAmfOnMH+/fvRu3dvjBkzBuvWrUNwcDB8fX2RmJiIkydPYvDgwRg1apRWBTg5OTU5JPHmzZvVloOCghAUFKTVeYiI6OkycvjrbV3CU0FwEPjtt99gb2+PDz74QDV6oEwmU30oDxw4EOHh4fyAJiIiakcEdxa8c+cO+vXrpzaE8KOX6/38/ODr64vDhw/rtkIiIiLSG8FBQKFQoEOHDqpliURSbwyAbt26IT09XWfFERERkX4JDgJ2dnYoLCxULUulUmRkZKi1KSws1GrSISIiImpbgoOAs7Mz7t69q1r28fHB9evXcfr0acjlcly6dAnnz5+Hi4uLXgql1mVnBEiNRLBjriMieqoJ7izYv39/bNu2Dbm5uZDJZJg0aRJ+//13bN68WdWr39jYuN5ERdQ+vW7PWRyJiAyB4CAQEhKCkJAQ1bJUKsXq1atx+PBh3Lt3D506dcKYMWPQvXt3fdRJREREeqD1yIKPkslkarMAEhERUfsiuI8AERERPX20viJQW1uLgoICjXMNaDthEBEREbUNrYLAoUOHcPjwYRQXF2tst3fv3hYVRURERK1DcBDYt28f9u/fDysrKwQHB8Pe3p5jBhAREbVzgoPAyZMnIZPJsGbNGrUpgomIiKj9EtxZsKSkBAEBAQwBRERETxHBQaBLly4oKyvTZy1ERETUygQHgdGjR+PixYsoKirSYzlERETUmgT3ERg9ejT++OMPfPjhh3jhhRfQs2fPRm8TSKVSnRVIRERE+qPV44M9evTAqVOnsHXr1kbbiEQi7Nmzp8WFERERkf4JDgLHjx/HN998AyMjI/j4+MDOzo6PDxIREbVzgoPA4cOHYWNjg08++QQymUyfNREREVErEdxZMC8vD4MHD2YIICIieooIDgL29vaNzi1ARERE7ZPgIBAcHIz4+HhUVFTosx4iIiJqRYKDwPPPPw83NzesWrUKiYmJDARERERPAcGdBV9++WXV3z/++ONG2/HxQSIiovZDcBDo1asXRCKRPmshIiKiViY4CKxYsUKPZRAREVFbENxHgIiIiJ4+DAJEREQGrNFbAz/99BMAYOzYsbCyslItCzFlypSWV0ZERER612gQiIiIAAAEBgbCyspKtSwEgwAREVH70GgQWL58OYD/TSlct0xERERPj0aDgLe3t8ZlIiIiav8EdxaMiopCRkaGxjZ37txBVFRUi4siIiKi1iE4CGzZsgWxsbEa28TFxWHLli0tLoqIiIhah04fH6ytreXog0RERO2IToNAdnY2LC0tdXlIIiIi0iONQww/fpk/NjYWubm59drV1tbi/v37SE5Ohr+/v24rJCIiIr3RGAQe7/iXnp6O9PT0Rtu7u7tj5syZOimMiIiI9E9jENi0aRMAQKlUYtGiRRg3bhzGjRtXr51YLIalpSXMzMz0UyURERHphcYg0KlTJ9Xfp0yZAh8fH7V1RERE1L4Jnob4xRdf1GcdRERE1AYEB4Hbt28jNTUVw4YNg4WFBQBALpdj27ZtiIuLg6mpKZ577rkGbx0QERHRk0nw44MHDx7EgQMHVCEAAH788UecOXMGSqUSJSUl2LlzJ65cuaKXQomIiEj3BAeBW7duwcfHR7VcU1ODqKgouLm54dtvv8WmTZtgbW2NX3/9VS+FEhERke4JDgLFxcXo2LGjajktLQ1yuRxhYWGQSCSwt7dHQEBAk/MREBER0ZNDq5EFFQqF6u/Xr18HoD4robW1NYqLi3VUGhEREemb4CAglUpx48YN1XJsbCw6duyIzp07q9YVFhbCyspKtxUSERGR3gh+amDIkCGIiIjAunXrYGJigtTUVDz77LNqbbKystSCARERET3ZBAeB8ePH48qVK7hw4QIAwNnZGVOmTFFtz83Nxc2bN/H8889rVUBmZiZ27NiB1NRUWFpaIjQ0FC+++CLEYs0XK8rLy/H9998jNjYWtbW16N+/P2bNmoUOHTpodX4iIiJDJjgImJmZYdWqVbhz5w4AwMnJqd6H9eLFi+Hq6ir45KWlpVi1ahWcnJywZMkS5OTk4IcffoBSqcRLL72kcd8NGzYgOzsbc+fOhVgsxu7du7F27Vp8/PHHgs9PRERk6AQHgTrdu3dvcL1MJoNMJtPqWMeOHUNVVRXeffddWFhYoG/fvqioqEBERAQmTpyoNmbBo1JTU3HlyhWsWLFC1VnR3t4eH3zwARISEtC3b1/tXhQREZGB0nj9PSkpCfn5+YIPlpGRUW/GQk0uX74MX19ftQ/8oKAgVFVVISkpqdH94uPjYWNjo/bEgpubG2QyGS5fviz4/ERERIZOYxBYuXIlTp06pbbu559/xmuvvdZg+wsXLmDLli2CT56VlYWuXbuqrZNKpTA1NUV2drbG/RwdHeutd3R0RFZWluDzExERGTqtbw1UV1ejrKxMJycvKyuDpaVlvfWWlpYoLS3VuF9Dtw0sLS2Rm5vb4D6RkZGIjIwEAISHh9cLIEJ13f2fZu1H1Jjf/vZCW5dAT5lZ85r3+40Mk1YDCrVnYWFhCA8PR3h4eFuX0m68//77bV0CPWX4niJd43uq5do0CFhaWqK8vLze+rKyMo0DE1laWqKioqLB/Rq6wkBEREQNa9Mg0NA9/fz8fFRWVmq8dN9YX4Ds7OwG+w4QERFRw9o0CPj5+eHKlStq3+6jo6MhkUjUngh4XL9+/VBUVKSa7wB4ODvivXv34Ofnp8+SDUpYWFhbl0BPGb6nSNf4nmq5Ng0Co0aNgomJCT7//HMkJCQgMjISERERGD9+vFpnwEWLFmHr1q2qZQ8PD/j6+mLTpk2IiYnBhQsX8NVXX8HLy4tjCOgQ/4GRrvE9RbrG91TLiZRKpbKxjdOmTWvWQffu3Su4bWZmJrZv3642xPDUqVPVRi1csGABvL29sWDBAtW6srIy7Ny5ExcuXIBSqYS/vz9mzZoFa2vrZtVMRERkiNo8CBAREVHb0RgEiIiI6Omm9YBC1H7k5uZi4cKFGDRoEN59910AwObNm9WGgRaJRDAzM0P37t0REhKC0NBQiEQiJCYmYuXKlQgMDMTbb79d79jLli3DjRs3MHbs2AZHmnzrrbeQm5uLHTt2NDpnBLVvde+vR5mamsLKygrdunVD7969ERISUu923b59+/DTTz8BAF5++WVMmjSpwePXvccA4PPPP290nhN6uujiffXofg4ODhg8eDDGjx8PiUSi9/rbIwYBAzV69GhYW1ujtrYWeXl5iImJQUpKCm7fvo05c+bA3d0dJiYmSE5OrrevXC5HWloaRCJRg9sLCgqQk5ODnj17MgQYAEdHRwwZMgQAUFVVhcLCQly/fh2XL1/G/v37MWfOHAwbNqzefkZGRoiKimowCGRmZuLGjRswMjKCQqHQ90ugJ1Bz31dBQUFwcHAAABQWFiI2NhZ79uxBYmIiPvzww1Z9De0Fg4CBGj16tNo3rEmTJuFvf/sbjh07hgkTJqBz585wc3NDcnIycnJy0KVLF1Xb1NRUKBQKDBgwAHFxcSgtLVUbAKpuwihNj4DS08PJyQlTp05VW6dUKnH27Fl8++232LRpEywtLeHv76/WxtfXF5cuXcLNmzfh5uamtu3UqVMwMjJCnz59OJGYgWru+2ro0KHo37+/avlPf/oTFi9ejKtXr+LatWvo3bt3q9TfnhjMEMOkWbdu3eDj4wOlUom0tDQAgI+PDwDUmwkyKSkJJiYmmDhxIpRKZb2rAnXt6/YnwyMSiTBs2DC8/vrrUCqV+OGHH/B4d6TAwECYmJjUm9istrYWZ86cga+vL2xsbFqxanrSCXlfPc7KygoBAQEAoPrdRuoYBKgekUgE4H/f6BsKAm5ubnB3d4e5uXm97cnJyRCJROjVq1frFExPrKFDh0ImkyErKwsZGRlq2ywtLREQEIDo6GjU1NSo1l+5cgWFhYUICQlp5WqpvdD0vtLEyMhIj1W1XwwCBODhPdmkpCSIRCL07NkTwMOBm0xMTNQ+6KuqqnDz5k306tULYrEYnp6eatuLioqQlZUFZ2dn9g8giEQieHl5AWj421hISAhKS0sRFxenWnfq1Cm1b3FEj2vqffWoR99fHh4eeq+tPWIfAQP122+/wdraGkqlUtVZsLKyEmPHjoVMJgMASCQSVT+B3NxcyGQypKamoqamRvVt38vLC3v37kV5eTksLCxUtwl4W4Dq2NnZAQBKSkrqbfP19YWdnR2ioqIwePBglJWVIS4uDqGhoTA25q8nalxj76uzZ8/i1q1bAP7XWbC4uBijRo2Cu7t7q9fZHvBfmoH67bffAPzv8UFnZ2eMGDECI0aMUGvn7e2N5ORkJCUlQSaTISkpCUZGRvD09FRtVyqVuH79Ovz9/dlRkLQiFosxbNgw/PLLL3jw4AFiYmJQXV3N2wLUbOfOnau3LiwsDK+//nobVNM+MAgYKKHPZXt7e2P//v1ISkpCSEgIkpOT4eLiAjMzMwCAq6ur6vZBXRBg/wB6VGFhIQA0Ovx3SEgIDh06hDNnziA6OhrdunWDq6tra5ZI7VBj76ulS5eif//+qKmpwd27d7Fjxw5ERkaiR48eGDNmTFuU+sRjHwHSyNPTE8bGxkhOTkZ1dTVSU1PVPuRNTEzg5uaGpKQklJSUIDMzE87OzrC0tGzDqulJUXe1CICq78njnJyc4OrqisOHD+PmzZsIDg5uzRKpHRLyvjI2NoaLiwvef/992NjYYNeuXbh//35rltluMAiQRnX9BO7du6e6bPv4t/1evXrh9u3biI+Ph1Kp5G0BUjl37hxyc3Ph6Oio8QpUSEgICgsLIRaLMXz48FaskNojoe8r4OHTKS+++CKqq6uxf//+VqqwfWEQoCbVfbD/+9//VuutW6dXr15QKBQ4ePCgWnsyXHUDv3zzzTcQiUR45ZVXVI+lNmT48OFYvHgxli1bBltb29YrlNoVbd9XdUJDQ9GxY0ecPHkS+fn5rVBp+8I+AtQkb29vHDhwAHfv3kWPHj3URhEEHt4+EIvFuHv3LvsHGKDMzEzs27cPAFBdXY3CwkIkJycjLy8P5ubmWLhwIfr166fxGObm5hg4cGBrlEvthC7eV3WMjY0xadIkbN++HQcOHMAbb7yhz9LbHQYBalJdP4FHHxt8lJmZGVxcXHDr1q0GgwI93bKyslSTvTw6OcyYMWManByGSAhdv69CQ0Px73//G6dOncLkyZMhlUr1UXa7xGmIiYiIDBj7CBARERkwBgEiIiIDxiBARERkwBgEiIiIDBiDABERkQFjECAiIjJgDAJEREQGjEGAiNpEYmIipk6diqlTp7Z1KUQGjSMLksGrqqpCVFQULl68iIyMDBQXF8PY2Bj29vbw8vJCUFAQevfurfEYCxYsQF5eXr31ZmZm6NSpE3r16oWxY8fCycmpXpsVK1YgKSlJUK3e3t5YsWKFoLZN1daQ4OBgLFiwQKvjP66srAy//PILAODZZ599KmeiPHXqFHJzc+Hj4wMfH5+2LoeoRRgEyKAlJCRg69atatOTmpubo6amBllZWcjKysLx48fRr18/LFy4EB06dNB4PBMTE1hYWAB4OEFKSUkJ7t69i7t37+L48eN4/fXXERoa2uC+RkZGTQ7P3JLhmx+trTFNbReirKxMNTRsSEhIo0HA1NQUXbt2bfH52sKpU6dU4Y1BgNo7BgEyWNHR0di4cSMUCgXs7e0xdepUDBw4UPVhm5WVhWPHjuHo0aOIj4/HsmXLsGrVKtjY2DR6zMDAQLVv1FVVVbh48SJ27NiBBw8e4JtvvoGrqyt69OhRb19PT0+tv+1r4/Ha2pqbmxu++OKLti6DyOCxjwAZpMzMTGzduhUKhQLdu3fHZ599htDQULVv3I6Ojnj11Vfx3nvvwdjYGDk5Ofjqq6+0Oo9EIsGQIUOwaNEiAEBtbS1+++03nb4WIqKW4BUBMkh79uxBZWUlTExM8Ne//lXjTGb+/v6YPHky9u3bh6tXr+LSpUvw9/fX6nx9+/aFnZ0dCgsLcevWrZaW36ru37+Pw4cPIyEhAXl5eVAoFOjQoQNsbW3Rq1cvDB06FG5ubgDq93dYuHCh2rEe7eOQmJiIlStXAoBqutk6p06dwpYtW9CpUyds3rwZycnJOHjwIG7evInKyko4ODhg7NixardZLl26hF9++QXp6emorKxEt27dMGHCBAQGBjb4unJzcxEdHY3ExETk5uaioKAAACCVSuHr64vx48fXm6Gurq46P/30k+o2SJ1NmzZBJpOplmtra3Hq1CmcOXMGd+7cQUVFBTp06ABPT0+MGTOm0VsLdT/LKVOmYPLkyfj1119x7tw55OTkoLy8HMuXL1ftm5WVhSNHjiApKQn379+HUqmEtbU17O3t4ePjg+DgYDg6OjZ4HiIGATI4hYWFiI2NBQAEBQUJuk89fvx4HD58GBUVFTh69KjWQQAA7O3tUVhYiIqKCq33bSvp6elYuXIlysrKAABisRjm5uYoKipCYWEhbt++jbKyMlUQsLKyQocOHVBSUgIA6NChA8Ti/114bE4fh+PHj+Obb74B8LD/RmVlJdLT0/H1118jJycHL7/8Mvbt24effvoJIpEI5ubmqKqqwq1bt/DFF1+gtLQUo0ePrnfcLVu2qEKLsbExzM3NUVpaquobcurUKbz//vvw8vJS7SORSGBjY4PS0lIoFAqYmprCzMxM7biPvt7y8nKsXbsWiYmJ9X5+58+fx/nz5zFhwgTMmDGj0ddfXV2NlStXIiUlBUZGRjAzM4NIJFJtT0hIwJo1a1BdXQ0Aqjb379/H/fv3cePGDRgbG/PpDGoUgwAZnMTERNTNvj1o0CBB+5iZmaFv376IiYlBcnIyFAoFjIyMtDpvXc/9lnT4a20//PADysrK4OLigtmzZ8Pd3R0ikQg1NTXIy8tDXFwcHp3JfPHixcjNzVVdCVi9erXat2NtFRcXY/v27Rg7dixeeOEFWFtbo7S0FDt37kRUVBQOHjwIS0tLHDhwAC+99BLGjh0LCwsLFBYWYuvWrbh8+TJ++OEHDB06tF5HSGdnZwwZMgR9+/ZF586dIRaLoVAocPv2bezbtw+XL1/Ghg0bsHHjRkgkEgAP+1kEBgaqvq1PmDBB4wfs1q1bkZiYCGNjY8yYMQOhoaEwNTVFUVER/vWvf+HkyZM4fPgwOnfu3GBYAYCjR48CAN58800EBgZCIpGgpKREFQa+/fZbVFdXw9fXFzNmzED37t0BPOyfcu/ePcTExNS7skH0KAYBMjiZmZmqv7u4uAjez9nZGTExMZDL5cjLy0OXLl0E73v+/HkUFxcDANzd3Rtsk5KSgtdff13jcWbNmtXope6mREdH4/LlyxrbLF68GJ6enmo1AcDs2bPh4eGhWm9sbAwHBwdMmDChWbUIVVlZidDQUMyaNUu1zsrKCvPnz0dycjJyc3Oxe/duvPTSS5g8ebKqjZ2dHd5++23MnTsXlZWViIuLw/Dhw9WO/eqrr9Y7n5GREdzc3PD+++9j6dKlyMjIwPnz5+vtK8SNGzcQExMDAHjttdcQFham2mZra4v58+ejvLwcMTEx2Lt3L0JCQlSB41FyuRxLlixBQECAal3d0ysPHjzAvXv3ADwMCnZ2dqo2EokE3bp1Q7du3bSunQwLOwuSwam7bA1o9+380UcHS0tLm2yvVCqRl5eHX3/9FVu3bgXw8AN0zJgxDbZXKBR48OCBxv+qqqoE1/u46urqJo9fU1Ojtk/do3+FhYXNPm9LTZo0qd46sVisGtvBxMQE48aNq9fGwsJCFV7u3Lmj1TnFYjF8fX0BANevX9ey4oeio6MBAB07dmz0kdFp06YBePieTEhIaLBNt27d1ELAo8zNzVVXBtry/xG1b7wiQKRDUVFRiIqKanCbmZkZFixYAAcHhwa3N2ewIG00Z7Agf39/HD9+HJs3b0ZKSgoCAgLg6uoKU1NTPVWpzsrKqtErL7a2tgAAJyenevfp69Q96tlYcEtOTsaJEydw48YN3L9/H5WVlfXa1HUi1FZaWhqAh+MMPNpv4FFOTk6wt7dHQUEB0tLSGvzAf/QKzeMkEgn69OmDhIQE/OMf/8CoUaPg7+8PFxcXGBvz1zsJw3cKGZzHv9nb29sL2k/IlYRHB+0RiUQwNTWFVCpFr169MHLkSHTs2LEFlbe+P//5z8jJyUFiYiKOHDmCI0eOQCwWw9nZGf7+/ggLCxP882sOc3PzRrfVfbhqalPXj0OhUNTb9n//9384dOiQ2vEsLS1VH6ByuRyVlZUNhgMhHjx4AABN/nw6duyIgoICVfvHaXqiBQDmzZuHNWvWICMjA/v378f+/fthbGwMV1dXDBgwoN5jsUSPYxAgg/PoML9paWmCP8hu374N4H/DBjfkSRu0p6UsLS2xfPlyXL9+HXFxcUhJSUFaWprqv0OHDmHevHkYOnRoW5eqlYSEBFUIGD16NEaPHg0nJye1b+579uzBgQMH1DpDtoXGribUkUqlWLNmDRISEhAfH4+UlBRkZGQgJSUFKSkp+Pe//4133323yWGyyXAxCJDB8fHxgUgkglKpRExMTKP3Xx8ll8tx9epVAECvXr20fmKgvfPy8lI9RldVVYWEhATs2bMHd+7cwdatW9G7d2/Vpfr24Ny5cwAAX19fzJkzp8E2RUVFLTqHjY0NsrOz1Yavbkjddk0jVjZFLBbDz88Pfn5+AICKigpcvHgRP/74I/Lz8/Hll19i69atvF1ADWJnQTI4dnZ2GDBgAICHHbqys7Ob3OfIkSOq5/8be8zLUEgkEgQEBGDx4sUAHnZCfLRDXVPfYJ8EdR++jT01olQqVc/+N+TR5/gb07NnTwAPH1etra1tsE1WVpaqD4Krq2uTxxTK3NwcQ4cOxbx58wA8vE2hbYdJMhxP/r9YIj2YNm0aJBIJqqursX79etWjfQ2Jj4/HgQMHADy8mtCcwYTaI4VC0egHGAC1R90e/fB/9J593UBET5q6fhwZGRkNbj927JjqsbyG1L1GTa8vKCgIwMPOhidOnGiwzd69ewE87LfSp0+fpgt/zONPeTzu0f9HQsILGSYGATJI3bp1w7x58yAWi3Hnzh0sXboUJ06cUPvFnp2djZ07d+Kzzz5DTU0NOnfujL/85S8G8wv1/v37+Mtf/oL9+/fj9u3bah3uMjIysHHjRgAPZxH09vZWbbO0tFT1uzh58mSDHfXaWt0l9Pj4ePz000+Qy+UAHn6wHzhwADt27NA402TdoD3x8fGNPlXg5uamGrBqx44d+O9//6vqeFhUVISvv/4a58+fB/C/YKqtlJQULF68GEeOHEFmZqYquCmVSqSkpGDbtm0AHnZIbGiiKyKAfQTIgA0dOhRWVlaqaYi//vprfP3117CwsEB1dbVqyFbg4b3kRYsWNdmDuyWEDCgEPBxJrjmEDCgklUqxevVq1fK9e/ewd+9e7N27F2KxGBYWFpDL5apvosbGxliwYEG9XumjRo3C3r178d///hfHjx+HtbU1xGIx3N3d8fbbbzerfl0aPnw4oqKikJycjH379iEiIgIWFhYoLy+HUqmEv78/nJ2dVVeCHhccHIzDhw8jJycH8+fPh7W1teqD/OOPP1Y9HTJ//nyUlJQgKSkJO3bswM6dO2FmZqY6DwBMmDChRbeb7ty5g127dmHXrl0wMjJSvY66AGZubo633nqrXdyyobbBIEAGzc/PDxs3bsSpU6dw8eJFZGRkoKSkBMbGxqrH/oKCgpp12VZbdQMK6UvdgEKaPPqt1N7eHkuWLEFiYiJSU1NVj7gZGRmhS5cu8PHxwbhx4xocF+H555+Hubk5zpw5o7oPrlQqG33aorUZGxtj2bJl+Pnnn3Hu3DnV8M9ubm4IDg5GWFhYvcmEHuXg4IDly5fj559/xo0bN1RzDwDqjypaWFjgo48+Uk06lJ6eDrlcDltbW3h4eGDs2LGNTjokhKurK9555x0kJibi5s2bKCwsRHFxMUxMTNCtWzf07dsX48aN0+sjntT+iZRt/WwMERERtRleKyIiIjJgDAJEREQGjEGAiIjIgDEIEBERGTAGASIiIgPGIEBERGTAGASIiIgMGIMAERGRAWMQICIiMmAMAkRERAbs/wFKVNNqqbJAkAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# by activating the `is_relative` option\n",
    "# we can visualize the estimated policy value of the evaluation policy\n",
    "# relative to the ground-truth policy value of the behavior policy\n",
    "ope.visualize_off_policy_estimates(\n",
    "    action_dist=action_dist,\n",
    "    estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
    "    n_bootstrap_samples=10000, # number of resampling performed in bootstrap sampling\n",
    "    is_relative=True,\n",
    "    random_state=12345,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the OPE demonstration here is with the small size example version of our dataset. \n",
    "\n",
    "Please use its full size version (https://research.zozo.com/data.html) to produce more reasonable results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (4) Evaluation of OPE\n",
    "\n",
    "Our final step is the **evaluation of OPE**, which evaluates the estimation accuracy of OPE estimators.\n",
    "\n",
    "Specifically, we asses the accuracy of DM, IPW, and DR by comparing their estimations with the ground-truth policy value estimated via the on-policy estimation from Open Bandit Dataset.\n",
    "\n",
    "This type of evaluation of OPE is possible, because Open Bandit Dataset contains a set of *multiple* different logged bandit datasets collected by running different policies on the same platform at the same time.\n",
    "\n",
    "Please refer to [the documentation](https://zr-obp.readthedocs.io/en/latest/evaluation_ope.html) for the details about the evaluation of OPE protocol."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (4-1) Approximate the Ground-truth Policy Value \n",
    "With Open Bandit Dataset, we can estimate the ground-truth policy value of the evaluation policy in an on-policy manner as follows.\n",
    "\n",
    "$V(\\pi_e) \\approx \\frac{1}{|\\mathcal{D}_{e}|} \\sum_{i=1}^{|\\mathcal{D}_{e}|} \\mathbb{E}_{n} [r_i]$\n",
    "\n",
    "$ \\mathcal{D}_e := \\{(x_i,a_i,r_i)\\} $ ($(x,a,r) \\sim p(x)\\pi_e(a | x)p(r | x,a) $) is the log data collected by the evaluation policy (, which is used only for approximating the ground-truth policy value).\n",
    "\n",
    "We can compare the policy values estimated by OPE estimators with this on-policy estimate to evaluate the accuracy of OPE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:obp.dataset.real:When `data_path` is not given, this class downloads the small-sized version of Open Bandit Dataset.\n"
     ]
    }
   ],
   "source": [
    "# we first calculate the ground-truth policy value of the evaluation policy\n",
    "# , which is estimated by averaging the factual (observed) rewards contained in the dataset (on-policy estimation)\n",
    "policy_value_bts = OpenBanditDataset.calc_on_policy_policy_value_estimate(\n",
    "    behavior_policy='bts', campaign='all'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (4-2) Evaluation of OPE\n",
    "\n",
    "We can evaluate the estimation performance of OPE estimators by comparing the estimated policy values of the evaluation with its ground-truth as follows.\n",
    "\n",
    "- $\\textit{relative-ee} (\\hat{V}; \\mathcal{D}_b) := \\left| \\frac{V(\\pi_e) - \\hat{V} (\\pi_e; \\mathcal{D}_b)}{V(\\pi_e)} \\right|$ (relative estimation error; relative-ee)\n",
    "- $\\textit{SE} (\\hat{V}; \\mathcal{D}_b) := \\left( V(\\pi_e) - \\hat{V} (\\pi_e; \\mathcal{D}_b) \\right)^2$ (squared error; se)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>relative-ee</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ipw</th>\n",
       "      <td>0.084019</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dm</th>\n",
       "      <td>0.156041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dr</th>\n",
       "      <td>0.125548</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     relative-ee\n",
       "ipw     0.084019\n",
       "dm      0.156041\n",
       "dr      0.125548"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# evaluate the estimation performance of the OPE estimators \n",
    "# `evaluate_performance_of_estimators` returns a dictionary containing estimation performance of given estimators \n",
    "relative_ee = ope.summarize_estimators_comparison(\n",
    "    ground_truth_policy_value=policy_value_bts,\n",
    "    action_dist=action_dist,\n",
    "    estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
    "    metric=\"relative-ee\", # \"relative-ee\" (relative estimation error) or \"se\" (squared error)\n",
    ")\n",
    "\n",
    "# estimation performances of the three estimators (lower means accurate)\n",
    "relative_ee"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can iterate the above process several times to get more reliable results.\n",
    "\n",
    "Please see [examples/obd](../obd) for a more sophisticated example of the evaluation of OPE with the Open Bandit Dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "2ff39f3b22306140fd87fd114528320b56c4f8c8e196b421a3ea939a2b6b4692"
  },
  "kernelspec": {
   "display_name": "Python 3.9.5 64-bit ('3.9.5': pyenv)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  },
  "orig_nbformat": 2
 },
 "nbformat": 4,
 "nbformat_minor": 2
}