{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "04e33286",
   "metadata": {},
   "source": [
    "# ProbabilityV0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0c83dca",
   "metadata": {},
   "source": [
    "![image](../images/probabilityv0.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "058cb0f2",
   "metadata": {},
   "source": [
    "## Description\n",
    "\n",
    "Reinforcement learning environment for training parameterized quantum circuits \n",
    "to approximate a target probability distribution over computational basis states.\n",
    "It is based on the `QuantumEnv` base class.\n",
    "\n",
    "The goal of `ProbabilityV0` is to optimize variational quantum circuits such that \n",
    "the measured probability distribution of outcomes matches a given target \n",
    "distribution. This is useful in tasks like quantum compilation, quantum generative \n",
    "modeling, and distribution learning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35ba685c",
   "metadata": {},
   "source": [
    "## Environment Dynamics\n",
    "- **State (observation):**\n",
    "    The current probability distribution over `2**n_qubits` basis states obtained \n",
    "    from the quantum circuit.\n",
    "\n",
    "- **Action:**\n",
    "    A continuous vector of parameter updates (`Box(low=-0.1, high=0.1, shape=(n_params,)`), \n",
    "    which perturbs the trainable parameters of the ansatz.\n",
    "\n",
    "- **Reward:**\n",
    "    Defined as the *negative* of a weighted cost combining:\n",
    "    - KL divergence between the target distribution and the circuit's output.\n",
    "    - L2 distance between the target and circuit distributions.\n",
    "    The weighting is controlled by `alpha` (for KL vs. L2) and `beta` (step penalty).\n",
    "\n",
    "- **Episode Termination:**\n",
    "    - If the reward is below the specified tolerance.\n",
    "    - If the number of steps reaches `max_steps`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd2c4c5a",
   "metadata": {},
   "source": [
    "\n",
    "**Key Parameters**\n",
    "-----------------\n",
    "- `n_qubits (int)`: Number of qubits in the circuit.\n",
    "- `target_distribution (np.ndarray)`: Target probability distribution (must sum to 1).\n",
    "- `ansatz (callable, optional)`: Custom circuit ansatz. Defaults to a simple \n",
    "    layer of RY rotations if not provided.\n",
    "- `max_steps (int, default=100)`: Maximum steps per episode.\n",
    "- `tolerance (float, default=-1e3)`: Reward threshold for termination.\n",
    "- `alpha (float, default=0.5)`: Weight between KL divergence and L2 error.\n",
    "- `beta (float, default=0.01)`: Penalty weight for step count.\n",
    "- `ffmpeg (bool, default=False)`: If `True`, uses FFmpeg for saving animations; otherwise uses Pillow (GIF)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a7dd570",
   "metadata": {},
   "source": [
    "**Visualization**\n",
    "----------------\n",
    "- `render()`: Creates an animation showing the evolution of the learned \n",
    "    distribution and corresponding rewards across training.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea2b3ed0",
   "metadata": {},
   "source": [
    "**Applications**\n",
    "----------------\n",
    "- Distribution learning with quantum circuits.\n",
    "- Testing expressivity of variational ansätze.\n",
    "- Benchmarking optimization of quantum neural networks.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "11368057",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step 0: Reward = -1.3208\n",
      "Step 1: Reward = -0.2667\n",
      "Step 2: Reward = -0.1845\n",
      "Step 3: Reward = -0.1294\n",
      "Step 4: Reward = -0.0906\n",
      "Step 5: Reward = -0.0626\n",
      "Step 6: Reward = -0.0421\n",
      "Step 7: Reward = -0.0270\n",
      "Step 8: Reward = -0.0158\n",
      "Step 9: Reward = -0.0076\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAHXCAYAAACLVgojAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIEklEQVR4nO3deVhV1f7H8c8BmRFQVEBFcR4K5yHMMSnU0swcKgs1s1sOqVxzSsXhmt0GpwYtNeuWlWbqLTXNUJKSckisTE1NwquCOIGiAsL+/dHPczsXdAMih+H9ep7zPJ6119r7ew77QT/utde2GIZhCAAAAABwQw72LgAAAAAAijuCEwAAAACYIDgBAAAAgAmCEwAAAACYIDgBAAAAgAmCEwAAAACYIDgBAAAAgAmCEwAAAACYIDgBAAAAgAmCEwCgUMXHx8tisSg6OtrepRQL7733niwWi+Lj4+1dCgDgFhCcAKCI/Pzzz+rbt69q1qwpV1dXVatWTffee69ef/11m34vvvii1q1bZ58i/9+ePXvUrVs3eXl5qXz58rrvvvsUFxdX6Mfp3LmzLBaL9eXm5qYmTZpo/vz5ys7OLvTjlTY7duxQ+/bt5e7uLn9/fz333HO6dOlSnscvW7ZMjRo1kqurq+rVq5fjXLzuxIkT6t+/v3x8fOTl5aUHH3xQv//+u02f6wHxRq8VK1bk2O/KlSsVEhIiDw8P+fj4qF27dtq6dWv+vgQAKCLl7F0AAJQFO3bsUJcuXVSjRg0NGzZM/v7+On78uL7//nstWLBAo0aNsvZ98cUX1bdvX/Xu3dsutf74449q3769AgMDFRkZqezsbL311lvq1KmTdu7cqQYNGhTq8apXr645c+ZIks6cOaOPPvpIY8eOVXJysmbPnl2oxypN4uLi1LVrVzVq1Ehz587Vf/7zH7366qs6fPiwvvzyS9Pxb7/9tp555hk9/PDDioiIUExMjJ577jldvnxZEyZMsPa7dOmSunTpopSUFE2ePFlOTk6aN2+eOnXqpLi4OPn6+kqSOnbsqA8++CDHcebNm6d9+/apa9euNu3Tp0/XzJkz1bdvXw0ePFiZmZn65ZdfdOLEiVv8ZgDgNjEAALddjx49jMqVKxvnz5/PsS0pKcnmvYeHhzFo0KCiKSwXPXr0MCpUqGCcOXPG2nby5EnD09PT6NOnj+n4Y8eOGZKMbdu2mfbt1KmTcccdd9i0XblyxahZs6ZRvnx549q1a/muv6hlZWUZV65cueH25cuXG5KMY8eOFepxu3fvbgQEBBgpKSnWtiVLlhiSjM2bN9907OXLlw1fX1/j/vvvt2kfOHCg4eHhYZw7d87a9s9//tOQZOzcudPaduDAAcPR0dGYNGmS6XHKly9v3HvvvTbtsbGxhsViMebOnWv6OQGguGCqHgAUgaNHj+qOO+6Qj49Pjm1VqlSx/tlisSgtLU3vv/++dYrT4MGDrdtPnDihJ598Un5+fnJxcdEdd9yhd99912Z/0dHRslgsWrlypSZPnix/f395eHioV69eOn78uGmtMTExCg0NtV5JkKSAgAB16tRJ69evz9dUsIJwdXVV69atdfHiRZ0+fdpm24cffqiWLVvKzc1NFStW1COPPGLzmRYuXChHR0dduHDB2vbaa6/JYrEoIiLC2paVlaXy5cvbXFl59dVX1a5dO/n6+srNzU0tW7bU6tWrc9RnsVg0cuRIrVixQnfccYdcXFy0adMmSdL+/ft1zz33yM3NTdWrV9c//vGPXKccpqSk6ODBg0pJSSnQd5SamqotW7bo8ccfl5eXl7U9PDxcnp6eWrVq1U3Hb9u2TWfPntXw4cNt2keMGKG0tDRt2LDB2rZ69Wq1bt1arVu3trY1bNhQXbt2NT3OF198oYsXL2rgwIE27fPnz5e/v79Gjx4twzBu+zkFAIWB4AQARaBmzZras2ePfvnll5v2++CDD+Ti4qIOHTrogw8+0AcffKC//e1vkqSkpCTddddd+vrrrzVy5EgtWLBAdevW1dChQzV//vwc+5o9e7Y2bNigCRMm6LnnntOWLVsUGhqqK1eu3LSG9PR0ubm55Wh3d3dXRkaG6WcoDNcXmPhr0Jw9e7bCw8NVr149zZ07V2PGjFFUVJQ6duxoDUodOnRQdna2vv32W+u4mJgYOTg4KCYmxtq2d+9eXbp0SR07drS2LViwQM2bN9fMmTP14osvqly5curXr59NiLhu69atGjt2rAYMGKAFCxYoKChIiYmJ6tKli+Li4jRx4kSNGTNG//rXv7RgwYIc49euXatGjRpp7dq1Bfp+fv75Z127dk2tWrWyaXd2dlazZs20d+/em46/vv1/x7ds2VIODg7W7dnZ2frpp59y9JOkNm3a6OjRo7p48eINj7NixQq5ubmpT58+Nu1RUVFq3bq1Fi5cqMqVK6t8+fIKCAjQG2+8cdO6AcCu7H3JCwDKgq+++spwdHQ0HB0djZCQEGP8+PHG5s2bjYyMjBx9bzRVb+jQoUZAQIDNFDrDMIxHHnnE8Pb2Ni5fvmwYhmFs27bNkGRUq1bNSE1NtfZbtWqVIclYsGDBTWsNDg426tevbzNNLj093ahRo4YhyVi9evVNx+d3ql7Dhg2N5ORkIzk52Th48KDx/PPPG5JsppHFx8cbjo6OxuzZs23G//zzz0a5cuWs7VlZWYaXl5cxfvx4wzAMIzs72/D19TX69etnODo6GhcvXjQMwzDmzp1rODg42EydvP79XZeRkWHceeedxj333GPTLslwcHAw9u/fb9M+ZswYQ5Lxww8/WNtOnz5teHt755iqd3363vLly02/o9x8+umnhiRj+/btObb169fP8Pf3v+n4ESNGGI6Ojrluq1y5svHII48YhmEYycnJhiRj5syZOfq9+eabhiTj4MGDue7n7NmzhrOzs9G/f3+b9nPnzhmSDF9fX8PT09N45ZVXjJUrVxrdunUzJBmLFy++ae0AYC9ccQKAInDvvfcqNjZWvXr10r59+/Tyyy8rLCxM1apV0+eff2463jAMffbZZ+rZs6cMw9CZM2esr7CwMKWkpOjHH3+0GRMeHq7y5ctb3/ft21cBAQHauHHjTY81fPhw/fbbbxo6dKh+/fVX/fLLLwoPD9epU6ckyfSKVX4dPHhQlStXVuXKldWwYUO98sor6tWrl9577z1rnzVr1ig7O1v9+/e3+ez+/v6qV6+etm3bJklycHBQu3bttH37dknSgQMHdPbsWU2cOFGGYSg2NlbSn1eh7rzzTpsrWn+9ynb+/HmlpKSoQ4cOOb5XSerUqZMaN25s07Zx40bdddddatOmjbWtcuXKOaapSdLgwYNlGIbNNMz8uP4zcHFxybHN1dXV9Gd05coVOTs757rtr+PNjvPXPv9r9erVysjIyPH5r0/LO3v2rJYuXapx48apf//+2rBhgxo3bqx//OMfN60dAOyF4AQARaR169Zas2aNzp8/r507d2rSpEm6ePGi+vbtq19//fWmY5OTk3XhwgW988471pBx/TVkyBBJynE/UL169WzeWywW1a1b1/R5Qs8884wmT56sjz76SHfccYeCg4N19OhRjR8/XpLk6emZz09+c0FBQdqyZYs2b96st956S9WqVVNycrL1H+aSdPjwYRmGoXr16uX4/AcOHLD57B06dNCePXt05coVxcTEKCAgQC1atFDTpk2t0/W+/fZbdejQwaaO9evX66677pKrq6sqVqyoypUra9GiRbneh1SrVq0cbX/88UeO71zSLa1CmJKSosTEROvr3Llzkv4b8tLT03OMuXr1aq5TLf/Kzc1NGRkZuW7763iz4/y1z/9asWKFKlasqO7du+c4tiQ5OTmpb9++1nYHBwcNGDBA//nPf5SQkHDT+gHAHliOHACKmLOzs/Vm+/r162vIkCH69NNPFRkZecMx1xcYePzxxzVo0KBc+zRp0qTQapw9e7bGjRun/fv3y9vbW8HBwZo8ebIkqX79+oV2HEny8PBQaGio9f3dd9+tFi1aaPLkyVq4cKGkPz+/xWLRl19+KUdHxxz7+GuYa9++vTIzMxUbG6uYmBhrQOrQoYNiYmJ08OBBJScn2wSnmJgY9erVSx07dtRbb72lgIAAOTk5afny5froo49yHM8smBSW0aNH6/3337e+79Spk6KjoxUQECBJ1quAf3Xq1ClVrVr1pvsNCAhQVlaWTp8+bbM4SUZGhs6ePWsdX7FiRbm4uNzwOJJyPVZCQoJiYmL09NNPy8nJyWZbxYoV5erqKh8fnxw/y+u1nD9/XjVq1LjpZwCAokZwAgA7un7T/V//YWqxWHL0u34DfVZWlk3IuJnDhw/bvDcMQ0eOHMlzwKpQoYLat29vff/111+revXqatiwYZ7GF1STJk30+OOP6+2339a4ceNUo0YN1alTR4ZhqFatWqbBrU2bNnJ2dlZMTIxiYmL0/PPPS/rzOUNLlixRVFSU9f11n332mVxdXbV582abaWnLly/Pc901a9bM8Z1L0qFDh/K8j/81fvx4Pf7449b3FSpUkCTdeeedKleunHbv3q3+/ftbt2dkZCguLs6mLTfNmjWTJO3evVs9evSwtu/evVvZ2dnW7Q4ODgoODtbu3btz7OOHH35Q7dq1baaDXvfxxx/LMIxcpyk6ODioWbNm2rVrlzIyMmymDJ48eVLSn+c7ABQ3TNUDgCKwbds2GYaRo/36/UZ/nc7l4eFhs5y2JDk6Ourhhx/WZ599luuqdsnJyTna/vWvf9mseLZ69WqdOnUqx9SpvFi5cqV27dqlMWPGyMHh9v/VMX78eGVmZmru3LmSpD59+sjR0VEzZszI8T0ahqGzZ89a319fzvzjjz9WQkKCzRWnK1euaOHChapTp471qo305/drsViUlZVlbYuPj9e6devyXHOPHj30/fffa+fOnda25ORkrVixIkffvC5H3rhxY4WGhlpfLVu2lCR5e3srNDRUH374oc3P+IMPPtClS5fUr18/a9vly5d18OBBnTlzxtp2zz33qGLFilq0aJHN8RYtWiR3d3fdf//91ra+fftq165dNuHp0KFD2rp1q81x/uqjjz5SjRo1bIL3Xw0YMEBZWVk2V9OuXr2qFStWqHHjxqZXzADALuy0KAUAlCl33HGHUatWLSMiIsJ45513jDfeeMN47LHHDEdHRyMoKMhmdbcePXoYHh4exmuvvWZ8/PHHxvfff28YhmEkJiYaNWvWNNzd3Y3Ro0cbb7/9tjFnzhyjX79+RoUKFazjr6+qFxwcbDRp0sSYN2+eMXHiRMPV1dWoW7eukZaWdtNav/nmG6Nr167GP//5T2Pp0qXGU089ZTg6OhrdunUzMjMzTT/rrT4A97r777/f8PDwsK4iOGfOHEOS0a5dO+Pll182Fi1aZIwfP96oV6+e8corr9iMnThxoiHJ8Pb2NrKysqztDRo0MCQZgwcPtukfFRVlSDI6dOhgLFq0yJgxY4ZRpUoVo0mTJsb//lUpyRgxYkSOek+ePGn4+voaFSpUMKZPn2688sorRr169az7KMxV9QzDMPbs2WO4uLgYzZs3NxYtWmS88MILhqurq3HffffZ9Lt+PkRGRtq0X18Vr2/fvsaSJUuM8PBwQ1KOlQtTU1ONOnXqGFWqVDFefvllY968eUZgYKBRtWpV4/Tp0znq+vnnnw1JxsSJE29Y++XLl4077rjDcHJyMsaNG2csXLjQaN26teHo6Ghs3LixwN8JANxOBCcAKAJffvml8eSTTxoNGzY0PD09DWdnZ6Nu3brGqFGjjKSkJJu+Bw8eNDp27Gi4ubkZkmyWJk9KSjJGjBhhBAYGGk5OToa/v7/RtWtX45133rH2uf4P5Y8//tiYNGmSUaVKFcPNzc24//77jT/++MO01iNHjhj33XefUalSJcPFxcVo2LChMWfOHCM9PT1Pn7WwglN0dHSOf/B/9tlnRvv27Q0PDw/Dw8PDaNiwoTFixAjj0KFDNmM3bNhgSDK6d+9u0/7UU08Zkoxly5blON6yZcuMevXqWT/z8uXLjcjIyDwHJ8MwjJ9++sno1KmT4erqalSrVs2YNWuWsWzZstsSnAzDMGJiYox27doZrq6uRuXKlY0RI0bYLEFvGDcOToZhGO+8847RoEEDw9nZ2ahTp44xb948Izs7O0e/48ePG3379jW8vLwMT09P44EHHjAOHz6ca03XQ+tPP/1009qTkpKMQYMGGRUrVjRcXFyMtm3bGps2bcr7hweAImYxjFzmjgAASqzo6Gh16dJFn376qc2qZUUlPj5etWrV0rZt29S5c+ciPz4AALcD9zgBAAAAgAmCEwAAAACYIDgBAAAAgAm7Bqft27erZ8+eqlq1qiwWS56WfY2OjlaLFi3k4uKiunXr6r333rvtdQJASdK5c2cZhmGX+5skKSgoSIZhcH8TAKBUsWtwSktLU9OmTfXmm2/mqf+xY8d0//33q0uXLoqLi9OYMWP01FNPafPmzbe5UgAAAABlWbFZVc9isWjt2rXq3bv3DftMmDBBGzZssHn44yOPPKILFy5o06ZNRVAlAAAAgLKonL0LyI/Y2FiFhobatIWFhWnMmDE3HJOenq709HTr++zsbJ07d06+vr6yWCy3q1QAAAAAxZxhGLp48aKqVq0qB4ebT8YrUcEpMTFRfn5+Nm1+fn5KTU3VlStX5ObmlmPMnDlzNGPGjKIqEQAAAEAJc/z4cVWvXv2mfUpUcCqISZMmKSIiwvo+JSVFNWrU0PHjx+Xl5WXHygAAAADYU2pqqgIDA1W+fHnTviUqOPn7+yspKcmmLSkpSV5eXrlebZIkFxcXubi45Gj38vIiOAEAAADI0y08Jeo5TiEhIYqKirJp27Jli0JCQuxUEQAAAICywK7B6dKlS4qLi1NcXJykP5cbj4uLU0JCgqQ/p9mFh4db+z/zzDP6/fffNX78eB08eFBvvfWWVq1apbFjx9qjfAAAAABlhF2D0+7du9W8eXM1b95ckhQREaHmzZtr2rRpkqRTp05ZQ5Qk1apVSxs2bNCWLVvUtGlTvfbaa1q6dKnCwsLsUj8AAACAsqHYPMepqKSmpsrb21spKSnc4wQAAIAcsrKylJmZae8yUEicnZ1vuNR4frJBiVocAgAAALhdDMNQYmKiLly4YO9SUIgcHBxUq1YtOTs739J+CE4AAACAZA1NVapUkbu7e55WWkPxlp2drZMnT+rUqVOqUaPGLf1MCU4AAAAo87KysqyhydfX197loBBVrlxZJ0+e1LVr1+Tk5FTg/ZSo5cgBAACA2+H6PU3u7u52rgSF7foUvaysrFvaD8EJAAAA+H9Mzyt9CutnSnACAAAAABMEJwAAAAAwweIQAAAAwE0ETdxQpMeLf+n+PPc1m4YWGRmp6dOn32JFBWOxWLR27Vr17t3bLscvbAQnAAAAoIQ6deqU9c8rV67UtGnTdOjQIWubp6dnvvaXkZFxy887Kq2YqgcAAACUUP7+/taXt7e3LBaL9X1aWpoGDhwoPz8/eXp6qnXr1vr6669txgcFBWnWrFkKDw+Xl5eXnn76aUnSkiVLFBgYKHd3dz300EOaO3eufHx8bMb++9//VosWLeTq6qratWtrxowZunbtmnW/kvTQQw/JYrFY35dkBCcAAACgFLp06ZJ69OihqKgo7d27V926dVPPnj2VkJBg0+/VV19V06ZNtXfvXk2dOlXfffednnnmGY0ePVpxcXG69957NXv2bJsxMTExCg8P1+jRo/Xrr7/q7bff1nvvvWftt2vXLknS8uXLderUKev7koypegAAAEAp1LRpUzVt2tT6ftasWVq7dq0+//xzjRw50tp+zz336O9//7v1/QsvvKDu3btr3LhxkqT69etrx44dWr9+vbXPjBkzNHHiRA0aNEiSVLt2bc2aNUvjx49XZGSkKleuLEny8fGRv7//bf2cRYUrTgAAAEApdOnSJY0bN06NGjWSj4+PPD09deDAgRxXnFq1amXz/tChQ2rTpo1N2/++37dvn2bOnClPT0/ra9iwYTp16pQuX758ez6QnXHFCQAAACiFxo0bpy1btujVV19V3bp15ebmpr59+yojI8Omn4eHR773fenSJc2YMUN9+vTJsc3V1bXANRdnBCcAAACgFPruu+80ePBgPfTQQ5L+DDvx8fGm4xo0aJDjnqT/fd+iRQsdOnRIdevWveF+nJyclJWVlf/CiymCEwAAAFAK1atXT2vWrFHPnj1lsVg0depUZWdnm44bNWqUOnbsqLlz56pnz57aunWrvvzyS5tnRk2bNk0PPPCAatSoob59+8rBwUH79u3TL7/8on/84x+S/lxZLyoqSnfffbdcXFxUoUKF2/ZZiwL3OAEAAACl0Ny5c1WhQgW1a9dOPXv2VFhYmFq0aGE67u6779bixYs1d+5cNW3aVJs2bdLYsWNtpuCFhYVp/fr1+uqrr9S6dWvdddddmjdvnmrWrGnt89prr2nLli0KDAxU8+bNb8tnLEoWwzAMexdRlFJTU+Xt7a2UlBR5eXnZuxwAAAAUA1evXtWxY8dUq1atUnuPzq0YNmyYDh48qJiYGHuXkm83+9nmJxswVQ8AAACAjVdffVX33nuvPDw89OWXX+r999/XW2+9Ze+y7IrgBAAAAMDGzp079fLLL+vixYuqXbu2Fi5cqKeeesreZdkVwQkAAACAjVWrVtm7hGKHxSEAAAAAwATBCQAAAABMEJwAAAAAwATBCQAAAABMEJwAAAAAwATBCQAAAABMEJwAAAAAmBo8eLB69+5tfd+5c2eNGTPmlvZZGPsoKjzHCQAAALiZ6d5FfLyUfHUfPHiw3n//fUmSk5OTatSoofDwcE2ePFnlyt2+f+6vWbNGTk5OeeobHR2tLl266Pz58/Lx8SnQPuyN4AQAAACUcN26ddPy5cuVnp6ujRs3asSIEXJyctKkSZNs+mVkZMjZ2blQjlmxYsVisY+iwlQ9AAAAoIRzcXGRv7+/atasqWeffVahoaH6/PPPrdPrZs+erapVq6pBgwaSpOPHj6t///7y8fFRxYoV9eCDDyo+Pt66v6ysLEVERMjHx0e+vr4aP368DMOwOeb/TrNLT0/XhAkTFBgYKBcXF9WtW1fLli1TfHy8unTpIkmqUKGCLBaLBg8enOs+zp8/r/DwcFWoUEHu7u7q3r27Dh8+bN3+3nvvycfHR5s3b1ajRo3k6empbt266dSpU4X7heaC4AQAAACUMm5ubsrIyJAkRUVF6dChQ9qyZYvWr1+vzMxMhYWFqXz58oqJidF3331nDSDXx7z22mt677339O677+rbb7/VuXPntHbt2pseMzw8XB9//LEWLlyoAwcO6O2335anp6cCAwP12WefSZIOHTqkU6dOacGCBbnuY/Dgwdq9e7c+//xzxcbGyjAM9ejRQ5mZmdY+ly9f1quvvqoPPvhA27dvV0JCgsaNG1cYX9tNMVUPAAAAKCUMw1BUVJQ2b96sUaNGKTk5WR4eHlq6dKl1it6HH36o7OxsLV26VBaLRZK0fPly+fj4KDo6Wvfdd5/mz5+vSZMmqU+fPpKkxYsXa/PmzTc87m+//aZVq1Zpy5YtCg0NlSTVrl3buv36lLwqVarY3OP0V4cPH9bnn3+u7777Tu3atZMkrVixQoGBgVq3bp369esnScrMzNTixYtVp04dSdLIkSM1c+bMgn5leUZwAgAAAEq49evXy9PTU5mZmcrOztZjjz2m6dOna8SIEQoODra5r2nfvn06cuSIypcvb7OPq1ev6ujRo0pJSdGpU6fUtm1b67Zy5cqpVatWOabrXRcXFydHR0d16tSpwJ/hwIEDKleunM1xfX191aBBAx04cMDa5u7ubg1NkhQQEKDTp08X+Lh5RXACAAAASrguXbpo0aJFcnZ2VtWqVW1W0/Pw8LDpe+nSJbVs2VIrVqzIsZ/KlSsX6Phubm4FGlcQ/7sKn8ViuWGgK0zc4wQAAACUcB4eHqpbt65q1KhhugR5ixYtdPjwYVWpUkV169a1eXl7e8vb21sBAQH64YcfrGOuXbumPXv23HCfwcHBys7O1jfffJPr9utXvLKysm64j0aNGunatWs2xz179qwOHTqkxo0b3/QzFQWCEwAAAFCGDBw4UJUqVdKDDz6omJgYHTt2TNHR0Xruuef0n//8R5I0evRovfTSS1q3bp0OHjyo4cOH68KFCzfcZ1BQkAYNGqQnn3xS69ats+5z1apVkqSaNWvKYrFo/fr1Sk5O1qVLl3Lso169enrwwQc1bNgwffvtt9q3b58ef/xxVatWTQ8++OBt+S7yg+AEAAAAlCHu7u7avn27atSooT59+qhRo0YaOnSorl69Ki8vL0nS3//+dz3xxBMaNGiQQkJCVL58eT300EM33e+iRYvUt29fDR8+XA0bNtSwYcOUlpYmSapWrZpmzJihiRMnys/PTyNHjsx1H8uXL1fLli31wAMPKCQkRIZhaOPGjcXiIbkWoygmBBYjqamp8vb2VkpKivXEAAAAQNl29epVHTt2TLVq1ZKrq6u9y0EhutnPNj/ZgCtOAAAAAGCC4AQAAAAAJghOAAAAAGCC4AQAAAAAJghOAAAAwP8rY+umlQmF9TMlOAEAAKDMu77c9eXLl+1cCQpbRkaGJMnR0fGW9nPzxwoDAAAAZYCjo6N8fHx0+vRpSX8+68hisdi5Ktyq7OxsJScny93dXeXK3Vr0ITgBAAAAkvz9/SXJGp5QOjg4OKhGjRq3HIQJTgAAAIAki8WigIAAValSRZmZmfYuB4XE2dlZDg63focSwQkAAAD4C0dHx1u+HwalD4tDAAAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJuwenN998U0FBQXJ1dVXbtm21c+fOm/afP3++GjRoIDc3NwUGBmrs2LG6evVqEVULAAAAoCyya3BauXKlIiIiFBkZqR9//FFNmzZVWFiYTp8+nWv/jz76SBMnTlRkZKQOHDigZcuWaeXKlZo8eXIRVw4AAACgLLFrcJo7d66GDRumIUOGqHHjxlq8eLHc3d317rvv5tp/x44duvvuu/XYY48pKChI9913nx599FHTq1QAAAAAcCvsFpwyMjK0Z88ehYaG/rcYBweFhoYqNjY21zHt2rXTnj17rEHp999/18aNG9WjR48bHic9PV2pqak2LwAAAADIj3L2OvCZM2eUlZUlPz8/m3Y/Pz8dPHgw1zGPPfaYzpw5o/bt28swDF27dk3PPPPMTafqzZkzRzNmzCjU2gEAAACULXZfHCI/oqOj9eKLL+qtt97Sjz/+qDVr1mjDhg2aNWvWDcdMmjRJKSkp1tfx48eLsGIAAAAApYHdrjhVqlRJjo6OSkpKsmlPSkqSv79/rmOmTp2qJ554Qk899ZQkKTg4WGlpaXr66af1wgsvyMEhZw50cXGRi4tL4X8AAAAAAGWG3a44OTs7q2XLloqKirK2ZWdnKyoqSiEhIbmOuXz5co5w5OjoKEkyDOP2FQsAAACgTLPbFSdJioiI0KBBg9SqVSu1adNG8+fPV1pamoYMGSJJCg8PV7Vq1TRnzhxJUs+ePTV37lw1b95cbdu21ZEjRzR16lT17NnTGqAAAAAAoLDZNTgNGDBAycnJmjZtmhITE9WsWTNt2rTJumBEQkKCzRWmKVOmyGKxaMqUKTpx4oQqV66snj17avbs2fb6CAAAAADKAItRxua4paamytvbWykpKfLy8rJ3OQAAAADsJD/ZoEStqgcAAAAA9kBwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATdg9Ob775poKCguTq6qq2bdtq586dN+1/4cIFjRgxQgEBAXJxcVH9+vW1cePGIqoWAAAAQFlUzp4HX7lypSIiIrR48WK1bdtW8+fPV1hYmA4dOqQqVark6J+RkaF7771XVapU0erVq1WtWjX98ccf8vHxKfriAQAAAJQZFsMwDHsdvG3btmrdurXeeOMNSVJ2drYCAwM1atQoTZw4MUf/xYsX65VXXtHBgwfl5ORUoGOmpqbK29tbKSkp8vLyuqX6AQAAAJRc+ckGdpuql5GRoT179ig0NPS/xTg4KDQ0VLGxsbmO+fzzzxUSEqIRI0bIz89Pd955p1588UVlZWXd8Djp6elKTU21eQEAAABAftgtOJ05c0ZZWVny8/Ozaffz81NiYmKuY37//XetXr1aWVlZ2rhxo6ZOnarXXntN//jHP254nDlz5sjb29v6CgwMLNTPAQAAAKD0s/viEPmRnZ2tKlWq6J133lHLli01YMAAvfDCC1q8ePENx0yaNEkpKSnW1/Hjx4uwYgAAAAClgd0Wh6hUqZIcHR2VlJRk056UlCR/f/9cxwQEBMjJyUmOjo7WtkaNGikxMVEZGRlydnbOMcbFxUUuLi6FWzwAAACAMsVuV5ycnZ3VsmVLRUVFWduys7MVFRWlkJCQXMfcfffdOnLkiLKzs61tv/32mwICAnINTQAAAABQGOw6VS8iIkJLlizR+++/rwMHDujZZ59VWlqahgwZIkkKDw/XpEmTrP2fffZZnTt3TqNHj9Zvv/2mDRs26MUXX9SIESPs9REAAAAAlAEFmqq3bds2denS5ZYPPmDAACUnJ2vatGlKTExUs2bNtGnTJuuCEQkJCXJw+G+2CwwM1ObNmzV27Fg1adJE1apV0+jRozVhwoRbrgUAAAAAbqRAz3FycXFR9erVNWTIEA0aNKhErVTHc5wAAAAASEXwHKcTJ05o5MiRWr16tWrXrq2wsDCtWrVKGRkZBSoYAAAAAIqzAgWnSpUqaezYsYqLi9MPP/yg+vXra/jw4apataqee+457du3r7DrBAAAAAC7ueXFIVq0aKFJkyZp5MiRunTpkt599121bNlSHTp00P79+wujRgAAAACwqwIHp8zMTK1evVo9evRQzZo1tXnzZr3xxhtKSkrSkSNHVLNmTfXr168wawUAAAAAuyjQ4hCjRo3Sxx9/LMMw9MQTT+ipp57SnXfeadMnMTFRVatWtXnmUnHA4hAAAAAApPxlgwItR/7rr7/q9ddfV58+feTi4pJrn0qVKmnbtm0F2T0AAAAAFCsFmqoXGRmpfv365QhN165d0/bt2yVJ5cqVU6dOnW69QgAAAACwswIFpy5duujcuXM52lNSUgrlwbgAAAAAUJwUKDgZhiGLxZKj/ezZs/Lw8LjlogAAAACgOMnXPU59+vSRJFksFg0ePNhmql5WVpZ++ukntWvXrnArBAAAAAA7y1dw8vb2lvTnFafy5cvLzc3Nus3Z2Vl33XWXhg0bVrgVAgAAAICd5Ss4LV++XJIUFBSkcePGMS0PAAAAQJlQoOc4lWQ8xwkAAACAdJue49SiRQtFRUWpQoUKat68ea6LQ1z3448/5r1aAAAAACjm8hycHnzwQetiEL17975d9QAAAABAscNUPQAAAABlUn6yQYGe4wQAAAAAZUmep+pVqFDhpvc1/dW5c+cKXBAAAAAAFDd5Dk7z58+/jWUAAAAAQPGV5+A0aNCg21kHAAAAABRbeQ5Oqamp1humUlNTb9qXRRcAAAAAlCb5usfp1KlTqlKlinx8fHK938kwDFksFmVlZRVqkQAAAABgT3kOTlu3blXFihUlSdu2bbttBQEAAABAccNznAAAAACUSfnJBnm+4vS/zp8/r2XLlunAgQOSpMaNG2vIkCHWq1IAAAAAUFoU6AG427dvV1BQkBYuXKjz58/r/PnzWrhwoWrVqqXt27cXdo0AAAAAYFcFmqoXHByskJAQLVq0SI6OjpKkrKwsDR8+XDt27NDPP/9c6IUWFqbqAQAAAJDylw0KdMXpyJEj+vvf/24NTZLk6OioiIgIHTlypCC7BAAAAIBiq0DBqUWLFtZ7m/7qwIEDatq06S0XBQAAAADFSZ4Xh/jpp5+sf37uuec0evRoHTlyRHfddZck6fvvv9ebb76pl156qfCrBAAAAAA7yvM9Tg4ODrJYLDLrXtwfgMs9TgAAAACk27Qc+bFjx265MAAAAAAoifIcnGrWrHk76wAAAACAYqvAD8CVpF9//VUJCQnKyMiwae/Vq9ctFQUAAAAAxUmBgtPvv/+uhx56SD///LPNfU8Wi0WSivU9TgAAAACQXwVajnz06NGqVauWTp8+LXd3d+3fv1/bt29Xq1atFB0dXcglAgAAAIB9FeiKU2xsrLZu3apKlSrJwcFBDg4Oat++vebMmaPnnntOe/fuLew6AQAAAMBuCnTFKSsrS+XLl5ckVapUSSdPnpT05wIShw4dKrzqAAAAAKAYKNAVpzvvvFP79u1TrVq11LZtW7388stydnbWO++8o9q1axd2jQAAAABgVwUKTlOmTFFaWpokaebMmXrggQfUoUMH+fr6auXKlYVaIAAAAADYm8W4viTeLTp37pwqVKhgXVmvuMrP04EBAAAAlF75yQa39BwnSTp+/LgkKTAw8FZ3BQAAAADFUoEWh7h27ZqmTp0qb29vBQUFKSgoSN7e3poyZYoyMzMLu0YAAAAAsKsCXXEaNWqU1qxZo5dfflkhISGS/lyifPr06Tp79qwWLVpUqEUCAAAAgD0V6B4nb29vffLJJ+revbtN+8aNG/Xoo48qJSWl0AosbNzjBAAAAEDKXzYo0FQ9FxcXBQUF5WivVauWnJ2dC7JLAAAAACi2ChScRo4cqVmzZik9Pd3alp6ertmzZ2vkyJGFVhwAAAAAFAd5vsepT58+Nu+//vprVa9eXU2bNpUk7du3TxkZGeratWvhVggAAAAAdpbn4OTt7W3z/uGHH7Z5z3LkAAAAAEqrPAen5cuX3846AAAAAKDYuqUH4CYnJ+vQoUOSpAYNGqhy5cqFUhQAAAAAFCcFWhwiLS1NTz75pAICAtSxY0d17NhRVatW1dChQ3X58uXCrhEAAAAA7KpAwSkiIkLffPONvvjiC124cEEXLlzQv//9b33zzTf6+9//Xtg1AgAAAIBdFegBuJUqVdLq1avVuXNnm/Zt27apf//+Sk5OLqz6Ch0PwAUAAAAgFcEDcC9fviw/P78c7VWqVGGqHgAAAIBSp0DBKSQkRJGRkbp69aq17cqVK5oxY4ZCQkIKrTgAAAAAKA4KtKre/Pnz1a1btxwPwHV1ddXmzZsLtUAAAAAAsLcC3eMk/Tldb8WKFTp48KAkqVGjRho4cKDc3NwKtcDCxj1OAAAAAKT8ZYN8X3HKzMxUw4YNtX79eg0bNqzARQIAAABASZHve5ycnJxs7m0CAAAAgNKuQItDjBgxQv/85z917dq1wq4HAAAAAIqdAi0OsWvXLkVFRemrr75ScHCwPDw8bLavWbOmUIoDAAAAgOKgQMHJx8dHDz/8cGHXAgAAAADFUr6CU3Z2tl555RX99ttvysjI0D333KPp06cX+5X0AAAAAOBW5Osep9mzZ2vy5Mny9PRUtWrVtHDhQo0YMeJ21QYAAAAAxUK+gtO//vUvvfXWW9q8ebPWrVunL774QitWrFB2dvbtqg8AAAAA7C5fwSkhIUE9evSwvg8NDZXFYtHJkydvqYg333xTQUFBcnV1Vdu2bbVz5848jfvkk09ksVjUu3fvWzo+AAAAANxMvoLTtWvX5OrqatPm5OSkzMzMAhewcuVKRUREKDIyUj/++KOaNm2qsLAwnT59+qbj4uPjNW7cOHXo0KHAxwYAAACAvLAYhmHktbODg4O6d+8uFxcXa9sXX3yhe+65x2ZJ8vwsR962bVu1bt1ab7zxhqQ/F6AIDAzUqFGjNHHixFzHZGVlqWPHjnryyScVExOjCxcuaN26dXk6Xmpqqry9vZWSkiIvL6881wkAAACgdMlPNsjXqnqDBg3K0fb444/nr7q/yMjI0J49ezRp0iRrm4ODg0JDQxUbG3vDcTNnzlSVKlU0dOhQxcTE3PQY6enpSk9Pt75PTU0tcL0AAAAAyqZ8Bafly5cX6sHPnDmjrKws+fn52bT7+fnp4MGDuY759ttvtWzZMsXFxeXpGHPmzNGMGTNutVQAAAAAZVi+7nGyt4sXL+qJJ57QkiVLVKlSpTyNmTRpklJSUqyv48eP3+YqAQAAAJQ2+briVNgqVaokR0dHJSUl2bQnJSXJ398/R/+jR48qPj5ePXv2tLZdXwq9XLlyOnTokOrUqWMzxsXFxeaeLAAAAADIL7tecXJ2dlbLli0VFRVlbcvOzlZUVJRCQkJy9G/YsKF+/vlnxcXFWV+9evVSly5dFBcXp8DAwKIsHwAAAEAZYdcrTpIUERGhQYMGqVWrVmrTpo3mz5+vtLQ0DRkyRJIUHh6uatWqac6cOXJ1ddWdd95pM97Hx0eScrQDAAAAQGGxe3AaMGCAkpOTNW3aNCUmJqpZs2batGmTdcGIhIQEOTiUqFuxAAAAAJQy+XqOU2nAc5wAAAAASPnLBlzKAQAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMFHO3gUAAIAyYrq3vSsoOaan2LsCFBTned6UwHOc4FQMBE3cYO8SSoT4l+63dwkoIM7xvOEcL7k4x/Mm3tXeFeBWcJ7nDed56UVwQsnB/+DkTQn8Hxz8P87xvOEcBwDYAfc4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAIAJghMAAAAAmCA4AQAAAICJYhGc3nzzTQUFBcnV1VVt27bVzp07b9h3yZIl6tChgypUqKAKFSooNDT0pv0BAAAA4FbZPTitXLlSERERioyM1I8//qimTZsqLCxMp0+fzrV/dHS0Hn30UW3btk2xsbEKDAzUfffdpxMnThRx5QAAAADKCrsHp7lz52rYsGEaMmSIGjdurMWLF8vd3V3vvvturv1XrFih4cOHq1mzZmrYsKGWLl2q7OxsRUVFFXHlAAAAAMoKuwanjIwM7dmzR6GhodY2BwcHhYaGKjY2Nk/7uHz5sjIzM1WxYsVct6enpys1NdXmBQAAAAD5YdfgdObMGWVlZcnPz8+m3c/PT4mJiXnax4QJE1S1alWb8PVXc+bMkbe3t/UVGBh4y3UDAAAAKFvsPlXvVrz00kv65JNPtHbtWrm6uubaZ9KkSUpJSbG+jh8/XsRVAgAAACjpytnz4JUqVZKjo6OSkpJs2pOSkuTv73/Tsa+++qpeeuklff3112rSpMkN+7m4uMjFxaVQ6gUAAABQNtn1ipOzs7Natmxps7DD9YUeQkJCbjju5Zdf1qxZs7Rp0ya1atWqKEoFAAAAUIbZ9YqTJEVERGjQoEFq1aqV2rRpo/nz5ystLU1DhgyRJIWHh6tatWqaM2eOJOmf//ynpk2bpo8++khBQUHWe6E8PT3l6elpt88BAAAAoPSye3AaMGCAkpOTNW3aNCUmJqpZs2batGmTdcGIhIQEOTj898LYokWLlJGRob59+9rsJzIyUtOnTy/K0gEAAACUEXYPTpI0cuRIjRw5Mtdt0dHRNu/j4+Nvf0EAAAAA8BclelU9AAAAACgKBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMEFwAgAAAAATBCcAAAAAMFEsgtObb76poKAgubq6qm3bttq5c+dN+3/66adq2LChXF1dFRwcrI0bNxZRpQAAAADKIrsHp5UrVyoiIkKRkZH68ccf1bRpU4WFhen06dO59t+xY4ceffRRDR06VHv37lXv3r3Vu3dv/fLLL0VcOQAAAICywu7Bae7cuRo2bJiGDBmixo0ba/HixXJ3d9e7776ba/8FCxaoW7duev7559WoUSPNmjVLLVq00BtvvFHElQMAAAAoK8rZ8+AZGRnas2ePJk2aZG1zcHBQaGioYmNjcx0TGxuriIgIm7awsDCtW7cu1/7p6elKT0+3vk9JSZEkpaam3mL1hSc7/bK9SygRUi2GvUsoGYrRuX0d53jecI7nEed4icU5ng+c5yUW53keFZNz/HomMAzzn5tdg9OZM2eUlZUlPz8/m3Y/Pz8dPHgw1zGJiYm59k9MTMy1/5w5czRjxowc7YGBgQWsGvbibe8CSoqX+KZKKn5yecQ5XmLxk8sHzvMSi59cHhWzc/zixYvy9r55TXYNTkVh0qRJNleosrOzde7cOfn6+spisdixMuRHamqqAgMDdfz4cXl5edm7HKDQcY6jtOMcR1nAeV7yGIahixcvqmrVqqZ97RqcKlWqJEdHRyUlJdm0JyUlyd/fP9cx/v7++erv4uIiFxcXmzYfH5+CFw278vLy4hcRSjXOcZR2nOMoCzjPSxazK03X2XVxCGdnZ7Vs2VJRUVHWtuzsbEVFRSkkJCTXMSEhITb9JWnLli037A8AAAAAt8ruU/UiIiI0aNAgtWrVSm3atNH8+fOVlpamIUOGSJLCw8NVrVo1zZkzR5I0evRoderUSa+99pruv/9+ffLJJ9q9e7feeecde34MAAAAAKWY3YPTgAEDlJycrGnTpikxMVHNmjXTpk2brAtAJCQkyMHhvxfG2rVrp48++khTpkzR5MmTVa9ePa1bt0533nmnvT4CioCLi4siIyNzTLsESgvOcZR2nOMoCzjPSzeLkZe19wAAAACgDLP7A3ABAAAAoLgjOAEAAACACYITAAAAAJggOAEAAACACYITio3o6GgFBQXluf+uXbv06aef3r6CgEKW33P8t99+09KlS29fQcBtwO9ylHac42UXwQnF1rlz5zRw4EB5eXnJx8dHQ4cO1aVLl6zbr127puHDhysrK8uOVQIFN3v2bLVr107u7u7y8fHJsd3T01PPPvuszp07V/TFAYUgPj5eQ4cOVa1ateTm5qY6deooMjJSGRkZ1j78LkdJ16tXL9WoUUOurq4KCAjQE088oZMnT1q3c46XHgQnFFsDBw7U/v37tWXLFq1fv17bt2/X008/bd3etm1bOTo6aseOHXasEii4jIwM9evXT88++2yu26tWrarmzZtrw4YNRVwZUDgOHjyo7Oxsvf3229q/f7/mzZunxYsXa/LkydY+/C5HSdelSxetWrVKhw4d0meffaajR4+qb9++1u2c46UHwQnF0oEDB7Rp0yYtXbpUbdu2Vfv27fX666/rk08+sf4vjoODgx544AH9+9//tnO1QMHMmDFDY8eOVXBw8A379OrVi3McJVa3bt20fPly3Xfffapdu7Z69eqlcePGac2aNdY+/C5HSTd27Fjdddddqlmzptq1a6eJEyfq+++/V2ZmpiTO8dKE4IRiKTY2Vj4+PmrVqpW1LTQ0VA4ODvrhhx+sbb169dIXX3xhjxKBItGrVy999dVXNlObgJIsJSVFFStWtGnjdzlKi3PnzmnFihVq166dnJycrO2c46UDwQnFUmJioqpUqWLTVq5cOVWsWFGJiYnWtnvvvVf/+c9/dPDgwaIuESgSTZo0ka+vr7Zu3WrvUoBbduTIEb3++uv629/+ZtPO73KUdBMmTJCHh4d8fX2VkJCQ4+oS53jpQHBCiebm5qbOnTtr48aN9i4FuG169OjBOY4S78SJE+rWrZv69eunYcOG2WzjdzlKuueff1579+7VV199JUdHR4WHh8swDOt2zvHSoZy9CwBy4+/vr9OnT9u0Xbt2TefOnZO/v79N+7Fjx/K1LChQ0hw7dkyhoaH2LgMosJMnT6pLly5q166d3nnnnVz78LscJVmlSpVUqVIl1a9fX40aNVJgYKC+//57hYSEWPtwjpd8XHFCsRQSEqILFy5oz5491ratW7cqOztbbdu2tbYdOXJEv//+u8LCwuxRJnDbpaWlaevWrerVq5e9SwEK5MSJE+rcubNatmyp5cuXy8Eh5z89+F2O0iQ7O1uSlJ6ebm3jHC8duOKEYqlRo0bq1q2bhg0bpsWLFyszM1MjR47UI488oqpVq1r7ff755+ratas8PDzsWC1QMAkJCTp37pwSEhKUlZWluLg4SVLdunXl6ekpSdq8ebPq1KmjunXr2rFSoGCuh6aaNWvq1VdfVXJysnXbX2cP8LscJdUPP/ygXbt2qX379qpQoYKOHj2qqVOnqk6dOjZXmzjHSweCE4qtFStWaOTIkeratascHBz08MMPa+HChTZ9/v3vf+uxxx6zU4XArZk2bZref/996/vmzZtLkrZt26bOnTtL+vMc52oTSqotW7boyJEjOnLkiKpXr26z7a/3f/C7HCWVu7u71qxZo8jISKWlpSkgIEDdunXTlClT5OLiYu3HOV46WIy//uYC7Cg6OlqDBw9WfHx8nvqfO3dOfn5++uOPP2yuQgHFVX7P8aysLPn7++vzzz+3+Z9LoDjjdzlKO87xsot7nFBibdiwQc2aNeOXEEqtHTt2yMHBwea+PqC04Xc5SjvO8dKD4IQS64knntCuXbvsXQZw23To0EFJSUm53kwPlBb8LkdpxzleevC3MYqNoKAgjRkzxt5lALcN5zjKAs5zlHac42UX9zgBAAAAgAmuOAEAAACACYITAAAAAJggOAEAAACACYITAKDEeu+99+Tj42PvMgAAZQDBCQBwWwwePFgWi8X68vX1Vbdu3fTTTz8V2jEGDBig3377rdD291dBQUGaP39+vsd17tyZFbcAoBQiOAEAbptu3brp1KlTOnXqlKKiolSuXDk98MADhbZ/Nzc3ValSpdD2BwDAjRCcAAC3jYuLi/z9/eXv769mzZpp4sSJOn78uJKTk619JkyYoPr168vd3V21a9fW1KlTlZmZad2+b98+denSReXLl5eXl5datmyp3bt3S8o5Ve9mff+XYRiaPn26atSoIRcXF1WtWlXPPfecpD+vGv3xxx8aO3as9YqZJJ09e1aPPvqoqlWrJnd3dwUHB+vjjz+27nPw4MH65ptvtGDBAuu4+Ph4SdIvv/yi7t27y9PTU35+fnriiSd05swZ69jVq1crODhYbm5u8vX1VWhoqNLS0m7tBwAAKDQEJwBAkbh06ZI+/PBD1a1bV76+vtb28uXL67333tOvv/6qBQsWaMmSJZo3b551+8CBA1W9enXt2rVLe/bs0cSJE+Xk5JTrMfLT97PPPtO8efP09ttv6/Dhw1q3bp2Cg4MlSWvWrFH16tU1c+ZM6xUzSbp69apatmypDRs26JdfftHTTz+tJ554Qjt37pQkLViwQCEhIRo2bJh1XGBgoC5cuKB77rlHzZs31+7du7Vp0yYlJSWpf//+kqRTp07p0Ucf1ZNPPqkDBw4oOjpaffr0EY9aBIDio5y9CwAAlF7r16+Xp6enJCktLU0BAQFav369HBz++/92U6ZMsf45KChI48aN0yeffKLx48dLkhISEvT888+rYcOGkqR69erd8Hj57evv76/Q0FA5OTmpRo0aatOmjSSpYsWKcnR0VPny5eXv728dU61aNY0bN876ftSoUdq8ebNWrVqlNm3ayNvbW87OznJ3d7cZ98Ybb6h58+Z68cUXrW3vvvuuAgMD9dtvv+nSpUu6du2a+vTpo5o1a0qSNcQBAIoHrjgBAG6bLl26KC4uTnFxcdq5c6fCwsLUvXt3/fHHH9Y+K1eu1N133y1/f395enpqypQpSkhIsG6PiIjQU089pdDQUL300ks6evToDY+Xn779+vXTlStXVLt2bQ0bNkxr167VtWvXbvp5srKyNGvWLAUHB6tixYry9PTU5s2bberNzb59+7Rt2zZ5enpaX9fD3dGjR9W0aVN17dpVwcHB6tevn5YsWaLz58/fdJ8AgKJFcAIA3DYeHh6qW7eu6tatq9atW2vp0qVKS0vTkiVLJEmxsbEaOHCgevToofXr12vv3r164YUXlJGRYd3H9OnTtX//ft1///3aunWrGjdurLVr1+Z6vPz0DQwM1KFDh/TWW2/Jzc1Nw4cPV8eOHW3ur/pfr7zyihYsWKAJEyZo27ZtiouLU1hYmE29ubl06ZJ69uxpDZHXX4cPH1bHjh3l6OioLVu26Msvv1Tjxo31+uuvq0GDBjp27JjZVwwAKCIEJwBAkbFYLHJwcNCVK1ckSTt27FDNmjX1wgsvqFWrVqpXr57N1ajr6tevr7Fjx+qrr75Snz59tHz58hseIz993dzc1LNnTy1cuFDR0dGKjY3Vzz//LElydnZWVlaWTf/vvvtODz74oB5//HE1bdpUtWvXzrEcem7jWrRoof379ysoKMgaJK+/PDw8rN/N3XffrRkzZmjv3r1ydna+YegDABQ9ghMA4LZJT09XYmKiEhMTdeDAAY0aNcp69UX68x6khIQEffLJJzp69KgWLlxoExauXLmikSNHKjo6Wn/88Ye+++477dq1S40aNcpxrPz0lf5ckW/ZsmX65Zdf9Pvvv+vDDz+Um5ub9R6joKAgbd++XSdOnLCuflevXj1t2bJFO3bs0IEDB/S3v/1NSUlJNvsNCgrSDz/8oPj4eJ05c0bZ2dkaMWKEzp07p0cffVS7du3S0aNHtXnzZg0ZMkRZWVn64Ycf9OKLL2r37t1KSEjQmjVrlJycfMPaAQBFj+AEALhtNm3apICAAAUEBKht27batWuXPv30U3Xu3FmS1KtXL40dO1YjR45Us2bNtGPHDk2dOtU63tHRUWfPnlV4eLjq16+v/v37q3v37poxY0aOY+WnryT5+PhoyZIluvvuu9WkSRN9/fXX+uKLL6wr/s2cOVPx8fGqU6eOKleuLOnPhSxatGihsLAwde7cWf7+/urdu7fNfseNGydHR0c1btxYlStXVkJCgqpWrarvvvtOWVlZuu+++xQcHKwxY8bIx8dHDg4O8vLy0vbt29WjRw/Vr19fU6ZM0Wuvvabu3bsXwk8BAFAYLAZrnQIAAADATXHFCQAAAABMEJwAAAAAwATBCQAAAABMEJwAAAAAwATBCQAAAABMEJwAAAAAwATBCQAAAABMEJwAAAAAwATBCQAAAABMEJwAAAAAwATBCQAAAABMEJwAAAAAwMT/AUFBMjBwpBqEAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from pennylane import numpy as np\n",
    "import pennylane as qml\n",
    "from qrl.env import ProbabilityV0\n",
    "\n",
    "n_qubits = 2\n",
    "target_distribution = np.array([0.25, 0.25, 0.25, 0.25])  # Example target distribution\n",
    "\n",
    "# initialize environment\n",
    "# set ffmpeg=True if you have ffmpeg installed to save as mp4, or ffmpeg=False to save as gif\n",
    "env = ProbabilityV0(\n",
    "    n_qubits=n_qubits,\n",
    "    target_distribution=target_distribution,\n",
    "    alpha=0.7,   # KL vs L2 weight\n",
    "    beta=0.01,   # step penalty\n",
    "    max_steps=10,\n",
    "    ffmpeg=False\n",
    ")\n",
    "\n",
    "# Reset environment\n",
    "params, _ = env.reset()\n",
    "\n",
    "\n",
    "# Use PennyLane's optimizer\n",
    "opt = qml.GradientDescentOptimizer(stepsize=0.2)\n",
    "\n",
    "# params = env.params.copy()\n",
    "for step in range(env.max_steps):\n",
    "    params, cost_val = opt.step_and_cost(env.get_reward, params)\n",
    "    probs = env.circuit(params)\n",
    "\n",
    "    # Save history for rendering\n",
    "    env.history.append(probs)\n",
    "    env.params = params  # update env params\n",
    "    reward = -cost_val\n",
    "    env.rewards.append(-cost_val)\n",
    "    print(f\"Step {step}: Reward = {reward:.4f}\")\n",
    "\n",
    "    if reward > -1e-2:  # close to perfect\n",
    "        break\n",
    "\n",
    "# Animate the full evolution\n",
    "env.render(save_path_without_extension='probabilityV0')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "qrl_env (3.12.3)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}