docs/qc/Adjoint_Sensitivity_Method_for_Kohn_Sham_Equations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "5oKf92DqRHFu5LBZddmUD7",
     "report_properties": {},
     "type": "MD"
    }
   },
   "source": [
    "# The power of Hellmann-Feynman theorem: Kohn-Sham DFT energy derivatives with respect to the parameters of exchange-correlation functional at linear cost (supplementary material)\n",
    "### Evgeny M. Kadilenko (ICKC SB RAS), Roland Grinis (MIPT, GrinisRIT)\n",
    "#### Assistants: Gregory Dushkin (MIPT), Sabina Abdiganieva (MIPT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "2rd8cXFrI617GyeNAptEQ6",
     "report_properties": {},
     "type": "MD"
    }
   },
   "source": [
    "### Preparing the globals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "pC39fktFfS3knkTgR6lG6m",
     "report_properties": {},
     "type": "MD"
    }
   },
   "source": [
    "#### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "V96IjHZ3bQiFZvggM3jn6C",
     "report_properties": {},
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "import dqc\n",
    "import pyscf\n",
    "from pyscf import gto, dft, scf, cc\n",
    "import sympy\n",
    "import inspect\n",
    "import numpy\n",
    "import torch\n",
    "import xitorch as xt\n",
    "from math import pi\n",
    "import gc\n",
    "import sys \n",
    "import time\n",
    "import pandas as pd\n",
    "from IPython.display import display, HTML, Math\n",
    "import numba as nb\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import ticker\n",
    "import functools\n",
    "from ast import literal_eval\n",
    "import copy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Software specifications:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PySCF version: 2.0.1\n",
      "DQC version: 0.1.0\n"
     ]
    }
   ],
   "source": [
    "print(f'PySCF version: {pyscf.__version__}')\n",
    "print(f'DQC version: {dqc.__version__}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### LDA and PBE constants:\n",
    "We use two different *ab initio* functionals as an example.\n",
    "The first one is LDA exchange:\n",
    "$$\n",
    "\\epsilon \\left[ \\rho \\right] = a \\cdot \\rho ^ p\n",
    "$$\n",
    "The second one is GGA-based PBE: [Perdew, J. P., Burke, K., & Ernzerhof, M. (1996). Generalized gradient approximation made simple. Physical review letters, 77(18), 3865.](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.77.3865)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "LDA_factor = -3.0/4.0 * (3.0 / pi)**(1.0/3.0)\n",
    "LDA_power = 4.0/3.0\n",
    "PW_a = 0.031091\n",
    "PW_a_modified = 0.0310907\n",
    "PW_alpha = 0.21370 \n",
    "PW_beta1 = 7.5957 \n",
    "PW_beta2 = 3.5876 \n",
    "PW_beta3 = 1.6382 \n",
    "PW_beta4 = 0.49294\n",
    "PBE_kappa = 0.8040\n",
    "PBE_mu = 0.2195149727645171\n",
    "PBE_beta = 0.06672455060314922\n",
    "PBE_gamma = (1 - numpy.log(2)) / pi ** 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "LDAX_default_parameters_dict = {'LDA_factor' : LDA_factor, 'LDA_power' : LDA_power}\n",
    "PBE_default_parameters_dict = {'LDA_factor' : LDA_factor, 'LDA_power' : LDA_power,\n",
    "                               'PBE_beta' : PBE_beta,     'PBE_gamma' : PBE_gamma,\n",
    "                               'PBE_kappa' : PBE_kappa,   'PBE_mu' : PBE_mu, \n",
    "                               'PW_a' : PW_a_modified,    'PW_alpha' : PW_alpha, \n",
    "                               'PW_beta1' : PW_beta1,     'PW_beta2' : PW_beta2, \n",
    "                               'PW_beta3' : PW_beta3,     'PW_beta4' : PW_beta4}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Correctness test: energy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "tErq97jOzVSbusn9Qpi8ju",
     "report_properties": {},
     "type": "MD"
    }
   },
   "source": [
    "#### Custom functionals\n",
    "\n",
    "Here we take a look at the ways to implement an LDA and PBE functionals and compare them to some built-in methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class XCSympy():\n",
    "    @staticmethod\n",
    "    def init():\n",
    "        XCSympy.rho = sympy.Symbol('rho', real=True)\n",
    "        XCSympy.sigma = sympy.Symbol('sigma', real=True)\n",
    "        \n",
    "        XCSympy.LDA_factor = sympy.Symbol('LDA_factor', real=True)\n",
    "        XCSympy.LDA_power  = sympy.Symbol('LDA_power', real=True)\n",
    "        XCSympy.PBE_kappa  = sympy.Symbol('PBE_kappa', real=True)\n",
    "        XCSympy.PBE_mu     = sympy.Symbol('PBE_mu', real=True)\n",
    "        XCSympy.PW_a       = sympy.Symbol('PW_a', real=True)\n",
    "        XCSympy.PW_alpha   = sympy.Symbol('PW_alpha', real=True)\n",
    "        XCSympy.PW_beta1   = sympy.Symbol('PW_beta1', real=True)\n",
    "        XCSympy.PW_beta2   = sympy.Symbol('PW_beta2', real=True)\n",
    "        XCSympy.PW_beta3   = sympy.Symbol('PW_beta3', real=True)\n",
    "        XCSympy.PW_beta4   = sympy.Symbol('PW_beta4', real=True)\n",
    "        XCSympy.PBE_beta   = sympy.Symbol('PBE_beta', real=True)\n",
    "        XCSympy.PBE_gamma  = sympy.Symbol('PBE_gamma', real=True)\n",
    "        \n",
    "        XCSympy.ex_lda_sympy = XCSympy.LDA_factor * XCSympy.rho ** (XCSympy.LDA_power - 1)\n",
    "        \n",
    "        rs = XCSympy.rho ** (-1 / 3) * (3 / (4 * pi)) ** (1 / 3)\n",
    "        g_aux = XCSympy.PW_beta1 * rs ** (1 / 2) + XCSympy.PW_beta2 * rs + XCSympy.PW_beta3 * rs ** (3 / 2) + XCSympy.PW_beta4 * rs ** 2\n",
    "        XCSympy.ec_pw_sympy = -2 * XCSympy.PW_a * (1 + XCSympy.PW_alpha * rs) * sympy.log(1 + 1/(2 * XCSympy.PW_a * g_aux))\n",
    "\n",
    "        s = XCSympy.sigma ** 0.5 / (2 * (3 * pi ** 2) ** (1 / 3) * XCSympy.rho ** (4 / 3))\n",
    "        pbe_x_factor_sympy = 1 + XCSympy.PBE_kappa - XCSympy.PBE_kappa ** 2 / (XCSympy.PBE_kappa + XCSympy.PBE_mu * s ** 2)\n",
    "        XCSympy.ex_pbe_sympy = XCSympy.ex_lda_sympy * pbe_x_factor_sympy\n",
    "        \n",
    "        t = XCSympy.sigma ** 0.5 * (pi / 3) ** (1 / 6) / (4 * XCSympy.rho ** (7 / 6))\n",
    "        A = XCSympy.PBE_beta / (XCSympy.PBE_gamma * (sympy.exp(-XCSympy.ec_pw_sympy / XCSympy.PBE_gamma) - 1))\n",
    "        f1 = t ** 2 + A * t ** 4\n",
    "        f2 = XCSympy.PBE_beta * f1 / (XCSympy.PBE_gamma * (1 + A * f1))\n",
    "        fH = XCSympy.PBE_gamma * sympy.log(1 + f2)\n",
    "        XCSympy.ec_pbe_sympy = XCSympy.ec_pw_sympy + fH\n",
    "        \n",
    "        XCSympy.exc_pbe_sympy = XCSympy.ec_pbe_sympy + XCSympy.ex_pbe_sympy\n",
    "\n",
    "    @staticmethod\n",
    "    def subs(expression, params):\n",
    "        for key, value in params.items():\n",
    "            expression = expression.subs(XCSympy.__getattribute__(XCSympy, key), value)\n",
    "        return expression\n",
    "    \n",
    "XCSympy.init()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "c806BJg47HMXR3Eqrppftl",
     "report_properties": {},
     "type": "MD"
    }
   },
   "source": [
    "#### PySCF\n",
    "\n",
    "PySCF accepts custom functionals as Python functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PySCFLDAXConstructor():\n",
    "    @staticmethod\n",
    "    def parse_data(data):\n",
    "        if torch.is_tensor(data):\n",
    "            torch.nan_to_num(data)\n",
    "        else:\n",
    "            numpy.nan_to_num(data) # delete NaNs\n",
    "        data.clip(0) # delete negative numbers\n",
    "        data += 1e-15\n",
    "        \n",
    "        return data\n",
    "\n",
    "    def __init__(self, params):\n",
    "        self.family = \"LDA\"\n",
    "        self.params = params\n",
    "\n",
    "        ex_lda_sympy = XCSympy.subs(XCSympy.ex_lda_sympy, self.params)\n",
    "    \n",
    "        dexlda_wrt_drho_sympy = sympy.diff(ex_lda_sympy, XCSympy.rho)\n",
    "    \n",
    "        ex_lda_numpy = sympy.lambdify((XCSympy.rho), ex_lda_sympy, \"numpy\", cse=True)\n",
    "        ex_lda_torch = sympy.lambdify((XCSympy.rho), ex_lda_sympy, \n",
    "                                      modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "        dexlda_wrt_drho_numpy = sympy.lambdify((XCSympy.rho), dexlda_wrt_drho_sympy, \"numpy\", cse=True)\n",
    "        dexlda_wrt_drho_torch = sympy.lambdify((XCSympy.rho), dexlda_wrt_drho_sympy, \n",
    "                                               modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "\n",
    "        def eval_xc(xc_code, data, *args, **kwargs):\n",
    "            rho = PySCFLDAXConstructor.parse_data(data)\n",
    "            if torch.is_tensor(rho):\n",
    "                ex_lda = ex_lda_torch(rho)\n",
    "                dexlda_wrt_drho = dexlda_wrt_drho_torch(rho)\n",
    "            else:\n",
    "                ex_lda = ex_lda_numpy(rho)\n",
    "                dexlda_wrt_drho = dexlda_wrt_drho_numpy(rho)\n",
    "            vrho_lda = ex_lda + rho * dexlda_wrt_drho\n",
    "\n",
    "            vxc_lda = (vrho_lda, None, None, None)\n",
    "            return ex_lda, vxc_lda, None, None\n",
    "        self.eval_xc = eval_xc\n",
    "        \n",
    "        d2exc_wrt_drho2_sympy = sympy.diff(dexlda_wrt_drho_sympy, XCSympy.rho)\n",
    "        d2exc_wrt_drho2_numpy = sympy.lambdify((XCSympy.rho), d2exc_wrt_drho2_sympy, \"numpy\", cse=True)\n",
    "\n",
    "        def eval_dvxc_wrt_drho(data, *args, **kwargs):\n",
    "            rho = PySCFLDAXConstructor.parse_data(data)\n",
    "            dexlda_wrt_drho = dexlda_wrt_drho_numpy(rho)\n",
    "            d2exc_wrt_drho2 = d2exc_wrt_drho2_numpy(rho)\n",
    "            \n",
    "            return 2 * dexlda_wrt_drho + rho * d2exc_wrt_drho2\n",
    "        self.eval_dvxc_wrt_drho = eval_dvxc_wrt_drho\n",
    "\n",
    "    @staticmethod\n",
    "    def decorate_cache_rho_on_grid(qc):\n",
    "        qc.grids.build(with_non0tab=True)\n",
    "        qc.grids.rho = numpy.empty((qc.grids.coords.shape[0],))\n",
    "        qc.grids.number_of_written_rho_values = 0\n",
    "\n",
    "        def cached_gen_rho_evaluator(*args, **kargs):\n",
    "            make_rho_clear, ndms, nao = dft.numint.NumInt()._gen_rho_evaluator(*args, **kargs)\n",
    "            def make_rho_cached(idm, ao, sindex, xctype):\n",
    "                rho = make_rho_clear(idm, ao, sindex, xctype)\n",
    "\n",
    "                if qc.grids.number_of_written_rho_values == qc.grids.rho.shape[0]:\n",
    "                    qc.grids.rho = numpy.empty((qc.grids.coords.shape[0],))\n",
    "                    qc.grids.number_of_written_rho_values = 0\n",
    "\n",
    "                n = qc.grids.number_of_written_rho_values\n",
    "                qc.grids.rho[n:n+rho.shape[0]] = rho\n",
    "                qc.grids.number_of_written_rho_values += rho.shape[0]\n",
    "\n",
    "                return rho\n",
    "            return make_rho_cached, ndms, nao\n",
    "\n",
    "        qc._numint._gen_rho_evaluator = cached_gen_rho_evaluator\n",
    "    \n",
    "    def construct_derivarives_calculator(self):\n",
    "        self.names = self.params.keys()\n",
    "        \n",
    "        ex_lda_derivative_sympy_list = []\n",
    "        vx_lda_derivative_sympy_list = []\n",
    "        \n",
    "        for key in self.params.keys():\n",
    "            ex_lda_derivative_sympy = sympy.diff(XCSympy.ex_lda_sympy, XCSympy.__getattribute__(XCSympy, key))\n",
    "            ex_lda_derivative_sympy = XCSympy.rho * XCSympy.subs(ex_lda_derivative_sympy, self.params)\n",
    "            ex_lda_derivative_sympy_list.append(ex_lda_derivative_sympy)\n",
    "            \n",
    "            vx_lda_derivative_sympy = sympy.diff(ex_lda_derivative_sympy, XCSympy.rho)\n",
    "            vx_lda_derivative_sympy_list.append(vx_lda_derivative_sympy)\n",
    "        \n",
    "        ex_lda_derivative_sympy_list = sympy.Array(ex_lda_derivative_sympy_list)\n",
    "        ex_lda_derivative_numpy_list = sympy.lambdify((XCSympy.rho), ex_lda_derivative_sympy_list, \n",
    "                                                      modules=[{'exp': numpy.exp, \n",
    "                                                                'log': numpy.log}], cse=True)\n",
    "        ex_lda_derivative_numpy_list = nb.njit(ex_lda_derivative_numpy_list)\n",
    "                \n",
    "        self.derivatives_calculator = ex_lda_derivative_numpy_list\n",
    "        \n",
    "        vx_lda_derivative_sympy_list = sympy.Array(vx_lda_derivative_sympy_list)\n",
    "        vx_lda_derivative_numpy_list = sympy.lambdify((XCSympy.rho), vx_lda_derivative_sympy_list, \n",
    "                                                      modules=[{'exp': numpy.exp, \n",
    "                                                                'log': numpy.log}], cse=True)\n",
    "        vx_lda_derivative_numpy_list = nb.njit(vx_lda_derivative_numpy_list)\n",
    "                \n",
    "        self.derivatives_calculator_vx = vx_lda_derivative_numpy_list\n",
    "\n",
    "    \n",
    "    def calculate_derivatives(self, data):\n",
    "        rho = PySCFLDAXConstructor.parse_data(data)\n",
    "        return numpy.stack(self.derivatives_calculator(rho))\n",
    "    \n",
    "    def calculate_vxc_derivatives(self, data):\n",
    "        rho = PySCFLDAXConstructor.parse_data(data)\n",
    "        return numpy.stack(self.derivatives_calculator_vx(rho))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PySCFPBEConstructor():\n",
    "    @staticmethod\n",
    "    def parse_data(data):\n",
    "        if type(data) is tuple:\n",
    "            rho = data[0]\n",
    "            dx, dy, dz = data[1]\n",
    "        else:\n",
    "            rho, dx, dy, dz = data[:4]\n",
    "\n",
    "        if torch.is_tensor(rho):\n",
    "            torch.nan_to_num(rho)\n",
    "        else:\n",
    "            numpy.nan_to_num(rho) # delete NaNs\n",
    "        rho.clip(0) # delete negative numbers\n",
    "        rho += 1e-15\n",
    "\n",
    "        sigma = dx**2 + dy**2 + dz**2\n",
    "        \n",
    "        return rho, sigma, (dx, dy, dz)\n",
    "\n",
    "    def __init__(self, params):\n",
    "        self.family = \"GGA\"\n",
    "        self.params = params\n",
    "\n",
    "        exc_pbe_sympy = XCSympy.subs(XCSympy.exc_pbe_sympy, self.params)\n",
    "\n",
    "        dexcpbe_wrt_drho_sympy = sympy.diff(exc_pbe_sympy, XCSympy.rho)\n",
    "        dexcpbe_wrt_dsigma_sympy = sympy.diff(exc_pbe_sympy, XCSympy.sigma)\n",
    "\n",
    "        exc_pbe_numpy = sympy.lambdify((XCSympy.rho, XCSympy.sigma), exc_pbe_sympy, \"numpy\", cse=True)\n",
    "        dexcpbe_wrt_drho_numpy = sympy.lambdify((XCSympy.rho, XCSympy.sigma), dexcpbe_wrt_drho_sympy, \"numpy\", cse=True)\n",
    "        dexcpbe_wrt_dsigma_numpy = sympy.lambdify((XCSympy.rho, XCSympy.sigma), dexcpbe_wrt_dsigma_sympy, \"numpy\", cse=True)\n",
    "\n",
    "        exc_pbe_torch = sympy.lambdify((XCSympy.rho, XCSympy.sigma), exc_pbe_sympy, \n",
    "                                       modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "        dexcpbe_wrt_drho_torch = sympy.lambdify((XCSympy.rho, XCSympy.sigma), dexcpbe_wrt_drho_sympy, \n",
    "                                                modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "        dexcpbe_wrt_dsigma_torch = sympy.lambdify((XCSympy.rho, XCSympy.sigma), dexcpbe_wrt_dsigma_sympy, \n",
    "                                                  modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "\n",
    "        \n",
    "        def eval_xc(xc_code, data, *args, **kwargs):\n",
    "            rho, sigma, _ = PySCFPBEConstructor.parse_data(data)\n",
    "\n",
    "            if torch.is_tensor(rho):\n",
    "                exc_pbe = exc_pbe_torch(rho, sigma)\n",
    "\n",
    "                dexcpbe_wrt_drho = dexcpbe_wrt_drho_torch(rho, sigma)\n",
    "                dexcpbe_wrt_dsigma = dexcpbe_wrt_dsigma_torch(rho, sigma)\n",
    "            else:\n",
    "                exc_pbe = exc_pbe_numpy(rho, sigma)\n",
    "\n",
    "                dexcpbe_wrt_drho = dexcpbe_wrt_drho_numpy(rho, sigma)\n",
    "                dexcpbe_wrt_dsigma = dexcpbe_wrt_dsigma_numpy(rho, sigma)\n",
    "\n",
    "            vrho_pbe = exc_pbe + rho * dexcpbe_wrt_drho\n",
    "            vsigma_pbe = rho * dexcpbe_wrt_dsigma\n",
    "\n",
    "            vxc_pbe = (vrho_pbe, vsigma_pbe, None, None)\n",
    "            return exc_pbe, vxc_pbe, None, None\n",
    "        self.eval_xc = eval_xc\n",
    "\n",
    "    @staticmethod\n",
    "    def decorate_cache_rho_on_grid(qc):\n",
    "        qc.grids.build(with_non0tab=True)\n",
    "        qc.grids.rho = numpy.empty((4, qc.grids.coords.shape[0]))\n",
    "        qc.grids.number_of_written_rho_values = 0\n",
    "\n",
    "        def cached_gen_rho_evaluator(*args, **kargs):\n",
    "            make_rho_clear, ndms, nao = dft.numint.NumInt()._gen_rho_evaluator(*args, **kargs)\n",
    "            def make_rho_cached(idm, ao, sindex, xctype):\n",
    "                rho = make_rho_clear(idm, ao, sindex, xctype)\n",
    "\n",
    "                if qc.grids.number_of_written_rho_values == qc.grids.rho.shape[1]:\n",
    "                    qc.grids.rho = numpy.empty((4, qc.grids.coords.shape[0]))\n",
    "                    qc.grids.number_of_written_rho_values = 0\n",
    "\n",
    "                n = qc.grids.number_of_written_rho_values\n",
    "                qc.grids.rho[:, n:n+rho.shape[1]] = rho\n",
    "                qc.grids.number_of_written_rho_values += rho.shape[1]\n",
    "\n",
    "                return rho\n",
    "            return make_rho_cached, ndms, nao\n",
    "\n",
    "        qc._numint._gen_rho_evaluator = cached_gen_rho_evaluator\n",
    "    \n",
    "    def construct_derivarives_calculator(self):\n",
    "        self.names = self.params.keys()\n",
    "        \n",
    "        exc_pbe_derivative_sympy_list = []\n",
    "        vxc_pbe_derivative_rho_sympy_list = []\n",
    "        vxc_pbe_derivative_sigma_sympy_list = []\n",
    "        \n",
    "        for key in self.params.keys():\n",
    "            exc_pbe_derivative_sympy = sympy.diff(XCSympy.exc_pbe_sympy, XCSympy.__getattribute__(XCSympy, key))\n",
    "            exc_pbe_derivative_sympy = XCSympy.rho * XCSympy.subs(exc_pbe_derivative_sympy, self.params)\n",
    "            exc_pbe_derivative_sympy_list.append(exc_pbe_derivative_sympy)\n",
    "            \n",
    "            vxc_pbe_derivative_rho_sympy = sympy.diff(exc_pbe_derivative_sympy, XCSympy.rho)\n",
    "            vxc_pbe_derivative_rho_sympy_list.append(vxc_pbe_derivative_rho_sympy)\n",
    "            vxc_pbe_derivative_sigma_sympy = sympy.diff(exc_pbe_derivative_sympy, XCSympy.sigma)\n",
    "            vxc_pbe_derivative_sigma_sympy_list.append(vxc_pbe_derivative_sigma_sympy)\n",
    "\n",
    "        exc_pbe_derivative_sympy_list = sympy.Array(exc_pbe_derivative_sympy_list)\n",
    "        exc_pbe_derivative_numpy_list = sympy.lambdify((XCSympy.rho, XCSympy.sigma), exc_pbe_derivative_sympy_list, \n",
    "                                                       modules=[{'exp': numpy.exp, \n",
    "                                                                 'log': numpy.log}], cse=True)\n",
    "        \n",
    "        self.derivatives_calculator = nb.njit(exc_pbe_derivative_numpy_list)\n",
    "        \n",
    "        vxc_pbe_derivative_rho_sympy_list = sympy.Array(vxc_pbe_derivative_rho_sympy_list)\n",
    "        vxc_pbe_derivative_rho_numpy_list = sympy.lambdify((XCSympy.rho, XCSympy.sigma), \n",
    "                                                           vxc_pbe_derivative_rho_sympy_list, \n",
    "                                                      modules=[{'exp': numpy.exp, \n",
    "                                                                'log': numpy.log}], cse=True)\n",
    "        vxc_pbe_derivative_rho_numpy_list = nb.njit(vxc_pbe_derivative_rho_numpy_list)\n",
    "                \n",
    "        self.derivatives_calculator_vxc_rho = vxc_pbe_derivative_rho_numpy_list\n",
    "        \n",
    "        vxc_pbe_derivative_sigma_sympy_list = sympy.Array(vxc_pbe_derivative_sigma_sympy_list)\n",
    "        vxc_pbe_derivative_sigma_numpy_list = sympy.lambdify((XCSympy.rho, XCSympy.sigma), \n",
    "                                                           vxc_pbe_derivative_sigma_sympy_list, \n",
    "                                                      modules=[{'exp': numpy.exp, \n",
    "                                                                'log': numpy.log}], cse=True)\n",
    "        vxc_pbe_derivative_sigma_numpy_list = nb.njit(vxc_pbe_derivative_sigma_numpy_list)\n",
    "                \n",
    "        self.derivatives_calculator_vxc_sigma = vxc_pbe_derivative_sigma_numpy_list\n",
    "\n",
    "\n",
    "    def calculate_derivatives(self, data):\n",
    "        rho, sigma, _ = PySCFPBEConstructor.parse_data(data)\n",
    "        return numpy.stack(self.derivatives_calculator(rho, sigma))\n",
    "\n",
    "    def calculate_vxc_derivatives(self, data):\n",
    "        rho, sigma, gradient = PySCFPBEConstructor.parse_data(data)\n",
    "        gradient = numpy.stack(gradient)\n",
    "        vxc_pbe_derivative_rho = numpy.stack(self.derivatives_calculator_vxc_rho(rho, sigma))\n",
    "        vxc_pbe_derivative_sigma = numpy.stack(self.derivatives_calculator_vxc_sigma(rho, sigma))\n",
    "        vxc_pbe_derivative_gradient = 2 * numpy.einsum(\"pg,xg->pxg\", vxc_pbe_derivative_sigma, gradient)\n",
    "        return (vxc_pbe_derivative_rho, vxc_pbe_derivative_gradient)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "iLDNI8SeDIrRd1K7AqA6eB",
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "class PySCFAdapterCustom():\n",
    "    def __init__(self, XCConstructor, params, mol, basis=\"6-31G\"):\n",
    "        self.scf_mol = gto.M(atom=mol, basis=basis, unit=\"Bohr\", verbose=0)\n",
    "    \n",
    "        # Timer\n",
    "        start = time.time()\n",
    "        self.qc = dft.RKS(self.scf_mol)\n",
    "        self.xc = XCConstructor(params)\n",
    "        self.qc = self.qc.define_xc_(self.xc.eval_xc, self.xc.family)\n",
    "        \n",
    "        self.total_energy = self.qc.kernel()\n",
    "        self.duration = time.time() - start\n",
    "        self.converged = self.qc.converged\n",
    "        \n",
    "    def construct_rho_squared(self, rho_exact):\n",
    "        dm = self.qc.make_rdm1()        \n",
    "        ao_value = pyscf.dft.numint.eval_ao(self.scf_mol, self.qc.grids.coords, deriv=0)\n",
    "        rho_on_grid = pyscf.dft.numint.eval_rho(self.scf_mol, ao_value, dm, xctype='LDA')\n",
    "    \n",
    "        diff_squared = (rho_on_grid - rho_exact) ** 2\n",
    "        self.rho_squared = (self.qc.grids.weights @ diff_squared)\n",
    "\n",
    "    def construct_rho_on_grid(self, grid):\n",
    "        dm_ao = self.qc.make_rdm1(ao_repr=True)\n",
    "\n",
    "        if hasattr(grid, \"numpy\"):\n",
    "            grid = grid.numpy()\n",
    "        \n",
    "        ao_value = pyscf.dft.numint.eval_ao(self.scf_mol, grid, deriv=0)\n",
    "        return pyscf.dft.numint.eval_rho(self.scf_mol, ao_value, dm_ao, xctype='LDA')\n",
    "\n",
    "# To compare the results, we would use PySCF's built-in implementation of the same functional\n",
    "class PySCFAdapterBuiltin():\n",
    "    def __init__(self, xc_name, mol, basis=\"6-31G\"):\n",
    "        self.scf_mol = gto.M(atom=mol, basis=basis, unit=\"Bohr\", verbose=0)\n",
    "        \n",
    "        # Timer\n",
    "        start = time.time()\n",
    "        self.qc = dft.RKS(self.scf_mol)\n",
    "        self.qc.xc = xc_name # Built-in functional implementation\n",
    "        \n",
    "        self.total_energy = self.qc.kernel()\n",
    "        self.duration = time.time() - start\n",
    "        self.converged = self.qc.converged\n",
    "        \n",
    "    def construct_rho_squared(self, rho_exact):\n",
    "        dm = self.qc.make_rdm1()        \n",
    "        ao_value = pyscf.dft.numint.eval_ao(self.scf_mol, self.qc.grids.coords, deriv=0)\n",
    "        rho_on_grid = pyscf.dft.numint.eval_rho(self.scf_mol, ao_value, dm, xctype='LDA')\n",
    "    \n",
    "        diff_squared = (rho_on_grid - rho_exact) ** 2\n",
    "        self.rho_squared = (self.qc.grids.weights @ diff_squared)\n",
    "\n",
    "    def construct_rho_on_grid(self, grid):\n",
    "        dm_ao = self.qc.make_rdm1(ao_repr=True)\n",
    "\n",
    "        if hasattr(grid, \"numpy\"):\n",
    "            grid = grid.numpy()\n",
    "        \n",
    "        ao_value = pyscf.dft.numint.eval_ao(self.scf_mol, grid, deriv=0)\n",
    "        return pyscf.dft.numint.eval_rho(self.scf_mol, ao_value, dm_ao, xctype='LDA')\n",
    "\n",
    "class PySCFAdapterAuto():\n",
    "    def __init__(self, XCConstructor, params, mol, basis=\"6-31G\", decorator=True):\n",
    "        self.scf_mol = gto.M(atom=mol, basis=basis, unit=\"Bohr\", verbose=0)\n",
    "        \n",
    "        # Timer        \n",
    "        start = time.time()\n",
    "        self.qc = dft.RKS(self.scf_mol)\n",
    "        self.xc = XCConstructor(params)\n",
    "        if (XCConstructor == PySCFLDAXConstructor) and (params == LDAX_default_parameters_dict):\n",
    "            self.qc.xc = \"lda,\" # Built-in functional implementation\n",
    "        elif (XCConstructor == PySCFPBEConstructor) and (params == PBE_default_parameters_dict):\n",
    "            self.qc.xc = \"pbe,pbe\" # Built-in functional implementation\n",
    "        else:\n",
    "            self.qc = self.qc.define_xc_(self.xc.eval_xc, self.xc.family)\n",
    "            \n",
    "        if decorator:\n",
    "            XCConstructor.decorate_cache_rho_on_grid(self.qc)\n",
    "        \n",
    "        self.total_energy = self.qc.kernel()\n",
    "        self.duration = time.time() - start\n",
    "        self.converged = self.qc.converged\n",
    "\n",
    "    def construct_rho_squared(self, rho_exact):\n",
    "        dm = self.qc.make_rdm1()        \n",
    "        ao_value = pyscf.dft.numint.eval_ao(self.scf_mol, self.qc.grids.coords, deriv=0)\n",
    "        rho_on_grid = pyscf.dft.numint.eval_rho(self.scf_mol, ao_value, dm, xctype='LDA')\n",
    "    \n",
    "        diff_squared = (rho_on_grid - rho_exact) ** 2\n",
    "        self.rho_squared = (self.qc.grids.weights @ diff_squared)\n",
    "\n",
    "    def construct_rho_on_grid(self, grid):\n",
    "        dm_ao = self.qc.make_rdm1(ao_repr=True)\n",
    "\n",
    "        if hasattr(grid, \"numpy\"):\n",
    "            grid = grid.numpy()\n",
    "        \n",
    "        ao_value = pyscf.dft.numint.eval_ao(self.scf_mol, grid, deriv=0)\n",
    "        return pyscf.dft.numint.eval_rho(self.scf_mol, ao_value, dm_ao, xctype='LDA')\n",
    "        \n",
    "# CCSD(T)\n",
    "class PySCFAdapterCCSDT():\n",
    "    def __init__(self, mol, basis=\"6-31G\"):\n",
    "        self.scf_mol = gto.M(atom=mol, basis=basis, unit=\"Bohr\", verbose=0)\n",
    "        \n",
    "        start = time.time()\n",
    "        hf_ = scf.HF(self.scf_mol)\n",
    "\n",
    "        hf = hf_.run()\n",
    "        self.qc = cc.CCSD(hf)\n",
    "        ccsd_energy = self.qc.kernel()\n",
    "        triples_energy = self.qc.ccsd_t()\n",
    "        self.total_energy = self.qc.e_tot + triples_energy\n",
    "        self.duration = time.time() - start\n",
    "        self.converged = self.qc.converged\n",
    "        \n",
    "    def construct_rho_on_grid(self, grid):\n",
    "        dm_ao = self.qc.make_rdm1(ao_repr=True)\n",
    "\n",
    "        if hasattr(grid, \"numpy\"):\n",
    "            grid = grid.numpy()\n",
    "        \n",
    "        ao_value = pyscf.dft.numint.eval_ao(self.scf_mol, grid, deriv=0)\n",
    "        return pyscf.dft.numint.eval_rho(self.scf_mol, ao_value, dm_ao, xctype='LDA')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "CabErmrEgIuTlRyQ4iQU4l",
     "type": "MD"
    }
   },
   "source": [
    "#### DQC\n",
    "\n",
    "DQC accepts custom functionals as objects of classes derived from `dqc.xc.CustomXC`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DQCLDAXConstructor(dqc.xc.CustomXC):\n",
    "    def __init__(self, params):\n",
    "        super().__init__()\n",
    "        self.number_of_parameters = len(params)\n",
    "        self.params = params\n",
    "\n",
    "        self.params_sympy = [XCSympy.__getattribute__(XCSympy, param_name) for param_name in params.keys()]\n",
    "        \n",
    "        ex_lda_sympy = XCSympy.rho * XCSympy.ex_lda_sympy\n",
    "    \n",
    "        self.ex_lda_torch = sympy.lambdify((XCSympy.rho, *self.params_sympy), \n",
    "                                           ex_lda_sympy, modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "        \n",
    "    @property\n",
    "    def family(self):\n",
    "        # 1 for LDA, 2 for GGA, 4 for MGGA\n",
    "        return 1\n",
    "\n",
    "    def get_edensityxc(self, densinfo):\n",
    "        # densinfo has up and down components\n",
    "        if isinstance(densinfo, dqc.utils.SpinParam):\n",
    "            raise NotImplementedError(\"polarized case is not implemented\")\n",
    "        else:\n",
    "            rho = densinfo.value.abs() + 1e-15  # safeguarding from nan\n",
    "            return self.ex_lda_torch(rho, *self.params.values())\n",
    "\n",
    "    def get_edensityxc_derivative(self, densinfo, number_of_parameter):\n",
    "        def derivative_calculator(ex_lda_sympy, symbol):\n",
    "            derivative_sympy = sympy.diff(ex_lda_sympy, symbol)\n",
    "            derivative_torch = sympy.lambdify((XCSympy.rho, *self.params_sympy), \n",
    "                                              derivative_sympy, \n",
    "                                              modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "            return derivative_torch(rho, *self.params.values())\n",
    "\n",
    "        # densinfo has up and down components\n",
    "        if isinstance(densinfo, dqc.utils.SpinParam):\n",
    "            # spin-scaling of the exchange energy\n",
    "            raise NotImplementedError(\"polarized case is not implemented\")\n",
    "        else:\n",
    "            rho = densinfo.value.abs() + 1e-15  # safeguarding from nan\n",
    "            ex_lda_sympy = XCSympy.rho * XCSympy.ex_lda_sympy\n",
    "            return derivative_calculator(ex_lda_sympy, self.params_sympy[number_of_parameter])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DQCPBEConstructor(dqc.xc.CustomXC):\n",
    "    def __init__(self, params):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.number_of_parameters = len(params)\n",
    "        self.params = params\n",
    "\n",
    "        self.params_sympy = [XCSympy.__getattribute__(XCSympy, param_name) for param_name in params.keys()]\n",
    "        \n",
    "        exc_pbe_sympy = XCSympy.rho * XCSympy.exc_pbe_sympy\n",
    "                \n",
    "        self.exc_pbe_torch = sympy.lambdify((XCSympy.rho, XCSympy.sigma, *self.params_sympy), exc_pbe_sympy, \n",
    "                                            modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "\n",
    "    @property\n",
    "    def family(self):\n",
    "        # 1 for LDA, 2 for GGA, 4 for MGGA\n",
    "        return 2\n",
    "\n",
    "    def get_edensityxc(self, densinfo):\n",
    "        # densinfo has up and down components\n",
    "        if isinstance(densinfo, dqc.utils.SpinParam):\n",
    "            raise NotImplementedError(\"polarized case is not implemented\")\n",
    "        else:\n",
    "            rho = densinfo.value.abs() + 1e-15  # safeguarding from nan\n",
    "            sigma = densinfo.grad[0]**2 + densinfo.grad[1]**2 + densinfo.grad[2]**2 + 1e-15  # safeguarding from nan\n",
    "            \n",
    "            return self.exc_pbe_torch(rho, sigma, *self.params.values())\n",
    "\n",
    "    def get_edensityxc_derivative(self, densinfo, number_of_parameter):\n",
    "        def derivative_calculator(exc_pbe_sympy, symbol):\n",
    "            derivative_sympy = sympy.diff(exc_pbe_sympy, symbol)\n",
    "            derivative_torch = sympy.lambdify((XCSympy.rho, XCSympy.sigma, *self.params_sympy), \n",
    "                                              derivative_sympy, \n",
    "                                              modules={'log': torch.log, 'exp': torch.exp}, cse=True)\n",
    "            return derivative_torch(rho, sigma, *self.params.values())\n",
    "\n",
    "        # densinfo has up and down components\n",
    "        if isinstance(densinfo, dqc.utils.SpinParam):\n",
    "            raise NotImplementedError(\"polarized case is not implemented\")\n",
    "        else:\n",
    "            rho = densinfo.value.abs() + 1e-15  # safeguarding from nan\n",
    "            sigma = densinfo.grad[0]**2 + densinfo.grad[1]**2 + densinfo.grad[2]**2 + 1e-15  # safeguarding from nan\n",
    "            exc_pbe_sympy = XCSympy.rho * XCSympy.exc_pbe_sympy\n",
    "            return derivative_calculator(exc_pbe_sympy, self.params_sympy[number_of_parameter])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "ZXuDyJAUqa73YFbnXcLJaF",
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "class DQCAdapterWithgrad():\n",
    "    def __init__(self, XCConstructor, params, mol, basis=\"6-31G\"):\n",
    "        atmozs, atomposs = dqc.parse_moldesc(mol)\n",
    "        dqc_mol = dqc.Mol((atmozs, atomposs), basis=basis)\n",
    "\n",
    "        start = time.time()\n",
    "        self.params = torch.nn.ParameterDict({k : torch.nn.Parameter(torch.tensor(v, dtype=torch.double)) for k, v in params.items()})\n",
    "        self.qc = dqc.KS(dqc_mol, xc=XCConstructor(self.params)).run()\n",
    "\n",
    "        self.total_energy = self.qc.energy()\n",
    "        self.converged = True\n",
    "        self.duration = time.time() - start\n",
    "    \n",
    "    def construct_rho_squared(self, rho_exact):\n",
    "        densinfo = SpinParam.apply_fcn(lambda dm_: self.qc.get_system().get_hamiltonian()._dm2densinfo(dm_), \n",
    "                                    self.qc._dm).value\n",
    "        diff_squared = (densinfo - torch.from_numpy(rho_exact)).pow(2)\n",
    "        self.rho_squared = self.qc.get_system().get_hamiltonian().dvolume @ diff_squared\n",
    "        self.rho_squared.requires_grad_()\n",
    "        \n",
    "class DQCAdapterNograd():\n",
    "    def __init__(self, XCConstructor, params, mol, basis=\"6-31G\"):\n",
    "        with torch.no_grad():\n",
    "            atmozs, atomposs = dqc.parse_moldesc(mol)\n",
    "            dqc_mol = dqc.Mol((atmozs, atomposs), basis=basis)\n",
    "\n",
    "            start = time.time()\n",
    "            self.params = torch.nn.ParameterDict({k : torch.nn.Parameter(torch.tensor(v, dtype=torch.double)) for k, v in params.items()})\n",
    "            self.qc = dqc.KS(dqc_mol, xc=XCConstructor(self.params)).run()\n",
    "            \n",
    "            self.total_energy = self.qc.energy().numpy()\n",
    "            self.converged = True\n",
    "            self.duration = time.time() - start\n",
    "            \n",
    "    def construct_rho_squared(self, rho_exact):\n",
    "        with torch.no_grad():\n",
    "            densinfo = SpinParam.apply_fcn(lambda dm_: self.qc.get_system().get_hamiltonian()._dm2densinfo(dm_), \n",
    "                                        self.qc._dm).value\n",
    "            diff_squared = (densinfo - rho_exact).pow(2)\n",
    "            self.rho_squared = (self.qc.get_system().get_hamiltonian().dvolume @ diff_squared).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "CSjHTRCB8P0ZifrjITux2s",
     "type": "MD"
    }
   },
   "source": [
    "#### Comparison test\n",
    "\n",
    "Let's compare all of the above methods (also, for convenience, all of the results are compiled at the end of the notebook in tables)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "lESczcDxOmGdqByoPm9dzW",
     "report_properties": {},
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "molecules_for_correctness_tests = {\n",
    "    \"h2\"       :  \"\"\"\n",
    "                  H 0 0 0; H 1.4 0 0\n",
    "                  \"\"\"\n",
    "    ,\n",
    "    \"water\"    : \"\"\"\n",
    "                 O        0.000000000      0.000000000      0.000000000;\n",
    "                 H        0.000000000      1.434938863      1.126357947;\n",
    "                 H        0.000000000     -1.434938863      1.12635794\n",
    "                 \"\"\"\n",
    "     ,\n",
    "    \"ethane\": \"\"\"\n",
    "                 C       -0.570532088      0.000000000     -0.925021293;\n",
    "                 H       -0.570532088      1.019690000     -1.324277293;\n",
    "                 H        0.312545912     -0.509845000     -1.324277293;\n",
    "                 H       -1.453610088     -0.509845000     -1.324277293;\n",
    "                 C       -0.570532088      0.000000000      0.604966707;\n",
    "                 H       -1.453610088      0.509845000      1.004222707;\n",
    "                 H       -0.570532088     -1.019690000      1.004222707;\n",
    "                 H        0.312545912      0.509845000      1.004222707\n",
    "              \"\"\"\n",
    "    ,\n",
    "    \"benzene\" : \"\"\"\n",
    "                C        0.598362921      0.000000000     -4.742986733;\n",
    "                C       -0.557705772     -0.354690359     -4.044822733;\n",
    "                C        1.754431614      0.354690359     -4.044822733;\n",
    "                H       -1.457130878     -0.630640582     -4.587995733;\n",
    "                H        2.653856720      0.630640582     -4.587995733;\n",
    "                C       -0.557705772     -0.354690359     -2.648492733;\n",
    "                C        1.754431614      0.354690359     -2.648492733;\n",
    "                H       -1.457130878     -0.630640582     -2.105319733;\n",
    "                H        2.653856720      0.630640582     -2.105319733;\n",
    "                C        0.598362921      0.000000000     -1.950328733;\n",
    "                H        0.598362921      0.000000000     -0.863981733;\n",
    "                H        0.598362921      0.000000000     -5.829333733\n",
    "                \"\"\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "5GuoHEv6WP9UDWCOUkmyGF",
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "energy_correctness_results = {\n",
    "    \"pyscf_builtin_lda\":    {},\n",
    "    \"pyscf_custom_lda\":     {},\n",
    "    \"pyscf_builtin_pbe\":    {},\n",
    "    \"pyscf_custom_pbe\":     {},\n",
    "    \"pyscf_ccsdt\":      {},\n",
    "    \"dqc_nograd_lda\":       {},\n",
    "    \"dqc_nograd_pbe\":       {},\n",
    "}\n",
    "\n",
    "for data_of_method in energy_correctness_results.values():\n",
    "    for mol_name in molecules_for_correctness_tests:\n",
    "        data_of_method[mol_name] = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "UY1XdShEAtxTvCwwmS7Jnx",
     "type": "CODE"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For h2 molecule and pyscf_ccsdt method: energy=-1.151679166076766 Eh, duration=0.053429365158081055 s\n",
      "For h2 molecule and pyscf_builtin_lda method: energy=-1.0386177856738814 Eh, duration=0.07524538040161133 s\n",
      "For h2 molecule and pyscf_custom_lda method: energy=-1.0386177856739316 Eh, duration=0.22424840927124023 s\n",
      "For h2 molecule and dqc_nograd_lda method: energy=-1.0386170699228439 Eh, duration=0.07260942459106445 s\n",
      "For h2 molecule and pyscf_builtin_pbe method: energy=-1.1619157577671007 Eh, duration=0.36080265045166016 s\n",
      "For h2 molecule and pyscf_custom_pbe method: energy=-1.16191575776717 Eh, duration=0.7622675895690918 s\n",
      "For h2 molecule and dqc_nograd_pbe method: energy=-1.1619148796560532 Eh, duration=0.19274139404296875 s\n",
      "---><---\n",
      "For water molecule and pyscf_ccsdt method: energy=-76.12064120720747 Eh, duration=0.0933375358581543 s\n",
      "For water molecule and pyscf_builtin_lda method: energy=-75.15470534025629 Eh, duration=0.10269498825073242 s\n",
      "For water molecule and pyscf_custom_lda method: energy=-75.15470534025643 Eh, duration=0.31499457359313965 s\n",
      "For water molecule and dqc_nograd_lda method: energy=-75.15469465302144 Eh, duration=0.16628503799438477 s\n",
      "For water molecule and pyscf_builtin_pbe method: energy=-76.29864670374658 Eh, duration=0.29820680618286133 s\n",
      "For water molecule and pyscf_custom_pbe method: energy=-76.29864670374668 Eh, duration=0.9797279834747314 s\n",
      "For water molecule and dqc_nograd_pbe method: energy=-76.29863515616559 Eh, duration=0.34073781967163086 s\n",
      "---><---\n",
      "For ethane molecule and pyscf_ccsdt method: energy=-74.72850157871227 Eh, duration=1.0693929195404053 s\n",
      "For ethane molecule and pyscf_builtin_lda method: energy=-73.23297713780947 Eh, duration=0.8192975521087646 s\n",
      "For ethane molecule and pyscf_custom_lda method: energy=-73.2329771378095 Eh, duration=0.8914637565612793 s\n",
      "For ethane molecule and dqc_nograd_lda method: energy=-73.23296447310341 Eh, duration=0.4466390609741211 s\n",
      "For ethane molecule and pyscf_builtin_pbe method: energy=-75.0796012544295 Eh, duration=1.2871887683868408 s\n",
      "For ethane molecule and pyscf_custom_pbe method: energy=-75.07960125442933 Eh, duration=1.7487683296203613 s\n",
      "For ethane molecule and dqc_nograd_pbe method: energy=-75.07958567449117 Eh, duration=1.3088188171386719 s\n",
      "---><---\n",
      "For benzene molecule and pyscf_ccsdt method: energy=-217.07797043621997 Eh, duration=9.228793144226074 s\n",
      "For benzene molecule and pyscf_builtin_lda method: energy=-213.4146842159044 Eh, duration=2.378046751022339 s\n",
      "For benzene molecule and pyscf_custom_lda method: energy=-213.41468421590793 Eh, duration=1.9744360446929932 s\n",
      "For benzene molecule and dqc_nograd_lda method: energy=-213.41430597685354 Eh, duration=2.33200740814209 s\n",
      "For benzene molecule and pyscf_builtin_pbe method: energy=-218.0955795467118 Eh, duration=3.636232614517212 s\n",
      "For benzene molecule and pyscf_custom_pbe method: energy=-218.0955795466976 Eh, duration=4.14870285987854 s\n",
      "For benzene molecule and dqc_nograd_pbe method: energy=-218.09528854425326 Eh, duration=4.828110456466675 s\n",
      "---><---\n"
     ]
    }
   ],
   "source": [
    "def print_results_energy(mol_name, method_name, e, c, d):\n",
    "    print(f\"For {mol_name} molecule and {method_name} method: energy={e} Eh, duration={d} s\")\n",
    "    if not c:\n",
    "        print(f\"Calculation for {mol_name} and method {method_name} did not converge\", file=sys.stderr)\n",
    "\n",
    "def print_and_save_results_energy(mol_name, method_name, e, c, d):\n",
    "    print_results_energy(mol_name, method_name, e, c, d)\n",
    "    energy_correctness_results[method_name][mol_name][\"energy\"] = e\n",
    "    energy_correctness_results[method_name][mol_name][\"converged\"] = c\n",
    "    energy_correctness_results[method_name][mol_name][\"time\"] = d\n",
    "\n",
    "for mol_name, mol in molecules_for_correctness_tests.items():\n",
    "    calc = PySCFAdapterCCSDT(mol)\n",
    "    print_and_save_results_energy(mol_name, \"pyscf_ccsdt\", calc.total_energy, calc.converged, calc.duration)\n",
    "    \n",
    "    calc = PySCFAdapterBuiltin(\"lda,\", mol)\n",
    "    print_and_save_results_energy(mol_name, \"pyscf_builtin_lda\", calc.total_energy, calc.converged, calc.duration)\n",
    "\n",
    "    calc = PySCFAdapterCustom(PySCFLDAXConstructor, LDAX_default_parameters_dict, mol)\n",
    "    print_and_save_results_energy(mol_name, \"pyscf_custom_lda\", calc.total_energy, calc.converged, calc.duration)\n",
    "\n",
    "    calc = DQCAdapterNograd(DQCLDAXConstructor, LDAX_default_parameters_dict, mol)\n",
    "    print_and_save_results_energy(mol_name, \"dqc_nograd_lda\", calc.total_energy, calc.converged, calc.duration)\n",
    "    \n",
    "    calc = PySCFAdapterBuiltin(\"pbe,pbe\", mol)\n",
    "    print_and_save_results_energy(mol_name, \"pyscf_builtin_pbe\", calc.total_energy, calc.converged, calc.duration)\n",
    "\n",
    "    calc = PySCFAdapterCustom(PySCFPBEConstructor, PBE_default_parameters_dict, mol)\n",
    "    print_and_save_results_energy(mol_name, \"pyscf_custom_pbe\", calc.total_energy, calc.converged, calc.duration)\n",
    "        \n",
    "    calc = DQCAdapterNograd(DQCPBEConstructor, PBE_default_parameters_dict, mol)\n",
    "    print_and_save_results_energy(mol_name, \"dqc_nograd_pbe\", calc.total_energy, calc.converged, calc.duration)\n",
    "    \n",
    "    print(\"---><---\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results of calculations with custom LDA functionals are in excellent agreement with the results of calculations with builtin LDA functional."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": false,
     "hide_output_from_viewers": false,
     "node_id": "oqSHKSEWrAT2qsYRaJyONL",
     "report_properties": {},
     "type": "MD"
    }
   },
   "source": [
    "### Existing methods for calculation of energy derivatives.\n",
    "\n",
    "The basic way to compute energy derivatives is to use finite differences. DQC also allows to make use of PyTorch's `autograd`. We will also explore the implicit differentiation method.\n",
    "\n",
    "From now on, all of the calculations will rely on user defined functionals."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "ZjOuChfOcdb0cfKcmfVCns",
     "type": "MD"
    }
   },
   "source": [
    "### Finite differences\n",
    "\n",
    "Calculating derivatives via finite difference is straighforward once there is a way to calculate energy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "ZS5Qm9R8TG3ohxZlQrEtW5",
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "def FiniteDifferenceTotalDerivativeCalculator(adapter_class, XCConstructor, params, getter, mol, \n",
    "                                              bump_factor=0.00001, basis=\"6-31G\"):\n",
    "    c = True\n",
    "    total_duration = 0\n",
    "    \n",
    "    calc = adapter_class(XCConstructor, params, mol, basis=basis)\n",
    "    value = getter(calc)\n",
    "    total_duration += calc.duration\n",
    "    c &= calc.converged\n",
    "    \n",
    "    derivatives = {}\n",
    "    \n",
    "    for key, param in params.items():\n",
    "        params_shifted = copy.deepcopy(params)\n",
    "\n",
    "        param_shifted = param * bump_factor\n",
    "        params_shifted[key] += param_shifted\n",
    "        \n",
    "        calc_shifted = adapter_class(XCConstructor, params_shifted, mol, basis=basis)\n",
    "        total_duration += calc_shifted.duration\n",
    "        c &= calc_shifted.converged\n",
    "        \n",
    "        value_shifted = getter(calc_shifted)\n",
    "        derivative = (value_shifted - value) / param_shifted\n",
    "        \n",
    "        derivatives[key] = derivative\n",
    "\n",
    "    if not c:\n",
    "        print(f\"One of finite difference computations for {mol} didn't converge with {method}\", file=sys.stderr)\n",
    "\n",
    "    return numpy.stack(list(derivatives.values())), total_duration, c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "qrTQdCyZr8mHVAKz5Dn9tM",
     "type": "MD"
    }
   },
   "source": [
    "### Iterative `torch.autograd`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "class IterativeAutogradTotalDerivativeCalculator():\n",
    "    def __init__(self, adapter):\n",
    "        self.adapter = adapter\n",
    "    \n",
    "    def construct_total_dE_wrt_dtheta(self, retain_graph=False):    \n",
    "        energy_tensor = self.adapter.total_energy\n",
    "        self.total_dE_wrt_dtheta = torch.stack(\n",
    "            torch.autograd.grad(energy_tensor, \n",
    "                                self.adapter.params.values(),\n",
    "                                retain_graph=retain_graph)).numpy()\n",
    "        \n",
    "    def construct_total_drho_squared_wrt_dtheta(self, rho_exact, retain_graph=False):\n",
    "        self.adapter.construct_rho_squared(rho_exact)\n",
    "        rho_squared = self.adapter.rho_squared\n",
    "\n",
    "        self.total_drho_squared_wrt_dtheta = torch.stack(\n",
    "            torch.autograd.grad(rho_squared, \n",
    "                                self.adapter.params.values(), \n",
    "                                retain_graph=retain_graph)).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "jE7QhzsPOXSZSHgKohPYF0",
     "type": "MD"
    }
   },
   "source": [
    "### New method: Implicit differentiation method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "fDE7wdj18sEXBFfslxXZRt",
     "type": "MD"
    }
   },
   "source": [
    "#### Full algorithm in PySCF and DQC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "Lm4lw1RuKp9OVUYRfwqBts",
     "type": "MD"
    }
   },
   "source": [
    "#### Preparing for calculation\n",
    "\n",
    "Initialize the system, PySCF, DQC, and set up some variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dqc.utils.datastruct import SpinParam, ValGrad\n",
    "\n",
    "class DQCDataAdapter(DQCAdapterNograd):\n",
    "    def __init__(self, *args, **kargs):\n",
    "        \n",
    "        super().__init__(*args, **kargs)\n",
    "        \n",
    "        self.noccorb = self.qc._engine.hf_engine._norb\n",
    "        self.nallorb = self.qc.get_system().get_hamiltonian().nao\n",
    "        self.orboccs = self.qc._engine.orb_weight.detach().numpy()\n",
    "        self.basis = (self.qc.get_system().get_hamiltonian().basis @ self.qc.get_system().get_hamiltonian()._orthozer._orthozer).numpy()\n",
    "        self.grid_size = self.basis.shape[0]\n",
    "        self.grid_volume = self.qc.get_system().get_hamiltonian().dvolume.detach().numpy()\n",
    "        self.grid_coords = self.qc.get_system().get_grid().get_rgrid()\n",
    "        self.number_of_parameters = self.qc._engine.hamilton.xc.number_of_parameters\n",
    "        \n",
    "        self.overlap = numpy.eye(self.nallorb)\n",
    "                        \n",
    "        self.dm = self.qc._dm\n",
    "        self.fockian = self.qc._engine.dm2scp(self.dm).detach()\n",
    "        self.energies, self.coefficients = self._eigendecomposition(self.fockian)\n",
    "        self._densinfo = SpinParam.apply_fcn(lambda dm_: self.qc.get_system().get_hamiltonian()._dm2densinfo(dm_), self.dm)\n",
    "        \n",
    "        self.fockian = self.fockian.numpy()\n",
    "        self.energies = self.energies.numpy()\n",
    "        self.coefficients = self.coefficients.numpy()\n",
    "        \n",
    "    def get_rho(self):\n",
    "        return self._densinfo.value.numpy()\n",
    "                    \n",
    "    def _eigendecomposition(self, fockian):\n",
    "        def _symm(scp):\n",
    "            # forcely symmetrize the tensor\n",
    "            return (scp + scp.transpose(-2, -1)) * 0.5\n",
    "\n",
    "        if not self.qc._engine.hf_engine._polarized:\n",
    "            fock = xt.LinearOperator.m(_symm(fockian), is_hermitian=True)\n",
    "            return self.qc._engine.hf_engine.diagonalize(fock, self.noccorb)\n",
    "        else:\n",
    "            raise NotImplementedError(\"polarized case is not implemented\")\n",
    "            \n",
    "    def contruct_fockian_from_dm(self, dm):\n",
    "        return self.qc._engine.dm2scp(dm)\n",
    "    \n",
    "    def construct_dexc_wrt_dtheta_on_grid(self):\n",
    "        list_of_tensors = []\n",
    "        for parameter_number in range(self.number_of_parameters):\n",
    "            current_tensor = self.qc.get_system().get_hamiltonian().xc.get_edensityxc_derivative(self._densinfo, parameter_number).detach()\n",
    "            list_of_tensors.append(current_tensor.unsqueeze(-1))\n",
    "        return torch.cat(list_of_tensors, 1).t().numpy()\n",
    "    \n",
    "    def get_elrep_tensor(self):\n",
    "        return self.qc._engine.hamilton.el_mat.detach().clone().numpy()\n",
    "    \n",
    "    def get_dvxc_wrt_drho_xc_on_grid(self):\n",
    "        # mark the densinfo components as requiring grads\n",
    "        with self.qc.get_system().get_hamiltonian().xc._enable_grad_densinfo(self._densinfo):\n",
    "            with torch.enable_grad():\n",
    "                edensity = self.qc.get_system().get_hamiltonian().xc.get_edensityxc(self._densinfo)\n",
    "            grad_outputs = torch.ones_like(edensity)\n",
    "            grad_enabled = torch.is_grad_enabled()\n",
    "\n",
    "            if not isinstance(self._densinfo, ValGrad):  # polarized case\n",
    "                raise NotImplementedError(\"polarized case is not implemented\")\n",
    "            else:  # unpolarized case\n",
    "                if self.qc.get_system().get_hamiltonian().xc.family == 1:  # LDA\n",
    "                    # Here we are taking autograd.grad derivative twice:\n",
    "                    potinfo, = torch.autograd.grad(\n",
    "                        edensity, self._densinfo.value, create_graph=grad_enabled,\n",
    "                        grad_outputs=grad_outputs)\n",
    "\n",
    "                    derivative_of_potinfo_wrt_rho, = torch.autograd.grad(\n",
    "                        potinfo, self._densinfo.value, grad_outputs=grad_outputs)\n",
    "                    return derivative_of_potinfo_wrt_rho\n",
    "                else:  # GGA and others\n",
    "                    raise NotImplementedError(\"Default dvxc wrt drho for this family is not implemented\")\n",
    "                    \n",
    "    def _get_dvxc_wrt_dtheta_on_grid(self, parameter_number):\n",
    "        # mark the densinfo components as requiring grads\n",
    "        with self.qc.get_system().get_hamiltonian().xc._enable_grad_densinfo(self._densinfo):\n",
    "            with torch.enable_grad():\n",
    "                edensity_derivative = self.qc.get_system().get_hamiltonian().xc.get_edensityxc_derivative(self._densinfo, parameter_number)\n",
    "            grad_outputs = torch.ones_like(edensity_derivative)\n",
    "            grad_enabled = torch.is_grad_enabled()\n",
    "\n",
    "            if not isinstance(self._densinfo, ValGrad):  # polarized case\n",
    "                raise NotImplementedError(\"polarized case is not implemented\")\n",
    "            else:  # unpolarized case\n",
    "                if self.qc.get_system().get_hamiltonian().xc.family == 1:  # LDA\n",
    "                    dedn, = torch.autograd.grad(\n",
    "                        edensity_derivative, self._densinfo.value, create_graph=grad_enabled,\n",
    "                        grad_outputs=grad_outputs)\n",
    "                    return ValGrad(value=dedn)\n",
    "                elif self.qc.get_system().get_hamiltonian().xc.family == 2: # GGA\n",
    "                    dedn, dedg = torch.autograd.grad(\n",
    "                        edensity_derivative, (self._densinfo.value, self._densinfo.grad), create_graph=grad_enabled,\n",
    "                        grad_outputs=grad_outputs)\n",
    "                    return ValGrad(value=dedn, grad=dedg)\n",
    "                else:  # MGGA and others\n",
    "                    raise NotImplementedError(\"Default dvxc wrt dtheta for this family is not implemented\")\n",
    "                    \n",
    "    def contruct_dvxc_wrt_dtheta(self, parameter_number):\n",
    "        potinfo = self._get_dvxc_wrt_dtheta_on_grid(parameter_number)  # value: (*BD, nr)\n",
    "        vxc_linop = self.qc.get_system().get_hamiltonian()._get_vxc_from_potinfo(potinfo)\n",
    "        return vxc_linop.fullmatrix().detach().numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PySCFDataAdapter(PySCFAdapterAuto):\n",
    "    def __init__(self, *args, **kargs):\n",
    "        super().__init__(*args, **kargs)\n",
    "        \n",
    "        self.nallorb = self.qc.mo_occ.shape[0]\n",
    "        self.orboccs = self.qc.mo_occ[self.qc.mo_occ != 0]\n",
    "        self.noccorb = self.orboccs.shape[0]\n",
    "\n",
    "        self.dm = self.qc.make_rdm1()\n",
    "        \n",
    "        self.fockian = self.qc.get_fock()\n",
    "\n",
    "        self.coefficients = self.qc.mo_coeff[:,:self.noccorb]\n",
    "        self.energies = self.qc.mo_energy[:self.noccorb]\n",
    "        \n",
    "        self.overlap = self.qc.get_ovlp()\n",
    "        \n",
    "        self.basis = self.qc.mol.eval_gto('GTOval', self.qc.grids.coords)\n",
    "        if self.xc.family == 'GGA':\n",
    "            self.basis_gradient = self.qc.mol.eval_gto('GTOval_ip', self.qc.grids.coords)\n",
    "        self.grid_size = self.qc.grids.weights.shape[0]\n",
    "        \n",
    "        self.number_of_parameters = len(self.xc.params)\n",
    "        \n",
    "        self.grid_coords = self.qc.grids.coords\n",
    "        self.grid_volume = self.qc.grids.weights\n",
    "        self._densinfo = self.qc.grids.rho\n",
    "        \n",
    "        self.xc.construct_derivarives_calculator()\n",
    "\n",
    "    def _eigendecomposition(self, fockian):\n",
    "        return numpy.linalg.eig(fockian)\n",
    "    \n",
    "    def get_elrep_tensor(self):\n",
    "        return self.qc.mol.intor('int2e')\n",
    "        \n",
    "    def get_rho(self):\n",
    "        if self.xc.family == 'LDA':\n",
    "            return self._densinfo\n",
    "        elif self.xc.family == 'GGA':\n",
    "            return self._densinfo[0]\n",
    "\n",
    "    def construct_dexc_wrt_dtheta_on_grid(self):\n",
    "        return self.xc.calculate_derivatives(self.qc.grids.rho)\n",
    "    \n",
    "    def _get_dvxc_wrt_dtheta_on_grid(self, parameter_number):\n",
    "        if self.xc.family == 'LDA':\n",
    "            return self.xc.calculate_vxc_derivatives(self.qc.grids.rho)[parameter_number]\n",
    "        elif self.xc.family == 'GGA':\n",
    "            return (self.xc.calculate_vxc_derivatives(self.qc.grids.rho)[0][parameter_number],\n",
    "                   self.xc.calculate_vxc_derivatives(self.qc.grids.rho)[1][parameter_number])\n",
    "    \n",
    "    def contruct_dvxc_wrt_dtheta(self, parameter_number):\n",
    "        if self.xc.family == 'LDA':\n",
    "            return numpy.einsum(\"g,g,gi,gj->ij\", self.grid_volume, self._get_dvxc_wrt_dtheta_on_grid(parameter_number), self.basis, self.basis)\n",
    "        elif self.xc.family == 'GGA':\n",
    "            answer = numpy.einsum(\"g,g,gi,gj->ij\", self.grid_volume, self._get_dvxc_wrt_dtheta_on_grid(parameter_number)[0], self.basis, self.basis)\n",
    "            answer_grad_ = numpy.einsum(\"g,xg,gi,xgj->ij\", self.grid_volume, self._get_dvxc_wrt_dtheta_on_grid(parameter_number)[1], self.basis, self.basis_gradient)\n",
    "            answer += (answer_grad_ + numpy.transpose(answer_grad_))\n",
    "            return answer\n",
    "\n",
    "    def get_dvxc_wrt_drho_xc_on_grid(self):\n",
    "        return self.xc.eval_dvxc_wrt_drho(self._densinfo)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "class AnalyticalPartialDerivativeCalculator():\n",
    "    def __init__(self, adapter):\n",
    "        self.adapter = adapter\n",
    "\n",
    "        self.noccorb = self.adapter.noccorb\n",
    "        self.nallorb = self.adapter.nallorb\n",
    "\n",
    "        self.fockian = self.adapter.fockian\n",
    "        self.energies = self.adapter.energies\n",
    "        self.coefficients = self.adapter.coefficients\n",
    "        self.orboccs = self.adapter.orboccs\n",
    "        \n",
    "        self.overlap = self.adapter.overlap\n",
    "        \n",
    "        self.basis = self.adapter.basis\n",
    "        self.grid_size = self.adapter.grid_size\n",
    "        self.grid_volume = self.adapter.grid_volume\n",
    "        \n",
    "        self.number_of_parameters = self.adapter.number_of_parameters\n",
    "\n",
    "    '''\n",
    "    Only canonical (i.e. eigenfunctions of fockian) orbitals can be used:\n",
    "    \\frac{\\partial E[\\rho](\\vec{\\theta})}{\\partial C_{bj}} = 2f_b \\sum_i C_{bi}F_{ij} = 2f_b \\epsilon_b C_{bj}\n",
    "    so\n",
    "    \\frac{\\partial E[\\rho](\\vec{\\theta})}{\\partial \\textbf{C}} = 2\\textbf{f}\\epsilon\\textbf{C}\n",
    "    \\epsilon here is matrix: \\epsilon_{ab}=\\delta_{ab}\\epsilon_a\n",
    "    '''\n",
    "    def construct_dE_wrt_dC(self):\n",
    "        self.dE_wrt_dC = 2 * numpy.einsum(\"b,ij,jb,b->bi\", self.orboccs, self.overlap, self.coefficients, self.energies)\n",
    "        \n",
    "    '''\n",
    "    \\frac{\\partial E[\\rho](\\vec{\\theta})}{\\partial \\epsilon_{b}} = 0 for all \\epsilon_b. So:    \n",
    "    '''\n",
    "    def construct_dE_wrt_depsilon(self):\n",
    "        self.dE_wrt_depsilon = numpy.zeros(self.noccorb)\n",
    "    \n",
    "    def construct_dE_wrt_dX(self):\n",
    "        self.construct_dE_wrt_dC()\n",
    "        self.construct_dE_wrt_depsilon()\n",
    "        self.dE_wrt_dX = numpy.concatenate((self.dE_wrt_dC, numpy.expand_dims(self.dE_wrt_depsilon, -1)), 1).transpose().reshape(-1, )\n",
    "    \n",
    "    '''\n",
    "    G_{Cou, ijkl} = \\iint b_i(\\vec{r}) \\frac{b_k(\\vec{r'})b_l(\\vec{r'})}{|\\vec{r}-\\vec{r'}|} b_j(\\vec{r})d\\vec{r'}d\\vec{r}\n",
    "    G_{XC, ijkl} = \n",
    "    \\int b_i(\\vec{r}) b_k(\\vec{r}) \\frac{\\partial V_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta})}{\\partial \\rho(\\vec{r})}b_l(\\vec{r})b_j(\\vec{r})d\\vec{r}\n",
    "    \\frac{\\partial \\Phi^{KS}_{ia}(\\textbf{C};\\;\\vec{\\theta})}{\\partial C_{ck}} = \n",
    "    (F_{ik}[\\rho](\\vec{\\theta}) - \\epsilon_c\\delta_{ik})\\delta_{ac} +\n",
    "    2f_c\\sum_j \\sum_{l}C_{cl}\\left(G_{Cou,ijkl} + G_{XC,ijkl}\\right)C_{aj}\n",
    "    '''    \n",
    "    def construct_dKS_wrt_dC(self):\n",
    "        occupied_orbitals_kronecker = numpy.eye(self.noccorb)\n",
    "        dKS_wrt_dC_first_term = numpy.einsum(\"ik,ac->iakc\", self.fockian, occupied_orbitals_kronecker)\n",
    "        dKS_wrt_dC_first_term -= numpy.einsum(\"c,ik,ac->iakc\", self.energies,\n",
    "                                             self.overlap, occupied_orbitals_kronecker)\n",
    "        \n",
    "        @nb.njit(parallel=True)\n",
    "        def _four_term_integral(m, b, o):\n",
    "            N = b.shape[1]  # size of basis set\n",
    "            n = o.shape[1]  # number of occupied orbitals\n",
    "            R = m.shape[0]  # size of grid\n",
    "            res = numpy.zeros((N, n, N, n))\n",
    "            for i in nb.prange(N):\n",
    "                for a in range(n):\n",
    "                    for k in range(i+1):\n",
    "                        for c in range(a+1):\n",
    "                            for r in range(R):\n",
    "                                value = m[r] * b[r, i] * o[r, a] * b[r, k] * o[r, c]\n",
    "                                res[i, a, k, c] += value\n",
    "                                if a != c:\n",
    "                                    res[i, c, k, a] += value\n",
    "                                    if i != k:\n",
    "                                        res[k, a, i, c] += value\n",
    "                                        res[k, c, i, a] += value\n",
    "                                elif i != k:\n",
    "                                    res[k, a, i, c] += value\n",
    "            return res\n",
    "        \n",
    "        def get_dvxc_wrt_drho_from_derivative_of_potinfo_wrt_rho(derivative_of_potinfo_wrt_rho):\n",
    "            print(f'started the four term integral computation for {self.grid_size}...')\n",
    "            molecular_orbitals_at_grid = self.basis @ self.coefficients\n",
    "            derivative_of_potinfo_wrt_rho_dvolume = derivative_of_potinfo_wrt_rho * self.grid_volume\n",
    "            \n",
    "            if hasattr(derivative_of_potinfo_wrt_rho_dvolume, \"numpy\"):\n",
    "                derivative_of_potinfo_wrt_rho_dvolume = derivative_of_potinfo_wrt_rho_dvolume.numpy()\n",
    "            \n",
    "            mat = _four_term_integral(derivative_of_potinfo_wrt_rho_dvolume,\n",
    "                                        self.basis,\n",
    "                                        molecular_orbitals_at_grid)\n",
    "            return mat\n",
    "\n",
    "        \n",
    "        def construct_vxc_derivative_tensor():\n",
    "            derivative_of_potinfo_wrt_rho = self.adapter.get_dvxc_wrt_drho_xc_on_grid()\n",
    "            dvxc_wrt_drho = get_dvxc_wrt_drho_from_derivative_of_potinfo_wrt_rho(derivative_of_potinfo_wrt_rho)\n",
    "            return dvxc_wrt_drho\n",
    "\n",
    "        \n",
    "        four_center_elrep_tensor = numpy.einsum(\"ijkl,lc,ja->iakc\",\n",
    "                                                self.adapter.get_elrep_tensor(),\n",
    "                                                self.coefficients, self.coefficients)\n",
    "        \n",
    "        fourDtensor = four_center_elrep_tensor + construct_vxc_derivative_tensor()\n",
    "        dKS_wrt_dC_second_term = 2 * numpy.einsum(\"c,iakc->iakc\", self.orboccs, fourDtensor)\n",
    "        \n",
    "        self.dKS_wrt_dC = dKS_wrt_dC_first_term + dKS_wrt_dC_second_term\n",
    "\n",
    "    \n",
    "    '''\n",
    "    \\frac{\\partial \\Phi^{KS}_{ia}(\\textbf{C};\\;\\vec{\\theta})}{\\partial \\epsilon_{c}} = -\\delta_{ac}C_{ai}\n",
    "    '''\n",
    "    def construct_dKS_wrt_depsilon(self):\n",
    "        occupied_orbitals_kronecker = numpy.eye(self.noccorb)\n",
    "        self.dKS_wrt_depsilon = -1 * numpy.einsum(\"ac,ij,ja->iac\", occupied_orbitals_kronecker, self.overlap, self.coefficients)\n",
    "\n",
    "    '''\n",
    "    \\frac{\\partial \\Phi^{n}_{a}(\\textbf{C})}{\\partial C_{ck}} = 2\\delta_{ac}C_{ck}\n",
    "    '''\n",
    "    def construct_dnorm_wrt_dC(self):\n",
    "        occupied_orbitals_kronecker = numpy.eye(self.noccorb)\n",
    "        self.dnorm_wrt_dC = 2 * numpy.einsum(\"ac,kj,jc->kac\", occupied_orbitals_kronecker, self.overlap, self.coefficients)\n",
    "\n",
    "    '''\n",
    "    \\frac{\\partial \\Phi^{n}_{a}(\\textbf{C})}{\\partial \\epsilon_b} = 0\n",
    "    '''\n",
    "    def construct_dnorm_wrt_depsilon(self):\n",
    "        self.dnorm_wrt_depsilon = numpy.zeros((self.noccorb, self.noccorb))\n",
    "\n",
    "    def construct_dY_wrt_dX(self):\n",
    "        self.construct_dKS_wrt_dC()\n",
    "        self.construct_dKS_wrt_depsilon()\n",
    "        self.construct_dnorm_wrt_dC()\n",
    "        self.construct_dnorm_wrt_depsilon()\n",
    "        \n",
    "        N = self.nallorb * self.noccorb\n",
    "        upper = numpy.concatenate((\n",
    "                            self.dKS_wrt_dC.reshape((N, N)),\n",
    "                            self.dKS_wrt_depsilon.reshape((N, self.noccorb))\n",
    "                        ), 1)\n",
    "        lower = numpy.concatenate((\n",
    "                            self.dnorm_wrt_dC.reshape(N, self.noccorb).transpose(),\n",
    "                            self.dnorm_wrt_depsilon\n",
    "                        ), 1)\n",
    "        self.dY_wrt_dX = numpy.concatenate((upper, lower), 0)\n",
    "\n",
    "    '''\n",
    "    \\frac{\\partial E[\\rho](\\vec{\\theta})}{\\partial \\vec{\\theta}} = \n",
    "    \\int \\frac{\\partial \\left(\\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta})\\right)}{\\partial \\vec{\\theta}}d\\vec{r} \n",
    "    so we just can put \\frac{\\partial}{\\partial \\vec{\\theta}}\\left( \\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta}) \\right) \n",
    "    instead of \\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta}) to DQC calculation.\n",
    "    '''   \n",
    "    def construct_dE_wrt_dtheta(self):\n",
    "        self.dE_wrt_dtheta = self.adapter.construct_dexc_wrt_dtheta_on_grid() @ self.grid_volume\n",
    "\n",
    "\n",
    "    '''\n",
    "    \\frac{\\partial \\Phi^{KS}_{ia}(\\textbf{C};\\;\\vec{\\theta})}{\\partial \\vec{\\theta}} =\n",
    "    \\sum_j C_{aj}\\int b_i(\\vec{r}) \\frac{\\partial V_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta})}{\\partial \\vec{\\theta}} b_j(\\vec{r})d\\vec{r}\n",
    "\n",
    "    As we know, V_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta}) = \\frac{\\delta E_{XC}[\\rho]}{\\delta\\rho(\\vec{r})} = \\frac{\\delta \\left(\\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta})\\right)}{\\delta\\rho(\\vec{r})}\n",
    "    So\n",
    "\n",
    "    \\frac{\\partial \\Phi^{KS}_{ia}(\\textbf{C};\\;\\vec{\\theta})}{\\partial \\vec{\\theta}} =\n",
    "    \\sum_j C_{aj}\\int b_i(\\vec{r}) \\frac{\\partial}{\\partial \\vec{\\theta}}\\frac{\\delta\\left(\\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta})\\right)}{\\delta\\rho(\\vec{r})} b_j(\\vec{r})d\\vec{r} = \n",
    "    \\sum_j C_{aj}\\int b_i(\\vec{r}) \\frac{\\delta}{\\delta\\rho(\\vec{r})} \\frac{\\partial \\left(\\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta})\\right)}{\\partial \\vec{\\theta}} b_j(\\vec{r})d\\vec{r}\n",
    "    \n",
    "    It means, we can use \\frac{\\partial}{\\partial \\vec{\\theta}}\\left( \\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta}) \\right) instead of \n",
    "    \\rho(\\vec{r})\\epsilon_{XC}[\\rho](\\vec{r};\\;\\vec{\\theta}) in DQC function`get_vxc()`and get suitable result. \n",
    "    This function takes densinfo and takes functional derivative with respect to density.\n",
    "    '''\n",
    "    def construct_dKS_wrt_dtheta(self):\n",
    "        list_of_tensors = []\n",
    "        for parameter_number in range(self.number_of_parameters):\n",
    "            current_tensor = self.adapter.contruct_dvxc_wrt_dtheta(parameter_number)\n",
    "\n",
    "            list_of_tensors.append(numpy.expand_dims(current_tensor, -1))\n",
    "        dvxc_wrt_dtheta = numpy.concatenate(list_of_tensors, 2)\n",
    "        self.dKS_wrt_dtheta = numpy.einsum(\"ijt,ja->iat\", dvxc_wrt_dtheta, self.coefficients)\n",
    "\n",
    "    '''\n",
    "    \\frac{\\partial \\Phi^{n}_{a}(\\textbf{C})}{\\partial\\vec{\\theta}}= 0\n",
    "    '''\n",
    "    def construct_dnorm_wrt_dtheta(self):\n",
    "        self.dnorm_wrt_dtheta = numpy.zeros((self.noccorb, self.number_of_parameters))\n",
    "\n",
    "    def construct_dY_wrt_dtheta(self):\n",
    "        self.construct_dKS_wrt_dtheta()\n",
    "        self.construct_dnorm_wrt_dtheta()\n",
    "        \n",
    "        N = self.nallorb * self.noccorb\n",
    "        \n",
    "        self.dY_wrt_dtheta = numpy.concatenate((\n",
    "                            self.dKS_wrt_dtheta.reshape(N, self.number_of_parameters),\n",
    "                            self.dnorm_wrt_dtheta\n",
    "                        ), 0)\n",
    "\n",
    "    '''\n",
    "    \\frac{\\partial \\rho(\\vec{r})}{\\partial \\textbf{C}} = 2\\vec{\\chi}(\\vec{r})\\vec{\\chi}^T(\\vec{r})\\textbf{C}\\textbf{f}\n",
    "    '''\n",
    "    def construct_drho_wrt_dC(self):    \n",
    "        self.drho_wrt_dC = 2 * numpy.einsum(\"gj,gi,ib,b->gbj\", self.basis, self.basis, \n",
    "                                            self.coefficients, self.orboccs)\n",
    "\n",
    "    '''\n",
    "    \\frac{\\partial \\rho(\\vec{r})}{\\partial \\vec{\\epsilon}} = 0\n",
    "    '''\n",
    "    def construct_drho_wrt_depsilon(self):\n",
    "        self.drho_wrt_depsilon = numpy.zeros((self.grid_size, self.noccorb))\n",
    "        \n",
    "    def construct_drho_wrt_dX(self):\n",
    "        self.construct_drho_wrt_dC()\n",
    "        self.construct_drho_wrt_depsilon()\n",
    "        self.drho_wrt_dX = numpy.concatenate((self.drho_wrt_dC, \n",
    "                                              numpy.expand_dims(self.drho_wrt_depsilon, -1)), 2).transpose((0, 2, 1)).reshape((self.grid_size, -1))\n",
    "\n",
    "    def construct_drho_wrt_dtheta(self):\n",
    "        self.drho_wrt_dtheta = numpy.zeros_like(self.basis)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate some partial derivatives using pytorch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "class FixedPointAutogradPartialDerivativeCalculator():\n",
    "    def __init__(self, adapter):\n",
    "        self.adapter = adapter\n",
    "\n",
    "        self.noccorb = self.adapter.noccorb\n",
    "        self.nallorb = self.adapter.nallorb\n",
    "\n",
    "        self.fockian = self.adapter.fockian\n",
    "        self.energies = self.adapter.energies\n",
    "        self.coefficients = self.adapter.coefficients\n",
    "        self.orboccs = self.adapter.orboccs\n",
    "        \n",
    "        self.overlap = self.adapter.overlap\n",
    "\n",
    "        self.basis = self.adapter.basis\n",
    "        if hasattr(self.adapter, \"basis_gradient\"):\n",
    "            self.basis_gradient = self.adapter.basis_gradient\n",
    "        self.grid_size = self.adapter.grid_size\n",
    "        self.grid_volume = self.adapter.grid_volume\n",
    "        \n",
    "        self.number_of_parameters = self.adapter.number_of_parameters\n",
    "        \n",
    "    def construct_dY_wrt_dX(self):       \n",
    "        energies_local = torch.from_numpy(self.energies)\n",
    "        energies_local.requires_grad_(True)\n",
    "        coefficients_local = torch.from_numpy(self.coefficients)\n",
    "        coefficients_local.requires_grad_(True)\n",
    "\n",
    "        orboccs_local = torch.from_numpy(self.orboccs)\n",
    "        \n",
    "        def Y(c, eps):\n",
    "            dm_ = torch.einsum(\"b,ib,jb->ij\", orboccs_local, c, c)\n",
    "            if hasattr(self.adapter, \"contruct_fockian_from_dm\"):\n",
    "                f_ = self.adapter.contruct_fockian_from_dm(dm_)\n",
    "            else:\n",
    "                f_ = self.contruct_fockian_from_dm(dm_)\n",
    "            \n",
    "            ks = f_ @ c - torch.from_numpy(self.overlap) @ torch.einsum(\"ib,b->ib\", c, eps)\n",
    "            norm = (c.t() @ torch.from_numpy(self.overlap) @ c).diagonal(0)\n",
    "            norm -= torch.ones_like(norm)\n",
    "            return torch.cat((ks, norm.unsqueeze(0)))\n",
    "        \n",
    "        self.dY_wrt_dX = torch.autograd.functional.jacobian(Y, \n",
    "                                                            (coefficients_local, energies_local), \n",
    "                                                            vectorize=True)\n",
    "        \n",
    "        size = self.noccorb * (self.nallorb + 1)\n",
    "        \n",
    "        self.dY_wrt_dX = torch.cat((self.dY_wrt_dX[0], self.dY_wrt_dX[1].unsqueeze(-2)), 2).reshape((size, size)).numpy()\n",
    "\n",
    "    def contruct_fockian_from_dm(self, dm):\n",
    "        rho_on_grid = torch.einsum(\"ij,gi,gj->g\", dm, torch.from_numpy(self.basis), torch.from_numpy(self.basis))\n",
    "        if self.adapter.xc.family == \"GGA\":\n",
    "            grad_on_grid = 2 * torch.einsum(\"ij,gi,xgj->xg\", dm, torch.from_numpy(self.basis),\n",
    "                                            torch.from_numpy(self.basis_gradient))\n",
    "            data_on_grid = (rho_on_grid, grad_on_grid)\n",
    "        else:\n",
    "            data_on_grid = rho_on_grid\n",
    "\n",
    "        _, vxc_on_grid, _, _ = self.adapter.xc.eval_xc(None, data_on_grid)\n",
    "\n",
    "        f_ = torch.from_numpy(self.adapter.qc.get_hcore())\n",
    "        f_ += torch.einsum(\"ijkl,ji->kl\", torch.from_numpy(self.adapter.get_elrep_tensor()), dm)\n",
    "\n",
    "        f_ += torch.einsum(\"g,g,gi,gj->ij\", vxc_on_grid[0], torch.from_numpy(self.grid_volume), \n",
    "                           torch.from_numpy(self.basis), torch.from_numpy(self.basis))\n",
    "        if self.adapter.xc.family == \"GGA\":\n",
    "            vxc_grad_on_grid = 2 * torch.einsum(\"g,xg->xg\", vxc_on_grid[1], grad_on_grid)\n",
    "            f_grad_ = torch.einsum(\"xg,g,gi,xgj->ij\", vxc_grad_on_grid, torch.from_numpy(self.grid_volume), \n",
    "                   torch.from_numpy(self.basis), torch.from_numpy(self.basis_gradient))\n",
    "            \n",
    "            f_ += (f_grad_ + f_grad_.t())\n",
    "        return f_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combine AnalyticalPartialDerivativeCalculator and FixedPointAutogradPartialDerivativeCalculator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MixedPartialDerivativeCalculator(AnalyticalPartialDerivativeCalculator):\n",
    "    def __init__(self, adapter):\n",
    "        super().__init__(adapter)\n",
    "        self.fixed = FixedPointAutogradPartialDerivativeCalculator(adapter)\n",
    "\n",
    "    def construct_dY_wrt_dX(self):\n",
    "        self.fixed.construct_dY_wrt_dX()\n",
    "        self.dY_wrt_dX = self.fixed.dY_wrt_dX"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PartialToTotalDerivativeAdjoint():\n",
    "    def __init__(self, partial_derivarive_calculator):\n",
    "        self.partial_derivarive_calculator = partial_derivarive_calculator\n",
    "        \n",
    "    def construct_total_dE_wrt_dtheta(self):\n",
    "        self.partial_derivarive_calculator.construct_dE_wrt_dX()\n",
    "        self.partial_derivarive_calculator.construct_dE_wrt_dtheta()\n",
    "        self.partial_derivarive_calculator.construct_dY_wrt_dX()\n",
    "        self.partial_derivarive_calculator.construct_dY_wrt_dtheta()\n",
    "        \n",
    "        # Find the adjoint vector:\n",
    "        self.energy_adjoint = numpy.linalg.solve(self.partial_derivarive_calculator.dY_wrt_dX, \n",
    "                                          self.partial_derivarive_calculator.dE_wrt_dX)\n",
    "\n",
    "        # Calculate total derivative:\n",
    "        self.total_dE_wrt_dtheta = self.partial_derivarive_calculator.dE_wrt_dtheta - \\\n",
    "        numpy.matmul(self.energy_adjoint, self.partial_derivarive_calculator.dY_wrt_dtheta)\n",
    "\n",
    "    def construct_total_drho_squared_wrt_dtheta(self, rho_exact, retain_graph=False):        \n",
    "        self.partial_derivarive_calculator.construct_drho_wrt_dX()\n",
    "        self.partial_derivarive_calculator.construct_drho_wrt_dtheta()\n",
    "        self.partial_derivarive_calculator.construct_dY_wrt_dX()\n",
    "        self.partial_derivarive_calculator.construct_dY_wrt_dtheta()\n",
    "        \n",
    "        # Find the adjoint vector:\n",
    "        self.rho_adjoint = numpy.linalg.solve(self.partial_derivarive_calculator.dY_wrt_dX, \n",
    "                                              self.partial_derivarive_calculator.drho_wrt_dX.transpose()).transpose()\n",
    "        \n",
    "        # Calculate total derivative:\n",
    "        self.total_drho_wrt_dtheta_on_grid = - numpy.matmul(self.rho_adjoint, self.partial_derivarive_calculator.dY_wrt_dtheta)        \n",
    "        \n",
    "        diff = self.partial_derivarive_calculator.adapter.get_rho() - rho_exact\n",
    "        \n",
    "        self.total_drho_squared_wrt_dtheta = 2 * (diff * \n",
    "                                                  self.partial_derivarive_calculator.adapter.grid_volume) @ self.total_drho_wrt_dtheta_on_grid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Energy derivatives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comparison of adjoint derivatives method with existing methods for LDA-X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "mol_name_ = \"water\"\n",
    "aj_mol = molecules_for_correctness_tests[mol_name_]\n",
    "\n",
    "analytic_partial_lda = AnalyticalPartialDerivativeCalculator(PySCFDataAdapter(PySCFLDAXConstructor, LDAX_default_parameters_dict, aj_mol))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "started the four term integral computation for 34310...\n",
      "For water:\n",
      "dE/dtheta = [ 10.94899516 -11.08012795]\n"
     ]
    }
   ],
   "source": [
    "analytic_total_adjoint_lda = PartialToTotalDerivativeAdjoint(analytic_partial_lda)\n",
    "analytic_total_adjoint_lda.construct_total_dE_wrt_dtheta()\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"dE/dtheta = {analytic_total_adjoint_lda.total_dE_wrt_dtheta}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "kN8tzGJftinTihqcAURdeo",
     "type": "CODE"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For water:\n",
      "dE/dtheta = [ 10.94899712 -11.08060462]\n"
     ]
    }
   ],
   "source": [
    "d_fd_lda, _, _ = FiniteDifferenceTotalDerivativeCalculator(PySCFAdapterCustom,\n",
    "                                                       PySCFLDAXConstructor,\n",
    "                                                       LDAX_default_parameters_dict,\n",
    "                                                       lambda a: a.total_energy,\n",
    "                                                       aj_mol)\n",
    "d_fd_lda = d_fd_lda\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"dE/dtheta = {d_fd_lda}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For water:\n",
      "dE/dtheta = [ 10.94897599 -11.08009856]\n"
     ]
    }
   ],
   "source": [
    "iterative_total_lda = IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(DQCLDAXConstructor, \n",
    "                                                                                LDAX_default_parameters_dict, \n",
    "                                                                                aj_mol))\n",
    "iterative_total_lda.construct_total_dE_wrt_dtheta(retain_graph=True)\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"dE/dtheta = {iterative_total_lda.total_dE_wrt_dtheta}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "vRYtdVxp2SNXg82o1Feolt",
     "type": "MD"
    }
   },
   "source": [
    "#### Comparison of adjoint derivatives method with existing methods for PBE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For water:\n",
      "dE/dtheta = [ 12.0970346  -12.00816415   2.18721901   3.08264406  -0.19324621\n",
      "  -3.01868819  -6.70059544  -0.36203076   0.02963464   0.0244972\n",
      "   0.02252879   0.0224657 ]\n"
     ]
    }
   ],
   "source": [
    "analytic_partial_pbe = MixedPartialDerivativeCalculator(PySCFDataAdapter(PySCFPBEConstructor, \n",
    "                                                                        PBE_default_parameters_dict, \n",
    "                                                                        aj_mol))\n",
    "analytic_total_adjoint_pbe = PartialToTotalDerivativeAdjoint(analytic_partial_pbe)\n",
    "analytic_total_adjoint_pbe.construct_total_dE_wrt_dtheta()\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"dE/dtheta = {analytic_total_adjoint_pbe.total_dE_wrt_dtheta}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For water:\n",
      "dE/dtheta = [ 12.09703717 -12.00869129   2.18720856   3.08261433  -0.19324482\n",
      "  -3.01868291  -6.70056653  -0.36203097   0.02963448   0.02449715\n",
      "   0.02252877   0.02246569]\n"
     ]
    }
   ],
   "source": [
    "d_fd_pbe, _, _ = FiniteDifferenceTotalDerivativeCalculator(PySCFAdapterCustom,\n",
    "                                                       PySCFPBEConstructor,\n",
    "                                                       PBE_default_parameters_dict,\n",
    "                                                       lambda a: a.total_energy,\n",
    "                                                       aj_mol)\n",
    "d_fd_pbe = d_fd_pbe\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"dE/dtheta = {d_fd_pbe}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For water:\n",
      "dE/dtheta = [ 12.09701535 -12.00812547   2.18721098   3.08265178  -0.19324674\n",
      "  -3.01868585  -6.70057983  -0.36203062   0.02963458   0.02449717\n",
      "   0.02252878   0.02246571]\n"
     ]
    }
   ],
   "source": [
    "iterative_total_pbe = IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(DQCPBEConstructor, \n",
    "                                                                                PBE_default_parameters_dict, \n",
    "                                                                                aj_mol))\n",
    "iterative_total_pbe.construct_total_dE_wrt_dtheta(retain_graph=True)\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"dE/dtheta = {iterative_total_pbe.total_dE_wrt_dtheta}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "qPKltQaHschZbAKnHbepXO",
     "type": "MD"
    }
   },
   "source": [
    "When calculating derivatives most of the time is spent computing calculations leading to `torch.matmul(adjoint, dY_wrt_dtheta)`, including one four-term integral. This is extremely expensive, but if we print out the result of these calculations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "OXqMaOVqdBauqLzVmpj8JL",
     "type": "CODE"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.18727711e-14,  4.18524666e-14])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.matmul(analytic_total_adjoint_lda.energy_adjoint, analytic_partial_lda.dY_wrt_dtheta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 7.53800867e-15, -3.98067301e-14,  2.82996810e-16,  2.36466535e-15,\n",
       "       -1.07208866e-16, -2.38826599e-15, -3.87346087e-16,  9.67522842e-17,\n",
       "       -3.89530438e-18, -7.19361397e-18, -5.93686826e-18, -3.14423868e-18])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.matmul(analytic_total_adjoint_pbe.energy_adjoint, analytic_partial_pbe.dY_wrt_dtheta)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "v6lLUYrE5D2KhVsKaOO1IJ",
     "type": "MD"
    }
   },
   "source": [
    "It could be seen that it is within margin of error from machine zero. In fact, this is expected from the Hellmann-Feynmann theorem in quantum mechanics and in our article we show it still holds exactly in the discrete case of Roothaan's equations. Therefore a majority of calculations could be omitted entirely."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "agFQLsE7RgKtOq8cTIEUgn",
     "type": "MD"
    }
   },
   "source": [
    "### Hellmann-Feynman method with PySCF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "class HellmannFeynmanTotalDerivative():\n",
    "    def __init__(self, partial_derivarive_calculator):\n",
    "        self.partial_derivarive_calculator = partial_derivarive_calculator\n",
    "        \n",
    "    def construct_total_dE_wrt_dtheta(self):\n",
    "        self.partial_derivarive_calculator.construct_dE_wrt_dtheta()\n",
    "        \n",
    "        # Calculate total derivative:\n",
    "        self.total_dE_wrt_dtheta = self.partial_derivarive_calculator.dE_wrt_dtheta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "ipuFdlwmyGdslpwgLzeyhf",
     "type": "MD"
    }
   },
   "source": [
    "#### Comparison of Hellmann-Feynman derivatives method with existing methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "5GuoHEv6WP9UDWCOUkmyGF",
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "derivative_correctness_results_lda = {\n",
    "    \"fd_of_pyscf_custom_lda\":       {},\n",
    "    \"dqc_autograd_lda\":             {},\n",
    "    \"pyscf_hellmann_feynman_lda\":   {},\n",
    "    \"implicit_differentiation_lda\": {}\n",
    "}\n",
    "\n",
    "derivative_correctness_results_pbe = {\n",
    "    \"fd_of_pyscf_custom_pbe\":       {},\n",
    "    \"dqc_autograd_pbe\":             {},\n",
    "    \"pyscf_hellmann_feynman_pbe\":   {},\n",
    "    \"implicit_differentiation_pbe\": {}\n",
    "}\n",
    "\n",
    "for data_of_method in derivative_correctness_results_lda.values():\n",
    "    for mol_name in molecules_for_correctness_tests:\n",
    "        data_of_method[mol_name] = {}\n",
    "\n",
    "for data_of_method in derivative_correctness_results_pbe.values():\n",
    "    for mol_name in molecules_for_correctness_tests:\n",
    "        data_of_method[mol_name] = {}\n",
    "\n",
    "        \n",
    "def print_results_derivatives(mol_name, method_name, derivatives, d):\n",
    "    print(f\"For {mol_name} molecule and {method_name} method: {derivatives=}, duration={d} s\")\n",
    "\n",
    "def print_and_save_results_derivatives_lda(mol_name, method_name, derivatives, dur):\n",
    "    print_results_derivatives(mol_name, method_name, derivatives, dur)\n",
    "    derivative_correctness_results_lda[method_name][mol_name][\"derivatives\"] = derivatives\n",
    "    derivative_correctness_results_lda[method_name][mol_name][\"time_d\"] = dur\n",
    "    \n",
    "def print_and_save_results_derivatives_pbe(mol_name, method_name, derivatives, dur):\n",
    "    print_results_derivatives(mol_name, method_name, derivatives, dur)\n",
    "    derivative_correctness_results_pbe[method_name][mol_name][\"derivatives\"] = derivatives\n",
    "    derivative_correctness_results_pbe[method_name][mol_name][\"time_d\"] = dur"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For h2 molecule and fd_of_pyscf_custom_lda method: derivatives=array([0.74835896, 1.47150197]), duration=0.3858170509338379 s\n",
      "For h2 molecule and dqc_autograd_lda method: derivatives=array([0.74835713, 1.47153201]), duration=0.1342601776123047 s\n",
      "For h2 molecule and pyscf_hellmann_feynman_lda method: derivatives=array([0.74835848, 1.47153331]), duration=0.36503052711486816 s\n",
      "started the four term integral computation for 20048...\n",
      "For h2 molecule and implicit_differentiation_lda method: derivatives=array([0.74835848, 1.47153331]), duration=1.542668104171753 s\n",
      "---><---\n",
      "For water molecule and fd_of_pyscf_custom_lda method: derivatives=array([ 10.94899713, -11.08060462]), duration=0.7466273307800293 s\n",
      "For water molecule and dqc_autograd_lda method: derivatives=array([ 10.94897599, -11.08009856]), duration=0.1818859577178955 s\n",
      "For water molecule and pyscf_hellmann_feynman_lda method: derivatives=array([ 10.94899516, -11.08012795]), duration=0.4459357261657715 s\n",
      "started the four term integral computation for 34310...\n",
      "For water molecule and implicit_differentiation_lda method: derivatives=array([ 10.94899516, -11.08012795]), duration=1.640470266342163 s\n",
      "---><---\n",
      "For ethane molecule and fd_of_pyscf_custom_lda method: derivatives=array([ 18.10967992, -10.01032567]), duration=1.6668262481689453 s\n",
      "For ethane molecule and dqc_autograd_lda method: derivatives=array([ 18.10965746, -10.00989673]), duration=0.6696631908416748 s\n",
      "For ethane molecule and pyscf_hellmann_feynman_lda method: derivatives=array([ 18.10967687, -10.00985709]), duration=0.9483811855316162 s\n",
      "started the four term integral computation for 87948...\n",
      "For ethane molecule and implicit_differentiation_lda method: derivatives=array([ 18.10967687, -10.00985709]), duration=4.8906214237213135 s\n",
      "---><---\n",
      "For benzene molecule and fd_of_pyscf_custom_lda method: derivatives=array([ 48.74784935, -40.54685998]), duration=7.486317873001099 s\n",
      "For benzene molecule and dqc_autograd_lda method: derivatives=array([ 48.74808178, -40.55343964]), duration=2.860684633255005 s\n",
      "For benzene molecule and pyscf_hellmann_feynman_lda method: derivatives=array([ 48.74784577, -40.54552819]), duration=3.6336164474487305 s\n",
      "For benzene molecule and implicit_differentiation_lda method: derivatives=array([nan, nan]), duration=nan s\n",
      "---><---\n"
     ]
    }
   ],
   "source": [
    "for mol_name, mol in molecules_for_correctness_tests.items():\n",
    "    der, dur, _ = FiniteDifferenceTotalDerivativeCalculator(PySCFAdapterCustom,\n",
    "                                                      PySCFLDAXConstructor,\n",
    "                                                      LDAX_default_parameters_dict,\n",
    "                                                      lambda scf: scf.total_energy,\n",
    "                                                      mol)\n",
    "    print_and_save_results_derivatives_lda(mol_name, \"fd_of_pyscf_custom_lda\", der, dur)\n",
    "    \n",
    "    start = time.time()\n",
    "    calc = IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(DQCLDAXConstructor, \n",
    "                                                                         LDAX_default_parameters_dict, \n",
    "                                                                         mol))\n",
    "    calc.construct_total_dE_wrt_dtheta()\n",
    "    duration = time.time() - start\n",
    "    print_and_save_results_derivatives_lda(mol_name, \"dqc_autograd_lda\", calc.total_dE_wrt_dtheta, duration)\n",
    "    \n",
    "    start_2 = time.time()\n",
    "    calc = HellmannFeynmanTotalDerivative(AnalyticalPartialDerivativeCalculator(PySCFDataAdapter(PySCFLDAXConstructor,\n",
    "                                                                                                 LDAX_default_parameters_dict,\n",
    "                                                                                                 mol)))\n",
    "    calc.construct_total_dE_wrt_dtheta()\n",
    "    dur_2 = time.time() - start_2\n",
    "    print_and_save_results_derivatives_lda(mol_name, \"pyscf_hellmann_feynman_lda\", calc.total_dE_wrt_dtheta, dur_2)\n",
    "    \n",
    "    if mol_name != \"benzene\":\n",
    "        start_2 = time.time()\n",
    "        calc = PartialToTotalDerivativeAdjoint(AnalyticalPartialDerivativeCalculator(PySCFDataAdapter(PySCFLDAXConstructor,\n",
    "                                                                                                 LDAX_default_parameters_dict,\n",
    "                                                                                                 mol)))\n",
    "        calc.construct_total_dE_wrt_dtheta()\n",
    "        dur_2 = time.time() - start_2\n",
    "        print_and_save_results_derivatives_lda(mol_name, \"implicit_differentiation_lda\", calc.total_dE_wrt_dtheta, dur_2)\n",
    "    else:\n",
    "        print_and_save_results_derivatives_lda(mol_name, \"implicit_differentiation_lda\", numpy.full(2, numpy.nan), numpy.nan)        \n",
    "    \n",
    "    print(\"---><---\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For h2 molecule and fd_of_pyscf_custom_pbe method: derivatives=array([ 0.87641519,  1.75662048,  0.33750333,  0.27029414, -0.03089221,\n",
      "       -0.25717842, -0.62576507, -0.08188992,  0.00360855,  0.00428855,\n",
      "        0.00523672,  0.00660081]), duration=6.091701984405518 s\n",
      "For h2 molecule and dqc_autograd_pbe method: derivatives=array([ 0.87641292,  1.7566578 ,  0.33750449,  0.27029633, -0.03089237,\n",
      "       -0.25717867, -0.62576756, -0.08188976,  0.00360856,  0.00428856,\n",
      "        0.00523672,  0.00660081]), duration=0.2459731101989746 s\n",
      "For h2 molecule and pyscf_hellmann_feynman_pbe method: derivatives=array([ 0.87641556,  1.75665907,  0.33750513,  0.27029721, -0.03089244,\n",
      "       -0.25717935, -0.62576954, -0.08188979,  0.00360857,  0.00428857,\n",
      "        0.00523673,  0.0066008 ]), duration=5.359846115112305 s\n",
      "For h2 molecule and implicit_differentiation_pbe method: derivatives=array([ 0.87641556,  1.75665907,  0.33750513,  0.27029721, -0.03089244,\n",
      "       -0.25717935, -0.62576954, -0.08188979,  0.00360857,  0.00428857,\n",
      "        0.00523673,  0.0066008 ]), duration=39.904908180236816 s\n",
      "---><---\n",
      "For water molecule and fd_of_pyscf_custom_pbe method: derivatives=array([ 12.09703716, -12.00869128,   2.18720858,   3.08261451,\n",
      "        -0.19324481,  -3.01868296,  -6.70056643,  -0.36203095,\n",
      "         0.02963448,   0.02449715,   0.02252878,   0.0224657 ]), duration=10.967376708984375 s\n",
      "For water molecule and dqc_autograd_pbe method: derivatives=array([ 12.09701535, -12.00812547,   2.18721098,   3.08265178,\n",
      "        -0.19324674,  -3.01868585,  -6.70057983,  -0.36203062,\n",
      "         0.02963458,   0.02449717,   0.02252878,   0.02246571]), duration=0.573073148727417 s\n",
      "For water molecule and pyscf_hellmann_feynman_pbe method: derivatives=array([ 12.0970346 , -12.00816415,   2.18721901,   3.08264406,\n",
      "        -0.19324621,  -3.01868819,  -6.70059544,  -0.36203076,\n",
      "         0.02963464,   0.0244972 ,   0.02252879,   0.0224657 ]), duration=6.108987331390381 s\n",
      "For water molecule and implicit_differentiation_pbe method: derivatives=array([ 12.0970346 , -12.00816415,   2.18721901,   3.08264406,\n",
      "        -0.19324621,  -3.01868819,  -6.70059544,  -0.36203076,\n",
      "         0.02963464,   0.0244972 ,   0.02252879,   0.0224657 ]), duration=43.65897870063782 s\n",
      "---><---\n",
      "For ethane molecule and fd_of_pyscf_custom_pbe method: derivatives=array([ 19.77885239, -10.70195581,   3.65489582,   4.86287431,\n",
      "        -0.27067548,  -4.40789038, -12.36850189,  -0.66172522,\n",
      "         0.05570527,   0.04613002,   0.0410357 ,   0.03865966]), duration=21.17627263069153 s\n",
      "For ethane molecule and dqc_autograd_pbe method: derivatives=array([ 19.77882674, -10.70147585,   3.65494082,   4.8629488 ,\n",
      "        -0.2706751 ,  -4.40790812, -12.36854246,  -0.66172534,\n",
      "         0.05570549,   0.0461301 ,   0.04103579,   0.0386598 ]), duration=2.097231149673462 s\n",
      "For ethane molecule and pyscf_hellmann_feynman_pbe method: derivatives=array([ 19.77884859, -10.70143711,   3.65491255,   4.86292048,\n",
      "        -0.27067741,  -4.40789748, -12.36855795,  -0.66172492,\n",
      "         0.05570555,   0.04613012,   0.04103576,   0.03865971]), duration=6.720547437667847 s\n",
      "For ethane molecule and implicit_differentiation_pbe method: derivatives=array([ 19.77884859, -10.70143711,   3.65491255,   4.86292048,\n",
      "        -0.27067741,  -4.40789748, -12.36855795,  -0.66172492,\n",
      "         0.05570555,   0.04613012,   0.04103576,   0.03865971]), duration=275.9585144519806 s\n",
      "---><---\n",
      "For benzene molecule and fd_of_pyscf_custom_pbe method: derivatives=array([ 52.97476248, -44.02647718,   8.81312265,  12.76040285,\n",
      "        -0.624974  , -11.51061883, -32.12266253,  -1.49100277,\n",
      "         0.13969964,   0.10898657,   0.09179907,   0.08250575]), duration=55.163607120513916 s\n",
      "For benzene molecule and dqc_autograd_pbe method: derivatives=array([ 52.97395646, -44.02782181,   8.81229   ,  12.75976111,\n",
      "        -0.6250228 , -11.51056925, -32.12137584,  -1.49100091,\n",
      "         0.13969352,   0.10898245,   0.09180116,   0.08251667]), duration=8.007707357406616 s\n",
      "For benzene molecule and pyscf_hellmann_feynman_pbe method: derivatives=array([ 52.97475356, -44.02501855,   8.81317358,  12.76052826,\n",
      "        -0.62497452, -11.51064468, -32.12273143,  -1.49101325,\n",
      "         0.13970011,   0.10898749,   0.09180211,   0.08250662]), duration=11.20598030090332 s\n",
      "For benzene molecule and implicit_differentiation_pbe method: derivatives=array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]), duration=nan s\n",
      "---><---\n"
     ]
    }
   ],
   "source": [
    "for mol_name, mol in molecules_for_correctness_tests.items():\n",
    "    der, dur, _ = FiniteDifferenceTotalDerivativeCalculator(PySCFAdapterCustom,\n",
    "                                                      PySCFPBEConstructor,\n",
    "                                                      PBE_default_parameters_dict,\n",
    "                                                      lambda scf: scf.total_energy,\n",
    "                                                      mol)\n",
    "    print_and_save_results_derivatives_pbe(mol_name, \"fd_of_pyscf_custom_pbe\", der, dur)\n",
    "    \n",
    "    start = time.time()\n",
    "    calc = IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(DQCPBEConstructor, \n",
    "                                                                         PBE_default_parameters_dict, \n",
    "                                                                         mol))\n",
    "    calc.construct_total_dE_wrt_dtheta()\n",
    "    der = calc.total_dE_wrt_dtheta\n",
    "    dur = time.time() - start\n",
    "    print_and_save_results_derivatives_pbe(mol_name, \"dqc_autograd_pbe\", der, dur)\n",
    "    \n",
    "    start_2 = time.time()\n",
    "    calc = HellmannFeynmanTotalDerivative(AnalyticalPartialDerivativeCalculator(PySCFDataAdapter(PySCFPBEConstructor,\n",
    "                                                                                                 PBE_default_parameters_dict,\n",
    "                                                                                                 mol)))\n",
    "    calc.construct_total_dE_wrt_dtheta()\n",
    "    dur_2 = time.time() - start_2\n",
    "    print_and_save_results_derivatives_pbe(mol_name, \"pyscf_hellmann_feynman_pbe\", calc.total_dE_wrt_dtheta, dur_2)\n",
    "    \n",
    "    if mol_name != \"benzene\":\n",
    "        start_2 = time.time()\n",
    "        calc = PartialToTotalDerivativeAdjoint(MixedPartialDerivativeCalculator(PySCFDataAdapter(PySCFPBEConstructor,\n",
    "                                                                                                 PBE_default_parameters_dict,\n",
    "                                                                                                 mol)))\n",
    "        calc.construct_total_dE_wrt_dtheta()\n",
    "        dur_2 = time.time() - start_2\n",
    "        print_and_save_results_derivatives_pbe(mol_name, \"implicit_differentiation_pbe\", calc.total_dE_wrt_dtheta, dur_2)\n",
    "    else:\n",
    "        print_and_save_results_derivatives_pbe(mol_name, \"implicit_differentiation_pbe\", numpy.full(12, numpy.nan), numpy.nan)        \n",
    "\n",
    "    print(\"---><---\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "4wniOikfCjUQCcWzy5XUY9",
     "type": "MD"
    }
   },
   "source": [
    "## Results\n",
    "In this section, the results of all computations above are assembled."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "hukIpEBzzZkQ1BKAoagtJH",
     "type": "MD"
    }
   },
   "source": [
    "### Energy computations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "ihPew4R2jd1Z7ld8hKACgc",
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "pretty_energy_method_names = {\n",
    "    \"pyscf_builtin_lda\": \"PySCF (Bulitin LDA)\",\n",
    "    \"pyscf_custom_lda\": \"PySCF (Custom LDA)\",\n",
    "    \"pyscf_ccsdt\": \"PySCF (CCST(T))\",\n",
    "    \"dqc_nograd_lda\": \"DQC (Custom LDA)\",\n",
    "    \"pyscf_builtin_pbe\": \"PySCF (Bulitin PBE)\",\n",
    "    \"pyscf_custom_pbe\": \"PySCF (Custom PBE)\",\n",
    "    \"dqc_nograd_pbe\": \"DQC (Custom PBE)\",\n",
    "}\n",
    "\n",
    "pretty_derivative_method_names = {\n",
    "    \"fd_of_pyscf_custom_lda\":       \"Δ(FD of PySCF, LDA)\",\n",
    "    \"dqc_autograd_lda\":             \"Δ(DQC, autograd, LDA)\",\n",
    "    \"pyscf_hellmann_feynman_lda\":   \"Hellmann-Feyman PySCF, LDA\",\n",
    "    \"implicit_differentiation_lda\": \"Δ(PySCF, Implicit differentiation, LDA)\",\n",
    "    \"fd_of_pyscf_custom_pbe\":       \"Δ(FD of PySCF, PBE)\",\n",
    "    \"dqc_autograd_pbe\":             \"Δ(DQC, autograd, PBE)\",\n",
    "    \"pyscf_hellmann_feynman_pbe\":   \"Hellmann-Feyman PySCF, PBE\",\n",
    "    \"implicit_differentiation_pbe\": \"Δ(PySCF, Implicit differentiation, PBE)\",\n",
    "}\n",
    "\n",
    "pretty_molecule_names = {\n",
    "    \"h2\": \"H<sub>2</sub>\",\n",
    "    \"water\": \"Water\",\n",
    "    \"ethane\": \"Ethane\",\n",
    "    \"benzene\": \"Benzene\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "o2vxywCndQXlqjHHSozaaj",
     "type": "CODE"
    }
   },
   "outputs": [],
   "source": [
    "energy_results = {}\n",
    "\n",
    "for mol_name in molecules_for_correctness_tests:\n",
    "    energy_results[mol_name] = {}\n",
    "    for method in energy_correctness_results:\n",
    "        mb_time = energy_correctness_results[method][mol_name][\"time\"]\n",
    "        time_ = mb_time.average if hasattr(mb_time, \"average\") else mb_time\n",
    "        mb_energy = energy_correctness_results[method][mol_name][\"energy\"]\n",
    "        energy_ = mb_energy.item() if hasattr(mb_energy, \"item\") else mb_energy\n",
    "        energy_results[mol_name][pretty_energy_method_names[method]] = {\n",
    "                                                        \"Energy\": energy_,\n",
    "                                                        \"Elapsed (s)\": time_\n",
    "                                                    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "2LRvBzITeYkatHea79GpOP",
     "type": "CODE"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h2>Energy computations results</h2>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>H<sub>2</sub></h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_6446d\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_6446d_level0_col0\" class=\"col_heading level0 col0\" >PySCF (Bulitin LDA)</th>\n",
       "      <th id=\"T_6446d_level0_col1\" class=\"col_heading level0 col1\" >PySCF (Custom LDA)</th>\n",
       "      <th id=\"T_6446d_level0_col2\" class=\"col_heading level0 col2\" >PySCF (Bulitin PBE)</th>\n",
       "      <th id=\"T_6446d_level0_col3\" class=\"col_heading level0 col3\" >PySCF (Custom PBE)</th>\n",
       "      <th id=\"T_6446d_level0_col4\" class=\"col_heading level0 col4\" >PySCF (CCST(T))</th>\n",
       "      <th id=\"T_6446d_level0_col5\" class=\"col_heading level0 col5\" >DQC (Custom LDA)</th>\n",
       "      <th id=\"T_6446d_level0_col6\" class=\"col_heading level0 col6\" >DQC (Custom PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_6446d_level0_row0\" class=\"row_heading level0 row0\" >Energy</th>\n",
       "      <td id=\"T_6446d_row0_col0\" class=\"data row0 col0\" >-1.038618</td>\n",
       "      <td id=\"T_6446d_row0_col1\" class=\"data row0 col1\" >-1.038618</td>\n",
       "      <td id=\"T_6446d_row0_col2\" class=\"data row0 col2\" >-1.161916</td>\n",
       "      <td id=\"T_6446d_row0_col3\" class=\"data row0 col3\" >-1.161916</td>\n",
       "      <td id=\"T_6446d_row0_col4\" class=\"data row0 col4\" >-1.151679</td>\n",
       "      <td id=\"T_6446d_row0_col5\" class=\"data row0 col5\" >-1.038617</td>\n",
       "      <td id=\"T_6446d_row0_col6\" class=\"data row0 col6\" >-1.161915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6446d_level0_row1\" class=\"row_heading level0 row1\" >Elapsed (s)</th>\n",
       "      <td id=\"T_6446d_row1_col0\" class=\"data row1 col0\" >0.075245</td>\n",
       "      <td id=\"T_6446d_row1_col1\" class=\"data row1 col1\" >0.224248</td>\n",
       "      <td id=\"T_6446d_row1_col2\" class=\"data row1 col2\" >0.360803</td>\n",
       "      <td id=\"T_6446d_row1_col3\" class=\"data row1 col3\" >0.762268</td>\n",
       "      <td id=\"T_6446d_row1_col4\" class=\"data row1 col4\" >0.053429</td>\n",
       "      <td id=\"T_6446d_row1_col5\" class=\"data row1 col5\" >0.072609</td>\n",
       "      <td id=\"T_6446d_row1_col6\" class=\"data row1 col6\" >0.192741</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Water</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_e5de9\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_e5de9_level0_col0\" class=\"col_heading level0 col0\" >PySCF (Bulitin LDA)</th>\n",
       "      <th id=\"T_e5de9_level0_col1\" class=\"col_heading level0 col1\" >PySCF (Custom LDA)</th>\n",
       "      <th id=\"T_e5de9_level0_col2\" class=\"col_heading level0 col2\" >PySCF (Bulitin PBE)</th>\n",
       "      <th id=\"T_e5de9_level0_col3\" class=\"col_heading level0 col3\" >PySCF (Custom PBE)</th>\n",
       "      <th id=\"T_e5de9_level0_col4\" class=\"col_heading level0 col4\" >PySCF (CCST(T))</th>\n",
       "      <th id=\"T_e5de9_level0_col5\" class=\"col_heading level0 col5\" >DQC (Custom LDA)</th>\n",
       "      <th id=\"T_e5de9_level0_col6\" class=\"col_heading level0 col6\" >DQC (Custom PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_e5de9_level0_row0\" class=\"row_heading level0 row0\" >Energy</th>\n",
       "      <td id=\"T_e5de9_row0_col0\" class=\"data row0 col0\" >-75.154705</td>\n",
       "      <td id=\"T_e5de9_row0_col1\" class=\"data row0 col1\" >-75.154705</td>\n",
       "      <td id=\"T_e5de9_row0_col2\" class=\"data row0 col2\" >-76.298647</td>\n",
       "      <td id=\"T_e5de9_row0_col3\" class=\"data row0 col3\" >-76.298647</td>\n",
       "      <td id=\"T_e5de9_row0_col4\" class=\"data row0 col4\" >-76.120641</td>\n",
       "      <td id=\"T_e5de9_row0_col5\" class=\"data row0 col5\" >-75.154695</td>\n",
       "      <td id=\"T_e5de9_row0_col6\" class=\"data row0 col6\" >-76.298635</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_e5de9_level0_row1\" class=\"row_heading level0 row1\" >Elapsed (s)</th>\n",
       "      <td id=\"T_e5de9_row1_col0\" class=\"data row1 col0\" >0.102695</td>\n",
       "      <td id=\"T_e5de9_row1_col1\" class=\"data row1 col1\" >0.314995</td>\n",
       "      <td id=\"T_e5de9_row1_col2\" class=\"data row1 col2\" >0.298207</td>\n",
       "      <td id=\"T_e5de9_row1_col3\" class=\"data row1 col3\" >0.979728</td>\n",
       "      <td id=\"T_e5de9_row1_col4\" class=\"data row1 col4\" >0.093338</td>\n",
       "      <td id=\"T_e5de9_row1_col5\" class=\"data row1 col5\" >0.166285</td>\n",
       "      <td id=\"T_e5de9_row1_col6\" class=\"data row1 col6\" >0.340738</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Ethane</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_7e00a\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_7e00a_level0_col0\" class=\"col_heading level0 col0\" >PySCF (Bulitin LDA)</th>\n",
       "      <th id=\"T_7e00a_level0_col1\" class=\"col_heading level0 col1\" >PySCF (Custom LDA)</th>\n",
       "      <th id=\"T_7e00a_level0_col2\" class=\"col_heading level0 col2\" >PySCF (Bulitin PBE)</th>\n",
       "      <th id=\"T_7e00a_level0_col3\" class=\"col_heading level0 col3\" >PySCF (Custom PBE)</th>\n",
       "      <th id=\"T_7e00a_level0_col4\" class=\"col_heading level0 col4\" >PySCF (CCST(T))</th>\n",
       "      <th id=\"T_7e00a_level0_col5\" class=\"col_heading level0 col5\" >DQC (Custom LDA)</th>\n",
       "      <th id=\"T_7e00a_level0_col6\" class=\"col_heading level0 col6\" >DQC (Custom PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_7e00a_level0_row0\" class=\"row_heading level0 row0\" >Energy</th>\n",
       "      <td id=\"T_7e00a_row0_col0\" class=\"data row0 col0\" >-73.232977</td>\n",
       "      <td id=\"T_7e00a_row0_col1\" class=\"data row0 col1\" >-73.232977</td>\n",
       "      <td id=\"T_7e00a_row0_col2\" class=\"data row0 col2\" >-75.079601</td>\n",
       "      <td id=\"T_7e00a_row0_col3\" class=\"data row0 col3\" >-75.079601</td>\n",
       "      <td id=\"T_7e00a_row0_col4\" class=\"data row0 col4\" >-74.728502</td>\n",
       "      <td id=\"T_7e00a_row0_col5\" class=\"data row0 col5\" >-73.232964</td>\n",
       "      <td id=\"T_7e00a_row0_col6\" class=\"data row0 col6\" >-75.079586</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7e00a_level0_row1\" class=\"row_heading level0 row1\" >Elapsed (s)</th>\n",
       "      <td id=\"T_7e00a_row1_col0\" class=\"data row1 col0\" >0.819298</td>\n",
       "      <td id=\"T_7e00a_row1_col1\" class=\"data row1 col1\" >0.891464</td>\n",
       "      <td id=\"T_7e00a_row1_col2\" class=\"data row1 col2\" >1.287189</td>\n",
       "      <td id=\"T_7e00a_row1_col3\" class=\"data row1 col3\" >1.748768</td>\n",
       "      <td id=\"T_7e00a_row1_col4\" class=\"data row1 col4\" >1.069393</td>\n",
       "      <td id=\"T_7e00a_row1_col5\" class=\"data row1 col5\" >0.446639</td>\n",
       "      <td id=\"T_7e00a_row1_col6\" class=\"data row1 col6\" >1.308819</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Benzene</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_eeae1\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_eeae1_level0_col0\" class=\"col_heading level0 col0\" >PySCF (Bulitin LDA)</th>\n",
       "      <th id=\"T_eeae1_level0_col1\" class=\"col_heading level0 col1\" >PySCF (Custom LDA)</th>\n",
       "      <th id=\"T_eeae1_level0_col2\" class=\"col_heading level0 col2\" >PySCF (Bulitin PBE)</th>\n",
       "      <th id=\"T_eeae1_level0_col3\" class=\"col_heading level0 col3\" >PySCF (Custom PBE)</th>\n",
       "      <th id=\"T_eeae1_level0_col4\" class=\"col_heading level0 col4\" >PySCF (CCST(T))</th>\n",
       "      <th id=\"T_eeae1_level0_col5\" class=\"col_heading level0 col5\" >DQC (Custom LDA)</th>\n",
       "      <th id=\"T_eeae1_level0_col6\" class=\"col_heading level0 col6\" >DQC (Custom PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_eeae1_level0_row0\" class=\"row_heading level0 row0\" >Energy</th>\n",
       "      <td id=\"T_eeae1_row0_col0\" class=\"data row0 col0\" >-213.414684</td>\n",
       "      <td id=\"T_eeae1_row0_col1\" class=\"data row0 col1\" >-213.414684</td>\n",
       "      <td id=\"T_eeae1_row0_col2\" class=\"data row0 col2\" >-218.095580</td>\n",
       "      <td id=\"T_eeae1_row0_col3\" class=\"data row0 col3\" >-218.095580</td>\n",
       "      <td id=\"T_eeae1_row0_col4\" class=\"data row0 col4\" >-217.077970</td>\n",
       "      <td id=\"T_eeae1_row0_col5\" class=\"data row0 col5\" >-213.414306</td>\n",
       "      <td id=\"T_eeae1_row0_col6\" class=\"data row0 col6\" >-218.095289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_eeae1_level0_row1\" class=\"row_heading level0 row1\" >Elapsed (s)</th>\n",
       "      <td id=\"T_eeae1_row1_col0\" class=\"data row1 col0\" >2.378047</td>\n",
       "      <td id=\"T_eeae1_row1_col1\" class=\"data row1 col1\" >1.974436</td>\n",
       "      <td id=\"T_eeae1_row1_col2\" class=\"data row1 col2\" >3.636233</td>\n",
       "      <td id=\"T_eeae1_row1_col3\" class=\"data row1 col3\" >4.148703</td>\n",
       "      <td id=\"T_eeae1_row1_col4\" class=\"data row1 col4\" >9.228793</td>\n",
       "      <td id=\"T_eeae1_row1_col5\" class=\"data row1 col5\" >2.332007</td>\n",
       "      <td id=\"T_eeae1_row1_col6\" class=\"data row1 col6\" >4.828110</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML(\"<h2>Energy computations results</h2>\"))\n",
    "for mol_name in molecules_for_correctness_tests:\n",
    "    df = pd.DataFrame(energy_results[mol_name])\n",
    "    html = df.style.to_html()\n",
    "    display(HTML(f\"<h3>{pretty_molecule_names[mol_name]}</h3>\" + html))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "TE24Up54Ze41j7n7VhWblo",
     "type": "MD"
    }
   },
   "source": [
    "### Derivative computations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "derivative_results_lda = {}\n",
    "\n",
    "for mol_name in molecules_for_correctness_tests:\n",
    "    derivative_results_lda[mol_name] = {}\n",
    "    for method in derivative_correctness_results_lda:\n",
    "        dictionary = {}\n",
    "        if method == \"pyscf_hellmann_feynman_lda\":\n",
    "            values = [f'{value:.3E}' for value in derivative_correctness_results_lda[method][mol_name][\"derivatives\"]]\n",
    "        else:\n",
    "            deltas = derivative_correctness_results_lda[method][mol_name][\"derivatives\"] - derivative_correctness_results_lda[\"pyscf_hellmann_feynman_lda\"][mol_name][\"derivatives\"]\n",
    "            values = [f'{value:.1E}' for value in deltas]\n",
    "        dictionary = dict(zip(LDAX_default_parameters_dict.keys(), values))\n",
    "            \n",
    "        mb_time = derivative_correctness_results_lda[method][mol_name][\"time_d\"]\n",
    "        time_ = mb_time.average if hasattr(mb_time, \"average\") else mb_time\n",
    "        dictionary[\"Elapsed (s)\"] = time_\n",
    "\n",
    "        derivative_results_lda[mol_name][pretty_derivative_method_names[method]] = dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "datalore": {
     "hide_input_from_viewers": true,
     "hide_output_from_viewers": true,
     "node_id": "Xy9H5ovh8u8rFtSTHr2DQd",
     "type": "CODE"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h2>Derivatives computations results</h2>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>H<sub>2</sub></h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_da21d\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_da21d_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, LDA)</th>\n",
       "      <th id=\"T_da21d_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, LDA)</th>\n",
       "      <th id=\"T_da21d_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, LDA</th>\n",
       "      <th id=\"T_da21d_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, LDA)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_da21d_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_da21d_row0_col0\" class=\"data row0 col0\" >4.8E-07</td>\n",
       "      <td id=\"T_da21d_row0_col1\" class=\"data row0 col1\" >-1.3E-06</td>\n",
       "      <td id=\"T_da21d_row0_col2\" class=\"data row0 col2\" >7.484E-01</td>\n",
       "      <td id=\"T_da21d_row0_col3\" class=\"data row0 col3\" >-3.3E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_da21d_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_da21d_row1_col0\" class=\"data row1 col0\" >-3.1E-05</td>\n",
       "      <td id=\"T_da21d_row1_col1\" class=\"data row1 col1\" >-1.3E-06</td>\n",
       "      <td id=\"T_da21d_row1_col2\" class=\"data row1 col2\" >1.472E+00</td>\n",
       "      <td id=\"T_da21d_row1_col3\" class=\"data row1 col3\" >-8.9E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_da21d_level0_row2\" class=\"row_heading level0 row2\" >Elapsed (s)</th>\n",
       "      <td id=\"T_da21d_row2_col0\" class=\"data row2 col0\" >0.385817</td>\n",
       "      <td id=\"T_da21d_row2_col1\" class=\"data row2 col1\" >0.134260</td>\n",
       "      <td id=\"T_da21d_row2_col2\" class=\"data row2 col2\" >0.365031</td>\n",
       "      <td id=\"T_da21d_row2_col3\" class=\"data row2 col3\" >1.542668</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Water</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_690bc\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_690bc_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, LDA)</th>\n",
       "      <th id=\"T_690bc_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, LDA)</th>\n",
       "      <th id=\"T_690bc_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, LDA</th>\n",
       "      <th id=\"T_690bc_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, LDA)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_690bc_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_690bc_row0_col0\" class=\"data row0 col0\" >2.0E-06</td>\n",
       "      <td id=\"T_690bc_row0_col1\" class=\"data row0 col1\" >-1.9E-05</td>\n",
       "      <td id=\"T_690bc_row0_col2\" class=\"data row0 col2\" >1.095E+01</td>\n",
       "      <td id=\"T_690bc_row0_col3\" class=\"data row0 col3\" >-2.7E-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_690bc_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_690bc_row1_col0\" class=\"data row1 col0\" >-4.8E-04</td>\n",
       "      <td id=\"T_690bc_row1_col1\" class=\"data row1 col1\" >2.9E-05</td>\n",
       "      <td id=\"T_690bc_row1_col2\" class=\"data row1 col2\" >-1.108E+01</td>\n",
       "      <td id=\"T_690bc_row1_col3\" class=\"data row1 col3\" >6.9E-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_690bc_level0_row2\" class=\"row_heading level0 row2\" >Elapsed (s)</th>\n",
       "      <td id=\"T_690bc_row2_col0\" class=\"data row2 col0\" >0.746627</td>\n",
       "      <td id=\"T_690bc_row2_col1\" class=\"data row2 col1\" >0.181886</td>\n",
       "      <td id=\"T_690bc_row2_col2\" class=\"data row2 col2\" >0.445936</td>\n",
       "      <td id=\"T_690bc_row2_col3\" class=\"data row2 col3\" >1.640470</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Ethane</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_101fd\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_101fd_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, LDA)</th>\n",
       "      <th id=\"T_101fd_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, LDA)</th>\n",
       "      <th id=\"T_101fd_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, LDA</th>\n",
       "      <th id=\"T_101fd_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, LDA)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_101fd_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_101fd_row0_col0\" class=\"data row0 col0\" >3.1E-06</td>\n",
       "      <td id=\"T_101fd_row0_col1\" class=\"data row0 col1\" >-1.9E-05</td>\n",
       "      <td id=\"T_101fd_row0_col2\" class=\"data row0 col2\" >1.811E+01</td>\n",
       "      <td id=\"T_101fd_row0_col3\" class=\"data row0 col3\" >3.9E-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_101fd_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_101fd_row1_col0\" class=\"data row1 col0\" >-4.7E-04</td>\n",
       "      <td id=\"T_101fd_row1_col1\" class=\"data row1 col1\" >-4.0E-05</td>\n",
       "      <td id=\"T_101fd_row1_col2\" class=\"data row1 col2\" >-1.001E+01</td>\n",
       "      <td id=\"T_101fd_row1_col3\" class=\"data row1 col3\" >1.8E-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_101fd_level0_row2\" class=\"row_heading level0 row2\" >Elapsed (s)</th>\n",
       "      <td id=\"T_101fd_row2_col0\" class=\"data row2 col0\" >1.666826</td>\n",
       "      <td id=\"T_101fd_row2_col1\" class=\"data row2 col1\" >0.669663</td>\n",
       "      <td id=\"T_101fd_row2_col2\" class=\"data row2 col2\" >0.948381</td>\n",
       "      <td id=\"T_101fd_row2_col3\" class=\"data row2 col3\" >4.890621</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Benzene</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_6d56e\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_6d56e_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, LDA)</th>\n",
       "      <th id=\"T_6d56e_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, LDA)</th>\n",
       "      <th id=\"T_6d56e_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, LDA</th>\n",
       "      <th id=\"T_6d56e_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, LDA)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_6d56e_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_6d56e_row0_col0\" class=\"data row0 col0\" >3.6E-06</td>\n",
       "      <td id=\"T_6d56e_row0_col1\" class=\"data row0 col1\" >2.4E-04</td>\n",
       "      <td id=\"T_6d56e_row0_col2\" class=\"data row0 col2\" >4.875E+01</td>\n",
       "      <td id=\"T_6d56e_row0_col3\" class=\"data row0 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6d56e_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_6d56e_row1_col0\" class=\"data row1 col0\" >-1.3E-03</td>\n",
       "      <td id=\"T_6d56e_row1_col1\" class=\"data row1 col1\" >-7.9E-03</td>\n",
       "      <td id=\"T_6d56e_row1_col2\" class=\"data row1 col2\" >-4.055E+01</td>\n",
       "      <td id=\"T_6d56e_row1_col3\" class=\"data row1 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6d56e_level0_row2\" class=\"row_heading level0 row2\" >Elapsed (s)</th>\n",
       "      <td id=\"T_6d56e_row2_col0\" class=\"data row2 col0\" >7.486318</td>\n",
       "      <td id=\"T_6d56e_row2_col1\" class=\"data row2 col1\" >2.860685</td>\n",
       "      <td id=\"T_6d56e_row2_col2\" class=\"data row2 col2\" >3.633616</td>\n",
       "      <td id=\"T_6d56e_row2_col3\" class=\"data row2 col3\" >nan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML(\"<h2>Derivatives computations results</h2>\"))\n",
    "for mol_name in molecules_for_correctness_tests:\n",
    "    df = pd.DataFrame(derivative_results_lda[mol_name])\n",
    "    html = df.style.to_html()\n",
    "    display(HTML(f\"<h3>{pretty_molecule_names[mol_name]}</h3>\" + html))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "derivative_results_pbe = {}\n",
    "\n",
    "for mol_name in molecules_for_correctness_tests:\n",
    "    derivative_results_pbe[mol_name] = {}\n",
    "    for method in derivative_correctness_results_pbe:\n",
    "        dictionary = {}\n",
    "        if method == \"pyscf_hellmann_feynman_pbe\":\n",
    "            values = [f'{value:.3E}' for value in derivative_correctness_results_pbe[method][mol_name][\"derivatives\"]]\n",
    "        else:\n",
    "            deltas = derivative_correctness_results_pbe[method][mol_name][\"derivatives\"] - derivative_correctness_results_pbe[\"pyscf_hellmann_feynman_pbe\"][mol_name][\"derivatives\"]\n",
    "            values = [f'{value:.1E}' for value in deltas]\n",
    "        dictionary = dict(zip(PBE_default_parameters_dict.keys(), values))\n",
    "        \n",
    "        mb_time = derivative_correctness_results_pbe[method][mol_name][\"time_d\"]\n",
    "        time_ = mb_time.average if hasattr(mb_time, \"average\") else mb_time\n",
    "        dictionary[\"Elapsed (s)\"] = time_\n",
    "\n",
    "        derivative_results_pbe[mol_name][pretty_derivative_method_names[method]] = dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h2>Derivatives computations results</h2>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>H<sub>2</sub></h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_87a8e\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_87a8e_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, PBE)</th>\n",
       "      <th id=\"T_87a8e_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, PBE)</th>\n",
       "      <th id=\"T_87a8e_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, PBE</th>\n",
       "      <th id=\"T_87a8e_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_87a8e_row0_col0\" class=\"data row0 col0\" >-3.7E-07</td>\n",
       "      <td id=\"T_87a8e_row0_col1\" class=\"data row0 col1\" >-2.6E-06</td>\n",
       "      <td id=\"T_87a8e_row0_col2\" class=\"data row0 col2\" >8.764E-01</td>\n",
       "      <td id=\"T_87a8e_row0_col3\" class=\"data row0 col3\" >0.0E+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_87a8e_row1_col0\" class=\"data row1 col0\" >-3.9E-05</td>\n",
       "      <td id=\"T_87a8e_row1_col1\" class=\"data row1 col1\" >-1.3E-06</td>\n",
       "      <td id=\"T_87a8e_row1_col2\" class=\"data row1 col2\" >1.757E+00</td>\n",
       "      <td id=\"T_87a8e_row1_col3\" class=\"data row1 col3\" >2.2E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row2\" class=\"row_heading level0 row2\" >PBE_beta</th>\n",
       "      <td id=\"T_87a8e_row2_col0\" class=\"data row2 col0\" >-1.8E-06</td>\n",
       "      <td id=\"T_87a8e_row2_col1\" class=\"data row2 col1\" >-6.4E-07</td>\n",
       "      <td id=\"T_87a8e_row2_col2\" class=\"data row2 col2\" >3.375E-01</td>\n",
       "      <td id=\"T_87a8e_row2_col3\" class=\"data row2 col3\" >1.7E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row3\" class=\"row_heading level0 row3\" >PBE_gamma</th>\n",
       "      <td id=\"T_87a8e_row3_col0\" class=\"data row3 col0\" >-3.1E-06</td>\n",
       "      <td id=\"T_87a8e_row3_col1\" class=\"data row3 col1\" >-8.9E-07</td>\n",
       "      <td id=\"T_87a8e_row3_col2\" class=\"data row3 col2\" >2.703E-01</td>\n",
       "      <td id=\"T_87a8e_row3_col3\" class=\"data row3 col3\" >1.1E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row4\" class=\"row_heading level0 row4\" >PBE_kappa</th>\n",
       "      <td id=\"T_87a8e_row4_col0\" class=\"data row4 col0\" >2.2E-07</td>\n",
       "      <td id=\"T_87a8e_row4_col1\" class=\"data row4 col1\" >7.1E-08</td>\n",
       "      <td id=\"T_87a8e_row4_col2\" class=\"data row4 col2\" >-3.089E-02</td>\n",
       "      <td id=\"T_87a8e_row4_col3\" class=\"data row4 col3\" >3.5E-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row5\" class=\"row_heading level0 row5\" >PBE_mu</th>\n",
       "      <td id=\"T_87a8e_row5_col0\" class=\"data row5 col0\" >9.3E-07</td>\n",
       "      <td id=\"T_87a8e_row5_col1\" class=\"data row5 col1\" >6.8E-07</td>\n",
       "      <td id=\"T_87a8e_row5_col2\" class=\"data row5 col2\" >-2.572E-01</td>\n",
       "      <td id=\"T_87a8e_row5_col3\" class=\"data row5 col3\" >-5.6E-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row6\" class=\"row_heading level0 row6\" >PW_a</th>\n",
       "      <td id=\"T_87a8e_row6_col0\" class=\"data row6 col0\" >4.5E-06</td>\n",
       "      <td id=\"T_87a8e_row6_col1\" class=\"data row6 col1\" >2.0E-06</td>\n",
       "      <td id=\"T_87a8e_row6_col2\" class=\"data row6 col2\" >-6.258E-01</td>\n",
       "      <td id=\"T_87a8e_row6_col3\" class=\"data row6 col3\" >-3.3E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row7\" class=\"row_heading level0 row7\" >PW_alpha</th>\n",
       "      <td id=\"T_87a8e_row7_col0\" class=\"data row7 col0\" >-1.3E-07</td>\n",
       "      <td id=\"T_87a8e_row7_col1\" class=\"data row7 col1\" >2.9E-08</td>\n",
       "      <td id=\"T_87a8e_row7_col2\" class=\"data row7 col2\" >-8.189E-02</td>\n",
       "      <td id=\"T_87a8e_row7_col3\" class=\"data row7 col3\" >0.0E+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row8\" class=\"row_heading level0 row8\" >PW_beta1</th>\n",
       "      <td id=\"T_87a8e_row8_col0\" class=\"data row8 col0\" >-1.9E-08</td>\n",
       "      <td id=\"T_87a8e_row8_col1\" class=\"data row8 col1\" >-9.1E-09</td>\n",
       "      <td id=\"T_87a8e_row8_col2\" class=\"data row8 col2\" >3.609E-03</td>\n",
       "      <td id=\"T_87a8e_row8_col3\" class=\"data row8 col3\" >-4.3E-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row9\" class=\"row_heading level0 row9\" >PW_beta2</th>\n",
       "      <td id=\"T_87a8e_row9_col0\" class=\"data row9 col0\" >-1.2E-08</td>\n",
       "      <td id=\"T_87a8e_row9_col1\" class=\"data row9 col1\" >-6.3E-09</td>\n",
       "      <td id=\"T_87a8e_row9_col2\" class=\"data row9 col2\" >4.289E-03</td>\n",
       "      <td id=\"T_87a8e_row9_col3\" class=\"data row9 col3\" >0.0E+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row10\" class=\"row_heading level0 row10\" >PW_beta3</th>\n",
       "      <td id=\"T_87a8e_row10_col0\" class=\"data row10 col0\" >-5.5E-09</td>\n",
       "      <td id=\"T_87a8e_row10_col1\" class=\"data row10 col1\" >-1.5E-09</td>\n",
       "      <td id=\"T_87a8e_row10_col2\" class=\"data row10 col2\" >5.237E-03</td>\n",
       "      <td id=\"T_87a8e_row10_col3\" class=\"data row10 col3\" >1.7E-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row11\" class=\"row_heading level0 row11\" >PW_beta4</th>\n",
       "      <td id=\"T_87a8e_row11_col0\" class=\"data row11 col0\" >3.7E-09</td>\n",
       "      <td id=\"T_87a8e_row11_col1\" class=\"data row11 col1\" >6.4E-09</td>\n",
       "      <td id=\"T_87a8e_row11_col2\" class=\"data row11 col2\" >6.601E-03</td>\n",
       "      <td id=\"T_87a8e_row11_col3\" class=\"data row11 col3\" >8.7E-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_87a8e_level0_row12\" class=\"row_heading level0 row12\" >Elapsed (s)</th>\n",
       "      <td id=\"T_87a8e_row12_col0\" class=\"data row12 col0\" >6.091702</td>\n",
       "      <td id=\"T_87a8e_row12_col1\" class=\"data row12 col1\" >0.245973</td>\n",
       "      <td id=\"T_87a8e_row12_col2\" class=\"data row12 col2\" >5.359846</td>\n",
       "      <td id=\"T_87a8e_row12_col3\" class=\"data row12 col3\" >39.904908</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Water</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_3785c\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_3785c_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, PBE)</th>\n",
       "      <th id=\"T_3785c_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, PBE)</th>\n",
       "      <th id=\"T_3785c_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, PBE</th>\n",
       "      <th id=\"T_3785c_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_3785c_row0_col0\" class=\"data row0 col0\" >2.6E-06</td>\n",
       "      <td id=\"T_3785c_row0_col1\" class=\"data row0 col1\" >-1.9E-05</td>\n",
       "      <td id=\"T_3785c_row0_col2\" class=\"data row0 col2\" >1.210E+01</td>\n",
       "      <td id=\"T_3785c_row0_col3\" class=\"data row0 col3\" >-1.1E-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_3785c_row1_col0\" class=\"data row1 col0\" >-5.3E-04</td>\n",
       "      <td id=\"T_3785c_row1_col1\" class=\"data row1 col1\" >3.9E-05</td>\n",
       "      <td id=\"T_3785c_row1_col2\" class=\"data row1 col2\" >-1.201E+01</td>\n",
       "      <td id=\"T_3785c_row1_col3\" class=\"data row1 col3\" >4.4E-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row2\" class=\"row_heading level0 row2\" >PBE_beta</th>\n",
       "      <td id=\"T_3785c_row2_col0\" class=\"data row2 col0\" >-1.0E-05</td>\n",
       "      <td id=\"T_3785c_row2_col1\" class=\"data row2 col1\" >-8.0E-06</td>\n",
       "      <td id=\"T_3785c_row2_col2\" class=\"data row2 col2\" >2.187E+00</td>\n",
       "      <td id=\"T_3785c_row2_col3\" class=\"data row2 col3\" >-4.4E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row3\" class=\"row_heading level0 row3\" >PBE_gamma</th>\n",
       "      <td id=\"T_3785c_row3_col0\" class=\"data row3 col0\" >-3.0E-05</td>\n",
       "      <td id=\"T_3785c_row3_col1\" class=\"data row3 col1\" >7.7E-06</td>\n",
       "      <td id=\"T_3785c_row3_col2\" class=\"data row3 col2\" >3.083E+00</td>\n",
       "      <td id=\"T_3785c_row3_col3\" class=\"data row3 col3\" >-3.1E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row4\" class=\"row_heading level0 row4\" >PBE_kappa</th>\n",
       "      <td id=\"T_3785c_row4_col0\" class=\"data row4 col0\" >1.4E-06</td>\n",
       "      <td id=\"T_3785c_row4_col1\" class=\"data row4 col1\" >-5.3E-07</td>\n",
       "      <td id=\"T_3785c_row4_col2\" class=\"data row4 col2\" >-1.932E-01</td>\n",
       "      <td id=\"T_3785c_row4_col3\" class=\"data row4 col3\" >2.8E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row5\" class=\"row_heading level0 row5\" >PBE_mu</th>\n",
       "      <td id=\"T_3785c_row5_col0\" class=\"data row5 col0\" >5.2E-06</td>\n",
       "      <td id=\"T_3785c_row5_col1\" class=\"data row5 col1\" >2.3E-06</td>\n",
       "      <td id=\"T_3785c_row5_col2\" class=\"data row5 col2\" >-3.019E+00</td>\n",
       "      <td id=\"T_3785c_row5_col3\" class=\"data row5 col3\" >2.7E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row6\" class=\"row_heading level0 row6\" >PW_a</th>\n",
       "      <td id=\"T_3785c_row6_col0\" class=\"data row6 col0\" >2.9E-05</td>\n",
       "      <td id=\"T_3785c_row6_col1\" class=\"data row6 col1\" >1.6E-05</td>\n",
       "      <td id=\"T_3785c_row6_col2\" class=\"data row6 col2\" >-6.701E+00</td>\n",
       "      <td id=\"T_3785c_row6_col3\" class=\"data row6 col3\" >1.8E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row7\" class=\"row_heading level0 row7\" >PW_alpha</th>\n",
       "      <td id=\"T_3785c_row7_col0\" class=\"data row7 col0\" >-1.9E-07</td>\n",
       "      <td id=\"T_3785c_row7_col1\" class=\"data row7 col1\" >1.4E-07</td>\n",
       "      <td id=\"T_3785c_row7_col2\" class=\"data row7 col2\" >-3.620E-01</td>\n",
       "      <td id=\"T_3785c_row7_col3\" class=\"data row7 col3\" >5.6E-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row8\" class=\"row_heading level0 row8\" >PW_beta1</th>\n",
       "      <td id=\"T_3785c_row8_col0\" class=\"data row8 col0\" >-1.6E-07</td>\n",
       "      <td id=\"T_3785c_row8_col1\" class=\"data row8 col1\" >-6.0E-08</td>\n",
       "      <td id=\"T_3785c_row8_col2\" class=\"data row8 col2\" >2.963E-02</td>\n",
       "      <td id=\"T_3785c_row8_col3\" class=\"data row8 col3\" >3.5E-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row9\" class=\"row_heading level0 row9\" >PW_beta2</th>\n",
       "      <td id=\"T_3785c_row9_col0\" class=\"data row9 col0\" >-5.2E-08</td>\n",
       "      <td id=\"T_3785c_row9_col1\" class=\"data row9 col1\" >-3.1E-08</td>\n",
       "      <td id=\"T_3785c_row9_col2\" class=\"data row9 col2\" >2.450E-02</td>\n",
       "      <td id=\"T_3785c_row9_col3\" class=\"data row9 col3\" >-3.5E-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row10\" class=\"row_heading level0 row10\" >PW_beta3</th>\n",
       "      <td id=\"T_3785c_row10_col0\" class=\"data row10 col0\" >-1.5E-08</td>\n",
       "      <td id=\"T_3785c_row10_col1\" class=\"data row10 col1\" >-6.0E-09</td>\n",
       "      <td id=\"T_3785c_row10_col2\" class=\"data row10 col2\" >2.253E-02</td>\n",
       "      <td id=\"T_3785c_row10_col3\" class=\"data row10 col3\" >-6.9E-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row11\" class=\"row_heading level0 row11\" >PW_beta4</th>\n",
       "      <td id=\"T_3785c_row11_col0\" class=\"data row11 col0\" >7.4E-09</td>\n",
       "      <td id=\"T_3785c_row11_col1\" class=\"data row11 col1\" >1.4E-08</td>\n",
       "      <td id=\"T_3785c_row11_col2\" class=\"data row11 col2\" >2.247E-02</td>\n",
       "      <td id=\"T_3785c_row11_col3\" class=\"data row11 col3\" >-1.7E-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_3785c_level0_row12\" class=\"row_heading level0 row12\" >Elapsed (s)</th>\n",
       "      <td id=\"T_3785c_row12_col0\" class=\"data row12 col0\" >10.967377</td>\n",
       "      <td id=\"T_3785c_row12_col1\" class=\"data row12 col1\" >0.573073</td>\n",
       "      <td id=\"T_3785c_row12_col2\" class=\"data row12 col2\" >6.108987</td>\n",
       "      <td id=\"T_3785c_row12_col3\" class=\"data row12 col3\" >43.658979</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Ethane</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_7d009\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_7d009_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, PBE)</th>\n",
       "      <th id=\"T_7d009_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, PBE)</th>\n",
       "      <th id=\"T_7d009_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, PBE</th>\n",
       "      <th id=\"T_7d009_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_7d009_row0_col0\" class=\"data row0 col0\" >3.8E-06</td>\n",
       "      <td id=\"T_7d009_row0_col1\" class=\"data row0 col1\" >-2.2E-05</td>\n",
       "      <td id=\"T_7d009_row0_col2\" class=\"data row0 col2\" >1.978E+01</td>\n",
       "      <td id=\"T_7d009_row0_col3\" class=\"data row0 col3\" >-7.1E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_7d009_row1_col0\" class=\"data row1 col0\" >-5.2E-04</td>\n",
       "      <td id=\"T_7d009_row1_col1\" class=\"data row1 col1\" >-3.9E-05</td>\n",
       "      <td id=\"T_7d009_row1_col2\" class=\"data row1 col2\" >-1.070E+01</td>\n",
       "      <td id=\"T_7d009_row1_col3\" class=\"data row1 col3\" >0.0E+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row2\" class=\"row_heading level0 row2\" >PBE_beta</th>\n",
       "      <td id=\"T_7d009_row2_col0\" class=\"data row2 col0\" >-1.7E-05</td>\n",
       "      <td id=\"T_7d009_row2_col1\" class=\"data row2 col1\" >2.8E-05</td>\n",
       "      <td id=\"T_7d009_row2_col2\" class=\"data row2 col2\" >3.655E+00</td>\n",
       "      <td id=\"T_7d009_row2_col3\" class=\"data row2 col3\" >6.7E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row3\" class=\"row_heading level0 row3\" >PBE_gamma</th>\n",
       "      <td id=\"T_7d009_row3_col0\" class=\"data row3 col0\" >-4.6E-05</td>\n",
       "      <td id=\"T_7d009_row3_col1\" class=\"data row3 col1\" >2.8E-05</td>\n",
       "      <td id=\"T_7d009_row3_col2\" class=\"data row3 col2\" >4.863E+00</td>\n",
       "      <td id=\"T_7d009_row3_col3\" class=\"data row3 col3\" >8.0E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row4\" class=\"row_heading level0 row4\" >PBE_kappa</th>\n",
       "      <td id=\"T_7d009_row4_col0\" class=\"data row4 col0\" >1.9E-06</td>\n",
       "      <td id=\"T_7d009_row4_col1\" class=\"data row4 col1\" >2.3E-06</td>\n",
       "      <td id=\"T_7d009_row4_col2\" class=\"data row4 col2\" >-2.707E-01</td>\n",
       "      <td id=\"T_7d009_row4_col3\" class=\"data row4 col3\" >9.4E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row5\" class=\"row_heading level0 row5\" >PBE_mu</th>\n",
       "      <td id=\"T_7d009_row5_col0\" class=\"data row5 col0\" >7.1E-06</td>\n",
       "      <td id=\"T_7d009_row5_col1\" class=\"data row5 col1\" >-1.1E-05</td>\n",
       "      <td id=\"T_7d009_row5_col2\" class=\"data row5 col2\" >-4.408E+00</td>\n",
       "      <td id=\"T_7d009_row5_col3\" class=\"data row5 col3\" >-2.7E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row6\" class=\"row_heading level0 row6\" >PW_a</th>\n",
       "      <td id=\"T_7d009_row6_col0\" class=\"data row6 col0\" >5.6E-05</td>\n",
       "      <td id=\"T_7d009_row6_col1\" class=\"data row6 col1\" >1.5E-05</td>\n",
       "      <td id=\"T_7d009_row6_col2\" class=\"data row6 col2\" >-1.237E+01</td>\n",
       "      <td id=\"T_7d009_row6_col3\" class=\"data row6 col3\" >-1.8E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row7\" class=\"row_heading level0 row7\" >PW_alpha</th>\n",
       "      <td id=\"T_7d009_row7_col0\" class=\"data row7 col0\" >-3.0E-07</td>\n",
       "      <td id=\"T_7d009_row7_col1\" class=\"data row7 col1\" >-4.2E-07</td>\n",
       "      <td id=\"T_7d009_row7_col2\" class=\"data row7 col2\" >-6.617E-01</td>\n",
       "      <td id=\"T_7d009_row7_col3\" class=\"data row7 col3\" >-2.4E-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row8\" class=\"row_heading level0 row8\" >PW_beta1</th>\n",
       "      <td id=\"T_7d009_row8_col0\" class=\"data row8 col0\" >-2.9E-07</td>\n",
       "      <td id=\"T_7d009_row8_col1\" class=\"data row8 col1\" >-6.1E-08</td>\n",
       "      <td id=\"T_7d009_row8_col2\" class=\"data row8 col2\" >5.571E-02</td>\n",
       "      <td id=\"T_7d009_row8_col3\" class=\"data row8 col3\" >1.4E-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row9\" class=\"row_heading level0 row9\" >PW_beta2</th>\n",
       "      <td id=\"T_7d009_row9_col0\" class=\"data row9 col0\" >-1.0E-07</td>\n",
       "      <td id=\"T_7d009_row9_col1\" class=\"data row9 col1\" >-1.9E-08</td>\n",
       "      <td id=\"T_7d009_row9_col2\" class=\"data row9 col2\" >4.613E-02</td>\n",
       "      <td id=\"T_7d009_row9_col3\" class=\"data row9 col3\" >6.2E-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row10\" class=\"row_heading level0 row10\" >PW_beta3</th>\n",
       "      <td id=\"T_7d009_row10_col0\" class=\"data row10 col0\" >-6.0E-08</td>\n",
       "      <td id=\"T_7d009_row10_col1\" class=\"data row10 col1\" >3.3E-08</td>\n",
       "      <td id=\"T_7d009_row10_col2\" class=\"data row10 col2\" >4.104E-02</td>\n",
       "      <td id=\"T_7d009_row10_col3\" class=\"data row10 col3\" >1.5E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row11\" class=\"row_heading level0 row11\" >PW_beta4</th>\n",
       "      <td id=\"T_7d009_row11_col0\" class=\"data row11 col0\" >-5.5E-08</td>\n",
       "      <td id=\"T_7d009_row11_col1\" class=\"data row11 col1\" >8.3E-08</td>\n",
       "      <td id=\"T_7d009_row11_col2\" class=\"data row11 col2\" >3.866E-02</td>\n",
       "      <td id=\"T_7d009_row11_col3\" class=\"data row11 col3\" >2.6E-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_7d009_level0_row12\" class=\"row_heading level0 row12\" >Elapsed (s)</th>\n",
       "      <td id=\"T_7d009_row12_col0\" class=\"data row12 col0\" >21.176273</td>\n",
       "      <td id=\"T_7d009_row12_col1\" class=\"data row12 col1\" >2.097231</td>\n",
       "      <td id=\"T_7d009_row12_col2\" class=\"data row12 col2\" >6.720547</td>\n",
       "      <td id=\"T_7d009_row12_col3\" class=\"data row12 col3\" >275.958514</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h3>Benzene</h3><style type=\"text/css\">\n",
       "</style>\n",
       "<table id=\"T_c2081\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_c2081_level0_col0\" class=\"col_heading level0 col0\" >Δ(FD of PySCF, PBE)</th>\n",
       "      <th id=\"T_c2081_level0_col1\" class=\"col_heading level0 col1\" >Δ(DQC, autograd, PBE)</th>\n",
       "      <th id=\"T_c2081_level0_col2\" class=\"col_heading level0 col2\" >Hellmann-Feyman PySCF, PBE</th>\n",
       "      <th id=\"T_c2081_level0_col3\" class=\"col_heading level0 col3\" >Δ(PySCF, Implicit differentiation, PBE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row0\" class=\"row_heading level0 row0\" >LDA_factor</th>\n",
       "      <td id=\"T_c2081_row0_col0\" class=\"data row0 col0\" >8.9E-06</td>\n",
       "      <td id=\"T_c2081_row0_col1\" class=\"data row0 col1\" >-8.0E-04</td>\n",
       "      <td id=\"T_c2081_row0_col2\" class=\"data row0 col2\" >5.297E+01</td>\n",
       "      <td id=\"T_c2081_row0_col3\" class=\"data row0 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row1\" class=\"row_heading level0 row1\" >LDA_power</th>\n",
       "      <td id=\"T_c2081_row1_col0\" class=\"data row1 col0\" >-1.5E-03</td>\n",
       "      <td id=\"T_c2081_row1_col1\" class=\"data row1 col1\" >-2.8E-03</td>\n",
       "      <td id=\"T_c2081_row1_col2\" class=\"data row1 col2\" >-4.403E+01</td>\n",
       "      <td id=\"T_c2081_row1_col3\" class=\"data row1 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row2\" class=\"row_heading level0 row2\" >PBE_beta</th>\n",
       "      <td id=\"T_c2081_row2_col0\" class=\"data row2 col0\" >-5.1E-05</td>\n",
       "      <td id=\"T_c2081_row2_col1\" class=\"data row2 col1\" >-8.8E-04</td>\n",
       "      <td id=\"T_c2081_row2_col2\" class=\"data row2 col2\" >8.813E+00</td>\n",
       "      <td id=\"T_c2081_row2_col3\" class=\"data row2 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row3\" class=\"row_heading level0 row3\" >PBE_gamma</th>\n",
       "      <td id=\"T_c2081_row3_col0\" class=\"data row3 col0\" >-1.3E-04</td>\n",
       "      <td id=\"T_c2081_row3_col1\" class=\"data row3 col1\" >-7.7E-04</td>\n",
       "      <td id=\"T_c2081_row3_col2\" class=\"data row3 col2\" >1.276E+01</td>\n",
       "      <td id=\"T_c2081_row3_col3\" class=\"data row3 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row4\" class=\"row_heading level0 row4\" >PBE_kappa</th>\n",
       "      <td id=\"T_c2081_row4_col0\" class=\"data row4 col0\" >5.1E-07</td>\n",
       "      <td id=\"T_c2081_row4_col1\" class=\"data row4 col1\" >-4.8E-05</td>\n",
       "      <td id=\"T_c2081_row4_col2\" class=\"data row4 col2\" >-6.250E-01</td>\n",
       "      <td id=\"T_c2081_row4_col3\" class=\"data row4 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row5\" class=\"row_heading level0 row5\" >PBE_mu</th>\n",
       "      <td id=\"T_c2081_row5_col0\" class=\"data row5 col0\" >2.6E-05</td>\n",
       "      <td id=\"T_c2081_row5_col1\" class=\"data row5 col1\" >7.5E-05</td>\n",
       "      <td id=\"T_c2081_row5_col2\" class=\"data row5 col2\" >-1.151E+01</td>\n",
       "      <td id=\"T_c2081_row5_col3\" class=\"data row5 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row6\" class=\"row_heading level0 row6\" >PW_a</th>\n",
       "      <td id=\"T_c2081_row6_col0\" class=\"data row6 col0\" >6.9E-05</td>\n",
       "      <td id=\"T_c2081_row6_col1\" class=\"data row6 col1\" >1.4E-03</td>\n",
       "      <td id=\"T_c2081_row6_col2\" class=\"data row6 col2\" >-3.212E+01</td>\n",
       "      <td id=\"T_c2081_row6_col3\" class=\"data row6 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row7\" class=\"row_heading level0 row7\" >PW_alpha</th>\n",
       "      <td id=\"T_c2081_row7_col0\" class=\"data row7 col0\" >1.0E-05</td>\n",
       "      <td id=\"T_c2081_row7_col1\" class=\"data row7 col1\" >1.2E-05</td>\n",
       "      <td id=\"T_c2081_row7_col2\" class=\"data row7 col2\" >-1.491E+00</td>\n",
       "      <td id=\"T_c2081_row7_col3\" class=\"data row7 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row8\" class=\"row_heading level0 row8\" >PW_beta1</th>\n",
       "      <td id=\"T_c2081_row8_col0\" class=\"data row8 col0\" >-4.8E-07</td>\n",
       "      <td id=\"T_c2081_row8_col1\" class=\"data row8 col1\" >-6.6E-06</td>\n",
       "      <td id=\"T_c2081_row8_col2\" class=\"data row8 col2\" >1.397E-01</td>\n",
       "      <td id=\"T_c2081_row8_col3\" class=\"data row8 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row9\" class=\"row_heading level0 row9\" >PW_beta2</th>\n",
       "      <td id=\"T_c2081_row9_col0\" class=\"data row9 col0\" >-9.2E-07</td>\n",
       "      <td id=\"T_c2081_row9_col1\" class=\"data row9 col1\" >-5.0E-06</td>\n",
       "      <td id=\"T_c2081_row9_col2\" class=\"data row9 col2\" >1.090E-01</td>\n",
       "      <td id=\"T_c2081_row9_col3\" class=\"data row9 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row10\" class=\"row_heading level0 row10\" >PW_beta3</th>\n",
       "      <td id=\"T_c2081_row10_col0\" class=\"data row10 col0\" >-3.0E-06</td>\n",
       "      <td id=\"T_c2081_row10_col1\" class=\"data row10 col1\" >-9.5E-07</td>\n",
       "      <td id=\"T_c2081_row10_col2\" class=\"data row10 col2\" >9.180E-02</td>\n",
       "      <td id=\"T_c2081_row10_col3\" class=\"data row10 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row11\" class=\"row_heading level0 row11\" >PW_beta4</th>\n",
       "      <td id=\"T_c2081_row11_col0\" class=\"data row11 col0\" >-8.6E-07</td>\n",
       "      <td id=\"T_c2081_row11_col1\" class=\"data row11 col1\" >1.0E-05</td>\n",
       "      <td id=\"T_c2081_row11_col2\" class=\"data row11 col2\" >8.251E-02</td>\n",
       "      <td id=\"T_c2081_row11_col3\" class=\"data row11 col3\" >NAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_c2081_level0_row12\" class=\"row_heading level0 row12\" >Elapsed (s)</th>\n",
       "      <td id=\"T_c2081_row12_col0\" class=\"data row12 col0\" >55.163607</td>\n",
       "      <td id=\"T_c2081_row12_col1\" class=\"data row12 col1\" >8.007707</td>\n",
       "      <td id=\"T_c2081_row12_col2\" class=\"data row12 col2\" >11.205980</td>\n",
       "      <td id=\"T_c2081_row12_col3\" class=\"data row12 col3\" >nan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML(\"<h2>Derivatives computations results</h2>\"))\n",
    "for mol_name in molecules_for_correctness_tests:\n",
    "    df = pd.DataFrame(derivative_results_pbe[mol_name])\n",
    "    html = df.style.to_html()\n",
    "    display(HTML(f\"<h3>{pretty_molecule_names[mol_name]}</h3>\" + html))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparison of perfomance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@functools.cache\n",
    "def pyscf_adjoint_time_of_derivatives_calculation(molecule, method_name):\n",
    "    if method_name == 'LDAX':\n",
    "        XCConstructor = PySCFLDAXConstructor\n",
    "        params = LDAX_default_parameters_dict\n",
    "    elif method_name == 'PBE':\n",
    "        XCConstructor = PySCFPBEConstructor\n",
    "        params = PBE_default_parameters_dict\n",
    "    else:\n",
    "        raise f\"Functional {method_name} was not implemented\"\n",
    "\n",
    "    calc = HellmannFeynmanTotalDerivative(AnalyticalPartialDerivativeCalculator(\n",
    "    PySCFDataAdapter(XCConstructor, params, molecule)))\n",
    "\n",
    "    calc.construct_total_dE_wrt_dtheta() # numba compilation\n",
    "    dur = %timeit -o -q -r100 calc.construct_total_dE_wrt_dtheta()\n",
    "        \n",
    "    return dur.average, dur.stdev, dur"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "@functools.cache\n",
    "def dqc_autograd_time_of_derivatives_calculation(molecule, method_name):\n",
    "    if method_name == 'LDAX':\n",
    "        XCConstructor = DQCLDAXContructor\n",
    "        params = LDAX_default_parameters_dict\n",
    "    elif method_name == 'PBE':\n",
    "        XCConstructor = DQCPBEContructor\n",
    "        params = PBE_default_parameters_dict\n",
    "    else:\n",
    "        raise f\"Functional {method_name} was not implemented\"\n",
    "        \n",
    "    time_with_derivatives_calculation = %timeit -o -q -r100 IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(XCConstructor, params, molecule)).construct_total_dE_wrt_dtheta()\n",
    "    time_without_derivatives_calculation = %timeit -o -q -r100 DQCAdapterNograd(XCConstructor, params, molecule)\n",
    "\n",
    "    total_average = time_with_derivatives_calculation.average - time_without_derivatives_calculation.average\n",
    "    total_std_dev = numpy.sqrt(time_with_derivatives_calculation.stdev ** 2 + time_without_derivatives_calculation.stdev ** 2)\n",
    "    \n",
    "    return total_average, total_std_dev, (time_with_derivatives_calculation, time_without_derivatives_calculation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('structures/alkanes_molecules.txt') as alkanes_molecules_file: \n",
    "    alkanes_molecules_file_as_string = alkanes_molecules_file.read() \n",
    "\n",
    "alkanes_molecules = literal_eval(alkanes_molecules_file_as_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "alkanes_performance = {\n",
    "    \"pyscf_adjoint_lda\":    {},\n",
    "    \"dqc_onlygrad_lda\":     {},\n",
    "    \"pyscf_adjoint_pbe\":    {},\n",
    "    \"dqc_onlygrad_pbe\":     {},\n",
    "}\n",
    "\n",
    "for data_of_method in alkanes_performance.values():\n",
    "    for mol_name in alkanes_molecules:\n",
    "        data_of_method[mol_name] = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For n-С4H10:\n",
      "* PySCF 'adjoint':\n",
      "27.1 ms ± 561 µs per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "3.06 s ± 123 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "2.21 s ± 126 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С6H14:\n",
      "* PySCF 'adjoint':\n",
      "39.7 ms ± 670 µs per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "8.8 s ± 210 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "6.81 s ± 201 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С8H18:\n",
      "* PySCF 'adjoint':\n",
      "52.1 ms ± 947 µs per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "17.8 s ± 498 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "14 s ± 439 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С10H22:\n",
      "* PySCF 'adjoint':\n",
      "63.3 ms ± 456 µs per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "32 s ± 822 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "26.1 s ± 643 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С12H26:\n",
      "* PySCF 'adjoint':\n",
      "79.8 ms ± 6.45 ms per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "57.7 s ± 1.54 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "48.9 s ± 1.09 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С14H30:\n",
      "* PySCF 'adjoint':\n",
      "88.4 ms ± 1.61 ms per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "1min 38s ± 1.66 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "1min 25s ± 1.69 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С16H34:\n",
      "* PySCF 'adjoint':\n",
      "102 ms ± 2.9 ms per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "5min 30s ± 11.6 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "5min 6s ± 9.35 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С18H38:\n",
      "* PySCF 'adjoint':\n",
      "116 ms ± 5.65 ms per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "5min 25s ± 14.6 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "4min 54s ± 12.3 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С20H42:\n",
      "* PySCF 'adjoint':\n",
      "127 ms ± 2.78 ms per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "5min 49s ± 7.84 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "5min 2s ± 6.26 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С22H46:\n",
      "* PySCF 'adjoint':\n",
      "138 ms ± 1.86 ms per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "* DQC:\n",
      "8min 45s ± 17.5 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "7min 46s ± 11 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С24H50:\n",
      "* PySCF 'adjoint':\n",
      "149 ms ± 240 µs per loop (mean ± std. dev. of 100 runs, 10 loops each)\n",
      "---><---\n",
      "For n-С32H66:\n",
      "* PySCF 'adjoint':\n",
      "200 ms ± 2.35 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С48H98:\n",
      "* PySCF 'adjoint':\n",
      "297 ms ± 4.17 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С64H130:\n",
      "* PySCF 'adjoint':\n",
      "398 ms ± 5.7 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n"
     ]
    }
   ],
   "source": [
    "for mol_name, mol in alkanes_molecules.items():\n",
    "    gc.collect()\n",
    "    print(f\"For n-С{mol_name}H{2*mol_name+2}:\")\n",
    "\n",
    "    print(\"* PySCF 'adjoint':\")\n",
    "    mean, std_dev, all_times = pyscf_adjoint_time_of_derivatives_calculation(mol, \"LDAX\")\n",
    "    alkanes_performance[\"pyscf_adjoint_lda\"][mol_name][\"mean\"] = mean\n",
    "    alkanes_performance[\"pyscf_adjoint_lda\"][mol_name][\"std_dev\"] = std_dev\n",
    "    print(all_times)\n",
    "\n",
    "    if mol_name <= 22:\n",
    "        print(\"* DQC:\")\n",
    "        mean, std_dev, all_times = dqc_autograd_time_of_derivatives_calculation(mol, \"LDAX\")\n",
    "        alkanes_performance[\"dqc_onlygrad_lda\"][mol_name][\"mean\"] = mean\n",
    "        alkanes_performance[\"dqc_onlygrad_lda\"][mol_name][\"std_dev\"] = std_dev\n",
    "        print(all_times[0])\n",
    "        print(all_times[1])\n",
    "    \n",
    "    print(\"---><---\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For n-С4H10:\n",
      "* PySCF 'adjoint':\n",
      "291 ms ± 8.84 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "7.86 s ± 440 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "5.28 s ± 310 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С6H14:\n",
      "* PySCF 'adjoint':\n",
      "412 ms ± 8.97 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "19.3 s ± 910 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "13 s ± 563 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С8H18:\n",
      "* PySCF 'adjoint':\n",
      "532 ms ± 10.3 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "33.8 s ± 1.68 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "24.1 s ± 1.32 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С10H22:\n",
      "* PySCF 'adjoint':\n",
      "661 ms ± 7.96 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "56.4 s ± 3.55 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "41.9 s ± 2.48 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С12H26:\n",
      "* PySCF 'adjoint':\n",
      "783 ms ± 5.72 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "1min 31s ± 5.3 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "1min 11s ± 3.2 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С14H30:\n",
      "* PySCF 'adjoint':\n",
      "915 ms ± 9.69 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "2min 30s ± 7.99 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "2min ± 5.51 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С16H34:\n",
      "* PySCF 'adjoint':\n",
      "1.07 s ± 56.9 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "3min 42s ± 9.65 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "3min 11s ± 7.62 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С18H38:\n",
      "* PySCF 'adjoint':\n",
      "1.17 s ± 26.8 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "6min 39s ± 10.5 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "5min 50s ± 17.7 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С20H42:\n",
      "* PySCF 'adjoint':\n",
      "1.3 s ± 41.8 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "7min 33s ± 25.4 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "6min 1s ± 9.93 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С22H46:\n",
      "* PySCF 'adjoint':\n",
      "1.43 s ± 28.4 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "* DQC:\n",
      "10min 51s ± 13.7 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "9min 11s ± 10 s per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С24H50:\n",
      "* PySCF 'adjoint':\n",
      "1.57 s ± 18.2 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С32H66:\n",
      "* PySCF 'adjoint':\n",
      "2.09 s ± 33.2 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С48H98:\n",
      "* PySCF 'adjoint':\n",
      "3.21 s ± 43.9 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n",
      "For n-С64H130:\n",
      "* PySCF 'adjoint':\n",
      "4.27 s ± 37.8 ms per loop (mean ± std. dev. of 100 runs, 1 loop each)\n",
      "---><---\n"
     ]
    }
   ],
   "source": [
    "for mol_name, mol in alkanes_molecules.items():\n",
    "    gc.collect()\n",
    "    print(f\"For n-С{mol_name}H{2*mol_name+2}:\")\n",
    "\n",
    "    print(\"* PySCF 'adjoint':\")\n",
    "    mean, std_dev, all_times = pyscf_adjoint_time_of_derivatives_calculation(mol, \"PBE\")\n",
    "    alkanes_performance[\"pyscf_adjoint_pbe\"][mol_name][\"mean\"] = mean\n",
    "    alkanes_performance[\"pyscf_adjoint_pbe\"][mol_name][\"std_dev\"] = std_dev\n",
    "    print(all_times)\n",
    "\n",
    "    if mol_name <= 22:\n",
    "        print(\"* DQC:\")\n",
    "        mean, std_dev, all_times = dqc_autograd_time_of_derivatives_calculation(mol, \"PBE\")\n",
    "        alkanes_performance[\"dqc_onlygrad_pbe\"][mol_name][\"mean\"] = mean\n",
    "        alkanes_performance[\"dqc_onlygrad_pbe\"][mol_name][\"std_dev\"] = std_dev\n",
    "        print(all_times[0])\n",
    "        print(all_times[1])\n",
    "    \n",
    "    print(\"---><---\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1kAAAHHCAYAAABN8pIpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB5FUlEQVR4nO3dd1iTV/sH8G/YGwVlKUuciAtR66qrVtyzjlYER/u2pUNtHW3fqh2votbRVrTaWkdtq1arVatYf+7W1gFiVbSO4gZxMmUl5/fHI4FAgAQSksD3c11ckpMn57nzkGBuzjn3kQkhBIiIiIiIiEgnzAwdABERERERUXXCJIuIiIiIiEiHmGQRERERERHpEJMsIiIiIiIiHWKSRUREREREpENMsoiIiIiIiHSISRYREREREZEOMckiIiIiIiLSISZZREREREREOsQki4iIiIiISIeYZBEREREREekQkywiKpdMJtPo69ChQ1i7di1kMhmuXbtm6LC1tmnTJjRv3hy2traQyWSIj4/X6vFz5syBTCbTeVxVcU2PHTuGOXPm4PHjx1V+7qqg7+dRmf5Lu/aV7beqVOR98/fff2P8+PHw9/eHjY0NHBwcEBwcjAULFuDhw4canbfg2pw6dUrt/QMGDICfn58Wz8T4lfecix5T8GVjYwMPDw/06NED8+bNQ0pKSpnn+OKLLyCTyRAUFKTr8IlqFAtDB0BExu/PP/9Uuf3JJ5/g4MGDOHDggEp7YGAgmjdvjj///BOenp5VGWKl3bt3D2FhYQgNDcXy5cthbW2Nxo0bGzosAED//v31fk2PHTuGjz76CBEREahVq1aVnrumK+3aA8Z//Svyvvn666/x+uuvo0mTJpg2bRoCAwORl5eHU6dO4auvvsKff/6Jbdu2VdEzqL7WrFmDpk2bIi8vDykpKfj9998xf/58fPbZZ9i0aROee+45tY/79ttvAQDnz5/H8ePH0aFDh6oMm6jaYJJFROV65plnVG7XrVsXZmZmJdqL3m9qLl26hLy8PIwdOxbdunUzdDgAgKysLNjZ2aFu3boGu6aGPDcZ//XX9n3z559/4rXXXkPv3r2xfft2WFtbK+/r3bs33nnnHcTExOgz5BojKCgIISEhytvDhw/HlClT0KVLFwwbNgyXL1+Gu7u7ymNOnTqFM2fOoH///vj111+xevVqJllEFcTpgkSkU8WnNxVMofv777/xwgsvwNnZGS4uLpg6dSry8/Pxzz//IDQ0FI6OjvDz88OCBQvU9nv58mW8+OKLcHNzg7W1NZo1a4bo6GiNYvr999/Rq1cvODo6ws7ODp06dcKvv/6qvD8iIgJdunQBAIwaNQoymQzdu3cvs89ff/0VrVu3hrW1Nfz9/fHZZ5+VeqwmsRdcp7i4OIwYMQK1a9dGQEAAgJLXdPv27ZDJZNi/f3+Jc61YsUJ5vQHgypUrGD9+PBo1agQ7OzvUq1cPAwcOxNmzZ1XOPW3aNACAv79/mdM/tTm3ps+9NBcvXsSYMWPg7u4Oa2tr+Pj4YNy4ccjJyVEeo8nzq0z/ERERaqecaTI1tLLXHlA/XbC813PR+M6fP48xY8bA2dkZ7u7umDBhAlJTU8u9NpqcpyLvm7lz50Imk2HVqlUqCVYBKysrDBo0SKP4SHs+Pj5YtGgR0tPTsXLlyhL3r169GgAQFRWFTp06YePGjcjKyqrqMImqBSZZRFQlRo4ciVatWmHr1q14+eWXsWTJEkyZMgVDhgxB//79sW3bNvTs2RMzZszAzz//rPLYhIQEtGvXDufOncOiRYuwa9cu9O/fH2+99RY++uijMs97+PBh9OzZE6mpqVi9ejV+/PFHODo6YuDAgdi0aRMA4MMPP1R+8J87dy7+/PNPLF++vNQ+9+/fj8GDB8PR0REbN27EwoULsXnzZqxZs6bEsdrGPmzYMDRs2BA//fQTvvrqK7XnHzBgANzc3NSeb+3atQgODkbLli0BAHfu3IGrqyuioqIQExOD6OhoWFhYoEOHDvjnn38AAJMmTcKbb74JAPj555/x559/4s8//0RwcHClzl2Zn9uZM2fQrl07/PXXX/j444+xZ88ezJs3Dzk5OcjNzVUep8nzq0z/laHraw9o9nouavjw4WjcuDG2bt2KmTNn4ocffsCUKVPKjV0f7xu5XI4DBw6gbdu28Pb2LjcGTcnlcuTn55f4EkLo7BzVSb9+/WBubo4jR46otD958gQ//vgj2rVrh6CgIEyYMAHp6en46aefDBQpkYkTRERaCg8PF/b29mrvW7NmjQAgEhMThRBCzJ49WwAQixYtUjmudevWAoD4+eeflW15eXmibt26YtiwYSrH9unTR9SvX1+kpqaqtL/xxhvCxsZGPHz4sNRYn3nmGeHm5ibS09OVbfn5+SIoKEjUr19fKBQKIYQQBw8eFADETz/9VO7z79Chg/Dy8hJPnjxRtqWlpQkXFxdR/NeqprEXXKdZs2aVOF/xayqEEFOnThW2trbi8ePHyraEhAQBQHz55Zelxp6fny9yc3NFo0aNxJQpU5TtCxcuLHGOyp67Mj+3nj17ilq1aomUlJRSj9Hm+RV/Hpr2Hx4eLnx9fUu0F/y8Sutfm9hKu/bq+tX09VwQ34IFC1T6e/3114WNjY3yuNLo432TnJwsAIjRo0eXe6wmCq5NWV9Ff3bZ2dkiIiJC1K9fXzg6OooOHTqIP/74QyexlEbX5yx4zidPnqzUMe7u7qJZs2YqbevXrxcAxFdffSWEECI9PV04ODiIrl27VjheopqMI1lEVCUGDBigcrtZs2aQyWTo27evss3CwgINGzbE9evXlW3Z2dnYv38/hg4dCjs7O5W/VPfr1w/Z2dn466+/1J4zMzMTx48fx4gRI+Dg4KBsNzc3R1hYGG7dulXmaEdpfZ48eRLDhg2DjY2Nsr3gr/xFVST24cOHaxTHhAkT8OTJE5XRizVr1sDa2hovvviisi0/Px9z585FYGAgrKysYGFhASsrK1y+fBkXLlzQ6rlrc+7K/NyysrJw+PBhjBw5stz1SBV5ftr0Xxm6vvYVeT0Xn3rXsmVLZGdnl1lhTh/vG31av349Tp48WeKrYCpjgfz8fPj7++OPP/7A48eP8dprr2HQoEGVmg4XERGBtWvXlnq/Ps6pC0LNKN/q1atha2uL0aNHAwAcHBzwwgsv4OjRo7h8+XJVh0hk8phkEVGVcHFxUbltZWUFOzs7lUSloD07O1t5+8GDB8jPz8eXX34JS0tLla9+/foBAO7fv6/2nI8ePYIQQm1lNi8vL2X/2nj06BEUCgU8PDxK3Fe8rSKxa1pFrnnz5mjXrp1y2p5cLseGDRswePBglWs9depUfPjhhxgyZAh27tyJ48eP4+TJk2jVqhWePHmi1XPX5tyV/bnJ5XLUr1+/3Fgq8vy06b8ydH3tK/J6dnV1VbldsA6qrPPr430DAHXq1IGdnR0SExM1On7RokUYM2YMJkyYgFq1aiEwMBBXrlwpcVyzZs0QEhJS4svZ2VnlOHt7e8yaNQs+Pj4wMzNDeHg4FAqFMoHQ9HzaKO+chpCZmYkHDx4of5aAtH7wyJEj6N+/P4QQePz4MR4/fowRI0YAKKw4SESaY3VBIjJqtWvXVv4FPTIyUu0x/v7+pT7WzMwMSUlJJe67c+cOAOmDn7bxyGQyJCcnl7iveFtFYtdmn63x48fj9ddfx4ULF/Dvv/8iKSkJ48ePVzlmw4YNGDduHObOnavSfv/+/RLlwrVR3rkr83NzcXGBubk5bt26VW4cFXl+2vRvY2OjUgijaP/6iK0s+ng9V+V5zM3N0atXL+zZswe3bt0qN8k9e/Ysjh49ip9++gnffPMNxo0bh6+//hrz58/X+tzqXLx4EU+ePFEWmNH3+dSd0xB+/fVXyOVylSIl3377LYQQ2LJlC7Zs2VLiMevWrcOnn34Kc3PzKoyUyLRxJIuIjJqdnR169OiB06dPo2XLlmr/Yl38r/UF7O3t0aFDB/z8888qf7lXKBTYsGED6tevr/VeWPb29mjfvj1+/vlnlRG39PR07Ny5U2exa2LMmDGwsbHB2rVrsXbtWtSrVw/PP/+8yjEymaxEFbdff/0Vt2/fVmnTZIRDm3NX5rnb2tqiW7du+Omnn8pNZjR9fhXt38/PDykpKbh7966yLTc3F3v37i3zcdrEpum118fruarP895770EIgZdfflltgZG8vDzl++js2bOYM2cOOnbsCDMzMwQEBOismEVWVhbCwsLw3//+VzklUp/nK+2cVe3GjRt499134ezsjP/85z8ApJHodevWISAgAAcPHizx9c477yApKQl79uwxSMxEpoojWURk9D7//HN06dIFXbt2xWuvvQY/Pz+kp6fjypUr2LlzZ4lNkYuaN28eevfujR49euDdd9+FlZUVli9fjnPnzuHHH3/UauSowCeffILQ0FDlvj5yuRzz58+Hvb09Hj58qLPYy1OrVi0MHToUa9euxePHj/Huu+/CzEz1b2cDBgzA2rVr0bRpU7Rs2RKxsbFYuHBhiVGEFi1aKOMNDw+HpaUlmjRpUqlzV+a5L168GF26dEGHDh0wc+ZMNGzYEHfv3sWOHTuwcuVKODo6avX8Ktr/qFGjMGvWLIwePRrTpk1DdnY2vvjiC8jl8jL71ya20q59QQxF6eP1rI6+ztOxY0esWLECr7/+Otq2bYvXXnsNzZs3R15eHk6fPo1Vq1YhKCgI/fr1Q0JCgnJqKSBtjlv0dkXl5eVh5MiRCAwMxPvvvw9ASjQ0Pd+AAQPw+++/A5ASp82bN2Py5MkAgJkzZ2LmzJkanbMomUyGbt26KUv3l+fAgQMqZf0LFI333LlzynWQKSkpOHr0KNasWQNzc3Ns27ZNuR5xz549uHPnDubPn6+2BH9QUBCWLVuG1atXl1hbS0RlMFjJDSIyWRWpLnjv3j2N+ujWrZto3rx5ifbExEQxYcIEUa9ePWFpaSnq1q0rOnXqJD799NNy4z169Kjo2bOnsLe3F7a2tuKZZ54RO3fuVDlGmyppQgixY8cO0bJlS2FlZSV8fHxEVFRUiYpz2sRe2nUSouzKdb/99puyktqlS5dK3P/o0SMxceJE4ebmJuzs7ESXLl3E0aNHRbdu3US3bt1Ujn3vvfeEl5eXMDMzEwDEwYMHK3VuTZ97aRISEsQLL7wgXF1dldc5IiJCZGdna/381D0PTfoXQojdu3eL1q1bC1tbW9GgQQOxbNkyjaoLVvbal9avJq/n0l5PmlRB1OY82r5vCsTHx4vw8HDh4+MjrKyshL29vWjTpo2YNWuWSElJERcvXhR169ZVeUyjRo1UKuaVV0Wvf//+JSpDyuVyMXr0aDFo0CCRl5enbNfkfOqEh4eLNWvWlHlMaecskJ6ernHVxfIqKiYmJpY4xsrKSri5uYlu3bqJuXPnlqioOWTIEGFlZVVmpc3Ro0cLCwsLkZycXG6MRCSRCcGNJIiIiMh4bNmyBatWrcJvv/0GQBoxqlWrFlJTU2Fra1vhfl9++WVcvnwZMTExKkV3Knq+iIgIdO/eHREREVqfs8Du3bsxYMAAnDlzRjmqSUSmj9MFiYiIyKicPXsWrVq1Ut4+f/48AgICKpVgXb9+Hd988w1sbGxUCnfs2bNHL+cr75xdu3YFABw8eBCjR49mgkVUzXAki4iIiIiISIdYXZCIiIiIiEiHmGQRERERERHpEJMsIiIiIiIiHWKSRUREREREpEOsLlgGhUKBO3fuwNHRUWcbPBIRERERkekRQiA9PR1eXl4wMyt7rIpJVhnu3LkDb29vQ4dBRERERERG4ubNm6hfv36Zx1T7JOvmzZsICwtDSkoKLCws8OGHH+KFF17Q6LGOjo7KPpycnPQZJhERERERGbG0tDR4e3src4SyVPt9spKSknD37l20bt0aKSkpCA4Oxj///AN7e/tyH5uWlgZnZ2ekpqYyySIiIiIiqsG0yQ2q/UiWp6cnPD09AQBubm5wcXHBw4cPNUqyiIiIiIiItGX01QWPHDmCgQMHwsvLCzKZDNu3by9xzPLly+Hv7w8bGxu0bdsWR48eVdvXqVOnoFAouM6KiIiIiIj0xuiTrMzMTLRq1QrLli1Te/+mTZswefJkfPDBBzh9+jS6du2Kvn374saNGyrHPXjwAOPGjcOqVauqImwiIiIiIqqhTGpNlkwmw7Zt2zBkyBBlW4cOHRAcHIwVK1Yo25o1a4YhQ4Zg3rx5AICcnBz07t0bL7/8MsLCwkrtPycnBzk5OcrbBYvbuCaLiIiIiKhm02ZNltGPZJUlNzcXsbGxeP7551Xan3/+eRw7dgyAVM8+IiICPXv2LDPBAoB58+bB2dlZ+cVphUREREREpC2TTrLu378PuVwOd3d3lXZ3d3ckJycDAP744w9s2rQJ27dvR+vWrdG6dWucPXtWbX/vvfceUlNTlV83b97U+3MgIiIiIqLqpVpUF5TJZCq3hRDKti5dukChUGjUj7W1NaytrXUeHxERERER1RwmPZJVp04dmJubK0etCqSkpJQY3SIiIiIiIqoKJp1kWVlZoW3btti3b59K+759+9CpUycDRUVERERERDWZ0U8XzMjIwJUrV5S3ExMTER8fDxcXF/j4+GDq1KkICwtDSEgIOnbsiFWrVuHGjRt49dVXK3zO6OhoREdHQy6X6+IpEBERERFRDWL0JdwPHTqEHj16lGgPDw/H2rVrAUibES9YsABJSUkICgrCkiVL8Oyzz1b63NqUaSQiIiIioupLm9zA6JMsQ2KSRUREREREQA3aJ4uIiIiIiMjYMMkiIiIiIiLSISZZakRHRyMwMBDt2rUzdChERESmLzcTmOMsfeVmGjoaIiK9Y5KlRmRkJBISEnDy5ElDh0JERERERCaGSRYREREREZEOMckiIiIiIiLSISZZREREREREOsQkSw0WviAiIiIioopikqUGC18QEREREVFFMckiIiIiIiLSISZZREREpF8KeeH314+p3iYiqoaYZBEREZH+JOwAotsX3v5+BLA0SGonIqqmmGQRERGRfiTsADaPA9KTVNvTkqR2JlpEVE0xySIiIiLdU8iBmBkAhJo7n7bFzOTUQSKqlphkqcES7kRERJV0/RiQdqeMAwSQdls6joiommGSpQZLuBMREVVSxl3dHkdEZEKYZBEREdV0uZnAHGfpKzdTN306uOv2OCIiE8Iki4iIiHTPtxPg5AVAVsoBMsCpnnQcEVE1wySLiIiIdM/MHAid//RG8UTr6e3QKOk4IqJqhkkWERFRTaevzYIDBwEj1wOOHqrtTl5Se+Ag3ZyHiMjIMMkiIiKqyfS9WXDgIGDS/sLbL20BJp9lgkVE1RqTLDVYwp2IiGoEfW8WnHwO2DUFiC7y/6lHS04RJKJqTyaEULdLIAFIS0uDs7MzUlNT4eTkZOhwiIioJsnNBOZ6Sd+/fwewstdt/wq5NGJV6l5WMmla3+Sz2iVF+TlScnbyG+DmXyXvf/cK4FC3QiETERmSNrmBRRXFRERERMZEm82C/buW39+j60DsGiDuOyDrvtRmZgE0HQC0GStNQwQAK7tKh05EZOyYZBEREdVEutgsWCEHruwHTq0GLu0F8HRyjKMXEDIeCB4nFb3Q1d5bREQmgkkWERFRTVSZzYIzHwCnvwNOfQs8vl7Y3qAH0G4i0LgvYM6PGERUc/E3IBERUU1UsFlwWhKUI1Aqnq7JKtgsWAjg1klprdX5bYA8V2q3cQZajwVCJgB1GlZV9ERERo1JFhERUU1UsFnw5nGQNgcummgV2Sw47wlw9ifg5Grg7tnCQ7zaAO0mAc2HcZ0VEVExTLKIiIhqqoLNgvdMVy3j7uQFdHwTuHYU+CUSyEmT2i1sgKARQLsJQL22homZiMgEMMkiIiKqyQIHAQ26A1He0u3OU6RpgXtnFh7jEiBNB2z9ImDnYpAwiYhMCZMsIiKimq5oBcE/lkj/ysyAJv2kQhb+3QEzM0NERkRkkphkqREdHY3o6GjI5XJDh0JERKQfCgXw70GpQuA/uwvb7d2AthFA23DAub7BwiMiMmVMstSIjIxEZGSkcldnIiKqwXIzgble0vfv3wGs7A0bT2VlPQTif5D2tnr4b8n73zgJ2Naq8rCIiKoTJllEREQ1we1YqULgua1AfrbUZu0EtBojfX3dXWozt9T9ua3sgTmpuu+XiMhIMckiIiKqrnKzpKTq5DdAUnxhu0cLqfx60AjA2kFqYxJERKQzTLKIiIiqm/uXpbVW8d8D2U+TJ3MraU+rdhOB+u0AmcywMRIRVWNMsoiIiKoDeb5UwOLkN0Di4cL2Wr5SYtV6LGDvarj4iIhqECZZRERExkhRpMLt9WNAQE/AzLzkcWlJQNw6IHYdkH7naaMMaBwqJVcBvVh+nYioijHJIiIiMjYJO4A90wtvfz8CcPICQudLmwcLASQekUatLv4KiKcJmV0dqfR62wiglo9BQiciIiZZRERExiVhB7B5HACh2p6WJLW3HgvcOg7cv1R4n08nadSq2UDAwrpKwyUiopKYZBERERkLhRyImYESCRZQ2Bb/nfSvlQPQcpSUXLk3r6oIiYhIA0yyiIiIjMX1Y0DanfKPe+Y1oMcHgLWj/mMiIiKtcSUsERGRsci4q9lx9UKYYBERGTEmWWpER0cjMDAQ7dq1M3QoRERUEyjkwMXdwJ/Rmh3v4K7feIiIqFI4XVCNyMhIREZGIi0tDc7OzoYOh4iIqquMFKn8+qm1QNotDR4gk6oM+nbSd2RERFQJTLKIiIiqkhDS2quT3wAXdgKKPKnd1gUIDgOcfYDd7xYcXOSBMumf0Cj1+2UREZHRYJJFRERUFbLTgL83ASdXA/cuFLbXby9VCAwcAljaSG0ObtI+WelJhcc5eUkJVuCgKg2biIi0xySLiIioLAp54ffXjwEBPbUbSUo+B5xaDZzZBORlSm2WdkDLkUDIRMCzZcnHBA4CGnQHoryl2y9t0f68RERkMEyyiIiISpOwQxpRKvD9iKcjSvPLHlHKzwESfpFGrW7+Vdhep4k0atVqNGBTzprfogmVbycmWEREJoRJFhERkToJO4DN41BiY+C0JKl95PqSidaj60DsGiDuOyDrvtRmZgE0HQC0mwT4dQFksioJn4iIDIdJFhERUXEKORAzAyUSLOBpmwyImQk07S81XdkvFbK4/FvhYxy9gJDxQPA4wNGjauImIiKjwCSLiIiouOvHgLQ7ZRwggLTbwK4pwL8Hgcc3Cu9q0EMatWocCpjzv1kiopqIv/2JiIiKy7ir2XFx66R/bWoBbcYCIRMA1wC9hUVERKaBSRYREVFxDu6aHefaCOgyGWg+DLCy02tIRERkOphkERERFefbSaoiWNaUQfu6QORxVv0jIqISzAwdABERkVHJzwUStgPWTqUcIJO++i9mgkVERGpxJIuIiAgAUm8BsWuB2HVAZsrTRhlgYQ3kZxce5+QFhEaVvU8WERHVaEyyiIio5lIopOqAJ1cDl/YAQiG1O7gDbSOA4HDA2hGI8pbaX9oCBPTkCBYREZWJSRYREdU8WQ+B+O+l5OpRYmG7X1ep/HrT/oC5pdSWm1l4v28nJlhERFQuJllERFQzCAHcjpM2DT7/c+EUQGsnoPWLUvn1uk0MGyMREVULTLLUiI6ORnR0NORyuaFDISKiysrNAs5tkUatkuIL2z1aSKNWLV4ArOwNFh4REVU/TLLUiIyMRGRkJNLS0uDs7GzocIiIqCLuXwZOfStNC8xOldrMrYHmQ6Xkqn4IIJMZNkYiIqqWmGQREVH1Ic8H/tktTQlMPFzYXssXaDcRaD0WsHc1XHxERFQjMMkiIiLTl5YExK2TSrCnJz1tlAGNQ6VRq4CegBm3hiQioqrBJIuIiEyTEEDiEWnU6uKvgHi6jta+LhA8TirBXsvHoCESEVHNxCSLiIhMy5PHwJkfpUIWDy4Xtvt0kqYENhsEWFgZLDwiIiImWUREZBruxAOnVgN//wTkP5HarByAlqOk5Mq9uUHDIyIiKsAki4iIjFdeNnB+mzQl8Papwna3QCmxajkKsHY0XHxERERqMMkiIiLj8/Bfqfz66Q3Ak0dSm5klEDhYKmTh8wzLrxMRkdFikkVERMZBng9c3iuttbq6v7Dd2RsIGQ+0CQMc3AwXHxERkYaYZBERkWGl3wVOrwdOrQXSbj1tlAENn5NGrRr1BszMDRkhERGRVphkERFR1RMCuH5MWmt1YQegyJfabV2A4DCg7XjAxd+wMRIREVUQkywiIqo62WnA35ukKYH3LhS2128vFbIIHAJY2khtuZnAXC/p+/fvAFb2VR4uERFRRTDJIiIi/Us+KyVWf28G8jKlNks7oMULUnLl2cqw8REREekQkywiItKP/Bwg4Rcpubr5V2F7ncbSWqtWowEb59Ifr5AXfn/9GBDQk2uziIjIJDDJIiIi3Xp0DTi1Riq/nnVfajOzAJoOkJIrvy7ll19P2AHsmV54+/sRgJMXEDofCBykt9CJiIh0gUkWERFVnkIOXPk/adTq8m8AhNTu6CWVXw8eBzh6aNZXwg5g87jCPgqkJUntI9cz0SIiIqPGJIuIiCou8z5w+jtp4+DHNwrbG/SQRq0ahwLmWvxXo5ADMTNQIsECnrbJgJiZQNP+nDpIRERGi0kWERFpRwjg5gmp/HrCdkCeK7Xb1ALajAVCJgCuARXr+/oxIO1OWScH0m5Lx/l3rdg5iIiI9IxJFhERaSYnAzi7WZoSePdcYbtXsFQhsPkwwMqucufIuKvb44iIiAyASRYREZUt5YKUWJ3ZCOSmS20WNkCLEUDIRKBesO7O5eCu2+OIiIgMgEkWERGVlJ8LXNwpJVfX/yhsdwmQRq1ajQHsXHR/Xt9OUhXBtCSoX5clk+737aT7cxMREekIkywiIir0+CYQuxaIWw9kpkhtMnOgSV+pkIV/N8DMTH/nNzOXyrRvHgdABtVE62nZ99AoFr0gIiKjViOSrKFDh+LQoUPo1asXtmzZYuhwiIiMi0IB/HtAGrW6FAMIhdTu4A60jQCCwwHnelUXT+AgqUz7nulAelJhu5OXlGDVlPLtVvbAnFRDR0FERBVQI5Kst956CxMmTMC6desMHQoRkfHIegjEfy8lV48SC9v9ukqjVk37A+aWhoktcBDQoDsQ5S3dfmkLENCTI1hERGQSakSS1aNHDxw6dMjQYRARGZ4QwO1YKbE6txWQ50jt1k5A6xel8ut1mxg2xgJFEyrfTkywiIjIZOhxYr1uHDlyBAMHDoSXlxdkMhm2b99e4pjly5fD398fNjY2aNu2LY4ePVr1gRIRGbPcLGmd1apuwDe9gDM/SAmWe3Pp/pw0oNcs40mwiIiITJjRJ1mZmZlo1aoVli1bpvb+TZs2YfLkyfjggw9w+vRpdO3aFX379sWNGzeqOFIiIiN0/zKwZyawqCmw400g6Qxgbi1VB5z4f8CE3wwdIRERUbVj9NMF+/bti759+5Z6/+LFizFx4kRMmjQJALB06VLs3bsXK1aswLx587Q6V05ODnJycpS309LSKhY0EZEhyfOAf3YDJ78BEo8Uttf2k6YDth4L2LtKbbmZBglRIyz8QEREJsrok6yy5ObmIjY2FjNnzlRpf/7553Hs2DGt+5s3bx4++ugjXYVHRFS10u4AseuAuHWFVflkZkCjPlIhi4CeJcuvK+SF318/xuISREREOmDSSdb9+/chl8vh7u6u0u7u7o7k5GTl7T59+iAuLg6ZmZmoX78+tm3bhnbt2pXo77333sPUqVOVt9PS0uDt7a2/J0BEVFlCAImHpUIWF38FxNOkyb4uEDxOKsFey0f9YxN2SGXSC3w/4mmZ9Pk1p0w6ERGRHph0klVAJpOp3BZCqLTt3btXo36sra1hbW2t09iIiPTiyWPgzI9ScvXgcmG7Tyeg3USg2SDAwqr0xyfseLrhr1BtT0uS2keuZ6JFRERUQSadZNWpUwfm5uYqo1YAkJKSUmJ0i4ioWrgTL621OrsFyH8itVk5AK1GAyETAffA8vtQyIGYGSiRYAFP22RAzExpnyxOHeTaMCIi0ppJJ1lWVlZo27Yt9u3bh6FDhyrb9+3bh8GDB1e43+joaERHR0Mul5d/MBGRvuU9Ac5vk5Kr27GF7W7NpVGrliMBa0fN+7t+TFq/VSoBpN2WjvPvWuGwiYiIaiqjT7IyMjJw5coV5e3ExETEx8fDxcUFPj4+mDp1KsLCwhASEoKOHTti1apVuHHjBl599dUKnzMyMhKRkZFIS0uDs7OzLp4GEZH2HlwFTn0LxH8PPHkktZlZAs2HSKNWPs8AxaZLayTjrm6PIyIiIhVGn2SdOnUKPXr0UN4uKEwRHh6OtWvXYtSoUXjw4AE+/vhjJCUlISgoCLt374avr6+hQiYiqjh5PnB5r7TW6ur+wnZnHyAkAmgzDnCoW7lzOGg4nVrT44iIiEiFTAihblI+AcqRrNTUVDg5ORk6HCKqztLvAqfXA6fWAmm3njbKgIbPSeXXG/XW3foohRxYGiQVuVC7LksmVRmcfJZrsoiIiJ7SJjcw+pEsIqJqSwjg+h/SqNWFHYAiX2q3dQGCw4C24wEXf92f18xcKtO+eRwAGVQTrafTD0OjmGARERFVEJMsNVj4goj0KjsN+HuTVMji3sXC9vrtpVGrwMGApY1+YwgcJJVp3zO9cONi4Ok+WVEs305ERFQJnC5YBk4XJCKdSj4rjVr9vRnIy5TaLO2k6oAhEwHPllUfU3YaEPV00/WXtgABPTmCRUREpAanCxIRGYv8HCDhFym5uvlXYXudJlL59VajARsDVjEtmlD5dmKCRUREpANMsoiI9OHRNeDUGuD0BiDrvtRmZgE0GyiNWvl1qVj5dSIiIjJ6TLKIiHRFIQeu/J80anX5NygLSjjVA9pGAMHjAEcPQ0ZIREREVYBJFhFRZWXeB05/J20c/PhGYXuDHlIhi8ahgDl/3RIREdUU/F9fDVYXJKJyCQHcPCFVCEzYDshzC+9r/x+gw38A1wCDhacxK3tgTqqhoyAiIqpWWF2wDKwuSEQl5GQAZzdLUwLvnits92wNJMVL379/R0peiIiIqNpgdUEiIl1LuSAlVmc2ArnpUpuFDdBihFTIwrVhYSn068dYCp2IiKgGY5JFRFSa/Fzg4k7g5LfA9d8L210CpLVWrccAtrWBhB3AxhcL7/9+xNNNfedzU18iIqIaiEkWEVFxj28CsWuBuPVAZorUJjMHmvaTRq38uwFmZlJ7wg5g8zgoKwkWSEuS2keuZ6JFRERUwzDJIiICAIUC+PeANGp1aQ8gFFK7gwfQNhwIDgec6xV7jByImYESCRbwtE0GxMwEmvbn1EEiIqIahEmWGqwuSFSDZD0E4r+X1ls9Sixs9+sqTQls2h8wt1T/2OvHgLQ7ZXQugLTb0nH+XXUaNhERERkvJllqREZGIjIyUllBhIiqGSGA23FS+fVzWwF5jtRu7SytswqZANRtUn4/GXc1O5+mxxEREVG1wCSLiGqO3Czg3BYpuUo6U9ju0VIatWoxQrvS6w7uuj2OiIiIqgUmWURU/d2/LE0HjP8ByHm68a65NRA0TEqu6rUFZDLt+/XtJFURTEuC+nVZMul+306ViZ6IiIhMDJMsIqqe5HnAP7ulUavEI4Xttf2kCoGtXwLsXSt3DjNzqUz75nEAZFBNtJ4mbaFRLHpBRERUwzDJIqLqJe0OELsOiFsHpCdJbTIzoHEo0G4i0KBnYfl1XQgcJJVp3zO98HzA032yoli+nYiIqAZikkVEpk8IIPGwNCXw4q+AeFoZ1L6uVHq9bQRQy1t/5w8cBDToDkQ9PcdLW4CAnhzBIiIiqqGYZBGR6XryCIj/ETj1LfDgcmG7b2epQmCzQYCFVdXEUjSh8u3EBIuIiKgGY5KlBvfJIjJyd05Lo1ZntwD5T6Q2K0eg1ShpvZV7oGHjIyIiohpNJoRQVxKLAOU+WampqXBycjJ0OEQ1W94T4Pw2qZDF7djCdrfm0lqrliMBa0fDxUdERETVmja5AUeyiMi4PbgqTQeM/16aHggAZpZA8yFS+XXvDhUrv05ERESkJ0yyiMj4yPOBy3ulUaurBwrbnX2AkPFAmzDAoa7h4iMiIiIqA5MsIjIe6XeBuPVA7Bog7fbTRhnQqLe01qpRbxaUICIiIqPHJIuI9C83E5jrJX3//h3Ayr7wPiGA639Io1YXdgKKfKndzlUasQoZL20gTERERGQimGQRkWFkpwJnNgGnVgP3Lha2e3eQRq0CBwOWNoaLj4iIiKiCmGQRkf4pimyHEP8DkHxWKr+elym1WdpL1QHbTQQ8WhgmRiIiIiIdYZJFRPqVsAPYM63w9u53C7+v00SqENhqFGDjXPWxEREREekBkyw1uBkxkY6c/Ab49Z3S7+/5gTQtkIiIiKga4WbEZeBmxEQVoJADV/4POPE1cGVfGQfKACcvYPJZVgwkIiIio8fNiImo6mXcA05/B5xaA6Te0OABQirTfv0Y4N9V7+ERERERVRUmWURUcUIAN49L0wITfgHkuVK7TS3Auz1w+bfy+8i4q9cQiYiIiKoakywi0l5OBnB2M3ByNXD3XGF7vbZSIYvmQ4FbpzRLshzc9RcnERERkQEwySIizaVckBKrMxuB3HSpzcIWaDFc2tuqXnDhsb6dpDVXaUkA1C39fLomy7dTVUROREREVGWYZBFR2fJzgYs7peTq+h+F7a4NpcSq9RjAtnbJx5mZA6Hzgc3jAMigmmjJpH9Co1j0goiIiKodJllEpN7jm0DsWiBuPZCZIrXJzIGm/aQpgf7dAJms7D4CBwEj1wN7pgPpSYXtTl5SghU4SG/hExERERkKkywiKqRQAP8ekEatLsUAQiG1O3gAbSOAtuFSgqSNwEFAg+5AlLd0+6UtQEBPjmARERFRtcUki4iArIdA/PdScvUosbDd/1lp1KpJP8DcsuL9F02ofDsxwSIiIqJqjUkWUU0lBHA7Vkqszm0F5DlSu7WztM4qZAJQt4lhYyQiIiIyQUyyiGqa3Czg3BZpb6ukM4XtHi2lUasWIwAre92e08oemJOq2z6JiIiIjBSTLDWio6MRHR0NuVxu6FCIdOf+ZWnUKv4HIOdpwmNuDQQNk5Krem3LL2RBREREROWSCSHUbWBDANLS0uDs7IzU1FQ4OTkZOhwi7cnzgH92S6NWiUcK22v7SeXX24wF7FwMFh4RERGRqdAmN+BIFlF1lHYHiF0HxK0rLJ0uMwMahwLtJgINegJmZoaNkYiIiKiaYpJFVF0IASQelqYEXvwVEE+nu9rXBYLDpRLstbwNGiIRERFRTcAki8jUPXkMnPlRSq4eXC5s9+0sVQhsNgiwsDJYeEREREQ1DZMsIlN1J15aa3V2C5D/RGqzcgRajZLWW7kHGjQ8IiIiopqKSRaRKcl7ApzfJiVXt2ML292aS2utWo4ErB0NFx8RERERVT7JSktLw4EDB9CkSRM0a9ZMFzERUXEPrgKnvgXivweePJLazCyB5kOkUSufZ1h+nYiIiMhIaJ1kjRw5Es8++yzeeOMNPHnyBCEhIbh27RqEENi4cSOGDx+ujziJah55PnB5r7TW6ur+wnZnHyAkAmgzDnCoa7DwiIiIiEg9rZOsI0eO4IMPPgAAbNu2DUIIPH78GOvWrcOnn37KJIuostLvAqfXA6fWAmm3njbKgEa9pVGrRr0BM3NDRkhEREREZdA6yUpNTYWLi7R5aUxMDIYPHw47Ozv0798f06ZN03mARDWCEMD1P6RRqws7AEW+1G7rAgSHAW3HAy7+ho2RiIiIiDSidZLl7e2NP//8Ey4uLoiJicHGjRsBAI8ePYKNjY3OAySq1rLTgL83SYUs7l0sbK/fHmg3CQgcDFjyfUVERERkSrROsiZPnoyXXnoJDg4O8PX1Rffu3QFI0whbtGih6/iIjF9uJjDXS/r+/TuAlX35j0k+K41a/b0ZyMuU2iztpOqAIRMBz5b6i5eIiIiI9ErrJOv1119Hhw4dcOPGDfTu3RtmZmYAgAYNGuDTTz/VeYBERk8hL/z++jEgoKf6NVP5OUDCL1JydfOvwvY6TaRRq1ajABtn/cdLRERERHolE0IIQwdhrNLS0uDs7IzU1FQ4OTkZOhwyRgk7gD3TgfSkwjYnLyB0PhA4SLr96Bpwag1wegOQdV9qM7MAmg2URq38urD8OhEREZGR0yY34GbERBWVsAPYPA5Asb9TpCVJ7V2nAsnngMu/FR7jVA9oGwEEjwMcPao4YCIiIiKqCkyyiCpCIQdiZqBEggUUth1dVNjUoIc0JbBxKGDOtx0RERFRdcZPe2pER0cjOjoacrm8/IOpZrp+DEi7U/5xgUOAXrMA1wC9h0RERERExsHM0AEYo8jISCQkJODkyZOGDoWMVcZdzY5rNpAJFhEREVENo9Mk68aNGxz9oeov5QJw7mfNjnVw128sRERERGR0dJpk+fn5ITAwED//rOEHUCJTkZ8LnNsKrOkPLH8G+OfXch4gk4pc+HaqkvCIiIiIyHjodE3WwYMHkZiYiC1btmDYsGG67JrIMB7fBGLXAnHrgcwUqU1mDjTtB7gFAocXPD2waAGMp+XYQ6PU75dFRERERNUa98kqA/fJqqEUCuDfg9KmwZf2AEIhtTt4AG3DgeBwwLme1KZ2n6x6UoJVsE8WEREREZk8ve+TlZ+fj0OHDuHq1at48cUX4ejoiDt37sDJyQkODg4VCprI4LIeAvHfS8nVo8TCdr+uUvn1pv0Bc0vVxwQOAhp0B6K8pdsvbQECenIEi4iIiKgG0zrJun79OkJDQ3Hjxg3k5OSgd+/ecHR0xIIFC5CdnY2vvvpKH3ES6YcQwO044OQ30poreY7Ubu0EtH4RCJkA1G1Sdh82TsCcVP3HSkREREQmQesk6+2330ZISAjOnDkDV1dXZfvQoUMxadIknQZHpDe5WcC5LVJylXSmsN2jpTRq1WIEYGVvuPiIiIiIyGRpnWT9/vvv+OOPP2BlZaXS7uvri9u3b+ssMCK9uH9Zmg545gcg++nok7k1EDRMSq7qtQVkMsPGSEREREQmTeskS6FQqN0L69atW3B0dNRJUEQ6Jc8D/tktjVolHilsr+0HhEwEWr8E2LuW+nAiIiIiIm1onWT17t0bS5cuxapVqwAAMpkMGRkZmD17Nvr166fzAIkqLO0OELsOiFtXWP1PZgY0DgXaTQQa9ATMdLpVHBERERGR9iXc79y5gx49esDc3ByXL19GSEgILl++jDp16uDIkSNwc3PTV6xVjiXcTZAQQOJhaUrgxV8B8XTU1b4uEDwOaBsB1PIxaIhEREREZHr0WsLdy8sL8fHx+PHHHxEXFweFQoGJEyfipZdegq2tbYWDJqqUJ4+BMz9KydWDy4Xtvp2lCoHNBgEWVqU+nIiIiIhIV7gZcRk4kmUC7pyWEquzW4D8J1KblSPQapS03so90LDxEREREVG1oPfNiG/fvo0//vgDKSkpUCgUKve99dZbFemSSHN5T4Dz26RCFrdjC9vdmktrrVqOBKxZhIWIiIiIDEPrJGvNmjV49dVXYWVlBVdXV8iKlLuWyWRMskh/HlwFTn0LxH8PPHkktZlZAs2HSKNWPs+w/DoRERERGZzW0wW9vb3x6quv4r333oNZNa/MxumCRkCeD1zeK41aXT1Q2O7sA4SMB9qEAQ51DRcfEREREdUIep0umJWVhdGjR1f7BIsMLP0uELceiF0LpN162igDGvWWRq0a9QbMzA0ZIRERERGRWlonWRMnTsRPP/2EmTNn6iMeqsmEAK7/IY1aXdgJKPKldjtXacQqZLy0gTARERERkRHTerqgXC7HgAED8OTJE7Ro0QKWlpYq9y9evFinARoSpwtWkexU4Mwm4NRq4N7FwnbvDtKoVeBgwNLGcPERERERUY2n1+mCc+fOxd69e9GkSRMAKFH4gkhjyWel8ut/bwbyMqU2S3ug5QtScuXZ0rDxERERERFVgNZJ1uLFi/Htt98iIiJCD+FQtZefAyT8Ik0JvHm8sL1OE6DdJGl/Kxtnw8VHRERERFRJWidZ1tbW6Ny5sz5ioers0TXg1Brg9HdA1gOpzcwCaDZQSq58O7P8OhERERFVC1qXCHz77bfx5Zdf6iMWvdi1axeaNGmCRo0a4ZtvvjF0ODWLQg5c2gt8/wLweWvgj6VSguVUD+jxX2DKeeCFtYBfFyZYRERERFRtaF34YujQoThw4ABcXV3RvHnzEoUvfv75Z50GWBn5+fkIDAzEwYMH4eTkhODgYBw/fhwuLi4aPZ6FLyoo4540YnVqDZB6o7A9oKc0atWoD2Cu9SAqEREREZHB6LXwRa1atTBs2LAKB1eVTpw4gebNm6NevXoAgH79+mHv3r0YM2aMgSOrhoSQ1lid/EZacyXPldptagFtxgIhEwDXAIOGSERERERUFbROstasWaOPONQ6cuQIFi5ciNjYWCQlJWHbtm0YMmSIyjHLly/HwoULkZSUhObNm2Pp0qXo2rUrAODOnTvKBAsA6tevj9u3b1dZ/DVCTrpUHfDUt8Ddc4Xt9dpKo1bNhwKWtoaLj4iIiIioihn1nK3MzEy0atUK48ePx/Dhw0vcv2nTJkyePBnLly9H586dsXLlSvTt2xcJCQnw8fGBupmQJltmPjcTmOslff/+HcDK3rDxpFyQyq+f2QjkpkttFrZAi+FS+fV6wYaNj4iIiIjIQDRKsoKDg7F//37Url0bbdq0KTNRiYuL01lwffv2Rd++fUu9f/HixZg4cSImTZoEAFi6dCn27t2LFStWYN68eahXr57KyNWtW7fQoUOHUvvLyclBTk6O8nZaWpoOnkU1kp8LXNwpJVfX/yhsd20oJVatxwC2tQ0XHxERERGREdAoyRo8eDCsra0BoMR0PUPJzc1FbGwsZs6cqdL+/PPP49ixYwCA9u3b49y5c7h9+zacnJywe/duzJo1q9Q+582bh48++kivcZukxzeB2LVA3HogM0Vqk5kDTftJUwL9u7E6IBERERHRUxolWbNnz8aECRPw+eefY/bs2fqOSSP379+HXC6Hu7u7Sru7uzuSk5MBABYWFli0aBF69OgBhUKB6dOnw9XVtdQ+33vvPUydOlV5Oy0tDd7e3vp5AtpSyAu/v35MqtRnZq7H8ymAfw9Io1aXYgChkNodPIC2EUDbcMDJS3/nJyIiIiIyURqvyVq3bh2ioqLg6Oioz3i0VnzqohBCpW3QoEEYNGiQRn1ZW1srR+yMSsIOYM/0wtvfj5ASnND5QKBmz01jWQ+B0xukQhaPEgvb/Z+VRq2a9APMLUt/PBERERFRDadxkqXldlp6V6dOHZibmytHrQqkpKSUGN0yaQk7gM3jABS7/mlJUvvI9ZVPtIQAbsdK5dfP/QzIn65Ls3YGWr8olV+v27hy5yAiIiIiqiG0qi5oTJX5rKys0LZtW+zbtw9Dhw5Vtu/btw+DBw+uVN/R0dGIjo6GXC4v/2B9UsiBmBkokWABT9tkQMxMoGn/ik0dzM0Ezm4BTq0Gks4Utnu0BNq/DAQNN3wVQyIiIiIiE6NVktW4ceNyE62HDx9WKqCiMjIycOXKFeXtxMRExMfHw8XFBT4+Ppg6dSrCwsIQEhKCjh07YtWqVbhx4wZeffXVSp03MjISkZGRyl2dDeb6MSDtThkHCCDttnScf1fN+713SUqs4n8EclKlNnNrKalqN1Ha48qIEmoiIiIiIlOiVZL10UcfVWnScerUKfTo0UN5u6AoRXh4ONauXYtRo0bhwYMH+Pjjj5GUlISgoCDs3r0bvr6+VRajXmXc1d1x8jzg4q9ScpV4pLC9tr80HbDNWMDOpWJxEhERERGRkkxouNjKzMwMycnJcHNz03dMRqNgJCs1NRVOTk5VH0DiUWDdgPKPC99V+khW2h0gdp1Ugj3j6fo1mRnQOFQatWrQEzAz01nIRERERETVkTa5gcYjWca0HqvG8O0kVRFMS4L6dVky6X7fTqrNQgD/HpJGrS7uBsTTtWX2bkDwOKkEey0jKU1PRERERFTNmGx1QX0ymsIXZuZSmfbN4wDIoJpoPU16Q6MKi148eSStszq1GnhQuJYNvp2lUaumAwELqyoKnoiIiIioZtJ4umBNZPDpggUK9slKTypsc6onJViBg4A7p6Xy62e3AvlPpPutHIFWo6Xkyq2ZYeImIiIiIqom9DJdkAwocBDQoDsQ9XSK30tbAJ+OQMIvwKoewJ24wmPdg6TEqsULgLVxbRxNRERERFQTMMkyFUX3wbq8D9g6Cch+LN02twICh0jJlXcHll8nIiIiIjIgJlmm4sZfhd+fWCn9W8vnafn1MMC+jmHiIiIiIiIiFUyy1DCawhdFpd0u/D6gF9DhP0DD51RHuIiIiIiIyOBY+KIMRlP4AgCyHgALGkjfv38HsLI3bDxERERERDUIC19UR3auwJxUQ0dBRERERETlMDN0AERERERERNUJkywiIiIiIiIdYpJFRERERESkQ0yyiIiIiIiIdIhJlhrR0dEIDAxEu3btDB0KERERERGZGJZwL4NRlXAnIiIiIiKD0SY34EgWERERERGRDjHJIiIiIiIi0iEmWURERERERDrEJIuIiIiIiEiHmGQRERERERHpEJMsIiIiIiIiHWKSpQb3ySIiIiIiooriPlll4D5ZREREREQEcJ8sIiIiIiIig2GSRUREREREpEMWhg6AiIiIiIioOLlC4ETiQ6SkZ8PN0Qbt/V1gbiYzdFgaYZJFRERERERGJeZcEj7amYCk1Gxlm6ezDWYPDERokKcBI9MMpwsSEREREZHRiDmXhNc2xKkkWACQnJqN1zbEIeZckoEi0xyTLCIiIiIiMgpyhcBHOxOgrvx5QdtHOxMgVxh3gXQmWUREREREZBROJD4sMYJVlACQlJqNE4kPqy6oCuCaLDWio6MRHR0NuVxu6FCIiIiIiKo1IQTuZeTgSkoGtsXd1ugxKemlJ2LGgJsRl4GbERMRERER6YZcIXDrURaupGQUft3LwNWUDKRl52vV148vP4OOAa56ilQ9bXIDjmQREREREZHO5OTLkXg/UzWZSslA4v1M5OQr1D7GTAb4uNghoK49/vr3ITJz1c8okwHwcJbKuRszJllERERERKS1tOw8XC02InUlJQM3HmahtLoUVhZmaFDHHg3dHFS+/FztYWNpDqCwuiAAlQIYBTtkzR4YaPT7ZTHJIiIiIiIitYQQuJeeo5pI3ZOSqbtpOaU+ztHGQkqg6qomU/Vr25WbIIUGeWLF2OAS+2R5mNA+WUyyiIiIiIhquIqul3J3skZDNwcEFCRTT/+t62gNmazio02hQZ7oHeiBE4kPkZKeDTdHaYqgsY9gFWCSRURERERUQ1RmvVRDNwcEFEmkAtwc4GRjqbdYzc1kVV7cQleYZBERERERVTP6Wi9FmmGSRURERERUxeQKUempcMXXSxWMSl29p7/1UqQZJllERERERFUo5lxSiaIOnmUUdShtvdSVlAykl7Feys3RWnVUSkfrpah8TLKIiIiIiKpIQXny4jP2klOz8dqGOHwwoBk8nGy0Wi/l7WKnsk6qoBCFs63+1ktR2ZhkERERERFVAblC4KOdCSUSLKBwP6hPd11Q+1iulzItTLLUiI6ORnR0NORy9TtNExERERGVRd16qdjrj1SmCJamoZs92njX5nopEyYTQpRSX4TS0tLg7OyM1NRUODk5GTocIiIiIjIyFV0vVZbPR7fG4Nb1dBwpVZY2uQFHsoiIiIiIylHR/aWKrpcCgJVH/i33XG6ONjqNnaoekywiIiIioqf0ub+UXCGw48wdJKdmq12XJQPg4SyVcyfTxiSLiIiIiGoUdeulrj79V5/7S5mbyTB7YCBe2xAHGaCSaBU8evbAQK69qgaYZBERERFRtVTaeqmrKRlIM9D+UqFBnlgxNrjEPlkeZeyTRaaHSRYRERERmTRdrJeqyv2lQoM80TvQAycSHyIlPRtujtIUQY5gVR9MsoiIiIjIJOhzvVRVMzeToWOAq8HOT/rFJIuIiIiIjIYQAvcypPVSV4uVRNfneikiXWKSRURERERVzhjXSxHpCpMsIiIiItIbU1svRaQLTLKIiIiIqNLSs/NKjEiZ6nopospikkVERERUTckVQqcV7LheikgzTLKIiIiIqqGYc0kl9mLy1HAvpqLrpQo26S344nopovIxySIiIiKqZmLOJeG1DXEoPksvOTUbr22Iw4qxwQgN8uR6KSI9YZJFREREVI3IFQIf7UwokWABULa9vTEeHk4XcPPRE66XItIDJllERERE1UDBeqkd8XdUpgiqk5OvwPWHTwBwvRSRPjDJUiM6OhrR0dGQy+WGDoWIiIhIhVwhcPvRE1y5l15iml9Z66WKe6NHAMZ19ON6KSI9kAkhShkkprS0NDg7OyM1NRVOTk6GDoeIiIhqEHXrpa7ey8S/9zLKXC9V19G6zEp/BX58+Rl0DHDVddhE1ZY2uQFHsoiIiIgMSNf7S1mam6HL/ANITs1Wuy5LBsDDWSrnTkT6wSSLiIiISM+qen+p2QMD8dqGOMgAlURLVuR+rrci0h8mWUREREQ6Yiz7S4UGeWLF2OAS+2R5aLhPFhFVDpMsIiIiIi2Zwv5SoUGe6B3ogROJD5GSng03R2mKIEewiPSPSRYRERFRKSq7XiqgyDQ//zpVv7+UuZmMxS2IDIBJFhEREZkkuULoZJSmqtdLEVH1xySLiIiITE7MuaQS6408y1lvZCzrpYio+mOSRURERCYl5lwSXtsQV6I8eXJqNl7bEIcvxrRGI3dHo14vRUTVG5MsIiIiMhlyhcBHOxPU7v9U0Pbmj/GlPr5gvVRAsWl+hlgvRUTVF5MsIiIi0qvKrJ0qvl7q98v3VaYIlsbW0gxNPJxKTPHzduF6KSLSPyZZREREpDearp2SKwRuP3qCK/fSS0zzK2u9VGmihrXE4Db1dPIciIi0xSSLiIiI9KK0tVNJqdl4dUMcBrT0hEwmw5WUDPx7L0Oj9VJ2VubY+XdSued2c7LRwTMgIqoYJllEREQ1nK5KoRf1OCsXH2w7p3btVIFdxZIlTdZLyRUCp64/QnJqttq+ZQA8nKXnQERkKEyyiIiIarCKlEIvUNH9pYoa094bvZq6a7xeytxMhtkDA/HahjjIAJVEq+CRswcGct0VERmUTAhR1h+ZarS0tDQ4OzsjNTUVTk5Ohg6HiIhIp0qbzleQnqwYG4zQIE+dr5cq6vPRrTG4tfZrpyqTHBIRVYQ2uQFHsoiIiGogTUqhT94UD7//u1zm/lIyGeDjYoeAuoVV/ALcHPAoKxeT1p0qNw43x4qtnQoN8kTvQA+dT3MkItIFJllERERGSB/rpAqkZ+dh++nb5ZZCz85T4GJyOgDt95eSKwQ8nW30unbK3EyGjgGuFX48EZG+MMkiIiIyMrqYCqeL9VIA8HJXf7zUwVfr/aW4doqIajKuySoD12QREVFV03SdVIGKrpdytrVE6pO8cuP58eVnKjVaxLVTRFRdcE0WERGRCdJkndR7P5/FhaR0/Hs/s9z9pWQywLu2nXJqX8F6qYZuDnCwtkCX+Qf0Xgqda6eIqCZikkVERGQkTiQ+LHed1KOsPHy+/7JKm5W5GRrULVwvVfBvg7rq10sVqKrpfFw7RUQ1TY1IsoYOHYpDhw6hV69e2LJli6HDISIiE6KPAhSlrZc6dztNo8e393dBz6ZuygIU2q6XKhAa5IkVY4NLTOfz4HQ+IqJKqRFrsg4ePIiMjAysW7dOqySLa7KIiGq2yq4nUigEbulhf6nKrpMqTp+VDImIqguuySqmR48eOHTokKHDICIiE1JaAYrk1Gy8tiFOpQBFTr4c1+5nFSZRT6v4abteyq+OPd78MQ4paTl6XSdVHKfzERHplsGTrCNHjmDhwoWIjY1FUlIStm3bhiFDhqgcs3z5cixcuBBJSUlo3rw5li5diq5duxomYCIiqvY0KUDxzk9n8NOpW/j3fiZuPMyCXKF+YoiVuRn869ijoVth0Ymy1kt9NKg5y54TEZk4gydZmZmZaNWqFcaPH4/hw4eXuH/Tpk2YPHkyli9fjs6dO2PlypXo27cvEhIS4OPjAwBo27YtcnJK7vnx22+/wcvLS+/PgYiIqg8hBH5LSC63AEVmjhz7L6YobztaWxQmUUU27K1f2xYW5mYan5/rpIiITJ9RrcmSyWQlRrI6dOiA4OBgrFixQtnWrFkzDBkyBPPmzdO470OHDmHZsmVlrsnKyclRSdbS0tLg7e3NNVlERNWQLtZLjQiuj6HB9dDQzQFujtaQyXQ3wsR1UkRExqXarMnKzc1FbGwsZs6cqdL+/PPP49ixYzo/37x58/DRRx/pvF8iItKerpKMiqyX0tTwtvX1tpaJ66SIiEyXUSdZ9+/fh1wuh7u7u0q7u7s7kpOTNe6nT58+iIuLQ2ZmJurXr49t27ahXbt2JY577733MHXqVOXtgpEsIiKqWhWp6peenYer9zJVRqSu3suo0HopX1c7PLf4sN436iUiourJqJOsAsWnXwghtJqSsXfvXo2Os7a2hrW1tVaxERGRbpVX1S9qeAv4uNjjyr0ie0ylZCA5rfQ1VAXrpQLqFlkz5eYA7zLWS1XVRr1ERFT9GHWSVadOHZibm5cYtUpJSSkxukVERKZPk6p+M7aeLfXxdR2tlQUnin5VZL0UC1AQEVFFGXWSZWVlhbZt22Lfvn0YOnSosn3fvn0YPHiwASMjIiJdKL5e6q/EB+VW9QMAN0drBNVzVk7vC3j6r7OdpU7jCw3yRO9ADxagICIirRg8ycrIyMCVK1eUtxMTExEfHw8XFxf4+Phg6tSpCAsLQ0hICDp27IhVq1bhxo0bePXVV/UWU3R0NKKjoyGXy/V2DiIiY1BVFewqsl6qLB/0b4bBrevpPE51WICCiIi0ZfAS7ocOHUKPHj1KtIeHh2Pt2rUApM2IFyxYgKSkJAQFBWHJkiV49tln9R6bNmUaiYhMTcy5JMzecR530wq3riivuERZhBC4n5GrrOCn7Xqphm4OsDSX4ccTN8s9148vP8PEh4iIqpQ2uYHBkyxjxiSLiKqr0opLFIxhrRgbXGqipVAI3H78RHVvqadl0VOf5JV6Tk3WS8kVAl3mHyi3qt/vM3pyyh4REVWparNPFhER6V55xSVkAD7amYBnG9fFzYdPcPWe6ka9/97PQHae+v2lZDLAu7ZdYRKl5XopczMZq/oREZHJY5JFRFTDnEh8WGZxCQEgKTUbQbP3orTlUlbmZmhQ116ZQBX826CuPWwszSsVH6v6ERGRqWOSpQYLXxBRdVN0vdTPp29p9BiFUF0vVTAy1dDNAd4udnodTWJVPyIiMmVck1UGrskiIlNT0fVS6ix/KRh9gzy03l+KiIioOuKaLCKiaq74/lIFidS/9zKQk1/6eikfFzs0qGOPE4kPkZmrfrS+oLhEn+ZMsIiIiCqCSRYRkRGryP5SxddLFUz1869TuF6qoLogwOISREREusYki4hIByqzqa8u9pfSdr0Ui0sQERHpD5MsNVj4goi0EXMuqUSyom5TX4VC4NajJ7hyL73IqFSmTvaXqggWlyAiItIPFr4oAwtfEFF5StvUt8CAlp6QyWQarZdSu7+UmwOcbcvfX4qIiIj0i4UviIiqwOOsXHyw7VypCRYA7Po7SeW2lbkZ/OvYo6FbYRKlq/2liIiIyDgwySIiKoMQAvcycnA1JVOr9VJFjWnvjV5N3dHQzQH1a9vCwtxMz1ETERGRITHJIiKC+vVSBWumtN1fqrhnGrjiuUB3HUVKRLoml8uRl1e59zkRVQ+WlpYwN6/8zBImWURUo1R0fyl166UeZeZi0vpT5Z7TzdFG10+DiHRACIHk5GQ8fvzY0KEQkRGpVasWPDwqt1ckkywiqpYqur+UNuul5AoBT2cbJKdmq12XVbCpb3t/F90+OSLSiYIEy83NDXZ2dtx8m6iGE0IgKysLKSkpAABPz4pvZ8IkSw2WcCcyDRXdX8qhYH+pYmXRvbVcL2VuJsPsgYF4bUMcZOCmvkSmRC6XKxMsV1dXQ4dDREbC1tYWAJCSkgI3N7cKTx1kCfcysIQ7kW5UZqNeQFovdfvxE5VRqYJpftrsLxXw9Ht3p8rtL1WcpvtkEZHxyM7ORmJiIvz8/JQfqoiIAODJkye4du0a/P39YWNTOOWfJdyJyGhok4Docr1Uw7oOcLarmv2luKkvkemq7B9csnLzEThrLwAg4eM+sLPiRysiU6eLP8TyNwER6U1pG/Ump2bj1Q1xCO/oC1srC72sl6pq5mYydAzglCMiIkPx8/PD5MmTMXnyZADSB+Vt27ZhyJAhBo2LaiZu1kJEepEvV2DWL+fVFoQoaFv353V8dfgq/u/CXSTez4RcIeBgbYFW3rUwPLg+ZoQ2xdfjQnDw3e5I+LgP9k55FtEvBWNq78YY1MoLgV5ORpFgEVHNVfQPQycSH5b6hyJdiYiIgEwmg0wmg6WlJRo0aIB3330XmZmZGvdx+vRpDBgwAG5ubrCxsYGfnx9GjRqF+/fvqxy3detWdO/eHc7OznBwcEDLli3x8ccf4+HDhwCAtWvXKmMp+vXNN99o9XzUJUGHDh2CTCar0ZUf58yZo/b6/t///Z+hQyMNcCSLiCqltPVSF5PSkJlbfvGY55q5oUvDOmjo5qiX9VJERPoScy4Js3ecV96OWHOyStZjhoaGYs2aNcjLy8PRo0cxadIkZGZmYsWKFeU+NiUlBc899xwGDhyIvXv3olatWkhMTMSOHTuQlZWlPO6DDz7A/PnzMWXKFMydOxdeXl64fPkyvvrqK3z33Xd4++23AQBOTk74559/VM7h7Oys2ydcgzVv3rxEUuXiwoq1poAjWUSkkZx8Of5JTsfus0n4Yv9lvPXjafT7/CgCZ8eg64KDGL/2JP63+wI2nbqJ2OuPNEqwAGBgKy9EdPZHl0Z14OFswwSLiExCwXTou2k5Ku3Jqdl4bUMcYs4l6e3c1tbW8PDwgLe3N1588UW89NJL2L59O4QQaNiwIT777DOV48+dOwczMzNcvXoVx44dQ1paGr755hu0adMG/v7+6NmzJ5YuXQofHx8AwIkTJzB37lwsWrQICxcuRKdOneDn54fevXtj69atCA8PV/Ytk8ng4eGh8qWvQiLHjh3Ds88+C1tbW3h7e+Ott97SeATv2rVrkMlk2Lx5M7p27QpbW1u0a9cOly5dwsmTJxESEgIHBweEhobi3r17ysedPHkSvXv3Rp06deDs7Ixu3bohLi5Ope+C0buhQ4fCzs4OjRo1wo4dO5T3F4zK7d+/HyEhIbCzs0OnTp1KJKfqWFhYlLi+VlZW5V6Pjz/+GC1atCjRX9u2bTFr1iwAhaOIn332GTw9PeHq6orIyEiVjbn9/Pzw6aefYty4cXBwcICvry9++eUX3Lt3D4MHD4aDgwNatGiBU6cK94x88OABxowZg/r168POzg4tWrTAjz/+qBJH9+7d8dZbb2H69OlwcXGBh4cH5syZU+71MCVMstSIjo5GYGAg2rVrZ+hQiKpcRk4+4m8+xpbYW4jacxGT1p1Cj88OIXDWXvRZegSvfx+HxfsuYceZO0hISkN2ngJW5mZo4u6I/i088VavRvhiTBtEDSv5y10dbtRLRKZGrhD4aGdCmdOhP9qZoPepgwVsbW2Rl5cHmUyGCRMmYM2aNSr3f/vtt+jatSsCAgLg4eGB/Px8bNu2DaUVmP7+++/h4OCA119/Xe39tWrV0vVTKNfZs2fRp08fDBs2DH///Tc2bdqE33//HW+88YZW/cyePRv//e9/ERcXBwsLC4wZMwbTp0/H559/jqNHj+Lq1avKJAQA0tPTER4ejqNHj+Kvv/5Co0aN0K9fP6Snp6v0+9FHH2HkyJH4+++/0a9fP7z00kvKaZUFPvjgAyxatAinTp2ChYUFJkyYoLfrMWHCBCQkJODkyZPKx/z99984ffo0IiIilG0HDx7E1atXcfDgQaxbtw5r167F2rVrVc61ZMkSdO7cGadPn0b//v0RFhaGcePGYezYsYiLi0PDhg0xbtw45espOzsbbdu2xa5du3Du3Dm88sorCAsLw/Hjx1X6XbduHezt7XH8+HEsWLAAH3/8Mfbt21fha2JsOF1QjcjISERGRirLNBJVN1Wxv5RcIfD5/svcqJeIqp0TiQ9VKqYWJwAkpWbjROJDvRfEOXHiBH744Qf06tULADB+/HjMmjULJ06cQPv27ZGXl4cNGzZg4cKFAIBnnnkG77//Pl588UW8+uqraN++PXr27Ilx48bB3d0dAHD58mU0aNAAlpblV2hNTU2Fg4OD8raDgwOSk5O1eg67du1S6QNAib1KFy5ciBdffFFZ1KJRo0b44osv0K1bN6xYsUKlzHZZ3n33XfTp0wcA8Pbbb2PMmDHYv38/OnfuDACYOHGiSpLRs2dPlcevXLkStWvXxuHDhzFgwABle0REBMaMGQMAmDt3Lr788kucOHECoaGhymP+97//oVu3bgCAmTNnon///sjOzi4z9rNnz6pcm8DAQJw4caLc61G/fn306dMHa9asUQ4arFmzBt26dUODBg2U/dWuXRvLli2Dubk5mjZtiv79+2P//v14+eWXlcf069cP//nPfwAAs2bNwooVK9CuXTu88MILAIAZM2agY8eOuHv3Ljw8PFCvXj28++67yse/+eabiImJwU8//YQOHToo21u2bInZs2cr41+2bBn279+P3r17l3o9TAmTLKJqTBf7SwXUta/Qeilu1EtE1VVKeukJVkWO01ZBUpKfn4+8vDwMHjwYX375JQDA09MT/fv3x7fffov27dtj165dyM7OVn4gBqQP+1OnTsWBAwfw119/4auvvsLcuXNx5MgRtGjRAkIIjX/XOzo6qkyfMzPTfpJUjx49SqwnO378OMaOHau8HRsbiytXruD7779XtgkhoFAokJiYiGbNmml0rpYtWyq/L0gqi06rc3d3R0pKivJ2SkoKZs2ahQMHDuDu3buQy+XIysrCjRs3Su3X3t4ejo6OKv0UP8bT01PZPyAlTwXef/99vP/++wCAJk2aqEw9tLa21vh6vPzyy5gwYQIWL14Mc3NzfP/991i0aJFKTM2bN1fZbNfT0xNnz57V+poVPBcPDw/I5XJERUVh06ZNuH37NnJycpCTkwN7e/tS+y04d/FrZsqYZBFVA7n5Clx7kKmaTKVk4N/7GcjOM9z+UqFBnlgxNrjEPlke3KiXiEyYptOc9TUduiApsbS0hJeXV4kRp0mTJiEsLAxLlizBmjVrMGrUKNjZ2akc4+rqihdeeAEvvPAC5s2bhzZt2uCzzz7DunXr0LhxY/z+++/Iy8srdzTLzMwMDRs2rNTzsbe3L9HHrVu3VG4rFAr85z//wVtvvVXi8QVryTRR9PkUJJLF2xSKwv83IyIicO/ePSxduhS+vr6wtrZGx44dkZubW2q/6vop7dwKhQL169dHfHy88r6ihS2srKzUXl9NrsfAgQNhbW2Nbdu2wdraGjk5ORg+fLhO4i7tuQDAokWLsGTJEixduhQtWrSAvb09Jk+eXKFrZsqYZBGZkIyc/MKpfU9HpK6mZOC6Ee8vxY16iai6ae/vAk9nG4NNh1aXlBTVr18/2NvbY8WKFdizZw+OHDlSZn9WVlYICAhQFk148cUX8cUXX2D58uXKKoJFPX78uMrXZQUHB+P8+fOVTui0dfToUSxfvhz9+vUDANy8ebNEqfvKsrCw0Pp5aXI9LCwsEB4ejjVr1sDa2hqjR48ukWzrw9GjRzF48GDlSKRCocDly5c1Hm2sLphkEVWSXCF0mkBUxXqpqsaNeomoOjH26dDm5uaIiIjAe++9h4YNG6Jjx47K+3bt2oWNGzdi9OjRaNy4MYQQ2LlzJ3bv3q0smNGhQwdMnz4d77zzDm7fvo2hQ4fCy8sLV65cwVdffYUuXbqoTb70acaMGXjmmWcQGRmJl19+Gfb29rhw4QL27dunnCqpDw0bNsR3332HkJAQpKWlYdq0aXqrnqgNTa/HpEmTlMnNH3/8USWxNWzYEFu3bsWxY8dQu3ZtLF68GMnJyUyyiEhzMeeSSkyF03SPFF2sl5LWTDlwfykioipWMB169o7zKmXcjWU69MSJEzF37twSFewCAwNhZ2eHd955Bzdv3oS1tTUaNWqEb775BmFhYcrj5s+fj7Zt2yI6OhpfffUVFAoFAgICMGLECJUS7mW5du0a/P39cfDgQXTv3r1Sz6dly5Y4fPgwPvjgA3Tt2hVCCAQEBGDUqFGV6rc83377LV555RW0adMGPj4+mDt3rkpRB0PR9Ho0atQInTp1woMHD1SKTujThx9+iMTERPTp0wd2dnZ45ZVXMGTIEKSmplbJ+Y2FTJRWv5OU1QVTU1Ph5ORk6HDIyBTskVL8DVSQ5qwYG4zQIE+jXS9FRFSTZWdnIzExEf7+/hpXplMnPTsPLeb8BgBYO74dujaqaxTTof/44w90794dt27dUhYmqGqHDh3C0KFD8e+//6J27doGiaGmE0KgadOm+M9//oOpU6caOhyTUdrvB21yA45kEVWAJnukvL0xHl7OF3Hj0ROjXS9FRESVUzShMob1pjk5Obh58yY+/PBDjBw50mAJFgDExMTg/fffZ4JlICkpKfjuu+9w+/ZtjB8/3tDh1DhMsoi0ULBeaseZ22XukQIAOfkKJD7IAgA4Pl0vFWCE66WIiKji7KwscC2qv6HDUPrxxx8xceJEtG7dGt99951BY4mKijLo+Ws6d3d31KlTB6tWrWKiawBMstSIjo5GdHR0iY3wqOao6Hqp4iJ7BGBcRz+4OXK9FBER6V9ERAQiIiIMHQYZAa4IMiwmWWpERkYiMjJSOe+Sqq+Krpeq62CNlPQctfcX1aVhXbg76WefFCIiIiIyTkyyqEbQ9f5SluZm6DL/gMH2SCEiIiIi48Uki6qNovtLXS1IpJ7+W9b6qYruL2XMe6QQERERkeEwySKTYyz7SxXskVJ8nyxj2SOFiIiIiAyDSRYZLVPYXyo0yBO9Az1wIvEhUtKz4eZoYxQlfImIiIjIcJhkkcHper1UVe8vZW4mQ8cA1yo9JxERGYncTGCul/T9+3cAK3vDxkNERoFJFlWJouulrtzLKEyqUjKQnKb79VJERERE+hQREYHHjx9j+/btOuuze/fuaN26NZYuXQoA8PPzw+TJkzF58mQAQHJyMsLCwnDs2DFYWlri8ePHatuMzaFDh9CjRw88evQItWrVMng/VYFJFulURddL1XGwRkM3e+WIVEM3x0qtlyIiIqoSiiJ7al4/BgT0BMz0N6MiIiIC69atAwBYWFjAxcUFLVu2xJgxYxAREQEzM9U/QB47dgyffvop/vzzTzx58gSNGjVCREQEJk+eDHNz1TgPHjyIhQsX4vjx43jy5An8/PzQt29fTJ06FfXq1dPbcyrO2D5IX7t2Df7+/jh9+jRat26tbP/888/1vhfVyZMnYW9fODq6ZMkSJCUlIT4+XrnNkLo2QyqeKAJAp06dkJSUpFV8uurHUJhkUYWYwnopIiIivUrYAeyZXnj7+xGAkxcQOh8IHKS304aGhmLNmjWQy+W4e/cuYmJi8Pbbb2PLli3YsWMHLCykj3fbtm3DyJEjMX78eBw8eBC1atXC//3f/2H69On466+/sHnzZuUfMleuXInXX38d4eHh2Lp1K/z8/HDjxg2sX78eixYtwuLFi/X2fExVVXzQr1u3rsrtq1evom3btmjUqFGZbdrKy8uDpaX+PotZWVnBw8PDaPqpEoJKlZqaKgCI1NRUQ4diMOnZeeL0jUfip1M3RdSeC2LSupOix8KDosF7vwrfGbvUfjV6f7d4fvFh8fqGWLHot3/EL/G3xfnbqeJJbr6hnw4REZEQQognT56IhIQE8eTJk4p1cP4XIWY7CzHbqdiXs/R1/hcdRlsoPDxcDB48uET7/v37BQDx9ddfCyGEyMjIEK6urmLYsGEljt2xY4cAIDZu3CiEEOLmzZvCyspKTJ48We05Hz16pHF8J06cEM8995xwdXUVTk5O4tlnnxWxsbHK+xMTEwUAcfr0aZX+AYiDBw8q7y/6FR4eLoQQIjs7W7z55puibt26wtraWnTu3FmcOHFC2c/BgwcFABETEyNat24tbGxsRI8ePcTdu3fF7t27RdOmTYWjo6MYPXq0yMzMVD5uz549onPnzsLZ2Vm4uLiI/v37iytXrijvLx5Pt27dhBAlfxZyuVxERUWJgIAAYWVlJby9vcWnn35a6rXKyMgQYWFhwt7eXnh4eIjPPvtMdOvWTbz99tvKY3x9fcWSJUuU3xe/LurahBDi8ePH4uWXXxZ169YVjo6OokePHiI+Pl7Z7+zZs0WrVq3E6tWrhb+/v5DJZEKhUGj8uPXr1wtfX1/h5OQkRo0aJdLS0pTXpPj1SkxMVP5sCl5L9+/fF6NHjxb16tUTtra2IigoSPzwww/K82jajxBCbNmyRQQGBgorKyvh6+srPvvsM5Xr7OvrK/73v/+J8ePHCwcHB+Ht7S1WrlxZ6s9FiNJ/P2iTG3Aky0TIFUJvFewE10sRERFpTiEHYmYAarejFwBkQMxMoGl/vU4dLKpnz55o1aoVfv75Z0yaNAm//fYbHjx4gHfffbfEsQMHDkTjxo3x448/YtSoUfjpp5+Qm5uL6dOnq+kZWk3ZS09PR3h4OL744gsAwKJFi9CvXz9cvnwZjo6O5T7e29sbW7duxfDhw/HPP//AyckJtra2AIDp06dj69atWLduHXx9fbFgwQL06dMHV65cgYuLi7KPOXPmYNmyZbCzs8PIkSMxcuRIWFtb44cffkBGRgaGDh2KL7/8EjNmzAAAZGZmYurUqWjRogUyMzMxa9YsDB06FPHx8TAzM8OJEyfQvn17/N///R+aN28OKysrtbG/9957+Prrr7FkyRJ06dIFSUlJuHjxYqnPddq0aTh48CC2bdsGDw8PvP/++4iNjVWZkljUyZMnMW7cODg5OeHzzz+Hra0tcnNzS7QJIdC/f3+4uLhg9+7dcHZ2xsqVK9GrVy9cunRJea2uXLmCzZs3Y+vWrcqpo5o87urVq9i+fTt27dqFR48eYeTIkYiKisL//vc/fP7557h06RKCgoLw8ccfA5BG465du6byXLKzs9G2bVvMmDEDTk5O+PXXXxEWFoYGDRqgQ4cOGvcTGxuLkSNHYs6cORg1ahSOHTuG119/Ha6uroiIiFAet2jRInzyySd4//33sWXLFrz22mt49tln0bRp01J/PpXFJEuN6OhoREdHQy6Xl39wFYg5l1RiLybPCuzFxPVSREREOnD9GJB2p4wDBJB2WzrOv2uVhdW0aVP8/fffAIBLly4BAJo1a1bqsQXHXL58GU5OTvD0rPz+jj179lS5vXLlStSuXRuHDx/GgAEDyn28ubm58sO8m5ubMsHLzMzEihUrsHbtWvTt2xcA8PXXX2Pfvn1YvXo1pk2bpuzj008/RefOnQEAEydOxHvvvYerV6+iQYMGAIARI0bg4MGDyiRr+PDhKjGsXr0abm5uSEhIQFBQkHLKnqura6lT1dLT0/H5559j2bJlCA8PBwAEBASgS5cuao/PyMjA6tWrsX79evTu3RsAsG7dOtSvX7/Ua1O3bl1YW1vD1tZWJY7ibQcOHMDZs2eRkpICa2trAMBnn32G7du3Y8uWLXjllVcAALm5ufjuu++Uz0/TxykUCqxdu1aZNIeFhWH//v343//+B2dnZ1hZWcHOzq7MaX316tVT+QPAm2++iZiYGPz000/o0KGDxv0sXrwYvXr1wocffggAaNy4MRISErBw4UKVJKtfv354/fXXAQAzZszAkiVLcOjQISZZVS0yMhKRkZFIS0sz+MK6mHNJeG1DXIm/lSWnZuO1DXFYMTa4RKJV2fVSAXXti6ybcuR6KSIioqIy7ur2OB0RQpT446copTCDEEI5IqPucRWVkpKCWbNm4cCBA7h79y7kcjmysrJw48aNSvV79epV5OXlKZMnALC0tET79u1x4cIFlWNbtmyp/N7d3R12dnbKBKug7cSJEyp9f/jhh/jrr79w//59KBTSZ6UbN24gKChIo/guXLiAnJwc9OrVS+Pnk5ubi44dOyrbXFxc0KRJE40eX5bY2FhkZGTA1VV1e5knT57g6tWrytu+vr4qa740fZyfn5/KqKSnpydSUlK0ilEulyMqKgqbNm3C7du3kZOTg5ycHJUiH5q4cOECBg8erNLWuXNnLF26FHK5XDlCV/Q1IZPJ4OHhoXXM2mKSZcTkCoGPdiaUNRkB/91+DmnZ+fj3Xiau3jOt/aWIiIhMkoO7bo/TkQsXLsDf3x8AlEUQLly4gE6dOpU49uLFi8ppaY0bN0ZqaiqSkpIqPZoVERGBe/fuYenSpfD19YW1tTU6duyI3NxcAFBWPyya/OXllT6bpkDB8eqSyOJtRQs4yGSyEgUdZDKZMpECpOmT3t7e+Prrr+Hl5QWFQoGgoCBlzJoomNKoqdKSX11QKBTw9PTEoUOHStxXdOpn8YRG08eVdz01sWjRIixZsgRLly5FixYtYG9vj8mTJ2t1zQHN/7Cgi5i1xSTLiJ1IfKgyRbA4AeB+Ri6mb/m7xH1cL0VERKQnvp2kKoJpSVC/Lksm3e9bMrnRl4KpXlOmTAEA9OnTBy4uLli0aFGJJGvHjh24fPmysjT2iBEjMHPmTCxYsABLliwp0ffjx481Xpd19OhRLF++HP369QMA3Lx5E/fv31feXzBykpSUhDZt2gAA4uPjVfooGGErumyjYcOGsLKywu+//44XX3wRgJScnTp1SrmHVEU8ePAAFy5cwMqVK9G1qzS18/fffy83nuIaNWoEW1tb7N+/H5MmTSr3vA0bNoSlpSX++usv+Pj4AAAePXqES5cuoVu3bhV9OgCA4OBgJCcnw8LCAn5+fnp/XHFWVlblLrk5evQoBg8ejLFjxwKQErzLly+rTG/VpJ/AwMASP69jx46hcePGJbYoqGpMsoxYSnrpCVZRDd3s8UwDV66XIiIiqgpm5lKZ9s3jIM0rKZpoPf2/NzRKb0UvcnJykJycrFLCfd68eRgwYADGjRsHQBqlWLlyJUaPHo1XXnkFb7zxBpycnLB//35MmzYNkyZNUiZC3t7eWLJkCd544w2kpaVh3Lhx8PPzw61bt7B+/Xo4ODhg0aJFGsXWsGFDfPfddwgJCUFaWhqmTZumMspja2uLZ555BlFRUfDz88P9+/fx3//+V6UPX19fyGQy7Nq1C/369YOtrS0cHBzw2muvYdq0aXBxcYGPjw8WLFiArKwsTJw4scLXsnbt2nB1dcWqVavg6emJGzduYObMmSrHuLm5wdbWFjExMahfvz5sbGxKLCexsbHBjBkzMH36dFhZWaFz5864d+8ezp8/rzY+BwcHTJw4EdOmTYOrqyvc3d3xwQcflNjnrCKee+45dOzYEUOGDMH8+fPRpEkT3LlzB7t378aQIUMQEhKi08cV5+fnh+PHj+PatWtwcHBQKUpSoGHDhti6dSuOHTuG2rVrY/HixUhOTlZJsjTp55133kG7du3wySefYNSoUfjzzz+xbNkyLF++XMOrpT8c0jBibo42Gh33yeAW+HRIC0R09keXRnXg4WzDBIuIiEifAgcBI9cDjsUW5Tt5Se163CcrJiYGnp6e8PPzQ2hoKA4ePIgvvvgCv/zyi8pf7wsKPNy4cQNdu3aFv78/Jk2ahBkzZuDrr79W6fP111/Hb7/9htu3b2Po0KFo2rQpJk2aBCcnJ5UCBTKZDGvXri01tm+//RaPHj1CmzZtEBYWhrfeegtubm4ljsnLy0NISAjefvttfPrppyr316tXDx999BFmzpwJd3d3vPHGGwCAqKgoDB8+HGFhYQgODsaVK1ewd+9e1K5du6KXEmZmZti4cSNiY2MRFBSEKVOmYOHChSrHWFhY4IsvvsDKlSvh5eVVYg1QgQ8//BDvvPMOZs2ahWbNmmHUqFFlrvtZuHAhnn32WQwaNAjPPfccunTpgrZt21b4uRSQyWTYvXs3nn32WUyYMAGNGzfG6NGjce3aNbi7lz6FtaKPK+7dd9+Fubk5AgMDUbduXbXr8T788EMEBwejT58+6N69Ozw8PDBkyBCt+wkODsbmzZuxceNGBAUFYdasWfj4449Vil4Yikzoc1KoiSsofJGamgonJ6cqP79cIdBl/gEkp2aXNhkBHs42+H1GT52VcyciIqoJsrOzkZiYCH9/f9jYaPZHTfUdpQFR3tL3L20BAnpWWdl2bWVnZ2Pw4MG4efMmDh8+XGKj2/Jcu3YNjRo1QkJCQqU2viUydqX9ftAmN+BIlhEzN5Nh9sBAAMrJB0oFt2cPDGSCRUREZChFEyrfTkabYAHSlLZffvkF48aNw5EjR7R+fExMDF555RUmWEQa4EhWGQw9klVAV/tkERERkURnI1lEVO3oYiSLhS9MQGiQJ3oHeuBE4kOkpGfDzdEG7f1dOIJFRERERGSEmGSZCHMzGToGuJZ/IBERERERGRTXZBEREREREekQkywiIiKqsbg0nYiK08XvBSZZREREVONYWloCALKysgwcCREZm4LfCwW/JyqCa7KIiIioxjE3N0etWrWUm8Xa2dlBJmNBKaKaTAiBrKwspKSkoFatWiqbe2uLSRYRERHVSB4eHgCgTLSIiACgVq1ayt8PFcUki4iIiGokmUwGT09PuLm5IS8vz9DhEJERsLS0rNQIVgEmWWpER0cjOjoacrnc0KEQERGRnpmbm+vkQxURUQGZYFmdUmmzqzMREREREVVf2uQGrC5IRERERESkQ0yyiIiIiIiIdIhrsspQMJMyLS3NwJEQEREREZEhFeQEmqy2YpJVhvT0dACAt7e3gSMhIiIiIiJjkJ6eDmdn5zKPYeGLMigUCty5cweOjo5GsUFhWloavL29cfPmTRbiMDL82ZSvJlwjU32Oxhy3scRmqDiq8rz6Ppe++jeW14gx4zUyXvzZlM+YrpEQAunp6fDy8oKZWdmrrjiSVQYzMzPUr1/f0GGU4OTkZPAXGanHn035asI1MtXnaMxxG0tshoqjKs+r73Ppq39jeY0YM14j48WfTfmM5RqVN4JVgIUviIiIiIiIdIhJFhERERERkQ4xyTIh1tbWmD17NqytrQ0dChXDn035asI1MtXnaMxxG0tshoqjKs+r73Ppq39jeY0YM14j48WfTflM9Rqx8AUREREREZEOcSSLiIiIiIhIh5hkERERERER6RCTLCIiIiIiIh1ikkVERERERKRDTLJMzLx58yCTyTB58mRDh0IAbt++jbFjx8LV1RV2dnZo3bo1YmNjDR2W0cjPz8d///tf+Pv7w9bWFg0aNMDHH38MhUJh6NAq7MiRIxg4cCC8vLwgk8mwfft2lfuFEJgzZw68vLxga2uL7t274/z584YJtpiyYs/Ly8OMGTPQokUL2Nvbw8vLC+PGjcOdO3cMHltx//nPfyCTybB06VKDxHHhwgUMGjQIzs7OcHR0xDPPPIMbN27o9bwZGRl44403UL9+fdja2qJZs2ZYsWKF1ueZN28e2rVrB0dHR7i5uWHIkCH4559/VI6pzGu4vP4r+jrTJO6i9PkaMVYrVqxAy5YtlRu2duzYEXv27AFg+Pc3qVL3WU5X73FTps1nKlN4jzPJMiEnT57EqlWr0LJlS0OHQgAePXqEzp07w9LSEnv27EFCQgIWLVqEWrVqGTo0ozF//nx89dVXWLZsGS5cuIAFCxZg4cKF+PLLLw0dWoVlZmaiVatWWLZsmdr7FyxYgMWLF2PZsmU4efIkPDw80Lt3b6Snp1dxpCWVFXtWVhbi4uLw4YcfIi4uDj///DMuXbqEQYMGGTy2orZv347jx4/Dy8vLIHFcvXoVXbp0QdOmTXHo0CGcOXMGH374IWxsbPR63ilTpiAmJgYbNmzAhQsXMGXKFLz55pv45ZdftDrP4cOHERkZib/++gv79u1Dfn4+nn/+eWRmZiqPqcxruLz+K/o60yTuAvp+jRir+vXrIyoqCqdOncKpU6fQs2dPDB48GOfPnzf4+5sKlfZZTlfvcVOlzWcqk3mPCzIJ6enpolGjRmLfvn2iW7du4u233zZ0SDXejBkzRJcuXQwdhlHr37+/mDBhgkrbsGHDxNixYw0UkW4BENu2bVPeVigUwsPDQ0RFRSnbsrOzhbOzs/jqq68MEGHpiseuzokTJwQAcf369aoJ6qnSYrt165aoV6+eOHfunPD19RVLliyp8jhGjRql99evuvM2b95cfPzxxyptwcHB4r///W+lzpWSkiIAiMOHDwshdP8aLt6/OhV5nZXWb1W/Roxd7dq1xTfffKP2PkO9v2uysj7L6es9bio0/UxlSu9xjmSZiMjISPTv3x/PPfecoUOhp3bs2IGQkBC88MILcHNzQ5s2bfD1118bOiyj0qVLF+zfvx+XLl0CAJw5cwa///47+vXrZ+DI9CMxMRHJycl4/vnnlW3W1tbo1q0bjh07ZsDIKiY1NRUymcwoRmcVCgXCwsIwbdo0NG/e3GAx/Prrr2jcuDH69OkDNzc3dOjQocypjbrSpUsX7NixA7dv34YQAgcPHsSlS5fQp0+fSvWbmpoKAHBxcQGg+9dw8f5LO0bb15m6fo3hNWIs5HI5Nm7ciMzMTHTs2FHtMcb0/q4pyvosp6/3uKnQ5DOVqb3HmWSZgI0bNyIuLg7z5s0zdChUxL///osVK1agUaNG2Lt3L1599VW89dZbWL9+vaFDMxozZszAmDFj0LRpU1haWqJNmzaYPHkyxowZY+jQ9CI5ORkA4O7urtLu7u6uvM9UZGdnY+bMmXjxxRfh5ORk6HAwf/58WFhY4K233jJYDCkpKcjIyEBUVBRCQ0Px22+/YejQoRg2bBgOHz6s13N/8cUXCAwMRP369WFlZYXQ0FAsX74cXbp0qXCfQghMnToVXbp0QVBQEADdvobV9V9cRV5npfVrDK8RQzt79iwcHBxgbW2NV199Fdu2bUNgYGCJ44zt/V0TlPdZTh/vcVOiyWcqU3uPWxg6ACrbzZs38fbbb+O3336r9Jx/0i2FQoGQkBDMnTsXANCmTRucP38eK1aswLhx4wwcnXHYtGkTNmzYgB9++AHNmzdHfHw8Jk+eDC8vL4SHhxs6PL2RyWQqt4UQJdqMWV5eHkaPHg2FQoHly5cbOhzExsbi888/R1xcnEGvY0HBlsGDB2PKlCkAgNatW+PYsWP46quv0K1bN72d+4svvsBff/2FHTt2wNfXF0eOHMHrr78OT0/PCs9weOONN/D333/j999/L3GfLl7DZfUPVPx1pq5fY3mNGFqTJk0QHx+Px48fY+vWrQgPD8fhw4dVEi1je3/XBJp8ltPHe9yUlPeZyiTf4wacqkga2LZtmwAgzM3NlV8AhEwmE+bm5iI/P9/QIdZYPj4+YuLEiSpty5cvF15eXgaKyPjUr19fLFu2TKXtk08+EU2aNDFQRLqFYmtnrl69KgCIuLg4leMGDRokxo0bV8XRla147AVyc3PFkCFDRMuWLcX9+/erPjBRMrYlS5Yof+cV/T1oZmYmfH19qyyOnJwcYWFhIT755BOV46ZPny46deqkt/NmZWUJS0tLsWvXLpXjJk6cKPr06VOhc7zxxhuifv364t9//1Vp19VruLT+C1T0dVZav4Z6jRi7Xr16iVdeeUV52xje3zVReZ/lMjIydP4eNzXlfaYyxfc4R7KMXK9evXD27FmVtvHjx6Np06aYMWMGzM3NDRQZde7cuUQJ4UuXLsHX19dAERmfrKwsmJmpzko2Nzc36RLuZfH394eHhwf27duHNm3aAAByc3Nx+PBhzJ8/38DRlS8vLw8jR47E5cuXcfDgQbi6uho6JABAWFhYib/k9unTB2FhYRg/fnyVxWFlZYV27dpV+fs+Ly8PeXl5OnkvCSHw5ptvYtu2bTh06BD8/f1V7q/sa7i8/guej7avs/L6NZbXiLERQiAnJweA8b6/a4LyPsvJ5XKdvcdNVXmfqUzxPc4ky8g5OjqWmMtub28PV1fXUue4U9WYMmUKOnXqhLlz52LkyJE4ceIEVq1ahVWrVhk6NKMxcOBA/O9//4OPjw+aN2+O06dPY/HixZgwYYKhQ6uwjIwMXLlyRXk7MTER8fHxcHFxgY+PDyZPnoy5c+eiUaNGaNSoEebOnQs7Ozu8+OKLBoxaUlbsXl5eGDFiBOLi4rBr1y7I5XLlGhwXFxdYWVkZLDYfH58SHwgtLS3h4eGBJk2aVGkc06ZNw6hRo/Dss8+iR48eiImJwc6dO3Ho0CG9nrdbt26YNm0abG1t4evri8OHD2P9+vVYvHixVueJjIzEDz/8gF9++QWOjo7Kn7GzszNsbW2Ve/dU9DVcXv/5+fkVep2V16+rq2uVvUaM1fvvv4++ffvC29sb6enp2LhxIw4dOoSYmJgKX3fSDU0+y+nqPW6qyvtMZZLvccMOpFFFsIS78di5c6cICgoS1tbWomnTpmLVqlWGDsmopKWlibffflv4+PgIGxsb0aBBA/HBBx+InJwcQ4dWYQcPHhQASnyFh4cLIaQS2LNnzxYeHh7C2tpaPPvss+Ls2bOGDfqpsmJPTExUex8AcfDgQYPGpo6+SvdqEsfq1atFw4YNhY2NjWjVqpXYvn273s+blJQkIiIihJeXl7CxsRFNmjQRixYtEgqFQqvzlPYzXrNmjfKYyryGy+u/oq8zTeIuztjLO+vahAkThK+vr7CyshJ169YVvXr1Er/99psQouLXnfSn+Gc5Xb3HTZm2n6mM/T0uE0IInWRrRERERERExBLuREREREREusQki4iIiIiISIeYZBEREREREekQkywiIiIiIiIdYpJFRERERESkQ0yyiIiIiIiIdIhJFhERERERkQ4xySIiIiIiItIhJllEREREREQ6xCSLiIiIiIhIhywMHQAREZGx6t69O1q2bAkbGxt88803sLKywquvvoo5c+YYOjQiIjJiHMkiIiIqw7p162Bvb4/jx49jwYIF+Pjjj7Fv3z5Dh0VEREZMJoQQhg6CiIjIGHXv3h1yuRxHjx5VtrVv3x49e/ZEVFSUASMjIiJjxpEsIiKiMrRs2VLltqenJ1JSUgwUDRERmQImWURERGWwtLRUuS2TyaBQKAwUDRERmQImWURERERERDrEJIuIiIiIiEiHmGQRERERERHpEKsLEhERERER6RBHsoiIiIiIiHSISRYREREREZEOMckiIiIiIiLSISZZREREREREOsQki4iIiIiISIeYZBEREREREekQkywiIiIiIiIdYpJFRERERESkQ0yyiIiIiIiIdIhJFhERERERkQ4xySIiIiIiItIhJllEREREREQ69P8OchC3aKqteQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "fig.set_figwidth(10)\n",
    "\n",
    "adjoint_x_alkanes = [l for l in alkanes_performance['pyscf_adjoint_lda'].keys() if alkanes_performance['pyscf_adjoint_lda'][l]]\n",
    "adjoint_y_alkanes = [l[\"mean\"] for l in alkanes_performance['pyscf_adjoint_lda'].values() if l]\n",
    "adjoint_yerr_alkanes = [l[\"std_dev\"] for l in alkanes_performance['pyscf_adjoint_lda'].values() if l]\n",
    "adjoint_coeff_alkanes = numpy.polyfit(numpy.log(adjoint_x_alkanes), numpy.log(adjoint_y_alkanes), deg=1)\n",
    "adjoint_poly_alkanes = numpy.poly1d(adjoint_coeff_alkanes)\n",
    "adjoint_yfit_alkanes = [numpy.exp(adjoint_poly_alkanes(numpy.log(x))) for x in adjoint_x_alkanes]\n",
    "ax.errorbar(adjoint_x_alkanes, adjoint_y_alkanes, yerr=adjoint_yerr_alkanes, \n",
    "            linestyle='None', marker='o', color='tab:blue', label='PySCF, Hellmann-Feynman')\n",
    "ax.plot(adjoint_x_alkanes, adjoint_yfit_alkanes, linestyle='solid', color='tab:blue')\n",
    "\n",
    "\n",
    "dqc_x_alkanes = [l for l in alkanes_performance['dqc_onlygrad_lda'].keys() if alkanes_performance['dqc_onlygrad_lda'][l]]\n",
    "dqc_y_alkanes = [l[\"mean\"] for l in alkanes_performance['dqc_onlygrad_lda'].values() if l]\n",
    "dqc_yerr_alkanes = [l[\"std_dev\"] for l in alkanes_performance['dqc_onlygrad_lda'].values() if l]\n",
    "dqc_coeff_alkanes = numpy.polyfit(numpy.log(dqc_x_alkanes), numpy.log(dqc_y_alkanes), deg=1)\n",
    "dqc_poly_alkanes = numpy.poly1d(dqc_coeff_alkanes)\n",
    "dqc_yfit_alkanes = [numpy.exp(dqc_poly_alkanes(numpy.log(x))) for x in dqc_x_alkanes]\n",
    "ax.errorbar(dqc_x_alkanes, dqc_y_alkanes, yerr=dqc_yerr_alkanes, \n",
    "            linestyle='None', marker='o', color='tab:orange', label='DQC, automatic differentiation')\n",
    "ax.plot(dqc_x_alkanes, dqc_yfit_alkanes, linestyle='solid', color='tab:orange')\n",
    "\n",
    "plt.xscale('log')\n",
    "plt.yscale('log')\n",
    "\n",
    "plt.title(r\"Time of derivative calculation of C$_n$H$_{2n+2}$, LDA\")\n",
    "plt.xlabel(\"n\")\n",
    "plt.ylabel(\"Time, s\")\n",
    "\n",
    "ax.set_xticks(adjoint_x_alkanes)\n",
    "ax.get_xaxis().set_major_formatter(ticker.ScalarFormatter())\n",
    "ax.get_xaxis().set_minor_locator(ticker.NullLocator())\n",
    "\n",
    "plt.legend(loc='lower right')\n",
    "\n",
    "plt.savefig('figure_lda_3.png')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1EAAAHHCAYAAABeJdLdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAACEcElEQVR4nO3dd3iT1dsH8G+SNt1NaaGLDsqmQCmbIsjwhzIExAGoQFkqS0UUcQKOFxBlKBQUVBD3BAEFRQRE2aNlFGRYoJOymnSP5Lx/PG3aNB1J2zRp+/1cVy+ak5Pz3HmalNw957mPTAghQERERERERCaRWzsAIiIiIiKiuoRJFBERERERkRmYRBEREREREZmBSRQREREREZEZmEQRERERERGZgUkUERERERGRGZhEERERERERmYFJFBERERERkRmYRBEREREREZmBSRQREREREZEZmEQRERERERGZgUkUEenJZDKTvvbu3YuNGzdCJpPhypUr1g7bbN9++y3at28PJycnyGQyREdHm/X4hQsXQiaT1XhctXFODxw4gIULFyItLa3Wj10bLP08qjN+eee+uuPWlqq8b06dOoVJkyYhJCQEjo6OcHV1RZcuXbB06VLcvn3bpOMWnZtjx46Vef/999+PZs2amfFMbF/Rcy76srOzQ0BAACZNmoTExMRy+8lkMjRp0gT9+/fH9u3bKxyzrN/rRGQ6O2sHQES24+DBgwa333rrLezZswd//vmnQXtoaCjat2+PgwcPws/PrzZDrLYbN25g/PjxGDx4MNasWQMHBwe0bt3a2mEBAIYNG2bxc3rgwAG88cYbmDhxIjw8PGr12A1deecesP3zX5X3zfr16zFjxgy0adMGc+fORWhoKPLz83Hs2DF8+OGHOHjwIDZv3lxLz6Bu2rBhA9q2bYvs7Gz89ddfWLx4Mfbt24fTp0/DxcXFqJ8QAikpKVi9ejWGDx+OrVu3Yvjw4WWOWVpoaKjFnw9RfcIkioj0evXqZXC7SZMmkMvlRu0l769rLly4gPz8fIwbNw79+vWzdjgAgKysLDg7O6NJkyZWO6fWPDbZ/vk3931z8OBBTJ8+HYMGDcKWLVvg4OCgv2/QoEF4/vnnsXPnTkuGXC906NAB3bp1AwAMGDAAWq0Wb731FrZs2YLHH3+8zH4AMHjwYDRq1Ahff/21URJVui8RVQ2X8xFRlZReflS0xO3UqVN45JFHoFKp4OnpiTlz5qCgoAD//vsvBg8eDDc3NzRr1gxLly4tc9yLFy/iscceg7e3NxwcHNCuXTtERUWZFNPff/+Ne+65B25ubnB2dkbv3r3xyy+/6O+fOHEi+vTpAwAYM2YMZDIZ+vfvX+GYv/zyC8LDw+Hg4ICQkBC899575fY1Jfai83TixAk8/PDDaNSoEVq0aAHA+Jxu2bIFMpkMu3fvNjrW2rVr9ecbAC5duoRJkyahVatWcHZ2RtOmTTF8+HCcPn3a4Nhz584FAISEhFS4PNOcY5v63Mtz/vx5PProo/Dx8YGDgwOCgoIwYcIE5Obm6vuY8vyqM/7EiRPLXBJmytLN6p57oOzlfJW9nkvGd/bsWTz66KNQqVTw8fHB5MmToVarKz03phynKu+bRYsWQSaTYd26dQYJVBGlUokRI0aYFB8VK/qD1tWrVyvs5+joCKVSCXt7+9oIi6hB4kwUEdWo0aNHY9y4cXjqqaewa9cuLF26FPn5+fjjjz8wY8YMvPDCC/jqq68wb948tGzZEg8++KD+sbGxsejduzeCgoKwbNky+Pr64rfffsMzzzyDmzdvYsGCBeUed9++fRg0aBDCwsLwySefwMHBAWvWrMHw4cPx9ddfY8yYMXj99dfRo0cPzJw5E4sWLcKAAQPg7u5e7pi7d+/GyJEjERERgW+++QZarRZLly7F9evXjfqaG/uDDz6IsWPHYtq0acjMzCzz+Pfffz+8vb2xYcMG3HPPPQb3bdy4EV26dEFYWBgAICkpCV5eXliyZAmaNGmC27dv47PPPkPPnj1x8uRJtGnTBlOnTsXt27exatUq/PTTT/qlY6GhoUbX4phz7Or83GJiYtCnTx80btwYb775Jlq1aoXk5GRs3boVeXl5+g/gpjy/6oxfHdU992Ux5fVc0kMPPYQxY8ZgypQpOH36NF5++WUAwKefflph7JZ432i1Wvz555/o2rUrAgMDzTmVFdJqtSgoKDBqF0LU2DFs3aVLlwAYrwIoOjdCCFy/fh3vvvsuMjMz8dhjjxmNUdZ5lMlkUCgUlgucqD4SRETliIyMFC4uLmXet2HDBgFAxMXFCSGEWLBggQAgli1bZtAvPDxcABA//fSTvi0/P180adJEPPjggwZ977vvPhEQECDUarVB+6xZs4Sjo6O4fft2ubH26tVLeHt7i/T0dH1bQUGB6NChgwgICBA6nU4IIcSePXsEAPH9999X+vx79uwp/P39RXZ2tr5No9EIT09PUfrXp6mxF52n+fPnGx2v9DkVQog5c+YIJycnkZaWpm+LjY0VAMSqVavKjb2goEDk5eWJVq1aieeee07f/u677xodo7rHrs7PbeDAgcLDw0OkpqaW28ec51f6eZg6fmRkpAgODjZqL/p5lTe+ObGVd+7LGtfU13NRfEuXLjUYb8aMGcLR0VHfrzyWeN+kpKQIAGLs2LGV9jVF0bmp6Kvkzy4nJ0dMnDhRBAQECDc3N9GzZ0/xzz//1Egs5anpYxY950OHDon8/HyRnp4utm/fLpo0aSLc3NxESkqKQb/SXw4ODmLNmjVljlnWl0KhqNbzJ2qIuJyPiGrU/fffb3C7Xbt2kMlkGDJkiL7Nzs4OLVu2NFiSkpOTg927d2PUqFFwdnZGQUGB/mvo0KHIycnBoUOHyjxmZmYmDh8+jIcffhiurq76doVCgfHjxyMhIQH//vuvWc8jMzMTR48exYMPPghHR0d9u5ubm9E1BlWJ/aGHHjIpjsmTJyM7Oxvffvutvm3Dhg1wcHAw+CtzQUEBFi1ahNDQUCiVStjZ2UGpVOLixYs4d+6cWc/dnGNX5+eWlZWFffv2YfTo0ZVeD1SV52fO+NVR0+e+Kq/n0kvjwsLCkJOTg9TU1Bo9jjVt2rQJR48eNfoqWmpYpKCgACEhIfjnn3+QlpaG6dOnY8SIEcjKyqrysSdOnIiNGzeWe78ljglIy/fs7e3h5uaG+++/H76+vtixYwd8fHwM+pU8Nzt27EBkZCRmzpyJ1atXG41Z1nk8fPhwteIkaoi4nI+IapSnp6fBbaVSCWdnZ4NEpKhdo9Hob9+6dQsFBQVYtWoVVq1aVebYN2/eLLP9zp07EEKUWdnM399fP7457ty5A51OB19fX6P7SrdVJXZTq7C1b98e3bt3x4YNG/Dkk09Cq9Xiiy++wMiRIw3O9Zw5cxAVFYV58+ahX79+aNSoEeRyOaZOnYrs7GyTjlWVY1f356bVahEQEFBpLFV5fuaMXx01fe6r8nr28vIyuF20TLGi41vifQMAjRs3hrOzM+Li4kzqv2zZMhw7dgxOTk746aef4O/vj61bt6Jly5YG/dq1a1dmQQSVSoX4+Hj9bRcXF8yfP19/OzIyEs899xwuXryITp06mXw8c1R2zKratGkT2rVrBzs7O/j4+JT7e6P0uRk8eDCuXr2KF198EePGjTOoBlneeSQi8zCJIiKb0KhRI/1fwGfOnFlmn5CQkHIfK5fLkZycbHRfUlISAOmDnbnxyGQypKSkGN1Xuq0qsZuzz9SkSZMwY8YMnDt3Dv/99x+Sk5MxadIkgz5ffPEFJkyYgEWLFhm037x506ictjkqO3Z1fm6enp5QKBRISEioNI6qPD9zxnd0dDQoNFFyfEvEVhFLvJ5r8zgKhQL33HMPduzYgYSEhEqT2NOnT2P//v34/vvv8fHHH2PChAlYv3493nnnHbOPXZbz588jOztbX8DF0scr65hVVZ2EJywsDL/99hsuXLiAHj16VCsOIjLG5XxEZBOcnZ0xYMAAnDx5EmFhYejWrZvRV+m/thdxcXFBz5498dNPPxn85V2n0+GLL75AQECA2XtBubi4oEePHvjpp5+Qk5Ojb09PT8e2bdtqLHZTPProo3B0dMTGjRuxceNGNG3aFPfee69BH5lMZlQk4ZdffjHYmBMwbYbCnGNX57k7OTmhX79++P777ytNVkx9flUdv1mzZkhNTTUoGpKXl4fffvutwseZE5up594Sr+faPs7LL78MIQSeeOIJ5OXlGd2fn5+vfx+dPn0aCxcuREREBORyOVq0aFFjxSKysrIwfvx4vPbaa/oli5Y8XnnHtIaizZBtuXQ+UV3GmSgishnvv/8++vTpg759+2L69Olo1qwZ0tPTcenSJWzbts1o09+SFi9ejEGDBmHAgAF44YUXoFQqsWbNGpw5cwZff/21WTM/Rd566y0MHjxYv6+NVqvFO++8AxcXF9y+fbvGYq+Mh4cHRo0ahY0bNyItLQ0vvPAC5HLDv4Hdf//92LhxI9q2bYuwsDAcP34c7777rtEsQMeOHfXxRkZGwt7evtzKdqYeuzrPffny5ejTpw969uyJl156CS1btsT169exdetWfPTRR3BzczPr+VV1/DFjxmD+/PkYO3Ys5s6di5ycHHzwwQfQarUVjm9ObOWd+6IYSrLE67ksljpOREQE1q5dixkzZqBr166YPn062rdvj/z8fJw8eRLr1q1Dhw4dMHToUMTGxmLo0KH6x549e9bgdlXl5+dj9OjRCA0NxSuvvAJAqkxn6vHuv/9+/P333wCkxOi7777D7NmzAQAvvfQSXnrpJZOOWZJMJkO/fv30pe1rypkzZ/QV927duoWffvoJu3btwqhRo4xmgkv2LalFixZMuIjMYc2qFkRk26pSne/GjRsmjdGvXz/Rvn17o/a4uDgxefJk0bRpU2Fvby+aNGkievfuLd5+++1K492/f78YOHCgcHFxEU5OTqJXr15i27ZtBn3MqTImhBBbt24VYWFhQqlUiqCgILFkyRKjim3mxF7eeRKi4spvv//+u76S1oULF4zuv3PnjpgyZYrw9vYWzs7Ook+fPmL//v2iX79+ol+/fgZ9X375ZeHv7y/kcrkAIPbs2VOtY5v63MsTGxsrHnnkEeHl5aU/zxMnThQ5OTlmP7+ynocp4wshxK+//irCw8OFk5OTaN68uVi9erVJ1fmqe+7LG9eU13N5rydTqgiacxxz3zdFoqOjRWRkpAgKChJKpVK4uLiIzp07i/nz54vU1FRx/vx50aRJE4PHtGrVShw9etTouZRsK2nYsGFGlRW1Wq0YO3asGDFihMjPz9e3m3K8skRGRooNGzZU2Ke8YxZJT083uWphZc+5dL+SXyqVSoSHh4vly5cbvMYrq3K4fv36SuMiomIyIRrQBgtERERkM3744QesW7cOv//+OwBpxsfDwwNqtRpOTk5VHveJJ57AxYsXsXPnToOiNlU93sSJE9G/f39MnDjR7GMW+fXXX3H//fcjJiZGPytJRHUXl/MRERGRVZw+fdqget3Zs2fRokWLaiVQV69exccffwxHR0eDwhg7duywyPEqO2bfvn0BAHv27MHYsWOZQBHVE5yJIiIiIiIiMgOr8xEREREREZmBSRQREREREZEZmEQRERERERGZgUkUERERERGRGRp8dT6dToekpCS4ubnV2OaFRERERERU9wghkJ6eDn9/f6PN5Utq8ElUUlISAgMDrR0GERERERHZiPj4eAQEBJR7f4NPotzc3ABIJ8rd3d3K0RARERERkbVoNBoEBgbqc4TyNPgkqmgJn7u7O5MoIiIiIiKq9DIfFpYgIiIiIiIyA5MoIiIiIiIiMzCJIiIiIiIiMgOTKCIiIiIiIjMwiSIiIiIiIjIDkygiIiIiIiIzMIkiIiIiIiIyA5MoIiIiIiIiMzCJIiIiIiIiMgOTKCIiIiIiIjMwiSIiIiIiIjIDkygiIiIiIiIzMIkiIiIiIiIyA5MoIiIiIiIiMzCJIiIiqu/yMoGFKukrL9Pa0RAR1XlMooiIiKh6mKQRUQPDJIqIiIiIiMgMTKKIiIiIiIjMUOeTqPj4ePTv3x+hoaEICwvD999/b+2QiIiIiIioHqvzSZSdnR1WrlyJ2NhY/PHHH3juueeQmcn12ERERLVGpy3+/uoBw9tERPVQnU+i/Pz8EB4eDgDw9vaGp6cnbt++bd2giIiIGorYrUBUj+LbXz4MrOwgtRMR1VNWT6L++usvDB8+HP7+/pDJZNiyZYtRnzVr1iAkJASOjo7o2rUr9u/fX+ZYx44dg06nQ2BgoIWjJiIiIsRuBb6bAKQnG7ZrkqV2JlJEVE9ZPYnKzMxEp06dsHr16jLv//bbbzF79my8+uqrOHnyJPr27YshQ4bg2rVrBv1u3bqFCRMmYN26dbURNhERUcOm0wI75wEQZdxZ2LbzJS7tI6J6yepJ1JAhQ/D222/jwQcfLPP+5cuXY8qUKZg6dSratWuHlStXIjAwEGvXrtX3yc3NxahRo/Dyyy+jd+/eFR4vNzcXGo3G4IuIiKhes8Q1S1cPAJqkCjoIQJMo9SMiqmesnkRVJC8vD8ePH8e9995r0H7vvffiwAHpl7IQAhMnTsTAgQMxfvz4SsdcvHgxVCqV/otL/4iIqF6z1DVLGddrth8RUR1i00nUzZs3odVq4ePjY9Du4+ODlJQUAMA///yDb7/9Flu2bEF4eDjCw8Nx+vTpcsd8+eWXoVar9V/x8fEWfQ5ERERWY8lrllx9Ku9jTj8iojrEztoBmEImkxncFkLo2/r06QOdTmfyWA4ODnBwcKjR+IiIiGxOpdcsyaRrltoOA+QK88cP7g24+0sJWZnHkEn3B1e8zJ6IqC6y6Zmoxo0bQ6FQ6GediqSmphrNThEREVEJlr5mSa4ABr9TeENW6s7C24OXVC1BIyKycTadRCmVSnTt2hW7du0yaN+1a1elBSSIiIgatNq4Zil0BDB6E+Dma9ju7i+1h46o+thERDbM6sv5MjIycOnSJf3tuLg4REdHw9PTE0FBQZgzZw7Gjx+Pbt26ISIiAuvWrcO1a9cwbdo0K0ZNRERk42rrmqXQEUDz/sCSwkJNj/8AtBjIGSgiqtesnkQdO3YMAwYM0N+eM2cOACAyMhIbN27EmDFjcOvWLbz55ptITk5Ghw4d8OuvvyI4OLhax42KikJUVBS0Wu5fQURE9VBtXrNUMmEK7s0EiojqPZkQoqzfrA2GRqOBSqWCWq2Gu7u7tcMhIiKqOUXV+QAYJlKF1yzV1JK7vExgkb/0/StJgNKl+mMSEVmBqbmBTV8TRURERNXAa5aIiCzC6sv5iIiIyIJ4zRIRUY1jEkVERFTfWfKapdx04MQmQBUEaPMAuX3NjU1EZKOYRBEREZH51InA4Q+B4xuBXI3U5twYuPkv4NvRqqEREVlag02iWJ2PiIioCpJPAQdXA2d+BHQFUptXKyBiJtBpLGDvZN34iIhqAavzsTofERHVd9WtnicEcOkP4MAqIG5fcXtwH6D300CrewE5a1URUd1nam7QYGeiiIiIqBIFucCp74CDUcCNc1KbTAG0fwCImAU07WLV8IiIrIVJFBERERnKug0c+wQ4sh7IuC61KV2BLpFAr2mAR5B14yMisjImUURERCS5/R9wcA0Q/SWQnyW1uflLiVOXSMDJw6rhERHZCiZRREREDd21w8DBVcC57QAKL5X27Qj0fgZoPwpQsGw5EVFJTKKIiIgaIp0WOL8dOLAaSDhS3N5yENB7FhDSD5DJrBcfEZENYxJFRETUkORlASe/BA5FAXeuSG0KJRA2WioW4d3OquEREdUFDTaJ4j5RRERkVdUtO15Vq7sBOWnS906NgO5Tge5PAG4+tXN8IqJ6oMEmUTNnzsTMmTP1teCJiIjqpdRzwN8rim/npAGNQqTNccMfq73kjYioHmmwSRQREVG9JQTw317g4Gppk9ySHvpEKhYhV1glNCKi+oBJFBERUX1RkAec/UkqFnH9tNQmkwNt7wd6Pw0E9rBufERE9QSTKCIiorouOw04vhE4/BGQniS12TsDnccDvaYDniHWjI6IqN5hEkVERFRX3bkKHP4QOLEJyMuQ2lx9gJ5PAV0nAc6e1o2PiKieYhJFRERU1yQel5bsxf4MiMIqs96hUonyjg8Ddg7WjY+IqJ5jEkVERFQX6HTAxd+AA6uAq/8UtzcfIG2O2+Iebo5LRFRLGmwSxX2iiIioTsjPBmK+Bg6uAW5dlNrkdkDHR6Qy5b4drRsfEVEDJBNCCGsHYU1F+0Sp1Wq4u7tbOxwiImooKttsN+MGcPRj4Oh6IOuW1OagArpNkq55cvev3XiJiBoAU3ODBjsTRUREZJNuXpT2d4r5BijIkdpUQUDEDKDzOMDBzbrxERERkygiIiKrEwK48rdULOLCjuJ2/y7S/k7tRgAK/pdNRGQr+BuZiIjI2jYOBZJjCm/IgDZDpWIRQREsFkFEZIOYRBEREdW23HTpeqciyTGAnSMQ/hjQaybQuKX1YiMiokoxiSIiIqot6kRpc9zjnwG56uL2jqOBe98G3HysFxsREZmMSRQREZGlJZ+SikWc+RHQFUhtMkXxRrmnvwOu/g0MfgcIHWG9OImIyCRyawdARERULwkBXPwD+GwE8FFf4NS3UgLVpF3h/aX2KdQkA99NAGK31n6sRERkFiZRRERENakgFzjxObAmAvjyISBunzTr1OFhYOpuw2V8Bgq3bdz5EqDjRvBERLaswS7ni4qKQlRUFLRa/kdFREQ1IOs2cOwT4Mh6IOO61KZ0A7pGSpvjegQBcfsBTVIFgwhAkwhcPQCE9K2VsImIyHwNNomaOXMmZs6cqd+VmIiIqEpu/wccXANEfwnkZ0lt7k2BntOkBMqxxP8xRclVZUztR0REVtFgkygiIqJquXYYOLgKOLcd+qV4vmHS5rjtRwEKe+PHuJpYfc/UfkREZBVMooiIiEyl0wLntwMHVgMJR4rbW90LRMwCQu6ueHPc4N6Au79URKIo8TIgk+4P7l3TkRMRUQ1iEkVERFSZvEzg5JfAoSjgzhWpTaEEwsZIyZN3W9PGkSukMubfTQAgg2EiVZh8DV4i9SMiIpvFJIqIiKg86SnAkXXA0U+AnDSpzakR0H0q0P2Jqm2OGzoCGL0J2PEikJ5c3O7uLyVQ3CeKiMjmMYkiIiIq7XoscDBK2gRXmye1eTYHes0Awh8DlC7VGz90BNC8P7AkULr9+A9Ai4GcgSIiqiOYRBEREQHS5rj/7QUOrgYu/VHcHtgL6D0LaDO0ZpOckmMF92YCRURUhzCJIiKihq0gDzj7k1Qs4vppqU0mB9oNByKeBgK7Wzc+IiKyOUyiiIioYcpOA45vBA6tBTJSpDZ7Z6DzeKDXdMAzxJrRERGRDWMSRUREDcudq8DhD4ETm4C8DMP7Zh0FVAHWiYuIiOoMJlFERNQwJB6XluzFbgGETmrzDgV6PAFsf0667dTIauEREVHdwSSKiIjqL50OuLBTKhZx9Z/i9uYDpGIRLe4B8rOKkygiIiITNNgkKioqClFRUdBqtdYOhYioYcrLBBb5S9+/klT9suEl5WcDMV9LZcpvXZLa5HZAx0eAiJmAb8eaOxYRETU4DTaJmjlzJmbOnAmNRgOVSmXtcIiIqCZk3ACOfgwcXQ9k3ZLaHFRAt0lAz6ekDW2JiIiqqcEmUUREVI/cvCgt2Yv+GtDmSm0eQdLmuJ3HAQ5u1o2PiIjqFSZRRERUNwkhXed0YDVwYUdxu38XoPfTQLsRgIL/zRERUc3j/y5ERFS3aAukCnsHVwNJJwsbZUCboVKxiKAIQCazZoRERFTPMYkiIqK6ITdd2tvp0IeA+prUZucIhD8G9JoJNG5p3fiIiKjBYBJFRES2TZ0obY57fCOQq5HanBsDPZ4Euk8BXBpbNTwiImp4mEQREZFtSj4lLdk78yOgK5DaGreWSpSHjQHsnawbHxERNVhMooiIyHYIAVzaDRz4AIjbV9zerC8QMQtodS8gl1svPiIiIjCJIiIiW1CQC5z5Sdoc98Y5qU2mANqPkopF+He2bnxEREQlMIkiIiLri+oJZKZK3yvdgK6RQM9pgEeg5Y+t0xZ/f/UA0GIgIFdY/rhERFRnMYkiIqLad/s/4O8VxbczUwE3f6DXdCmBclTVThyxW4EdLxbf/vJhwN0fGPwOEDqidmIgIqI6h0kUERHVnmuHgYOrgHPbyr6/UbPaTaC+mwBAGLZrkqX20ZuYSBERUZl4dS4REVmWTgvE/gx8PAj49N7yE6j0wuQldmvtxLRzHowSKKC4bedLhkv9iIiICnEmioiILCMvEzj5JXAoCrhzRWqT2wMKeyA/q4wHCAAyKXlpO8yy1yVdPQBokiroIABNotQvpK/l4iAiojqJSRQREdWs9BTgyDrg6CdATprU5tQI6D4V8A0DvhtfwYNrKXnJuF6z/YiIqEFpsElUVFQUoqKioNVyqQYRUY24HiuVKD/9HaDNk9oahUib44Y/BihdgNM/mDaWpZMXV5+a7UdERA1Kg02iZs6ciZkzZ0Kj0UClqqWLmImI6hshgP/2AgdXA5f+KG4P7CXt79RmqOGyPFtJXoJ7S1X4NMko+7oomXR/cG/LxkFERHVSg02iiIioGgrygLM/AQdWA9dPS20yOdBuOBDxNBDYvezH2UryIldIZcy/myAd0yAWmfTP4CWWvS5L6QIsVFtufCIishgmUUREZLrsNOD4RuDwR0B6YWEGe2eg83hpjyfPkIofbwvJS5HQEVIZ8x0vSpUBi7j7SzGwvDkREZVDJoQo60+BDUbRcj61Wg13d3drh0NEZJvuXAUOfwic2ATkZUhtrr5AzyeBrpMAZ0/zxiva5NYgeWlqneQlRwMsCZS+f/wHoMXA2kniiIjI5piaG3AmioiIypd4XFqyF7sFEDqpzTsUiJgFdHwYsHOo2rihI4Dm/W0jeSl5zODeTKCIiKhSTKKIiOqzvExgkb/0/StJ0nU4ldHpgAs7pWIRV/8pbm8+QCoW0eIeQCarfmxMXoiIqI5iEkVERJL8bCDma6lM+a1LUpvcDuj4iFSm3LejdeMjIiKyEUyiiIgauowbwNGPgaPrgaxbUpuDCug2Cej5lFRogYiIiPSYRBERNVQ3L0pL9qK/BrS5UptHENBrBtB5HODgZt34iIiIbBSTKCKihkQI6TqnA6uBCzuK2/27AHc9A7QdDij4XwMREVFF+D8lEVFDcXYLcGQdkBxd2CAD2gyVikUERdRMsQgiIqIGgEkUEVF9lptR/P3PM6R/7RyB8MeAXjOBxi2tExcREVEdxiSKiKg+UidKm+Me31Dc5uwF9HgK6D4FcGlsvdiIiIjqOCZRRET1SXKMdL3T2Z8AXYHhfTOPMHkiIiKqAXJrB0BERNUkBHBxF/DZcOCju4HT30kJVLO+wMMlZqKSTgI6rfXiJCIiqic4E0VEVFcV5AKnvpPKlN84L7XJFED7UVKxiLR4YMeLxf2/fFja82nwO0DoCOvETEREVA8wiSIiqmuybgPHPgEOrwMyU6U2pRvQNRLoOQ3wCARitwLfTQAgDB+rSZbaR29iIkVERFRFTKKIiOqKW5eBQ2uAk18CBdlSm3tTKXHqGgk4qqQ2nRbYOQ9GCRRQ2CYDdr4EtB0GyBW1FDwREVH9wSSKiMjWXTsMHPgAOP8L9ImRbxjQ+2lp6Z7C3rD/1QOAJqmCAQWgSZT6hfS1VNRERET1FpMoIiJbpNMC57cDB1YBCUeL21vdC0TMAkLuLn9z3Izrph3D1H5ERERkgEkUEZEtyc0Aor+Ulu3duSK1KZRA2BgpefJuW/kYrj6mHcvUfpaidAEWqq0bAxERURU02CQqKioKUVFR0GpZ7peIbEB6CnD4I+DYp0BOmtTm1AjoPhXo/gTgZkbCE9xbqsKnSUbZ10XJpPuDe9dA4ERERA2PTAhR1v+wDYZGo4FKpYJarYa7u7u1wyGihuZ6rFSi/NR3gC5favNsDvSaAYQ/Js3WVIW+Oh9gmEgVLgFkdb5ieZnAIn/p+1eSqn7OiYiozjM1N2iwM1FERFYjBPDfXul6p8u7i9sDe0n7O7UZWv2qeaEjpERpx4tAenJxu7s/MHgJEygiIqJqYBJFRGSKmpitKMgDzvwozTxdPyO1yeRAu+FAxNNAYPeaixeQEqXm/YElgdLtx38AWgxkWXMiIqJqYhJFRGRp2WnA8Q3SNU9Fs0L2zkDn8UCv6YBniOWOXTJhCu7NBIqIiKgGMIkiIrKUO1eBQ2uBk58DeRlSm6sv0PNJoOskwNnTuvGRhFUCiYjITEyiiIhqWsJx4OAqIPZnQOikNu9QqUR5x4cBOwfrxkdERETVwiSKiKgm6HTAhR3AgdXAtQPF7c0HSMUiWtxT/ua4REREVKcwiSIiqo78bCD6K2lz3FuXpDa5HdDxESBiJuDb0brxERERUY1jEkVEVBUZN4Cj64GjHwNZt6Q2BxXQbRLQ8ymplDgRERHVS0yiiIjM9etc4PQPgDZXuq0KAiJmAJ3HAQ5u1o2NiIiILI5JFBFRZYQArh4svh39pfSvfxeg99NAuxGAgr9OiYiIGgr+r09EVB5tvlRh78AqIDm6uL31fcBds4GgCBaLICIiaoCYRBERlZajAU5sAg5/CKjjpTaFQ/Hyve5PAIE9mUARERE1UEyiiIiKqBOkxOn4Z0CuRmpzbgyE3A1c/QfIuC61ffmwVDhi8DtA6AjrxUtERERWwSSKiCg5Rtrf6exPgK5AamvcWipRrnQFfpwKQBg+RpMMfDcBGL2JiRQREVEDwySKiBomIYBLfwAHPgDi/ipub9YXiJgFtLoXgABWdoBRAiUNAEAG7HwJaDsMkCtqJ24iIiKyOiZRRNSw5OcAp78DDkYBN85LbTIF0H4U0HsW4N+5uG/cfkCTVMFgAtAkAlcPACF9LRo2ERER2Q4mUUTUMGTdBo5+AhxZB2SmSm1KN6BrJNBzGuARaPyYomugKmNqP2tQugAL1daOgoiIqF5hEkVE9duty8ChNcDJL4GCbKnNvamUOHWNBBxV5T/W1ce0Y5jaj4iIiOoFJlFEVD9dOyxd73T+F+ivafINkzbHbT8KUNhXPkZwb6kKnyYZZV8XJZPuD+5dg4ETERGRrWMSRUT1h04LnN8ubY6bcLS4vdW9UrGIkLvN29tJrpDKmH83AYAMholU4TiDl7CoBBERUQPDJIqI6r7cDCD6S2nZ3p0rUptCCYSNkZIn77ZVHzt0hFTGfMeLQHpycbu7v5RAsbw5ERFRg8MkiojqrvQU4PBHwLFPgZw0qc2pEdB9KtD9CcCthq5VCh0BNO8PLCksPvH4D0CLgZyBIiIiaqCYRBFR3XM9Fji4Gjj1HaDLl9o8mwO9ZgDhj0kV6WpayYQpuDcTKCIiogaMSRQR1Q1CAP/tla53ury7uD2wl7S/U5uhTGyIiIioVjCJIiLbVpAHnP1JSp6un5HaZHKg3XAg4mkgsLt14yMiIqIGh0kUEdmm7DTg+Ebg8IfFBR3sXYDO44Be0wHPEGtGR0RERA0Ykygisi13rgKH1gInPwfyMqQ2V1+g55NA10mAs6d14yMiIqIGj0kUEVVfXiawyF/6/pWkqhV2SDwuLdmL/RkQOqnNO1QqUd7xYcDOoebiJSIiIqoGJlFEZD06HXBhp5Q8XTtQ3N58gFQsosU95m2OS0RERFQLmEQRUe3LzwZivgYORgG3LkltcntpxiliJuDb0brxEREREVWgXiRRo0aNwt69e3HPPffghx9+sHY4RFSejBvA0Y+Bo+uBrFtSm4MK6DYJ6PkU4O5v3fiIiIiITFAvkqhnnnkGkydPxmeffWbtUIioLDcuSJvjxnwDaHOlNo8gaXPczuMABzfrxkdERERkhnqRRA0YMAB79+61dhhEVJIQwNV/pOudLuwsbvfvAtz1DNB2OKCoF7+CiIiIqIGRWzuAv/76C8OHD4e/vz9kMhm2bNli1GfNmjUICQmBo6Mjunbtiv3799d+oERUPp22+Pu4/cCp74B1/YGNwwoTKBnQZhgwaQfwxJ9A+1FMoIiIiKjOsvqnmMzMTHTq1AmTJk3CQw89ZHT/t99+i9mzZ2PNmjW466678NFHH2HIkCGIjY1FUFCQFSImIgOxW4EdLxbf/npM8fd2jkD4Y0CvmUDjlrUfGxEREZEFWD2JGjJkCIYMGVLu/cuXL8eUKVMwdepUAMDKlSvx22+/Ye3atVi8eLHZx8vNzUVubq7+tkajMT9oIpLEbgW+mwBAlH3/sOVA58drNSSLUboAC9XWjoKIiIhsgNWX81UkLy8Px48fx7333mvQfu+99+LAgQPlPKpiixcvhkql0n8FBgbWRKhEDU/iCWDzkyg3gYIM2PN/hkv9iIiIiOoBm06ibt68Ca1WCx8fH4N2Hx8fpKSk6G/fd999eOSRR/Drr78iICAAR48eLXfMl19+GWq1Wv8VHx9vsfiJ6h0hgIu7gM+GA+sHSPs9ld8Z0CQCV6v2Bw8iIiIiW2X15XymkMlkBreFEAZtv/32m8ljOTg4wMHBocZiI2oQCnKlYhEHVwM3zhc2ylD+LFQJGdctGRkRERFRrbPpJKpx48ZQKBQGs04AkJqaajQ7RUQWkHUbOPYJcHgdkJkqtSndgK6RUqnyHydXPoYr36tERERUv9h0EqVUKtG1a1fs2rULo0aN0rfv2rULI0eOtGJkRPXcrcvAoTXAyS+BgsIle+5NgZ7TpATKUSVd67TrNUCTjLJnpGSAuz8Q3Ls2IyciIiKyOKsnURkZGbh06ZL+dlxcHKKjo+Hp6YmgoCDMmTMH48ePR7du3RAREYF169bh2rVrmDZtmhWjJqqnrh0GDnwAnP8F+sTINwzo/QzQ/gFAYV/cV64ABr9TWJ2v9NK+wuW2g5dI/YiIiIjqEasnUceOHcOAAQP0t+fMmQMAiIyMxMaNGzFmzBjcunULb775JpKTk9GhQwf8+uuvCA4OrtZxo6KiEBUVBa2WlcOogdNpgfPbgQOrgIQSRVla3Qv0fhpo1hcodV2iXugIYPQmaZ+o9OTidnd/KYEKHWHZ2ImIiIisQCaEMOHK8PJpNBr8+eefaNOmDdq1a1dTcdUajUYDlUoFtVoNd3d3a4dDVHtyM4DoL6Vle3euSG0KJdBprLQ5rndb08fK0QBLCrcLePwHoMVAzkARERFRnWNqbmD2TNTo0aNx9913Y9asWcjOzka3bt1w5coVCCHwzTff4KGHHqpW4ERkYekpwOGPgGOfAjlpUptTI6D7VKD7E4BbFQpBlEyYgnszgSIiIqJ6zewk6q+//sKrr74KANi8eTOEEEhLS8Nnn32Gt99+m0kUka26HiuVKD/1HaDLl9o8mwMRM4FOjwFKZ+vGR0RERFRHmJ1EqdVqeHp6AgB27tyJhx56CM7Ozhg2bBjmzp1b4wESUTUIAfy3V7re6fLu4vbAXtL1Tm2GcNaIiIiIyExmJ1GBgYE4ePAgPD09sXPnTnzzzTcAgDt37sDR0bHGAySiKijIA878KM08XT8jtcnkQLvhQMTTQGB368ZHREREVIeZnUTNnj0bjz/+OFxdXREcHIz+/fsDkJb5dezYsabjIyJzZKcBxzdI1zwVVcuzdwG6jJf2ePIMsWp4RERERPWB2UnUjBkz0LNnT1y7dg2DBg2CXC4HADRv3hxvv/12jQdoKSxxTvXKnavAobXAyc+BvAypzdUX6PkU0G2SVDiCiIiIiGpEtUuc13UscU51WsJx4OAqIPZnQOikNu9Q6XqnDg8DdsraiSMvE1jkL33/ShKgdKmd4xIRERHVIIuVOCciK9PpgAs7gAOrgWsHitubD5CSpxYDy98cl4iIiIiqjUkUUV2RlwXEfC1tjnvrktQmtwc6PiKVKfftYL3YlC7AQrX1jk9ERERUi5hEEVXG2kvVMm4AR9cDR9YD2belNkcV0G0y0OMpwN2vduMhIiIiauCYRBHZqhsXpBLlMd8A2lypzSMI6DUT6DwOcHC1bnxEREREDVSNJlHXrl1D06ZNoVBw806iKhECuPK3lDxd2Fnc3rSrdL1T2+GAgn/7ICIiIrKmGv001qxZM7Rq1QqLFy/Ggw8+WJND1ziWOCebos2XKuwdWAUkRxc2yoC2w4CIWUBQLxaLICIiIrIRNVrifN++fYiLi8Pvv/+Or776qqaGtSiWOKdK5WiAJYHS94//IFW/k9fQbGuOBjixCTj8IaCOl9rsHIHwx6ViEV4tauY4RERERFQpU3MD7hPFJIoqErsV2PEikJ5c3ObuDwx+BwgdUfVx1QlS4nT8MyBXI7W5NAF6PAl0mwK4eFUvbiIiIiIym0X3iSooKMDevXtx+fJlPPbYY3Bzc0NSUhLc3d3h6sqL3ameiN0KfDcBQKm/M2iSpfbRm8xPpJJjpP2dzv4E6AqktsZtpFmnsDGAvWONhE5ERERElmN2EnX16lUMHjwY165dQ25uLgYNGgQ3NzcsXboUOTk5+PDDDy0RJ1Ht0mmBnfNglEABhW0yYOdL0jVLlS3t0+mAS38AB1cBcX8VtzfrKxWLaDkIkMtrMHgiIiIisiSzk6hnn30W3bp1Q0xMDLy8ipccjRo1ClOnTq3R4Iis5uoBQJNUQQcBaBKlfiF9y+6SnwOc/g44GAXcOC+1yRRAhwelYhH+4TUdNRERERHVArOTqL///hv//PMPlEqlQXtwcDASExNrLDAiq8q4XvV+WbeBo58ARz4CMm9IbUo3oGsk0HMa4BFYc3ESERERUa0zO4nS6XRllgVPSEiAm5tbjQRFZHWuPub3u3VZmnWK/gooyJba3AOAXtOALhMAR1XNx0lEREREtc7sJGrQoEFYuXIl1q1bBwCQyWTIyMjAggULMHTo0BoP0FK4TxRVKLi3VIVPk4yyr4uSSfcHRQDXDkn7O53/pbivXycg4mmg/QOAwr724iYiIiIiizO7xHlSUhIGDBgAhUKBixcvolu3brh48SIaN26Mv/76C97e3paK1SJY4pzKpa/OBxgmUoWb3kbMAq4dBBKPFd/V6j6g9yypaAQ3xyUiIiKqUyy6T1R2dja+/vprnDhxAjqdDl26dMHjjz8OJyenagVtDUyiqEJl7RPlqALkSiCr8HonhQPQaYyUVDVpY504iYiIiKjauNmuiZhEUaVyNMCSwmIQ9i5Afqb0vZMn0H0q0OMJwLVuzcASERERkTGLbrabmJiIf/75B6mpqdDpdAb3PfPMM1UZksg2XT8L/LOy+HZ+JuDZQtoct9OjgNLZaqERERERkXWYnURt2LAB06ZNg1KphJeXF2QlrvuQyWRMoqjuEwL4b49ULOLyn4b3PfQJ0H5U5RvsEhEREVG9ZXYSNX/+fMyfPx8vv/wy5HK5JWIiso6CPODMD1KZ8utnpDaZHGgzDDi/TbrdZggTKCIiIqIGzuwkKisrC2PHjmUCRfVH9h3g+Ebg8EfFBSTsXYAu44Fe0wGXJsCibVYNkYiIiIhsh9lJ1JQpU/D999/jpZdeskQ8RLXnzhXg0FrgxOfFxSJcfYGeTwHdJgFOjYr7LlRbJUQiIiIisj1mV+fTarW4//77kZ2djY4dO8Le3nAj0eXLl9dogJbG6nwNUMIx6Xqnc1sBUVgYxbs90PtpoMNDgJ3SuvERERERkVVYrDrfokWL8Ntvv6FNG2k/nNKFJYhskk4HXNghJU/XDha3txgoJU/NB3BzXCIiIiIyidlJ1PLly/Hpp59i4sSJFgin9kRFRSEqKgpardbaoZAl5WUBMV9LxSJuX5ba5PZAx0ekMuW+HawbHxERERHVOWYv5/P19cX+/fvRqlUrS8VUq7icr57KSAWOrAeOfgxk35baHFVAt8lAj6cAdz/rxkdERERENsdiy/meffZZrFq1Ch988EG1AiSyiBv/AgdXAzHfAtpcqc0jWJp1Cn8ccHC1bnxEREREVOeZnUQdOXIEf/75J7Zv34727dsbFZb46aefaiw4IpMIAVz5W7re6eJvxe1NuwG9ZwFthwMKs1/qRERERGRBWp3AkbjbSE3PgbebI3qEeEIhrxvXqJv9ydLDwwMPPvigJWIhMo82Hzi7BTi4CkiOKWyUAW2HScUiAnuyWAQRERGRDdp5JhlvbItFsjpH3+ancsSC4aEY3MH2L7sw+5qo+obXRNVBORrgxCZpjydNgtRm5wSEPyYt2/NqYd34iIiIiKhcO88kY/oXJ1A6CSn60/facV2slkhZ7JooIqtRJxRujrsJyNVIbS5NgB5PAt2mAC5e1o2PiIiIiCqk1Qm8sS3WKIECAAEpkXpjWywGhfra9NI+k5KoLl26YPfu3WjUqBE6d+5c4X5QJ06cqLHgiAAASdFSsYizmwFdgdTWuI10vVPH0YC9o1XDIyIiIiLTHIm7bbCErzQBIFmdgyNxtxHRwnb/QG5SEjVy5Eg4ODgAAB544AFLxkMk0emAS38ABz4Aruwvbm/WV7reqeUgQC63XnxEREREZLbU9PITqKr0sxaTkqgFCxZg8uTJeP/997FgwQJLx0QNWX4OcOpbaXPcm/9KbTIF0OFBIGIW4B9u1fCIiIiIqOq83UxbQWRqP2sx+Zqozz77DEuWLIGbm5sl46GGKvMWcOwT4Mg6IPOG1KZ0A7pGAj2nAR6B1o2PiIiIiMyWmVuAM4lqRMenISYhDSev3qmwvwyAr0oqd27LTE6iGngRP7KUW5elWafor4CCbKnNPQDoNQ3oMgFwVFk3PiIiIiIySYFWh4upGVLCFJ+G6Pg0XLieDp2JaURR1YUFw0NtuqgEYGZ1vooKStQ1UVFRiIqKglartXYoDY8QQPxhaXPc878ARfVZ/DoBEU8D7R8AFPYVjUBEREREViSEQJI6B9HXpBmm6GtpOJ2oRna+8WdrP5UjwgM90CnQA50CPNAxQIW/L94w2ifKtz7uEyWXy6FSqSpNpG7fvl0jgdUW7hNVi7QFwPltwIHVQOKx4vZW90nFIpr14ea4RERERDZInZ2PUwlFM0zS8rybGblG/Vwd7BAWoNInTeGBHvBxL/v6Jq1O4EjcbaSm58DbTVrCZ+0ZKIvsE/XGG29ApeLyKjJTbgZw8gvg0Bog7arUpnAAOo2RikU0aWPd+IiIiIhIL69Ah3PJGmmGqXBZ3n83Mo362cllaOfnjk6BKnQK8EDnIA80b+wKuYmJkEIus+ky5hUxK4kaO3YsvL29LRUL1TeaZODIR8CxT4EctdTm5Al0nwr0eAJw5WuJiIiIyJqEELhyK0t/DVN0fBpikzTI0+qM+gZ5OhvMMLX3d4ejvcIKUVufyUlUfboeiizs+llpyd7p7wFdvtTm2QKImAF0egxQOls3PiIiIqIG6lZGbuEMk1pfAEKdnW/Uz8PZXkqYAjwQHiT96+mitELEtonV+ahmCAFc/hM4uFr6t0hQhLRkr80QQN4w/1JBREREZA05+Vp9efGiEuPxt7ON+int5Gjv747wwhmm8EAPBHk6cxKlAiYnUTqd8ZQeEQrygDM/SDNPqWelNpkcaDdCKhYR0M268RERERE1AFqdwOUbGcUJU3wazqekQ1tGffEWTVwQHtgI4YEqhAc2QhtfNyjt5FaIuu4y65ooIr3sO8CxDdLmuOnJUpu9C9BlPNBrOtComVXDIyIiIrImS1eeS1HnGCRMpxPVyMgtMOrXxM1BP7vUKcADYYEquDtyK5nqYhJF5rlzBTi0FjjxOZBfWKXF1Rfo+RTQbRLg1Miq4RERERFZ284zyUZ7IPlVYw+kjNyCwvLiakTH30FMvBopmhyjfs5KBTo2NSwv7qdy5LI8C2ASRaZJOCZtjntuKyAKl3Z6twd6zwI6PAzY8UJDIiIiop1nkjH9ixMovYguRZ2D6V+cwNpxXSpMpPK1Ovybkq7fwDYmIQ0XUzNQujyBXAa09nFD56Di4g8tm7jCTsFlebWBSRSVT6cF/t0hFYu4drC4vcVAqVhEi4HcHJeIiIiokFYn8Ma2WKMECgAEABmAN7bFYlCoLxRyGYQQSLiTbbAs70ySGjn5xrUImno4Fc4wSdcxdWjqDmclP8pbC888GcvLAmK+Ag6uAW5fltrk9kDHR4CImYBvB+vGR0RERGSDjsTdNljCV5oAkKzOwbwfTuF2Vh5i4tNwKzPPqJ+bo53+GqZOhYmTt5ujBSMnczGJomIZqcCR9cDRj4Hs21KbowroNhno8RTgbv4aXiIiIqKGIjW9/ASqpB9OJOi/t1fI0M7P3WBPphAvF8hrsAgF1TwmUQTc+FdashfzLaDNldo8goFeM4DO4wAHV+vGR0RERGTDdDqBuFuZOJ+cblL/u1p44X+hPggP9EA7P3c42nMvzbqGSZStyMsEFvlL37+SBChdLHs8IYAr+6X9nS7+VtzetKu0v1Pb4YCCLw8iIiKi0m6k5yKmxAa2MfFp0OQYlxcvTQbAV+WITVN61mi5c6p9/JTc0GjzgbNbgIOrgOSYwkYZ0HaYVCwiqBeLRRAREREVysorwJlEjb60eHR8GhLTso36OdjJ0bGpCione+w+n2p0f9GnqwXDQ5lA1QMNNomKiopCVFQUtFqttUOpHTka4MRnwKEPAU3hOlw7JyD8MalYhFcL68ZHREREZGVancDF1HR9afHoeDUuXE+HVmdYb08mA1o2cTXYj6mNrxvsC8uLl7VPlG819oki2yMTonTV+YZFo9FApVJBrVbD3d3deoFYajmfOqFwc9xNQK5GanNpAvR4Eug2BXDxqpnjEBEREdUhQggkq3P0pcVPxqfhTKIaWXnGf2D3cXcwSJg6NlXBzdG+wvG1OoEjcbeRmp4DbzdH9Ajx5AxUHWBqbtBgZ6LqvaRoqVjEmZ8AUfjLoHEbadYpbAxgzzKZRERE1HBocvJxOkGt35MpOj4NN9Jzjfq5KBUICyhOmMIDPeCrMv9zk0IuQ0QL/rG6vmISZSt0Jf7qcfWAtJGt3MxKLTodcGkXcGCVVDSiSLO+UrGIloMAOXexJiIiovotr0CHf1PSER1/B9HxakTH38HlG5lG/RRyGdr6ukkJU2F58RZNXDljRJViEmULYrcCO14svv3lw4C7PzD4HSB0ROWPz88BTn0LHIwCbv4rtckUQIcHpWIR/uEWCZuIiIjI2oQQuHY7Sz+7FBOfhjNJGuQV6Iz6BjRy0s8uhQd6oL2/Ck5Klhcn8zGJsrbYrcB3EyDtYV2CJllqH72p/EQq8xZw7BPgyDog84bUpnQDukYCPacBHoEWDZ2IiIiott3OzNOXFy8qMZ6WlW/UT+VkXzjDpEJ4kAfCAjzQ2NXBChFTfcQkypp0WmDnPBglUEBhmwzY+ZJUfrzk0r5bl6VZp+ivgILCEpvuAUCvaUCXCYCjqhaCJyIiIrKsnHwtziZp9DNM0fFpuHY7y6ifUiFHqL+7foapU6AHmnk5Q8ZtW8hCmERZ09UDgCapgg4C0CRK/Zr1Aa4dkopFnP8F+sTLrxMQ8TTQ/gFAUXGVGCIiIiJbpdMJ/HczAyf15cXTcD45HQU64z82N2/igvASxR/a+rnBwY7L8qj2MImypozrpvU7vx34YyGQeKy4rdV9QO9ZUtEI/pWFiIiI6phUTQ5OFs4wxSSk4VS8Gum5BUb9GrsqpdmlwsIPYU09oHLmH47JuphEWZOrj2n9Dn8o/atQAp3GSsUimrSxXFxERERENSgztwCnEtTSDFPhTFPJjWiLONkr0LGpCp0CVQgPbIROgSo09XDisjyyOUyirCm4t1SFT5OMsq+LKuTYCOjxhPTl6l1r4RERERGZq0Crw4XrGQbXMV1MTUfpVXlyGdDax00/w9QpwAOtfVxhp+B2LGT7mERZk1whlTH/bkL5fbpEAoOXAErn2ouLiIiIyARCCCSmZRskTKcT1cjJNy4v7q9yRKfA4uuYOjZVwcWBH0WpbuIr19pCR0hlzLc9C2TfLm538gTuXyEVjCAiIiKyAeqsfMQkFCdMMQlpuJmRZ9TPzcEOYYEqaZapMGnydne0QsRElsEkyhaEjgBC+gHvBEm3H/8BaDHQsKw5ERERUS3KLdDiXHJ6ccIUn4b/bmYa9bOTy9DOz11/HVN4oArNG7tCLud1TFR/MYmyFYoSP4rg3kygiIiIqNYIIXDlVhai4+8gJl6Nk/FpOJekQZ7WeFlesJezfoapU6AH2vu7w9Gen1uoYWESRURERFRHaXUCR+JuIzU9B95ujugR4gmFCTNANzNypdLi8Wk4GZ+GUwlqqLPzjfo1craXrmMqUfzB00VpiadCVKcwiSIiIiKqg3aeScYb22INSoX7qRyxYHgoBnfw07dl52lxJkmtT5hi4tOQcCfbaDylnRwd/N31hR/CAz0Q5OnM8uJEZWASRURERFTH7DyTjOlfnDDaICVFnYNpX5zA+F7BKNAJxMSn4d/r6dCWri8OoKW3q36GKTzAA2183aC0Y3lxIlMwiSIiIiKqQ7Q6gTe2xZa5w2RR2+eHrhq0N3Fz0M8uhQd6oGOACu6O9haPlai+YhJFREREVAek5+TjdIIaW2OSDJbwlWd4Jz8M7eCHToEe8FM5clkeUQ1iEkVERERkY/K1Ovybkm6wie2lGxkQZU0/leN/7XwwpKNf5R2JyGxMomyF0gVYqLZ2FERERFTLhBBIuJOtL/oQHZ+GM4lq5BYYlxdv6uGEwEZOOBR3u9Jxvd24uS2RpTTYJCoqKgpRUVHQarXWDoWIiIgakLSsvMIZJrW0L1OCGrcz84z6uTna6a9h6hQg7cnUxM0BWp1An3f+RIo6p8zromQAfFVSuXMisgyZEOZMDNc/Go0GKpUKarUa7u7u1g6HiIiI6pGcfC1ikzX6GaaY+DRcuZVl1M9eIUOon7t+A9tOgR4I8XKBvJw9n4qq8wEwSKSKeq8d18WgzDkRmcbU3KDBzkQRERER1SSdTuC/m5nSJrYJUtJ0LlmDfK3x36tDGrsUzjCp0CnQA6H+7nCwU5h8rMEd/LB2XBejfaJ8y9gniohqHpMoIiIioiq4kZ5rUPghJiEN6TkFRv28XJT6DWw7FSZOHs7Kah9/cAc/DAr1xZG420hNz4G3m7SET1HO7BUR1RwmUURERESVyMorwOkEtX6GKSZejcS0bKN+DnZydGyq0idM4YEeCGjkZLHy4gq5DBEtvCwyNhGVj0kUERERUQlancCF6+n6ZXknr6XhwvV06EqtypPJgFbergYJU2sfN9gr5NYJnIhqDZMoIiIiarCEEEhW5+iX5Z0sLC+elWdcvdfX3bFE4QcVwgI84OrAj1JEDRHf+URERNRgaHLycSperZ9hiklIw430XKN+rg52CCss+tApQJpl8lVx3yUikjCJIiIionopr0CH8ylF5cWlPZku38g06qeQy9DW100/y9Q50APNm7iyQAMRlYtJFBEREdkkrU6YXHlOCIFrt7MQXVgpLzo+DWeTNMgr0Bn1DfR0QnhgI3QKUKFzkAfa+6vgaG96eXEiIiZRREREZHN2nkk22gPJr8QeSLcz8/SlxYvKi6dl5RuN4+Fsj04BxTNMYQEqeLk61OZTIaJ6iEkUERER2ZSdZ5Ix/YsTKL1FbbI6B9O+OIHGrkrczMgzepzSTo72/u76a5jCAz0Q7OVssfLiRNRwMYkiIiIim5FfoMNrW84YJVAlFSVQLZq46EuLhwd6oK2vO5R2LC9ORJbHJIqIiIisJlWTg5OF5cWj49Nw8todZOcbX8dU2seR3fC/dj61ECERkTEmUURERFQtphaAyMwtwKkEqbx4dGF58ZLXPJkjM7egumETEVUZkygiIiKqsvIKQLw2rB2aNXZBTGFp8Zh4NS6mpkNXap2eXAa09ikuLy6XAfN+PF3pcb3duGcTEVkPkygiIiKqkooKQMz86mSZj/FXOSI8qHgD2w5NVXBxKP44otUJrPzjIlLUOWVeFyUD4KuSZruIiKyFSRQRERGZRZ2Vj5PX7uCF709VWABCBiCihSc6BzXSJ03e7hXPICnkMiwYHorpX5yADDAYv2iB4ILhodwIl4isikkUERERlSu3QItzyen6wg8x8Wn472amSY8VAJ4e2BoRLbzMOubgDn5YO66L0TJB3xL7RBERWROTKCIionrO1MIPQgjE3czUF36ITlDjXJIGeVrjannl7dVUWmp61QpHDO7gh0GhvibFTURU25hEERER1WPlFX5YMDwU3Zp5IqZwdulkfBpOJaihzs43GqORs71+P6ZOgR4ID/DA+ZR0PLr+UKXHr04BCIVcZvYsFhFRbWASRUREVE9VVPhh2hcnynyM0k6ODv7uCA9shE6BKoQHeiDI0xkymeEMUI8QT/ipHFkAgogaJCZRRERE9YxWJ/BvSjrm/Xi6wsIPANCiiYtU+KFwhqmNrxuUdvJKj8ECEETUkDGJIiIiquOS1dn6JXkx8Wk4naBGZp7WpMe+/UDHKi+ZYwEIImqomEQRERHVIek5+TidoNYnTDEJabiuyTXqp1TIkKetbB6q6oUfirAABBE1REyiiIiIrMCUinn5Wh3+TUlHdIny4pduZECUyo3kMqCNrzvCAz0QHqhCp0AP3MrIw+MfH640juoUfijCAhBE1NAwiSIiIqplZVXM81U5Ymb/llA52yP6mjTDdCZRjdwC4/LiTT2cEB4kXcPUKdADHZq6w1lp+F+6VidY+IGIyEKYRBEREdWi8irmpahz8PrPZ4z6uznaFc4weaBTYdLUxM2h0uOw8AMRkeUwiSIiIrKwnHwtYpM1OHH1Dt77/d8KK+bZK2QY2z0QnYMaITzQA828XCCvYqLDwg9ERJbBJIqIiKgG6XQC/93MREzRdUwJaTiXrEG+CUUeACBfKzC0o3+NXWPEwg9ERDWPSRQRETVophR4qEhqeg5i4tWIjr+DmHg1YhLSkJ5TYNTPy0UJX5UjziZpTBqzJrHwAxFRzWISRUREDVZZBR78KljqlplbgDOJav0MU0y8Golp2Ub9HO3l6NhUpb+GKTzQAwGNnHDov9t4dP2hSuOqiYp5RERkOfUiidq+fTuef/556HQ6zJs3D1OnTrV2SEREZOMqKvAw/YsTWP1YZzRv4qpflhcdn4YL19OhK/UAmQxo7e2GToWlxcMDPdDaxw32CrnRMXuEeLJiHhFRPSATovRuE3VLQUEBQkNDsWfPHri7u6NLly44fPgwPD1N+w9Io9FApVJBrVbD3d3dwtESEZEt0OoE+rzzp8EMVGmlK9oV8VM5olOAB8KDpGp5HQNUcHUw/W+SRckbUHbFvLXjurDgAxGRlZiaG9T5magjR46gffv2aNq0KQBg6NCh+O233/Doo49aOTIiIrJVe/5NrTCBAqQEx9Feji6FVfKKZpl83Ku31I4V84iI6j6rJ1F//fUX3n33XRw/fhzJycnYvHkzHnjgAYM+a9aswbvvvovk5GS0b98eK1euRN++fQEASUlJ+gQKAAICApCYmFibT4GIiGxYXoEO51M0+iV5MfFpuHwj06THLnkwDA90blp5RzOxYh4RUd1m9SQqMzMTnTp1wqRJk/DQQw8Z3f/tt99i9uzZWLNmDe666y589NFHGDJkCGJjYxEUFISyViPKZPxPiIioIRJC4OqtrOKEKSENZ5M0yCvQVWm86s46VYQV84iI6i6rJ1FDhgzBkCFDyr1/+fLlmDJlir5YxMqVK/Hbb79h7dq1WLx4MZo2bWow85SQkICePXuWO15ubi5yc3P1tzWaykvNEhGRbbqVkYuYhDREx6sRU5g0pWXlG/XzcLaXrmMqXJLXoakKI1b/zQIPRERUJVZPoiqSl5eH48eP46WXXjJov/fee3HgwAEAQI8ePXDmzBkkJibC3d0dv/76K+bPn1/umIsXL8Ybb7xh0biJiKhy5u7PlJOvLVFeXNqXKf62cXlxpZ0c7f3d9QlTeKAHgjydjVYpLBgeiulfnDAqICErcT+X1xERUVlsOom6efMmtFotfHx8DNp9fHyQkpICALCzs8OyZcswYMAA6HQ6vPjii/DyKn95xMsvv4w5c+bob2s0GgQGBlrmCRARUZkq259JqxO4fCNDfw1TdHwa/k1JR0Hp+uIAWjRxQXhgI4QXlhhv6+sOpZ1xefHSWOCBiIiqyqaTqCKl/3oohDBoGzFiBEaMGGHSWA4ODnBwcKjR+IiIyHTl7c+UrM7BtC9OoLWPK5LScpCRW2D02MauDggP9EDnwvLiYYEquDvaVzkWFnggIqKqsOkkqnHjxlAoFPpZpyKpqalGs1NERGT7tDqBBVvPlnkdUpEL1zMAAE72CnQMUKFzifLifirHGi8exAIPRERkLptOopRKJbp27Ypdu3Zh1KhR+vZdu3Zh5MiRVoyMiIhMka/V4d+UdKn4w7U0HPzvJq5rcit93DsPdcRDXQJgp6h8WR4REVFts3oSlZGRgUuXLulvx8XFITo6Gp6enggKCsKcOXMwfvx4dOvWDREREVi3bh2uXbuGadOmWTFqIiIqTQiBhDvZBtcxnUlSIyff/PLijvYKJlBERGSzrJ5EHTt2DAMGDNDfLir6EBkZiY0bN2LMmDG4desW3nzzTSQnJ6NDhw749ddfERwcXK3jRkVFISoqClqttlrjEBE1VOqsfEQnFCdMMfFpuJWZZ9TPzdFOX17c0V6O936/UOnY3m6W25+JiIioumSirN1qGxCNRgOVSgW1Wg13d3drh0NEZJNyC7SITdIUJ0wJasTdzDTqZ6+QoZ2fuz5p6hTogeaNXSAvLNSg1Qn0eefPSvdn+nveQBZ3ICKiWmdqbmD1mSgiIrItOp1A3K1Mgxmm2GQN8rXGaU8zL2d90YdOgR4I9XOHo72i3LEVchn3ZyIiojqPSRQRUT1myoa2N9JzEROfJhV/KEyaNDnG5cU9XZToFKBCeGAjdApUoVOABxq5KM2OifszERFRXcckioioniprQ1tfdwc83jMYjvYKRBfONCWmZRs91sFOjg5NVfoZpvAADwR6OtVYeXHuz0RERHUZr4niNVFEVA+Vt6FtWWQyoGUTV/2yvPBAD7TxdYM9q+MREVEDw2uiiIgaECEEktU5iIlPw4n4O/jsnysVJlAOdnI8fU9LdAlshI4BKrg52tdarERERHVdg02iWOKciOoyTU4+Tieo9UvyYuLTkJpe+Sa2RXILdOga5ImIFl4WjJKIiKh+4nI+LucjIhuXr9XhfHI6ohPSEH1NKgBx+UYGSv/2VshlaOPjBpWzPQ5evlXpuO+PDcfI8KYWipqIiKju4XI+IqI6SAiBa7ezDGaYziRpkFegM+ob0MgJnQI90Lmw+EMHfxWclAocvHzLpCSKG9oSERFVDZMoIiIrup2ZJ5UWL5xhiolPw52sfKN+7o52BglTp0APNHZ1KHPMHiGe8FM5VrqhbY8Qz5p9MkRERA0EkygiolqSk6/F2SRN8Sa2CWm4eivLqJ9SIUc7f/fChEnajymksYvJ5cW5oS0REZFlMYkiIjKBKZvWlqTTCfx3MwPR8WpEx99BTLwa55I1KNAZzw01b+yi34+pU6AH2vm5wcFOUa14uaEtERGR5TCJIiKqRFmb1vqVSkZSNTnF1zElpOFUvBrpuQVGY3m5KPV7MXUK9ECnAA+onC1TXpwb2hIREVlGg63OV7LE+YULF1idj4jKVNmmtZ0DPZCiyTFIsIo42svRsalKnzCFB3qgqYeTycvyiIiIqHaZWp2vwSZRRVjinIjKo9UJ3LXkT6RojBOk0mQyoLW3m0HC1NrHFXYKeS1ESkRERDWBJc6JiMwkhEBiWjZiCq9j2n/hpkkJ1OvD2mFMjyC4OvBXKhERUUPA//GJqMFSZ+fjVIny4tHxatzMyDV7nMZuDkygiIiIGhD+r09EDUJugRbnk9P1G9hGx6fhv5uZRv3s5DK083NHp0AVXJV2+PCv/yodm5vWEhERNSxMooio3hFC4MqtLH1p8ej4NMQmaZCn1Rn1DfJ0NriOqb2/OxztpfLiWp3AzzFJ3LSWiIiIDDCJIqI671ZGrrQc71oaohPUiIlPgzo736ifh7O9lDAFeCA8SPrX00VZ7rjctJaIiIjKwiSKiOqU7DwtziapDfZkir+dbdRPaSdHe393/Z5M4YEeCPJ0Nru8ODetJSIiotKYRBGRzdLqBC7fyCicYZJmmv69ng6tznhxXYsmLggPbITwQBXCAxuhja8blHY1U16cm9YSERFRSQ02iSq52S4R2YYUdQ6i4+8gOl5aknc6UY2M3AKjfk3cHAxmmDoGqODuaG/R2BRyGSJaeFn0GERERFQ3cLNdbrZLVG1anTB7liY9Jx+nE9UG1fKua4zLizsrFejYVKVPmDoFesBP5Wj2sjwiIiKiynCzXSKqFTvPJBtdL+RX6nqhfK0O/6YUlxePSUjDxdQMlP4TjlwGtPZxQ+eg4uIPLZu4wk5RM8vyiIiIiGoCkygiqrKdZ5Ix/YsTRuW/k9U5mPbFCdzT1hvq7HycSVIjJ9+4vHhTD6fC2SXpOqYOTd3hrOSvJSIiIrJt/LRCRFWi1Qm8sS22zP2Tiuw+n6r/3s3RTl9evFNh4sRNaomIiKguYhJFRCbLydfiXLIG0fFp+CP2usESvvJM79cCD3cLQIiXC+SsZkdERET1AJMoIiqTTicQdysT0deka5ii49NwLlmDfK15tWja+rmhRRNXC0VJRFQ5rVaL/HzjDbiJqOGxt7eHQqGo9jhMoogIAHAjPdegUl5MQhrSc4zLi3u6KBEe6AFPZyV+OJFQ6bhcskdE1iKEQEpKCtLS0qwdChHZEA8PD/j6+lar0i+TKKIGKCuvAKcT1IhJSENMvFRmPDEt26ifg50cHZuqCq9h8kDnQA8ENHKCTCaDVifwz+WbSFHnlHldlAyAr0oqd05EZA1FCZS3tzecnZ25NQJRAyeEQFZWFlJTpWu2/fz8qjwWkyiiek6rE7hwPV1fWvzktTRcuJ4OXanMRyYDWnm76gs/hAd6oI2vG+zLKS+ukMuwYHgopn9xAjLAIJEq+piyYHhopftFERFZglar1SdQXl7cKJuIJE5OTgCA1NRUeHt7V3lpX4NNoqKiohAVFQWtVmvtUIhqjBACyeoc/bK8k/FpOJOoRlae8evcx91Bv3lteKAHOjZVwc3R3qzjDe7gh7XjuhjtE+Vbap8oIqLaVnQNlLOzs5UjISJbU/R7IT8/v8pJlEyI0ttdNiym7kpMZIs0Ofk4Fa/WzzDFJKThRnquUT8XpQJhJWaYwgM94KuquWuVtDqBI3G3kZqeA283aQkfZ6CIyJpycnIQFxeHkJAQODpW/fddVl4BQuf/BgCIffM+7mVHVA9U9PvB1NyAvwmI6oi8Ah3Op2gKCz+oER1/B5dvZBr1U8hlaOvrZpAwtWjiatGkRiGXIaIFl8sQEZHlNGvWDLNnz8bs2bMBADKZDJs3b8YDDzxg1bioYSr7YgcisiohBK7czMTP0Yl4Y9tZjFrzDzos/A0jVv+D138+ix9PJOgTqEBPJ9wf5ofXhrXDD9MicGbhffjlmb5YNKojRncLRGsfN84KERFVkbbEBaRH4m4b3LaEiRMnQiaTQSaTwd7eHs2bN8cLL7yAzEzjP5qV5+TJk7j//vvh7e0NR0dHNGvWDGPGjMHNmzcN+v3444/o378/VCoVXF1dERYWhjfffBO3b98GAGzcuFEfS8mvjz/+2KznU1aSs3fvXshksgZdOXHhwoVlnt8//vjD2qGRCTgTRWQDbmfm6UuLF5UXT8sy3tNE5WRfYoZJhbAADzR2dbBCxERE9d/OM8lYsPWs/vbEDUfhVwvXfA4ePBgbNmxAfn4+9u/fj6lTpyIzMxNr166t9LGpqan43//+h+HDh+O3336Dh4cH4uLisHXrVmRlZen7vfrqq3jnnXfw3HPPYdGiRfD398fFixfx4Ycf4vPPP8ezzz4LAHB3d8e///5rcAyVSlWzT7gBa9++vVHS5OnJqrZ1AWeiiCqh1QkcvHwLP0cn4uDlW9X+K2ROvhbHr97GJ3/H4emvT+LupXvQ5a1dmLTxKN7ffRH7LtxAWlY+lAo5wgM9MLF3M6wcE449L/RH9PxB2DS5B+YMao2BbX2YQBERWcjOM8mY/sUJXNcYXmeaos7B9C9OYOeZZIsd28HBAb6+vggMDMRjjz2Gxx9/HFu2bIEQAi1btsR7771n0P/MmTOQy+W4fPkyDhw4AI1Gg48//hidO3dGSEgIBg4ciJUrVyIoKAgAcOTIESxatAjLli3Du+++i969e6NZs2YYNGgQfvzxR0RGRurHlslk8PX1Nfgqqm5W0w4cOIC7774bTk5OCAwMxDPPPGPyDNyVK1cgk8nw3XffoW/fvnByckL37t1x4cIFHD16FN26dYOrqysGDx6MGzdu6B939OhRDBo0CI0bN4ZKpUK/fv1w4sQJg7GLZt9GjRoFZ2dntGrVClu3btXfXzSrtnv3bnTr1g3Ozs7o3bu3UfJZFjs7O6Pzq1QqKz0fb775Jjp27Gg0XteuXTF//nwAxbOA7733Hvz8/ODl5YWZM2cabDzdrFkzvP3225gwYQJcXV0RHByMn3/+GTdu3MDIkSPh6uqKjh074tixY/rH3Lp1C48++igCAgLg7OyMjh074uuvvzaIo3///njmmWfw4osvwtPTE76+vli4cGGl56MuYRJFVIGdZ5LR550/8ej6Q3j2m2g8uv4Q+rzzp8n/eep0Ahevp+P7Y/F4dfNp3L9qPzos+A0PrT2It7bHYltMEq7dlv4y2LyJCx7s3BRvjmyPn2fehTNv3IctM+/CwhHt8UDnpghp7MI9ToiIaoFWJ/DGttgy98ArantjW6zFl/YVcXJyQn5+PmQyGSZPnowNGzYY3P/pp5+ib9++aNGiBXx9fVFQUIDNmzejvNphX375JVxdXTFjxowy7/fw8Kjpp1Cp06dP47777sODDz6IU6dO4dtvv8Xff/+NWbNmmTXOggUL8Nprr+HEiROws7PDo48+ihdffBHvv/8+9u/fj8uXL+uTDABIT09HZGQk9u/fj0OHDqFVq1YYOnQo0tPTDcZ94403MHr0aJw6dQpDhw7F448/rl/2WOTVV1/FsmXLcOzYMdjZ2WHy5MkWOx+TJ09GbGwsjh49qn/MqVOncPLkSUycOFHftmfPHly+fBl79uzBZ599ho0bN2Ljxo0Gx1qxYgXuuusunDx5EsOGDcP48eMxYcIEjBs3DidOnEDLli0xYcIE/espJycHXbt2xfbt23HmzBk8+eSTGD9+PA4fPmww7meffQYXFxccPnwYS5cuxZtvvoldu3ZV+ZzYHNHAqdVqAUCo1Wprh0I2ZsfpJNFs3nYRXOqrWeHXjtNJRo9JUWeLnWeSxZId58Sj6w6K9vN3Gj0+eN520fWt38WUjUfEqt0XxF8XUkVaVp4VniERUf2VnZ0tYmNjRXZ2ttmPPXDpZpm/u0t/Hbh0s8bjjoyMFCNHjtTfPnz4sPDy8hKjR48WQgiRlJQkFAqFOHz4sBBCiLy8PNGkSROxceNG/WNeeeUVYWdnJzw9PcXgwYPF0qVLRUpKiv7+IUOGiLCwsEpj2bBhgwAgXFxc9F8+Pj5mPx+FQmEwhouLi3B0dBQAxJ07d4QQQowfP148+eSTBo/dv3+/kMvl+p9hcHCwWLFihf5+AGLz5s1CCCHi4uIEAPHxxx/r7//6668FALF792592+LFi0WbNm3KjbegoEC4ubmJbdu2GRzntdde09/OyMgQMplM7NixQwghxJ49ewQA8ccff+j7/PLLLwJAha+/BQsWCLlcbnBeunfvbvL5GDJkiJg+fbr+/tmzZ4v+/fvrb0dGRorg4GBRUFCgb3vkkUfEmDFj9LeDg4PFuHHj9LeTk5MFAPH666/r2w4ePCgAiOTk5HKfy9ChQ8Xzzz+vv92vXz/Rp08fgz7du3cX8+bNK3eM2lTR7wdTcwNeE0VUhsr+CikDsHBrLNwc7HEqUa3fyLbkXklFnOwV6NhUhfAgj8KNbFVo6uHEWSUiIhuVmm78u7w6/cy1fft2uLq6oqCgAPn5+Rg5ciRWrVoFAPDz88OwYcPw6aefokePHti+fTtycnLwyCOP6B//f//3f5gzZw7+/PNPHDp0CB9++CEWLVqEv/76Cx07doQQwuT/g9zc3AyWt8nl5i9iGjBggNH1XIcPH8a4ceP0t48fP45Lly7hyy+/1LcJIaDT6RAXF4d27dqZdKywsDD99z4+PgBgsOzNx8cHqamp+tupqamYP38+/vzzT1y/fh1arRZZWVm4du1aueO6uLjAzc3NYJzSffz8/PTjA0BoaKj+vldeeQWvvPIKAKBNmzYGSwMdHBxMPh9PPPEEJk+ejOXLl0OhUODLL7/EsmXLDGJq3769wT5Ifn5+OH36tNnnrOi5+Pr6QqvVYsmSJfj222+RmJiI3Nxc5ObmwsXFpdxxi45d+pzVZUyiiMpwJO52mQlREQEgRZODxz8xnLqWy4DWPm76TWw7BXigtY8r7BRcOUtEVFd4u5m2r5Sp/cxVlHTY29vD398f9vaGG6FPnToV48ePx4oVK7BhwwaMGTPGaFNhLy8vPPLII3jkkUewePFidO7cGe+99x4+++wztG7dGn///Tfy8/ONxi5NLpejZcuW1Xo+Li4uRmMkJCQY3NbpdHjqqafwzDPPGD2+6FouU5R8PkWJYuk2nU6nvz1x4kTcuHEDK1euRHBwMBwcHBAREYG8vLxyxy1rnPKOrdPpEBAQgOjoaP19JQtHKJXKMs+vKedj+PDhcHBwwObNm+Hg4IDc3Fw89NBDNRJ3ec8FAJYtW4YVK1Zg5cqV6NixI1xcXDB79uwqnbO6jEkUUSlCCJxP1pjU18PJHhEtvPRJU8emKrg48G1FRFSX9QjxhJ/KESnqnDJXJMgA+KqkjcUtoayko6ShQ4fCxcUFa9euxY4dO/DXX39VOJ5SqUSLFi30RQkee+wxfPDBB1izZo2+Cl9JaWlptX5dVJcuXXD27NlqJ2zm2r9/P9asWYOhQ4cCAOLj441KwVeXnZ2d2c/LlPNhZ2eHyMhIbNiwAQ4ODhg7dqxRMm0J+/fvx8iRI/UziTqdDhcvXjR5trC+4Kc9avDUWfmISSgsLV64LO9mRl7lDwSwdlxXbjJLRFTPKOQyLBgeiulfnIAMMEikihbBLRgearU9+BQKBSZOnIiXX34ZLVu2REREhP6+7du345tvvsHYsWPRunVrCCGwbds2/Prrr/qCFD179sSLL76I559/HomJiRg1ahT8/f1x6dIlfPjhh+jTp0+ZyZUlzZs3D7169cLMmTPxxBNPwMXFBefOncOuXbv0SxktoWXLlvj888/RrVs3aDQazJ0712LVB81h6vmYOnWqPnn5559/aiW2li1b4scff8SBAwfQqFEjLF++HCkpKUyiGoqoqChERUVBq9VaOxSqRbkFWpxLTkf0tTuISZCuZfrvpnH5VIUMkMtkyC+n8pKl/wpJRETWNbiDH9aO64IFW88alDn3rYV9okwxZcoULFq0yKgCXGhoKJydnfH8888jPj4eDg4OaNWqFT7++GOMHz9e3++dd95B165dERUVhQ8//BA6nQ4tWrTAww8/bFDivCJXrlxBSEgI9uzZg/79+1fr+YSFhWHfvn149dVX0bdvXwgh0KJFC4wZM6Za41bm008/xZNPPonOnTsjKCgIixYtwgsvvGDRY5rC1PPRqlUr9O7dG7du3ULPnj1rJbbXX38dcXFxuO++++Ds7Iwnn3wSDzzwANRqda0c31bIhCin/mUDodFooFKpoFar4e7ubu1wqAYJIRB3M1OaZbqWhugENc4laZCnNV6PG+zlLC3JC/BAeJAHQv3csfffVEz/QrqYtqy/Qq4d18Xq/4kSEVHZcnJyEBcXh5CQEDg6Vv3apfScfHRc+DsAYOOk7ujbqonVZqBK+ueff9C/f38kJCToL/yvbXv37sWoUaPw33//oVGjRlaJoaETQqBt27Z46qmnMGfOHGuHU2dU9PvB1Nygwc5EUf1zMyMXMfHSsryipXmanAKjfo2c7YsLPwR6IDzAA41clEb9iv4K+ca2WIMiE7byV0giIrK8kglTjxBPqydQubm5iI+Px+uvv47Ro0dbLYECgJ07d+KVV15hAmUlqamp+Pzzz5GYmIhJkyZZO5wGh0kU1UnZeVqcSZKW450sTJgS7mQb9XOwk6NDU5W+tHjnwEYI9DS9vPjgDn4YFOqLI3G3kZqeA283R5v4T5SIiGqHs9IOV5YMs3YYel9//TWmTJmC8PBwfP7551aNZcmSJVY9fkPn4+ODxo0bY926dUxkrYBJFNk8rU7gUmqGQcL07/V0o53iZTKgRRNX/SxT50APtPF1g301y4sr5DIWjyAiIpswceJETJw40dphkA1o4FfkWB2TKLI5yepsg4TpdIIamXnGBUC83RwMEqYOASq4O1a83wURERERUXUxiSKrSs/Jx+kEtT5hiklIM6iCVMRZqUBYgEqfMHUK9ICvu6PJy/KIiIiIiGoKkyiqNflaHf5NSTco/HDpRgZKz0Yr5DK08XGTij4EqhAe2AgtvV15HRIRERER2QQmUWQRQgjE385GdGF58ZiENJxJVCO3wLi8eEAjJ32VvPAgD7T3d4ezki9NIiIiIrJN/KRKNeJOZp60H5N+WZ4atzPzjPq5O9oVzjBJX2EBHmji5mCFiImIiIiIqoZJFJktJ1+L2GSNfk+mmPg0XLmVZdRPqZCjnb87wgNUCA+SNrJt5uUCOZflERFRXZGXCSzyl75/JQlQulg3HiKyCUyiqEI6ncB/NzOLE6aENJxL1iBfa1xWs3ljF/0sU6dAD7Tzc4ODncIKURMRERFVz8SJE5GWloYtW7bU2Jj9+/dHeHg4Vq5cCQBo1qwZZs+ejdmzZwMAUlJSMH78eBw4cAD29vZIS0srs83W7N27FwMGDMCdO3fg4eFh9XFqA5MoMpCanoOYeDWi4+8gJl6NmIQ0pOcUGPXzclHqkyVpWZ4KHs5KK0RMRERkQboSW2xcPQC0GAjILfcHwokTJ+Kzzz4DANjZ2cHT0xNhYWF49NFHMXHiRMjlhnsfHjhwAG+//TYOHjyI7OxstGrVChMnTsTs2bOhUBjGuWfPHrz77rs4fPgwsrOz0axZMwwZMgRz5sxB06ZNLfacSrO1D8pXrlxBSEgITp48ifDwcH37+++/b/G9mI4ePQoXl+LZzRUrViA5ORnR0dFQqVTltllT6UQQAHr37o3k5GSz4qupcayFSZSN0OoEjsTdRmp6DrzdHNEjxNPi1egycwtwJlGtn2GKiVcjMS3bqJ+jvRwdm6rQqbDwQ6cADwQ0cmJ5cSIiqt9itwI7Xiy+/eXDgLs/MPgdIHSExQ47ePBgbNiwAVqtFtevX8fOnTvx7LPP4ocffsDWrVthZyd9fNu8eTNGjx6NSZMmYc+ePfDw8MAff/yBF198EYcOHcJ3332n/7/6o48+wowZMxAZGYkff/wRzZo1w7Vr17Bp0yYsW7YMy5cvt9jzqatq44N8kyZNDG5fvnwZXbt2RatWrSpsM1d+fj7s7S23l6ZSqYSvr6/NjFMrRAOnVqsFAKFWq60Ww47TSaLXoj9E8Lzt+q9ei/4QO04n1dgx8gu0IjZJLb4+fFXM+yFG3Ldinwh5abvBMYPnbRfNXtou7l2+T8z9Plp8eeiqOJOYJvIKtDUWBxERUW3Izs4WsbGxIjs7u2oDnP1ZiAUqIRa4l/pSSV9nf67BaItFRkaKkSNHGrXv3r1bABDr168XQgiRkZEhvLy8xIMPPmjUd+vWrQKA+Oabb4QQQsTHxwulUilmz55d5jHv3LljcnxHjhwR//vf/4SXl5dwd3cXd999tzh+/Lj+/ri4OAFAnDx50mB8AGLPnj36+0t+RUZGCiGEyMnJEU8//bRo0qSJcHBwEHfddZc4cuSIfpw9e/YIAGLnzp0iPDxcODo6igEDBojr16+LX3/9VbRt21a4ubmJsWPHiszMTP3jduzYIe666y6hUqmEp6enGDZsmLh06ZL+/tLx9OvXTwhh/LPQarViyZIlokWLFkKpVIrAwEDx9ttvl3uuMjIyxPjx44WLi4vw9fUV7733nujXr5949tln9X2Cg4PFihUr9N+XPi9ltQkhRFpamnjiiSdEkyZNhJubmxgwYICIjo7Wj7tgwQLRqVMn8cknn4iQkBAhk8mETqcz+XGbNm0SwcHBwt3dXYwZM0ZoNBr9OSl9vuLi4vQ/m6LX0s2bN8XYsWNF06ZNhZOTk+jQoYP46quv9McxdRwhhPjhhx9EaGioUCqVIjg4WLz33nsG5zk4OFj83//9n5g0aZJwdXUVgYGB4qOPPir35yJExb8fTM0NOBNlZTvPJGP6FydQerI4RZ2D6V+cwNpxXTC4g59ZYwohkKTO0V/HFB0vlRfPytMa9fVTORrMMHUMUMHVgS8LIiJqwHRaYOc8wOh/ZxS2yYCdLwFth1l0aV9JAwcORKdOnfDTTz9h6tSp+P3333Hr1i288MILRn2HDx+O1q1b4+uvv8aYMWPw/fffIy8vDy+++GIZI8OsJXXp6emIjIzEBx98AABYtmwZhg4diosXL8LNza3SxwcGBuLHH3/EQw89hH///Rfu7u5wcnICALz44ov48ccf8dlnnyE4OBhLly7Ffffdh0uXLsHT01M/xsKFC7F69Wo4Oztj9OjRGD16NBwcHPDVV18hIyMDo0aNwqpVqzBv3jwAQGZmJubMmYOOHTsiMzMT8+fPx6hRoxAdHQ25XI4jR46gR48e+OOPP9C+fXsolWVfnvDyyy9j/fr1WLFiBfr06YPk5GScP3++3Oc6d+5c7NmzB5s3b4avry9eeeUVHD9+3GDJYElHjx7FhAkT4O7ujvfffx9OTk7Iy8szahNCYNiwYfD09MSvv/4KlUqFjz76CPfccw8uXLigP1eXLl3Cd999hx9//FG/tNOUx12+fBlbtmzB9u3bcefOHYwePRpLlizB//3f/+H999/HhQsX0KFDB7z55psApNm0K1euGDyXnJwcdO3aFfPmzYO7uzt++eUXjB8/Hs2bN0fPnj1NHuf48eMYPXo0Fi5ciDFjxuDAgQOYMWMGvLy8MHHiRH2/ZcuW4a233sIrr7yCH374AdOnT8fdd9+Ntm3blvvzqa4G+2k5KioKUVFR0GqNE4vaotUJvLEttqJf0XhjWywGhfpWuLRPnZ2P0wnSdUzRhdcx3UjPNern6mCHsACVwbVMPu6ONfZ8iIiI6oWrBwBNUgUdBKBJlPqF9K21sNq2bYtTp04BAC5cuAAAaNeuXbl9i/pcvHgR7u7u8PMz74+yZRk4cKDB7Y8++giNGjXCvn37cP/991f6eIVCof+w7u3trU/gMjMzsXbtWmzcuBFDhgwBAKxfvx67du3CJ598grlz5+rHePvtt3HXXXcBAKZMmYKXX34Zly9fRvPmzQEADz/8MPbs2aNPoh566CGDGD755BN4e3sjNjYWHTp00C+p8/LyKncpWXp6Ot5//32sXr0akZGRAIAWLVqgT58+ZfbPyMjAJ598gk2bNmHQoEEAgM8++wwBAQHlnpsmTZrAwcEBTk5OBnGUbvvzzz9x+vRppKamwsFB2ibmvffew5YtW/DDDz/gySefBADk5eXh888/1z8/Ux+n0+mwceNGfVI8fvx47N69G//3f/8HlUoFpVIJZ2fnCpfdNW3a1CDBf/rpp7Fz5058//336Nmzp8njLF++HPfccw9ef/11AEDr1q0RGxuLd9991yCJGjp0KGbMmAEAmDdvHlasWIG9e/cyibKEmTNnYubMmdBoNFa7eO1I3G0kq3PKvV8ASFbn4EjcbUS08AIA5BXocD5FKi9+srC8+OUbmUaPtZPL0NbPTUqYAqSEqUUTV5YXJyIiqkzG9ZrtV0OEEEbXI4tyCh8IIfQzKmU9rqpSU1Mxf/58/Pnnn7h+/Tq0Wi2ysrJw7dq1ao17+fJl5Ofn65MjALC3t0ePHj1w7tw5g75hYWH67318fODs7KxPoIrajhw5YjD266+/jkOHDuHmzZvQ6XQAgGvXrqFDhw4mxXfu3Dnk5ubinnvuMfn55OXlISIiQt/m6emJNm3amPT4ihw/fhwZGRnw8vIyaM/Ozsbly5f1t4ODgw2uuTL1cc2aNTOYVfTz80NqaqpZMWq1WixZsgTffvstEhMTkZubi9zcXIMiGqY4d+4cRo4cadB21113YeXKldBqtfoZtpKvCZlMBl9fX7NjNleDTaJsQWp6+QlUSTvOJOO3symISUjD2SQN8gp0Rn2CPJ1LbGKrQnt/FRztWV6ciIjIbK4+Nduvhpw7dw4hISEAoC8ycO7cOfTu3duo7/nz5/XLxlq3bg21Wo3k5ORqz0ZNnDgRN27cwMqVKxEcHAwHBwdEREQgLy8PAPTVA0smd/n5+ZWOW9S/rCSxdFvJAgkymcyoYIJMJtMnSoC0vDEwMBDr16+Hv78/dDodOnTooI/ZFEVLDk1VXnJbE3Q6Hfz8/LB3716j+0ouzSydsJj6uMrOpymWLVuGFStWYOXKlejYsSNcXFwwe/Zss845YPofDmoiZnMxibIibzfTltJtOnjV4LaHs71+dqloaZ6nC8uLExER1Yjg3lIVPk0yyr4uSibdH2ycvFhK0VKs5557DgBw3333wdPTE8uWLTNKorZu3YqLFy/qS0c//PDDeOmll7B06VKsWLHCaOy0tDSTr4vav38/1qxZg6FDhwIA4uPjcfPmTf39RTMfycnJ6Ny5MwAgOjraYIyiGbKSl1S0bNkSSqUSf//9Nx577DEAUvJ17Ngx/R5KVXHr1i2cO3cOH330Efr2lZZe/v3335XGU1qrVq3g5OSE3bt3Y+rUqZUet2XLlrC3t8ehQ4cQFBQEALhz5w4uXLiAfv36VfXpAAC6dOmClJQU2NnZoVmzZhZ/XGlKpbLSy2H279+PkSNHYty4cQCkBO7ixYsGy09NGSc0NNTo53XgwAG0bt3aqIR/bWMSZUU9Qjzhp3JEijqnzF/RRcIDVegc1EifNAV5OrO8OBERkaXIFVIZ8+8mQLpCueT/0oX//w5eYrGiErm5uUhJSTEocb548WLcf//9mDBhAgBpluGjjz7C2LFj8eSTT2LWrFlwd3fH7t27MXfuXEydOlWf6AQGBmLFihWYNWsWNBoNJkyYgGbNmiEhIQGbNm2Cq6srli1bZlJsLVu2xOeff45u3bpBo9Fg7ty5BrM0Tk5O6NWrF5YsWYJmzZrh5s2beO211wzGCA4Ohkwmw/bt2zF06FA4OTnB1dUV06dPx9y5c+Hp6YmgoCAsXboUWVlZmDJlSpXPZaNGjeDl5YV169bBz88P165dw0svvWTQx9vbG05OTti5cycCAgLg6OhodKmHo6Mj5s2bhxdffBFKpRJ33XUXbty4gbNnz5YZn6urK6ZMmYK5c+fCy8sLPj4+ePXVV432+aqK//3vf4iIiMADDzyAd955B23atEFSUhJ+/fVXPPDAA+jWrVuNPq60Zs2a4fDhw7hy5QpcXV0Nin4UadmyJX788UccOHAAjRo1wvLly5GSkmKQRJkyzvPPP4/u3bvjrbfewpgxY3Dw4EGsXr0aa9asMfFsWU71f5JUZQq5DAuGh1bYZ9WjnbFlZh8sGN4eI8ObItjLhQkUERGRpYWOAEZvAtxKXfTu7i+1W3CfqJ07d8LPzw/NmjXD4MGDsWfPHnzwwQf4+eefDf76XlRA4dq1a+jbty9CQkIwdepUzJs3D+vXrzcYc8aMGfj999+RmJiIUaNGoW3btpg6dSrc3d0NCgDIZDJs3Lix3Ng+/fRT3LlzB507d8b48ePxzDPPwNvb26hPfn4+unXrhmeffRZvv/22wf1NmzbFG2+8gZdeegk+Pj6YNWsWAGDJkiV46KGHMH78eHTp0gWXLl3Cb7/9hkaNGlX1VEIul+Obb77B8ePH0aFDBzz33HN49913DfrY2dnhgw8+wEcffQR/f3+ja3CKvP7663j++ecxf/58tGvXDmPGjKnwupt3330Xd999N0aMGIH//e9/6NOnD7p27Vrl51JEJpPh119/xd13343JkyejdevWGDt2LK5cuQIfn/KXmFb1caW98MILUCgUCA0NRZMmTcq8Hu71119Hly5dcN9996F///7w9fXFAw88YPY4Xbp0wXfffYdvvvkGHTp0wPz58/Hmm28aFJWwFpmw5KLNOqCosIRarYa7u7tVYth5JhkLt8UipUSRCT+VIxYMDzW7vDkRERFJJZbj4uIQEhICR8dqVKLN0QBLAqXvH/8BaDGw1sqamysnJwcjR45EfHw89u3bZ7SRa2WuXLmCVq1aITY2tlobuxLZuop+P5iaG3A5nw0Y3MEPg0J9cSTuNlLTc+Dt5ogeIZ4VljUnIiKiWlAyYQrubbMJFCAtOfv555+xcuVK/PXXX0alvSuzc+dOPPnkk0ygiEzAmSgbmIkiIiKimlVjM1FEVO/UxEwUr4kiIiIiIiIyA5MoIiIiIiIiMzCJIiIiIiIiMgOTKCIiIqq3Gvil30RUhpr4vcAkioiIiOode3t7AEBWVpaVIyEiW1P0e6Ho90RVsMQ5ERER1TsKhQIeHh76zVCdnZ25WT1RAyeEQFZWFlJTU+Hh4WGwebW5mEQRERFRveTr6wsA+kSKiAgAPDw89L8fqopJFBEREdVLMpkMfn5+8Pb2Rn5+vrXDISIbYG9vX60ZqCJMooiIiKheUygUNfKhiYioCAtLEBERERERmYFJFBERERERkRmYRBEREREREZmhwV8TVbTZlkajsXIkRERERERkTUU5QWUb8jb4JCo9PR0AEBgYaOVIiIiIiIjIFqSnp0OlUpV7v0xUlmbVczqdDklJSXBzc7P6JnwajQaBgYGIj4+Hu7u7VWMhQ/zZVK4hnKO6+hxtOW5bic1acdTmcS19LEuNbyuvEVvGc2S7+LOpnK2dIyEE0tPT4e/vD7m8/CufGvxMlFwuR0BAgLXDMODu7m4TLyIyxp9N5RrCOaqrz9GW47aV2KwVR20e19LHstT4tvIasWU8R7aLP5vK2dI5qmgGqggLSxAREREREZmBSRQREREREZEZmETZEAcHByxYsAAODg7WDoVK4c+mcg3hHNXV52jLcdtKbNaKozaPa+ljWWp8W3mN2DKeI9vFn03l6uo5avCFJYiIiIiIiMzBmSgiIiIiIiIzMIkiIiIiIiIyA5MoIiIiIiIiMzCJIiIiIiIiMgOTKBuzePFiyGQyzJ4929qhEIDExESMGzcOXl5ecHZ2Rnh4OI4fP27tsGxKQUEBXnvtNYSEhMDJyQnNmzfHm2++CZ1OZ+3QquSvv/7C8OHD4e/vD5lMhi1bthjcL4TAwoUL4e/vDycnJ/Tv3x9nz561TrClVBR7fn4+5s2bh44dO8LFxQX+/v6YMGECkpKSrB5baU899RRkMhlWrlxplTjOnTuHESNGQKVSwc3NDb169cK1a9csetyMjAzMmjULAQEBcHJyQrt27bB27Vqzj7N48WJ0794dbm5u8Pb2xgMPPIB///3XoE91XsOVjV/V15kpcZdkydeIrVq7di3CwsL0G5JGRERgx44dAKz//iZDZX2Wq6n3eF1mzmequvAeZxJlQ44ePYp169YhLCzM2qEQgDt37uCuu+6Cvb09duzYgdjYWCxbtgweHh7WDs2mvPPOO/jwww+xevVqnDt3DkuXLsW7776LVatWWTu0KsnMzESnTp2wevXqMu9funQpli9fjtWrV+Po0aPw9fXFoEGDkJ6eXsuRGqso9qysLJw4cQKvv/46Tpw4gZ9++gkXLlzAiBEjrB5bSVu2bMHhw4fh7+9vlTguX76MPn36oG3btti7dy9iYmLw+uuvw9HR0aLHfe6557Bz50588cUXOHfuHJ577jk8/fTT+Pnnn806zr59+zBz5kwcOnQIu3btQkFBAe69915kZmbq+1TnNVzZ+FV9nZkSdxFLv0ZsVUBAAJYsWYJjx47h2LFjGDhwIEaOHImzZ89a/f1Nxcr7LFdT7/G6ypzPVHXmPS7IJqSnp4tWrVqJXbt2iX79+olnn33W2iE1ePPmzRN9+vSxdhg2b9iwYWLy5MkGbQ8++KAYN26clSKqOQDE5s2b9bd1Op3w9fUVS5Ys0bfl5OQIlUolPvzwQytEWL7SsZflyJEjAoC4evVq7QRVqLzYEhISRNOmTcWZM2dEcHCwWLFiRa3HMWbMGIu/dss6bvv27cWbb75p0NalSxfx2muvVetYqampAoDYt2+fEKLmX8Olxy9LVV5n5Y1b268RW9eoUSPx8ccfl3mftd7fDVlFn+Us9R6vK0z9TFWX3uOcibIRM2fOxLBhw/C///3P2qFQoa1bt6Jbt2545JFH4O3tjc6dO2P9+vXWDsvm9OnTB7t378aFCxcAADExMfj7778xdOhQK0dW8+Li4pCSkoJ7771X3+bg4IB+/frhwIEDVoysatRqNWQymU3Mrup0OowfPx5z585F+/btrRbDL7/8gtatW+O+++6Dt7c3evbsWeHSw5rSp08fbN26FYmJiRBCYM+ePbhw4QLuu+++ao2rVqsBAJ6engBq/jVcevzy+pj7OitrXFt4jdgKrVaLb775BpmZmYiIiCizjy29vxuKij7LWeo9XleY8pmqrr3HmUTZgG+++QYnTpzA4sWLrR0KlfDff/9h7dq1aNWqFX777TdMmzYNzzzzDDZt2mTt0GzKvHnz8Oijj6Jt27awt7dH586dMXv2bDz66KPWDq3GpaSkAAB8fHwM2n18fPT31RU5OTl46aWX8Nhjj8Hd3d3a4eCdd96BnZ0dnnnmGavFkJqaioyMDCxZsgSDBw/G77//jlGjRuHBBx/Evn37LHrsDz74AKGhoQgICIBSqcTgwYOxZs0a9OnTp8pjCiEwZ84c9OnTBx06dABQs6/hssYvrSqvs/LGtYXXiLWdPn0arq6ucHBwwLRp07B582aEhoYa9bO193dDUNlnOUu8x+sSUz5T1bX3uJ21A2jo4uPj8eyzz+L333+v9pp7qlk6nQ7dunXDokWLAACdO3fG2bNnsXbtWkyYMMHK0dmOb7/9Fl988QW++uortG/fHtHR0Zg9ezb8/f0RGRlp7fAsQiaTGdwWQhi12bL8/HyMHTsWOp0Oa9assXY4OH78ON5//32cOHHCquexqBjKyJEj8dxzzwEAwsPDceDAAXz44Yfo16+fxY79wQcf4NChQ9i6dSuCg4Px119/YcaMGfDz86vyCoVZs2bh1KlT+Pvvv43uq4nXcEXjA1V/nZU1rq28RqytTZs2iI6ORlpaGn788UdERkZi3759BomUrb2/GwJTPstZ4j1el1T2mapOvsetuJSQhBCbN28WAIRCodB/ARAymUwoFApRUFBg7RAbrKCgIDFlyhSDtjVr1gh/f38rRWSbAgICxOrVqw3a3nrrLdGmTRsrRVRzUOralcuXLwsA4sSJEwb9RowYISZMmFDL0VWsdOxF8vLyxAMPPCDCwsLEzZs3az8wYRzbihUr9L/zSv4elMvlIjg4uNbiyM3NFXZ2duKtt94y6Pfiiy+K3r17W+y4WVlZwt7eXmzfvt2g35QpU8R9991XpWPMmjVLBAQEiP/++8+gvaZew+WNX6Sqr7PyxrXWa8TW3XPPPeLJJ5/U37aF93dDVNlnuYyMjBp/j9c1lX2mqovvcc5EWdk999yD06dPG7RNmjQJbdu2xbx586BQKKwUGd11111GJXYvXLiA4OBgK0Vkm7KysiCXG64MVigUdbbEeUVCQkLg6+uLXbt2oXPnzgCAvLw87Nu3D++8846Vo6tcfn4+Ro8ejYsXL2LPnj3w8vKydkgAgPHjxxv9Jfa+++7D+PHjMWnSpFqLQ6lUonv37rX+vs/Pz0d+fn6NvI+EEHj66aexefNm7N27FyEhIQb3V/c1XNn4Rc/H3NdZZePaymvE1gghkJubC8B2398NQWWf5bRabY29x+uqyj5T1cX3OJMoK3NzczNaS+7i4gIvL69y15hT7XjuuefQu3dvLFq0CKNHj8aRI0ewbt06rFu3ztqh2ZThw4fj//7v/xAUFIT27dvj5MmTWL58OSZPnmzt0KokIyMDly5d0t+Oi4tDdHQ0PD09ERQUhNmzZ2PRokVo1aoVWrVqhUWLFsHZ2RmPPfaYFaOWVBS7v78/Hn74YZw4cQLbt2+HVqvVXwPj6ekJpVJptdiCgoKMPvDZ29vD19cXbdq0qdU45s6dizFjxuDuu+/GgAEDsHPnTmzbtg179+616HH79euHuXPnwsnJCcHBwdi3bx82bdqE5cuXm3WcmTNn4quvvsLPP/8MNzc3/c9YpVLByclJv3dNVV/DlY1fUFBQpddZZeN6eXnV2mvEVr3yyisYMmQIAgMDkZ6ejm+++QZ79+7Fzp07q3zeqWaY8lmupt7jdVVln6nq5HvcuhNhVBaWOLcd27ZtEx06dBAODg6ibdu2Yt26ddYOyeZoNBrx7LPPiqCgIOHo6CiaN28uXn31VZGbm2vt0Kpkz549AoDRV2RkpBBCKhG9YMEC4evrKxwcHMTdd98tTp8+bd2gC1UUe1xcXJn3ARB79uyxamxlsVRpW1Pi+OSTT0TLli2Fo6Oj6NSpk9iyZYvFj5ucnCwmTpwo/P39haOjo2jTpo1YtmyZ0Ol0Zh2nvJ/xhg0b9H2q8xqubPyqvs5Mibs0Wy9/XNMmT54sgoODhVKpFE2aNBH33HOP+P3334UQVT/vZDmlP8vV1Hu8LjP3M5Wtv8dlQghRI9kYERERERFRA8AS50RERERERGZgEkVERERERGQGJlFERERERERmYBJFRERERERkBiZRREREREREZmASRUREREREZAYmUURERERERGZgEkVERERERGQGJlFERERERERmYBJFRERERERkBjtrB0BERGQt/fv3R1hYGBwdHfHxxx9DqVRi2rRpWLhwobVDIyIiG8aZKCIiatA+++wzuLi44PDhw1i6dCnefPNN7Nq1y9phERGRDZMJIYS1gyAiIrKG/v37Q6vVYv/+/fq2Hj16YODAgViyZIkVIyMiIlvGmSgiImrQwsLCDG77+fkhNTXVStEQEVFdwCSKiIgaNHt7e4PbMpkMOp3OStEQEVFdwCSKiIiIiIjIDEyiiIiIiIiIzMAkioiIiIiIyAyszkdERERERGQGzkQRERERERGZgUkUERERERGRGZhEERERERERmYFJFBERERERkRmYRBEREREREZmBSRQREREREZEZmEQRERERERGZgUkUERERERGRGZhEERERERERmYFJFBERERERkRmYRBEREREREZmBSRQREREREZEZ/h8OO/HWDT1RggAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "fig.set_figwidth(10)\n",
    "\n",
    "adjoint_x_alkanes = [l for l in alkanes_performance['pyscf_adjoint_pbe'].keys() if alkanes_performance['pyscf_adjoint_pbe'][l]]\n",
    "adjoint_y_alkanes = [l[\"mean\"] for l in alkanes_performance['pyscf_adjoint_pbe'].values() if l]\n",
    "adjoint_yerr_alkanes = [l[\"std_dev\"] for l in alkanes_performance['pyscf_adjoint_pbe'].values() if l]\n",
    "adjoint_coeff_alkanes = numpy.polyfit(numpy.log(adjoint_x_alkanes), numpy.log(adjoint_y_alkanes), deg=1)\n",
    "adjoint_poly_alkanes = numpy.poly1d(adjoint_coeff_alkanes)\n",
    "adjoint_yfit_alkanes = [numpy.exp(adjoint_poly_alkanes(numpy.log(x))) for x in adjoint_x_alkanes]\n",
    "ax.errorbar(adjoint_x_alkanes, adjoint_y_alkanes, yerr=adjoint_yerr_alkanes, \n",
    "            linestyle='None', marker='o', color='tab:blue', label='PySCF, Hellmann-Feynman')\n",
    "ax.plot(adjoint_x_alkanes, adjoint_yfit_alkanes, linestyle='solid', color='tab:blue')\n",
    "\n",
    "dqc_x_alkanes = [l for l in alkanes_performance['dqc_onlygrad_pbe'].keys() if alkanes_performance['dqc_onlygrad_pbe'][l]]\n",
    "dqc_y_alkanes = [l[\"mean\"] for l in alkanes_performance['dqc_onlygrad_pbe'].values() if l]\n",
    "dqc_yerr_alkanes = [l[\"std_dev\"] for l in alkanes_performance['dqc_onlygrad_pbe'].values() if l]\n",
    "dqc_coeff_alkanes = numpy.polyfit(numpy.log(dqc_x_alkanes), numpy.log(dqc_y_alkanes), deg=1)\n",
    "dqc_poly_alkanes = numpy.poly1d(dqc_coeff_alkanes)\n",
    "dqc_yfit_alkanes = [numpy.exp(dqc_poly_alkanes(numpy.log(x))) for x in dqc_x_alkanes]\n",
    "ax.errorbar(dqc_x_alkanes, dqc_y_alkanes, yerr=dqc_yerr_alkanes, \n",
    "            linestyle='None', marker='o', color='tab:orange', label='DQC, automatic differentiation')\n",
    "ax.plot(dqc_x_alkanes, dqc_yfit_alkanes, linestyle='solid', color='tab:orange')\n",
    "\n",
    "plt.xscale('log')\n",
    "plt.yscale('log')\n",
    "\n",
    "plt.title(r\"Time of derivative calculation of C$_n$H$_{2n+2}$, PBE\")\n",
    "plt.xlabel(\"n\")\n",
    "plt.ylabel(\"Time, s\")\n",
    "\n",
    "ax.set_xticks(adjoint_x_alkanes)\n",
    "ax.get_xaxis().set_major_formatter(ticker.ScalarFormatter())\n",
    "ax.get_xaxis().set_minor_locator(ticker.NullLocator())\n",
    "\n",
    "plt.legend(loc='lower right')\n",
    "\n",
    "plt.savefig('figure_pbe_3.png')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Density derivatives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Hellmann-Feynman theorem holds only for variationally optimized energy. For other quantities, such as the electron density distribution, other methods must be used. Below we compare the accuracy of the automatic differentiation method and implicit differentiation one for the LDA and PBE functionals for the water molecule."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "mol_name_ = \"water\"\n",
    "aj_mol = molecules_for_correctness_tests[mol_name_]\n",
    "analytic_partial_lda = AnalyticalPartialDerivativeCalculator(PySCFDataAdapter(PySCFLDAXConstructor, LDAX_default_parameters_dict, aj_mol))\n",
    "analytic_partial_lda_dqc = AnalyticalPartialDerivativeCalculator(DQCDataAdapter(DQCLDAXConstructor, LDAX_default_parameters_dict, aj_mol))\n",
    "rho_exact = PySCFAdapterCCSDT(aj_mol).construct_rho_on_grid(analytic_partial_lda.adapter.grid_coords)\n",
    "rho_exact_dqc = PySCFAdapterCCSDT(aj_mol).construct_rho_on_grid(analytic_partial_lda_dqc.adapter.grid_coords)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.00688876, -0.02054143])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iterative_total_lda = IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(DQCLDAXConstructor, \n",
    "                                                                                LDAX_default_parameters_dict, \n",
    "                                                                                aj_mol))\n",
    "\n",
    "iterative_total_lda.construct_total_drho_squared_wrt_dtheta(rho_exact_dqc, retain_graph=True)\n",
    "iterative_total_lda.total_drho_squared_wrt_dtheta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "started the four term integral computation for 34310...\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([ 0.00688652, -0.02053853])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "analytic_total_adjoint_lda = PartialToTotalDerivativeAdjoint(analytic_partial_lda)\n",
    "analytic_total_adjoint_lda.construct_total_drho_squared_wrt_dtheta(rho_exact)\n",
    "analytic_total_adjoint_lda.total_drho_squared_wrt_dtheta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For water:\n",
      "drho/dtheta = [ 0.00688512 -0.02052864]\n"
     ]
    }
   ],
   "source": [
    "def density_getter(a, rho_exact):\n",
    "    a.construct_rho_squared(rho_exact)\n",
    "    return a.rho_squared\n",
    "\n",
    "drho_fd, _, _ = FiniteDifferenceTotalDerivativeCalculator(PySCFAdapterAuto,\n",
    "                                                          PySCFLDAXConstructor,\n",
    "                                                          LDAX_default_parameters_dict,\n",
    "                                                          lambda a: density_getter(a, rho_exact),\n",
    "                                                          aj_mol)\n",
    "drho_fd = drho_fd\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"drho/dtheta = {drho_fd}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-7.69513197e-03,  9.24275044e-03, -1.21229734e-03, -2.76373012e-03,\n",
       "       -9.29343644e-05,  9.19616614e-04,  5.69091737e-03, -2.72468659e-04,\n",
       "       -1.70749409e-05,  1.55159955e-06,  1.89934622e-05,  3.70781820e-05])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iterative_total_pbe = IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(DQCPBEConstructor, \n",
    "                                                                                PBE_default_parameters_dict, \n",
    "                                                                                aj_mol))\n",
    "\n",
    "iterative_total_pbe.construct_total_drho_squared_wrt_dtheta(rho_exact_dqc, retain_graph=True)\n",
    "iterative_total_pbe.total_drho_squared_wrt_dtheta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-7.69741939e-03,  9.24589216e-03, -1.21281155e-03, -2.76507367e-03,\n",
       "       -9.30082072e-05,  9.19787589e-04,  5.69295304e-03, -2.72516071e-04,\n",
       "       -1.70816331e-05,  1.55043435e-06,  1.89969747e-05,  3.70859055e-05])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mixed_partial_pbe = MixedPartialDerivativeCalculator(PySCFDataAdapter(PySCFPBEConstructor, PBE_default_parameters_dict, aj_mol))\n",
    "mixed_total_adjoint_pbe = PartialToTotalDerivativeAdjoint(mixed_partial_pbe)\n",
    "mixed_total_adjoint_pbe.construct_total_drho_squared_wrt_dtheta(rho_exact)\n",
    "mixed_total_adjoint_pbe.total_drho_squared_wrt_dtheta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For water:\n",
      "drho/dtheta = [-7.69882077e-03  9.25611484e-03 -1.21288306e-03 -2.76512278e-03\n",
      " -9.30228920e-05  9.19819685e-04  5.69295189e-03 -2.72498452e-04\n",
      " -1.70816189e-05  1.54991538e-06  1.89958262e-05  3.70835777e-05]\n"
     ]
    }
   ],
   "source": [
    "drho_fd, _, _ = FiniteDifferenceTotalDerivativeCalculator(PySCFAdapterAuto,\n",
    "                                                          PySCFPBEConstructor,\n",
    "                                                          PBE_default_parameters_dict,\n",
    "                                                          lambda a: density_getter(a, rho_exact),\n",
    "                                                          aj_mol)\n",
    "drho_fd = drho_fd\n",
    "print(f\"For {mol_name_}:\")\n",
    "print(f\"drho/dtheta = {drho_fd}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dependence of derivatives on the basis set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For molecule h2:\n",
      "For cc-pVDZ:\n",
      "dE/dtheta, Hellmann-Feynman = [0.74829134 1.47555614]\n",
      "started the four term integral computation for 20048...\n",
      "Δ(dE/dtheta), analytic = [7.31636973e-14 4.15889545e-13]\n",
      "Δ(dE/dtheta), FD = [ 5.84862286e-07 -3.12686778e-05]\n",
      "Δ(dE/dtheta), AD = [-1.23801119e-06 -1.21011720e-06]\n",
      "For cc-pVTZ:\n",
      "dE/dtheta, Hellmann-Feynman = [0.74660888 1.4610596 ]\n",
      "started the four term integral computation for 20048...\n",
      "Δ(dE/dtheta), analytic = [8.27116153e-14 4.76951811e-13]\n",
      "Δ(dE/dtheta), FD = [ 6.40289452e-07 -3.14948752e-05]\n",
      "Δ(dE/dtheta), AD = [-1.28817092e-06 -1.22650078e-06]\n",
      "For cc-pVQZ:\n",
      "dE/dtheta, Hellmann-Feynman = [0.74648306 1.46114144]\n",
      "started the four term integral computation for 20048...\n",
      "Δ(dE/dtheta), analytic = [8.38218384e-14 5.08260101e-13]\n",
      "Δ(dE/dtheta), FD = [ 6.39311672e-07 -3.15497458e-05]\n",
      "Δ(dE/dtheta), AD = [-1.20529197e-06 -1.18085394e-06]\n",
      "For cc-pV5Z:\n",
      "dE/dtheta, Hellmann-Feynman = [0.74641917 1.46091701]\n",
      "started the four term integral computation for 20048...\n",
      "Δ(dE/dtheta), analytic = [8.83737528e-14 5.47784040e-13]\n",
      "Δ(dE/dtheta), FD = [ 6.41629988e-07 -3.15690520e-05]\n",
      "Δ(dE/dtheta), AD = [-1.25346669e-06 -1.20321057e-06]\n",
      "For molecule water:\n",
      "For cc-pVDZ:\n",
      "dE/dtheta, Hellmann-Feynman = [ 10.96441393 -11.08504652]\n",
      "started the four term integral computation for 34310...\n",
      "Δ(dE/dtheta), analytic = [9.94759830e-14 3.94351218e-13]\n",
      "Δ(dE/dtheta), FD = [ 9.78940210e-06 -4.89931338e-04]\n",
      "Δ(dE/dtheta), AD = [-1.01503775e-05  1.55920936e-05]\n",
      "For cc-pVTZ:\n",
      "dE/dtheta, Hellmann-Feynman = [ 10.91731969 -11.01619714]\n",
      "started the four term integral computation for 34310...\n",
      "Δ(dE/dtheta), analytic = [1.29674049e-13 3.92574862e-13]\n",
      "Δ(dE/dtheta), FD = [ 8.12506554e-06 -5.09922697e-04]\n",
      "Δ(dE/dtheta), AD = [-9.22832205e-06  1.68023000e-05]\n",
      "For cc-pVQZ:\n",
      "dE/dtheta, Hellmann-Feynman = [ 10.90462975 -11.02383787]\n",
      "started the four term integral computation for 34310...\n",
      "Δ(dE/dtheta), analytic = [1.10134124e-13 4.54747351e-13]\n",
      "Δ(dE/dtheta), FD = [ 3.58174770e-06 -5.08149048e-04]\n",
      "Δ(dE/dtheta), AD = [-1.64694257e-05  1.97517245e-05]\n"
     ]
    }
   ],
   "source": [
    "basis_names = [\"cc-pVDZ\", \"cc-pVTZ\", \"cc-pVQZ\", \"cc-pV5Z\"]\n",
    "\n",
    "for mol_name in [\"h2\", \"water\"]:\n",
    "    print(f\"For molecule {mol_name}:\")\n",
    "    aj_mol = molecules_for_correctness_tests[mol_name]\n",
    "    for basis_name in basis_names:\n",
    "        if (mol_name == \"water\" and basis_name == \"cc-pV5Z\"):\n",
    "            continue\n",
    "        print(f\"For {basis_name}:\")    \n",
    "        analytic_partial = AnalyticalPartialDerivativeCalculator(PySCFDataAdapter(PySCFLDAXConstructor, LDAX_default_parameters_dict, aj_mol, basis=basis_name))\n",
    "        \n",
    "        hellmann_feynman = HellmannFeynmanTotalDerivative(analytic_partial)\n",
    "        hellmann_feynman.construct_total_dE_wrt_dtheta()\n",
    "        print(f\"dE/dtheta, Hellmann-Feynman = {hellmann_feynman.total_dE_wrt_dtheta}\")\n",
    "\n",
    "        analytic_total_adjoint = PartialToTotalDerivativeAdjoint(analytic_partial)\n",
    "        analytic_total_adjoint.construct_total_dE_wrt_dtheta()\n",
    "        print(f\"Δ(dE/dtheta), analytic = {analytic_total_adjoint.total_dE_wrt_dtheta - hellmann_feynman.total_dE_wrt_dtheta}\")\n",
    "        \n",
    "        der, _, _ = FiniteDifferenceTotalDerivativeCalculator(PySCFAdapterCustom,\n",
    "                                                   PySCFLDAXConstructor,\n",
    "                                                   LDAX_default_parameters_dict,\n",
    "                                                   lambda a: a.total_energy,\n",
    "                                                   aj_mol,\n",
    "                                                   basis=basis_name)\n",
    "        print(f\"Δ(dE/dtheta), FD = {der - hellmann_feynman.total_dE_wrt_dtheta}\")\n",
    "        \n",
    "        iterative_total = IterativeAutogradTotalDerivativeCalculator(DQCAdapterWithgrad(DQCLDAXConstructor,\n",
    "                                                   LDAX_default_parameters_dict, aj_mol, basis=basis_name))\n",
    "        iterative_total.construct_total_dE_wrt_dtheta()\n",
    "        print(f\"Δ(dE/dtheta), AD = {iterative_total.total_dE_wrt_dtheta - hellmann_feynman.total_dE_wrt_dtheta}\")"
   ]
  }
 ],
 "metadata": {
  "datalore": {
   "base_environment": "default",
   "computation_mode": "JUPYTER",
   "package_manager": "pip",
   "packages": [
    {
     "name": "pyscf",
     "source": "PIP",
     "version": "2.1.1"
    },
    {
     "name": "pydqc",
     "source": "PIP",
     "version": "0.1.0"
    }
   ],
   "version": 1
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}