excel: Rework code to import side-python file.

* Prepare for windows installer.
* cmdline: Add --excelrun.
* Rename wltp_runner --> WltpExcelRunner
* Replace (some) Flashbang with just `python`.
* Add WinPython_requirments.txt.

MANIFEST.in

@@ -1,8 +1,8 @@
 include LICENSE.txt
 include CHANGES.rst
 
-include wltp/excel/wltp_runner.xlsm
-include wltp/excel/wltp_runner.py
+include wltp/excel/wltp_excel_runner.xlsm
+include wltp/excel/wltp_excel_runner.py
 
 recursive-include docs *.docx
 

README.rst

@@ -64,7 +64,9 @@ For a quick-start, and assuming a working python-environment, open a normal "con
 #. Install: ``pip install wltp --pre -U``
 #. Check version: ``wltpcmd.py --version``  
 #. Run with the TkUI and explore model: ``wltpcmd.py --gui``
-#. (*Windows* or *OS X*) Create Excel file in your current-working folder: ``wltpcmd.py --excel -v``
+#. (*Windows* or *OS X*) Create a sample Excel file in your current-working folder: ``wltpcmd.py --excel -v``
+   Open the excel, (enable macros and) select the python-code at the left and click 
+   the :menuselection:`Run Selection as Pyhon` button; one sheet per vehicle will be created.
 
 
 
@@ -169,6 +171,14 @@ on your system (or virtualenv), enter:
     $ pip install -r requirements.txt
 
 
+Particularly for the latest *WinPython* environments you can install dependencies with: 
+
+.. code-block:: console
+
+    $ pip install -r WinPython_requirements.txt
+
+
+
 Project files and folders
 -------------------------
 The files and folders of the project are listed below::
@@ -250,47 +260,53 @@ Excel usage
 In *Windows* and *OS X* you may utilize the excellent `xlwings <http://xlwings.org/quickstart/>`_ library 
 to provide input and output to wltp from Excel files.
 
-To create the necessary sample files in your current-directory by entering:
+To create the necessary template-files in your current-directory you should enter:
 
 .. code-block:: console
 
      $ wltpcmd.py --excel
      
-The above command creates two files:
-
-:file:`wltp_runner.xlsm`
-    The python-enabled excel-file (macro-enabled and with *VBA* code) where input and output data are written.
-    It invokes python-functions declared in the python-script, below.
-
-:file:`wltp_runner.py`   
-    Utility python functions used by the above xls-file.  
-    The functions included provide for running a batch of experiments, but you can edit it to 
-    fit your needs.
-
-You may type instead :samp:`wltpcmd.py --excel {xls_file_path}` to specify a  different filename/path.
 
-The xls-file contains the appropriate *VBA* modules and macros allowing you to invoke *Python code* 
-present in *selected cells* with a click of a button, as seen in the screenshot below.
+You could type instead :samp:`wltpcmd.py --excel {xls_file_path}` to specify a  different filename/path.
 
-.. image:: docs/xlwings_screenshot.png
-    :scale: 50%
-    :alt: Screenshot of the `wltp_runner.xlsm` file.
+The above command creates two files:
 
+:file:`wltp_excel_runner.xlsm`
+    The python-enabled excel-file where input and output data are written, as seen in the screenshot below:
+
+    .. image:: docs/xlwings_screenshot.png
+        :scale: 50%
+        :alt: Screenshot of the `wltp_excel_runner.xlsm` file.
+
+    The excel-file contains additionally appropriate *VBA* modules allowing you to invoke *Python code* 
+    present in *selected cells* with a click of a button, and python-functions declared in the python-script, below,
+    using the `mypy` namespace. 
+
+    To add more input-columns, you need to set as column *Headers* the *json-pointers* path of the desired 
+    model item (see `Python usage`_ below,).
 
-The particular code already loaded in the demo-workbook reads multiple vehicles from tables with various 
-vehicle characteristics and experiment data and generates new worksheets with the cycle-run tables 
-for each vehicle.  
+:file:`wltp_excel_runner.py`   
+    Utility python functions used by the above xls-file for running a batch of experiments.  
+
+    The particular functions included reads multiple vehicles from the input table with various  
+    vehicle characteristics and/or experiment parameters, and then it adds a new worksheet containing 
+    the cycle-run of each vehicle . 
+    Of course you can edit it to further fit your needs.
 
-If you need to add/remove fields (see `Python usage`_ below) 
-it should be pretty obvious that you need to change the *json-pointers* in the header of the table.
 
+.. Note:: You may reverse the procedure described above and run the python-script instead.
+    The script will open the excel-file, run the experiments and add the new sheets, but in case any errors occur, 
+    this time you can debug them, if you had executed the script through *LiClipse*, or *IPython*! 
 
 Some general notes regarding the python-code in excel-cells:
 
 * The *VBA* `xlwings` module contains the code from the respective library; do not edit, but you may replace it 
   with a latest version. 
 * You can read & modify the *VBA* `xlwings_ext` module with code that will run on each invocation 
   to import libraries such as 'numpy' and 'pandas', or pre-define utility python functions.
+* The name of the python-module to import is automatically calculated from the name of the Excel-file,
+  and it must be valid as a python module-name.  Therefore do not use non-alphanumeric characters such as 
+  spaces(` `), dashes(`-`) and dots(`.`) on the Excel-file.
 * Double-quotes(") do not work for denoting python-strings in the cells; use single-quotes(') instead.
 * You cannot enter multiline or indentated python-code such as functions and/or  ```if-then-else`` expressions; 
   move such code into the python-file. 
@@ -312,11 +328,13 @@ Some general notes regarding the python-code in excel-cells:
     the *VBA* modules of the demo-excel file ``xlwings`` and ``xlwings-ext`` into 
     your :file:`PERSONAL.XLSB` workbook, as explaine here: 
     http://office.microsoft.com/en-001/excel-help/copy-your-macros-to-a-personal-macro-workbook-HA102174076.aspx.
-
+    
     You can even `add a new Ribbon-button <http://msdn.microsoft.com/en-us/library/bb386104.aspx>`_ 
     to execute the selected cells as python-code.  Set this new button to invoke the ``RunSelectionAsPython()``
     *VBA* function.
 
+    If you do the above, remember that *VBA*-code in your personal-workbook takes precedance over any code
+    present in your currently open workbook.
 
 
 Python usage

WinPython_requirements.txt

@@ -0,0 +1,9 @@
+six
+jsonschema>=2.4,
+jsonpointer
+#prefixtree
+numpy
+pandas #openpyxl, xlrd,
+matplotlib
+Pillow
+xlwings>=0.2.3

postinstall.py

@@ -0,0 +1,54 @@
+#! python
+#-*- coding: utf-8 -*-
+#
+# Copyright 2013-2014 European Commission (JRC);
+# Licensed under the EUPL (the 'Licence');
+# You may not use this work except in compliance with the Licence.
+# You may obtain a copy of the Licence at: http://ec.europa.eu/idabc/eupl
+#! python
+# -*- coding: utf-8 -*-
+
+from __future__ import division, unicode_literals
+
+import os
+from os.path import expanduser
+import sys
+
+from cx_Freeze.dist import install
+
+PROG_GROUP='PythonWltp'
+
+def install():
+    start_menu = get_special_folder_path("CSIDL_STARTMENU")                 #@UndefinedVariable
+    prog_group = os.path.join(PROG_GROUP, start_menu)
+    menu_shortcuts = [
+        dict(path=os.path.join(sys.prefix, 'pythonw.exe'), 
+            description='Start GUI to run a single experiment.', 
+            filename=os.path(prog_group, 'TkWltp.lnk'),
+            arguments="wltpcmd.py --gui", 
+            #workdir=expanduser('~'), 
+            #iconpath=os.path.join(os.path.dirname(my_package.__file__), 'favicon.ico'), 
+            #iconindex=0
+        ),
+        dict(path=os.path.join(sys.prefix, 'pythonw.exe'), 
+            description='Copy `xlwings` excel & python template files into USERDIR and open Excel-file, to run a batch of experiments.', 
+            filename=os.path(prog_group, 'Open Wltp Excel.lnk'),
+            arguments="wltpcmd.py --excelrun", 
+            workdir=expanduser('~'), 
+        ),
+    ]
+
+    os.mkdir(prog_group) 
+    directory_created(prog_group)                                           #@UndefinedVariable
+
+    for shortcut in menu_shortcuts:    
+        create_shortcut(**shortcut)                                         #@UndefinedVariable
+        file_created(shortcut['filename'])                                  #@UndefinedVariable
+
+
+os.mkdir('D:\\tt')
+if __name__ == '__main__':
+    if sys.argv[1] == '-install':
+        install()
+    elif sys.argv[1] == '-remove':
+        print("Wltp: Nothing to uninstall.", file=sys.stderr)

setup.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#! python
 #-*- coding: utf-8 -*-
 #
 # Copyright 2013-2014 European Commission (JRC);
@@ -170,7 +170,7 @@ def clean_line(line):
         "Topic :: Scientific/Engineering",
         "Topic :: Software Development :: Libraries :: Python Modules",
     ],
-    scripts = ['wltpcmd.py'],
+    scripts = ['wltpcmd.py', 'postinstall.py'],
     install_requires = install_deps,
     setup_requires = [
         'setuptools>=3.4.4',
@@ -185,6 +185,11 @@ def clean_line(line):
     ],
     include_package_data = True,
     zip_safe=True,
+    options={
+        'bdist_wininst': {
+            'install_script': "postinstall.py",
+        },
+    }
 )
 
 

wltp/cmdline.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#! python
 #-*- coding: utf-8 -*-
 #
 # Copyright 2013-2014 European Commission (JRC);
@@ -157,6 +157,24 @@ def main(argv=None):
             copy_excel_template_files(opts.excel)
             return
 
+        if opts.excelrun:
+            from os.path import expanduser
+            destdir = expanduser("~")
+            files_copied = copy_excel_template_files(destdir)          #@UnusedVariable
+            xls_file = files_copied[0]
+
+            ## From http://stackoverflow.com/questions/434597/open-document-with-default-application-in-python
+            #     and http://www.dwheeler.com/essays/open-files-urls.html
+            import subprocess
+            try:
+                os.startfile(xls_file)
+            except AttributeError:
+                if sys.platform.startswith('darwin'):
+                    subprocess.call(('open', xls_file))
+                elif os.name == 'posix':
+                    subprocess.call(('xdg-open', xls_file))
+            return
+
         opts = validate_file_opts(opts)
 
         infiles     = parse_many_file_args(opts.I, 'r', opts.irenames)
@@ -197,26 +215,28 @@ def main(argv=None):
 
 
 def copy_excel_template_files(dest_dir):
-            import pkg_resources as pkg
+    import pkg_resources as pkg
+
+    if dest_dir == '<CWD>':
+        dest_dir = os.getcwd()
+    else:
+        dest_dir = os.path.abspath(dest_dir)
+
+    files_to_copy = ['excel\wltp_excel_runner.xlsm', 'excel\wltp_excel_runner.py']
+    files_to_copy = [pkg.resource_filename('wltp', f) for f in files_to_copy] #@UndefinedVariable
+    files_copied = []
+    for src_fname in files_to_copy:
+        dest_fname = os.path.basename(src_fname)
+        fname_genor = generate_filenames(os.path.join(dest_dir, dest_fname))
+        dest_fname = next(fname_genor)
+        while os.path.exists(dest_fname):
+            dest_fname = next(fname_genor)
 
-            if dest_dir == '<CWD>':
-                dest_dir = os.getcwd()
-            else:
-                dest_dir = os.path.abspath(dest_dir)
-
-            files_to_copy = ['excel\wltp_runner.xlsm', 'excel\wltp_runner.py']
-            files_to_copy = [pkg.resource_filename('wltp', f) for f in files_to_copy] #@UndefinedVariable
-            for src_fname in files_to_copy:
-                dest_fname = os.path.basename(src_fname)
-                fname_genor = generate_filenames(os.path.join(dest_dir, dest_fname))
-                dest_fname = next(fname_genor)
-                while os.path.exists(dest_fname):
-                    dest_fname = next(fname_genor)
-
-                log.info('Copying `xlwings` template-file: %s --> %s', src_fname, dest_fname)
-                shutil.copy(src_fname, dest_fname)
+        log.info('Copying `xlwings` template-file: %s --> %s', src_fname, dest_fname)
+        shutil.copy(src_fname, dest_fname)
+        files_copied.append(dest_fname)
 
-
+    return files_copied
 
 
 ## The value of file_frmt=VALUE to decide which
@@ -619,10 +639,13 @@ def build_args_parser(program_name, version, desc, epilog):
                         metavar='ARG')
 
 
-    grp_various = parser.add_argument_group('Various', 'Options controlling various other aspects.')
-    parser.add_argument('--gui', help='start in GUI mode', action='store_true')
-    parser.add_argument('--excel', help='Copy `xlwings` excel & python template files into DESTPATH or current-working dir', 
+    xlusive_group = parser.add_mutually_exclusive_group()
+    xlusive_group.add_argument('--gui', help='start GUI to run a single experiment', action='store_true')
+    xlusive_group.add_argument('--excel', help="copy `xlwings` excel & python template files into DESTPATH or current-working dir, to run a batch of experiments", 
         nargs='?', const='<CWD>', metavar='DESTPATH')
+    xlusive_group.add_argument('--excelrun', help="Copy `xlwings` excel & python template files into USERDIR and open Excel-file, to run a batch of experiments", action='store_true')
+
+    grp_various = parser.add_argument_group('Various', 'Options controlling various other aspects.')
     grp_various.add_argument('-d', "--debug", action="store_true", help=dedent("""\
             set debug-mode with various checks and error-traces
             Suggested combining with --verbose counter-flag.

wltp/excel/wltp_excel_runner.py

@@ -0,0 +1 @@
+#!python#-*- coding: utf-8 -*-"""Sample `xlwings` script####################### The functions included provide for running a batch of experiments in an excel-table with json-pointer paths as headers (see accompanying `wltp_excel_runner.xlsm` file).You can debug it by running it directly as a python script, as suggested by :   http://docs.xlwings.org/debugging.html<< EDIT THIS SCRIPTTO PUT YOUR EXCEL/XLWINGS PYTHON-CODE, BELOW >>"""from __future__ import division, print_function, unicode_literalsimport loggingimport osimport sysfrom wltp.experiment import Experimentimport pandas as pdimport wltp.pandel as pdlimport xlwings as xwdef _init_logging(loglevel):    logging.basicConfig(level=loglevel)    logging.getLogger().setLevel(level=loglevel)_init_logging(logging.INFO)log = logging.getLogger(__name__) def add_cycle_run_as_sheet(veh_id, mdl_out):    sheet = "cycle_%s"%veh_id    try:        xw.Sheet.add(sheet)    except Exception as ex:        log.info("Sheet(%s) already exists(%s).", sheet, ex)        xw.Sheet(sheet).clear()    xw.Range(sheet, 'A1').value = mdl_out['cycle_run']        def read_table_as_df(range):    """    Expects excel-table with jsonpointer paths as Header and 1st column named `id` as index,     like this::            id     vehicle/test_mass  vehicle/p_rated           vehicle/gear_ratios         veh_1               1500              100  [120.75, 75, 50, 43, 37, 32]         veh_2               1600               80  [120.75, 75, 50, 43, 37, 32]         veh_3               1200               60  [120.75, 75, 50, 43, 37, 32]             """    vehs = xw.Range(range).table.value    vehs = pd.DataFrame(vehs[1:], columns=vehs[0]).set_index('id')        return vehsdef build_models(vehs_df):    """    Builds all input-dataframes as Experiment classes and returns them in a list of (veh_id, exp) pairs.        :param vehs_df:     A dataframe indexed by veh_id, and with columns *json-pointer* paths into the model    :return: a list of (veh_id, :class:`wltp.experiment.Experiment`) tuples    """    experiment_pairs = []    for veh_id, row in vehs_df.iterrows():        try:            mdl_in = {}            for k, v in row.items():                log.debug('veh_id(%s): Column(%s): %s', veh_id, k, v)                if isinstance(v, str):                    v = eval(v)                pdl.set_jsonpointer(mdl_in, k, v)                        exp = Experiment(mdl_in)                        experiment_pairs.append((veh_id, exp))        except Exception as ex:            raise Exception('Invalid model for vehicle(%s): %s' % (veh_id, ex)) from ex             return experiment_pairsdef run_experiments(experiment_pairs):    """    Iterates `veh_df` and runs experiment_pairs.        :param vehs_df:     A dataframe indexed by veh_id, and with columns *json-pointer* paths into the model    """    for veh_id, exp in experiment_pairs:        try:            mdl_out = exp.run()                        add_cycle_run_as_sheet(veh_id, mdl_out)        except Exception as ex:            raise Exception('Experiment failed for vehicle(%s): %s' % (veh_id, ex)) from ex         if __name__ == '__main__':    ## Open and run experiments    #    log.info('CWD: %s', os.getcwd())    excel_fname = os.path.join(os.path.dirname(__file__), '%s.xlsm' % os.path.splitext(os.path.basename(__file__))[0])    wb_path = os.path.abspath(excel_fname)    xw.Workbook(wb_path)        vehs_df = read_table_as_df('D2')    log.info('%s', vehs_df)    exp_pairs = build_models(vehs_df)    run_experiments(exp_pairs)        

wltp/utils.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#! python
 #-*- coding: utf-8 -*-
 #
 # Copyright 2013-2014 European Commission (JRC);
@@ -113,6 +113,6 @@ def generate_filenames(filename):
     yield filename
     i = 1
     while True:
-        yield '%s.%i%s' % (f, i, e)
+        yield '%s%i%s' % (f, i, e)
         i += 1
 

wltpcmd.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#! python
 #-*- coding: utf-8 -*-
 #
 # Copyright 2013-2014 European Commission (JRC);