v0.2 commit

.gitignore

@@ -2,6 +2,8 @@
 other/
 tmp/
 download/
+sql/
+stats_server/
 
 # macOS stuff
 .DS_Store

README.md

@@ -1,15 +1,17 @@
-<!--- // cSpell:words killboard, blops, hics, killboard's, cynos, ccp's, pyspy, psf's, pyperclip, pyinstaller, executables, jojo, unported --->
+<!--- // cSpell:words killboard, blops, hics, killboard's, cynos, ccp's, pyspy, psf's, pyperclip, pyinstaller, executables, jojo, unported, killmails --->
 
 # PySpy - A simple EVE Online character intel tool using CCP's ESI API
 
 <p align="left">
+  <a href=https://github.com/WhiteRusssian/PySpy/releases/latest>
+    <img alt="Current Version" src="https://img.shields.io/github/release/whiterusssian/pyspy.svg">
+  </a>
   <a href=https://github.com/WhiteRusssian/PySpy/releases/latest>
     <img alt="Number of releases downloaded" src="https://img.shields.io/github/downloads/WhiteRusssian/PySpy/total.svg">
   </a>
 </p>
 
-**Download the latest release [here](https://github.com/WhiteRusssian/PySpy/releases/latest).** 
-
+**Download the latest release [here](https://github.com/WhiteRusssian/PySpy/releases/latest).**
 
 ## Background
 
@@ -26,24 +28,26 @@ PySpy connects to [CCP's ESI API](https://esi.evetech.net/ui/) and the
 3. Wait until PySpy is done and inspect the results.
 4. Double-click a name to open the respective zKillboard in your browser.
 
-**Note**: PySpy will save its window location, size, column sizes and transparency (slider on bottom right) settings automatically and restore them the next time you launch it. PySpy will stay on top of the EVE client so long as the game runs in *window mode*.
+**Note**: PySpy will save its window location, size, column sizes and transparency (slider on bottom right) and any other settings automatically and restore them the next time you launch it. PySpy will stay on top of the EVE client so long as the game runs in *window mode*.
 
 ## Information Provided by PySpy
 
 <p align="center">
-  <img alt="PySpy in action" src="https://github.com/WhiteRusssian/PySpy/blob/master/assets/v0.1_screenshot.png?raw=true">
+  <img alt="PySpy in action" src="https://github.com/WhiteRusssian/PySpy/blob/master/assets/v0.2_screenshot.png?raw=true">
 </p>
 
 * **Character**: Character name.
 * **Corporation**: Corporation of character.
-* **Alliance**: Alliance of Character's Corporation, if any.
+* **Alliance**: Alliance of character's Corporation, if any.
+* **Faction**: Faction of character, if any.
+* **Last Wk**: Number of kills over past 7 days.
 * **K**: Total number of kills.
 * **B**: Number of Black Ops Battleships (BLOPS) killed.
 * **H**: Number of lost Heavy Interdiction Cruisers (HIC).
 
-**Note**: Characters that have killed BLOPS or have lost HICs are high-lighted *orange*.
+**Note**: Characters that have killed BLOPS or have lost HICs are high-lighted *orange*. This can now be changed in the *View* menu.
 
-**Current Limitations**: To avoid undue strain on zKillboard's API, PySpy will run the *K-B-H* analysis only for the first 20 characters in the list.
+**Current Limitations**: To avoid undue strain on zKillboard's API, PySpy will run the *Last Wk* and *K-B-H* analyses only for the first 30 characters in the list.
 
 ## Installation
 
@@ -58,14 +62,14 @@ PySpy comes as a single-file executable both in Windows and macOS. On both platf
 Delete the PySpy executable and remove the following files manually:
 
 * **Windows**: PySpy saves preference and log files in a folder called  `PySpy` located at `%LocalAppData%`.
-* **macOS**: PySpy creates `pyspy.log` under `~/Library/Logs` and `pyspy.cfg` under `~/Library/Preferences`.
+* **macOS**: PySpy creates `pyspy.log` under `~/Library/Logs` and `pyspy.cfg` as well as `pyspy.pickle` under `~/Library/Preferences`.
 
 ## Future Features
 
 Below is a non-exhaustive list of additional features I plan to add to PySpy as and when the ESI and zKillboard APIs support them:
 
 * **Standings**: Only show characters that are non-blue, i.e. neutral or hostile.
-* **Cynos**: Indicate if a character has in the past lost ships fitted with regular or covert cynos.
+* **Cynos**: Indicate if a character has in the past lost ships fitted with regular or covert cynos. I am currently putting together the underlying database of killmails and expect to add this feature in August.
 * **Improved GUI**: The current GUI is very basic and while it works, I do appreciate that it is not ideal for people who cannot use it on a second screen but actually have to overlay it on-top of their EVE client.
 
 Please feel free to add a [feature request](https://github.com/WhiteRusssian/PySpy/issues/new?template=pyspy-feature-request.md) for any improvements you would like to see in future releases.
@@ -81,7 +85,7 @@ Despite PySpy's simplicity and extensive testing, you may encounter the odd bug.
 * Clipboard monitoring is implemented with the help of [pyperclip](https://github.com/asweigart/pyperclip) (v1.6.2), licensed under the [3-Clause BSD License](https://github.com/asweigart/pyperclip/blob/master/LICENSE.txt).
 * The GUI is powered by [wxPython](https://www.wxpython.org/) (v4.0.3), licensed under the [wxWindows Library Licence](https://wxpython.org/pages/license/index.html).
 * The Windows and macOS executables are built using [pyinstaller](https://www.pyinstaller.org/), licensed under [its own modified GPL license](https://raw.githubusercontent.com/pyinstaller/pyinstaller/develop/COPYING.txt).
-* PySpy's icon was created by Jojo Mendoza and is licensed under [Creative Commons (Attribution-Noncommercial 3.0 Unported)](https://creativecommons.org/licenses/by-nc/3.0/). It is available on https://www.iconfinder.com/icons/1218719/cyber_hat_spy_undercover_user_icon.
+* PySpy's icon was created by Jojo Mendoza and is licensed under [Creative Commons (Attribution-Noncommercial 3.0 Unported)](https://creativecommons.org/licenses/by-nc/3.0/). It is available on [IconFinder](https://www.iconfinder.com/icons/1218719/cyber_hat_spy_undercover_user_icon).
 
 ## License
 

VERSION

@@ -1 +1 @@
-0.1
+v0.2

__main__.py

@@ -22,6 +22,7 @@
 import chkversion
 import config
 import gui
+import reportstats
 import statusmsg
 # cSpell Checker - Correct Words****************************************
 # // cSpell:words russsian, ccp's, pyperclip, chkversion, clpbd, gui
@@ -31,6 +32,7 @@
 
 
 def watch_clpbd():
+    valid = False
     recent_value = None
     while True:
         clipboard = pyperclip.paste()
@@ -61,11 +63,13 @@ def analyze_chars(char_names):
     wx.CallAfter(app.PySpy.list.DeleteAllItems)
     try:
         outlist = analyze.main(char_names)
+        duration = round(time.time() - start_time, 1)
+        reportstats.ReportStats(outlist, duration).start()
         if outlist is not None:
-            wx.CallAfter(app.PySpy.update_list, outlist, start_time)
+            wx.CallAfter(app.PySpy.updateList, outlist, duration)
         else:
             statusmsg.push_status(
-                  "No valid character names found. Please try again..."
+                "No valid character names found. Please try again..."
                 )
     except Exception:
         Logger.error(

analyze.py

@@ -39,7 +39,8 @@ def main(char_names):
         t.join()
         zkill_stats = q_main.get()
         query_string = (
-            '''UPDATE characters SET kills=?, blops_kills=?, hic_losses=?
+            '''UPDATE characters SET kills=?, blops_kills=?, hic_losses=?,
+            week_kills=?
             WHERE char_id=?'''
             )
         db.write_many_to_db(conn, cur, query_string, zkill_stats)
@@ -77,16 +78,18 @@ def get_char_affiliations(conn, cur):
     except:
         Logger.info("Failed to obtain character affiliations.", exc_info=True)
         raise Exception
+
     records = ()
     for r in affiliations:
-        if "alliance_id" in r:
-            records = records + (
-                (r["corporation_id"], r["alliance_id"], r["character_id"]),
-                )
-        else:
-            records = records + ((r["corporation_id"], 0, r["character_id"]),)
+        corp_id = r["corporation_id"]
+        alliance_id = r["alliance_id"] if "alliance_id" in r else 0
+        faction_id = r["faction_id"] if "faction_id" in r else 0
+        char_id = r["character_id"]
+        records = records + ((corp_id, alliance_id, faction_id, char_id), )
+
     query_string = (
-        '''UPDATE characters SET corp_id=?, alliance_id=? WHERE char_id=?'''
+        '''UPDATE characters SET corp_id=?, alliance_id=?, faction_id=?
+        WHERE char_id=?'''
         )
     records_added = db.write_many_to_db(conn, cur, query_string, records)
 
@@ -136,15 +139,15 @@ def __init__(self, char_ids, q_main):
 
     def run(self):
         count = 0
-        max = 20
+        max = config.ZKILL_CALLS
         threads = []
         q_sub = queue.Queue()
         for id in self._char_ids:
             t = apis.Query_zKill(id[0], q_sub)
             threads.append(t)
             t.start()
             count += 1
-            time.sleep(0.05)
+            time.sleep(config.ZKILL_DELAY)
             if count >= max:
                 break
         for t in threads:
@@ -156,20 +159,22 @@ def run(self):
             kills = str(s[0])
             blops_kills = str(s[1])
             hic_losses = str(s[2])
-            id = str(s[3])
-            zkill_stats.append([kills, blops_kills, hic_losses, id])
+            week_kills = str(s[3])
+            id = str(s[4])
+            zkill_stats.append([kills, blops_kills, hic_losses, week_kills, id])
         self._q_main.put(zkill_stats)
         return
 
 
 def output_list(cur):
     query_string = (
         '''SELECT
-        ch.char_id, ch.char_name, co.name, al.name, ac.numid, ch.kills,
-        ch.blops_kills, hic_losses
+        ch.char_id, ch.faction_id, ch.char_name, co.name, al.name, fa.name,
+        ac.numid, ch.week_kills, ch.kills, ch.blops_kills, hic_losses
         FROM characters AS ch
         LEFT JOIN alliances AS al ON ch.alliance_id = al.id
         LEFT JOIN corporations AS co ON ch.corp_id = co.id
+        LEFT JOIN factions AS fa ON ch.faction_id = fa.id
         LEFT JOIN (SELECT alliance_id, COUNT(alliance_id) AS numid FROM characters GROUP BY alliance_id) AS ac ON
         ch.alliance_id = ac.alliance_id
         ORDER BY ch.char_name'''

apis.py

@@ -17,7 +17,7 @@
 import statusmsg
 # cSpell Checker - Correct Words****************************************
 # // cSpell:words wrusssian, ZKILL, gmail, blops, toon, HICs, russsian,
-# // cSpell:words ccp's
+# // cSpell:words ccp's, activepvp
 # **********************************************************************
 Logger = logging.getLogger(__name__)
 # Example call: Logger.info("Something badhappened", exc_info=True) ****
@@ -113,5 +113,13 @@ def run(self):
         except KeyError:
             hic_losses = 0
 
-        self._queue.put([kills, blops_kills, hic_losses, self._char_id])
+        try:
+            # Kills over past 7 days
+            week_kills = r["activepvp"]["kills"]["count"]
+        except KeyError:
+            week_kills = 0
+
+        self._queue.put(
+            [kills, blops_kills, hic_losses, week_kills, self._char_id]
+            )
         return

chkversion.py

@@ -8,28 +8,34 @@
 import logging
 import os
 import sys
+import datetime
 
 import requests
 import wx
 
 import __main__
 import config
+# cSpell Checker - Correct Words****************************************
+# // cSpell:words russsian
 # **********************************************************************
 Logger = logging.getLogger(__name__)
 # Example call: Logger.info("Something badhappened", exc_info=True) ****
 CURRENT_VER = config.CURRENT_VER
 
 
 def chk_github_update():
-    # Get latest version available on GitHub
-    GIT_URL = "https://raw.githubusercontent.com/WhiteRusssian/PySpy/master/VERSION"
-    try:
-        latest_ver = requests.get(GIT_URL).text.replace('\n', '')
-        Logger.info(
-            "You are running v" + CURRENT_VER + " and v" +
-            latest_ver + " is the latest version available on GitHub."
-            )
-    except:
-        Logger.info("Could not check GitHub for potential available updates.")
-    if latest_ver != CURRENT_VER:
-        wx.CallAfter(__main__.app.PySpy.update_alert, latest_ver, CURRENT_VER)
+    last_check = config.OPTIONS_OBJECT.Get("last_update_check", 0)
+    if last_check == 0 or last_check < datetime.date.today():
+        # Get latest version available on GitHub
+        GIT_URL = "https://api.github.com/repos/WhiteRusssian/PySpy/releases/latest"
+        try:
+            latest_ver = requests.get(GIT_URL).json()["tag_name"]
+            Logger.info(
+                "You are running " + CURRENT_VER + " and " +
+                latest_ver + " is the latest version available on GitHub."
+                )
+        except:
+            Logger.info("Could not check GitHub for potential available updates.")
+        config.OPTIONS_OBJECT.Set("last_update_check", datetime.date.today())
+        if latest_ver != CURRENT_VER:
+            wx.CallAfter(__main__.app.PySpy.updateAlert, latest_ver, CURRENT_VER)

config.py

@@ -8,10 +8,14 @@
 import logging
 import os
 import sys
+import uuid
 
 import requests
+
+import optstore
 # cSpell Checker - Correct Words****************************************
 # // cSpell:words MEIPASS, datefmt, russsian, pyinstaller, posix, pyspy
+# // cSpell:words zkill, amarr, caldari, gallente, minmatar
 # **********************************************************************
 Logger = logging.getLogger(__name__)
 # Example call: Logger.info("Something badhappened", exc_info=True) ****
@@ -28,15 +32,6 @@ def resource_path(relative_path):
 
     return os.path.join(base_path, relative_path)
 
-# Read current version from VERSION file
-with open(resource_path('VERSION'), 'r') as ver_file:
-    CURRENT_VER = ver_file.read().replace('\n', '')
-
-# Various constants
-MAX_NAMES = 500  # The max number of char names to be processed
-GUI_TITLE = "PySpy v" + CURRENT_VER
-
-
 # If application is frozen executable
 if getattr(sys, 'frozen', False):
     ABOUT_ICON = resource_path("pyspy_mid.png")
@@ -68,6 +63,47 @@ def resource_path(relative_path):
 
 LOG_FILE = os.path.join(LOG_PATH, "pyspy.log")
 GUI_CFG_FILE = os.path.join(PREF_PATH, "pyspy.cfg")
+OPTIONS_FILE = os.path.join(PREF_PATH, "pyspy.pickle")
+
+# Persisten options object
+OPTIONS_OBJECT = optstore.PersistentOptions(OPTIONS_FILE)
+
+# Read current version from VERSION file
+with open(resource_path('VERSION'), 'r') as ver_file:
+    CURRENT_VER = ver_file.read().replace('\n', '')
+
+# Clean up old GUI_CFG_FILES
+if os.path.isfile(GUI_CFG_FILE) and not os.path.isfile(OPTIONS_FILE):
+    try:
+        os.remove(GUI_CFG_FILE)
+    except:
+        pass
+if OPTIONS_OBJECT.Get("version", 0) != CURRENT_VER:
+    try:
+        os.remove(GUI_CFG_FILE)
+    except:
+        pass
+
+# Unique identifier for usage statistics reporting
+if OPTIONS_OBJECT.Get("uuid", "not set") == "not set":
+    OPTIONS_OBJECT.Set("uuid", str(uuid.uuid4()))
+
+# Various constants
+MAX_NAMES = 500  # The max number of char names to be processed
+ZKILL_DELAY = 0.05  # API rate limit is 10/second, pushing it a little...
+ZKILL_CALLS = 30
+GUI_TITLE = "PySpy " + CURRENT_VER
+
+# Note, Amarr and Caldari are allied and have IDs ending on uneven integers.
+# Likewise, Gallente and Minmatar, also allied, have even IDs.
+# We will use this to block certain faction alliances.
+FACTION_IDS = (
+    (("500001", "Caldari"), ) +
+    (("500002", "Minmatar"), ) +
+    (("500003", "Amarr"), ) +
+    (("500004", "Gallente"), )
+)
+IGNORED_FACTIONS = OPTIONS_OBJECT.Get("IgnoredFactions", 0)
 
 # Logging setup
 ''' For each module that requires logging, make sure to import modules

db.py

@@ -5,8 +5,10 @@
 '''Establishes an in memory SQLite3 database and creates a few tables as
 well as provides a function to write records to the database.'''
 # **********************************************************************
-import sqlite3
 import logging
+import sqlite3
+
+import config
 # cSpell Checker - Correct Words****************************************
 # // cSpell:words wrusssian, sqlite, blops, russsian
 # **********************************************************************
@@ -31,15 +33,22 @@ def prepare_tables(conn, cur):
     function.'''
     cur.execute(
         '''CREATE TABLE IF NOT EXISTS characters (char_name TEXT, char_id INT,
-        corp_id INT, alliance_id INT, kills INT, blops_kills INT,
-        hic_losses INT)'''
+        corp_id INT, alliance_id INT, faction_id INT, kills INT,
+        blops_kills INT, hic_losses INT, week_kills INT)'''
         )
     cur.execute(
         '''CREATE TABLE IF NOT EXISTS corporations (id INT, name TEXT)'''
         )
     cur.execute(
         '''CREATE TABLE IF NOT EXISTS alliances (id INT, name TEXT)'''
         )
+    cur.execute(
+        '''CREATE TABLE IF NOT EXISTS factions (id INT, name TEXT)'''
+        )
+    cur.executemany(
+        '''INSERT INTO factions (id, name) VALUES (?, ?)''',
+        config.FACTION_IDS
+    )
     conn.commit()