Merge branch 'dev_0.7.0' into gh-pages

.github/workflows/python-fhem-test.yaml

@@ -0,0 +1,61 @@
+# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+
+name: Python package
+
+on:
+  push:
+    branches: [ "master", "dev_0.7.0" ]
+  pull_request:
+    branches: [ "master" ]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.8", "3.10", "3.11"]
+
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v3
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Get perl stuff
+      run: |
+        sudo apt install perl libio-socket-ssl-perl
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install twine build
+    - name: Build python-fhem package
+      run: |
+        cd fhem
+        cp ../README.md .
+        python -m build
+    - name: Install python-fhem
+      run: |
+        python -m pip install ./fhem/dist/*.gz
+        rm ./fhem/dist/*
+#     - name: Install fhem server
+#       run: |
+#         cd selftest
+#         wget -nv https://fhem.de/fhem-6.0.tar.gz
+#         mkdir fhem
+#         cd fhem
+#         tar -xzf ../fhem-6.0.tar.gz
+#         cd fhem-6.0
+#         mkdir certs
+#         cd certs
+#         openssl req -newkey rsa:2048 -nodes -keyout server-key.pem -x509 -days 36500 -out server-cert.pem -subj "/C=DE/ST=NRW/L=Earth/O=CompanyName/OU=IT/CN=www.example.com/[email protected]"
+#         cd ..
+#         cp ../../fhem-config-addon.cfg fhem.cfg
+#         perl fhem.pl fhem.cfg
+    - name: Test with selftest.py
+      run: |
+        cd selftest
+        python selftest.py
+# python selftest.py --reuse

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2016-2018 Dominik Schlösser, [email protected]
+Copyright (c) 2016-2023 Dominik Schlösser, [email protected]
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,6 +1,5 @@
 [![PyPI version](https://badge.fury.io/py/fhem.svg)](https://badge.fury.io/py/fhem)
-[![TravisCI Test Status](https://travis-ci.org/domschl/python-fhem.svg?branch=master)](https://travis-ci.org/domschl/python-fhem)
-[![Codacy Badge](https://api.codacy.com/project/badge/Grade/116e9e988d934aaa9cfbfa5b8aef7f78)](https://www.codacy.com/app/dominik.schloesser/python-fhem?utm_source=github.com&utm_medium=referral&utm_content=domschl/python-fhem&utm_campaign=Badge_Grade)
+[![Python package](https://github.com/domschl/python-fhem/actions/workflows/python-fhem-test.yaml/badge.svg)](https://github.com/domschl/python-fhem/actions/workflows/python-fhem-test.yaml)
 [![License](http://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat)](LICENSE)
 [![Docs](https://img.shields.io/badge/docs-stable-blue.svg)](https://domschl.github.io/python-fhem/index.html)
 
@@ -10,7 +9,7 @@ Python FHEM (home automation server) API
 
 Simple API to connect to the [FHEM home automation server](https://fhem.de/) via sockets or http(s), using the telnet or web port on FHEM with optional SSL (TLS) and password or basicAuth support.
 
-**Note:** Python 2.x deprecation warning. `python-fhem` versions 0.6.x will be the last versions supporting Python 2.x.
+**Note:** Starting with verson 0.7.0, Python 2.x is no longer supported with `python-fhem`. If you still require support for Python 2, use versions 0.6.5.
 
 ## Installation
 
@@ -24,28 +23,22 @@ pip install [-U] fhem
 
 ### From source
 
-In `python-fhem/fhem`:
-
-Get a copy of README for the install (required by setup.py):
-
-```bash
-cp ../README.md .
-```
-
-then:
+To build your own package, install `python-build` and run:
 
 ```bash
-pip install [-U] .
+cd fhem
+python -m build
 ```
 
-or, as developer installation, allowing inplace editing:
+This will create a `dist` directory with the package. Install with:
 
 ```bash
-pip install [-U] -e .
+pip install [-U] dist/fhem-<version>.tar.gz
 ```
-
+    
 ## History
 
+* 0.7.0 (2023-08-17): [unpublished] Ongoing: move Travis CI -> Github actions, Python 2.x support removed, modernize python packaging, global states for SSL and authentication removed (support for multiple sessions).
 * 0.6.6 (2022-11-09): [unpublished] Fix for new option that produces fractional seconds in event data.
 * 0.6.5 (2020-03-24): New option `raw_value` for `FhemEventQueue`. Default `False` (old behavior), on `True`, the full, unparsed reading is returned, without looking for a unit.
 * 0.6.4 (2020-03-24): Bug fix for [#21](https://github.com/domschl/python-fhem/issues/21), Index out-of-range in event loop background thread for non-standard event formats.  
@@ -142,12 +135,7 @@ The library can create an event queue that uses a background thread to receive
 and dispatch FHEM events:
 
 ```python
-try:
-    # Python 3.x
-    import queue
-except:
-    # Python 2.x
-    import Queue as queue
+import queue
 import fhem
 
 que = queue.Queue()
@@ -160,6 +148,11 @@ while True:
     que.task_done()
 ```
 
+## Selftest
+
+For a more complete example, you can look at [`selftest/selftest.py`](https://github.com/domschl/python-fhem/tree/master/selftest). This automatically installs an FHEM server, and runs a number of tests,
+creating devices and checking their state using the various different transports.
+
 # Documentation
 
 see: [python-fhem documentation](https://domschl.github.io/python-fhem/index.html)

fhem/fhem/__init__.py

@@ -69,6 +69,7 @@ def __init__(
         self.bsock = None
         self.sock = None
         self.https_handler = None
+        self.opener = None
 
         # Set LogLevel
         # self.set_loglevel(loglevel)
@@ -96,31 +97,42 @@ def __init__(
     def connect(self):
         """create socket connection to server (telnet protocol only)"""
         if self.protocol == "telnet":
-            try:
-                self.log.debug("Creating socket...")
-                if self.ssl:
-                    self.bsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-                    self.sock = ssl.wrap_socket(self.bsock)
-                    self.log.info(
-                        "Connecting to {}:{} with SSL (TLS)".format(
-                            self.server, self.port
-                        )
-                    )
-                else:
-                    self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-                    self.log.info(
-                        "Connecting to {}:{} without SSL".format(self.server, self.port)
-                    )
-
-                self.sock.connect((self.server, self.port))
-                self.connection = True
-                self.log.info("Connected to {}:{}".format(self.server, self.port))
-            except socket.error:
-                self.connection = False
-                self.log.error(
-                    "Failed to connect to {}:{}".format(self.server, self.port)
+            # try:
+            self.log.debug("Creating socket...")
+            if self.ssl:
+                self.bsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+                context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
+                context.check_hostname = False
+                context.verify_mode = ssl.CERT_NONE
+                self.sock = context.wrap_socket(self.bsock)
+                self.log.info(
+                    "Connecting to {}:{} with SSL (TLS)".format(self.server, self.port)
                 )
-                return
+            else:
+                self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+                self.log.info(
+                    "Connecting to {}:{} without SSL".format(self.server, self.port)
+                )
+            # except Exception as e:
+            #    self.connection = False
+            #     self.log.error(
+            #         "Failed to create socket to {}:{}: {}".format(self.server, self.port, e)
+            #     )
+            #     return
+
+            self.log.debug("pre-connect (no try/except)")
+            # try:
+            # self.sock.timeout = 5.0
+            self.sock.connect((self.server, self.port))
+            self.log.debug("post-connect")
+            # except Exception as e:
+            #     self.connection = False
+            #     self.log.error(
+            #         "Failed to connect to {}:{}: {}".format(self.server, self.port, e)
+            #     )
+            #     return
+            self.connection = True
+            self.log.info("Connected to {}:{}".format(self.server, self.port))
 
             if self.password != "":
                 # time.sleep(1.0)
@@ -198,6 +210,9 @@ def close(self):
 
     def _install_opener(self):
         self.opener = None
+        self.auth_handler = None
+        self.password_mgr = None
+        self.context = None
         if self.username != "":
             self.password_mgr = HTTPPasswordMgrWithDefaultRealm()
             self.password_mgr.add_password(
@@ -221,9 +236,9 @@ def _install_opener(self):
         else:
             if self.username != "":
                 self.opener = build_opener(self.auth_handler)
-        if self.opener is not None:
-            self.log.debug("Setting up opener on: {}".format(self.baseurlauth))
-            install_opener(self.opener)
+        # if self.opener is not None:
+        #     self.log.debug("Setting up opener on: {}".format(self.baseurlauth))
+        #     install_opener(self.opener)
 
     def send(self, buf, timeout=10):
         """Sends a buffer to server
@@ -255,6 +270,9 @@ def send(self, buf, timeout=10):
                 return None
         else:  # HTTP(S)
             paramdata = None
+            # if self.opener is not None:
+            #     install_opener(self.opener)
+
             if self.csrf and len(buf) > 0:
                 if len(self.csrftoken) == 0:
                     self.log.error("CSRF token not available!")
@@ -263,36 +281,46 @@ def send(self, buf, timeout=10):
                     datas = {"fwcsrf": self.csrftoken}
                     paramdata = urlencode(datas).encode("UTF-8")
 
-            try:
+            # try:
+            if len(buf) > 0:
                 self.log.debug("Cmd: {}".format(buf))
                 cmd = quote(buf)
                 self.log.debug("Cmd-enc: {}".format(cmd))
+            else:
+                cmd = ""
+            if len(cmd) > 0:
+                ccmd = self.baseurl + cmd
+            else:
+                ccmd = self.baseurltoken
 
-                if len(cmd) > 0:
-                    ccmd = self.baseurl + cmd
-                else:
-                    ccmd = self.baseurltoken
-
-                self.log.info("Request: {}".format(ccmd))
-                if ccmd.lower().startswith("http"):
-                    ans = urlopen(ccmd, paramdata, timeout=timeout)
+            self.log.info("Request: {}".format(ccmd))
+            if ccmd.lower().startswith("http"):
+                if self.opener is not None:
+                    ans = self.opener.open(ccmd, paramdata, timeout=timeout)
                 else:
-                    self.log.error(
-                        "Invalid URL {}, Failed to send msg, len={}, {}".format(
-                            ccmd, len(buf), err
+                    if self.context is None:
+                        ans = urlopen(ccmd, paramdata, timeout=timeout)
+                    else:
+                        ans = urlopen(
+                            ccmd, paramdata, timeout=timeout, context=self.context
                         )
+            else:
+                self.log.error(
+                    "Invalid URL {}, Failed to send msg, len={}, {}".format(
+                        ccmd, len(buf), err
                     )
-                    return None
-                data = ans.read()
-                return data
-            except URLError as err:
-                self.connection = False
-                self.log.error("Failed to send msg, len={}, {}".format(len(buf), err))
-                return None
-            except socket.timeout as err:
-                # Python 2.7 fix
-                self.log.error("Failed to send msg, len={}, {}".format(len(buf), err))
+                )
                 return None
+            data = ans.read()
+            return data
+            # except URLError as err:
+            #     self.connection = False
+            #     self.log.error("Failed to send msg, len={}, {}".format(len(buf), err))
+            #     return None
+            # except socket.timeout as err:
+            #     # Python 2.7 fix
+            #     self.log.error("Failed to send msg, len={}, {}".format(len(buf), err))
+            #     return None
 
     def send_cmd(self, msg, timeout=10.0):
         """Sends a command to server.

fhem/setup.py

@@ -1,23 +1,25 @@
-from setuptools import setup
+import setuptools
 
 with open("README.md", "r") as fh:
     long_description = fh.read()
 
-setup(
+setuptools.setup(
     name="fhem",
     version="0.7.0",
+    author="Dominik Schloesser",
+    author_email="[email protected]",
     description="Python API for FHEM home automation server",
     long_description=long_description,
     long_description_content_type="text/markdown",
+    url="http://github.com/domschl/python-fhem",
+    project_urls={"Bug Tracker": "https://github.com/domschl/python-fhem/issues"},
     classifiers=[
+        "Programming Language :: Python :: 3",
         "Development Status :: 5 - Production/Stable",
         "License :: OSI Approved :: MIT License",
+        "Operating System :: OS Independent",
     ],
-    keywords="fhem home automation",
-    url="http://github.com/domschl/python-fhem",
-    author="Dominik Schloesser",
-    author_email="[email protected]",
-    license="MIT",
-    packages=["fhem"],
-    zip_safe=False,
+    package_dir={"": "."},
+    packages=setuptools.find_packages(where="."),
+    python_requires=">=3.6",
 )

publish.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+if [ ! -f ~/.pypirc ]; then
+    echo "Please configure .pypirc for pypi access first"
+    exit -2
+fi
+if [[ ! -d fhem/dist ]]; then
+    mkdir fhem/dist
+fi
+cd fhem
+cp ../README.md .
+rm dist/*
+export PIP_USER=
+python -m build
+# twine upload dist/*
+