V4.3.0

[4.3.0] - 2022-05-17

Added

• New Per a user request, I added the ability to use sshpass with the STDIO over SSH access method.
  This allows you to authenticate with user/pw instead of having to use rsa keys (passwordless shh). There
  are now two new keys (see the doc) for providing the path to the sshpass executable and the parameters to use.
  The ssh key and other keys for this configuration stay the same.

Changed

• Enhanced The sd2df methods require multiple interactions (code submissions) to the SAS server and if any of these
  intermediate steps fail or have some issue, then the method invocation fails. But, I wasn't catching these intermediate
  problems which could cause non-obvious exceptions and tracebacks which were confusing. I've added code to catch failures
  for these intermediate steps and throw a more clear exception if that happens. Nothing about how the methods works has
  changed, just better error handling.

• Enhanced The code that checks for an 'ERROR:' in the log and issues a warning to alert you to look to see what the
  error was, wasn't only looking for it to start in column 1 of the log. So, it could pick up 'ERROR:' in a comment in the
  code or anywhere in the log. I've enhanced this to limit false positives by only flagging 'ERROR:' starting in the first
  column of the LOG.

• Tweak Fixed a typo in an error message.

Fixed

• Fix The COM Access Method had a bug where the encoding was being used to transcode HTML results returned to
  SASPy when it shouldn't, since the HTML results are already utf-8. Other code paths retrieving data from SAS did need to
  use the provided encoding for transcoding from SAS Session encoding to utf-8. So, this one path was changed to not try to
  transcode the HTML results. This was a fix for issue 454.

Removed

• None Nothing removed

V4.2.0

[4.2.0] - 2022-03-22

Added

• None Nothing Added

Changed

• Tweak If the initial connection to SAS (a SASsession) fails then there will be a failure error provided, but there
  is also an exception thrown in subsequent code, trying to submit some of the initial code to gather info about the
  session. This was subordinate and inconsequential, yet was confusing and needed to be addressed. So, in this release, there
  is a specific exception being thrown instead of the one that happened to be thrown, which makes the problem more clear.

Fixed

• Fix There was an issue opened for the STDIO Access Method where if the code being submitted was longer than 128K,
  SASPy would deadlock with SAS due to both being single threaded and the way Pipes work; trying to flush STDIN would block
  in SASPy because SAS was blocked on writing to STDOUT/ERR, so they would both deadlock, not being able to read off of the
  other Pipes. I reworked how submit works in STDIO, so that there's a blocked amount of STDIN sent before trying to pull off
  of both STDOUT and STDERR, iterating till done, to circumvent this behavior. That works great, but it caused issues with
  ATTN handling that I support in that Access Method. I needed to rework that to compensate and it's handling it much better.

Removed

• None Nothing removed

V4.1.0

Added

• New I added a new method, lib_path(), off the SASsession object which returns a list of the path(s) for
  the libref. Depending upon the engine, 'path' may mean different things. For BASE engines (Linux/PC), it's a directory
  or multiple directories if a concatenated libref. For database engines, it varies; may be the database name that you're
  connected to. On MVS, it could be then name of a bound library file, or a linux directory path.

Changed

• Tweak In the submit method of the HTTP access method, for results='text', I removed extraneous empty lines before
  and after the code. These resulted in extra 'return's being submitted. Removing these allows correct behavior when using
  saspy to interactively debug (w/ pdb) PROC PYTHON code. With the extra 'return' before and after the command you tried
  to submit, pdb executed extra commands. Now it's 1:1 with clean debugging using submit().

Fixed

• None No changes

Removed

• None Nothing removed

V4.0.0

Note the Major release number being incremented! This means there is a breaking change in this release (yes, the first time so far).

Also note the addition of the CHANGELOG.md file in the repo, where all release changes will be logged, starting with this release!

The breaking change is as follows. sasdata2dataframe (sd2df or other aliases), has returned None when the SAS Data Set provided doesn't exist. Of course, trying to use that as a dataframe results in an exception. But, if you coded a check for this and conditionally didn't use it, then this change will break that code. A FileNotFoundError exception is now thrown, as that's the appropriate Pythonic thing to do in this situation.

If you happened to code this:

df = sas.sd2df('table','libref')
if df is None:
   # do something because the data set wasn't found in the SAS session

Then you would need to change that to:

try:
   df = sas.sd2df('table','libref')
except FileNotFoundError:
   # do something because the data set wasn't found in the SAS session

There is another change that I had to make for the log4j problems. If you recently used the new 'log4j' configuration key in the pervious release, you may need to update the value. SASPy V3.7.8 had log4j 2.12.2 and 2.16.0 jars, but now 2.12.4 and 2.17.1 are included instead.

So, if you happened to use log4j='2.16.0', you need to change that to log4j='2.17.1' in this release. Be aware that this can continue to change if more vulnerabilities are found and fixed in log4j.

Note also that none of the vulnerabilities are exposed by SASPy as it doesn't use log4j, so there's no actual problem with these regardless of their version.

V3.7.8

This release addresses the current issues with log4j by providing both the fixed version (https://logging.apache.org/log4j/2.x/index.html) of log4j that still supports Java 7, saspy's minimum version of Java, using log4j 2.12.2, but also includes the 2.16.0 versions which you can choose to use, if you want and have Java 8 or higher. There is a new configuration definition key, 'log4j', that you can provide in either your config definition or on the SASsession() method (using 2.12.2 is the default - you don't need to provide this unless trying to use 2.16.0). This is documented in the SASPy V3.7.8 documentation under configuring the IOM using Java section.
Note that SASPy doesn't expose any log4j vulnerabilities, regardless of the log4j version, because it doesn't use or even initialize log4j. But, having these version in the deployment will mitigate security scans and 'false positives'. There is more discussion on all of this in the #429 issue. Feel free to read more there, an respond if you want or have any other questions or concerns.

V3.7.7

This version updates the log4j jar versions to patch the security issues reported in the last few days.
The details are found in #429.

Note that this change now causes the minimum Java version to move to Java 8, from the previous minimum of Java 7 (log4j version 2.12, which is what saspy included until now, was the highest version still supporting java 7). Also note that saspy does not expose any of the the log4j vulnerabilities, so it's not necessary nor required to move to this version. None of the known, or still to be found, log4j vulnerabilities are exposed by saspy because it neither uses nor even initializes log4j, so no logging is done via log4j; neither internal nor any containing user provided content.

V3.7.6

This release just has 2 fixes in it, from the previous version. Fixes for issues #421 and #422.

V3.7.5

This release has a user requested enhancement and another user contributed enhancement. Both of these are doc'ed in the Advanced Topics section of the saspy doc (https://sassoftware.github.io/saspy/advanced-topics.html#).

The first is an enhancement to support the Python logging facility, instead of saspy using print() to write messages to the output. This is a great feature and provides a lot of way to configure it yo your needs; from setting what level of messages to get, and to specifying whether the show up in the output or in a file if you like. You can even provide formatting, like timestamps prepended to the messages or whatever you want.

The other is a contribution (#401) which allows the SASsession object to be used as a Context Manager, like in the 'with' statement. This automatically ends the session once the 'with' block of code has completed. Very convenient.

Another thing is for getting the Script directory (containing the user contributed script which was added in V3.7.3) actually installed with SASPy. I forget to add that directory to the selection criteria for what get's deployed when you pip install. My bad on that one. But, now the run_sas.py script will be available in the saspy deployment when you install it.

Other than that there were some doc updates and changes and a couple other tweaks, but the above are the main things added to this release.

V3.7.4

This release has just two enhancements. The first is to support embedded quotes in SAS variable and table names. SAS supports this via the n-literal syntax (which SASPy has always used in it's code gen). For example: 'other"name'n or "other''name"n. or 'other''name'n
you can't tell here (embeded quotes are a pain everywhere), but some of those are 2 single quotes a some one double quote ... (took me 30 min just to get these examples to resolve in this editor!)

The n-literal syntax supports all kinds of things, not just quotes; blanks, special chars,...
So saspy has generated that syntax all along, but I misses the embedded quotes case and it tangles up parsing because they require special handling compared to anything else in the n-lit syntax (have to double single quotes, double quote single quotes, . bla bla bla).
So this version should handle all of this appropriately. Here's a SAS example using this:

data "cars''2"n; set sashelp.cars( rename=( Make='Make''s'n Model='Model"''s'n Cylinders="Cylinder''s"n horsepower="HP''s"n ) ); run;

For methods that take table or variable names explicitly, you can use Python string syntax to provide then; no need for the n-lit syntax; I generate the corect SAS code for those. But some methods take arbitrary string that are embedded directly into the code w/ no massaging from SASPy (no idea what the syntax is to be able to do so), so for those cases, you have to provide correct SAS (including n-literal as necessary) syntax yourself:

For known table or variable name just use Python string syntax for embedded quotes
cars = sas.sasdata("cars'2")
cars.heatmap('msrp',"hp's")

For arbitrary strings that are just included into the code, use SAS syntax
cars.where(''' "HP's"n > 400 ''').head()
stat_results = stat.reg(model=" 'hp''s'n = 'Cylinder''s'n EngineSize ",data=cars)

The other enhancement is is specific to Viya 4 (the HTTP access method), for authenticating to Viya. Originally user/pw were the only way. Right before SGF 2021, SSO (Single Sign On) was implemented which uses a client_it, client_secret and authcode, and a client_id/secret was created for SASPy which now ships w/ Viya 4, so all you actually need to do is just provide the authcode, which is like when you log onto a secure account and it text's you a code to type in to prove it's your, even though you've already authenticated, as an extra security mechanism. So that's already there. But what's new in this release is another variation on that.

The HTTP access to Viya 4 now support providing a JWT witch is an authentication token from other than SASLogon, which (if both systems are configured to share), can be used to acquire the SASLongon authentication token that is required for all API calls into Viya. This allows, for instance, an Azure ML instance to connect to Viya by providing the Azure auth token to saspy which then uses that (as opposed to user/pw or authcode) to get the SASLogon auth token. This config option is jwt= on the SASsesion() method. Support for passing a SASLogon token directly in is also available (meaning you've already somehow made calls to SASLogon yourself, or some other tool, and have a valid authtoken), via authtoken= on the SASsession() method. Both of these are really expected to be used under the covers by other tools/processes, but can be used by advanced users who can interface to the authenticating providers and acquire these tokens themselves directly.

V3.7.3

This release has a few enhancements. The IOM access method uses the Java IOM client (some jars files), and the log4j.jar in that is pretty old; 1.2. That version if past EOL, and raises flags on some security scans. So, I've upgraded to 2.12, which isn't the absolute newest, but it's close and the highest version that still supports Java7; one higher and Java8 is the minimum. Since saspy still supports back to Java7, and it isn't EOL till end of this year, I've gone w/ log4j 2.12. Maybe upgrade it in the future after java7 is dead.

Another cool thing is a user contributed script (see it in the new saspy/scripts dir, and pr #392 ) for batch submitting a .sas file, similar to how you could batch submit a .sas file with SAS from the command line. Check it out.

Another little enhancements is the the char_lengths parm of df2sd, which now accepts only the subset of columns you want to overrise; the others get calculated per usual. Before you had to provide the whole list.

There's a fix for SAS other missing values ('.A'-'.Z' and '._') from issue #390, so now those are all NaN,s when transferred, as they should have been.

Some doc changes for things, and that's about it.