2.6.0 - 2024-04-15
reindex
: base reindexer functionflagGeneric
,processGeneric
: target broadcasting and numpy array supportSaQC
: automatic translation of incoming flags- Option to change the flagging scheme after initialization
flagByClick
: manually assign flags using a graphical user interfaceSaQC
: support for selection, slicing and setting of items by subscription onSaQC
objectstransferFlags
is a multivariate functionplot
: addedyscope
keywordsetFlags
: function to replaceflagManual
flagUniLOF
: added parameterslope_correct
to correct for overflagging at relatively steep data value slopesHistory
: added option to change aggregation behavior- "horizontal" axis / multivariate mode for
rolling
- Translation scheme
AnnotatedFloatScheme
SaQC.flags
always returns aDictOfSeries
SaQC
methods deprecated in version 2.4:interpolate
,interpolateIndex
,interpolateInvalid
,roll
,linear
,shift
,flagCrossStatistics
- Method
Flags.toDios
deprecated in version 2.4 - Method
DictOfSeries.index_of
method deprecated in version 2.4 - Option
"complete"
for parameterhistory
of methodplot
- Option
"cycleskip"
for parameterax_kwargs
of methodplot
- Parameter
phaseplot
from methodplot
flagConstants
: fixed flagging of rolling rampsFlags
: add meta entry to imported flags- group operations were overwriting existing flags
SaQC._construct
: was not working for inherited classesprocessgeneric
: improved numpy function compatability
flagManual
in favor ofsetFlags
inverse_**
options forconcatFlags
parametermethod
in favor ofinvert=True
flagRaise
with delegation to better replacementsflagZScore
,flagUniLOF
,flagJumps
orflagOffset
flagByGrubbs
with delegation to better replacementsflagZScore
,flagUniLOF
sflagMVScore
with delegation to manual application of the steps
2.5.0 - 2023-09-05
- WMO standard mean aggregations
- Function selection via strings for most function-expecting parameters
SaQC.plot
:- enable multivariate plots
- keyword
plot_kwargs
to pass matplotlib related arguments
- CLI:
--version
to print the SaQC version-ll
as a shorthand for--log-level
--json-field
to use a non-root element of a json file.- basic json support for CLI config files, which are detected by
.json
-extension.
SaQC.flagScatterLowpass
: option to select function based on string names.- Checks and unified error message for common function inputs.
- Require pandas >= 2.0
SaQC.flagUniLOF
andSaQC.assignUniLOF
: changed parameterfill_na
to typebool
.SaQC.plot
:- changed default color for single variables to
black
with80% transparency
- added seperate legend for flags
- changed default color for single variables to
SaQC.plot
: option to plot with complete history (history="complete"
)- Support for Python 3.8
SaQC.assignChangePointCluster
andSaQC.flagChangePoints
: A tuple passedmin_period
was only recognised ifwindow
was also a tuple.SaQC.propagateFlags
was overwriting existing flags
SaQC.andGroup
andSaQC.orGroup
: option to pass dictionaries togroup
.SaQC.plot
:phaseplot
in favor of usage withmode="biplot"
cyclestart
in favor of usage withmarker_kwargs
SaQC.flagStatLowPass
in favor ofSaQC.flagScatterLowpass
2.4.1 - 2023-06-22
- pin pandas to versions >= 2.0
- removed deprecated
DictOfSeries.to_df
2.4.0 - 2023-04-25
- Methods
logicalAnd
andlogicalOr
Flags
support slicing and column selection withlist
or apd.Index
.- Expose the
History
viaSaQC._history
- Config function
cv
(coefficient of variation)
- Rename
interplateInvalid
tointerpolate
- Rename
interpolateIndex
toalign
- Rewrite of
dios.DictOfSeries
- Parameter
limit
fromalign
- Parameter
max_na_group_flags
,max_na_flags
,flag_func
,freq_check
fromresample
func
arguments in text configurations were not parsed correctly- fail on duplicated arguments to test methods
reample
was not writing meta entriesflagByScatterLowpass
was overwriting existing flagsflagUniLOF
andflagLOF
were overwriting existing flags
- Deprecate
flagMVScore
parameters:partition
in favor ofwindow
,partition_min
in favor ofmin_periods
,min_periods
in favor ofmin_periods_r
- Deprecate
interpolate
,linear
andshift
in favor ofalign
- Deprecate
roll
in favor ofrolling
- Deprecate
DictOfSeries.to_df
in favor ofDictOfSeries.to_pandas
2.3.0 - 2023-01-17
- add option to not overwrite existing flags to
concatFlags
- add option to pass existing axis object to
plot
- python 3.11 support
- added Local Outlier Factor functionality
- Remove all flag value restrictions from the default flagging scheme
FloatTranslator
- Renamed
TranslationScheme.forward
toTranslationScheme.toInternal
- Renamed
TranslationScheme.backward
toTranslationScheme.toExternal
- Changed default value of the parameter
limit
forSaQC.interpolateIndex
andSaQC.interpolateInvalid
toNone
- Changed default value of the parameter
overwrite
forconcatFlags
toFalse
- Deprecate
transferFlags
in favor ofconcatFlags
- python 3.7 support
- Error for interpolations with limits set to be greater than 2 (
interpolateNANs
) - Error when fitting polynomials to irregularly sampled data (
fitPolynomial
)
2.2.1 - 2022-10-29
- data label to
plot
legend
dfilter
default value inference to respect the function default value ofplot
- functions not handling
target
failed to overwrite existing variables
2.2.0 - 2022-10-28
- translation of
dfilter
- new generic function
clip
- parameter
min_periods
toSaQC.flagConstants
- function
fitLowpassFilter
- tracking interpolation routines in
History
- test function interface changed to
func(saqc: SaQC, field: str | Sequence[str], *args, **kwargs)
- lib function
butterFilter
returnsNaN
for too-short series dfilter
default value precedence order
closed
keyword inflagJumps
- fixed undesired behavior in
flagIsolated
for not harmonized data - fixed failing translation of
dfilter
-defaults - fixed unbound recursion error when interpolating with order-independent methods in
interpolateIndex
- fixed not working min_periods condition if
window=None
inassignZScore
- fixed Exception occuring when fitting polynomials via
polyfit
to harmonized data, containing all-NaN gaps wider than the polynomial fitting window size. - fixed bug in function parameter checking
- fixed bug one-off bug in
flagJumps
2.1.0 - 2022-06-14
- documentation of global keywords
- generic documentation module
docurator.py
- documentation of flagging constants
pyproject.toml
- new function
progagateFlags
- include function typehints in parameter documentation
label
parameter to the generic functionisflagged
flagOffsets
parametersthresh
andthresh_relative
are optional- corrected false notion of the term residual (replace all occurences of residue by residual)
FILTER_NONE
andFILTER_ALL
are top level constants (imported insaqc.__init__
)- renamed
maskTime
toselectTime
SaQC.data
returnsdios.DictOfSeries
SaQC.flags
returnsdios.DictOfSeries
orpd.DataFrame
SaQC.data
andSaQC.flags
are not mutated by function calls- renamed
History.max
toHistory.squeeze
- renamed parameter
freq
of function flagByStray towindow
DmpScheme
: setDFILTER_DEFAULT
to 1 in order to not mask the flag 'OK'
- data accessors
SaQC.result
,SaQC.data_raw
,SaQC.flags_raw
flagOffset
failure on falsythresh
flagCrossStatistics
failure on unaligned input variablesplot
data loss when using dfilter kwargcorrectDrift
: failure on single value intervalsconcatFlags
: information loss by appending squeezed historiesinterpolateInvalid
: replace flags by interpolated valuesresample
: pass resampling function tohistory.appy()
tools.seasonalMask
: mask swapping withinclude_bounds=True
flagGeneric
:- fixed inconsistent history meta writing
- fixed handling of existing flags
proGeneeric
: fixed inconsistent history meta writingdocs
: removed documentation of data/flags parameters from automatic sphinx doc
2.0.1 - 2021-12-20
- CLI now accepts remote configuration and data files as URL
- new function
transferFlags
- improved error messages from
flagGeneric
andprocessGeneric
- new
ax_kwargs
keyword toSaQC.plot
function
- generate documentation from the
develop
branch - doctest is now ran upon push to the
develop
branch, failing doc snippets cause CI-pipeline to fail - renamed function
flagCrossStatistic
toflagCrossStatistics
- removed function
flagDriftFromScaledNorm
- removed
stats
keywords and functionality fromSaQC.plot
function
- RDM/UFZ logos:
- use the English versions of the respective images
- use full URLs instead of the repository local URLs in
README.md
- fix code snippets in
README.md
- fix version confusion
copyField
: fix misleading error messageflagGeneric
: fix failure on empty data- existing
target
variables led to function calls ontarget
instead offield
- the functions
flagDriftFromNorm
,flagDriftFromReference
,flagCrossStatistics
andflagMVScores
now properly support the field-target workflow field
was not masked for resampling functions- allow custom registered functions to overwrite built-ins.
2.0.0 - 2021-11-25
This release marks the beginning of a new release cycle. Basically the entire system got reworked between versions 1.4 and 2.0, a detailed changelog is not recoverable and/or useful.