Skip to content

Amoro-0.5.1
Compare
Choose a tag to compare
@wangtaohz wangtaohz released this 10 Oct 12:10
· 626 commits to master since this release
v0.5.1
8f554b1

Changelog for v0.5.1

Features:

  • [Feature]: Show summary details of snapshots in the table transaction tab #1827
  • [Subtask]: Implement displaying metrics on the front end of AMS #1771
  • [Feature]: Support table filters to make AMS ignore tables that are not needed #1870
  • [Feature]: Support MOR with Flink Engine in SQL/Table API #1422
  • [Feature]: Add PostgresSQL RDS as system database #1966
  • [Subtask]: Support creating mixed-iceberg format tables in any catalog which support iceberg. #1336

Imporvements:

  • [Improvement]: modify the parameter transfer method for flink optimize #1789
  • [Subtask]: Remove IcebergContentFile and related codes #1839
  • [Improvement]: Extract Spark Common Module to reuse code between different spark versions. #1449
  • [Improvement]: Show correct resource occupation for external optimizer #1845
  • [Improvement] Polish optimizing executor code #1895
  • [Improvement]: Remove AmsClient in UnkeyedTable and KeyedTable #1904
  • [Improvement]: add mysql schema init #1899
  • [Improvement]: Perform file filtering as early as possible when during optimizing plan process #1883
  • [Improvement] Add bounds of timestamp type for mixed-hive table #1920
  • [Improvement]: Flink Logstore supports 'group-offsets' and 'specific-offsets' Startup mode. #1931
  • [Improvement]: Using the parameters in TableConfiguration instead of from Table properties #1943
  • [Improvement]: Improve the triggering conditions for segment file rewriting #1953
  • [Improvement]: Helm Chart Support Overwrite JVM Optional #1962
  • [Subtask]: Do not cache DataFile in memory when evaluating tables. #1840
  • [Improvement]: Introduce a table property to set the task splitting size of self-optimizing #1935
  • doc: update dashboard readme #1984
  • [Improvement]: Flink optimizer configure managed memory explicitly in both the doc and the startup script #1907
  • [Improvement]: CI run check style first #1293
  • [Improvement]: Add basic configration of spotless #1926
  • [Improvement]: The version of the Amoro in the Docker startup script should be the latest by default #2010
  • support running ams in jdk17 environment. #1896

BugFixs:

  • [Bug]: the historical metadata file has not been deleted #1864
  • [Bug]: Segment file first binpack strategy leads to inability to optimize SplitTasks #1866
  • [Bug]: The water mark is not displayed properly for tables without primary keys #1603
  • [Bug]: Fix unit conversion for parameters -msz to spillmap configuration #1893
  • [Bug]: Snapshot file not found after Flink ingestion job has been running for a while #1882
  • [Bug]: BasicUnkeyedTable does not Override the name method #1914
  • [Bug]: Can not set configuration when setting UGI #1912
  • [Bug]: If the optimizing of a Mixed Hive table fails to alter the Hive location, the files will be moved to the old Hive location. #1898
  • [Bug]: MAJOR Optimizing is running repeatedly #1924
  • [Bug]: The optimizing task has been in a suspended state all along. #1939
  • [Bug]: Failed to persist optimizing process records #1869
  • [Bug]: Mix-iceberg format with hive metastore not working on spark #1963
  • [Bug]: Mix-iceberg format with hive metastore not working on spark session catalog #1964
  • [Bug]:When using an HDFS cluster protected by Kerberos, the fs.hdfs.impl.disable.cache=false configuration is invalid, FileSystem is unable to hit the cache #1857
  • [Bug]: When using s3a schema, orphan files could not be deleted. #1981
  • [Bug]: Using mix iceberg in spark has some docs problems #1908
  • [Bug]: Avoid NPE during disposing ArcticServiceContainer #1985
  • [Bug]: Orphan files clean mistakenly deleted the Iceberg statistic file #1959
  • [Bug]: If the iceberg table has a primary key of date type, in upsert mode, after Amoro merge, the primary key duplicate data can be found #1979
  • [Bug]: If the projected schema of the Lookup Table Source doesn't include all primary keys and requires automatic addition #1891
  • [Bug]: ams-optimizer FLINK job expired oom due to frequent requests for Kerberos. #1980
  • [Bug]: Terminal is unable to create the iceberg table, when Kerberos authentication is enabled #1996
  • [Bug] Trino failed to query Amoro Mixed Format Table after restarting AMS #1977
  • [Bug]: The startup script has a judgment error #2006
  • [Bug]: Spark SQL using amoro spark runtime error class cannot find ResolveProcedures #2032
  • [Bug]: Dashboard can't show files page with kerberos and mixed-iceberg format in external catalog #2029
  • [Bug]: When a time value that is too large or too small is written, it will encounter a 'long overflow' exception. #2044
  • [Bug]: Ams Server always executes initSqlScript when using mysql8 #2049
Assets 11
Loading