Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-43284: [Release] Fix version detection timing for bump deb package names on post-12-bump-versions.sh script #43294

Merged
merged 3 commits into from
Jul 20, 2024

Conversation

raulcd
Copy link
Member

@raulcd raulcd commented Jul 17, 2024

Rationale for this change

The script fails to bump deb package names at the moment due to an unmatching if.

What changes are included in this PR?

Use current_version before bumping the versions in order to match debian packages to be updated.

Are these changes tested?

It has been tested locally to bump the versions for the debian package names on main for 17.0.0

Are there any user-facing changes?

No

Copy link

⚠️ GitHub issue #43284 has been automatically assigned in GitHub to PR creator.

@github-actions github-actions bot added the awaiting committer review Awaiting committer review label Jul 17, 2024
@raulcd
Copy link
Member Author

raulcd commented Jul 17, 2024

I am not sure why update_deb_package_names is failing on the git mv here: https://github.com/apache/arrow/blob/main/dev/release/utils-prepare.sh#L239-L241
I was doing some tests locally but I don't understand what the issue is:

+ git mv 'debian*/lib*1701.install' debian/libarrow1800.install debian/libarrow-acero1800.install debian/libarrow-cuda1800.install debian/libarrow-cuda-glib1800.install debian/libarrow-dataset1800.install debian/libarrow-dataset-glib1800.install debian/libarrow-flight1800.install debian/libarrow-flight-glib1800.install debian/libarrow-flight-sql1800.install debian/libarrow-flight-sql-glib1800.install debian/libarrow-glib1800.install debian/libgandiva1800.install debian/libgandiva-glib1800.install debian/libparquet1800.install debian/libparquet-glib1800.install
fatal: destination 'dev/tasks/linux-packages/apache-arrow/debian/libparquet-glib1800.install' is not a directory

cc @kou

@@ -64,7 +64,7 @@ if [ ${BUMP_VERSION_POST_TAG} -gt 0 ]; then
fi

if [ ${BUMP_DEB_PACKAGE_NAMES} -gt 0 ] && \
[ "${next_version}" != "$(current_version)" ]; then
[ "${next_version}" != "${version}" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm.
$(current_version) is a function call not a variable reference.
So current_version function call is intentional here.

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Jul 17, 2024
@kou
Copy link
Member

kou commented Jul 17, 2024

I am not sure why update_deb_package_names is failing on the git mv here: https://github.com/apache/arrow/blob/main/dev/release/utils-prepare.sh#L239-L241 I was doing some tests locally but I don't understand what the issue is:

+ git mv 'debian*/lib*1701.install' debian/libarrow1800.install debian/libarrow-acero1800.install debian/libarrow-cuda1800.install debian/libarrow-cuda-glib1800.install debian/libarrow-dataset1800.install debian/libarrow-dataset-glib1800.install debian/libarrow-flight1800.install debian/libarrow-flight-glib1800.install debian/libarrow-flight-sql1800.install debian/libarrow-flight-sql-glib1800.install debian/libarrow-glib1800.install debian/libgandiva1800.install debian/libgandiva-glib1800.install debian/libparquet1800.install debian/libparquet-glib1800.install
fatal: destination 'dev/tasks/linux-packages/apache-arrow/debian/libparquet-glib1800.install' is not a directory

This is happen because there are no files that are matched against debian*/lib*1701.install.

The pattern at

for target in debian*/lib*${deb_lib_suffix}.install; do
is used as-is for git mv. So
local version=$1
value is strange.

@kou
Copy link
Member

kou commented Jul 17, 2024

Could you share the command line you tried?
I want to try it too.

@kou
Copy link
Member

kou commented Jul 17, 2024

Ah, we need to run current_version before we call update_versions:

diff --git a/dev/release/post-12-bump-versions.sh b/dev/release/post-12-bump-versions.sh
index 422821a66b..206da6ccb4 100755
--- a/dev/release/post-12-bump-versions.sh
+++ b/dev/release/post-12-bump-versions.sh
@@ -41,6 +41,8 @@ version=$1
 next_version=$2
 next_version_snapshot="${next_version}-SNAPSHOT"
 
+current_version_before_bump="$(current_version)"
+
 case "${version}" in
   *.0.0)
     is_major_release=1
@@ -64,7 +66,7 @@ if [ ${BUMP_VERSION_POST_TAG} -gt 0 ]; then
 fi
 
 if [ ${BUMP_DEB_PACKAGE_NAMES} -gt 0 ] && \
-     [ "${next_version}" != "$(current_version)" ]; then
+     [ "${next_version}" != "${current_version_before_bump}" ]; then
   update_deb_package_names "${version}" "${next_version}"
 fi
 

@raulcd
Copy link
Member Author

raulcd commented Jul 18, 2024

Could you share the command line you tried? I want to try it too.

I was just trying what the test was doing on the branch:

BUMP_DEFAULT=0  BUMP_DEB_PACKAGE_NAMES=1 dev/release/post-12-bump-versions.sh 17.1.0 18.0.0

I also added the set -x to the post-12-bump-versions.sh:

diff --git a/dev/release/post-12-bump-versions.sh b/dev/release/post-12-bump-versions.sh
index 2eeb715..5ffc7d3 100755
--- a/dev/release/post-12-bump-versions.sh
+++ b/dev/release/post-12-bump-versions.sh
@@ -17,7 +17,7 @@
 # specific language governing permissions and limitations
 # under the License.
 #
-set -ue
+set -uex
 
 SOURCE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 source "${SOURCE_DIR}/git-vars.sh"

@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jul 18, 2024
@raulcd raulcd changed the title GH-43284: [Release] Fix variable name on post-12-bump-versions.sh script GH-43284: [Release] Fix bump deb package names on post-12-bump-versions.sh script Jul 18, 2024
@kou
Copy link
Member

kou commented Jul 19, 2024

(I'm writing a test for this case...)

@kou
Copy link
Member

kou commented Jul 19, 2024

Could you apply this?

diff --git a/dev/release/post-12-bump-versions-test.rb b/dev/release/post-12-bump-versions-test.rb
index 2bd1458746..f31e1a3122 100644
--- a/dev/release/post-12-bump-versions-test.rb
+++ b/dev/release/post-12-bump-versions-test.rb
@@ -358,8 +358,15 @@ class PostBumpVersionsTest < Test::Unit::TestCase
   def test_deb_package_names
     omit_on_release_branch unless bump_type.nil?
     current_commit = git_current_commit
-    stdout = bump_versions("DEB_PACKAGE_NAMES")
-    changes = parse_patch(git("log", "-p", "#{current_commit}.."))
+    stdout = bump_versions("VERSION_POST_TAG", "DEB_PACKAGE_NAMES")
+    log = git("log", "-p", "#{current_commit}..")
+    # Remove a commit for VERSION_POST_TAG
+    if log.scan(/^commit/).size == 1
+      log = ""
+    else
+      log.gsub!(/\A(commit.*?)^commit .*\z/um, "\\1")
+    end
+    changes = parse_patch(log)
     sampled_changes = changes.collect do |change|
       first_hunk = change[:hunks][0]
       first_removed_line = first_hunk.find { |line| line.start_with?("-") }

Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Jul 20, 2024
@kou kou changed the title GH-43284: [Release] Fix bump deb package names on post-12-bump-versions.sh script GH-43284: [Release] Fix version detection timing for bump deb package names on post-12-bump-versions.sh script Jul 20, 2024
@github-actions github-actions bot added the awaiting review Awaiting review label Jul 20, 2024
@kou kou merged commit fe51029 into apache:main Jul 20, 2024
8 checks passed
@kou kou removed awaiting review Awaiting review awaiting merge Awaiting merge labels Jul 20, 2024
Copy link

After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit fe51029.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants