Backwards Compatibility Requirements for Jakarta EE Specifications is Ambiguous wrt to deprecation and removal of features

[mkarg]

Describe the bug
The document "Backwards Compatibility Requirements for Jakarta EE Specifications" is ambiguous wrt to deprecation and removal of features:

• In Chapter 6 it allows deprecation and removal:

  A specification project may decide to designate a method or interface as being “deprecated” when its use is no longer recommended or when the specification project considers that a different interface or method should be used instead. This can be stated in the specification and in the relevant javadoc.

• In the FAQ it forbids deprecation and removal:

  Q2. Is it permissible to remove an existing interface from an existing specification?
  No, because this would violate rules 2 and 3 above. Note that it is also not permissible to mark it as deprecated.

  Q4. Is it permissible to remove an existing method from an existing interface?
  No, because this would violates rules 2 and 3 above.

As a result, (at least) the Jakarta REST committers and the Jakarta JSON-B committers need urgently a decision if they MAY or MAY NOT deprecate and / or remove features in their next major release. Both teams currently plan / discuss deprecations / removals and are stuck without an official clarification of this question.

Expected behavior
The document shall unambiguously clarify in what cases / under what conditions deprecation and removal are allowed, or clearly say that it is never allowed to deprecate / remove features.

Proposal
What many people told me what they expect is to align our rules with SemVer.

This could be achieved by an agreement like this:

• Removal or incompatible change of features may ONLY happen in MAJOR releases, and only if the previous major or minor release deprecated that feature.
  @deprecate(forRemoval = true) MUST be used for removal, as the application programmer MUST have a clear and unambiguous warning regarding the upcoming removal.
• Compatible change of features, and addition of features, may ONLY happen in MINOR and MAJOR releases.
  @deprecate(forRemoval = false) MAY be used (as there is no essential need to forbid it), but SHOULD NOT get used (as it is noisy).
• As a result, PATCH releases MUST NOT add, change, deprecate or remove features.

AB#46

[mkarg]

@ivargrimstad I kindly like to ask you to put this issue on the agenda of the next meeting.

[ivargrimstad]

@ivargrimstad I kindly like to ask you to put this issue on the agenda of the next meeting.

It was discussed in the Platform Call on January 31. It was decided to seek input from the Jakarta EE Specification Committee, which is currently working on updating the documentation on how to handle backward compatibility.

I will make them aware of the urgency, so you can do your planning accordingly.

For the upcoming plan due May 30th, you may assume SemVer. The very strict rules in the original document and FAQ is most likely going to be relaxed.

[starksm64]

From still unpublished spec committee meeting minutes of Apr 19, May 5, there was a general agreement that backwards incompatible changes are to be allowed, the current Java EE legacy doc "Backwards Compatibility Requirements for Jakarta EE Specifications" is completely out of date and will be replaced with something based on Semantic versioning with deprecation of features to be removed or changed in a backwards incompatible way.

It is guaranteed you will be able to deprecate features for removal. What has not been determined is if you can deprecate in one release and remove in the next. I think we will push for the ability to deprecate and then remove features after one major release, but that has not been ratified as yet.

[quiram]

From @ivargrimstad

For the upcoming plan due May 30th, you may assume SemVer.

I'm not sure this is correct, I thought we had agreed not to follow SemVer, at least not to the letter. The reason for this is that Jakarta EE's major version has some marketing connotations and we cannot guarantee that a new major version will always imply an incompatible API change like SemVer suggests. This requirement could get particularly gnarly when taking into account that different profiles include different specs, and therefore that a new version of Jakarta EE may imply braking changes in one profile but not in another: applying SemVer strictly would mean that the major version of different profiles evolve independently, which is undesired.

[ivargrimstad]

From @ivargrimstad

For the upcoming plan due May 30th, you may assume SemVer.

I'm not sure this is correct, I thought we had agreed not to follow SemVer, at least not to the letter. The reason for this is that Jakarta EE's major version has some marketing connotations and we cannot guarantee that a new major version will always imply an incompatible API change like SemVer suggests. This requirement could get particularly gnarly when taking into account that different profiles include different specs, and therefore that a new version of Jakarta EE may imply braking changes in one profile but not in another: applying SemVer strictly would mean that the major version of different profiles evolve independently, which is undesired.

SemVer will only be applied in some form for the individual component specifications. Not for the Platform, Web Profile, and Core Profile. As you say, they are more marketing related and will be numbered accordingly.

[mkarg]

It seems there is a misunderstanding in SemVer. We can safely apply it at the Platform level, as SemVer is unidirectional in its effects chain:

• A breaking change enforces a major release.
• A major release does not enforce the existence of any change, neither breaking nor non-breaking.

[ivargrimstad]

It seems there is a misunderstanding in SemVer. We can safely apply it at the Platform level, as SemVer is unidirectional in its effects chain:

• A breaking change enforces a major release.
• A major release does not enforce the existence of any change, neither breaking nor non-breaking.

Thanks, @mkarg!

[quiram]

It seems there is a misunderstanding in SemVer. We can safely apply it at the Platform level, as SemVer is unidirectional in its effects chain:

• A breaking change enforces a major release.
• A major release does not enforce the existence of any change, neither breaking nor non-breaking.

Thanks @mkarg, this is great but, just to be on the safe side, could you please point at the part on SemVer spec that specifies this? I tried to find this myself and couldn't. My concern is that the current wording of SemVer may be open to interpretation, which could lead to a few gotchas if we officially say that we are using it at Platform level.

Personally (and this is where interpretation comes in), if SemVer is meant to "convey meaning about the underlying code", then the relationship between major version and breaking changes seems bidirectional to me; without specific text to say the opposite, that's how I would understand SemVer.

[mkarg]

I completely respect your opinion and interpretation, but please understand that people daily engaged with specs and legal texts have a common understanding, that is totally different from your personal opinion.

You do not need explicit wording for this, as it is common sense in technical specs and mathematical, psychologic and legal texts that a ⇒ b does neither imply a ⇐ b nor a ⇔ b, so an explicit exclusion of a ⇐ b and a ⇔ b is not needed and would even be semantically redundant. So we are on the safe side already. That common sense is the reason why you do not find any clarification on neither a ⇐ b nor a ⇔ b in the SemVer spec.

Edit: I asked a question at SemVer to get the autoritative answer you demand. :-)

[ivargrimstad]

The Specification Committee is working on composing a deprecation process to ensure that backward incompatible changes can be done in a controlled manner. Conceptually, this is where we are headed:

The deprecation process that is being worked on broadly has two steps:

1. A feature is marked as deprecated in release N (N can be a MINOR release)
2. A feature is then removed in release N+1 (or beyond where N+1 is a MAJOR release)

More details and descriptions will follow, but this is basically the concept.

Ivar, on behalf of the Jakarta EE Specification Committee

[mkarg]

@quiram So here is the definitive authoritative answer of the SemVer team: semver/semver#946 (comment). Hence we MAY increase major version without any actual need. SemVer does not prohibit that.

[quiram]

Apologies for late reply, I’ve been busy with work.

@mkarg the comment that you linked to says:

I'd argue it violates the spirit of semver, and is a gross and icky thing to do, but neither of those are definitive/objective nor answered by the spec.

What I take from this is thar there is no “official” answer and that, if anything, the unofficial recommendation is not to do it for it would be a “gross and icky thing to do”.

It seems the crux of the matter here is interpretation. I tried to gather how the wider community feels about this topic so I asked the LJC to run a little
poll. IMHO there are too few results to take this with any seriousness, but it may serve as anecdotic evidence that there isn’t a clear-cut, universal understanding of what a new major version conveys.

With this, and in order to prevent any potential misunderstandings, I still think the safest course of action is to say that Jakarta EE as a whole does not follow SemVer, although individual specs might.

PS: I must confess I found your comment rather patronising. “Common sense” is subjective. I never “demanded” anything. I kindly request that you use a more conciliatory tone so as to not discourage participation.

[mkarg]

Unfortunately your poll was vaguely phrased. If you need a clear answer then the question must be unambiguous and closed, like "Does SemVer forbid increasing MAJOR in case there is no breaking change?", so you solely receive clear YES/NO answers.

I do not share your interpretation. Let's agree to disagree on SemVer. Here is my view, and I think we can close the topic with this, as everything is already said and we should not repeat the same arguments:

• IMHO the answer is official (Edit: and unambiguous), as we had been told that the authors of the comments are officials.
  Edit: It is clearlay told so in that part of the comment that you stripped from your quote:

  definitively, it would not definitely violate semver, per two Semver Spec team members above.

• "Neither of those are ... objective" IMHO means that doing so does not violate the spec. The comment means "we don't like it, but it is ok".

Independent of your or mine interpretation, we will not get a better answer than that: The SemVer spec does not forbid it, so it is allowed, even if not appreciated.

I am really sorry if you found any tone at all in my comments. I am not a native English speaker and actually do not have the ability to select any particular tone intentionally, I just write as good as I can. Sorry for that. My comments were not intended anything other than clarifying the situation. I do not want to make you feel bad or dictate the outcome of the discussion. BTW, in fact I need to say that demanding such high English skills to be able to select particular nuances in the tone makes me feel like getting segregated, and makes me feel like discouraged from further participation. As you see, this medal has two sides, so I kindly request you do not read between the lines and do not feel easily offended, as it makes others feel bad and discouraged in turn. In FIDO times we had a saying like "Do not offend others. Do not feel offended by others." which I think is a good agreement and expresses the two sides of the medal.

I do really not discourage participation. But I discourage picking on issues that apparently already have been discussed, clarified and decided by the particular committee. They already haven seen your point and heared you, but they decided in a way you do not shore. Ok. I am sorry if you have a different view and I absolutely respect that view, but apparently others do not share your view. Also please understand that after more than three decades in the software industry I do have gathered enough experience and knowledge to say that "Common sense" is not subjective, but it is not inclusive and not absolute. This means "Common sense" might only be shared by a small majority, but it is the majority. Even if you find many people sharing your view, it is not the majority's view, as your poll proofs.

Edit: The particular use of "Common Sense" by me was on how maths and psychology uses the "follows" operator. You can simply check maths and psychology books and will find proof of my claim. Hence that particular comment of my is even academically proven.

[quiram]

@mkarg you are being needlessly adversarial, I am trying to avoid conflict, you are trying to win conflict. I was asked to participate in representation of the community, but I'm a volunteer here and I'm not keen on spending my free time dealing with direct confrontation.

You say that the decision on SemVer has already been made, can you please specify where and when? The last news that I have on this matter (and I appreciate that there could be additional decisions that I'm not aware of) was at the Specs Committee, where it was still being debated. The last agenda suggests that a decision on this has not been made, and that there are ideas still being floated (I will not copy any content so as to avoid being accused of "stripping quotes" again, you can read the document yourself); I could not attend the last meeting and the minutes haven't been published yet, so if a decision was made during that meeting then I have missed it.

[mkarg]

There is no conflict, so I cannot trying to win it; I already explained what my targets are and will not repeat, as we come into a loop simply, which nobody has a benefit of. Neither am I "adversarial" (in fact I learned that word just now from you), I am just clear and direct and have a strong opinion. I do not "accuse" you of anything, I just reported the fact that you left out a part of the quote that was essential. I am sorry if you feel attacked, but it really is just perception, not intent. Your contribution is appreciated, I already said that. So can we please end this "tone" thread in this public forum, as it leads to nothing and is actually OT? Feel free to continue by PM if you think we are still not good. Thanks.

The decision of the committee is documented in the above history:

SemVer will only be applied in some form for the individual component specifications. Not for the Platform, Web Profile, and Core Profile.

[quiram]

That's what I was advocating for, we've been having an argument for no reason.

[edburns]

Hello @mkarg nice to see you again. As it happens, the EE Specification committee is hard at work revising the backward compatibility requirements right now. I expect they'll have a process on this in 2023-07 or 2023-08. I suggest you take a close look at that when it comes out and we can decide what to do with this particular issue.

[mkarg]

Hello @mkarg nice to see you again. As it happens, the EE Specification committee is hard at work revising the backward compatibility requirements right now. I expect they'll have a process on this in 2023-07 or 2023-08. I suggest you take a close look at that when it comes out and we can decide what to do with this particular issue.

Hello Ed, thank you for the update! :-)