feat: Handle Error in Background Jobs - Improve the way we store errors and defects in DB #1218

FabioPinheiro · 2024-06-21T16:25:56Z

Description:

ATL-6931
Note the branch name is wrong

Checklist:

My PR follows the contribution guidelines of this project
My PR is free of third-party dependencies that don't comply with the Allowlist
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
I have checked the PR title to follow the conventional commit specification

...ce/server/src/main/scala/org/hyperledger/identus/agent/server/jobs/IssueBackgroundJobs.scala

github-actions · 2024-06-21T16:30:34Z

Unit Test Results

96 files ±0 96 suites ±0 21m 52s ⏱️ -35s
826 tests ±0 818 ✅ ±0 8 💤 ±0 0 ❌ ±0
833 runs ±0 825 ✅ ±0 8 💤 ±0 0 ❌ ±0

Results for commit 26ae6a0. ± Comparison against base commit e5e3411.

♻️ This comment has been updated with latest results.

github-actions · 2024-06-24T15:44:14Z

Integration Test Results

16 files ±0 16 suites ±0 2s ⏱️ ±0s
34 tests ±0 34 ✅ ±0 0 💤 ±0 0 ❌ ±0
59 runs ±0 59 ✅ ±0 0 💤 ±0 0 ❌ ±0

Results for commit 26ae6a0. ± Comparison against base commit e5e3411.

♻️ This comment has been updated with latest results.

FabioPinheiro · 2024-06-25T10:51:24Z

@bvoiturier can you review?

shared/core/src/main/scala/org/hyperledger/identus/shared/models/Failure.scala

FabioPinheiro · 2024-06-25T10:54:31Z

pollux/sql-doobie/src/main/resources/sql/pollux/V23__clear_content_of_meta_last_failure.sql

@@ -0,0 +1,3 @@
+-- Clear content of meta_last_failure
+UPDATE public.issue_credential_records SET meta_last_failure=NULL;


I am cleanup the meta_last_failure columns like this.

wip Signed-off-by: FabioPinheiro <[email protected]> wip fix tests cleanup Fix tests scalafmt one more fix and rebase remove FixmeStatusCode scalafmt cleanup

bvoiturier

Good job @FabioPinheiro! I left some comments...

...c/main/scala/org/hyperledger/identus/agent/walletapi/model/error/DIDSecretStorageError.scala

bvoiturier · 2024-06-26T08:13:14Z

...erver/src/main/scala/org/hyperledger/identus/didcomm/controller/DIDCommControllerError.scala

-) extends Failure {
-  override val namespace = "DIDCommControllerError"
+sealed trait DIDCommControllerError extends Failure {
+  override def namespace = "DIDCommControllerError"


Any particular reason why you changed the previous construct and the below case class declaration? If not, I propose to stay consistent with the rest of the codebase.

To make final case class RecipientNotFoundError() an object

Let's keep it as a case class, then. I don't think it is worth the trouble. Who knows, maybe we will need to some kind of message ID as a parameter in the future.

bvoiturier · 2024-06-26T08:28:36Z

mercury/models/src/main/scala/org/hyperledger/identus/mercury/model/error/package.scala

+import org.hyperledger.identus.shared.models._
+
+import java.io.IOException
+
 package object error {


Do we really need all this complex error types hierarchy? It seems @CryptoKnightIOG was able to simplify it nicely in his last PR, but it was rolled back in this one. To be honest, this MercuryException|Throwable|Error construct is something I can't really wrap my head around for some time.

hum this is weird because I also remove this.
I did after making MercuryError extends Failure

It was most certainly merge conflict.

@bvoiturier That MercuryThrowable was a hackys solution to integration with the rest of the Open Enterprise Agent. That required Throwable in many places. So instead of adding Throwable in a few more places and making the problem even bigger.
I add the type alias MercuryThrowable for me to just know where I need to remove in the future. I even have a comment // TODO REMOVE Throwable

Awesome, thanks for the explanation! 👌

FabioPinheiro · 2024-06-26T10:19:51Z

Hey @CryptoKnightIOG do you want to have a look at this PR. Since you are doing ATL-6933:

The ErrorResponse stored in DB should then be exposed through the REST API for later retrieval by the user. Those should be returned as part of the connect, issue, and present flow records.

In shared/core/src/main/scala/org/hyperledger/identus/shared/models/Failure.scala I'm adding a case class FailureInfo just to read the error from the DB.
We don't have access to the error type (because we do not encode that) but we have all the necessary information to inform the user. What do you think?

Signed-off-by: FabioPinheiro <[email protected]>

# [1.37.0-snapshot.test.1](cloud-agent-v1.36.1...cloud-agent-v1.37.0-snapshot.test.1) (2024-07-01) ### Bug Fixes * avoid name and operationId conflict when creating oas client ([#1233](#1233)) ([73e8e24](73e8e24)) * incorrect parsing of public key coordinates and enable some tests again ([#1215](#1215)) ([5398a75](5398a75)) * KeyID from String to Opaque Type and presentation job cleanup ([#1190](#1190)) ([b813faf](b813faf)) * make init-script.sh idempotent [#1173](#1173) ([#1194](#1194)) ([1712062](1712062)) * migrate and repair in case of renaming issues ([#1211](#1211)) ([cb4d479](cb4d479)) * Present Error Handling (Part 1: Repo Changes) ([#1172](#1172)) ([13e2447](13e2447)) * Present Error Handling Job ([#1204](#1204)) ([3191d8b](3191d8b)) * present error handling Part 2 ([#1177](#1177)) ([9ac6e52](9ac6e52)) * Verification Policy Error Handling ([#1228](#1228)) ([6117a3c](6117a3c)) ### Features * add oidc4vci protocol MVP ([#1182](#1182)) ([3ae91dc](3ae91dc)) * add schemaId to the issuance session ([#1199](#1199)) ([97f5d83](97f5d83)) * add support for EcdsaSecp256k1Signature2019Proof and fix pk encoding for EddsaJcs2022Proof ([#1127](#1127)) ([e617ded](e617ded)) * **agent:** add browser fingerprint label to http metrics ([#1231](#1231)) ([f090554](f090554)) * ATL 6829 - Integrate ZIO failures and defects ADR in credential status list ([#1175](#1175)) ([dffad1d](dffad1d)) * ATL-6832 ZIO failures and defects in entity controller ([#1203](#1203)) ([9050094](9050094)) * ATL-6833 integrate ZIO failures and defects in wallet event controller ([#1186](#1186)) ([8bc2018](8bc2018)) * ATL-6834 Use ZIO Failures and Defects effectively in the Issue flow ([#1139](#1139)) ([ede7b77](ede7b77)) * Handle Error in Background Jobs - Improve the way we store errors and defects in DB ([#1218](#1218)) ([e3cadc9](e3cadc9)) * implement ADR Use ZIO Failures and Defects Effectively - Mercury should not throw exceptions ([#1192](#1192)) ([a4ce87f](a4ce87f)) * improve OpenAPI's PresentProof to make more consistent ([#1130](#1130)) ([bdc5d20](bdc5d20)) * Integrate ZIO failures and defects ADR in DID Registrar Controller and Mercury ([#1180](#1180)) ([d8e2120](d8e2120)) * SDJWT holder key binding ([#1185](#1185)) ([628f2f0](628f2f0)) * use kid in the jwt proof header of OID4VCI CredentialIssue request ([#1184](#1184)) ([ee53eda](ee53eda)) * use the compact format in SD-JWT ([#1169](#1169)) ([65da651](65da651)) Signed-off-by: Allain Magyar <[email protected]>

# [1.37.0-snapshot.test.1](cloud-agent-v1.36.1...cloud-agent-v1.37.0-snapshot.test.1) (2024-07-01) ### Bug Fixes * avoid name and operationId conflict when creating oas client ([#1233](#1233)) ([73e8e24](73e8e24)) * incorrect parsing of public key coordinates and enable some tests again ([#1215](#1215)) ([5398a75](5398a75)) * KeyID from String to Opaque Type and presentation job cleanup ([#1190](#1190)) ([b813faf](b813faf)) * make init-script.sh idempotent [#1173](#1173) ([#1194](#1194)) ([1712062](1712062)) * migrate and repair in case of renaming issues ([#1211](#1211)) ([cb4d479](cb4d479)) * Present Error Handling (Part 1: Repo Changes) ([#1172](#1172)) ([13e2447](13e2447)) * Present Error Handling Job ([#1204](#1204)) ([3191d8b](3191d8b)) * present error handling Part 2 ([#1177](#1177)) ([9ac6e52](9ac6e52)) * Verification Policy Error Handling ([#1228](#1228)) ([6117a3c](6117a3c)) ### Features * add oidc4vci protocol MVP ([#1182](#1182)) ([3ae91dc](3ae91dc)) * add schemaId to the issuance session ([#1199](#1199)) ([97f5d83](97f5d83)) * add support for EcdsaSecp256k1Signature2019Proof and fix pk encoding for EddsaJcs2022Proof ([#1127](#1127)) ([e617ded](e617ded)) * **agent:** add browser fingerprint label to http metrics ([#1231](#1231)) ([f090554](f090554)) * ATL 6829 - Integrate ZIO failures and defects ADR in credential status list ([#1175](#1175)) ([dffad1d](dffad1d)) * ATL-6832 ZIO failures and defects in entity controller ([#1203](#1203)) ([9050094](9050094)) * ATL-6833 integrate ZIO failures and defects in wallet event controller ([#1186](#1186)) ([8bc2018](8bc2018)) * ATL-6834 Use ZIO Failures and Defects effectively in the Issue flow ([#1139](#1139)) ([ede7b77](ede7b77)) * Handle Error in Background Jobs - Improve the way we store errors and defects in DB ([#1218](#1218)) ([e3cadc9](e3cadc9)) * implement ADR Use ZIO Failures and Defects Effectively - Mercury should not throw exceptions ([#1192](#1192)) ([a4ce87f](a4ce87f)) * improve OpenAPI's PresentProof to make more consistent ([#1130](#1130)) ([bdc5d20](bdc5d20)) * Integrate ZIO failures and defects ADR in DID Registrar Controller and Mercury ([#1180](#1180)) ([d8e2120](d8e2120)) * SDJWT holder key binding ([#1185](#1185)) ([628f2f0](628f2f0)) * use kid in the jwt proof header of OID4VCI CredentialIssue request ([#1184](#1184)) ([ee53eda](ee53eda)) * use the compact format in SD-JWT ([#1169](#1169)) ([65da651](65da651)) Signed-off-by: Allain Magyar <[email protected]> Signed-off-by: Pat Losoponkul <[email protected]>

Signed-off-by: Pat Losoponkul <[email protected]> test: remove println Signed-off-by: Pat Losoponkul <[email protected]> chore(release): cut Identus Cloud agent 1.37.0-snapshot.test.1 release * avoid name and operationId conflict when creating oas client ([#1233](#1233)) ([73e8e24](73e8e24)) * incorrect parsing of public key coordinates and enable some tests again ([#1215](#1215)) ([5398a75](5398a75)) * KeyID from String to Opaque Type and presentation job cleanup ([#1190](#1190)) ([b813faf](b813faf)) * make init-script.sh idempotent [#1173](#1173) ([#1194](#1194)) ([1712062](1712062)) * migrate and repair in case of renaming issues ([#1211](#1211)) ([cb4d479](cb4d479)) * Present Error Handling (Part 1: Repo Changes) ([#1172](#1172)) ([13e2447](13e2447)) * Present Error Handling Job ([#1204](#1204)) ([3191d8b](3191d8b)) * present error handling Part 2 ([#1177](#1177)) ([9ac6e52](9ac6e52)) * Verification Policy Error Handling ([#1228](#1228)) ([6117a3c](6117a3c)) * add oidc4vci protocol MVP ([#1182](#1182)) ([3ae91dc](3ae91dc)) * add schemaId to the issuance session ([#1199](#1199)) ([97f5d83](97f5d83)) * add support for EcdsaSecp256k1Signature2019Proof and fix pk encoding for EddsaJcs2022Proof ([#1127](#1127)) ([e617ded](e617ded)) * **agent:** add browser fingerprint label to http metrics ([#1231](#1231)) ([f090554](f090554)) * ATL 6829 - Integrate ZIO failures and defects ADR in credential status list ([#1175](#1175)) ([dffad1d](dffad1d)) * ATL-6832 ZIO failures and defects in entity controller ([#1203](#1203)) ([9050094](9050094)) * ATL-6833 integrate ZIO failures and defects in wallet event controller ([#1186](#1186)) ([8bc2018](8bc2018)) * ATL-6834 Use ZIO Failures and Defects effectively in the Issue flow ([#1139](#1139)) ([ede7b77](ede7b77)) * Handle Error in Background Jobs - Improve the way we store errors and defects in DB ([#1218](#1218)) ([e3cadc9](e3cadc9)) * implement ADR Use ZIO Failures and Defects Effectively - Mercury should not throw exceptions ([#1192](#1192)) ([a4ce87f](a4ce87f)) * improve OpenAPI's PresentProof to make more consistent ([#1130](#1130)) ([bdc5d20](bdc5d20)) * Integrate ZIO failures and defects ADR in DID Registrar Controller and Mercury ([#1180](#1180)) ([d8e2120](d8e2120)) * SDJWT holder key binding ([#1185](#1185)) ([628f2f0](628f2f0)) * use kid in the jwt proof header of OID4VCI CredentialIssue request ([#1184](#1184)) ([ee53eda](ee53eda)) * use the compact format in SD-JWT ([#1169](#1169)) ([65da651](65da651)) Signed-off-by: Allain Magyar <[email protected]> Signed-off-by: Pat Losoponkul <[email protected]>

# [1.37.0](cloud-agent-v1.36.1...cloud-agent-v1.37.0) (2024-07-01) ### Bug Fixes * avoid name and operationId conflict when creating oas client ([#1233](#1233)) ([73e8e24](73e8e24)) * incorrect parsing of public key coordinates and enable some tests again ([#1215](#1215)) ([5398a75](5398a75)) * KeyID from String to Opaque Type and presentation job cleanup ([#1190](#1190)) ([b813faf](b813faf)) * make init-script.sh idempotent [#1173](#1173) ([#1194](#1194)) ([1712062](1712062)) * migrate and repair in case of renaming issues ([#1211](#1211)) ([cb4d479](cb4d479)) * Present Error Handling (Part 1: Repo Changes) ([#1172](#1172)) ([13e2447](13e2447)) * Present Error Handling Job ([#1204](#1204)) ([3191d8b](3191d8b)) * present error handling Part 2 ([#1177](#1177)) ([9ac6e52](9ac6e52)) * Verification Policy Error Handling ([#1228](#1228)) ([6117a3c](6117a3c)) ### Features * add oidc4vci protocol MVP ([#1182](#1182)) ([3ae91dc](3ae91dc)) * add schemaId to the issuance session ([#1199](#1199)) ([97f5d83](97f5d83)) * add support for EcdsaSecp256k1Signature2019Proof and fix pk encoding for EddsaJcs2022Proof ([#1127](#1127)) ([e617ded](e617ded)) * **agent:** add browser fingerprint label to http metrics ([#1231](#1231)) ([f090554](f090554)) * ATL 6829 - Integrate ZIO failures and defects ADR in credential status list ([#1175](#1175)) ([dffad1d](dffad1d)) * ATL-6832 ZIO failures and defects in entity controller ([#1203](#1203)) ([9050094](9050094)) * ATL-6833 integrate ZIO failures and defects in wallet event controller ([#1186](#1186)) ([8bc2018](8bc2018)) * ATL-6834 Use ZIO Failures and Defects effectively in the Issue flow ([#1139](#1139)) ([ede7b77](ede7b77)) * Handle Error in Background Jobs - Improve the way we store errors and defects in DB ([#1218](#1218)) ([e3cadc9](e3cadc9)) * implement ADR Use ZIO Failures and Defects Effectively - Mercury should not throw exceptions ([#1192](#1192)) ([a4ce87f](a4ce87f)) * improve OpenAPI's PresentProof to make more consistent ([#1130](#1130)) ([bdc5d20](bdc5d20)) * Integrate ZIO failures and defects ADR in DID Registrar Controller and Mercury ([#1180](#1180)) ([d8e2120](d8e2120)) * SDJWT holder key binding ([#1185](#1185)) ([628f2f0](628f2f0)) * use kid in the jwt proof header of OID4VCI CredentialIssue request ([#1184](#1184)) ([ee53eda](ee53eda)) * use the compact format in SD-JWT ([#1169](#1169)) ([65da651](65da651)) Signed-off-by: Allain Magyar <[email protected]>

FabioPinheiro requested review from patlo-iog, mineme0110, yshyn-iohk, bvoiturier, shotexa and CryptoKnightIOG as code owners June 21, 2024 16:25

FabioPinheiro marked this pull request as draft June 21, 2024 16:26

github-actions bot added pollux build shared mercury connect cloud-agent labels Jun 21, 2024

FabioPinheiro force-pushed the feat/ATL-6932 branch from 3b9637b to 4d42c6f Compare June 21, 2024 16:28

FabioPinheiro commented Jun 21, 2024

View reviewed changes

...ce/server/src/main/scala/org/hyperledger/identus/agent/server/jobs/IssueBackgroundJobs.scala Outdated Show resolved Hide resolved

yshyn-iohk approved these changes Jun 24, 2024

View reviewed changes

FabioPinheiro force-pushed the feat/ATL-6932 branch from 4d42c6f to 90395eb Compare June 24, 2024 11:54

FabioPinheiro marked this pull request as ready for review June 24, 2024 16:42

FabioPinheiro force-pushed the feat/ATL-6932 branch 2 times, most recently from b13b4dc to 59b30d0 Compare June 25, 2024 10:50

FabioPinheiro commented Jun 25, 2024

View reviewed changes

shared/core/src/main/scala/org/hyperledger/identus/shared/models/Failure.scala Outdated Show resolved Hide resolved

FabioPinheiro commented Jun 25, 2024

View reviewed changes

ATL-6931

eedcaa7

wip Signed-off-by: FabioPinheiro <[email protected]> wip fix tests cleanup Fix tests scalafmt one more fix and rebase remove FixmeStatusCode scalafmt cleanup

FabioPinheiro force-pushed the feat/ATL-6932 branch from b5ce26d to eedcaa7 Compare June 25, 2024 12:33

bvoiturier reviewed Jun 26, 2024

View reviewed changes

Fixes after Review

25079db

Signed-off-by: FabioPinheiro <[email protected]>

FabioPinheiro force-pushed the feat/ATL-6932 branch from 417e6e8 to 25079db Compare June 26, 2024 10:20

FabioPinheiro enabled auto-merge (squash) June 27, 2024 10:43

Merge branch 'main' into feat/ATL-6932

26ae6a0

FabioPinheiro merged commit e3cadc9 into main Jun 27, 2024
12 checks passed

FabioPinheiro deleted the feat/ATL-6932 branch June 27, 2024 13:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Handle Error in Background Jobs - Improve the way we store errors and defects in DB #1218

feat: Handle Error in Background Jobs - Improve the way we store errors and defects in DB #1218

FabioPinheiro commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 24, 2024 •

edited

Loading

FabioPinheiro commented Jun 25, 2024

FabioPinheiro Jun 25, 2024 •

edited

Loading

bvoiturier left a comment

bvoiturier Jun 26, 2024

FabioPinheiro Jun 26, 2024

bvoiturier Jun 26, 2024

bvoiturier Jun 26, 2024

FabioPinheiro Jun 26, 2024

bvoiturier Jun 26, 2024

FabioPinheiro commented Jun 26, 2024

		@@ -0,0 +1,3 @@
		-- Clear content of meta_last_failure
		UPDATE public.issue_credential_records SET meta_last_failure=NULL;

feat: Handle Error in Background Jobs - Improve the way we store errors and defects in DB #1218

feat: Handle Error in Background Jobs - Improve the way we store errors and defects in DB #1218

Conversation

FabioPinheiro commented Jun 21, 2024 • edited Loading

Description:

Checklist:

github-actions bot commented Jun 21, 2024 • edited Loading

Unit Test Results

github-actions bot commented Jun 24, 2024 • edited Loading

Integration Test Results

FabioPinheiro commented Jun 25, 2024

FabioPinheiro Jun 25, 2024 • edited Loading

Choose a reason for hiding this comment

bvoiturier left a comment

Choose a reason for hiding this comment

bvoiturier Jun 26, 2024

Choose a reason for hiding this comment

FabioPinheiro Jun 26, 2024

Choose a reason for hiding this comment

bvoiturier Jun 26, 2024

Choose a reason for hiding this comment

bvoiturier Jun 26, 2024

Choose a reason for hiding this comment

FabioPinheiro Jun 26, 2024

Choose a reason for hiding this comment

bvoiturier Jun 26, 2024

Choose a reason for hiding this comment

FabioPinheiro commented Jun 26, 2024

FabioPinheiro commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 24, 2024 •

edited

Loading

FabioPinheiro Jun 25, 2024 •

edited

Loading