You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've tried to use your example to have checkpointing in s3 using minio. I'm using docker compose setup and what I've noticed that checkpointing doesn't really and I see following in job manager logs:
2020-06-04 19:22:19,450 INFO com.amazonaws.latency - ServiceName=[Amazon S3], AWSErrorCode=[404 Not Found], StatusCode=[404], ServiceEndpoint=[http://minio-service:9000], Exception=[com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: 16156D972F7BCCCA; S3 Extended Request ID: null), S3 Extended Request ID: null], RequestType=[GetObjectMetadataRequest], AWSRequestID=[16156D972F7BCCCA], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, Exception=1, HttpClientPoolLeasedCount=0, ClientExecuteTime=[3.172], HttpClientSendRequestTime=[0.113], HttpRequestTime=[2.379], RequestSigningTime=[0.294], CredentialsRequestTime=[0.003, 0.001], HttpClientReceiveResponseTime=[0.675],
2020-06-04 19:22:19,454 INFO com.amazonaws.latency - ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://minio-service:9000], RequestType=[ListObjectsRequest], AWSRequestID=[16156D972FA9FC2F], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[1.077], ClientExecuteTime=[3.384], HttpClientSendRequestTime=[0.104], HttpRequestTime=[1.511], RequestSigningTime=[0.289], CredentialsRequestTime=[0.001, 0.001], HttpClientReceiveResponseTime=[0.967],
2020-06-04 19:22:22,419 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 33 @ 1591298542418 for job 44a454e5e0a555403eb80ef36e2dc375.
2020-06-04 19:22:32,420 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint 33 of job 44a454e5e0a555403eb80ef36e2dc375 expired before completing.
Btw, in order to pick up the flink-conf.yaml file I had to add following to docker-compose:
Hello @alpinegizmo
I've tried to use your example to have checkpointing in s3 using minio. I'm using docker compose setup and what I've noticed that checkpointing doesn't really and I see following in job manager logs:
Btw, in order to pick up the
flink-conf.yaml
file I had to add following to docker-compose:Any pointers on the error above?
Thanks in advance
The text was updated successfully, but these errors were encountered: