pk_chunking is not working

[hrishy]

Hi

I tried the example in the documentation

job = bulk.create_query_job("Contact", contentType='CSV', pk_chunking=True)
batch = bulk.query(job, "select Id,LastName from Contact")
bulk.close_job(job)
while not bulk.is_batch_done(batch):
    sleep(10)

for result in bulk.get_all_results_for_query_batch(batch):
    reader = unicodecsv.DictReader(result, encoding='utf-8')
    for row in reader:
        print(row) # dictionary rows

BulkBatchFailed Traceback (most recent call last)
in
8 batch = bulk.query(job, "select Id,LastName from Contact")
9 bulk.close_job(job)
---> 10 while not bulk.is_batch_done(batch):
11 sleep(10)
12

c:\python37\lib\site-packages\salesforce_bulk\salesforce_bulk.py in is_batch_done(self, batch_id, job_id)
426 if batch_state in bulk_states.ERROR_STATES:
427 status = self.batch_status(batch_id, job_id)
--> 428 raise BulkBatchFailed(job_id, batch_id, status.get('stateMessage'), batch_state)
429 return batch_state == bulk_states.COMPLETED
430

BulkBatchFailed: Batch 123456789 of job None failed: None

Without pk_chunking it works though.
Any ideas what am i doing wrong here any help would be appreciated

[kunal4422]

Hi @hrishy did u find solution to this?

[hrishy]

Pk_chunking actually refers to chunk_size and the default value is 2048 I used vs code ide to figure this out

…

On Mon 23 Dec, 2019, 11:48 AM kunal4422, ***@***.***> wrote: Hi @hrishy <https://github.com/hrishy> did u find solution to this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#74?email_source=notifications&email_token=AJCOEWPWM2BQO44YHTPOJJDQ2BJ3FA5CNFSM4IO3G36KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHQKJXA#issuecomment-568370396>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJCOEWOA2IAUEY6WJOIH6GLQ2BJ3FANCNFSM4IO3G36A> .

[kunal4422]

@hrishy didnt got you. I am having same code as you and also same error. Instead of pk_chunking=True i tried to give pk_chunking=100000; as given in example. What exactly should i do to rectify this error?
Somewhere we have to use get_batch_items() for iterating between batches. How did u solve it?

[hrishy]

Hi My apologies I don't have acesses to my code now but I do remember vaguely that I used visual studio code to explore the method defination and that is when I came to know that pk_chunk is actually chunk_size in the method and default value is 2048 if you don't specify it

…

On Mon 23 Dec, 2019, 3:48 PM kunal4422, ***@***.***> wrote: @hrishy <https://github.com/hrishy> didnt got you. I am having same code as you and also same error. Instead of pk_chunking=True i tried to give pk_chunking=100000; as given in example. What exactly should i do to rectify this error? Somewhere we have to use get_batch_items() for iterating between batches. How did u solve it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#74?email_source=notifications&email_token=AJCOEWNAKRQ2CVV7WVWDMOTQ2CF7FA5CNFSM4IO3G36KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHQZHHA#issuecomment-568431516>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJCOEWMEFWLYWLY53WNCPDLQ2CF7FANCNFSM4IO3G36A> .

[roshin8]

I added an example to the README which shows how to use PK Chunking. Let me know if that helps you.
https://github.com/heroku/salesforce-bulk/pull/77/files

[fungss]

I added an example to the README which shows how to use PK Chunking. Let me know if that helps you. https://github.com/heroku/salesforce-bulk/pull/77/files

This should be merged into README. It works really well. Thanks a lot.