-
-
Notifications
You must be signed in to change notification settings - Fork 207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multithreading printing of duplicate results #263
Comments
Hi, That is caused by the multiple refreshes of Anyway, I think what you want is something like this: import random, time
from concurrent import futures
def requester(url):
# print(f'requesting: {url}')
time.sleep(random.random())
with futures.ThreadPoolExecutor(max_workers=3) as executor, alive_bar(100) as bar:
future_to_url = {executor.submit(requester, url): url for url in range(100)}
for future in futures.as_completed(future_to_url):
url = future_to_url[future]
print(f'finished: {url}')
bar() If you uncomment that line everything breaks again. Does that solve it? |
This would work for printing: import random, time, queue
from concurrent import futures
def requester(url):
q.put(f'requesting: {url}')
time.sleep(random.random())
q = queue.Queue()
with futures.ThreadPoolExecutor(max_workers=3) as executor, alive_bar(100) as bar:
future_to_url = {executor.submit(requester, url): url for url in range(100)}
for future in futures.as_completed(future_to_url):
while not q.empty():
print(q.get())
url = future_to_url[future]
print(f'finished: {url}')
bar() You'll get output like this:
|
Hmm, having strange behaviour when printing from threads sounds like a recipe for disaster. Lots of code, often beyond a developer's control, might try to print in a separate thread. |
Actually, you're right @TheTechRobo. |
Found it, and fixed it 👍 I've also improved them a bit along the way. I've never noticed that an empty So, using these new synced print hooks, I've tested this code: import random, time
from concurrent import futures
def requester(url):
print(f'requesting: {url}\n', end='') # this is printed from the thread pool.
time.sleep(2*random.random())
with futures.ThreadPoolExecutor(max_workers=3) as executor, alive_bar(10) as bar:
print()
future_to_url = {executor.submit(requester, url): url for url in range(10)}
for future in futures.as_completed(future_to_url):
url = future_to_url[future]
print(f'finished: {url}') # this from the main thread.
bar() It was printing this: ...:
on 0:
on 0: requesting: 0
requesting: 1
requesting: 2
requesting: 3
finished: 0
on 1: requesting: 4
finished: 2
on 2: requesting: 5
finished: 3
on 3: requesting: 6
finished: 1
on 4: requesting: 7
finished: 4
on 5: requesting: 8
finished: 7
on 6: requesting: 9
finished: 6
on 7: finished: 5
on 8: finished: 9
on 9: finished: 8
|████████████████████████████████████████| 10/10 [100%] in 3.8s (2.53/s) And now it will print this: ...:
on 0: requesting: 0
requesting: 1
requesting: 2
requesting: 3
finished: 0
on 1: requesting: 4
finished: 2
on 2: requesting: 5
finished: 1
on 3: requesting: 6
finished: 4
on 4: requesting: 7
finished: 3
on 5: requesting: 8
finished: 6
on 6: requesting: 9
finished: 5
on 7: finished: 8
on 8: finished: 9
on 9: finished: 7
|████████████████████████████████████████| 10/10 [100%] in 3.0s (3.37/s) |
Important to say that, thanks to the way that Python calls ...:
on 0: requesting: 0
requesting: 1
requesting: 2
requesting: 3
requesting: 4
requesting: 5
finished: 4
on 1: finished: 1
on 2: finished: 0
on 3: requesting: 6
finished: 2requesting: 7
requesting: 9
requesting: 8
on 4: finished: 5
on 5: finished: 3
on 6: finished: 6
on 7: finished: 9
on 8: finished: 8
on 9: finished: 7
|████████████████████████████████████████| 10/10 [100%] in 0.0s (8656.14/s) Unfortunately, Python calls |
@rsalmei Thanks! Referring to your code, it is currently running normally. |
You're welcome! |
Hello, When I use multithreading, the runtime prints duplicate results.
Running result
What should I do to print the result once? Thanks!!!
The text was updated successfully, but these errors were encountered: