Race condition in test_queue.test_shutdown_immediate_put_join

[colesbury]

There is a race condition in test_shutdown_immediate_put_join that leads to ValueError: task_done() called too many times. This happens frequently in the free-threaded build. It may also happen in the default build depending on when the GIL is released. You can reliably reproduce the issue by adding a short sleep at the start of Queue.task_done().

The problem is that we call q.task_done() in a thread concurrently with q.shutdown(immediate=True).

cpython/Lib/test/test_queue.py

Lines 581 to 586 in 3e45030

	self.assertEqual(q.unfinished_tasks, nb)
	for i in range(nb):
	t = threading.Thread(target=q.task_done)
	t.start()
	threads.append(t)
	q.shutdown(immediate)

The queue has one unfinished task. If, q.task_done() finishes first then everything is okay. However, if the q.shutdown(immediate=True) happens first, it will remove the single outstanding task and the q.task_done() will raise an exception.

cc @EpicWink

Linked PRs

• gh-116631: Fix race condition in test_shutdown_immediate_put_join #116670

[EpicWink]

CC @YvesDup