Clarifying Concurrency

Clarifying Concurrency explores various aspects of concurrent programming across different programming languages with a focus on Python. It evaluates concurrency models, the integration of languages like Rust, and the evolution of Python's concurrency features, aiming to deepen understanding and application of concurrent programming techniques.

Concurrency in Python Programming Language Integration Concurrency Models Structured Concurrency Coroutines and Asyncio Concurrency in Go Task and Error Management Context Managers in Python Software Development Practices Publishing and Writing

The hottest Substack posts of Clarifying Concurrency

And their main takeaways

Denouement

33 implied HN points • 31 May 24
🕹 Technology Programming Software Tools Conferences
  1. Pittsburgh surprised many with its beauty and many bridges. The Pycon conference there was a great experience and will be held again next year.
  2. Concurrency is a complex topic, and many people underestimate its difficulties. The writer realized it's okay to focus on broader topics rather than an expert niche like concurrency.
  3. Writing for a general audience feels more impactful. There are other programming techniques that can improve productivity for everyone, which is more inspiring to share.

Python Threads vs. OS Threads

16 implied HN points • 07 Sep 23
  1. Python threads in CPython can achieve concurrency, even if not true parallelism due to GIL, allowing for one thread to run while another waits.
  2. CPython allows interfacing with C/C++ extensions that can release the GIL for true parallel computation by mapping Python threads to OS threads.
  3. By using native OS threads, Python threads can utilize native OS APIs, simplify external interactions, and maintain consistency across different Python implementations.

Working Title

8 implied HN points • 19 Sep 23
  1. Using Substack for blogging can simplify the writing process.
  2. Having a solid mental foundation is crucial in understanding complex topics like concurrency.
  3. When looking for a publisher, finding one committed to good promotion and respecting the author's style is key.

Python Subinterpreters

8 implied HN points • 21 Jul 23
  1. Python subinterpreters' future is unclear
  2. Some are hesitant about the impact on libraries like NumPy
  3. PEP 554 may only be available in Python 3.13

The Python-Rust Connection

8 implied HN points • 22 Jun 23
  1. Rust was designed with a focus on safety and performance, making it stand out with its rigorous type system and no need for a garbage collector.
  2. Python and Rust can complement each other well, with Python for rapid development and Rust for performance enhancement through extensions.
  3. Converting Python code to Rust can lead to significant speedups, evident from experiments showing Rust extensions outperforming other methods like Cython.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Golang and the Simplest Approach

3 HN points • 18 Oct 23
  1. Go's default concurrency model is simple and may not handle tasks completion, cancellation, or error handling automatically.
  2. Goroutines in Go are designed as daemon threads, running in the background with no direct connection to the main code.
  3. Using Go's default concurrency model can be beneficial for beginners, but as projects mature, the limitations of this approach might become apparent.

Concurrent Failure

2 HN points • 16 Oct 23
  1. Tasks can conclude by finishing normally, reporting an error, or being forcibly terminated.
  2. Structured concurrency is crucial for managing errors and handling tasks as cohesive groups.
  3. Structured concurrency allows for automatic cancellation of tasks upon failure and collection of all errors produced by tasks.

Coroutines: Single- vs. Multi-threaded

2 HN points • 02 Oct 23
  1. Programming community is still exploring concurrency solutions, with multiple strategies but no universal answer.
  2. Learning concurrency involves understanding that blocking operations are not just due to IO, and async/await doesn't solve all concurrency problems.
  3. Using single-threaded coroutines can be a default choice for reducing data race risk, especially in business software where there may be less experienced concurrency programmers.

Python 3.11 asyncio.TaskGroup

2 HN points • 03 Aug 23
  1. Trio automates task joining and cancellation, simplifying work for users.
  2. Python 3.11 introduces task groups and exception groups, potentially offering a better solution than Trio.
  3. By using asyncio's task groups, users can manage tasks within a scope and ensure all tasks complete without explicit joining.

Python Context Managers

2 HN points • 06 Jul 23
  1. Context managers help with setup and cleanup in Python code.
  2. The @contextmanager function decorator can simplify the creation of context managers using generators.
  3. The 'as' keyword in context management allows for yielding objects and passing information into and out of contexts.

Cancellation in Concurrency

1 HN point • 27 Aug 23
  1. Concurrency involves breaking up a program into independently-running tasks to potentially speed up its execution.
  2. Cancellation in concurrency can be complicated, especially when it comes to safely stopping tasks and cleaning up resources.
  3. Python's asyncio makes cancellation easy and safe at coroutine suspension points, reducing the complexity compared to using threads or multiprocessing.