Python Desidirata

9 thoughts
last posted May 24, 2013, 7:59 a.m.

6 earlier thoughts


Share More Stuff Between Processes

In C, every copy of an executable (or shared library) shares the code in memory. It opens the shared library with mmap. Therefore, if you have a multi-process program that has a 100 megabyte executable, you use 100 megabytes of RAM for the code, regardless of how many copies of that process there are.

In Python, by contrast, .pyc files are opened and read by the read() syscall. This means that every process has its own copy of all the code in memory. In addition to all the other interpreter overhead, this imposes a non-trivial tax on spinning up additional processes to make use of additional cores.

2 later thoughts