{"id":387,"date":"2013-02-06T13:33:43","date_gmt":"2013-02-06T18:33:43","guid":{"rendered":"http:\/\/www.virtualroadside.com\/blog\/?p=387"},"modified":"2013-02-06T13:33:43","modified_gmt":"2013-02-06T18:33:43","slug":"problems-with-file-descriptors-being-inherited-by-default-in-python","status":"publish","type":"post","link":"https:\/\/www.virtualroadside.com\/blog\/index.php\/2013\/02\/06\/problems-with-file-descriptors-being-inherited-by-default-in-python\/","title":{"rendered":"Problems with file descriptors being inherited by default in Python"},"content":{"rendered":"<p>Have you ever run into a traceback that ends with something like this?<\/p>\n<pre>  File \"C:\\Python27\\lib\\logging\\handlers.py\", line 141, in doRollover\r\n    os.rename(self.baseFilename, dfn)\r\nWindowsError: [Error 32] The process cannot access the file because it is being used by another process<\/pre>\n<p>I certainly have, in a few places. The basic problem is that when python creates file objects on Windows (and I think on *nix as well), by default Python will mark the handle as being inheritable (I&#8217;m sure there&#8217;s a reason why&#8230; but, doesn&#8217;t make a whole lot of sense for this to be the default behavior to me). So if your script spawns a new process, that new process will inherit all the file handles from your script &#8212; and of course since it doesn&#8217;t realize that it even has those handles, it&#8217;ll never close them. A great example of this is launching a process, then exiting. When you launch your script again and try to open that handle&#8230; the other process still has it open, and depending on how the file was opened, you may not be able to open it due to a sharing violation. <\/p>\n<p>It looks like they&#8217;re trying to provide ways to fix the problem in <a href=\"http:\/\/www.python.org\/dev\/peps\/pep-0433\/\">PEP 433<\/a> for Python 3.3, but that doesn&#8217;t help those of us still using Python 2.7. Here&#8217;s a snippet that you can put at the very beginning of your script to fix this problem on Windows:<\/p>\n<pre>\r\nimport sys\r\n\r\nif sys.platform == 'win32':\r\n    from ctypes import *\r\n    import msvcrt\r\n    \r\n    __builtins__open = __builtins__.open\r\n    \r\n    def __open_inheritance_hack(*args, **kwargs):\r\n        result = __builtins__open(*args, **kwargs)\r\n        handle = msvcrt.get_osfhandle(result.fileno())\r\n        windll.kernel32.SetHandleInformation(handle, 1, 0)\r\n        return result\r\n        \r\n    __builtins__.open = __open_inheritance_hack\r\n<\/pre>\n<p>Now, I admit, this is a bit of a hack&#8230; but it solves the problem for me. Hope you find this useful!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Have you ever run into a traceback that ends with something like this? File &#8220;C:\\Python27\\lib\\logging\\handlers.py&#8221;, line 141, in doRollover os.rename(self.baseFilename, dfn) WindowsError: [Error 32] The process cannot access the file because it is being used by another process I certainly have, in a few places. The basic problem is that when python creates file objects [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45,40,47,37],"tags":[],"_links":{"self":[{"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/387"}],"collection":[{"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=387"}],"version-history":[{"count":2,"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/387\/revisions"}],"predecessor-version":[{"id":389,"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/387\/revisions\/389"}],"wp:attachment":[{"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=387"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=387"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.virtualroadside.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=387"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}