Archive for the ‘Snippet’ Category

Problems with file descriptors being inherited by default in Python

Wednesday, February 6th, 2013

Have you ever run into a traceback that ends with something like this?

  File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
    os.rename(self.baseFilename, dfn)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process

I certainly have, in a few places. The basic problem is that when python creates file objects on Windows (and I think on *nix as well), by default Python will mark the handle as being inheritable (I’m sure there’s a reason why… but, doesn’t make a whole lot of sense for this to be the default behavior to me). So if your script spawns a new process, that new process will inherit all the file handles from your script — and of course since it doesn’t realize that it even has those handles, it’ll never close them. A great example of this is launching a process, then exiting. When you launch your script again and try to open that handle… the other process still has it open, and depending on how the file was opened, you may not be able to open it due to a sharing violation.

It looks like they’re trying to provide ways to fix the problem in PEP 433 for Python 3.3, but that doesn’t help those of us still using Python 2.7. Here’s a snippet that you can put at the very beginning of your script to fix this problem on Windows:

import sys

if sys.platform == 'win32':
    from ctypes import *
    import msvcrt
    
    __builtins__open = __builtins__.open
    
    def __open_inheritance_hack(*args, **kwargs):
        result = __builtins__open(*args, **kwargs)
        handle = msvcrt.get_osfhandle(result.fileno())
        windll.kernel32.SetHandleInformation(handle, 1, 0)
        return result
        
    __builtins__.open = __open_inheritance_hack

Now, I admit, this is a bit of a hack… but it solves the problem for me. Hope you find this useful!

Posted in python, Snippet, tips, Windows | 1 Comment »

C++ template-fu: delete_if

Sunday, November 9th, 2008

I’ve been doing a lot of work with templates lately, for my R* tree implementation (which is morphing into a R* tree and a B tree library) and some things that I’ve been playing around with for Roadnav, especially in regards to serialization.

Heres a neat snippet that I created:

template <bool>
struct delete_if {
	template <typename T>
	static void del(T t) { delete t; }
};

template <>
struct delete_if
{
	template <typename T>
	static void del(T t) { /* no-op, doesn't delete */ }
};

This uses a partial specialization to do the decision portion, and a static member template to eliminate some of the typing (you could get rid of the member template, and you would end up calling delete_if<type, delete_item>::del ).

So what this is intended for is those cases where you are implementing some generic container, and you don’t know whether the type you have is a pointer or not. You *could* use a normal bool as a parameter:

if (delete_it) delete var;

But primary problem with this is that it won’t compile if the type isn’t a pointer. So a sample of something similar to what I’m using it for is something like this:

template < typename T, bool delete_item >
class test {
public:

	~test()
	{
		// only release the item if the client specifies
		// that they want to
		delete_if<delete_item>::del(val);
	}

private:
	T val;
};

Obviously this is just a really simple example of how you can use this. One could also use boost::is_pointer to detect whether the type is a pointer, and pass *that* to delete_if while using a ‘normal’ variable to determine whether the client wants to have the item be deleted or not. I leave that as an exercise for the reader.

Posted in Snippet, Software | No Comments »