Archive for the ‘General’ Category

Detect the ID of the docker container your process is running in

Tuesday, August 16th, 2016

Couldn’t find this exact thing elsewhere, so I’m publishing it in case I ever need it again. This trick depends on the fact that docker uses cgroups and the cgroup ID seems to always be equal to the container ID. This probably only works on linux-based docker containers.

import re

def my_container_id():
    '''
        If this process is in a container, this will return the container's ID,
        otherwise it will return None
    '''
    
    try:
        fp = open('/proc/self/cgroup', 'r')
    except IOError:
        return
    else:
        with fp:
            for line in fp:
                m = re.match(r'^.*\:/docker/(.*)$', line)
                if m:
                    return m.group(1)

Download and unpack source RPM in one command

Saturday, October 31st, 2015

One thing I constantly find myself doing when developing GTK on python is referring to the C source code to figure out just exactly why it’s doing the things that it does. It’s easier to just download the source package from my distro instead of finding the upstream source repository.. etc etc etc. Of course, downloading and unpacking an RPM is magic that I always seem to forget.

Here’s a script that does it all in one swoop.

Simple python wrapper to give SSH a password for automation purposes (with output capture)

Saturday, December 20th, 2014

A very simple thing that most developers want to do at one time or another is automate SSH. However, since OpenSSH reads the password from the terminal, this can make it a very annoying thing to automate. Indeed, searching google for a solution to this problem yields all sorts of bad answers (and very few good ones).

There are a number of ways to solve the problem in python other than a wrapper around the SSH command:

  • Use public keys for authentication (this is a generally good practice for this sort of automation, but sometimes you can’t do this)
  • Use paramiko to talk the SSH protocol directly
  • Use pexpect‘s ssh wrapper to wrap openssh instead of rolling your own wrapper

However, if you can’t use external dependencies for some reason, this script is for you. It’s derived from this script and many other sources of information on the net, but I think it’s a bit simpler to use. It has been tested on Linux and OSX, on Python 2.7 and 3.4.

Let me know if you have problems with the script!

Concept Map playlist visualization generated by Exaile 3.4 beta3

Tuesday, July 15th, 2014

At work, I’ve been playing a little bit with visualization of data in D3. After hours, I DJ at local Lindy Hop dance events, and it occurred to me recently that it could be interesting to visualize my playlists to understand more about what I actually played. Using Exaile, I’m able to easily add a lot of metadata to the tracks in my collection, and its plugin framework made it easy to take that data and do something interesting with it.

From that initial investigation, I’ve built in a playlist visualization templating engine plugin for Exaile that can do some very simple visualization stuff, and I’m planning on extending the types of things I can do with it. Here’s an interesting one I created from a recent playlist. This visualization was inspired by The Concept Map, except the source code for this isn’t minified. 🙂

Check out the visualization at  bl.ocks.org (works best in Chrome).

Tool to implement ruby-inspired DSLs in python

Sunday, November 17th, 2013

This weekend, I created library to aid developers in creating neat little embedded DSLs when using python, without having to do any complex parsing or anything like that. The resulting DSLs look a bit more like english than python does.

The idea for this was inspired by some ruby stuff that I’ve been using lately. I’ve been using ruby quite a bit lately, and while I am still not a huge fan of the language, I do like the idea of easy to code up DSLs that I can use to populate objects without too much effort. Since I wanted a DSL for a python project I was working on, I played with a few ideas and ended up with this tool. (more…)

Versioned chef environments

Sunday, November 3rd, 2013

I was just recently introduced to chef, and it has turned out to be a pretty useful tool for automating infrastructure. One feature that I’m using in our environment is chef environments. A bunch of developers are all updating this file, and their cookbooks depend on attributes in it*.  I keep getting bit by various user errors that all could be solved if a cookbook could state ‘make sure you have at least this version of the environment’.

I’m sure there’s a good reason for it, but chef does not currently support versioning environments. To work around this problem, I created a cookbook with a library function that can check a node attribute and compare the version information there to determine if the environment is out of date. It’s a bit of a hack for now, but it gets the job done.

I thought someone else might find this useful, so I uploaded the code to github and to the opscode community site. If you have a need for versioned environments in chef, this might work for you too. Let me know if you find it useful!

*Yes, this could be considered an anti-pattern, but for our use case using environments to override attributes makes perfect sense, and allows us to not have to fork every community cookbook that we want to use.

Another Python SIP wrapper for the tesseract OCR library

Friday, July 26th, 2013

Tesseract is a pretty decent open source OCR engine that was developed by HP back in the day, but is now maintained as open source by Google. It has a C++ API that you can program to, and as you would expect there are a number of wrappers (of varying quality) that allow you to use libtesseract from Python. For various reasons, none of those fit my needs, so I created my own SIP-based wrapper instead. I will not be maintaining this as a format project, but if you want an apache licensed python wrapper, you can find the code on github. 🙂

GitHub repository for Python Tesseract SIP wrapper

glib.idle_add for tkinter in python

Saturday, November 10th, 2012

After doing a lot of python GTK+ work on Exaile, I’ve found that it uses glib.idle_add() extensively — and usually with good reason. idle_add is great if you want to ensure that whatever you’re calling is being called on the GUI thread, so that way you don’t have to worry too much about thread interactions as long as you keep things separate.

Another mentor and I are developing a GUI video game along with a ‘fake wpilib’ for our FIRST Robotics programming students to help them learn how to program, and as such we’ve decided to use TKinter for the GUI toolkit (since it supports Python 3, and usually doesn’t require the kids to install anything special to make it work). However, as I started making things I couldn’t find the equivalent of idle_add() for TKinter, and I guess there isn’t one — after_idle() apparently blocks until the event loop is idle, and so that isn’t what I wanted.

A number of posts I found online advocated to poll a queue for input… but I *really* dislike polling, and try to avoid it when I can. So I wrote up this set of routines that is roughly equivalent to idle_add() in tkinter, and uses a queue while avoiding polling.

        
import Tkinter as tk
from queue import Queue, Empty

def idle_add(callable, *args):
    '''Call this with a function and optional arguments, and that function
       will be called on the GUI thread via an event.
       
       This function returns immediately.
    '''
    queue.put((callable, args))
    root.event_generate('<<Idle>>', when='tail')
    
def _on_idle(event):
    '''This should never be called directly, it is called via an 
       event, and should always be on the GUI thread
    '''
    while True:
        try:
            callable, args = queue.get(block=False)
        except queue.Empty:
            break
        callable(*args)
        
queue = Queue()
root = tk.Tk()
root.bind('<<Idle>>', _on_idle)

Comments inadvertently disabled

Wednesday, October 10th, 2012

Apparently at some point my anti-spam plugin decided to stop working. No wonder I haven’t had any comments in a really long time…. should be working now!

cRio Netconsole implemented in Python

Friday, November 12th, 2010

The FIRST Robotics Competition uses the NI-cRio platform for the controller for the robots in the competition. There is a bit of functionality called ‘NetConsole’ which sends the stdout from vxWorks out to a waiting client (refer to the WPI FIRST website for instructions to enable this). It turns out that the protocol used to implement the NetConsole for the cRio is incredibly simple… it just sends the raw output data as a bunch of UDP packets out to the broadcast address on port 6666 (which I suppose is slightly amusing). Here’s a dirt-simple python script that catches the output (TODO: need to send it input… haven’t gotten around to looking at that yet).

#!/usr/bin/env python

import socket
import select

UDP_PORT=6666

sock = socket.socket( socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP )

sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
sock.bind( ('',UDP_PORT) )
sock.setblocking(0)

while True:
    result = select.select( [sock], [], [] )
    msg = result[0][0].recv( 8196 )
    print msg