Archive for the ‘Howto’ Category

BPM autodetection using python + GStreamer’s bpmdetect plugin

Sunday, August 16th, 2015

I recently found the bpmdetect element in GStreamer, and thought it would be neat to try it out and see how well it works. The GStreamer bpmdetect plugin is mostly undocumented, so I had to dig in the source code to figure out how to extract tags from it. The operation is pretty simple:

  • Setup a pipeline to read in a file, put a fakesink at the end of the pipeline and set ‘sync’ to false
  • Insert the bpmdetect element
    • However, due to this bug, insert a capsfilter before the element to mix it down to a single channel!
  • Attach a message handler, and listen for taglist messages
    • If you already have the BPM tag on the file, then it will be emitted by whatever is decoding the audio too. So, what I do is look for messages that *only* have the beats-per-minute tag in the tag list
    • The bpm is accumulated, so the last BPM message you get will be the calculated rate

Pretty simple! Of course, the results are only as reliable as libsoundtouch’s BPM detection is… but it seems to be correct at least some of the time. Expect to see this code in Exaile soon as a companion to the manual BPM counter! 🙂

Code is available in a gist on github: https://gist.github.com/virtuald/c30032a5b8cdacd1a6c0

Connect to RoboRIO USB on Linux

Saturday, January 3rd, 2015

We got our RoboRIO imaged today, and my first thought was whether I could use it on Linux without needing to plug it into a network. Turns out, this is a pretty simple thing to do!

  1. Plug the RoboRIO in to your computer
  2. Identify the network device, it should be something like enp0s29u1u2
  3. Assign it an IP address like so:
    sudo ip addr add 172.22.11.1/24 dev enp0s29u1u2

Now you need to start an DHCP server to give it an address, because FIRST didn’t give it a static address for some reason. You could modify the configuration on the RoboRIO… but let’s assume you don’t want to do that. Instead:

  1. Download this python script that acts like a DHCP server
  2. Run this:
    sudo python simple-dhcpd -a 172.22.11.1 -i enp0s29u1u2 -f 172.22.11.2 -t 172.22.11.2

And that’s it! When it works, you should get a message that says “Leased: 172.22.11.2”.

If it’s not working, here are some things to watch out for:

  • Make sure your firewall allows port 67 on UDP
  • Make sure dnsmasq or some similar program isn’t listening on port 67 (use “netstat -ln | grep 67” to check)

Things that would be nice to change:

  • It’d be nice if FIRST changed the usb0 device to have a static address instead of depending on dhcp
  • Probably should create a udev rule to make the network device something pretty
  • When you disconnect the device, you have to run the scripts again. Probably should set something more permanent up, but this is good enough for now

Hope this helps you out!

Use GroupTagger to rapidly organize your audio files with Exaile 3.3.0 using the Grouping tag

Saturday, October 6th, 2012

Well, I haven’t blogged here in quite awhile, but it’s been a pretty busy year! One project I’ve been spending a fair amount of time on is Exaile, a cross platform music player for GTK+ (its free and works on Windows, Linux, and even OSX — though, the installation for OSX is rather tricky). I’ve been DJ’ing Lindy Hop dances for almost a year now, and I’ve been fixing up Exaile to be an awesome music player for DJing — adding a BPM counter, secondary output device support (sometimes called pre-listening), and other things that I’ve found to be useful.

One feature in particular I want to highlight is a plugin I’ve created called GroupTagger. This super-useful plugin is distributed with the latest stable version of Exaile. This plugin allows you to easily and rapidly organize your music using the ‘Grouping’ tag in your audio files. The best part about this is that the data is stored in your MP3/OGG/FLAC/whatever audio files, so you can use the data with your favorite audio player if it supports it (Winamp, iTunes, others already do). A lot of people already categorize their music using this tag, and this plugin can manage music that is already tagged this way. To illustrate how useful this is, here are some screenshots and a mini-tutorial.

(more…)

Changing variables using a web interface and embedded HTTP server

Tuesday, April 21st, 2009

When walking around during the Boston Regional, I had been talking to some people about code, and they mentioned that LabView was great because they could tune their PID controllers on the fly while the robot was operating. So I thought to myself, “why can’t I do this with C++?”. And… so I did. WebDMA was created to allow our FIRST Robotics team to tune our robot in an easy to use and intuitive way via any modern web browser.

Using C++ operator overloading, WebDMA provides proxy objects that your application can use as normal variables which can be manipulated or displayed by your application via a configurable jQuery/javascript powered Web 2.0 interface hosted by an lightweight embedded web server.

Despite that WebDMA was specifically created for use in FIRST Robotics on the NI-cRio/vxWorks platform, it uses the Boost ASIO portable networking library and Boost Thread portable threads library and is usable on any platform supported by these Boost libraries (tested on Boost 1.38, requires a patch for vxWorks).

A non-functional (but very shiny) demo of the interface is available at http://www.virtualroadside.com/botface/index.html

Visit the Google Code project site for WebDMA

Update: Go here for a video: http://www.virtualroadside.com/blog/index.php/2009/04/25/webdma-demo-video/

Installing Monroe Quick Struts on my Grand Am

Saturday, April 26th, 2008

Apparently, the Boston area is notorious for having bad roads and huge potholes. The other day I was driving my 1999 Pontiac Grand Am around here, and I hit some pothole or something really hard… didn’t notice anything wrong though. So a few weeks later when it started really acting up, so I brought it somewhere and appearantly I totally snapped one of the springs in my front left tire, and my struts were bad in the front. With how expensive things are here, they wanted $430 * 2, plus labor costs to fix it… and no, that was not going to happen. Heres a picture of the broken spring/strut next to a new one:

So I found these parts called a quick strut — I got the parts from rockauto.com, and they were around $170 each. They come with the strut, springs, and everything you need, and they’re extremely easy to install! Even though they come with decent instructions, heres a quick howto in case you ever want to do this yourself:

(more…)

Make BitTorrent/P2P less annoying at home using Linux, Iptables, and QoS

Saturday, November 17th, 2007

At home, I’ve been using a Linux router for as long as I can remember… since, a rather long time ago, I didn’t have money for a router but I happened to have an extra computer that I had been playing with Linux on it. So, I developed an ipchains script (which I eventually converted to iptables) to do NAT on it, and its worked pretty well ever since then. I honestly don’t remember where I derived the script from, however.

When I was in the dorms and afterwards in an apartment with others, we had used my Linux router, since it was pretty simple and theres a ton of things you can do with a Linux server. However, something we noticed (which wasn’t specific to this router) is that whenever someone was using BitTorrent or some other P2P app (you know, to download Linux distros and CC-licensed stuff), it would totally kill our internet access unless the person made their upload rate ridiculously low.. which is fine, unless a visitor stops by, plugs in, and forgets to turn their torrents down.

So, sometime last year I decided that there was probably a good way to filter different types of traffic so they don’t get excessive. Turns out, you can combine iptables and the QoS functionality of the kernel to do just that.

(more…)

prnRename: an AutoIt based utility to rename IPP printers in Windows

Wednesday, October 17th, 2007

Using CUPS in conjunction with Windows is pretty easy to do, except there is one big major annoyance: you can’t rename the printer to something “friendly”, its always something long and annoying with the URL of the printer in it.

So, I jumped into the registry and figured out how to change it, and developed a tool using AutoIt (which works really well for these types of simple things) that allows you to change the URL and the name displayed for the printer in Windows Explorer.

See, at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Print\Printers lies registry keys for all of the printers you have there, and you’ll notice that the key for the IPP printer starts with two ,, and the URL. So, you just need to rename that to whatever you want, then change the following keys/values under that key to the same thing (this is AutoIt code, but I think you get the point).

RegWrite($key,"Name","REG_SZ",$newPrinterName)
RegWrite($key & "\DsSpooler","printerName","REG_SZ",$newPrinterName)
RegWrite($key & "\DsSpooler","uNCName","REG_SZ", "\\" & @ComputerName & "\\\" & $newPrinterName)

After making the changes, restart the spooler service

net stop spooler
net start spooler

And, thats pretty much all there is to it. Obviously, use this at your own risk, seeing as this is modifying undocumented registry entries. Source code is included.

Download Windows 2000/XP IPP Printer Rename Utility 

Visual Studio 2005 not putting User Controls into Toolbox

Saturday, October 6th, 2007

I’ve been developing in C# for my Senior Design project, and I have a lot of user controls that I’ve been using in the project, and while most of them are instantiated at runtime, a few needed to be embedded into the forms. And of course, I couldn’t actually do that since I couldn’t drag any of the controls into the Toolbox, even with a successful build. After playing with it for awhile, I found the solution.

It was as easy as this:

Tools -> Options ->  Windows Form Designer  -> General -> AutoToolboxPopulate = true

For some reason, it was not set on my desktop but was set on my laptop. So odd… hope that helps ya if you happen to have that problem!

Obsessive Web Statistics (OWS) analysis plugin tutorial

Saturday, September 1st, 2007

This is a short tutorial on how you can write an analysis plugin for Obsessive Website Statistics (OWS). OWS is designed first and foremost to be plugin friendly, and as you will see, adding useful functionality in the form of plugins is not hard at all, and can be done in just a few lines of code. We are going to add DNS hostname resolving to OWS.

What is an analysis plugin?

An analysis plugin performs analysis on the parsed logfile data, and stores that information in the database dimensions. OWS has wrapped all of this stuff in a nice easy to use abstraction layer so that you won’t need to make actual SQL queries if you don’t want to.

Implementation

All OWS plugins are implemented as PHP classes. This is the bare skeleton that all OWS plugins should define.

class OWSDNS implements iPlugin{

	// this should return a unique ID identifying the plugin, should start with an alpha,
	// should use basename instead of just __FILE__ otherwise it could expose path information
	public function getPluginId();//{
		return 'p'. md5(basename(__FILE__) . get_class());
	}

	// returns an associative array describing the plugin
	public function getPluginInformation(){

		return array(

			'pluginName' => 'Name of plugin',
			'aboutUrl' => 'http://information.about.plugin',

			'author' => 'author',
			'url' => 'http://developers.website',

			'description' => 'Description of what plugin does'
		);
	}
}

You should notice we define two functions — getPluginId() and getPluginInformation(). These must be defined by any OWS plugin, and are used to identify the plugin in a number of instances. This plugin also implements iPlugin. All interfaces are defined (with plenty of comments) in include/plugin_interfaces.inc.php. A plugin can implement as many interfaces as it needs to. There are a few types, but the one we are going to implement is iAnalysisPlugin. We will do so by changing the first part to:

class OWSDNS implements iPlugin, iAnalysisPlugin {

Additionally, we need to register the plugin with OWS so that it knows what kind of plugin you are defining. Add this to the end of your source file:

register_plugin('analysis',new OWSDNS());

An analysis plugin needs to implement the following functions:

define_dimensions
InitializeAnalysis
preAnalysis
getPrimaryNode
getAttributes
postAnalysis

All of these functions are documented in include/plugin_interfaces.inc.php if you need more comprehensive information.

Now, OWS stores data in multiple dimensions. Each dimension has a ‘primary node’ which is the main data element of the dimension. Each primary node can have mutliple attributes which are defined about it, and always has the same name as the dimension. Plugins can define new dimensions or extend existing dimensions.

Right now, OWS stores only the host address — which is an IP address representing the visitor. What our plugin needs to do is resolve this address, and store it as an attribute of the dimension. So, we need to extend the dimension ‘host’, which we can do using the function define_dimensions().

// this function should return a set of arrays that define the dimensions
// and attributes that this plugin defines. You should not specify an attribute
// that another plugin defines. This is not website dependent.
public function define_dimensions(){

	return array(
		'host' => array(
			'hostname' => attribute_defn('varchar',254,16)
		)
	);
}

Pretty simple, eh? See, the array returned means that we are defining inside dimension ‘host’, an attribute named ‘hostname’. The function attribute_defn is used to define the SQL type that our attribute has, so the installer can create it for us. Now, we can write the actual analysis part.

At the beginning of analysis, the function InitializeAnalysis is called in case the plugin needs to do something before the analysis begins. This function is called once per website analyzed. Our plugin isn’t going to need this, so we just return true.

public function InitializeAnalysis($website){
	return true;
}

Now, after all plugins are initialized, then the logfile lines are read from the logfile (or from the database in the case of an install or in the case of reanalysis). It is read in phases, which consist of 4 steps:

preAnalysis
getPrimaryNode
getAttributes
postAnalysis

Now, preAnalysis and postAnalysis are only called once per phase, but getPrimaryNode is typically called at least once per logfile line. Our plugin doesn’t use getPrimaryNode — getPrimaryNode is only used for plugins that define new dimensions in define_dimensions. If you don’t define a primary node, then you should return false and show an error.

It should also be noted that our plugin doesn’t need to do any preAnalysis or postAnalysis, so we can just return true.

public function preAnalysis($website,&$ids){
	return true;
}

public function getPrimaryNode($website, $dimension, $line){
	return show_error("Invalid dimension passed to plugin\"" . get_class() . "\"");
}

public function postAnalysis($website,&$ids){
	return true;
}

Now we get to the part that actually does the work. The function getAttributes needs to return an array representing the attributes that the plugin defines per dimension. The $dimension argument is passed in to the function, and we should only do analysis on the primary node. The contents of the primary node are passed in to the function as well. This makes sense, because attributes of the primary node should be discernable by only looking at the primary node itself. If this is not the case, then you should probably be defining a new dimension instead.

This function should return an array of attributes/values in the form of:

	array('attribute' => 'value', ...)

Note: The returned values can be cached (for performance reasons), so this function may NOT always be called for each row. You should ALWAYS return an array with the same keys each time, in the same order that you defined them in define_dimensions. Of course, if you do not define any attributes in the dimension passed in the $dimension parameter, or if there is an error, then return false.

Anyways, heres the code for this function:


public function getAttributes($website, $dimension, $pnode){

	if ($dimension != 'host')
		return show_error("Invalid dimension passed to plugin\"" . get_class() . "\" in getAttributes!");

	// return the hostname
	return array('hostname' => gethostbyaddr($pnode));

}

And thats it! Wasn’t that easy? Of course, theres a lot more useful things we could probably implement, and make this more polished. Now, after you install the plugin and run the analysis, the only filter you’ll be able to use on your new dimension attribute in the web interface is the manual analysis, since it allows analysis on all defined dimensions. But, it would be a pretty trivial matter to either modify an existing filter plugin or create a new filter plugin. We’ll discuss this in the future.

Hope this helps you out. If you need help with OWS, or developing for OWS, don’t hesitate to ask! Leave your comments, or join the obsessive-compulsive mailing list!

Download this
Obsessive Website Statistics Website

10 things you can do now to improve the security and performance of your Windows PC

Thursday, July 19th, 2007

Lets face it, secure and Windows are two words almost never used together, and for good reason. Security on Windows has traditionally been extremely horrible, and has singlehandedly brought about the rise of the antivirus industry and its billions of dollars of revenue. Another problem with Windows is that it gets slow over time. If your computer is slow, most of the time because of one of the following reasons:

  1. You have spyware, viruses, or both
  2. You have too many startup programs
  3. You have too many ‘temp’ files
  4. Your temporary internet files folder has too many files (internet explorer)
  5. You have toolbars or other browser helper objects installed
  6. You have antivirus installed (this has a big impact on performance)
  7. You’re using Windows Vista

However, it is possible to live (relatively) securely using a Windows PC with decent performance. Here are some useful tips (most that you already should know, and some that may surprise you).

Security Tips

  • Do your Windows Updates. This should be a given. If you don’t, you are asking for trouble. Sure they’re annoying at times, and can even cause problems, but you better do them. Personally, I do the optional updates too, just in case.
  • Disable Windows file sharing and the Remote Registry service. If you don’t use it, then disable it. Most users don’t share things on the network, so this isn’t a problem. File and Printer sharing can be dangerous to have on a public network, as shown by the next item.
  • Have a strong Administrator password. Many viruses/exploits rely on the fact that you have file and printer sharing enabled and that the Administrator password is blank. See, Windows shares out all of your hard drives to the world with names like C$ or ADMIN$, and by default makes them available to anyone who has the Administrator password. Check for yourself: Click Start -> Run -> type ‘fsmgmt.msc’ and click on ‘Shared Folders’. I bet you didn’t know those were being shared. If you’re really paranoid, rename the Administrator account.
  • BACK YOUR STUFF UP. Seriously, everyone says to do this, but nobody does it regularly. The best defense you can have against malware and viruses is the ability to restore your information anytime you need to without losing too much data. Theres really no excuse with how cheap CD-R and DVD-R media/burners are now.

Performance Tips

  • Don’t use antivirus. Get rid of it. Conventional wisdom says that you should use antivirus on your windows PC to make sure that you don’t get any nasty viruses. I disagree. Not only does antivirus routinely fail to protect your machine from viruses (especially new ones), but most vendors products slow down your computer a TON, and cause even more problems. I have not had antivirus on my PC for over 5 years, and I have never had problems with viruses or spyware. The key to keeping viruses off your computer is surprisingly simple: don’t visit questionable websites, don’t download questionable software or attachments, and do your Windows Updates. And if you happen to make a mistake, you’ve got those backups, right?
  • Remove Viruses and Spyware. Cmon, you don’t have these on your computer, right? NEVER pay money for an anti-spyware product, there are way too many free resources out there to take care of the problem for you (there are also a LOT of bad free anti-spyware products as well, so beware!). My small howto  details some programs that you can use to do the job manually without too much trouble.
  • Delete those temporary files. Having too many temporary internet files can slow your computer down a lot, especially on startup and shutdown. For some reason, Windows can sometimes accumulate thousands of files in the temp directory. I’ve seen up to 10,000 files before in someone’s temp directory. Wait you say, I thought that ‘temporary’ means ‘temporary’? No, not in Windows. Most of the trash is stuff left behind by lazy installers or careless third party programs. I recommend you use a program like Pocket Killbox (http://killbox.net/) to delete the files, its much quicker than doing it manually. Also, don’t forget about those temporary internet files. If you’re not using Firefox (you are, aren’t you?), then internet explorer can get extremely slow because of its inefficient caching methods. Pocket Killbox can help you here too.
  • Stop programs from running on startup. There are a lot of tools (including MSCONFIG, which comes with Windows, despite being rather annoying to use). There are a lot of free tools you can use to examine the programs that are starting up when your computer is, and more! HijackThis and AutoRuns are excellent programs to use for this purpose. Refer to my antivirus and spyware howto about good ways of doing this.
  • Delete your system restore points (All except the most recent one), or just disable it completely. In my experience, System Restore rarely ever fixes problems (though, I’ve heard rumors to the contrary). It just wastes 10% of your hard drive.
  • Defrag regularly. This isn’t as important as it used to be, but its still a good way to keep your system running smoothly. I schedule mine to run every night around 4am when I’m not using my computer.

I hope this helps you out, let me know what you do to improve the performance and security of your system!