Quick Tip: Installing MySQLdb for Python on Mac OSX

This quick tip is intended to serve as a quick reference for setting up MySQLdb package for Python on Mac OSX (but this will probably apply on Linux too).

Firstly, download the latest MySQLdb package from sourceforge. Then unzip the file anywhere on the file system. On the command prompt, activate your virtualenv and then change to the unpacked directory;

(myvirtualenv) kevin$ cd ~/Downloads/MySQL-python-1.2.3

Now install it using the setup.py script

(myvirtualenv) kevin$ python setup.py install

If everything went well, it will install correctly and you can confirm by entering a Python shell and trying to import the package.

(myvirtualenv) kevin$ python
Python 2.7.2 (default, Jan 31 2012, 18:01:16) 
[GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.10.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
>>>

If no errors occur then you’re good to go. If however you get the error;

dlopen(/Users/kevin/.python-eggs/MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): Library not loaded: libmysqlclient.18.dylib
  Referenced from: /Users/kevin/.python-eggs/MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so
  Reason: image not found

Then you will need to locate the libmysqlclient.18.dylib file and set DYLD_LIBRARY_PATH environment variable. For me, the library was located here -> /usr/local/mysql-5.5.15-osx10.6-x86_64/lib/libmysqlclient.18.dylib. You can set this in a few places such as your .bash_profile or in the bin/activate file of virtualenv.

export DYLD_LIBRARY_PATH=/usr/local/mysql-5.5.15-osx10.6-x86_64/lib/

Python

mac osx, mysql, python, tip

February 1, 2012

Leave a comment

Getting Python Tornado Web Autoreload to work

I’ve been doing a bit of Tornado work recently, and one of the things I really found annoying was the getting the auto reload functionality working when developing. I just couldn’t get it working, but eventually did it using the following code.

import tornado
from tornado import autoreload

tornado.options.parse_command_line()
app = MyApp()
app.listen(8888)
ioloop = tornado.ioloop.IOLoop().instance()
autoreload.start(ioloop)
ioloop.start()

I also had debug set to True in my tornado settings. After that, the server automatically restarted when my modules changes.

I hope this helps someone else getting autoreload to work.

Python

python, tornadoweb

October 15, 2011

2 Comments

Synchronising Git repositories in Redmine

Lately, I’ve been using Git more and more for development, edging away from Subversion. One of the things I found slightly frustrating was setting up Redmine to sync the project repositories. Of course, this is easy in Subversion as it’s a remote call to the SVN server, however when using Git, a different approach is needed.

Essentially, what we need is a mirrored repository on the same file system as the Redmine installation, then using a sheduling task to fetch the updates.

For this guide, I’m going to assume you know how to install Git and Redmine and how to install and manage SSH keys so won’t cover these points. In my situation, I host all my repositories on my NAS which is a separate location to my Redmine installation.

For these examples, I am using Redmine v1.2.1 and Git 1.7.6 on Fedora 15.

First, initialize a new repository.

$ mkdir test.git
$ cd test.git
$ git --bare init

Next, create a directory inside your Redmine installation that will hold the mirrored repositories and clone the newly created Git repository.

$ cd /home/kevin/vhosts/redmine
$ mkdir repos
$ cd repos
$ git clone --mirror git@192.168.0.9:repos/test.git

Now, you need to create a new scheduled cron job to fetch all the changes. I personally do this at 5 minute intervals but alter it to suit your needs.

$ crontab -e

*/5 * * * * cd /home/kevin/vhosts/redmine/repos/test.git && git fetch -q --all

With all that done, you just need to set the repository path inside your Redmine project which in this case would be /home/kevin/vhosts/redmine/repos/test.git.

General Development

Git, Redmine

July 25, 2011

Leave a comment

Python Tutorials: List Comprehensions

One of the thing that I like about the Python language is the level of versatility you have at your disposal. List comprehensions is concept that enables you to generate a list quickly with minimal code. At first glance, they appear confusing to read, but once the syntax is understood, they can prove to be very powerful. As I am a PHP developer first, I will make comparison syntax that will hopefully make things easier to understand.

In PHP we can generate a simple array like this;

$lang = array('PHP', 'Python', 'Ruby', 'Java');

Now say we wanted to alter that array making all of it’s values uppercase. We might do this;

$newLang = array();
foreach ($lang as $name) {
    $newLang[] = strtotupper($name);
}

Now, in Python using list comprehensions, we would do this;

lang = ['PHP', 'Python', 'Ruby', 'Java']
newLang = [name.upper() for name in lang]

Okay, that’s really simple yes? If it doesn’t make sense, read if from right to left, so ‘for name in lang’, then ‘name.upper()’. Let’s move on.

Say we wanted to do some string substitution. In PHP we may do this;

$lang = array('PHP', 'Python', 'Ruby', 'Java');
$newLang = array();
foreach ($lang as $name) {
	$newLang[] = sprintf('I like %s', $name);
}

Using list comprehension in Python it looks like this;

lang = ['PHP', 'Python', 'Ruby', 'Java']
newLang = ['I like %s' % name for name in lang]

See how elegant and easy it is? You can even expand on this and add conditionals too. The following example creates a new list if the language begins with the letter ‘P’.

lang = ['PHP', 'Python', 'Ruby', 'Java']
newLang = [name for name in lang if name[0] == 'P']

In PHP, we would do this to achieve the same result;

$lang = array('PHP', 'Python', 'Ruby', 'Java');
$newLang = array();
foreach ($lang as $name) {
	if (substr($name,0,1) === 'P') {
		$newLang[] = $name;
	}
}

To go one step further again, we can create the final list using two loops inside the comprehension. Take a look at this example;

gridx = range(0,5) # [0,1,2,3,4]
gridy = range(1,4) # [1,2,3]
mix = [[x, y] for x in gridx for y in gridy]

This outputs the following result;

[[0, 1], [0, 2], [0, 3], [1, 1], [1, 2], [1, 3], [2, 1], [2, 2], [2, 3], [3, 1], [3, 2], [3, 3], [4, 1], [4, 2], [4, 3]]

In PHP, we would use two nested loops to create the same result;

$gridx = array(0,1,2,3,4);
$gridy = array(1,2,3);
$mix = array();
foreach ($gridx as $x) {
	foreach ($gridy as $y) {
		$mix[] = array($x,$y);
	}
}

And that’s pretty much all you need to know about Python list comprehensions. Their a great way of simplifying code and can be very expressive when wanting to generate lists. Have fun!

Python

python

July 21, 2011

Leave a comment

Setting up Xdebug with a remote host and Netbeans

I’ve just struggled for the last hour trying to set up xdebug remote debugging using NetBeans. Having found the solution, I’ve decided to write about the step to get this working to help anyone else out who may be stuck.

On my set up I have two machines. The development machine (that has xdebug installed) on 192.168.0.109 and my local work machine that is on 192.168.0.108. In the xdebug configuration ini, enter the following parameters;

zend_extension=/path/to/your/xdebug.so
xdebug.remote_enable=on
xdebug.remote_handler="dbgp"
xdebug.remote_host=192.168.0.108
xdebug.remote_port=9000
xdebug.idekey="netbeans-xdebug"

What may strike you as a little odd when reading it is the remote host address is my local machine. This is what caught me out. I’m not 100% sure why this has to be but it won’t work without it. After you have done this, restart your web server and make some configuration changes in NetBeans.

In NetBeans, go to Tools > Options > PHP (General Tab). In the debugging section, make sure the port is at 9000 (or whatever you specified in the config ini) and the session ID is what you set the xdebug.idekey value at (netbeans-xdebug).

Now you’re good to go!

PHP Development

xdebug

March 21, 2011

Leave a comment

Integrating Doctrine MongoDB ODM and Zend Framework

Doctrine is one of the coolest Open Source PHP database frameworks of recent years. Their MongoDB ODM project is still in Beta development but having used it recently I have to say it is really, really good. I’ve been integrating it with Zend Framework recently and have developed a handy container class that makes integration simple. The project is hosted on Google code and I will give an overview here on how to use it.

Download the latest version of my library and place the directories inside your directory. For example;

application
public
bin
library
--Doctrine
--Wildkat
--Zend

Configuration

If you look at the example application.ini, you’ll notice all configuration is placed under the ‘doctirne’ root.

There are 3 section that you need to configure (cache, connection and document managers).

; --- @ Configure Cache instances @ ---
doctrine.cache.default.driver = Doctrine\Common\Cache\ArrayCache
doctrine.cache.default.namespace = 'Wildkat'
;doctrine.cache.memcache.driver = Doctrine\Common\Cache\MemcacheCache
;doctrine.cache.memcache.servers.0.host = 127.0.0.1
;doctrine.cache.memcache.servers.0.port = 11211


; --- @ Configure connections @ ---
doctrine.connection.default.cache = default
doctrine.connection.default.host = localhost
doctrine.connection.default.port = 27017
;doctrine.connection.default.connect = true
;doctrine.connection.default.replicaSet = true
;doctrine.connection.default.persist = 'wildkat_presist'
;doctrine.connection.default.username =
;doctrine.connection.default.password =
;doctrine.connection.default.mongoCmd = $
;doctrine.connection.default.loggerClass =
;doctrine.connection.default.loggerCallback =

; --- @ Configure the document manager instances @ ---
doctrine.dms.default.connection = default
doctrine.dms.default.autoGenerateHydrationClasses = false
doctrine.dms.default.autoGenerateProxyClasses = false
doctrine.dms.default.documentNamespaces[] = Wildkat\Documents
doctrine.dms.default.hydratorDir = /tmp
doctrine.dms.default.hydratorNamespace = Wildkat
doctrine.dms.default.metadataCache = default
doctrine.dms.default.metadataDir = /tmp
doctrine.dms.default.proxyDir = /tmp
doctrine.dms.default.proxyNamespace = WildkatProxy
doctrine.dms.default.metadataDriver = AnnotationDriver
;doctrine.dms.default.defaultDb =

For detailed information on what each parameter does, consult the documentation or look around the sourcecode. Most of the ini lines are access to the setters of each component so for example autoGenerateProxyClasses = false will invoke setAutoGenerateProxyClasses(false).

Bootstrap

In your bootstrap, create a new init method that instantiates the container:

use Wildkat\Application\Container\DoctrineContainer;

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    /**
     * Loads the container
     *
     * @return Wildkat\Application\Container\DoctrineContainer
     */
    public function _initDoctrineMongoContainer()
    {
        $container = new DoctrineContainer($this->getOption('doctrine'));
        Zend_Registry::set('Wildkat\DoctrineContainer', $container);

        return $container;
    }
}

You can the obtain the container in a number of ways. You can access it via the registry by using the registry:

Zend_Registry::get('Wildkat\DoctrineContainer')

or via the bootstrap

$bootstrap->getResource('DoctrineMongoContainer');

Using the container

Once you have the container object there are three methods you’ll want to use.

$dm = $container->getDocumentManager('default');
$cache = $container->getCacheInstance('default');
$conn = $container->getConnection('default');

The ‘default’ word in each method parameter represents the sections specified in your application.ini.

Command Line Tool

Once you have everything configured, the command line tool will work with the container. The tool is located inside the bin directory. The format for invoking the cli tool is as follows;

php -q doctrinemongo.php [dm name e.g. 'default'] [cli commands]

This container is by no means complete but does covers most usage scenarios when working with MongoDB.

PHP Development

doctrine2, mognodb, zend framework

March 18, 2011

Leave a comment