How to run Django's test database only in memory?


Question

My Django unit tests take a long time to run, so I'm looking for ways to speed that up. I'm considering installing an SSD, but I know that has its downsides too. Of course, there are things I could do with my code, but I'm looking for a structural fix. Even running a single test is slow since the database needs to be rebuilt / south migrated every time. So here's my idea...

Since I know the test database will always be quite small, why can't I just configure the system to always keep the entire test database in RAM? Never touch the disk at all. How do I configure this in Django? I'd prefer to keep using MySQL since that's what I use in production, but if SQLite 3 or something else makes this easy, I'd go that way.

Does SQLite or MySQL have an option to run entirely in memory? It should be possible to configure a RAM disk and then configure the test database to store its data there, but I'm not sure how to tell Django / MySQL to use a different data directory for a certain database, especially since it keeps getting erased and recreated each run. (I'm on a Mac FWIW.)

1
116
11/10/2015 6:47:23 PM

Accepted Answer

If you set your database engine to sqlite3 when you run your tests, Django will use a in-memory database.

I'm using code like this in my settings.py to set the engine to sqlite when running my tests:

if 'test' in sys.argv:
    DATABASE_ENGINE = 'sqlite3'

Or in Django 1.2:

if 'test' in sys.argv:
    DATABASES['default'] = {'ENGINE': 'sqlite3'}

And finally in Django 1.3 and 1.4:

if 'test' in sys.argv:
    DATABASES['default'] = {'ENGINE': 'django.db.backends.sqlite3'}

(The full path to the backend isn't strictly necessary with Django 1.3, but makes the setting forward compatible.)

You can also add the following line, in case you are having problems with South migrations:

    SOUTH_TESTS_MIGRATE = False
155
2/17/2019 11:05:03 AM

I usually create a separate settings file for tests and use it in test command e.g.

python manage.py test --settings=mysite.test_settings myapp

It has two benefits:

  1. You don't have to check for test or any such magic word in sys.argv, test_settings.py can simply be

    from settings import *
    
    # make tests faster
    SOUTH_TESTS_MIGRATE = False
    DATABASES['default'] = {'ENGINE': 'django.db.backends.sqlite3'}
    

    Or you can further tweak it for your needs, cleanly separating test settings from production settings.

  2. Another benefit is that you can run test with production database engine instead of sqlite3 avoiding subtle bugs, so while developing use

    python manage.py test --settings=mysite.test_settings myapp
    

    and before committing code run once

    python manage.py test myapp
    

    just to be sure that all test are really passing.


Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Icon