Monday, February 4, 2013

GeoDjango + Spatialite (SQLite3) Setup Issues

Because there are fresh ones every time I start a new project!

As of May 03 2013 I have opted to install virtualenv --system-site-packages and running aptitude-install python-pysqlite2 spatialite-bin gdal-bin; the site are not as self contained any more but sure beats fighting with the issues.

All my Django projects are self contained (as much as possible) with Virtualenv and use Spatialite database (SQLite3 +geo spatial stuff) installed so to enable GeoDjango functionality. This post is a list of issues I encounter related to that setup and how I resolved them; as I seem to run into a new one on every project start. And as always if you found this useful do +1, or follow me on twitter @danielsokolows

/usr/include/spatialite.h:61: note: previous declaration of ‘spatialite_init’ was here

Say what? --- this one threw me for a three hour fun time. It happened because I executed 'pip install pyspatialite' will install the 3.X vesion which is is broken and craps out with:

setup.py", line 97, in get_amalgamation 
        download_file = pattern.findall(download_page)[0] 
    IndexError: list index out of range

So then I tried an older 'pip install pyspatialite==2.6.2-spatialite.2.4.0-4' which results in:

...
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DMODULE_NAME="spatialite.dbapi2" -DVERSION="3.0.1" -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 -DOMIT_FREEXL=1 -I/Library/Frameworks/GEOS.framework/unix/include/ -I/Library/Frameworks/PROJ.framework/unix/include/ -Iamalgamation -I/usr/include/python2.6 -c src/module.c -o build/temp.linux-i686-2.6/src/module.o
In file included from src/module.c:24:src/connection.h:36: error: conflicting types for ‘spatialite_init’
/usr/include/spatialite.h:61: note: previous declaration of ‘spatialite_init’ was here
error: command 'gcc' failed with exit status 1
----------------------------------------
Command /srv/www/django/development..com/virtualenv/bin/python -c "import setuptools; __file__='/srv/www/django/development..com/virtualenv/build/pyspatialite/setup.py'; execfile('/srv/www/django/development..com/virtualenv/build/pyspatialite/setup.py')" install --single-version-externally-managed --record /tmp/pip-vq4xsp-record/install-record.txt --install-headers /srv/www/django/development..com/virtualenv/include/site/python2.6 failed with error code 1
Storing complete log in /root/.pip/pip.log

Which after pulling hair out I finally realized is because pip is caching the previous partial install and does not attempt a new download. To resolve this (clearing pip cache did nothing as there was no '~/.pip/cache' to clear) install another package and uninstall it and then try again:

(virtualenv)root@klisrv03:development..com>$  pip install pysqlite
Downloading/unpacking pysqlite
  Downloading pysqlite-2.6.3.tar.gz (74Kb): 74Kb downloaded
 ...
Successfully installed pysqlite
Cleaning up...
(virtualenv)root@klisrv03:development..com>$  pip uninstall pysqlite
Uninstalling pysqlite:
 ...
Proceed (y/n)? y
  Successfully uninstalled pysqlite
(virtualenv)root@klisrv03:development..com>$  pip install pyspatialite==2.6.2-spatialite.2.4.0-4
...
Successfully installed pyspatialite
Cleaning up..

That was easy and obvious!

ImproperlyConfigured: Only versions of pysqlite 2.5+ are compatible with SpatiaLite and GeoDjango.

EASY! Just run 'pip install pysqlite' to get a newer version then what your current python stack includes...not so fast as then you get:

ImproperlyConfigured: The pysqlite library does not support C extension loading. Both SQLite and pysqlite must be configured to allow the loading of extensions to use SpatiaLite.

The solution is to try installing from source as per django docs or using something like 'pip install pyspatiallite==2.6.2-spatialite.2.4.0-4'

Still getting: ImproperlyConfigured: Only versions of pysqlite 2.5+ are compatible with SpatiaLite and GeoDjango then make sure to patch in support as per GeoDjango missing pyspatialite support or grab my github-danielsokolowski-django patched version.

error: geos_c.h: No such file or directory

   gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DMODULE_NAME="spatialite.dbapi2" -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 -I/Library/Frameworks/GEOS.framework/unix/include/ -I/Library/Frameworks/PROJ.framework/unix/include/ -Iamalgamation -I/usr/include/python2.6 -c amalgamation/spatialite.c -o build/temp.linux-i686-2.6/amalgamation/spatialite.o
    amalgamation/spatialite.c:94:20: error: geos_c.h: No such file or directory

To resolve run aptitude install libgeos-dev.

error: proj_api.h: No such file or directory

    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DMODULE_NAME="spatialite.dbapi2" -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 -I/Library/Frameworks/GEOS.framework/unix/include/ -I/Library/Frameworks/PROJ.framework/unix/include/ -Iamalgamation -I/usr/include/python2.6 -c amalgamation/spatialite.c -o build/temp.linux-i686-2.6/amalgamation/spatialite.o
    amalgamation/spatialite.c:98:22: error: proj_api.h: No such file or directory

Execute aptitude install libproj-dev.

error: python.h: No such file or directory

Execute aptitude install python-dev.

src/connection.h:33:21: fatal error: sqlite3.h: No such file or directory

Execute aptitude install libsqlite3-dev.

src/connection.h:34:24: fatal error: spatialite.h: No such file or directory

Execute aptitude install libspatialite-dev.

ImportError: cannot import name SpatialReference

You are missing the required for Spatialite + Geodjango GDAL library, install it by running aptitude install gdal-bin

2 comments:

  1. i did the last steps and i stil get

    sudo apt-get install python-dev libsqlite3-dev libspatialite-dev gdal-bin
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    libsqlite3-dev is already the newest version.
    python-dev is already the newest version.
    python-dev set to manually installed.
    libspatialite-dev is already the newest version.
    libspatialite-dev set to manually installed.
    Suggested packages:
    python-gdal
    The following NEW packages will be installed:
    gdal-bin
    0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
    Need to get 221 kB of archives.
    After this operation, 639 kB of additional disk space will be used.
    Do you want to continue [Y/n]? Y
    Get:1 http://archive.ubuntu.com/ubuntu/ saucy/universe gdal-bin i386 1.9.0-3.1ubuntu4 [221 kB]
    Fetched 221 kB in 0s (523 kB/s)
    Selecting previously unselected package gdal-bin.
    (Reading database ... 291756 files and directories currently installed.)
    Unpacking gdal-bin (from .../gdal-bin_1.9.0-3.1ubuntu4_i386.deb) ...
    Processing triggers for man-db ...
    Setting up gdal-bin (1.9.0-3.1ubuntu4) ...
    vivi@vivi-X55SV:~/libosmpbf/osm-history-splitter/osm-history-renderer/importer$ make
    g++ -g -O3 -Wall -Wextra -pedantic `getconf LFS_CFLAGS` -I/usr/include/postgresql/ -I/usr/include/libxml2/ -DOSMIUM_WITH_GEOS -o osm-history-importer importer.cpp -lexpat -lpq -lproj -lz -lprotobuf-lite -losmpbf -lpthread -lgeos
    In file included from geombuilder.hpp:11:0,
    from handler.hpp:28,
    from importer.cpp:25:
    project.hpp:4:22: fatal error: proj_api.h: No such file or directory
    #include
    ^
    compilation terminated.
    make: *** [osm-history-importer] Error 1
    vivi@vivi-X55SV:~/libosmpbf/osm-history-splitter/osm-history-renderer/importer$

    ReplyDelete
    Replies
    1. Please read the blog again, the error is addressed and the solution is to run 'aptitude install libproj-dev'.

      Delete