INRIA's Fisher vector implementation

What is this?

This is a Matlab implementation of the Fisher processing chain used in

Aggregating local image descriptors into compact codes Hervé Jégou; Florent Perronnin; Matthijs Douze; Jorge Sánchez; Patrick Pérez; Cordelia Schmid, PAMI 2011.
Combining attributes and Fisher vectors for efficient image retrieval, Matthijs Douze; Arnau Ramisa; Cordelia Schmid, CVPR 2011.

It is INRIA's implementation, as opposed to that of Xerox, which is closed-source.

How to use

You will need:

Matlab on Linux or Mac OS X (making it work with Octave requires some work).
A recent version of Yael.
This package.
These image descriptors for the Holidays dataset
These datafiles (centroid files, PCA matrices).
If you want to compute SIFT descriptors for your own images, you will need compute_descriptors (linux and mac versions). This page explains how to preprocess images before extracting descriptors.

On 64-bit Linux, if everything works well, the following steps should be enough:

http://lear.inrialpes.fr/src/inria_fisher/inria_fisher_v1.tgz
tar xvzf inria_fisher_v1.tgz
cd inria_fisher_v1
wget http://lear.inrialpes.fr/src/inria_fisher/inria_fisher_data_v1.tgz
tar xvzf inria_fisher_data_v1.tgz
wget http://pascal.inrialpes.fr/data/holidays/siftgeo.tar.gz
tar xzf siftgeo.tar.gz
mkdir -p yael/matlab
cd yael/matlab
wget https://gforge.inria.fr/frs/download.php/30399/yael_matlab_linux64_v277.tar.gz
tar xvzf yael_matlab_linux64_v277.tar.gz 
cd ../..

Then run test_fisher in Matlab. This computes Fisher descriptors (k=64) for the Holidays dataset (from the local descriptors in siftgeo/). Then it performs exact NN-searches with the L2 distance on the Holidays query images. After about 10 minutes, it should display:

Fisher k=64                     4096D   mAP = 0.599
Fisher + PCA (D'=128)            128D   mAP = 0.561

These numbers correspond to the 59.5 and 56.5 figures in Table I of the PAMI paper (the implementation is not exactly the same).

The PQ compression is not implemented in this version. GMM learning is implemented in Yael but not interfaced in Matlab. Dense descriptors local are implemented in compute_descriptors (-dense option) but no corresponding GMM is provided yet. You may be interested in this package, that also implements compression with PQ codes (on different descriptors).

What to cite

If you use the package in a publication, please cite

@article{JEGOU-2011-633013,
    url = {http://hal.inria.fr/inria-00633013},
    title = {{Aggregating local image descriptors into compact codes}},
    author = {J{\'e}gou, Herv{\'e} and Perronnin, Florent and Douze, Matthijs and S{\'a}nchez, Jorge and P{\'e}rez, Patrick and Schmid, Cordelia},
    publisher = {IEEE},
    journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
    year = {2011},
    pdf = {http://hal.inria.fr/inria-00633013/PDF/jegou\_aggregate.pdf},
}

Contact, license

If the package does not work, you may complain to

matthijs dot douze at inria dot fr.

It is licensed under Cecill, similar to the GPL. Please note that some techniques (eg. SIFT and Fisher) may be covered by non-INRIA software patents.

Last modification on 2012-04-09 by Matthijs Douze.