Yorg the image organizer


Yorg is a photo management tool that includes some image analysis modules to assist the user organizing her data. This document is a short user manual of the software. The less useful functions are not described.

1. Before

Download the software from http://lear.inrialpes.fr/src/yorg/
 
Install the software according to the instructions in the README file.

Launch the program from its main directory with

source setvars.bash
python organizer_gui.py

2. Windows

There are two main windows.

2.1 Database window

This window is visible at startup.

To add images to the collection (= database), use the Database > Add images... menu and select a directory (not a set of images).

Database window

The left pane shows the images of the current database as thumbnails. The pane at the right shows different organization options. The size slider resizes the thumbnails.

Images can be selected by clicking or drawing a rectangle around them. The displayed images can be restricted to a subset of the database with the work on selection button (work on all switches back to the whole collection).

The file menu has the usual items. The image collections are stored in a directory rather than a file. By convention, the extension of the directory name is .iorg. The collection's images themselves are not stored, so if they are moved somewhere else, they will be lost when the collection is re-opened.

2.2 Selection window

Open this window via Selection > Selection window.

Image window

It displays information about the selected image(s). There are several tabs showing different types of information.

Click on in GIMP to open the image(s) in the GIMP.

3. Information about images & organization modes

Information can be extracted from each image in the collection. A series of analysis modules perform more or less complicated analysis taks on an image. The results, image attributes, are shown in the selection window.

In the database window, the images can be organized according to one or several attributes. The attribute used is selected in the by: dropdown menu on the right, which shows the appropriate pane.

The attributes are computed when needed, but this often introduces tedious delays in the GUI. Therefore, you can explicitly request an analysis of the whole collection via the Database > Analyze... menu items and go for a cup of tea while the computer works.

3.1 File information

This is the most basic organization type, based only on the image file attributes (name, size, extension, etc.). Clicking on a radio menu item immediately orders the collection according to the selected attribute.

vrac
means "without a particular order" (but it at least reflects the order in which the images were added).

Here, images are ordered by file size:

Organize by file size

3.2 Exif information

JPEG images generated by most digital cameras contain EXIF meta-information.

For example, you can order the images by the date they were shot:

order by date

3.3 Tags

You can manually annotate images with tags (=keywords).

3.3.1 Defining and assigning tags

Open the selection window to define or add tags to images.

Create a new tag with the new... button. Enter its name (without spaces) in the dialog box.

The defined tags appear in the first column of a table (the tag list). The which column indicates how many among the selected images have this tag.

You can assign tags to the currently selected image(s) in two ways:
Tags can be un-assigned from the selected images with the - button.

3.3.2 Organizing with tags

You can use tacgs to select the images to display in the database window.

To display images having a given tag, double-click on that tag in the database window's tag list.

Select several tags (shift-click) in the list and click on the and or or button to compute a boolean combination of tags. Here, the images having both the evening and outside tags are displayed:

evening and outside

3.3.3 Tips

If you need complex boolean combinations of tags, use re-tagging. For example:
Use tags to keep selections or search results from the other modules.

3.4 Color

Yorg extracts global color histograms from the images. They are full 3D histograms with 10*10*10 bins. A reduced version of the histogram, as separate R, G and B histograms can be displayed in the color tab of the selection window:

view histogram

In the database window, the images can be ordered by luminance, hue or saturation of their average color.

When a reference image is selected, you may order the rest of the collection by increasing color distance to this image:

order by similarity (color)

The color distance used is the Earth Mover Distance.

You can also arrange the images in a diagram that attempts to group close images (in terms of color distance). All pairwise distances must be computed (slow!). This is done with a Multi Dimensional Scaling algorithm (MDS):

MDS color

Since there is no spatial information in the color histograms and because of the very relative way humans perceive colors, the ordering based only on a dominant color often seems non-intuitive. Representations based on distances between color histograms are more useful.

This part is inspired and based on EMD code by Yossi Rubner (Y. Rubner, L. Guibas, C. Tomasi, The Earth Mover's Distance, Multi-Dimensional Scaling and Color-Based Image Retrieval, ARPA Image Understanding Workshop, 1997).

3.5 Geometry

The images are analyzed to extract keypoints. The keypoints can be matched to find correspondances between images.

3.5.1 Viewing correspondances

The keypoints tab of the selection window displays the keypoints of an image.

view keypoints 

The blue parallelograms indicate a region used to compute the keypoint's signature.

When two images are selected, the tab displays the matched keypoints:

Matching keypoints

By default, matches are filtered to show only the ones that are geometrically coherent (e.g. there must be an affine 2D transform that maps the set of points on one image onto the set on the second). Ticking the raw option shows correspondances without this geometric check.

3.5.2 Image distances

The number of matching keypoints can be interpreted as a "geometrical distance" between images. The smaller this distance, the more likely it is that the images show the same scene.

You can group similar images together in the database window according to this distance:

Grouping of similar images

You can also use the distance to make an MDS diagram:

MDS with geom distances

3.6 Person detection

Yorg can detect persons on the images. The detector works best on upright and entirely visible persons.

You can see detected persons in the persons tab of the selection window:

Person detection

The number in the corner of the detected bounding box is a confidence value.

In the collection window, you can order the images by the number of detected persons:

by nb of persons

3.7 Texture

3.7.1 Recognizing textures

Yorg is able to recognize textured zones in the images. Given a kind of texture to recognize, it extracts patches from each image and evaluates whether the patch contains that texture.

You can choose the texture of interest in the texture bar of the collection window.

The patches defined in an image are displayed in the texture tab of the selection window (a foliage texture in this case):

imageshow patches foliage

Each patch has a shade between red and green, green indicating that the current texture is recognized in the patch.

3.7.2 Classification based on textures

You can order the collection of images by the amout of the texture they contain. For example, this is the collection ordered by the amount of foliage:

order by foliage content

The figures (between 0 and 1) in the corners of the images indicate their amount of texture. By default, for speed reasons, clicking on organize only handles a subset of the database (30 images). Checking the all box orders the whole database.

3.7.3 User-defined texture classifiers

You can define new texture classifiers by indicating positive and negative image regions. Let's assume that you want to recognize a sea texture:
In the texture list, the second and third columns indicate how many regions are defined. (1+2m = 1 full image + 2 images with drawn regions). The fourth column has a "*" if a classifier was trained.

After training, the texture classifier can be used like the predefined ones.

3.7.4 Tips

Do not expect too much from this image analysis: it recognizes only simple textures. Selecting cars or faces will not learn an object detector.

Very often, there are more negative training examples than positive ones. This explains why the image patches are mostly red. This is not a problem however, since only relative amounts of texure coverage are used.

The learning time and storage is grows with the number of example patches and thus images. To avoid too big an overhead, do not use more than 4 to 6 training images.

3.8 Refinement

The attributes extracted by the previous modules can be combined to quickly find images from a few examples. To do this, go to the refine pane in the database window.

Select a few images you like and click on positive. Select a few irrelevant images and click on negative. The pane shows something like 3p + 4n, which means 3 positive images and 4 negative ones.

Click on organize. Here again, only a subset of images is organized by default (unless all is checked). Yorg allows asks what attributes of the images should be used:

attrs

Attribute names should be self-explanatory: exif_* refers to exif information, nperson and npixel to the number of persons and pixels, and tex_* are the texture classifier results. You can uncheck the attributes you think are not relevant, or too slow to compute, etc.

This is an example result where we search for rocks next to the sea (the figures are confidence values):

rock next to sea, first guess

The 6th image is wrong, so select it and click on negative and organize. The second classification is much faster.

second guess

After a few refinement iterations, you can classify the whole set:

final

If you want to cancel the whole process, click on clear.

This classifier uses an SVM and has many integer or boolean attributes. Therefore, adding similar images to the positive or negative sets may not change the ordering at all (the support vectors are the same). Refrain from adding only negative images, the positive and negative sets should remain balanced.

4. Miscelaneous

A lot of information is written to the console. If there is a Python exception (which should not happen, of course), it is caught by the GUI, which does not exit: it just prints a stacktrace to the console. If you want, you can send
me this information as a bug report. Of course, C and C++ errors crash the application.

4.1 Tag import/export

Tags can also be imported and exported with Database > load tags and Database > save tags. They are stored as a serialized (= "pickled") Python data structure:

stored=(tag_list,tag_dict)
tag_list= all defined tags, as a list of strings
tag_dict= dictionary that maps an image filename to a list of tags associated with the image.

4.2 Built-in debugger

The menu item Developer > debugger opens a python debugger command line which allows to examine data, set breakpoints, etc.