The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies




  • Paper topic: Images
  • Software type: Code
  • Able to run a replicability test: True
  • Replicability score: 4
  • Software language: Python
  • License: MIT
  • Build mechanism: Not applicable (python, Matlab..)
  • Dependencies: Caffe
  • Documentation score {0,1,2}: 2
  • Reviewer: Nicolas Mellado <>
  • Time spent for the test (build->first run, timeout at 100min): 50min

Source code information


The github repsitory provides two scripts, which both require to download the" sketch and photos"dataset (
- Retrieval_Example.ipynb, which demonstrates how to retrieve images from a sketch. I had to flatten the content of the directory "256x256/photo/tx_00000000000" to be able to load in the photo database (8th code block in jupyter notebook).
- Compute_Sketchy_score.ipynb, which compute the network matching accuracy. I tried to flatten the directory "256x256/sketch/tx_00000000000", and replace the use of the file test_img_list.txt (which is not provided) by test_img_list = os.listdir(photo_paths). Unfortunatly, even with this change, I couldn't run the script, and I couldn't find a way to format the data as required by the last code block. No instruction is given on how to modify or adapt file loading and directory structure.

Bug fixes (for both scripts):
 - scripts use python2 syntax for the print function (ie. without parenthesis). Can be adapted to python3 by using parenthesis.
 - scripts attempt to load LSHForest from sklearn.neighbors, which is deprecated and not provided by modern sklearn packages. Replace 
from sklearn.neighbors import NearestNeighbors,LSHForest
from sklearn.neighbors import NearestNeighbors

I have tested both scripts using caffe_cpu (see 4th code block in both scripts: caffe.set_mode_cpu()).

The provided scripts does not demonstrate how to retrieve sketches from an image. The broken script does not demonstrate a functionality of the approach but rather its performances, and so its failure does not impact the replicability of the approach.

If you want to contribute with another review, please follow these instructions.

Please consider to cut/paste/edit the raw JSON data attached to this paper.