We developed Picasso to help protect against situations where evaluation metrics like loss and accuracy may not tell the whole story in training neural networks on image classification tasks. At Merantix, we work with a variety of neural network architectures; we developed Picasso to make it easy to see standard visualizations across our models in our various verticals: including applications in automotive such as understanding when road segmentation or object detection fail; advertisement, such as understanding why certain creatives receive higher click-through rates; and medical imaging, such as analyzing what regions in a CT or X-ray image contain irregularities.
The Picasso visualizer tool is an open-source deep neural network visualization tool. It can be used to understand neural nets' decisions in the field of computer vision.
The scope of Picasso is definded as per below:
- Provide a web framework to connect DNNs trained in various frameworks (currently Tensorflow and Keras) to visualizations
- Handle upload of inputs (images)
-
Provide some useful default visualizations, and a frontend for navigating them:
- Class probabilities
- Occlusion of parts of objects (Zeiler, 2014)
- Saliency maps (Simonyan, 2014)
Additionally, we strive for:
- Ease of working with new models
- Ease of adding new visualizations
This is not a program for monitoring evaluation metrics or visualizating the computational graph itself. Thus, functionality like that provided in Tensorboard is out-of-scope.
Picasso is a free open-source (Eclipse Public License) web application written in Python for rendering standard visualizations useful for training convolutional neural networks. Picasso ships with occlusion maps and saliency maps, two visualizations which help reveal issues that evaluation metrics like loss and accuracy might hide: for example, learning a proxy classification task. Picasso works with the Keras and Tensorflow deep learning frameworks. Picasso can be used with minimal configuration by deep learning researchers and engineers alike across various neural network architectures. Adding new visualizations is simple: the user can specify their visualization code and HTML template separately from the application code.
For more in-depth explanation, see our Medium story or JORS submission.
Autonomous driving systems and other applications of computer vision are becoming more and more prevelant, while neural nets remain black boxes and can easily be fooled, thus the demand for a visualization tool to better understand the inner workings of NNs is becoming increasingly in demand. This is a result of a joint effort of Bosch and applied researchers at Merantix to develop a practicle application for this purpose. The development of reliable autonomous driving systems requires a lot of training and validation before they can be safely operated. Understanding the inner processes of such systems is thus an important step in the validation process.
The Eclipse Foundation creates a professionally managed environment, with relevant infrastructure such as governance, licensing and IP management, which optimally support the future development of the Picasso Visualizer. This environment facilitates the publication and development of an open source project, such as the Picasso Visualizer.
We believe these factors, and others will help provide the perfect environment for our project, as it encourages engagement, and benefits the evolution and adoption of our tool, while maintaining valuable standards.
Some plans include:
- Interactive visualizations
- RESTful API
- Segmentation visualizations
Our tool is already published.
Picasso is already in a complete, usable state. It is written in Python 3.5 using the Flask web application framework. Visualization classes and HTML templates must be defined separately by the user, but do not require modifying any other source files to use. Picasso handles the uploading of user-supplied images and generates temporary folders containing input and output images. If the visualization class has a "settings" attribute, Picasso automatically renders the settings selection as a separate page.
Application-level settings are handled via a configuration file, where the user may specify the deep learning framework (Keras or Tensorflow) as well as the location of the checkpoint files for their chosen model. The user must also supply a function to preprocess the image (reshape the image into appropriate input dimensions) and decode the output of the model (provide class labels).
Picasso has already received over 600 stars on Github, and we have merged external pull requests (with explicit assent to the Eclipse Contributor Agreement). No third-party libraries are distributed with the code, but various external open-source dependencies are downloaded through the Python Package Index. A list of these dependencies is available in the "setup.py" file at the project root.
- Log in to post comments