We developed Picasso to help protect against situations where evaluation metrics like loss and accuracy may not tell the whole story in training neural networks on image classification tasks. At Merantix, we work with a variety of neural network architectures; we developed Picasso to make it easy to see standard visualizations across our models in our various verticals: including applications in automotive such as understanding when road segmentation or object detection fail; advertisement, such as understanding why certain creatives receive higher click-through rates; and medical imaging, such as analyzing what regions in a CT or X-ray image contain irregularities.
The Picasso visualizer tool is an open-source deep neural network visualization tool. It can be used to understand neural nets' decisions in the field of computer vision.
The scope of Picasso is definded as per below:
- Provide a web framework to connect DNNs trained in various frameworks (currently Tensorflow and Keras) to visualizations
- Handle upload of inputs (images)
Provide some useful default visualizations, and a frontend for navigating them:
- Class probabilities
- Occlusion of parts of objects (Zeiler, 2014)
- Saliency maps (Simonyan, 2014)
Additionally, we strive for:
- Ease of working with new models
- Ease of adding new visualizations
This is not a program for monitoring evaluation metrics or visualizating the computational graph itself. Thus, functionality like that provided in Tensorboard is out-of-scope.
Picasso is a free open-source (Eclipse Public License) web application written in Python for rendering standard visualizations useful for training convolutional neural networks. Picasso ships with occlusion maps and saliency maps, two visualizations which help reveal issues that evaluation metrics like loss and accuracy might hide: for example, learning a proxy classification task. Picasso works with the Keras and Tensorflow deep learning frameworks. Picasso can be used with minimal configuration by deep learning researchers and engineers alike across various neural network architectures. Adding new visualizations is simple: the user can specify their visualization code and HTML template separately from the application code.
For more in-depth explanation, see our Medium story or JORS submission.
Autonomous driving systems and other applications of computer vision are becoming more and more prevelant, while neural nets remain black boxes and can easily be fooled, thus the demand for a visualization tool to better understand the inner workings of NNs is becoming increasingly in demand. This is a result of a joint effort of Bosch and applied researchers at Merantix to develop a practicle application for this purpose. The development of reliable autonomous driving systems requires a lot of training and validation before they can be safely operated. Understanding the inner processes of such systems is thus an important step in the validation process.
The Eclipse Foundation creates a professionally managed environment, with relevant infrastructure such as governance, licensing and IP management, which optimally support the future development of the Picasso Visualizer. This environment facilitates the publication and development of an open source project, such as the Picasso Visualizer.
We believe these factors, and others will help provide the perfect environment for our project, as it encourages engagement, and benefits the evolution and adoption of our tool, while maintaining valuable standards.
Our tool is already published.
Some plans include:
- Interactive visualizations
- RESTful API
- Segmentation visualizations