Practical Data Visualization with Python - Part 4: Library Deep-Dive (Plotly)¶

Setup¶

Library Deep-Dive (Plotly)¶

Having now covered some of the more theoretical components of a robust data visualization toolkit, explored the menu of package options from which one can choose, and gone through a few examples of statistical visualization problems one might face on the job, it seems apposite to round out this material with a deep-dive into a particular package. There are a few reasons for doing this with Plotly specifically:

1. Popularity of Plotly
   • Plotly is a popular suite of visualization options for python and R users alike.
2. Synergies Between Plotly and Dash
   • Plotly plays nicely with Dash--a popular python web app framework.
3. Reinforces a Key Skill: Hunting Through Documentation
   • Plotly is a vast library with documentation that varies in degrees of ease of use, so it is an excellent package with which to hone one's skills at navigating documentation to find the answer you need.

Plotly is, as of March of 2019, more than one library. I'm simplifying a bit here for the sake of concision, but there are essentially two main variants of Plotly visualizations: base Plotly, and Plotly Express. We'll dive into both here shortly, but suffice it to say that the names do a decent job of explaining what differentiates both packages.

• Base Plotly allows the user a high degree of customizability in crafting a wide array of interactive javascript-based visualizations. - Plotly Express takes much of the functionality of base Plotly, but simplifies the API a great deal at the cost of sacrificing some small degree of customizability.

Now, we'll jump into examples for both!

Plotly Express.¶

In describing Plotly Express, I don't think one can do better than the announcement post for the package:

Plotly Express is a new high-level Python visualization library: it’s a wrapper for Plotly.py that exposes a simple syntax for complex charts. Inspired by Seaborn and ggplot2, it was specifically designed to have a terse, consistent and easy-to-learn API: with just a single import, you can make richly interactive plots in just a single function call, including faceting, maps, animations, and trendlines.

My personal take then, is that Plotly Express is, simply put, Plotly's competitor to Seaborn. It is particularly well-suited for visualization problems where the chart / mark-type you need isn't very niche. Differently put, if you are looking to build a very particular type of choropleth map, for example, stick with base Plotly (or some other package). If, on the other hand, you want to make a bar chart, scatter plot, etc. (see a fuller yet not exhaustive list of potential chart types here), Plotly Express is a great tool to leverage. If you are looking for more of a walkthrough of Plotly Express, see here for that type of documentation as well. Also, note that Plotly, like most packages, is constantly evolving. As one example of this, a recently released updated for Plotly -- Plotly 4.0 -- will impact how we go about updating things like Plotly Express layouts, etc. As such, it pays to keep an ear to the ground with respect to package updates and best practices.

Finally, note that if you are trying to trouble-shoot or explore with Plotly Express, my preferred source of help is always the official Plotly Express documentation. If the docs fail you, you can of course always try ?name_of_plotly_function_here, and last but certainly the most hacky of solutions: you can make up parameters within the function that you know don't exist, as sometimes the error traceback that will result can yield yelpful insights about allowable values, etc.

Now, let's jump into a deeper exploration of Plotly Express by diving into the details of the facetted histogram we briefly went over early in this material, dissecting its various components, and also improving on / updating it.

Note: the best place to get started (aside from googling the exact question you have / reading the formal documentation) when looking for how to get an API like plotly express to do what you want is to look at the possible parameters of relevant functions / methods to see what options they take. For example, in the above viz. we ensure that the y-axis label only appears once on the facetted chart, and this is thanks to the row and col arguments. If you were looking for this functionality but didn't know what arguments to use, the best place to start hunting would be to call ?lender_fico_hist.update_xaxes. We'll dive into some further examples of these kinds of trouble-shooting techniques below.

While we've made a few improvements to the chart--axis labels, title, etc.--I still don't like the legend location (which as mentioned above is messing with the skewed sizing of the entire chart) or the facet labels. How would I go about fixing this? Well... if we go to the documentation for Plotly Express barcharts, we don't see anything about legends. Now what?

We'll, I would start by digging into the anatomy of our specific chart by calling ?lender_fico_hist, and we'll begin to see what lies beneath the Plotly Express API. We'll see some basic information about the object, for example: it is of type "Express Figure", and under "Init docstring:" we start to see how it is made.

Then, if we look a bit further down in the output, we'll start to see the two main components that make up a Plotly figure (both Express, and otherwise): the data and the layout parameters.

Everything listed under layout ... Supporting dict properties: let's you see all of the layout parameters that could be callable! There are a lot of them, for sure. Remeber, we're looking to learn about how to mess with the legend, so we scroll through these properties until we get to legend, as shown below.

From this, we can tell that there is a legend parameter that is a part of this figure's layout and that is callable as a dictionnary... but what value within that dictionnary controls the legend's location? Well, for that, one hacky but useful method I have discovered is to guess, and read the error traceback. For example, I might write the chunk of code below...

Which will ultimately throw an error containing, among other things this traceback here:

Eureka! A list of parameteres that our legend dictionnary can take, a few of which have to deal with the legend's location. We can thus apply this guess-and-check method to fix the chart as we'd like, arriving at the following layout updates below.

After all of that work, howeover, our chart labels still don't look as we'd like. As it turns out, this is one of the areas where the Plotly API, no matter how deep we go in the docstrings, doesn't afford us the flexibility we might want. As such, in order to clean up the labels for both the facets and the color legend, we'll need to transform our starting data slighty--something that is actually quite common when making a custom viz. This just goes to show that sometimes it is better to transform your data to suit the API, than to force the API to do something it wasn't designed to do.

Now that we have transformed our data to suit the constraints of our API, after having throughougly explored the various options and parameters with which we can work, let's dive into the creation of the final viz.

After all of this, one final thought you might have could be: isn't it redundant to have the facet labels say the exact same thing as the legend? Yes! The tough part about that is that there simply isn't a parameter or an update method as of yet that exists to deal with hiding these labels. This isn't something that is intuitive, either. Rather, after a long journey through the docs, I eventually went with the nuclear option (not recommended) and simply tweeted at the head of product for Plotly about this, and he very graciously responded, as shown below. As Nicolas shows us, the only way to deal with this is by dropping all annotations (for now; an update method is in the works).

The lesson here is not to tweet at package devs whenever you encouter a problem, but rather to build a community around you to whom you can turn for answers when you hit a particularly hard problem.

Finally, we can put all the code together into a final version of the figure with all of our improvements made in a single cell, as shown below.

Base Plotly.¶

Now that we've explored the wonders of Plotly Express, a logical question would be: why bother returning to base Plotly given all the functionality of Plotly Express? Well, there are two mains answers to that question:

• Access to certain figure types that are non achievable (yet, to my knowledge) with Plotly Express
  • Some figure varieties are so domain-specific or otherwise niche that there simply isn't yet a Plotly Express API built to produce that particular variety of viz, for example, the quiver plot
  • The example we'll go over pertaining to this justification entails exploring the creation of distribution plots (aka distplots) with Plotly's figure factory, where chart types go prior to being integrated into the broader Plotly.js library of plot types.
• Less abstraction in syntax = more granular control over all details of the viz
  • Think about how, as we went through the Plotly Express examples, we had to dig into the skeletal structure of the plot (Data and Layout) in order to find what parameters we might be able to pass to an update() method call to get our desired plot. With base plotly, we start out at this level of detail, which is both more work, but also more all-encompassing from the get-go.
  • The example we'll go over pertaining to this justification entails creating a custom heatmap.

Note of Clarification: When mentioning "base Plotly", I am referring to everything in the Plotly ecosystem that is not Plotly Express. Generally, this entails Plotly "Graphical Objects" (aka "Go") and the aforementioned Plotly "Figure Factory" (aka "FF"). If Plotly Express is mostly used as the quickest way to interact with Plotly visualizations, you can think of Go as the largest overarching granular framework for making most any plot type with Plotly, and Figure Factory as a more experimental branch of the Plotly API, as previously described.

With all that context in mind, let's jump into exploring these aforementioned example use cases for base Plotly, below.

Now that we've seen an example of the utility of base Plotly in building niche chart types via Plotly's Figure Factory, let's move to the other main use case of base Plotly: very granular control over your visualization. For this, we'll turn to Plotly Go figures.

In this example we'll be able to see most clearly how Plotly works on the back end, creating a figure by combining a dictionnary specifying what data are to be included, and what layout is to be used.

Let's imagine a business scenario where we want to see how much UPB (origination amount) we have concentrated across different LTV and FICO cohorts. Do we have large pockets of UPB at any modal points for FICO or LTV? We want a visualization to help us quickly identify these UPB hotspots across these two axes. There are many visuals we could build for this, but for this vignette, we'll turn to a heatmap (aka Contour Plot).

We'll start by preparing our data by rounding our FICO and LTV data (mainly for speed, given that super-precise granularity isn't important for a chart like this). Then we'll calculate UPB concentration within each FICO and LTV bucket, and finally, we'll build the heatmap. Let's dig in!

The last preparatory step we'll want to take care of prior to building the heatmap is to figure out how we want our colors to scale. Plotly will automatically pick a color gradient for the heatmap if we don't pre-specify one, but seeing as Plotly gives us the power to control the gradient, let's take advantage of that functionality!

Hope You Enjoyed!¶

It's my hope that over the course of this seminar (or just by reading these materials) you picked up some fresh insights into the theory and techniques behind practical data visualization. If you're looking for more material, check out the homework in this repository, and feel free to reach out with any feedback!

Thank you, and happy visualizing!