Interactive dashboard with Python

Using CO2 Emissions and Climate Change data

By Mundy Jim, 24.05.2022


  1. Introduction
  2. Installing Packages and Data upload
  3. Data Cleansing
  4. Continets CO2 emission over time
  5. CO2 Emission Table over time
  6. CO2 sources of top 10 Manufacturing countries
  7. The Interactive Dashboard
  8. Conclusion

You can download the updated notebook from my Github page

1. Introduction¶

This project creates a dashboard to show the CO2 emissions using Python with with Panel and hvplot. The data is obtained from Our World in Data. Reference is made to Statistica for Top 10 countries by share of global manufacturing output in 2020 .

I found Panel, a high-level app and dashboarding framework for Python very interesting and useful. Panel is a fully open source Python library which enables users to create interactive, web-based data dashboards. It connects user widgets to plots, images, tables, and text. It works with IPython notebook files which can leverage the Jupyter server or can be used with a normal web server, using code. I give credit to Sophia Yang for the inspiration. Out of curiosity, I decided to try it out as a personal project to see the effects of CO2 emission in different countries/continents. I filter out the data a little bit more using Pandas and somehow separated the data into two different dataframes:

Objective:¶

Create an Interactive dashboard with Panel and Python to display and filter¶
  1. The CO2 reading of six continents: Africa, Asia, Europe, North America, South America and Oceania
  2. Top 10 manufaturing countries by share of global manufacturing output in (2020) (interacting with their CO2 emission)

Ultimately we shall get a deplyable interactive dashboard as a local host exactly as the image shown below in case you decide to use my notebook. Ofcourse exploring the dataset might give anyone a different analysis mindset. Within the actual local host at the end, we can easily see the interactivity with the slider adjustment in the side panel or by tuggling between the desired CO2widgets.

Interactive dashboard

2. Installing Packages and Data upload¶

We need Panel, Pandas, Numpy and Hvplot. Hvplot is a high-level plotting API for the PyData ecosystem built on Holoviz which we can use to quickly generate interactive plots from our data. First we import the csv dataset and take a quick look. The columns of the first 5 rows show different CO2 sources, population, GDP, energy per capita, etc for different countries/continents.

In [128]:
import pandas as pd # DataFrame, Series (columnar/tabular data)
import numpy as np # for arrays and matrices i/o
import panel as pn
pn.extension('tabulator')

import hvplot.pandas # Hvplotis high-level plotting API for the PyData ecosystem built on HoloViews.
In [129]:
df = pd.read_csv('https://www.obilorjim.com/wp-content/uploads/2022/05/covid-co2-data.csv')
print('Data downloaded and read into a dataframe!')
# Check the link in Introduction for an optional data download source
Data downloaded and read into a dataframe!
In [130]:
# cache data to improve dashboard performance
if 'data' not in pn.state.cache.keys():
    df = pd.read_csv('https://www.obilorjim.com/wp-content/uploads/2022/05/covid-co2-data.csv')
    pn.state.cache['data'] = df.copy()
else: 
    df = pn.state.cache['data']
In [131]:
df.head() # view first 5 rows
Out[131]:
iso_code country year co2 co2_per_capita trade_co2 cement_co2 cement_co2_per_capita coal_co2 coal_co2_per_capita ... ghg_excluding_lucf_per_capita methane methane_per_capita nitrous_oxide nitrous_oxide_per_capita population gdp primary_energy_consumption energy_per_capita energy_per_gdp
0 AFG Afghanistan 1949 0.015 0.002 NaN NaN NaN 0.015 0.002 ... NaN NaN NaN NaN NaN 7624058.0 NaN NaN NaN NaN
1 AFG Afghanistan 1950 0.084 0.011 NaN NaN NaN 0.021 0.003 ... NaN NaN NaN NaN NaN 7752117.0 9.421400e+09 NaN NaN NaN
2 AFG Afghanistan 1951 0.092 0.012 NaN NaN NaN 0.026 0.003 ... NaN NaN NaN NaN NaN 7840151.0 9.692280e+09 NaN NaN NaN
3 AFG Afghanistan 1952 0.092 0.012 NaN NaN NaN 0.032 0.004 ... NaN NaN NaN NaN NaN 7935996.0 1.001733e+10 NaN NaN NaN
4 AFG Afghanistan 1953 0.106 0.013 NaN NaN NaN 0.038 0.005 ... NaN NaN NaN NaN NaN 8039684.0 1.063052e+10 NaN NaN NaN

5 rows × 60 columns

We can have a better overview of the data column names. Different kinds of analysis to think of but we are interested in a few of the columns.

In [132]:
df.columns
Out[132]:
Index(['iso_code', 'country', 'year', 'co2', 'co2_per_capita', 'trade_co2',
       'cement_co2', 'cement_co2_per_capita', 'coal_co2',
       'coal_co2_per_capita', 'flaring_co2', 'flaring_co2_per_capita',
       'gas_co2', 'gas_co2_per_capita', 'oil_co2', 'oil_co2_per_capita',
       'other_industry_co2', 'other_co2_per_capita', 'co2_growth_prct',
       'co2_growth_abs', 'co2_per_gdp', 'co2_per_unit_energy',
       'consumption_co2', 'consumption_co2_per_capita',
       'consumption_co2_per_gdp', 'cumulative_co2', 'cumulative_cement_co2',
       'cumulative_coal_co2', 'cumulative_flaring_co2', 'cumulative_gas_co2',
       'cumulative_oil_co2', 'cumulative_other_co2', 'trade_co2_share',
       'share_global_co2', 'share_global_cement_co2', 'share_global_coal_co2',
       'share_global_flaring_co2', 'share_global_gas_co2',
       'share_global_oil_co2', 'share_global_other_co2',
       'share_global_cumulative_co2', 'share_global_cumulative_cement_co2',
       'share_global_cumulative_coal_co2',
       'share_global_cumulative_flaring_co2',
       'share_global_cumulative_gas_co2', 'share_global_cumulative_oil_co2',
       'share_global_cumulative_other_co2', 'total_ghg', 'ghg_per_capita',
       'total_ghg_excluding_lucf', 'ghg_excluding_lucf_per_capita', 'methane',
       'methane_per_capita', 'nitrous_oxide', 'nitrous_oxide_per_capita',
       'population', 'gdp', 'primary_energy_consumption', 'energy_per_capita',
       'energy_per_gdp'],
      dtype='object')

3. Data Cleansing and preparation¶

  • Replace missing values with zeroes
  • Calculate GDP per Capita where it does not exist
  • Drop ISO_code column for easier data filtering
  • Create a new column 'gdp per capita'. Note that GDP/Capita = Real GDP/Population
  • Export dataframe as a new csv file to reference externally ina spreadsheet
  • Change the column 'country' to 'place' as the column contains both countries and continents
In [133]:
df = df.fillna(0)# Replace missing values with zeroes
df = df.rename(columns = {'country':'Place'})
df = df.drop(['iso_code'] , axis=1)
df["Place"].replace({"United States": "US", "United Kingdom": "UK"}, inplace=True)
df['gdp_per_capita'] = np.where(df['population']!= 0, df['gdp']/ df['population'], 0)
In [134]:
df.head() # View 5 rows
Out[134]:
Place year co2 co2_per_capita trade_co2 cement_co2 cement_co2_per_capita coal_co2 coal_co2_per_capita flaring_co2 ... methane methane_per_capita nitrous_oxide nitrous_oxide_per_capita population gdp primary_energy_consumption energy_per_capita energy_per_gdp gdp_per_capita
0 Afghanistan 1949 0.015 0.002 0.0 0.0 0.0 0.015 0.002 0.0 ... 0.0 0.0 0.0 0.0 7624058.0 0.000000e+00 0.0 0.0 0.0 0.000000
1 Afghanistan 1950 0.084 0.011 0.0 0.0 0.0 0.021 0.003 0.0 ... 0.0 0.0 0.0 0.0 7752117.0 9.421400e+09 0.0 0.0 0.0 1215.332543
2 Afghanistan 1951 0.092 0.012 0.0 0.0 0.0 0.026 0.003 0.0 ... 0.0 0.0 0.0 0.0 7840151.0 9.692280e+09 0.0 0.0 0.0 1236.236369
3 Afghanistan 1952 0.092 0.012 0.0 0.0 0.0 0.032 0.004 0.0 ... 0.0 0.0 0.0 0.0 7935996.0 1.001733e+10 0.0 0.0 0.0 1262.264378
4 Afghanistan 1953 0.106 0.013 0.0 0.0 0.0 0.038 0.005 0.0 ... 0.0 0.0 0.0 0.0 8039684.0 1.063052e+10 0.0 0.0 0.0 1322.255925

5 rows × 60 columns

In [135]:
#df.to_csv('co2_data2.csv', sep=',', header=True, index=True) 
In [136]:
df.shape # may want to check the total number of columns and rows
Out[136]:
(25989, 60)

4. Continents CO2 emission over time¶

We can create widgets for our visualisations such as sliders, radio buttons or drop-down menus that can be used for configurations by users. First we create a panel widget for a 'year slider' using the minimum and maximum value for the years. For various widgets, please take a look at this section of the Panel User Guide.
Make Year slider:

In [137]:
year_slider = pn.widgets.IntSlider(name='Year slider', start=1750, end=2020, step=5, value=1960)
year_slider
Out[137]:
Make Radio buttons for CO2 and 'CO2 per Capita'. This will show alongside the axes of our first chart in the dashboard.¶
In [138]:
co2_values = pn.widgets.RadioButtonGroup(
    name='Y axis', options=['co2', 'co2_per_capita'], button_type='success')
co2_values
Out[138]:

Make dataframe interactive and connect the data pipeline with the widgets in such a way that, as the widgets change, the underlying data for our visualization is also updated. Next we group the data by year and country.

In [139]:
interactive_df = df.interactive() # make dataframe interactive
In [140]:
Regions = ['Africa', 'Asia', 'Europe', 'North America', 'South America', 'Oceania']
co2_pipeline = (
    interactive_df[
        (interactive_df.year <= year_slider) &
        (interactive_df.Place.isin(Regions))
       ]
    .groupby(['Place', 'year'])[co2_values].mean()
    .to_frame()
    .reset_index()
    .sort_values(by='year')  
    .reset_index(drop=True)
)
In [141]:
co2_pipeline.head() # View the 1st 5 rows
Out[141]:

We create a chart using the above pipeline: A display of CO2 as a function of the year and we notice that, as the slider is moved, both CO2 and CO2 per capita values get adjusted, making our chat quite interactive.

In [142]:
co2_plot = co2_pipeline.hvplot(x = 'year', by='Place', y=co2_values,line_width=1.6, title = "CO2 Emission by Continent")
co2_plot
Out[142]:

5. CO2 Emission Table over time¶

It is a good idea to showcase the data in a table format using the tabulator extension that we have already imported and making use of the same widgets.

In [143]:
co2_table = co2_pipeline.pipe(pn.widgets.Tabulator, pagination='remote', page_size = 10, sizing_mode='stretch_width') 
co2_table
Out[143]:

6. CO2 sources of top 10 Manufacturing countries: (Bar Chart)¶

The goal is to create an interactive bar chart showing the CO2 sources of 10 manufacturing countries (call them superpowers): China, United States, Germany, Japan, India, South Korea, United Kingdom, Italy, France and Indonesia. We do this by creating a new radio button for three different sources of CO2 emissions with reference to coal, oil and gas. As above, we create a data pipeline and connect it with the widgets and make the display with Hvplot.

In [144]:
Top10_Countries = pn.widgets.RadioButtonGroup(
    name='Y axis', 
    options=['coal_co2', 'oil_co2', 'gas_co2'], 
    button_type='success'
)

continents_excl_world = ['China', 'US', 'Germany', 'Japan', 'India',
                                   'South Korea', 'UK', 'Italy', 'France', 'Indonesia']

co2_source_bar_pipeline = (
    interactive_df[
        (interactive_df.year == year_slider) &
        (interactive_df.Place.isin(continents_excl_world))
        
    ]
    .groupby(['year', 'Place'])[Top10_Countries].sum()
    .to_frame()
    .reset_index()
    .sort_values(by='year')  
    .reset_index(drop=True)
)
In [145]:
co2_source_bar_plot = co2_source_bar_pipeline.hvplot(kind='bar', 
                                                     x='Place', 
                                                     y=Top10_Countries, 
                                                     title='CO2 Emissions: Top 10 Manufacturing Countries',
                                                     height=400, width=850)
co2_source_bar_plot
Out[145]:

7. The Interactive Dashboard¶

In [146]:
#Layout using Template
template = pn.template.FastListTemplate(
    title='World CO2 emission dashboard', 
    sidebar=[pn.pane.Markdown("## Climate Change and CO2 Emissions"), 
             pn.pane.Markdown("Many believe that carbon dioxide emissions are the primary cause of global climate change. The recommendation to curb this is for the world  to strategically reduce emissions. How this goal can be achieved is a subject of discussions in different countries and among different experts."), 
             pn.pane.PNG('climate_day.png', sizing_mode='scale_both'),
             pn.pane.Markdown("## Settings"),   
             year_slider],
    main=[pn.Row(pn.Column(co2_values, 
                           co2_plot.panel(width=700), margin=(0,25)), 
                 co2_table.panel(width=500)), 
          pn.Row(pn.Column(Top10_Countries, co2_source_bar_plot.panel(width=800)))],
    accent_base_color="#88d8b0",
    header_background="#88d8b0",
)
template.show()
template.servable();
Launching server at http://localhost:59873

8. Conclusion¶

Overall, by running the above codes, we can easily obtain an interactive dashboard with panel in addition to only utilizing numpy and pandas libraries. Commenting out the code line template.show() does not automatically launch the dashboard. Note that we can simply run in the terminal(for Mac) the text command, <panel serve Notebook_name.ipynb to play around with our interactive dashboard.

Since we have our dashboard up and running, the next question is: how can this be consumed or how can the app be deployed? There are several ways for deployment. Panel apps are supported by Jupyter, Bokeh, and Voilà servers. So you can configure your application to run on any of these given servers. A good place to look at when it comes to Panel App deployment is in Panel User Guide, 'Depoy and export'

There are other Python-server deployment procedures on different platforms like AWS, Google Cloud, Heroku, and Anaconda Enterprise. In many cases, Panel objects can also be exported to static, standalone HTML. But the goal of this project is to show that we can use Panel to create an Interactive dashboard using Python librararies. Really cool!

Next move:¶

  1. I will like to deploy this dashboard in Azure or Google cloud and see how it comes out
  2. I am a big fan of reponsiveness when it comes to web design. It appears that dashboard will need some extra work done to it in order to be enjoyed visually on mobile devices (tablets and smartphones)

Thank you¶