Python Package Trends: Visualize Package Download Stats in Django

• Download source code - 11.5 KB

Introduction

PyPi is an official package repository for Python and serves as go to tool for developer to install packages required for your application. PyPi contains over 400 thousand packages. Choosing a better package among similar packages becomes a great challenge. By analyzing package download trends, we will have better understanding of the popularity of packages, which are being maintained and which ones are losing its ground. In this article, we will discover how to use Django and pypistats API for showing download trends of Python packages.

Prerequisites

• Django
• pypistats
• CanvasJS

Project Setup

Create a new django project & a new app within the project:

$ django-admin startproject pypi_download_stats
$ python manage.py startapp download_stats_app 

Install pypistats package that provides API to fetch the download data for given package.

$ pip install pypistats

Fetching the Data

In views.py file, we will define view(get_download_stats) which will return download data received from pypistats.overall API for any given packages in JSON format.

from django.http import HttpResponse, JsonResponse
import pypistats, json

def get_download_stats(request):
  if request.method == "POST":
    post_body = json.loads(request.body)
    package_name = post_body.get("packageName")
    if(package_name != ""):
      try:
        json_data = json.loads(pypistats.overall
                    (package_name, total=True, format="json"))
        filtered_data = { "with_mirrors": [], "without_mirrors": []}
        for stats in json_data['data']:
          filtered_data[stats['category']].append
          ({k:v for (k,v) in stats.items() if ('downloads' in k or 'date' in k)})
        return JsonResponse({ 'data': filtered_data, 'package': json_data['package'] })
      except:
        return HttpResponse(json.dumps({
          "error": package_name + " Package doesnot exist"
        }), content_type="application/json")  

  return HttpResponse(json.dumps({
          "error": "Method Not Supported"
      }), content_type="application/json")

Create Download Trend Chart

Create a template index.html which contains form which takes package name as input to pass it on view for getting download data of the package. Along with form, we will add a chart container and include CanvasJS script in it.

<!-- index.html -->
<div class="content">
  <section>
    <header>
      <h2 class="text-center">PyPi Download Stats</h2>
    </header>
    <content>
      <form id="package-name" class="input-section" 
       action="{% url 'download_stats_app:index' %}" method="POST">
          {% csrf_token %}
          <input type="text" placeholder="Package name" 
           name="package-name" id="package-name" class=""/>
          <button type="submit">Show Stats</button>
      </form>
      <div class="messages" id="messages">
      </div>
    </content>
  </section>
  <section class="chart-section">
    <div id="chartContainer"></div>
  </section>
  </div>
<script src="https://canvasjs.com/assets/script/canvasjs.min.js"></script>
<script src="{% static 'assets/script.js' %}"></script>

Create script.js file in static directory to perform the ajax request and parse the data received for plotting the chart inside chart-container.

/* script.js */

//https://docs.djangoproject.com/en/4.1/howto/csrf/#using-csrf-protection-with-ajax
function getCookie(name) {
    let cookieValue = null;
    if (document.cookie && document.cookie !== '') {
        const cookies = document.cookie.split(';');
        for (let i = 0; i < cookies.length; i++) {
            const cookie = cookies[i].trim();
            // Does this cookie string begin with the name we want?
            if (cookie.substring(0, name.length + 1) === (name + '=')) {
                cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                break;
            }
        }
    }
    return cookieValue;
}

let csrftoken = getCookie('csrftoken');
let messageDOM = document.getElementById('messages');

function displayStats(packageName) {
    let requestOptions = {
        method: 'POST',
        headers: { 'Content-Type': 'application/json', 'X-CSRFToken': csrftoken },
        body: JSON.stringify({ packageName: packageName })
    };
    fetch("/downloads-stats/get-stats", requestOptions)
    .then(response => response.json())
    .then(dataArr => {
        if(dataArr.error) {
            chart.options.data.forEach(dataSeries => { dataSeries.dataPoints = [] })
            chart.options.title.text = ``;   
            chart.render();
            messageDOM.innerHTML = `<div class='error'>${dataArr.error}</div>`
            return;
        }

        let statsMirrosDataPoints = [], statsWithoutMirrosDataPoints = []
        dataArr.data["with_mirrors"].forEach( stat => {
            statsMirrosDataPoints.push({x: new Date(stat.date), y: stat.downloads})
        })
        dataArr.data["without_mirrors"].forEach( stat => {
            statsWithoutMirrosDataPoints.push({x: new Date(stat.date), y: stat.downloads})
        })
        messageDOM.innerHTML = '';
        chart.options.title.text = `Daily Download Stats for ${dataArr.package}`;
        chart.options.data[0].dataPoints = statsMirrosDataPoints;
        chart.options.data[1].dataPoints = statsWithoutMirrosDataPoints;
        chart.render();
    })
}

var chart = new CanvasJS.Chart("chartContainer", {
    theme: "light2",
    animationEnabled: true,
    title: {
      text: "With Mirror Download Stats for <Package:name>",
      fontFamily: "Poppins"
    },
    toolTip: {
      shared: true
    },
    legend: {
        cursor: "pointer",
        itemclick: hideUnhideDataSeries
    },
    data:[{
        type: "spline",
        xValueType: "dateTime",
        name: "With Mirrors",
        showInLegend: true,
        dataPoints: []
    },{
        type: "spline",
        xValueType: "dateTime",
        name: "Without Mirrors",
        showInLegend: true,
        dataPoints: []
    }]
});

function hideUnhideDataSeries(e) {
    if (typeof (e.dataSeries.visible) === "undefined" || e.dataSeries.visible) {
        e.dataSeries.visible = false;
    } else {
        e.dataSeries.visible = true;
    }
    e.chart.render();
}

document.getElementById("package-name").addEventListener('submit', function(e) {
    e.preventDefault();
    let formData = new FormData(e.target)
    let packageName = formData.get('package-name')
    if(packageName != "")
        displayStats(packageName);
});

In urls.py file, map the url with respective views.

from django.urls import path
from . import views

app_name = 'download_stats_app'
urlpatterns = [
    path('', views.index, name='index'),
    path('get-stats', views.get_download_stats, name='get-stats'),
]

Run the application using runserver:

$ py manage.py runserver 

Ta-da, we have just created Django application to show the download trends of various package available in PyPi. To analyze the trends further, you can compare plotting the download trends for multiple packages, different range of data, etc. pypistats also provides API to get the download data for different versions of python, platform, etc. which can be used further for analyzing the package and choose the best among the different packages available.

History

• 16^th February, 2023: Initial version