Optional Challenge: Parameterize the Data Viewer

Now we have a web site that consumes a Python service. But it does so only for data from The Netherlands. This section contains an optional challenge for those of you that have interest (and time!) to take it a bit further: to create a parameterized version where the users can make their own choice of country to show the bar chart for. This would make the Data Viewer page you created earlier quite a bit more useful...!

The Challenge

We will not tell you step-by-step how to do this, you should try to be creative and pull together all the techniques you have learned to achieve the following:

The web page should offer the users a list that lets them choose between several countries. It is up to you how you create this list, there are many options ranging from
- Simple (create a fixed list of a few options), to
- Complicated (getting the country list from the actual DB);
Based on the choice of country, the chart should show population numbers per province of that country.
Based on that same choice, the map should show the provinces of the country chosen.

A Solution

There are many different ways this can be achieved. One possible solution is in the listings below, it will be revealed later...

There are any ways to achieve this. The solution as implemented on the WEBSITE HERE was implemented with the code changes and additions shown below:

import cgi, cgitb
cgitb.enable()
import psycopg2
import matplotlib.pyplot as plt
import sys

############# Get Parameters ################
theRequest = cgi.FieldStorage()
# get the country input
theCountry = theRequest.getfirst("country", "undefined")

############# Block 1: Get Data ################
# connect to the db
db = psycopg2.connect("host='gisedu.itc.utwente.nl' port='5434' \
                       dbname='exercises' user='exercises' password='exercises'")
# Get a cursor
dbcursor = db.cursor()
#create and execute query
SQL = """SELECT admin_name, pop_admin 
         FROM world.prov_pop 
         WHERE cntry_name = '%s' 
         ORDER BY pop_admin DESC;"""
dbcursor.execute(SQL % (theCountry))
data = dbcursor.fetchall()

############# Block 2: create figure ################
labellist = []
valuelist = []
# extract the lists from the Dict:
for item in data:
    labellist.append(item[0])
    valuelist.append(item[1])

# Create a matplotlib figure
plt.figure(figsize=(10,4))
fig = plt.subplot()
indices = range(len(labellist))
fig.bar(indices, valuelist)
fig.ticklabel_format(style='plain') #avoid scientific notation
fig.set_ylabel('Population')
fig.set_xticks(indices)
fig.set_xticklabels(labellist)
fig.tick_params(axis='x', rotation=90)
plt.subplots_adjust(bottom=0.3) # make room for labels

############# write as SVG to CGI Response Object ################
# first we need to output http header for an SVG web graphic:
print ('Content-type: image/svg+xml')
print ('') # this empty print is required!

# we cannot use simple plt.show() because that plots to the screen
# we use plt.savefig() to save as a SVG drawing,
# but we do not save as a file, but to the sys.stdout buffer,
# which in CGI python writes to the Response Object!
plt.savefig(sys.stdout.buffer, format='svg')

import sys

############# Get Parameters ################

theRequest = cgi.FieldStorage()

# get the country input

theCountry = theRequest.getfirst("country", "undefined")

############# Block 1: Get Data ################

# connect to the db

db = psycopg2.connect("host='gisedu.itc.utwente.nl' port='5434' \

dbname='exercises' user='exercises' password='exercises'")

# Get a cursor

dbcursor = db.cursor()

#create and execute query

SQL = """SELECT admin_name, pop_admin

FROM world.prov_pop

WHERE cntry_name = '%s'

ORDER BY pop_admin DESC;"""

dbcursor.execute(SQL % (theCountry))

data = dbcursor.fetchall()

import cgi, cgitb
cgitb.enable()
import psycopg2
import json

############# Get Parameters ################
theRequest = cgi.FieldStorage()
# get the country input
theCountry = theRequest.getfirst("country", "undefined")

############# Get Data ################
# connect to the db
db = psycopg2.connect("host='gisedu.itc.utwente.nl' port='5434' \
                       dbname='exercises' user='exercises' password='exercises'")
# Get a cursor
dbcursor = db.cursor()
#create and execute query
SQL = """
    SELECT json_build_object(
        'type', 'FeatureCollection',
        'features', json_agg(
            json_build_object(
            'type',       'Feature',
            'properties', jsonb_set(row_to_json(prov)::jsonb,'{geom}','0',false),
            'geometry',   ST_AsGeoJSON(geom)::json
            )
        )
    )
    FROM (SELECT gid, admin_name, pop_admin,geom, cntry_name FROM world.prov_pop) as prov
    WHERE cntry_name = '%s'; 
"""
dbcursor.execute(SQL % (theCountry))
# using json.dumps to create valid json, using [0] to only get single object
jsonstring = json.dumps(dbcursor.fetchone()[0])

############# write to Response Object ################
# first we need to output http header for JSON data:
print ('Content-type: application/json; charset=utf-8')
print ('') # this empty print is required!
print ( jsonstring )

import json

############# Get Parameters ################

theRequest = cgi.FieldStorage()

# get the country input

theCountry = theRequest.getfirst("country", "undefined")

############# Get Data ################

# connect to the db

db = psycopg2.connect("host='gisedu.itc.utwente.nl' port='5434' \

dbname='exercises' user='exercises' password='exercises'")

# Get a cursor

dbcursor = db.cursor()

#create and execute query

SQL = """

SELECT json_build_object(

'type', 'FeatureCollection',

'features', json_agg(

json_build_object(

'type', 'Feature',

'properties', jsonb_set(row_to_json(prov)::jsonb,'{geom}','0',false),

'geometry', ST_AsGeoJSON(geom)::json

)

FROM (SELECT gid, admin_name, pop_admin,geom, cntry_name FROM world.prov_pop) as prov

WHERE cntry_name = '%s';

"""

dbcursor.execute(SQL % (theCountry))

# using json.dumps to create valid json, using [0] to only get single object

jsonstring = json.dumps(dbcursor.fetchone()[0])

<!doctype html>
<html lang="en">

<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
  <title>A Simple Website</title>
  <!-- styles -->
  <link rel='stylesheet' href='https://fonts.googleapis.com/css?family=Open+Sans|Baumans' type='text/css'>
  <link rel="stylesheet" href="styles/main.css" type="text/css" media="screen" />
  <link rel="stylesheet" href="styles/viewer.css" type="text/css" media="screen" />
  <link rel="stylesheet" href="https://gisedu.itc.utwente.nl/exercise/libraries/openlayers/v4.6.4/ol.css" type="text/css"/>
  <!-- scripts -->
  <script src="https://gisedu.itc.utwente.nl/exercise/libraries/openlayers/v4.6.4/ol.js"></script>
  <script src="scripts/viewer.js"></script>
</head>

<body onload="init()">

<div class="container header">
  <header>
    <h1>A Simple Website</h1>
    <p>Using HTML5 and CSS3</p>
    <nav>
      <ul>
        <li><a href="./index.html">Home</a></li>
        <li class="active"><a href="./data-viewer.html">Data Viewer</a></li>
        <li><a href="">Services</a></li>
      </ul>
    </nav>
  </header>
</div>

<section id="spacer">
  <!-- Info Bar -->
  <p>Visualizations of the country:
    <select id="countryChooser" class="form-control" onchange="setCountry(this)">
      <option value="Netherlands" selected>Netherlands</option>
    </select></p>
</section> <!-- spacer -->

<!-- Chart Section -->
<section class="one-column">
  <article id="bar_chart">
    <h2>Bar Chart</h2>
    <div id="chart_container">
      <img id="population_chart" height="300px" src="../services/pop_bars_svg_params.py?country=Netherlands">
    </div>
  </article> <!-- bar_chart -->
  <br class="clear"/>
</section>

<!-- Map Section -->
<section class="one-column">
  <article id="web_map">
    <h2>Spatial Representation</h2>
    <div id="map_container" style="width:1000px; height: 800px;"></div>
  </article> <!-- web_map -->
  <br class="clear"/>
</section>

<footer>
  <!-- Footer content -->
  <section class="wrapper">
    <article class="column">
      <h4>About Me</h4>
      Some information about the author...
    </article>
    <article class="column midlist">
      <h4>Email</h4>
      <a href="mailto:myemail@student.utwente.nl">myemail@student.utwente.nl</a>
    </article>
    <article class="column rightlist">
      <h4>Socials</h4><a href="">A link to my socials.</a>
    </article>
  </section>
  <!-- Copyright notice -->
  <section id="copyright">
    <div class="wrapper">
      &copy; Copyright 2024 by The Author
  </section> <!-- copyright -->
</footer>

</body>
</html>

<p>Visualizations of the country:

<option value="Netherlands" selected>Netherlands</option>

</select></p>

</section>

<h2>Bar Chart</h2>

</div>

</article>

</section>

/*----------------------------------------------------------------
  Name:        viewer.js
  Date:        November 2023
  Description: Base code for the - Web Programming - exercise
               (data viewer page)
  Version:     2.1 - for the CHALLENGE
----------------------------------------------------------------*/

var navMap;
var CountryName = "Netherlands";
var CountryNameList = "empty";

// simple solution is just create a limited list :
function getCountryList() {
    CountryNameList = ["Netherlands", "Denmark", "Germany", "Belgium", "Luxembourg", "Poland"];
    countryChooser = document.getElementById("countryChooser");
    for (i in CountryNameList) {
        var option = document.createElement("option");
        option.text = CountryNameList[i];
        countryChooser.add(option);
    }
}

// function to change CountryName based on the dropdowm menu choice in data-viewer.html
function setCountry(menuChoice) {
    CountryName = menuChoice.value;
    // redraw Chart by setting a new SRC
    var popChart = document.getElementById("population_chart");
    popChart.src = "../services/pop_bars_svg_params.py?country=" + CountryName;
}


/*-- Initialization function --*/
function init() {
    if (CountryNameList === "empty") {
        getCountryList();
    }
    //define map object & link to placeholder div:
    navMap = new ol.Map({target: "map_container"});
    // define layer as tiled map:
    osmLayer = new ol.layer.Tile({
        // load OSM (a connector predefined in the API) as source:
        source: new ol.source.OSM()
    });
    // add layer to map:
    navMap.addLayer(osmLayer);
    // create a map view:
    mapView = new ol.View({
        center: ol.proj.transform([0, 0], 'EPSG:4326', 'EPSG:3857'),
        zoom: 2
    });
    navMap.setView(mapView);

    navMap.addControl(new ol.control.Zoom());
    navMap.addControl(new ol.control.MousePosition({
            projection: 'EPSG:4326',
            coordinateFormat: ol.coordinate.createStringXY(4)
        })
    );

    let prov_style = new ol.style.Style({
        stroke: new ol.style.Stroke({
            color: 'MediumPurple',
            width: 2
        }),
        fill: new ol.style.Fill({
            color: 'rgba(147, 112, 219, 0.2)'
        })
    });

    country_provinces = new ol.layer.Vector({
        source: new ol.source.Vector({
            url: '../services/provinces_params.py?country=' + CountryName,
            format: new ol.format.GeoJSON({
                defaultDataProjection: 'EPSG:4326',
                projection: 'EPSG:3857'
            })
        }),
        style: prov_style,
        name: 'Country Provinces'
    });
    navMap.addLayer(country_provinces);
    // wait until the 'change' event is fired (all data loaded)
    country_provinces.getSource().once('change', function () {
        // then zoom in to extent:
        mapView.fit(country_provinces.getSource().getExtent());
    });
}

// The most complete solution uses a services/getcountrylist.py service to get all names from DB
// This function to get Country Names from Python CGI (which gets it from the DB)
// has to be asynchronous as we do not know when service is finished
async function getCountryListfromDB() {
    var response = await fetch("../services/getcountrylist.py"); //wait for response
    if (response.ok) { // if HTTP-status is 200-299 = OK
        var rawdata = await response.text(); // wait for data to be extracted
        CountryNameList = rawdata.split("|"); // creates array
        countryChooser = document.getElementById("countryChooser");
        for (i in CountryNameList) {
            var option = document.createElement("option");
            option.text = CountryNameList[i];
            countryChooser.add(option);
        }
    } else {
        CountryNameList = "undefined";
        SetMessage("Error getcountrylist.py :\n[" + response.status + "] " + response.statusText, errorMsg);
    }
}
/*---*/

/*----------------------------------------------------------------

Name: viewer.js

Date: November 2023

Description: Base code for the - Web Programming - exercise

(data viewer page)

Version: 2.1 - for the CHALLENGE

----------------------------------------------------------------*/

var navMap;

var CountryName = "Netherlands";

var CountryNameList = "empty";

// simple solution is just create a limited list :

function getCountryList() {

CountryNameList = ["Netherlands", "Denmark", "Germany", "Belgium", "Luxembourg", "Poland"];

countryChooser = document.getElementById("countryChooser");

for (i in CountryNameList) {

var option = document.createElement("option");

option.text = CountryNameList[i];

countryChooser.add(option);

}

// function to change CountryName based on the dropdowm menu choice in data-viewer.html

function setCountry(menuChoice) {

CountryName = menuChoice.value;

// redraw Chart by setting a new SRC

var popChart = document.getElementById("population_chart");

popChart.src = "../services/pop_bars_svg_params.py?country=" + CountryName;

}

/*-- Initialization function --*/

function init() {

if (CountryNameList === "empty") {

getCountryList();

}

//define map object & link to placeholder div:

navMap = new ol.Map({target: "map_container"});

// define layer as tiled map:

osmLayer = new ol.layer.Tile({

// load OSM (a connector predefined in the API) as source:

source: new ol.source.OSM()

});

// add layer to map:

navMap.addLayer(osmLayer);

// create a map view:

mapView = new ol.View({

center: ol.proj.transform([0, 0], 'EPSG:4326', 'EPSG:3857'),

zoom: 2

});

navMap.setView(mapView);

navMap.addControl(new ol.control.Zoom());

navMap.addControl(new ol.control.MousePosition({

projection: 'EPSG:4326',

coordinateFormat: ol.coordinate.createStringXY(4)

})

);

let prov_style = new ol.style.Style({

stroke: new ol.style.Stroke({

color: 'MediumPurple',

width: 2

}),

fill: new ol.style.Fill({

color: 'rgba(147, 112, 219, 0.2)'

})

});

country_provinces = new ol.layer.Vector({

source: new ol.source.Vector({

url: '../services/provinces_params.py?country=' + CountryName,

format: new ol.format.GeoJSON({

defaultDataProjection: 'EPSG:4326',

projection: 'EPSG:3857'

})

}),

style: prov_style,

name: 'Country Provinces'

});

navMap.addLayer(country_provinces);

// wait until the 'change' event is fired (all data loaded)

country_provinces.getSource().once('change', function () {

// then zoom in to extent:

mapView.fit(country_provinces.getSource().getExtent());

});

}

If you want to create a pulldown menu from the DB, you need a Python service such as this:

import cgi, cgitb
cgitb.enable()
import psycopg2

############# Get Data ################
# connect to the db
db = psycopg2.connect("host='gisedu.itc.utwente.nl' port='5434' \
                       dbname='exercises' user='exercises' password='exercises'")
# Get a cursor
dbcursor = db.cursor()
#create and execute query
SQL = """SELECT cntry_name
         FROM world.prov_pop
         GROUP BY cntry_name
         ORDER BY cntry_name;"""
dbcursor.execute(SQL)
data = dbcursor.fetchall()
namelist = ''
for item in data:
    namelist = namelist + item[0] + '|' # use [0] to get 1st item from Dict
    # the '|' is used as delimiter and later in JS to split the array...

############ write to Response Object ################
# first we need to output http header:
print ('Content-type: Content-Type: text/plain')
print ('') # this empty print is required!
# dump as one string
print( namelist )

import cgi, cgitb

cgitb.enable()

import psycopg2

############# Get Data ################

# connect to the db

db = psycopg2.connect("host='gisedu.itc.utwente.nl' port='5434' \

dbname='exercises' user='exercises' password='exercises'")

# Get a cursor

dbcursor = db.cursor()

#create and execute query

SQL = """SELECT cntry_name

FROM world.prov_pop

GROUP BY cntry_name

ORDER BY cntry_name;"""

dbcursor.execute(SQL)

data = dbcursor.fetchall()

namelist = ''

for item in data:

namelist = namelist + item[0] + '|' # use [0] to get 1st item from Dict

# the '|' is used as delimiter and later in JS to split the array...

############ write to Response Object ################

# first we need to output http header:

print ('Content-type: Content-Type: text/plain')

print ('') # this empty print is required!

# dump as one string

print( namelist )

And then change the function getCountryList() to this:

/*----------------------------------------------------------------
  Name:        viewer.js
  Date:        November 2023
  Description: Base code for the - Web Programming - exercise
               (data viewer page)
  Version:     2.1 - for the CHALLENGE
----------------------------------------------------------------*/

var navMap;
var CountryName = "Netherlands";
var CountryNameList = "empty";

// simple solution is just create a limited list :
function getCountryList() {
    CountryNameList = ["Netherlands", "Denmark", "Germany", "Belgium", "Luxembourg", "Poland"];
    countryChooser = document.getElementById("countryChooser");
    for (i in CountryNameList) {
        var option = document.createElement("option");
        option.text = CountryNameList[i];
        countryChooser.add(option);
    }
}

// function to change CountryName based on the dropdowm menu choice in data-viewer.html
function setCountry(menuChoice) {
    CountryName = menuChoice.value;
    // redraw Chart by setting a new SRC
    var popChart = document.getElementById("population_chart");
    popChart.src = "../services/pop_bars_svg_params.py?country=" + CountryName;
}


/*-- Initialization function --*/
function init() {
    if (CountryNameList === "empty") {
        getCountryList();
    }
    //define map object & link to placeholder div:
    navMap = new ol.Map({target: "map_container"});
    // define layer as tiled map:
    osmLayer = new ol.layer.Tile({
        // load OSM (a connector predefined in the API) as source:
        source: new ol.source.OSM()
    });
    // add layer to map:
    navMap.addLayer(osmLayer);
    // create a map view:
    mapView = new ol.View({
        center: ol.proj.transform([0, 0], 'EPSG:4326', 'EPSG:3857'),
        zoom: 2
    });
    navMap.setView(mapView);

    navMap.addControl(new ol.control.Zoom());
    navMap.addControl(new ol.control.MousePosition({
            projection: 'EPSG:4326',
            coordinateFormat: ol.coordinate.createStringXY(4)
        })
    );

    let prov_style = new ol.style.Style({
        stroke: new ol.style.Stroke({
            color: 'MediumPurple',
            width: 2
        }),
        fill: new ol.style.Fill({
            color: 'rgba(147, 112, 219, 0.2)'
        })
    });

    country_provinces = new ol.layer.Vector({
        source: new ol.source.Vector({
            url: '../services/provinces_params.py?country=' + CountryName,
            format: new ol.format.GeoJSON({
                defaultDataProjection: 'EPSG:4326',
                projection: 'EPSG:3857'
            })
        }),
        style: prov_style,
        name: 'Country Provinces'
    });
    navMap.addLayer(country_provinces);
    // wait until the 'change' event is fired (all data loaded)
    country_provinces.getSource().once('change', function () {
        // then zoom in to extent:
        mapView.fit(country_provinces.getSource().getExtent());
    });
}

// The most complete solution uses a services/getcountrylist.py service to get all names from DB
// This function to get Country Names from Python CGI (which gets it from the DB)
// has to be asynchronous as we do not know when service is finished
async function getCountryListfromDB() {
    var response = await fetch("../services/getcountrylist.py"); //wait for response
    if (response.ok) { // if HTTP-status is 200-299 = OK
        var rawdata = await response.text(); // wait for data to be extracted
        CountryNameList = rawdata.split("|"); // creates array
        countryChooser = document.getElementById("countryChooser");
        for (i in CountryNameList) {
            var option = document.createElement("option");
            option.text = CountryNameList[i];
            countryChooser.add(option);
        }
    } else {
        CountryNameList = "undefined";
        SetMessage("Error getcountrylist.py :\n[" + response.status + "] " + response.statusText, errorMsg);
    }
}
/*---*/

100

101

102

103

104

105

106

107

108

// The most complete solution uses a services/getcountrylist.py service to get all names from DB

// This function to get Country Names from Python CGI (which gets it from the DB)

// has to be asynchronous as we do not know when service is finished

async function getCountryListfromDB() {

var response = await fetch("../services/getcountrylist.py"); //wait for response

if (response.ok) { // if HTTP-status is 200-299 = OK

var rawdata = await response.text(); // wait for data to be extracted

CountryNameList = rawdata.split("|"); // creates array

countryChooser = document.getElementById("countryChooser");

for (i in CountryNameList) {

var option = document.createElement("option");

option.text = CountryNameList[i];

countryChooser.add(option);

}

} else {

CountryNameList = "undefined";

SetMessage("Error getcountrylist.py :\n[" + response.status + "] " + response.statusText, errorMsg);

}

Web Programming - Short Introduction

Optional Challenge: Parameterize the Data Viewer

The Challenge

A Solution

Exercise Sections

Page Content