COVID-19 in Singapore: Looking at every case since it happened

Since January, the world has been dealing with this new bug pandemic known as COVID-19 (used to be called coronavirus - but maybe the beer people complained?). Like SARS, H1N1, MERS, this flu like disease has weird DNA sequence-ish name (which changes from time to time), but is not fun to people who caught them.

Many people have helpfully assembled a lot of information. But I thought it will be interesting to look at local data using visualisation tool. There are already folks doing it. See here and here. Still, I see no harm riding on the shoulders of giants ahead of me, to see if I could come up with an improved version.

Data

The first thing I need to decide is what information to present and where to get the data. I want to look at how infection is spreading in various parts of Singapore. Ministry of Health puts up daily press release to update on the profile of new and existing infected cases. So that will be my source.

Collating the data is going to be a manual process. I wish there is a simple tool to parse the texts in the press releases into something that could be sent to a database directly. I also know this is just pipe dream. It is not so straight forward to convert messy press releases into a nice set of data. So let's forget about web scraping and what-have-you. ?

I want to show where COVID-19 is going around in Singapore. So we need to map all the places where the infected persons have stayed or visited. That's straight forward enough.

However, I don't think it is meaningful to present the information anchoring on the date of the press releases. I mean, what matters to me is not when the information is out, but the exact time when the person displayed symptoms due to the virus. Knowing this means that the longer time has passed, the "safer" that place should be for me, if I had been there. That is what I wanted to tell people.

Data persistence, aka, where it is stored

After the information is organised, I shall store it in the cloud, and use it to generate the required data for the website. For experimental purpose, I signed up a free account at Mongodb Atlas. I guess it is ok for the purpose of this demo site. Also, it is fun to use a NoSQL database since I do not need to get into creating and managing database schema, although there are plus points for doing that. More later.

Tools

For mapping tool, I decided to try out leaflet.js. I first saw data.gov.sg folks using it for demo sites; I am really a sucker for beautiful interfaces. There is nothing wrong with Google map, but so is nothing wrong for a change! What I like best initially is that leaflet works together with Mapbox which allows you to customise nice map tiles for your map.

Obviously there is a ton of additional stuff that could be done with Mapbox. There is an online studio for you to create map styles and collaborate with others. But, my focus was to just get a nice map, so I did not really explore further.

As for leaflet.js, there is adequate documentation and examples to get me started. It looks like there is also a good community of guys working on various types of problems, so I know I won't be lonely. I took the sample codes from one tutorial, put in some coordinate data in the prescribed geojson format, and quite quickly got the hang of loading and unloading markers on a map.

Filtering data

For user interactivity, my idea is to let the person use a slider to choose and filter out the information she wants. I implemented sliders using an old (but looks very robust) javascript library by seiyria.

I could spend more effort to improve on the look and feel of the sliders, but I wanted to concentrate on getting the sliders to work properly and (more importantly) to be able to load and unload the necessary markers on the map.

I must say the slider library comes with a lot of flexibility to handle all the tasks that I could think of. I took my hat off folks out there for having trotted a tougher road before me, and freely offered tips. Essentially, to handle data filtering using slider, it boils down to coding the routines that should respond to the slider that is changing its value, like follows:

$('#input').slider({<options dictionary>}).on('change', function(event){//what to with the event.value while sliding})

Bear in mind I am using jquery all the time. Can't think of any reason why not.

Visualisation Work Flow

Here is the brief synopsis of the micro site visualisation work flow.

First, the geojson data is declared in a linked .js file

var impactPoint = {...}

Then we use leaflet to set up a map, and add in the MapBox tile.

var map = L.map('map').setView([1.355548,103.8248537], 11);

L.tileLayer(<url>,{<options>}).addTo(map);

Next, the geojson is loaded into a layer and then added to the map. Note that the createLayer() function here actually wraps the leaflet's L.geoJson() function to load/add layer to a map.

let layer = createLayer(impactPoint);

We also create and add a small legend to the map.

var legend = L.control({position:'bottomright'});

legend.onAdd = function(map) {//codes that return the html elements using L.DomUtil.create() function };

legend.addTo(map);

There are two sliders on the map. Using the sliders .on('change', fn) event handler, we change the markers on the map, based on either the range of symptom dates, or by individual case number. This is done by passing the value from the slider into the createLayer() function for it to filter out the markers

$('#input').slider().on('change', function(){

map.removeLayer(layer);

layer = createLayer(impactPoint,ev.value.newValue,'symptom_date');

})

Where else has he gone?

I noticed that in Ministry of Health's press releases, they mentioned about the places each patient visited, and the place he had stayed. While plotting these points on the map, I thought it would be interesting to see what are the routes this person might have taken from say his home to the various locations he visited.

I am sure Google has a free service to do this. But since I am in the mood of shunning them, I decided to look around for alternatives. Imagine my joy when I came across Open Route Service. Thank you very much Heidelberg University! They provide you a free API through the web. You send the API two sets of latlng coordinates (a start-point and an end-point)

https://api.openrouteservice.org/v2/directions/driving-car?api_key=<api>&start=<start-point>&end=<start-point>

And vioala it returns you a bunch of geojson lines (ie sets of two coordinates). You can then use them to plot the lines on the map, showing how to travel (along the road) from the start-point to the end-point.

At this point in time I realised I needed to include additional attributes for the database. This is where NoSQL like mongodb shines. I just went ahead and changed the underlying database for some records - no need for every record, mind you - so in other words the data is no longer structured/uniformed. You can't do this with SQL type of database.

Afterthought

It was interesting creating this microsite. The most painful part of this process was sorting out (and keying in) the data. Mongodb does not control how you add records. I needed to track the places that each patient has visited. This part of the record ideally should have some relation to another set of data setting out all the locations. However, there is no "foreign key" concept for NoSQL database. This type of mismatch is of course a well known issue and it looks like all sorts of experts have all sorts of opinion. I guess if one is serious about using NoSQL then one needs to ensure that the problem is well managed in the application.

Routine Job

The backend is a very simple python Flask application consisting of one file with 8 lines. This runs a webserver to serve the static website. Sorry. This is no production grade.

from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')

def hello_world():

f = open('static/time.txt', 'r') ; l = f.readlines(); f.close()

return render_template ('h.html', date=l[0].split(',')[1], goedate=l[0].split(',')[0])

if __name__ == '__main__':

app.run(debug=False)

It might be useful to have a batch process to automatically generate the data (ie the geojson file) from mongodb. Then the only human effort required left is to update the database from time to time. Should I do it? You tell me. ?