This is another multi-tasking week.
First, I continued the the work started last week โ to create workflows that integrate analysis modules with visualization modules. Specifically, I developed a new module called PairWiseComparisonMatrix that is used to extract a matrix to pair-wisely compare different models. The supported statistics include: correlation, root mean squared difference, mean absolute difference. The output is a symmetric matrix that can be used as the input of HeatMap visualization module. With this module, I created a workflow that reads in outputs from different climate models, processes and summarize them, visualize their pair-wise correlation using HeatMap.
So far we have two runnable workflows:
- Dayment summary: summarize Daymet climate data (e.g. getting long term temporal mean, temporal aggregation, summary along different axis etc.)
- Model comparison: compare different models using analysis (e.g. summary statistics) and visualization modules (bar chart and heat map)
On Monday we had a Telecon with a USGS expert Colin Talbert who is a GIS Analyst and very experienced in Vistrails module development. We showed him our package and modules. We also had a great discussion that would be very beneficial to our future work.
The second piece of work of this week is to integration our package with DataONE cyber-infrastructure. With the modules we developed and the Vistrails DataONE package which is a wrapper of DataONE python client (Part of the DataONE Investigator Toolkit), I created a workflow that can read data from DataONE, process them and then write data back to DataONE. This is a proof-of-concept workflow. It serves as the initial step of interacting with DataONE cyber-infrastructure. Currently the Vistrails DataONE package only has limited functions. It doesn’t have data search modules. If a search module is there, we can build the workflow that can search DataONE repository, show the result. Users can select datasets they are interested in to proceed running the following modules in our package.
The third piece of work is web server setup. Currently all the package development is desktop based. Users need to install Vistrails/UV-CDAT to run our modules. To facilitate their usage, we want to set up a web server so that user can run the workflow simply using a web browser. This is a non-trivial change as we have to create user interface, write server side script, run Vistrials in server and retrieve result, show the result to user through a browser. I started some initial work this week. I mainly worked on writing server side script to manipulate Vistrials. Now it can control Vistrails. The other steps (e.g input interface, result retrieval and display) would be the focus of my next week.