Category Archives: Provenance from Digital Notebooks

As mentioned last week, we’d like to develop our own tool to capture provenance information of digital notebooks, so writing an iPython extension is good choice. An iPython extension is like an user defined library, and defined APIs can be registered that they can be invoked in the notebook as functions. So following Continue reading Week 9 – The road ahead→

As we’re approaching the end of this summer project, it’s time to think about the end game. We’re targeting to study the provenance of notebooks and we choose iPython notebook as a use case and we studied NoWorkflow system in detail, so it’s natural to combine these two together. Since Continue reading Week 8 – Notebook provenance→

Let’s continue on the front of capturing provenance information on the Python script that visualizes a NetCDF file, using NoWorkflow system. In previous blogs, I mentioned that NoWorkflow provenance has three types: definition, development and execution. For execution provenance, only function activation and file access information are captured, there is not very Continue reading Week 7 – Provenance questions→

This week, I spent some time on the new tools as I mentioned last week, however, we ran into some difficulties that sadly we may need to change the target since we cannot run the tool. The tool is called Prov-O-Matic, which intends to capture provenance of iPython notebooks and Continue reading Week 6 – The new tool→

This week, I spent some time to find out what NoWorkflow can actually do for us. So first, we start with a summary of NoWorkflow system including what kind of provenance can be captured and how we can access that. NoWorkflow system stores all the provenance captured in a local Sqlite Continue reading Week 5 – Provenance Reunion→

This week’s report is actually a combination of three 1/3 weeks, since I attended a conference in Germany and due to the visa issue, I stayed in China for two weeks after the conference. So I did travel around the world. The conference is called ProvenanceWeek 2014, which took place in Continue reading Week 4 – Around the world…→

The goal of this week is implementing the use cases identified last week, running NoWorklfow system on it and analyzing the result. Implementing the first use case (MsTMIP script in Matlab) is a little bit challenging because 1). making exactly mapping from Matlab to Python is difficult and time-consuming or Continue reading Week 3 – Capturing provenance→

Last week, we have made a plan and this week’s task is focusing on the use cases. So several use cases have been identified: 1. An example simulation data processing script from EVA WG. The script works a set of simulation datasets, performs several diagnoses on the data based on different Continue reading Week 2 – Use cases and more→

We’re targeting to study provenance in digital notebooks, i.e. iPython. IPython notebook is an a web-based interactive computational environment for Python programming. One of our group members along with other collaborators has created a tool called “NoWorkflow”, which intends to capture script provenance of a Python script. During the joint section of Continue reading Week 1 Report – Getting started→