You are a guest. Restricted access. Read more.
Command disabled: backlink
SCaVis manual

Cross platform I/O

HBook class

Essentially any object (data arrays, histograms, functions) of SCaVis cab be represented as XML file without using the Java serialization. The Java class which writes data in XML is called HBook. This class, unlike the standard Java serialization (including XML) discussed in serialized, the current XMP implementation is very similar to the “AIDA” implementation. Notably:

  • jhplot.io.HBook keeps only information content of objects (values, titles, labels), without graphical attributes (Color, fonts)
  • jhplot.io.HBook does not use XML tags to keep separate values. This means that this class generates substantially smaller outputs since the standard tags for data values are not used. This is especially important for large data volumes, 2D matrices and histograms.
  • jhplot.io.HBook is better suited for cross platform. In particular, C++ and fortran packages are available to read and write data in this format. See the package CFBook.

Here is a small example of how to write a few SCaVis objects:

writer.py
from jhplot  import *
from jhplot.io import *
hb = HBook("output.jdat","w") # HBook object 
p1=P1D("test")
p1.add(1.0,2.0)
p1.add(2.0,3.0)
hb.write("data",p1)      # we add data using "data" key
 
f1=F1D("x*cos(x*10)")
hb.write(10,f1)          # we can also add a function using numeric key (integer)
hb.close()               # write and close the file. 

The output of the JDAT file look as:

Click to display ⇲

Click to hide ⇱

Click to hide ⇱

<scavis>
<created-by>JDAT file. Work.ORG: @S.Chekanov</created-by>
<created-on>Sat Jan 26 20:43:42 CST 2013</created-on>
<description>writer</description>
<version>2</version>
 
<f1d>
<id>10</id>
<title>x*cos(x*10)</title>
<name>x*cos(x*10)</name>
</f1d>
 
<p1d>
<id>data</id>
<title>test</title>
<size>2</size>
<dimen>2</dimen>
<labelx>X</labelx>
<labely>Y</labely>
<data>
1.0,2.0
2.0,3.0
</data>
</p1d>
 
</scavis>

All XML tags of JDAT are self-explanatory. Now we will read both objects using their keys:

from jhplot  import *
from jhplot.io import *
hb = HBook("output.jdat")  # read HBook object
print hb.getKeys()         # print all the keys
p1=hb.get("data")
f1=hb.get('10')
print p1.toString()
print f1.getName()

The expected number is:

array(java.lang.String, [u'10', u'data'])
# title=test  dimension=2
#  X       Y
1.0  2.0
2.0  3.0

x*cos(x*10)

You can browser the data as explained in Sect.input_output.

Here is a more detailed example. We write data arrays, histogram and a function to 3 types of files with the extensions .jdat (HBook), .jser (HFile), .jpbu (PFile, protocol buffer). Then we open the DataBrowser for each file automatically and then plot the objects using the mouse clicks.

You are not full member and have a limited access to this section. One can unlock advanced pages after becoming a full member. You can also request to edit this manual and insert comments.

The data can be organized in “directories” inside the HBook XML files and can be viewed in the browser as a trees. Here is an example showing this:

You are not full member and have a limited access to this section. One can unlock advanced pages after becoming a full member. You can also request to edit this manual and insert comments.

Parse HBook files in CPython

HBook files with the extension “.jdat” can be read in CPython (Python implemented in C) using “xml.dom” Python module. In this case, one can use the output in a platform-specific languages (for example, CPython can be interfaced with C++/C libraries). Here is a small example how to parse jdat files in the CPython:

You are not full member and have a limited access to this section. One can unlock advanced pages after becoming a full member. You can also request to edit this manual and insert comments.

In this example “.jdat” file includes several P1D objects. We read them all objects with the keys and create a map where the ID of the object is the key. The array of this map has data.

PFile class

This class is an attempt to build a multi-platform (and multi-language) I/O format based the Google's protocol buffer package. This package can be used to write files using a C++ program (or any other language) and read using Java, or write data in Java and read by a C/C++ programs.

The SCaVis Java class which implements the Google's protocol buffer format is called PFile. The class PFile is designed to store mainly SCaVis containers (arrays) described in data_structures and data projected to 1,2 and 3 dimensions (histograms) described in histograms.

A C/C++ package which is used to write data arrays, histograms and structural data (ntuples) in C++ to be read by Java PFile class is called CBook.

Once data are written in C/C++ with the help of the CBook package, one can read such data with the PFile class or even to open data in a interactive Browser using PFileBrowser class.

Here is a more complicated example: We write a 2D histogram and 1D array into a file. Then we read the objects back and plot them. One can use the class HFile as well (just replace PFile by HFile). But, in case of PFile, you can write histograms and arrays using a C++ code and read data back using Java/Jython!

Unlike the HFile class, only pre-defined data containers can be stored in PFile files (all jhplot arrays, functions, histograms). PFile class is optimized for write/read speed and small output. In addition, one can read/write such files in C++.

Code example

  Download for this example is disabled for non-members
 1: from jhplot.io import *
 2: from jhplot import *
 3: from java.util import Random
 4:
 5: c1 = HPlot3D("Canvas",600,400,2,1)
 6: c1.visible(1)
 7: c1.setAutoRange()
 8:
 9: h1 = H2D("input2D",5,-2, 2.0, 5, -3,3)
10: r = Random(33)
11: for i in range(100):
12:   h1.fill(r.nextGaussian(), r.nextGaussian())
13: c1.draw(h1)
14: f=PFile('tmp.jpbu','w')
15: f.write(h1)
16: f.write(P0D("sss"))
17: f.close()
18:
19: c1.cd(2,1)
20: c1.setAutoRange()
21: f=PFile('tmp.jpbu','r')
22: print f.size()        # how many objects are stored
23: print f.listEntries() # list all entries, the size of objects and the compression level
24: c1.draw( f.read("input2D") )
25: f.close()

In this example, instead of sequential reading the objects one by one, we retrieve the objects using its key “input2D” (which is the title of the histogram).

Open the created file in a browser for easy plotting:

Code example

  Download for this example is disabled for non-members
1: from jhplot.io  import *
2: from jhplot  import *
3: c1=HPlot("Browser")
4: c1.visible()
5: f=PFile("tmp.jpbu")
6: PFileBrowser(c1,f,1)

You will see a pop-up windows with all data entries. Click on the entry - the object will be plotted on the canvas.

Using DataBrowser to open ".jpbu" files

All SCaVis objects stored in compressed Java-serialized files can be viewed using a browser. For example, if a serialized file contains P1D, P0D, H1D, etc. objects, one can view them and plot them using a mouse-click approach.

If you have a file with the extension “.jpbu”, you can view it using the DataBrowser. Go to the toolbar, select [Plot}→[HPlot canvas]→ [File]→ [Open data file]. The you can plot the objects using the mouse click. Read input_output.

Working with C++ external programs

Here we discuss how to create files with SCaVis data containers (arrays, histograms, functions) using an C++ code and then how to read them using 100% Java code of the SCaVis.

Unregistered users have a limited access to this section. You can unlock advanced pages after becoming a full member.

I/O performance and benchmarks

Here we compare performance of the PFile and HFile classes for read and write mode. Benchmark results are given together with the code.

Unregistered users have a limited access to this section. You can unlock advanced pages after becoming a full member.

Third-party IO classes

click here if you want to know more

click here if you want to know more

click here if you want to know more

here are a lot of other Java-based I/O classes designed for storing and retrieving data. A complete description of how to use Java, Jython and SCaVis for scientific analysis is described in the book Scientific data analysis using Jython and Java published by Springer Verlag, London, 2010 (by S.V.Chekanov)

Sergei Chekanov 2010/03/07 17:35

man/io/crossplatform.txt · Last modified: 2014/12/13 20:48 by admin
CC Attribution-Share Alike 3.0 Unported
Powered by PHP Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0 Valid HTML5