Table of Contents
Using Java serialization
SCaVis Serialized class
Data can be serialised into a file using the Serialized class. The only limitation is the computer memory, since the object has to be created first. In this example we create a list with two 2D arrays and write it to the file. Then we restore it.
1: from jhplot import * 2: from jhplot.io import * 3: 4: p1=P1D("test") # create 2D array and fill it 5: p1.add(10,20) 6: p1.add(12,40) 7: print p1.toString() 8: 9: p2=P0D("test2") # create 2D arrays and fill it 10: for i in range(10): 11: p2.add(i*i) 12: 13: a=[p1,p2] # make a list 14: Serialized.write(a,'file.ser') # write the list to the file 15: 16: fas=Serialized.read('file.ser') # read the data from the file 17: 18: p1=fas; p2=fas # read 2D arryas from the list 19: print "After serialization:n" 20: print p1.toString(); print p2.toString()
One can also use SerializedXML class, which writes/reads data to/from XML files. Let us take a look how to write histograms into XML serialized file and then read it back
1: from jhplot import * 2: from jhplot.io import * 3: from java.util import Random 4: 5: h1=H1D("test H1D",20,-3,3) 6: h1.setFill(1) 7: 8: p0=P0D("test P0D") 9: r=Random() # fill with random numbers 10: for i in range(100): 11: h1.fill(2*r.nextGaussian()) 12: p0.add(r.nextGaussian()+10) 13: a= # put both objects to the list 14: a.append(h1) 15: a.append(p0) 16: 17: Serialized.writeXML(a,'file.xml') # write to a XML file 18: print Serialized.toXML(a) # convert to XML string 19: 20: b=Serialized.readXML('file.xml') # read back from XML 21: print b
The class HFile is designed to store any Java object (including containers) in a compact serialized form. It is designed to store large data volumes without memory limitations (unlike the class Serialized considered above). It is well suited for sequential input and outputs. For example one can store data containers and functions described Data structures and functions. The class is based on the standard Java serialization mechanism (compression is by default, but can be switched off). A typical extension for such files is ”.ser”.
Essentially, almost any Java object can be stored and retrieved from the HFile file
One can also store data in the XML form using the class HFileXML.
Let us give an example: we create 3 objects (array and two histograms) and write then into compressed serialized file (use HFileXML to write into XML format). The we read this file and create these objects:
from jhplot.io import * from jhplot import * x=P0D('X'); y=P0D('Y') x.randomUniform(10,-3,3) y.randomUniform(10,0,30) p1=P1D("XY",x,y) h1 = H1D("fixed bins",10, -2, 2.0) h2 = H1D("variable",[-1,2,4,7]) r = Random() for i in range(100): h1.fill(r.nextGaussian()) h2.fill(5*r.nextGaussian()) f=HFile('tmp.jser','w') f.write(p1) f.write(h1) f.write(h2) f.close() # reading objects f=HFile('tmp.jser','r') p=f.read() h1=f.read() h2=f.read() print "Created the file=" print h1.toString() print h1.toString() f.close()
You can append any any number of objects to this file. You can even make a Jython map or list from different objects and write such containers in one go.
There is no any restriction on which Java object is written. One can write arbitrary complicated data in form of arrays, strings, lists, tuples, maps, dictionaries, SCaVis functions, histograms etc. Any Java, Jython, SCaVis or any third-party Java container which can hold data can be written into a file. If you want to write objects which can be retrieved using keys, use maps or dictionaries.
Using the keys
Before we considered data records organized sequentially. One can also store objects using keys in form of string. The keys should be unique.
from jhplot import * from jhplot.io import * p1=P0D() p1.randomNormal(1000,0,2) # 1000 random numbers p2=P0D() p2.randomNormal(1000,1,2) # 1000 random numbers pp=P1D("test",p1,p2); g=HFile("output.jser","w") g.write("key1",p1) g.write("key2",p2) g.write("key3",pp) g.write("directory/key4",p1) # store the objects in the directory g.write("directory/key5",p2) g.close() # now reading the objects back g=HFile("output.jser") print g.get("directory/key4") # prints p1
The notion of “directory” is important. Now one can organize data using some meaningful logic and open data in a browser as will be discussed below.
DataBrowser to open ".jser" files
All SCaVis objects stored in compressed Java-serialized files can be viewed using a browser. For example, if a serialized file contains P1D, P0D, H1D objects, one can view them and plot them using a mouse-click approach.
If you have a file with the extension ”.jser”, you can view it using the DataBrowser. Go to the toolbar, select [Plot}→[HPlot canvas]→ [File]→ [Open data file]. The you can plot the objects using the mouse click. Read input_output.
One can also open the browser fin a macro as:
1: from jhplot.io import * 2: from jhplot import * 3: c1=HPlot("Browser") 4: c1.visible() 5: f=HFile("test.ser") 6: HFileBrowser(c1,f,1)
I/O performance and benchmarks
Here we compare performance of the PFile and HFile classes for read and write mode. Benchmark results are given together with the code.
Third-party IO classes
click here if you want to know more
click here if you want to know more
here are a lot of other Java-based I/O classes designed for storing and retrieving data. A complete description of how to use Java, Jython and SCaVis for scientific analysis is described in the book Scientific data analysis using Jython and Java published by Springer Verlag, London, 2010 (by S.V.Chekanov)
— Sergei Chekanov 2010/03/07 17:35