DMelt:IO/3 Java Serialization

Table of contents

Using Java serialization

Serialization is the process of converting data or object state into a format that can be stored or transmitted and reconstructed later.

DataMelt Serialized class

Data can be serialized into a file using the  jhplot.IO class. The only limitation is the computer memory, since the object has to be created first. In this example we create a list with two 2D arrays and write it to the file. Then we restore it.

from jhplot  import *
from jhplot.io import *

p1=P1D("test") # create 2D array and fill it
p1.add(10,20)
p1.add(12,40)
print p1.toString()

p2=P0D("test2")  # create 2D arrays and fill it
for i in range(10):
     p2.add(i*i)

a=[p1,p2]                      # make a list
Serialized.write(a,'file.ser') # write the list to the file

fas=Serialized.read('file.ser') # read the data from the file

p1=fas[0];  p2=fas[1]           # read 2D arryas from the list
print "After serialization:n"
print p1.toString(); print p2.toString()

One can also use SerializedXML class, which writes/reads data to/from XML files. Let us take a look how to write histograms into XML serialized file and then read it back

from jhplot  import *
from java.util import Random

h1=H1D("test H1D",20,-3,3)
h1.setFill(1)

p0=P0D("test P0D")
r=Random()           # fill with random numbers
for i in range(100):
      h1.fill(2*r.nextGaussian())
      p0.add(r.nextGaussian()+10)
a=[] # put both objects to the list
a.append(h1)
a.append(p0)

IO.writeXML(a,'file.xml')  # write to a XML file
print IO.toXML(a)          # convert to XML string

b=IO.readXML('file.xml')   # read back from XML
print b

HFile class

The class  jhplot.io.HFile is designed to store any Java object (including containers) in a compact serialized form. It is designed to store large data volumes without memory limitations (unlike the class Serialized considered above). It is well suited for sequential input and outputs.

For example one can store data containers and functions described Data structures,

The class is based on the standard Java serialization mechanism (compression is by default, but can be switched off). A typical extension for such files is ".jser". Essentially, almost any Java object supported by DMelt can be stored and retrieved from the HFile file.

Data stored in a file created by the class  jhplot.io.HFile can be viewed in the browser based on the class  jhplot.io.HFileBrowser One can also store data in the XML form using the class  jhplot.io.HFileXML.

Let us give an example: we create 3 objects (array and two histograms) and write then into compressed serialized file (use HFileXML to write into XML format). The we read this file and create these objects:

from jhplot.io import *
from jhplot import *

x=P0D('X'); y=P0D('Y') 
x.randomUniform(10,-3,3)
y.randomUniform(10,0,30)
p1=P1D("XY",x,y)

h1 = H1D("fixed bins",10, -2, 2.0)
h2 = H1D("variable",[-1,2,4,7])
r = Random()
for i in range(100):
  h1.fill(r.nextGaussian())
  h2.fill(5*r.nextGaussian())

f=HFile('tmp.jser','w')
f.write(p1)
f.write(h1)
f.write(h2)
f.close()

# reading objects
f=HFile('tmp.jser','r')
p=f.read()
h1=f.read()
h2=f.read()
print "Created the file="
print h1.toString()
print h1.toString()
f.close()

You can append any any number of objects to this file. You can even make a Jython map or list from different objects and write such containers in one go.

There is no any restriction on which Java object is written. One can write arbitrary complicated data in form of arrays, strings, lists, tuples, maps, dictionaries, DataMelt functions, histograms etc. Any Java, Jython, DataMelt or any third-party Java container which can hold data can be written into a file. If you want to write objects which can be retrieved using keys, use maps or dictionaries.

Using the keys

Before we considered data records organized sequentially. One can also store objects using keys in form of string. The keys should be unique.

from jhplot import *
from jhplot.io import *

p1=P0D()
p1.randomNormal(1000,0,2) # 1000 random numbers
p2=P0D()
p2.randomNormal(1000,1,2) # 1000 random numbers
pp=P1D("test",p1,p2)
g=HFile("output.jser","w")
g.write("key1",p1)
g.write("key2",p2)
g.write("key3",pp)
g.write("directory/key4",p1) # store the objects in the directory
g.write("directory/key5",p2)
g.close()

# now reading the objects back
g=HFile("output.jser")
print g.get("directory/key4")  # prints p1

The notion of "directory" is important. Now one can organize data using some meaningful logic and open data in a browser as will be discussed below.

DataBrowser to open ".jser" files

All DataMelt objects stored in compressed Java-serialized files can be viewed using a browser. For example, if a serialized file contains P1D, P0D, H1D objects, one can view them and plot them using a mouse-click approach.

If you have a file with the extension ".jser", you can view it using the DataBrowser. Go to the toolbar, select [Plot}->[HPlot canvas]-> [File]-> [Open data file].

One can also open the browser fin a macro as:

from jhplot.io  import *
from jhplot  import *
c1=HPlot("Browser")
c1.visible()
f=HFile("test.ser")
HFileBrowser(c1,f,1)