developer

From autoplot.org

Jump to: navigation, search

Purpose: Links for miscellaneous developer pages.

Contents

  1. Development Versions
  2. Use methods
    1. Servlet
    2. Applet
    3. WebStart
    4. Jython
    5. IDL/MATLAB
    6. Command line
      1. WebStart
      2. Non-Webstart
        1. Linux
        2. Mac
      3. Non-WebStart + Nailgun (faster)
  3. Development
    1. Development Versions
    2. Feature requests
    3. Report a bug
    4. Automated Testing
    5. Manual Testing
    6. Email List
    7. Presentations
    8. Source Code
    9. List of URIs for Testing
  4. Who's Using Autoplot
  5. Design Documents
    1. Misc.
    2. Autoplot 2011
      1. TimeSeriesBrowse
      2. Data Sources
      3. Graphics
      4. GUI
      5. Layout
      6. DOM version 1.8
      7. User Options
    3. Autoplot 2010
      1. Major Limitations of Current Autoplot
      2. Scalability from Delegation
      3. DOM Tree for Autoplot
      4. First Implementation
      5. Controller for Autoplot
        1. Methods
      6. Use Cases
      7. Layout Specification
  6. Autoplot Objects
    1. DOM Controllers
    2. DOM Nodes
      1. syncTo
      2. diffs
      3. copy

1 Development Versions

The development versions are located here: Autoplot_Change_Log

2 Use methods

Autoplot is typically run as a "thick client" on the scientist's workstation. The ~25MB application is downloaded and run on top of the Java platform. This is not the only way to use the application, however, and this section covers other modes.

2.1 Servlet

Autoplot can be run in a server mode in which case the user specifies a URL to file in a web browser and a PNG image is returned. See servlet_guide.

2.2 Applet

See applet_guide.

2.3 WebStart

See webstart_guide.

2.4 Jython

Autoplot uses Jython (Python in Java) for scripting data sources and the application itself. Selecting Options->Enable Feature->Script Panel will reveal a tab named "script". See developer.scripting.

2.5 IDL/MATLAB

See developer.idlMatlab.

2.6 Command line

2.6.1 WebStart

To start Autoplot from the command line, use 

javaws http://autoplot.org/autoplot.jnlp

To start Autoplot with an initial URI in the address bar, use the following script. 

!/bin/bash
# Usage: autoplot.sh
#        autoplot.sh URL
#        autoplot.sh FILE
#
# FILE may be an absolute or relative path, e.g.,
# /tmp/autoplot.cdf or file:///autoplot.cdf, or ./test/autoplot.cdf
#
# Downloads autoplot.org/autoplot.jnlp, modifies the input argument in
# autoplot.jnlp to be URL or FILE, and calls javaws on the modified autoplot.jnlp
#
# TODO:
#   check $JAVA_HOME 
if [ -z "$(which javaws 2>/dev/null)" ]; then
    echo "autoplot: error: Java Web Start must be installed and on your PATH to run Autoplot."
    exit 1
fi
if [ -z "$(which wget 2>/dev/null)" ]; then
    echo "autoplot: error: wget must be installed to run this script."
    exit 1
fi
 
mydir=`pwd`
infile=$1
 
if [ "http://" != "${infile:0:7}" ]; then
    # If leading ./, remove it
    if [ "./" == "${infile:0:2}" ]; then
        infile=${infile:2:${#infile}}
    fi
    # If no leading /, add absolute path
    if [ "/" == "${mydir:0:1}" ]; then
        infile="$mydir/$infile"
    fi
    # If no leading file://, add it
    if [ "file://" != "${infile:0:7}" ]; then
        infile="file://$infile"
    fi
fi
 
# See http://autoplot.org/webstart_guide for the syntax of jnlp.cgi
 
cmd="wget -q -O - \"http://autoplot.org/autoplot.jnlp?open="$infile"\" > /tmp/autoplot.jnlp"
echo $cmd
eval $cmd
# non-Sun Java WebStart is not full implementation of Sun WebStart
/usr/lib/jvm/java-6-sun/bin/javaws /tmp/autoplot.jnlp

2.6.2 Non-Webstart

Note: documentation needs URLs for non-64-bit libcdfNativeLibrary binary files. Note on note: in the current 2011 version, vap+cdfj will resolve vap+cdf if the binaries aren't available.

To see command line options, type 

java -Djava.awt.headless=true -Djava.library.path=. -cp AutoplotAll.jar org.virbo.autoplot.AutoplotServer

2.6.2.1 Linux

On Linux systems the default install of Java is not complete. On a Debian-based system, use 

apt-get install openjdk-6-jre

and call java using 

/usr/lib/jvm/java-6-openjdk/bin/java

wget "http://autoplot.org:8080/hudson/job/autoplot-jar-all/lastSuccessfulBuild/artifact/autoplot/VirboAutoplot/dist/AutoplotAll.jar"
 
# 64-bit Linux
wget http://autoplot.svn.sourceforge.net/viewvc/autoplot/autoplot/branches/agu2009/VirboAutoplot/libcdfNativeLibrary.so
 
java -Djava.awt.headless=true -Djava.library.path=. -cp AutoplotAll.jar org.virbo.autoplot.AutoplotServer -u "http://autoplot.org/data/autoplot.cdf?BGSM" -f png -o BGSM.png

2.6.2.2 Mac 

curl "http://autoplot.org:8080/hudson/job/autoplot-jar-all/lastSuccessfulBuild/artifact/autoplot/VirboAutoplot/dist/AutoplotAll.jar" > AutoplotAll.jar
# 64-bit mac
curl "http://sscweb.gsfc.nasa.gov/skteditor/cdf/lib/3.3.1/mac/x86_64/nativeCdfLibs.jar" > nativeCdfLibs.jar   
# 32-bit mac
#curl "http://sscweb.gsfc.nasa.gov/skteditor/cdf/lib/3.3.1/mac/x86/nativeCdfLibs.jar" > nativeCdfLibs.jar 
 
mv nativeCdfLibs.jar nativeCdfLibs.zip
unzip nativeCdfLibs.zip
 
# Test operation:
java -Djava.awt.headless=true -Djava.library.path=. -cp AutoplotAll.jar org.virbo.autoplot.AutoplotServer -u "http://autoplot.org/data/autoplot.cdf?BGSM" -f png -o BGSM.png

will create Image:BGSM.png

2.6.3 Non-WebStart + Nailgun (faster)

If you would like to call Autoplot repeatedly from the command line, you may use Nailgun to avoid the overhead of starting a JVM each time Autoplot is called (but setting up an Autoplot server is probably a better solution).

This [1] program may be used to compare the speed of nailgun for a trivial "hello world" application written in java and c.

First follow the instructions for running Autoplot in #Basic. Then 

wget "http://sourceforge.net/projects/nailgun/files/nailgun/0.7.1/nailgun-0.7.1.zip/download"
unzip nailgun-0.7.1.zip
 
java  -Djava.awt.headless=true -Djava.library.path=. -cp AutoplotAll.jar -cp ./AutoplotAll.jar:nailgun-0.7.1/nailgun-0.7.1.jar:. com.martiansoftware.nailgun.NGServer &
 
# Give the server some time to start
sleep 1
 
nailgun-0.7.1/ng org.virbo.autoplot.AutoplotServer -u "http://autoplot.org/data/autoplot.cdf?BGSM" -f png -o BGSM1.png 
nailgun-0.7.1/ng ng-stats
 
nailgun-0.7.1/ng org.virbo.autoplot.AutoplotServer -u "http://autoplot.org/data/autoplot.cdf?BGSM" -f png -o BGSM2.png
nailgun-0.7.1/ng ng-stats

3 Development

3.1 Development Versions

3.2 Feature requests

Autoplot was initially developed as part of a NASA VxO grant for the ViRBO project. Development will continue and we welcome requests for new features. Submit requests to http://sourceforge.net/projects/autoplot or send an email to the list given at http://groups.google.com/group/autoplot.

3.3 Report a bug

The preferred method of reporting a bug is through the sourceforge bug tracker system. The feature "Log Console" records internal logging messages and catches stdout and stderr. This information is useful for debugging, and the console's Save button saves these messages to a file.

3.4 Automated Testing

Hudson is a Java framework for managing automated testing and reporting. Once each night and shortly after any commit of source code, hundreds of tests are performed to detect if a change has been made that breaks some aspect of the system. These are currently broken up in to roughly 40 functional areas in these the tests here: http://papco.org:8080/hudson/. These make static images of vap products, test layout, test the IDL/Matlab interface, do PNGWalk generation, etc. This is not a thorough test, but it often catches errors that before would have been reported in a desperate email the week before a meeting. http://apps-pw.physics.uiowa.edu/hudson/job/autoplot-trunk/ also does FindBugs analysis that analyses the code for suspicious and oddly-worded code, and has identified a number of mistakes as well. Last, http://autoplot.org:8080/hudson/job/autoplot-release/ provides a convenient means for making one-of releases and greatly simplifies the release process.

3.5 Manual Testing

We have a script for human-driven testing here: developer.tests. These are procedures for tests that don't translate into automatic tests, such as GUI operations and multiple-platform testing.

3.6 Email List

For discussion about Autoplot, send email to autoplot@googlegroups.com. You may browse the email archive at http://groups.google.com/group/autoplot

3.7 Presentations

  • AGU 2010 poster: ppt
  • QDataSet Poster talks about how data is represented: ppt

3.8 Source Code

3.9 List of URIs for Testing

Here is a List Of URIs for testing. These include links that demonstrate that things work, and may quickly demonstrate a sources that don't work. Feel free to add to the list, adding a comment if it doesn't work properly.

4 Who's Using Autoplot

5 Design Documents

5.1 Misc.

5.2 Autoplot 2011

It would be useful to identify the feature set of the Autoplot2011 branch.

5.2.1 TimeSeriesBrowse

  • TimeSeriesBrowse will reconnect itself to application timerange if the result is not a timeseries.
    • (TODO: this doesn't work when you reload the file.)
  • Automatic aggregation detection. Plotting "20100101.dat" will look for "$Y$m$d" and use the aggregation automatically. [NOTYET]

5.2.2 Data Sources

  • pure-java cdf reader handles ".cdf" (NOT YET)
  • inline specification of filters, such as
    • vap+bin:file:///tmp/foo.wav?column=1&recLength=3|histogram (NOT YET)
  • introduce the general way to specify depend datasets, such as: (NO, 2012)
    • vap+bin:file:///tmp/foo.dat?recLength=45&depend0.column=0&column=2 , instead of
    • vap+bin:file:///tmp/foo.dat?recLength=45&depend0offset=0&column=2
  • introduce other standard parameters for easily modifying the dataset, like title, label, rendertype, etc.
  • csv is handled by 3rd party and correct csv reader. (YES, DONE)

5.2.3 Graphics

  • legend position, and legend outside plot (DONE, though not yet in vap)
  • plots.xaxis.ticklen (DONE)
  • improve cadence logic, there's a "corner case" of just four corners that I don't handle, where the histogram resolution is too fine so the non-zero peaks are spaced regularly.
  • new annotations in labels
    •  %{VERSION} from QDataSet
    •  %{CONTEXT} shows any context, e.g. slice position, associated with a dataset.
    •  %{COMPONENT} shows slice positions, components, or additional processing.
  • Time Axis annotations (ephemeris, TCA. It goes by a number of names).
  • RGB image renderer (DONE)
  • Events bar renderer (DONE)
  • rank 0 renderer that allows data to be reported. (DONE, digital supports this)
  • digital renderer used for enumeration data.
  • Improve list of available renderers to only those that handle dataset scheme, checkmark next to active render type.

5.2.4 GUI

  • Optional x-axis (time) range controller on canvas tab. (done)
  • Option to use $Y-$j day of year for ticks instead of $Y-$m-$d (done)
  • Links over to help documentation (done)
  • Flip y axis (Garcia) (NOTYET)
  • Labels option to only face two directions, many publishers require this. (Bill K.) (NOTYET)

5.2.5 Layout

  • Automatic layout inserts extra spaces for multiline plots and axes. (Need to specify how this is going to work.) developer.autolayout
  • Handle stack plots that don't have x ticks.
  • plot above/plot below, insert plot below, insert plot above. (done)
  • convert current plot to M by N
  • easy way reduce/enlarge height

5.2.6 DOM version 1.8

Everyone has figured out there there are lots of accessible Das2 features that don't make it into the DOM. Bob has requested that "everything" be in the DOM, and to the extent that this is possible, I'll do this. These features include:

  • legend position (NOTYET)
  • plots.xaxis.ticklen (NOTYET)
  • overriding cadence (NOTYET)
  • Flip y axis (Garcia)
  • Labels option to only face two directions (THIS MISSED the 2011 branch, but is a trivial change. I'll probably introduce it for 2011a_8 as a user preference.

5.2.7 User Options

  • basic/expert mode hides large sections of the DOM to help to get started. [DONE]
  • basic mode will:
    • hide the controller nodes
    • show [?] in section labels to link to documentation
    • hide the "data" "script" "console" tabs
    • show "add plot from...cdaweb..." in basic mode
  • expert mode will:
    • show the "data" "script" "console" tabs
    • show [?] in section labels to link to documentation
  • see developer.novice

5.3 Autoplot 2010

5.3.1 Major Limitations of Current Autoplot

A couple of easily described limitations of autoplot present a nice use case and hopefully help to specify the beefy version.

  • Only one dataset can be viewed at a time.
    • no stack plots
    • no overplots
    • no correlation plots
  • Some properties can be changed, but changes are not saved.
    • row, column layout
    • contour numbers

The more configurable the application becomes, the more pressing the need for a beefy system for storing application state. This has been a lingering problem with das2 for years.

5.3.2 Scalability from Delegation

The way complex state is achieved is to delete object state storage out to the objects. The application knows how to store itself by delegating to the objects that implement it.

5.3.3 DOM Tree for Autoplot

Upper-case nodes represent nodes with children, lower case nodes are leaf nodes that can be set. 'Italics' are simply comments about the node. An example node may exist below a parent to show an example child, and an ID will show how it might be accessed.

Style is a node below many nodes, which contain properties that affect the application view but not the model. DOMs may be deserialized while ignoring these, leaving user preferences in place. Further "Style Sheets" may be applied to a DOM with affecting the interpretation the application state.

EmSpec is a string like 50%+5em, which are to be interpreted relative to a column or row. 20% means 20% of the way from min to max. 5em means 5 ems, the height of the font. 20px or 20pt means 20 pixels.

Propose/Specify DOM for autoplot application.

5.3.4 First Implementation

Here's how the DOM is shaping up:

Some differences to note:

  • less hierarchical than first conceived. It's easier to manage when the model's structure doesn't change, so have plots contain panels by reference rather than object hierarchy. panel.plotId contains the plot that contains it. This also means moving a panel is just resetting the plotId.
  • many controllers. Back in the Spring I expected there would be one massive controller, but it works nicely to have a controller for each node.
  • controllers in the tree. Right now, the controllers are accessible in the tree with each node's getController method. This may change, since it pollutes the model with the controller. But it does make it very easy to find things.
  • application.options rather than application.style. This is closer to the legacy structure and this may change as the idea of "style sheets" is introduced. Options are per-user inter-session persistent.

5.3.5 Controller for Autoplot

The controller contains methods for controlling the DOM.

5.3.5.1 Methods

  • bind( 'Canvas[0].Plot[0].Xaxis.range', property( DOM.getElementById('annotation_1'), 'text' )
  • property( Object o, String property )
  • insertRow( ) adds below the current focus
  • insertColumn( )
  • focusElement( )
  • addDataSource( DataSource )

5.3.6 Use Cases

  • plot with two datasets overlayed.
  • correlation between two datasets.
  • add ContoursRenderer
  • add TimeRangeSelector
  • add page for collecting digitize times
  • run daily plots for year
  • style sheets demo similar to CSS Zen Garden

5.3.7 Layout Specification

"Rows" and "columns" are used to control the position of plots. Autoplot uses strings "5em,100%-10em" to control the position of these objects. The comma delineates min and max specifications, which correspond to top,left and bottom, right for rows and columns respectively. The text "100%-10em" means 100% of the way from the way left (min) to right (max), and the -10em means come back 10 em heights.

6 Autoplot Objects

6.1 DOM Controllers

The DOM Controllers are objects that manage nodes of the DOM. For example, when the slice index of a DataSourceFilter node is changed, the controller slices the original dataset to create a new dataset, and fires off an event notification of the new dataset.

DataSourceController: Converts the DataSourceFilter node URI string into a dataset. (Filter comes from that this once would slice as well, but that function may be removed. We may allow one node to simply filter another node.)

PlotElementController: A PlotElement has references to a DataSourceFilter,a Plot node, and render type string identifying the rendering method. This takes the QDataSet produced by the DataSourceController and renders it onto the plot. Note that this may also filter the data. When vectors are displayed, several PlotElements are added for each component. Also, each PlotElement contains default plot ranges and labels.

PlotController: The Plot node has range and label settings. This controller will detect when autoranging should occur and combines the plot ranges into a default plot range.

TimeSeriesBrowseController: When a DataSource indicates it has the TimeSeriesBrowse capability, this controller is created to manage loading new data when new ranges are displayed.

ApplicationController: It's job is to simply add and remove listeners as the dom is manipulated, to keep track of the application's history, and to implement load and save.

Note a number of things are handled as part of the Das2 graphics library or Java:

  • Updates to the plot when axes are moved or labels changed via property editor.
  • Connectors between plots are painted and managed by das2.
  • Bindings are simply Java Beans Bindings between property nodes.

6.2 DOM Nodes

6.2.1 syncTo

syncTo method should set all the properties of a node to those of another node. This should include the id. This should include references to id's.

6.2.2 diffs

diffs returns an array of diff descriptor objects that can do and undo the diff. The returned diffs should include the id. The returned diffs should include changes in references to other

6.2.3 copy

copy is a deep copy of the node, including id and references. However, this is often implemented with clone() it's extremely important that the controller not be copied. Most copy implementations will explicitly set the controller to null.

Retrieved from "http://www.autoplot.org/developer"
Personal tools