Matrix Data Viewer

The Matrix data viewer renders ObservedValue? objects in a matrix form with seamlessly attached the respective Target and Feature properties.

Brainstorm

Joeri and Morris made a brainstorm picture which merges concepts with GUI. You can edit it on Google docs (assuming we shared it with you).

Important considerations

From Ate and Erik:

• Ability to store matrix views (which columns to show, how to sort etc.)
• User can right-click one or more cells, or entire rows or columns and get a context-menu to do stuff with the contents:
  • Make cells editable, not only value but also the other fields of the underlying ObservedValue?
  • Make cells deletable
• Handle multiple values for a target-feature combination, e.g. show only most recent but provide a + button to show them all
• Provide data visualization, e.g. you select a series of values and on the right of the matrix a plot shows up
• Search, filter and sort
• Save selection to Batch (Batchesplugin)

Use cases (Ate en Erik)

1) A simple one: Ate would like to be able to generate a simple overview of the animals currently alive, in a second stage maybe filtered or grouped by a second(/third/n th) criterion such as location.

--> steps involved (for as far as I can see now)

• Filter targets by rights: (only show the targets I am entitled to see), this has to happen under the hood, currently not implemented I guess.
• Filter targets by type: --> only show targets of type animal. This is currently implemented in Eriks matrix by filtering on labeled groups prior to generating the matrix, the matrix therefore only contains observable targets of one type. Should this be reconsidered for a generic matrixviewer?
• Add an EAV --> in the case of AnimalDB: the Status event.
• filter by the most recent application of the status event. (animals can have multiple protocol applications, in the case of the status feature only the most recent protocol application is valid.
• filter by active/inactive values. --> if endtime is present, then the animal is dead/not present, if it is not set, the animal is still alive. In this case the value istself (active) is irrelevant but the presence of endtime siganls the relevant change. In Erik's current implementation it is impossible to filter/select/group by the times attached to the event, making it impossible to generate this overview.

Joris' Matrix

HIGHLIGHT

Implementation

add path to your code here.

GUI

add path to your code here

Regression test

ADD HERE

Demo

ADD HERE

Erik's Matrix

The strong point of this matrix implementation is the front-end, a jQuery DataTable?. This provides the user with a lot of standard functionality like paging, filtering and searching. However, since the intermediate layer is a servlet, we had to do a lot of programming ourselves. But on the other hand, this enabled us to add nice features like the ability to add and remove feature columns at will.

In order to stay compatible with Joeri's work, we based our matrix back-end on the Matrix interface, but the version we used seems to be outdated now.

Implementation

Interface: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/plugins/listplugin/Matrix.java

Implementation: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/plugins/listplugin/PhenoMatrix.java

GUI

​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/plugins/listplugin/ListPlugin.java

​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/plugins/listplugin/ListPlugin.ftl

(JavaScripts at the bottom)

Regression test

None yet...

Demo

Should be at ​http://demo.animaldb.org/ but seems to be broken at the moment...

You can of course always check out gcc, do gcc-generate, then animaldb-generate and -updatedb, run the application and go to Events -> Make/view lists.

Joeri's Matrix

The clever thing is the abstract class. This takes care of most matrix abilities, because it runs on a set of basic core functions that have to be implemented by each specific backend. (ie. retrieving elements)

Implementation

Interface: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/matrix/DataMatrixInstance.java

Abstract: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/matrix/AbstractDataMatrixInstance.java

Backend implementations: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/matrix/implementations/

TODO: Queries on values, matrix operations such as join/union/intersect etc.

GUI

Several plugins that make use of the matrix: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/plugins/matrix/

An important intermediate is the Browser, this enables paging through a matrix. Code at: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/plugins/matrix/manager/Browser.java

TODO: Add advanced actions such as filters, queries, operations.

Regression test

A JUnit test for all 4 implementations. Created matrices filled with random data. Makes all kinds of combinations of annotations, datatypes, dimension sizes, etc. Then tests every possible way to extract data from the matrix. TODO: proper tests for queries.

Test main class at: ​http://www.molgenis.org/svn/gcc/trunk/handwritten/java/matrix/test/implementations/RunTestMatrix.java

Demo

See the plugins in action at: ​http://vm7.target.rug.nl:8080/xqtldemo/molgenis.do?__target=Investigationmenu&select=Data

Older stuff

This is the page describing everything surrounding the Matrix module to be developed. This includes:

• MatrixInterface - backend to deal with matrix-like data with implementations for database, file, binary file, plink, and so on. What methods should this interface at least provide?
  • Matrix LoadDataFromPhenoModel(DB)
• MatrixViewer - MOLGENIS user interface plugin to interact with matrices in a useful manner (edit, filter, extract, linkout, annotate, etc)
  • List<Feature> GetColumnHeaders()
  • List<Target> GetRowHeaders(StartRowNr = 0, EndRowNr = null)
  • GetValue(Column, Row, Version)
  • GetValueTimestampStart(Column, Row, Version)
  • GetValueTimestampEnd(Column, Row, Version)
  • GetCurrentValueVersion(Column, Row)
  • SetValue(Column, Row)
  • SetValueTimestampStart(Column, Row)
  • SetValueTimestampEnd(Column, Row)
  • SortOnColumn(ColumnNr, StartRowNr = 0, EndRowNr = null)
  • List<Feature> RestrictOnEventtype(List<Eventtype>)
  • List<Feature> RestrictOnFeature(SearchString)
  • Tuple<Column, Row, Version> Search(SearchString)
• MatrixDisplay - Extensions upon the MatrixViewer that allow users to make sense of the data like heatmap, pointcloud, network

At the backend, we want a class that loads data from a Pheno OM style data model into a Matrix object. We then want to be able to interact with this Matrix, getting column headers, row headers, and cell values with different versions, each with its own timestamp. In addition, we want to be able to:

• Sort on columns
• Restrict the returned row set
• Restrict the columns (Features), based on
  • A search term
  • One or more Eventtypes of which the Features have to be part. If a Feature is part of more than one Eventtype, the corresponding column should be returned more than once, one time for each Eventtype (true?).
• Search the Matrix for a specific search string.

Old design

• MatrixInterface?: Interface class with function definitions and all documentation.
• Matrix: Abstract class with most functions implemented.
• xxxMatrix: Regular class which is a flavour of Matrix. Functions implemented here work together with the Abstract functions but are specific for this flavour.

Error: Failed to load processor graphviz

No macro or processor named 'graphviz' found

New design:

• ReadOnlyMatrix?: Interface class of the simplest Matrix, so no dependancies and only getter functions.
• EntityMatrix?: Interface class of typed ReadOnlyMatrix?, uses org.molgenis.util.Entity to type its rows/colums.
• EditableMatrix?: Interface class of read/writeable Matrix, has add and remove functions for manipulation.
• MatrixInterface?: XGAP specific stuff and complex functions??
• AbstractMatrix?: Other complex functions??
• AbstractEntityMatrix?: Integrates functionality of high level functions with Entities??
• xxxMatrix: Regular class which is a flavour of Matrix. Functions implemented here work together with the Abstract functions but are specific for this flavour.

Current design in xgap_1_4_distro_hax:

Error: Failed to load processor graphviz

No macro or processor named 'graphviz' found

Corrected new design??

Error: Failed to load processor graphviz

No macro or processor named 'graphviz' found

Back to Modules