Changes between Version 11 and Version 12 of Modules


Ignore:
Timestamp:
2010-05-26T18:18:28+02:00 (14 years ago)
Author:
Morris Swertz
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Modules

    v11 v12  
    22Don't expect this if you use MOLGENIS 3.x!
    33
    4 MOLGENIS is organized as a set of modules which we call plugins. Each plugin adds a combination of MolgenisModels, MolgenisDomainSpecificLanguage, MolgenisGenerators and/or handwritten code. Other plugins may extend other plugins to yet add more functionality. The first plugin where all others build upon is the MOLGENIS database core. The last plugin is typically a fully functional application that assembles other plugins
     4= Architectural overview =
     5
     6MOLGENIS is organized as a set of modules which we call plugins. Each plugin adds a combination of
     7 * MolgenisModels - models that use the domain specific language to define functionality compactly
     8 * MolgenisGenerators - templates that expand the models into working software
     9 * MolgenisComponents -
     10and/or handwritten code. Other plugins may extend other plugins to yet add more functionality. The first plugin where all others build upon is the MOLGENIS database core. The last plugin is typically a fully functional application that assembles other plugins
    511
    612TODO core architectural picture.
    713
    8 = Software oriented plugins =
    9  * wiki:Plugins/OrgMolgenisCore - core platform of model parsers, generators, framework and utils
    10  * wiki:Plugins/OrgMolgenisDb - database interface framework on top of MySQL, PostgreSQL and HypersonicSQL (extends core)
    11  * wiki:Plugins/OrgMolgenisUi - user interface framework that can be run on the internet (extends db)
    12  * wiki:Plugins/OrgMolgenisAuth - authentication and secure sharing framework (extends db, ui)
    13  * wiki:Plugins/OrgMolgenisProcessing - for easy modeling, integration and running of analysis tools (extends ui, db, core)
     14TODO directory structure
    1415
    15 = Data oriented plugins =
    16  * wiki:Plugins/MatrixData - 
    17  * wiki:Plugins/PhenoData -
    18  * wiki:Plugins/MutationData - for representing and working with mutations
     16TODO concept of extension points
    1917
    20 = Systems =
    21 Systems are ready to use and download:
     18= Extension mechanism =
    2219
    23 In the wet lab:
    24  * wiki:Plugins/
     20Each building block may add framework code, model extensions, generators and/or models.
     21In this last case the module may regeneration against the last versions before it can be used (which we can do automatically?).
     22After that we could use a osgi drop in mechanism to add the produce funtionality into a running system.
    2523
    26 Repositories:
    27  * wiki:Plugins/ShortReadMolgenis - for storing short read data
     24Design 0: simply link to projects
     25In this procedure the model files are puth on classpath and can be simply imported.
     26The code has to be generated by the imported projects themselves.
     27This is ideal for development (as long as re-generation is a one push on a button event and checks if generators are changed (hash or something)).
    2828
    29 Downstream:
    30  * wiki:Plugins/GwasMolgenis - for GWAS studies like in EU-GEN2PHN, LifeLines and BBMRI-NL
    31  * wiki:Plugins/QtlMolgenis - for QTL studies in model organisms like EU-Panacea
     29Design 1: linking in code
     30We imagine a procedure where when a user runs the generator (ant script, nothing else)
     31- downloads the core molgenis and starts the model parser/generator
     32- when <import="db" .../> is in your local model dependencies are automatically checked out from svn to /generated/java (incl libs!)
     33- each of these dependencies is regenerated in dependency order
     34- tests run to verify that the modules are sanitized (no compile errors, test cases work)
     35- and then produced as osgi modules that are simply loaded by the current project during the rest of development
     36And have a 'clean' action to redo the procedure.
     37
     38Design 2: downloading maven/osgi modules.
     39We imagine a procedure where when a user runs the generator from maven (maven pom + molgenis core as plugin, nothing else)
     40- downloads all depedencies using maven model (all MOLGENIS settings should be in this model as well to keep it simple?)
     41- add generate tasks to maven to get all stuff done
     42Complicating factor here is that the maven repos should be rebuild now and then.
     43
     44We could allow both design 1 and 2 in order to speed up development?
     45
     46i.e. organize maven compatible but still allow ant.xml type of interactions to do this low-tech.
     47Or use maven in the back and don't necessarily bother the user with it.
     48
     49= Core modules =
     50These are the foundational building blocks that can be reused between all MOLGENIS applications.
     51Package org.molgenis.*
     52
     53Core:
     54 * [wiki:Plugins/OrgMolgenis core] - core platform of model (parsers), generators and util packages
     55 * [wiki:Plugins/OrgMolgenisDb db] - database interface framework on top of MySQL, PostgreSQL and HypersonicSQL (extends core)
     56 * [wiki:Plugins/OrgMolgenisExchange exchange] - file import/export mechanisms from and to csv, tab, excel, (xml), etc (extends db)
     57 * [wiki:Plugins/OrgMolgenisApi api] - application programming interfaces to Java, R, REST, SOAP, RDF, etc on top of Db (extends exchange)
     58 * [wiki:Plugins/OrgMolgenisUi ui] - web user interface framework with forms, menus and plugins (extends api)
     59 * [wiki:Plugins/OrgMolgenisCompute compute] - compute interface framework for short, long and parallel running computations (extends ui?)
     60
     61Discussion:
     62ui could be moved to depend on core only, and then have other packages extend on that to add particular UI components for db, compute, etc.
     63Or have separate ui extensions like ui.db, ui.compute, etc.
     64
     65= Components =
     66These are the semi-finished building blocks that can be shared between different applications to reuse common work.
     67These components typically provide data model (modules) that need to be generated before a plugin is ready to use.
     68Hence, there is a need to regenerate on the first installation of the plugin.
     69package org.molgenis.*
     70
     71Cross cutting:
     72 * [wiki:Component/OrgXgapAuth auth] - secure authentication and sharing framework (extends ui)
     73 * [wiki:Component/OrgXgapVersionable versionable] - mechanism to have versioned data elements by simply extending an interface
     74
     75Discussion: need ability to weave this in later, like a special decorator that is applied everywhere?
     76Or should we just make these core features and move it into db package?
     77
     78Generic:
     79 * [wiki:Component/OrgXgapOnto molgenisdata.onto] - for using ontologies including tree based browser, search box (extends ui)
     80 * [wiki:Component/OrgXgapMatrix molgenisdata.matrix] - for working with matrix like data having 'dimensionelements' and 'dataelements'
     81 * [wiki:Component/OrgXgapProtocol molgenisdata.protocol] - for track and tracing of materials/data, protocols, protocolapplications (compatible with MAGE-TAB SDRF)
     82 * [wiki:Component/OrgXgapTool tool] - for integrating R scripts onto the application (extends protocol, compute)
     83 * [wiki:Component/OrgXgapMaterialTracker tool] - for integrating R scripts onto the application (extends protocol)
     84
     85Domain:
     86 * [wiki:Component/OrgXgapLocus locus] - tools for managing and browsing genomic sequence annotations
     87 * [wiki:Component/OrgXgapPheno pheno] - for representing deep phenotypic data collected in clinics and the field (extends onto, tracker, matrix)
     88 * [wiki:Component/OrgXgapMutation muta] - for working with mutating data conforming to HGVS nomenclature (extends pheno)
     89
     90More ideas: pedigree, comments
     91
     92
     93= Applications =
     94Systems are ready to use and download. Ideally end users would be able to choose from these online.
     95
     96 * [wiki:Apps/PhenoTrackerMolgenis PhenoTracker] - for storing phenotypics annotations on individuals and panels
     97 * [wiki:Apps/GwasMolgenis GwasPlatform] - for GWAS studies like in EU-GEN2PHN, LifeLines and BBMRI-NL
     98 * [wiki:Apps/QtlMolgenis QtlPlatform] - for QTL studies in model organisms like EU-Panacea
     99 * [wiki:Apps/MutationMolgenis MutationTracker] - for locus specific annotations (extends pheno, muta)
     100 * [wiki:Apps/ShortReadMolgenis SequenceArchive] - for storing, QC-ing and releasing short read data from NGS
    32101 *
     102
     103
     104= Ideas for more enhancements =
     105
     106== Core ==
     107 * [wiki:Plugins/OrgMolgenisUi/Theme ui.theme] - catalog of UI web themes for MOLGENIS applications the use can choose from (extends ui)
     108 * [wiki:Plugins/OrgMolgenisUi/Iphone ui.iphone] - iphone user interface (extends ui)
     109 * [wiki:Plugins/OrgMOlgenisDb/Versionable db.versionable] - to allow versionable data entry
     110 * [wiki:Plugins/OrgMolgenisApi/Observer api.observer] - mechanism to observe and log activity for tracking and tracing
     111 * [wiki:Plugins/OrgMolgenisApi/Feeds api.feed] - atom feeds + command in UI to observe data, queries and/or transactions (extends api, ui) ui
     112 * [wiki:Plugins/OrgMolgenisUi/EclipeR ui.gwt] - rich client application generator based on GWT