wiki:InfrastructureOverview

Version 10 (modified by Dieter Van Uytvanck, 8 years ago) (diff)

added numbered headings

1. Technology

1.1. Gateway applications

1.1.1. Virtual Language Observatory

Metadata exploration web application, with faceted browser.

Own software. Stable.

Public info: http://www.clarin.eu/vlo Technical info: https://trac.clarin.eu/wiki/CmdiVirtualLanguageObservatory

1.1.2. Federated Content Search engine

Search engine to interact with the federated content search endpoints.

Own software. Stable.

Public info: https://www.clarin.eu/content/content-search Technical info: https://trac.clarin.eu/wiki/FCS%20Aggregator

1.1.3. Virtual Collection Registry

Web application to manage and publish digital bookmark sets (virtual collections).

Own software. Stable.

Public info: http://www.clarin.eu/content/virtual-collections Technical info: https://trac.clarin.eu/wiki/VirtualCollectionRegistry

1.2. Infrastructure applications

1.2.1. Centre Registry

Database with information about the CLARIN centres.

Own software. Stable.

Public info: https://www.clarin.eu/clarin-eric-datatables/centres Technical info: https://centres.clarin.eu/about

1.2.2. Metadata

1.2.2.1. Metadata Harvester

Application to harvest the metadata descriptions from centres.

Own software. Stable.

Public info: http://clarin.eu/harvester Technical info: https://trac.clarin.eu/wiki/OAIHarvester

1.2.2.2. Component Registry

Application to manage and create CLARIN-compliant metadata profiles.

Own software. Stable.

Public info: http://clarin.eu/componentregistry Technical info: https://trac.clarin.eu/wiki/ComponentRegistryAndEditor

1.2.2.3. Concept Registry

Application to manage and create CLARIN-compliant concept definitions

Existing software (OpenSKOS) from CatchPLUS, with some modifications. Stable.

Public info: http://www.clarin.eu/ccr Technical info: http://openskos.org/

1.2.2.4. Curation module

Application to detect and correct errors in CLARIN-compliant metadata descriptions.

Experimental, under development by OEAW

Technical info: https://github.com/acdh-oeaw/clarin-curation-module

1.2.3. Federated Identity

1.2.3.1. SAML metadata aggregation: PyFF

Application to merge technical information from national identity federations into 1 file.

Existing software from NORDUnet. Stable.

Technical info: https://github.com/leifj/pyFF

1.2.3.2. CLARIN Identity Provider

Application for the registration of accounts for those who do not have an academic identity provider.

Mix of existing software (drupal, OpenDJ, Shibboleth) and own connecting components. Will be replaced by one overarching and externally maintained application (!UnityIDM), which is also used by EUDAT.

Public info: https://user.clarin.eu/ Technical info: http://www.clarin.eu/content/clarin-identity-provider

1.2.3.3. Discovery Service

Application to select an identity provider upon federated login.

Mix of existing software (DiscoJuice) and own connecting components. Stable.

Technical info: http://www.clarin.eu/content/clarin-discovery-service

1.2.3.4. Authorisation Service

Middleware that allows a web application or service to perform user delegation: act on behalf of an authenticated user (e.g. writing to her/his protected workspace).

Experimental

Will be provided by UnityIDM Technical info: https://www.clarin.eu/content/user-delegation

1.2.4. Piwik

Application that keeps usage statistics of various web applications and sites.

Third-party software (self hosted). Stable.

Technical info: https://trac.clarin.eu/wiki/AccessStatistics

1.2.5. Validators

1.2.5.1. OAI-PMH (metadata) validator

Application to check the validity of an OAI-PMH endpoint for metadata harvesting.

Third-party software (self hosted). Stable.

Public info: http://clarin.eu/oaivalidator Technical info: http://dl.cs.uct.ac.za/projects/re/

1.2.5.2. CMDI (metadata) validator

Command-line application to check the validity of large sets of CMDI files.

Own software. Stable.

Technical info: https://trac.clarin.eu/browser/CMDIValidator

1.2.5.3. SRU-CQL (FCS) validator

Application to check the validity of an SRU-CQL endpoint for content search. Own software.

Stable Public info: http://clarin.eu/oaivalidator Technical info: http://dl.cs.uct.ac.za/projects/re/

1.2.6. Language Resource Switchboard

Application to suggest web applications that can process a specific file type.

Experimental, under development by EKUT

Technical info: https://github.com/clarin-eric/LRSwitchboard

1.3. Software development

1.3.1. Subversion repository

Source code repository for self-developed software. Will be gradually migrated to GitHub.

Third-party software (self hosted)

Technical info: http://www.clarin.eu/svn

1.3.2. Trac

Issue tracker for software development and wiki. The issue tracker will be gradually migrated to GitHub, while the wiki will remain for internal information exchange.

Third-party software (self hosted)

Technical info: http://www.clarin.eu/svn

1.3.3. Nexus maven repository

Repository to share compiled Java packages (jars) and Maven files (poms).

Third-party software (self hosted).

Technical info: https://trac.clarin.eu/wiki/Nexus

1.3.4. Docker repository

Third-party software (self hosted). Stable.

Technical info: https://trac.clarin.eu/wiki/ServerConfig/docker

1.3.5. Third-party infrastructure (hosted in the cloud)

1.3.5.1. GitHub

Issue tracking and code repository.

Technical info: https://github.com/clarin-eric

1.3.5.2. Travis

Continuous Integration platform, tightly coupled with GitHub.

Technical info: https://travis-ci.org/clarin-eric

1.3.5.3. Jenkins

Continuous Integration platform, used to build and test a number of tools (co)-developed at MPI-PL. Plans to migrate this to Travis.

Technical info: https://lux16.mpi.nl/jenkins/ (not publicly accessible)

1.4. System management

1.4.1. Icinga

Monitoring system.

Third-party software (hosted at computing centre).

Technical info: https://trac.clarin.eu/wiki/Monitoring

1.4.2. Grafana

Computing Resource Dashboard.

Third-party software (self hosted).

Technical info: https://trac.clarin.eu/wiki/ServerConfig/collectd

1.5. Office systems

1.5.1. Document (CE) archive

Simple web application for the CLARIN document archive.

Own software. Stable.

Technical info: https://github.com/dietervu/ce-archive (private)

1.5.2. People and Organisation database

Simple web application to manage the CLARIN contacts. Will be replaced by e.g. a lightweight Drupal-based CRM solution.

Own software. Stable.

1.6. Communication systems

1.6.1. Website

Drupal-based content management system.

Third-party software, with a few own additions. Stable.

Public info: http://www.clarin.eu/ Technical info: https://trac.clarin.eu/wiki/ServerConfig/vz07-clarin-eu.im.hum.uu.nl

1.6.2. Mailing Lists

Mailman system.

Third-party software, hosted at Utrecht University.

Public info: http://www.clarin.eu/content/mailing-lists Technical info: https://trac.clarin.eu/wiki/lists.clarin.eu

1.6.3. MailChimp

Newsflash distribution system.

Third-party software, hosted in the cloud.

Public info: http://www.clarin.eu/content/newsflash

1.6.4. Basecamp

Internal project management platform, used for CLARIN-PLUS.

Third-party software, hosted in the cloud.

Public info: https://basecamp.com/3008549

1.6.5. Slack

Chat platform for developers. Experimental.

Third-party software, hosted in the cloud.

Public info: https://clarineric.slack.com

1.6.6. Social media

1.6.6.1. Facebook

https://www.facebook.com/ClarinEric/

1.6.6.2. Twitter

https://twitter.com/CLARINERIC

1.6.6.3. LinkedIn

https://www.linkedin.com/company/9411460

1.6.6.4. YouTube

https://www.youtube.com/channel/UCJPks1mzisqsS4NrBFKIWag

1.7. Third-party infrastructure applications & services

1.7.1. B2DROP [EUDAT]

Cloud-storage platform based on ownCloud, similar to DropBox. Also referred to (within CLARIN) as “workspaces”.

Public info: https://b2drop.eudat.eu/ Technical info: https://trac.clarin.eu/wiki/Workspaces

1.7.2. B2SHARE [EUDAT]

Simple data upload platform, “scientific youtube”, based on CERN’s Invenio repository.

Public info: https://b2share.eudat.eu/

1.7.3. B2SAFE [EUDAT]

Data replication service based on iRODS.

Public info: https://eudat.eu/services/b2safe Technical info: http://www.clarin.eu/event/2015/clarin-b2safe-workshop

1.7.4. B2ACCESS [EUDAT]

Authentication module (proxy Service Provider) for EUDAT services.

Public info: https://b2access.eudat.eu Technical info: https://eudat.eu/services/b2access

1.7.5. Potential A-services

To be extended with services provided by CLARIN centres; a first candidate list is available at http://www.clarin.eu/node/4002