Interconnected Private Clouds for Universities and Researchers
OpenCloudMesh is a joint international initiative under the umbrella of the GÉANT Association that is built on ownCloud’s open Federated Cloud sharing application programming interface (API) taking Universal File Access beyond the borders of individual Clouds and into a globally interconnected mesh of research clouds - without sacrificing any of the advantages in privacy, control and security an on-premises cloud provides. OpenCloudMesh provides a common file access layer across an organization and across globally interconnected organizations, whether the data resides on internal servers, on object storage, in applications like SharePoint or Jive, other ownClouds, or even external cloud systems such as Dropbox and Google (syncing them to desktops or mobile apps, making them available offline).
Concept document
The OpenCloudMesh concept document was produced by ownCloud Inc. and first distributed on 23 July 2015
Statement
All leading partners of the OpenCloudMesh project - GÉANT, CERN and ownCloud Inc. - are fully committed to the open API design principle. This means that - from day one - the OCM sharing API should be discussed, designed and developed as a vendor neutral protocol to be adopted by any on-premise sync&share product vendor or service provider. We acknowledge the fact that the piloting of the first working interface prototype will be carried out in an ownCloud environment that should not effect the adoption of the open API in any other vendor and provider domain.
Community effort - open for participation
A collaborative project was established under the umbrella of GÉANT called the OpenCloudMesh project on 22 October 2015. The kick-off meeting was held in Vienna, Austria.
The project is co-managed by Peter Szegedi (GÉANT), Jakub Moscicki (CERN) and Christian Schmitz (ownCloud). This combination of project management ensures that all the major stakeholders – GÉANT National Research and Education Networks, CERN research community and ownCloud Inc. as a commercial company with its open source developers community – are equally represented in the project and the technical, management and business aspects are well-covered.
The collaborative project is fully open to any participation and in-kind contributions. Interested parties can subscribe to the mailing list at:
Key stakeholders
Name | Organization | Interest / Involvement / Role | RACI | Stakeholder Comments |
---|---|---|---|---|
Peter Szegedi | GÉANT | Project management (about 200h/year) | A, R | Committed |
Jakub Moscicki Massimo Lamanna | CERN | Project management | A, R | Committed |
Frank Karlitschek Christian Schmitz | ownCloud Inc. | Project management | A, R | Committed |
Charles du Jeu David Gillard | Pydio | Contribute to the specifications and development | R, C | Committed |
Rogier Spoor | SURFnet | Contribute to the specifications and development | R, C |
|
Ron Trompert | SURFsara | Contribute to the specifications and development | R, C | Committed |
Christoph Herzog Simon Leinen | SWITCH | Contribute to the specifications and development | R, C | Committed |
Guido Aben David Jericho | AARNet | Contribute to the specifications and development | R, C | Committed |
Holger Angenent | Sciebo / Uni Münster | Contribute to the specifications and development | R, C | Committed |
David Antoš | CESNET | Contribute to the specifications and development | R, C | Committed |
Frederik Orellana | DeIC | Contribute to the specifications and development | R, C |
|
Kurt Bauer | ACOnet | Contribute to the specifications and development | R, C |
|
Christian Kracher | University of Vienna | Contribute to the specifications and development | R, C | Committed |
Jari Miettinen | CSC/Funet / EUDAT | Contribute to the specifications and development | R, C |
|
Andreas Eckey | Technische Universität Berlin | Contribute to the specifications and development | R, C | |
Woojin Seok | KISTI | Interest from South Korea | I | |
Benedikt Wegmann Ralph Krimmel | GWDG | Interest | I | |
Hrachya Astsatryan Andranik Hayrapetyan Wahi | ASNET-AM | Contribute to the specifications and development | R, C | Committed |
The project will be delivered in phases.
Project plan (Phase II.)
The Phase II. objectives and project planning will be defined as we go. A demonstration is expected at TNC'16 in June 2016.
PHASE II. | DESCRIPTION | COMMENT | ASSIGNED TO | START DATE | END DATE | STATUS | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1. Pre-project (preparation) | Initiation Pick up the results on Phase I. See section 4.2 Deliverable of Phase I. below... | Create the new structure of two WGs:
Assign mailing lists above | Peter Szegedi | 19 January 2016 | 5 February 2016 |
| ||||||||
2. Initiation | ||||||||||||||
2.1 | Call for a kick-off video conference | What are the next steps for OCM? 1. Understanding current usage of Federated Sharing feature - How many ownCloud sites in our community have this feature enabled? - How many admins know about this feature? - What prevents them from enabling it? - If enabled, how many users have already used this functionality.
2. General feeling is that we do not have enough understanding of this technology Tilo proposed to setup a test bed where we could check various technical aspects (ref: http://cs3.ethz.ch/presentations/Interoperability/02Moscicki.pdf) and get confidence.
3. Involvement of pioneer users Tilo proposed to involve pioneer users early on to get real user feedback. The extent of exposure of the users should be function of progress in point (2).
Possible candidate users identified already: - Physicists from ETH and CERN. - Others... | Christian Schmitz | 19 January 2016 | 11 February 2016 |
| ||||||||
2.2 | Define objectives, key results and timelines |
| Peter Szegedi | 11 February 2016 | 11 March 2016 |
| ||||||||
3. Subsequent stages (execution) | ||||||||||||||
3.1 | Build the OCM community | ownCloud v.8.2 or higher: Uni Münster, Uni Vienna, AARnet, SWITCH, CESNET, GWDG Others very close: SURFnet, CERN Pydio v.x?: ASNET-AM, University of Lausanne (TBC) | Peter Szegedi | 19 January 2016 |
| |||||||||
3.2 | KEY - Involvement of other vendors... | Locate other vendors and identify user cases, universities with two products and cross-sharing needs. 18-02-2015 News from Christian: Pydio will most likely join OCM. They are actually aiming for test implementation by end of March. 16-03-2016 Charles du Jeu and David Gillard from Pydio (https://pydio.com/) joined the OCM project. Charles, the CEO/CTO of Pydio, confirmed that that are preparing the release of an important dev version, that actually contains the Federated Sharing API implementation. See https://pydio.com/en/community/releases/pydio-core/pydio-core-631-development-release for more info. Pydio is making its best to have this release transformed to a stable one by the end of this month.
| Christian Schmitz Peter Szegedi | 19 January 2016 | 16 March 2016 |
| ||||||||
3.3 | Demonstrate inter-vendor OCM functionality | 29-03-2016 OCM call at 2pm CET 1) Charles (Pydio) demonstrated via screen-share the OCM functionality that has been implemented by Pydio in their software. Federated sharing of files was shown between two local independent instances of Pydio v.6.4.0.(to be released on 30 March) running locally. Comments:
2) Charles (Pydio) and Frank (ownCloud) also tested the interoperability (technical feasibility) of the OCM protocol implementations between Pydio v.6.4 and ownCloud v.9.0 offline. It was reported successful but yet to be seen...
| Charles du Jeu David Gillard Frank and Lukas | 16 March 2016 | 29 March 2016 |
| ||||||||
3.4 | Multi-vendor OCM validation by the Community ownCloud - Pydio | 1) AARNet (Guido) initiated some initial testing with ASNET-AM (Hrachya) AARNet (Australia) uses ownCloud and ASNET-AM (Armenia) uses Pydio.
2) More Pydio users and test scenarios to be defined (see 3.1)
| Guido Aben Hrachya Astsatryan Andranik Hayrapetyan Wahi |
| ||||||||||
4. Delivery | ||||||||||||||
4.1 | Demonstration at the TNC'16 Conference in Prague, Czech Republic | Booth demos: ownCloud, Pydio?, Interoperability demo at GÉANT bootkbooth Lightning talk by Guido Aben: submitted and approved DEMO IDEAS: We can redo the same demo as shown at CS3 but fancier and bigger, also involving more sites. Additional sites which would like to join this time for the demo:
| Christian Schmitz Guido Aben | 19 January 2016 | 13 June 2016 |
| ||||||||
4.2 | Demo feedback | Collect and made available... | 30 June 2016 |
| ||||||||||
5. Closing | Publish results and define next phase | 30 June 2016 |
|
Project plan (Phase I.)
Phase I. aims at demonstrating the first working prototype of the OCM protocol (API v1.0 BETA) functionally working between two separate administrative ownCloud domains (i.e. between two NRENs).
October 2015 - February 2016
PHASE I. | DESCRIPTION | COMMENT | ASSIGNED TO | START DATE | END DATE | STATUS | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1. Pre-project (preparation) | Start collecting organizations and people interested in joining the initiative. | Mailing list to be created. Announcements to be made. | Peter Szegedi Christian Schmitz | 8 February 2015 | 15 June 2015 |
| ||||||||
2. Initiation | ||||||||||||||
2.1 | ownCloud to release the first version of the API code and documentation. | DELAYED Code v.0.002 has been released on 27 July 2015 by ownCloud Inc. Comments have been provided by CERN. | Christian Schmitz | 8 February 2015 | 27 July 2015 |
| ||||||||
2.2 | Create a project team, estimate budget and organize a kick-off meeting. | VC for coordination on 24 August 2015.
Pre-launch meeting organized by ownCloud on 28 August 2015 in Berlin. Peter (GÉANT), Guido (AARnet) and Kuba (CERN) and others.
NIF/PID to be submitted and mailing list migration to be done. Kick-off meeting: 22 October, 2015 in Vienna, Austria The slides of the event can be found here. | Peter Szegedi Christian Schmitz | 8 February 2015 | 22 October 2015 |
| ||||||||
3. Subsequent stages (execution) | ||||||||||||||
3.1 | Get the API v.0.004 code and documentation, define the participating domains, initiate the first tests. | DRAFT protocol definition v.0.004 released OpenCloudMesh = ownCloudMesh | Christian Schmitz | 15 June 2015 | 28 Augustus 2015 |
| ||||||||
3.2 | Demonstrate the first working prototype | Uni Münster server-to-server sharing (i.e. federated cloud sharing) feature has been demonstrated by Holger to Kuba and Peter.
There was an agreement to
Comments from Kuba:
| Holger Angenent | 22 October 2015 | 18 November 2015 |
| ||||||||
3.3 | Prepare for a demonstration of the federated cloud sharing feature between two administrative domains, say SURFsara and Uni Münster | We are looking for volunteers with ownCloud server version v.8 or above to test the feature. Ron, SURFsara pointed out the the Federated Cloud Sharing feature does not work together with SAML/Shibboleth based authentication. This is a showstopper for a planned SURFdrive - Uni Münster demonstration. Ticket has been created: https://github.com/owncloud/core/issues/21227 ownCloud is working on a quick work-around. FCS just needs a user name. Partners to demonstrate Federated Cloud Sharing on 19 January 2016:
h_zimm01@uni-muenster.de@uni-muenster.sciebo.de
david.jericho@aarnet.edu.au@cloudstor.aarnet.edu.au/plus
krachec6@ucloud-test-files.univie.ac.at Demonstration has been successfully done:
| Holger Angenent Andreas Wilmer Ron Trompert David Jericho Guido Aben Christian Kracher Simon Leinen | 18 November 2015 | 19 January 2016 |
| ||||||||
3.4 | Initiate discussions about policies, metadata release, directories, legal issues, etc. | Two main topics have been identified (18 November 2015)
Mind map (15 January 2016): Kuba at CERN talked about the open issues and Simon at SWITCH talked about the standardization aspects (19 January 2016)
Kuba: http://cs3.ethz.ch/presentations/Interoperability/02Moscicki.pdf Simon: http://cs3.ethz.ch/presentations/Interoperability/04Leinen.pdf | Jakub Moscicki Simon Leinen All | 18 November 2015 | 19 January 2016 |
| ||||||||
4. Delivery | ||||||||||||||
4.1 Workshop | Open API v.1.0 documented and released at least in BETA version with the intention to come up v.2.0 vendor agnostic version (IETF WG) | Cloud Services for Synchronization and Sharing (CS3) Workshop ETH Zürich, Switzerland; January 18-19 2016 http://cs3.ethz.ch/program.html Slides and presentation/demo materials are available! Praying for OCM to work... | Peter Szegedi Christian Schmitz Kuba Moscicki Simon Leinen | 15 November 2015 | 19 January 2016 |
| ||||||||
4.2 Deliverable | Come up with recommendations for the development of the API towards a new v.1.0 according to the requirements of an open standard. | Set of recommendations: Integration with Macaroons
Cloud user lookup service
David:
Protocol negotiation handshake Holger:
David:
Security and Trust
Data path for federated shares
Pick a preferred home for standardization
All the recommendations have been fed into the Phase II initiation phase. | All | 19 January 2016 | 05 February 2016 |
| ||||||||
5. Closing | Summary | The OCM team agreed to continue and split into two working groups (WGs):
The key objective of the Strategic WG will be to reach the ultimate "Open Cloud Mesh standard" and to oversee (not overdrive) the Technical WG making sure that the open API design principles are properly addressed. The key objective of the Technical WG will be to deliver one or more working prototypes of OCM and to provide (not force) input to the Strategic WG making sure that their assumptions are realistic. The detailed objectives and key results of the Phase II will be determined in the initiation phase. Phase I is considered to be CLOSED. News item: http://www.geant.org/News_and_Events/Pages/OpenCloudMesh.aspx | Peter Szegedi | 19 January 2016 | 05 February 2016 |
|
History
- Around early 2012, TF-Storage participants started to actively look into data storage software platforms in order to provide on-premise file-based sync&share (aka. Dropbox-like) services to their constituencies.
- Some NRENs even ventured into the development of a proof-of-concept tool called the Trusted Cloud Drive (TCD) under TERENA
- By mid 2013, ownCloud appeared to be the most promising one with a growing open-source development community behind.
- In December 2013, the GÉANT Association (formerly known as TERENA) and ownCloud Inc. made an agreement that serves to facilitate the desire of various National Research and Education Networking organisations (NRENs) to introduce services based on ownCloud technology and/or to offer the technology to their constituencies.
- As part of this collaboration effort, in January 2015, ownCloud initiated an idea (aka. OpenCloudMesh) to interconnect the individual on-premise private cloud domains at the server side in order to provide federated sharing and syncing functionality between the different administrative domains.