Linux.com

Home News Enterprise Computing Cloud Computing Building an Inter-University Private Cloud with Open Source ownCloud

Building an Inter-University Private Cloud with Open Source ownCloud

Munster

In late 2011, a lively discussion (we enjoy lively discussions here in Germany) among the IT managers of the publicly-funded research universities in Northrhine-Westfalia (NRW), Germany’s most populous federal state, started over a set of interrelated topics:

·         Endorsement or ban of public cloud services at universities: Since commercial, consumer-grade cloud services like Google Mail and Google Apps or Dropbox were growing in acceptance, also amongst researches at universities, we had to define our position on this issue. Should we endorse them as cost saving alternatives, e.g. for student mail accounts, or should we fight them, because we consider them insecure with respect to data privacy and confidentiality – a very serious matter in Germany in general and for research institutions in particular.

Raimund Vogl, director of IT at Münster University·         Need for regulation of researchers’ cloud service usage: When storing data from work (such as personal information on student work or confidential research results) in cloud services, researchers put themselves in peril of violating their office duties. But regulations imposed by a university banning the use of cloud services altogether for data protection reasons do not take into account the realities of research life – you have to be able to offer a convenient alternative. So this is what university IT service providers have to come up with.

·         Inter-university IT co-operation: With the science ministry boosting the universities’ autonomy by bailing out of direct involvement in university operations in 2007, self-organized cooperation was expected to save costs, improve quality and thereby justify the ministry’s move to grant this new freedom. Cost intensive IT projects especially were under scrutiny in times of dwindling budgets. But the discussion on possible scenarios for inter-university IT cooperation actually did stall in a quite early stage – since virtually all IT services used by university affiliates were already provided by local IT centers.

With the demand for an on-premise alternative to cloud storage services like Dropbox – a demand strongly articulated by both researchers and students, even back in 2011 long before the Edward Snowden NSA disclosures – things were different. This field was not covered yet, and here, the creation of an inter-university private cloud made perfect sense. So in Spring 2012, the council of university IT managers in NRW (ARNW) started a project for an inter-university sync & share cloud storage service and the IT center of Münster University (ZIV) was designated as the project lead.

An extensive market research and product evaluation started soon after. Even at this stage, in the summer of 2012, ownCloud, back then a complete newcomer to the open source scene, was seen as the most promising candidate for our project – envisioned to provide about 6 Petabyte of free-to-use cloud storage to 500,000 affiliates of more than 30 public research and applied science universities in NRW (the storage was ours, access to it comes via ownCloud).

For a project of this scale, with the investment of substantial public funding, an empirically well-founded decision-making process was indispensable. In collaboration with the Information Systems department of Münster University, extensive studies on user expectations, demands and projected adoption were carried out and will be continued throughout the upcoming 5 years of service operation for targeted marketing campaigns and continuous service improvement. Already, valuable scientific insights resulted from this and have been published:

  • We had a paper at ECIS 2014 (European Conference on Information Systems). The proceedings are open access and the link to our paper is here.
  • Just last week, we had a paper at HCII2014 in Session S104: "Cloud Storage Services in Higher Education – Results of a Preliminary Study in the Context of the Sync&Share-Project in Germany," Christian Meske, Stefan Stieglitz, Raimund Vogl, Dominik Rudolph, Ayten, Öksüz, University of Muenster, Germany.

As to the sync & share software solution, the continuous market research of nearly 2 years had shown that ownCloud developed well over time. By the time of final decision-making in April 2014, it featured a software suite that was almost complete for the demands articulated by our users – with ownCloud committed to provide the add-ons still missing for a 500,000 user setup and storage locations dispersed at three university IT centers across the state.

The open source nature of ownCloud was another important argument to see this product as singled out amongst its competitors and to directly award the contract without a public tendering process. Open source creates trust by ensuring users that there are no back doors and it gives confidence that the development of new desirable features will continue for at least the upcoming 5 years our service is scheduled to run – driven by a broad community of contributors, and possibly also through student and research projects from the universities participating.

After successfully completing a peer review in early 2014, mandatory to receive the substantial funding grant from the NRW ministry of science, the project is currently in the procurement phase, with the procurement of the scale-out storage platform still being in the public tendering process. We will be tracking our progress publicly and will report back here periodically.

 

Comments

Subscribe to Comments Feed
  • Jay Jimenez Said:

    Hi Raimund, What a huge implementation! " 6 Petabyte of free-to-use cloud storage to 500,000 affiliates of more than 30 public research and applied science universities in NRW" I'm wondering how many ownCloud servers you will actually use per university per X amount of users. In my setup, I will assign a maximum of 1000 users per server per site. So If I have 5000 users on 5 international remote sites, I will have 5 ownCloud servers.I created a back-end replication service to synchronize shares information of those 5 servers. The result will be a federated Own Cloud servers utilizing existing corporate AD environment. I will be glad to see how you do yours. Cheers Jay

  • Raimund Vogl Said:

    Hi We will not provide an ownCloud server to every single university but have only three sites with ownCloud running. So each site will have over 150.000 users - potentialy - we expect 70%-80% adoption of this service by the eligeble users. At the beginning, these 3 sites will be coupled with the server to server mechanism introduced with ownCloud 7 (https://owncloud.org/blog/owncloud-7-sneak-peek-improved-sharing/). Later on, we will develop this further to make sharing between different servers as transparent as possible to the users. Regards Raimund Vogl

  • Jay Jimenez Said:

    Hi Raimund. I think Ownlocud 7 server to server sharing will work great as long as the user X of server instance A has a knowledge that user Y is located on server instance B and not expecting that user Y is searchable on the share text box of server instance A. For example, if users X and Y are students of different universities, user X shouldn't expect that user Y's name is searchable on share link text box. Instead, user X should send the link through email to user Y. Opposite logic applies if users X and Y are both students of the same university, user X should expect that user Y's name is searchable on the share textbox. It could drill down to 1 server instance per college or 1 server instance per department of a college of a university. Ultimately, proper rules must be set from day 1 in order not to confuse the users on how to share to users of the same department,college,university or other university. I hope I did not interpret that 150,000 users per site will be handled only by 1 ownCloud server. At 1,000 files per user, that's 150,000,000 rows of a single database table oc_filecache. I have done a transparent sharing federated setup(3 server instances) for few hundreds of users(mostly geologists and engineers with thousands of files each user) for testing before scaling it to 5000 users. I just couldn't imagine how my databases will perform for 150,000 to 500,000 users that you have. Luckily, I don't have that numbers. Good luck Raimund. Hope to see more updates on your implementation soon. Regards Jay Jimenez

Upcoming Linux Foundation Courses

  1. LFS220 Linux System Administration
    05 Jan » 08 Jan - Virtual
    Details
  2. LFD331 Developing Linux Device Drivers
    12 Jan » 16 Jan - Virtual
    Details
  3. LFS520 OpenStack Cloud Architecture and Deployment
    12 Jan » 15 Jan - Virtual
    Details

View All Upcoming Courses

Become an Individual Member
Check out the Friday Funnies

Sign Up For the Linux.com Newsletter


Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board