Use both Cirrus CLI and Podman to better manage and secure container environments for rootless builds.
Read More at Enable Sysadmin
How Cirrus CLI uses Podman to achieve rootless builds
CNCF: Fostering the Evolution of TiKV
PingCAP had high hopes that its TiKV project would develop into a building block for the next generation of distributed systems by providing reliable high quality and practical storage foundation. To accomplish that, it decided to contribute TiKV to the Cloud Native Computing Foundation (CNCF) to make it vendor-neutral and widely used across organizations. It seems headed in that direction, especially now that the project recently graduated, further demonstrating its maturity and sustainability. On behalf of the Linux Foundation, Swapnil Bhartiya, founder and host of TFiR, sat down with two members of the TiKV project, Siddon Tang and Calvin Weng, to learn more about the project’s evolution.
Here is a transcript of the discussion:
Swapnil Bhartiya: What is TiKV project and what problem are you trying to solve?
Siddon Tang: TiKV is an open source, distributed transactional key value database. TiKV is inspired by Google Spanner and HBase, but the design is simpler and more practical. Why did we develop TiKV at PingCAP? We want to build a distributed database with SQL compatibility. We built the SQL and then we wanted to build a distributed key-value storage layer that supported our database. At first, we tried to use HBase, but its performance was not what we expected so we decided to build our own distributed key-value database. That’s how TiKV started.
Calvin Weng: It was originally created to complement TiDB, but we soon realized that the TiKV project could be decoupled from TiDB and serve as a unified distributed storage layer that supported distributed transactions, horizontal scalability, and cloud-native architecture.
We also realized that with the amount of data we generate, there could be a demand for such a solution in the cloud-native communities. So, we contributed it to the CNCF to develop it as a building block for the next generation of distributed systems by providing a reliable high quality and practical storage foundation.
Swapnil Bhartiya: How is CNCF helping the TiKV project and the community?
Calvin Weng: Thanks for the question. I am a liaison between the CNCF and the TiKV project. The CNCF has been immensely helpful in shaping TiKV into what it is today in terms of both the project and the community. There are a few things that I would like to elaborate on and the first is neutrality. CNCF provides a neutral home to projects like ours, so that developers from different organizations are willing to collaborate, contribute and eventually become the leaders in the project. This is very important for the broader community to perceive TiKV as a vendor-neutral and universal project that belongs to the community instead of a single company like PingCAP. People will feel comfortable adopting it or developing their own apps on TiKV.
Another important aspect is exposure, which includes publicity and marketing support that we get from CNCF so that we are known by the broader community. More people and more companies could get involved, which also means more adoption.
Last but not least is diversity in the maintainer and the contributor structure. This is a very important criterion for CNCF graduation.
Swapnil Bhartiya: Since you mentioned graduation, can you talk about what it means for a project like TiKV to become a graduated project? How does it affect the project and what does it mean for its users?
Calvin Weng: TiKV has a lot of adoptions. There are more than 1,000 deployments in production. It is battle-tested. Moving from incubation to graduation is a very solid and convincing validation of the technology, its open governance, its vision, maturity and sustainability.
From a user’s perspective, graduation means the credibility and reliability of the project. It means that the TiKV project is a mature enough project for cloud-native architecture. It also means that the TiKV community is an active and healthy community. It boosts the confidence of users.
Swapnil Bhartiya: One last question before we wrap this up: can you talk about the roadmap of the TiKV project?
Siddon Tang: Our focus is on making it faster, easier-to-use, and cost-effective. We just released the 4.0 version and in the next major release of 5.0, we want it to be more cloud-friendly and be able to smoothly run on AWS S3, AWS EBS Cloud Disk or any other cloud storage. We are also working on making TiKV handle different workloads. We are also working on adding support for other database engines so it can support different workloads. The long-term goal is to introduce AI so it can use different engines to certify different workloads.
Communication by example: Which methods do high-performing open source communities use?
“Good words are worth much, and cost little.” George Herbert
Although effective communication is an essential life skill, it is the most critical element in any business [2]. Lack of accurate communication is the common cause of any organization’s issues, causing conflicts, reducing client relationships, team effectiveness, and profitability [2]. According to the Project Management Institute (PMI), ineffective communication is the main contributor to project failure one-third of the time. It has a negative impact on project success more than half of the time [1].
In open source projects where there is a diverse and world spread community, effective communication is the key to projects’ success. Using the right technology is crucial for that. So, which tools do open source communities use for communication?
Open Source community communication by example
Ubuntu
The Ubuntu community uses mailing lists for development and team coordination. The mailing lists are split into announcements and news, support, development, testing and quality assurance, and general (such as translation, marketing, and documentation) [3]. Despite the mailing lists, IRC (Internet Relay Chat) channels are used for informal daily chats and short-term coordination tasks [3]. If someone wants to know what is going on on Ubuntu, but doesn’t want to subscribe to the high traffic mailing list, the web forum can be used to get support and discuss the future of Ubuntu. Finally, Ask Ubuntu can be used to ask technical questions.
Linux Kernel
Mailing lists are the main communication channels in the Linux Kernel. For newcomers that would like to learn more about the Linux kernel development, there is the kernelnewbies resource and #kernelnewbies IRC channel on OFTC. This online resource provides information on basic kernel development questions. Additionally, the kernelnewbies IRC channel is a vehicle for contributors to ask questions in real-time and get help from experts in the kernel community. The Linux Kernel Mailing List (LKML) is where most development discussions and announcements are made. Kernel developers send patches to the mailing lists as outlined in the Submitting patches: the essential guide to getting your code into the kernel. The archives from each mailing list can be found at https://lore.kernel.org/lists.html.
Shuah Khan, a Linux Fellow, mentioned in an interview [4] that before contributing to the Linux Kernel, it’s important to subscribe to the kernel-related mailing lists “to understand the dynamics.” Khan said, “The process works like this: you walk into a room. People are gathering in small groups and are talking to each other. You have to break into one of these conversations. That is the process of watching the mailing lists, watching the interaction, and learning from that before you start sending out a patch.”
OpenStack
OpenStack has many communication channels such as IRC channels for both public meetings and projects as well as mailing lists. The mailing lists are used to asynchronously communicate and share information, team communication, and cross-project communication. Additionally, mailing lists in OpenStack are used to communicate with non-developer community members of OpenStack [5].
GNOME
IRC channels are one of the most important communication methods in Gnome. They are a google place to know what the community is talking about and also ask for help. There are many channels on Discourse, including discussions about Gnome’s sub-projects, community-related topics, internationalization, etc. Similar to other communities, mailing lists can be used for discussing specific topics. Finally, PlanetGnome and GnomeNews can be used to follow the latest news of the project.
So, where does communication occur in open source projects?
As observed in our previous discussion, mailing lists seem to be the most used communication method. Previous work has also found that “mailing lists are the bread and butter of project communications” [11] and that “the developer mailing list is the primary communication channel for an OSS project” [12]. However, as we have previously mentioned, mailing lists are not the only communication channel used in OSS. Other channels (such as IRC channels and forums) also play an important role.
Guzzy et. al [10] mention that when more than one communication repository exists, the policy of most OSS is to transfer all official decisions and useful discussions to the mailing lists, so that they can later be retrieved. Thus, traceability and transparency of information is an important matter here.
The benefit of using mailing lists is that it is an asynchronous form of communication, and it is an easy resource to share information with the entire community. Additionally, mailing lists allow people that are in different timezones to engage, as well as people that have different levels of English proficiency, may better manage it in text messages [5].
However, mailing lists might also have their disadvantages. Previous work [10] found that developers have problems maintaining awareness of each other’s work when discussing on the mailing lists. Additionally, recovering traceability links among different communication repositories might help researchers and community members to have a more complete picture of the development process.
What are the common DOs and DON’Ts when using OSS mailing lists?
Given that mailing lists are one of the common ways to communicate in open source projects, it is worth knowing how to communicate in mailing lists. Although each project has its own set of rules, certain conventions should be followed.
DOs
Subject
-
-
- Prefix the subject with topic tags in square brackets. This makes email threads easier for readers to categorize and decide what they should read quickly. For example, OpenStack has documentation [13] establishing how to prefix the subject, i.e., community members should use [docs] to address any kind of documentation discussions that are cross-projects and so on.
-
-
-
- Sometimes it’s appropriate to change the subject rather than start a new thread.
- Exceptions: Linux Kernel mailing lists use “bottom post” protocol (writing the message below the original text) rather than “top post” (writing the message above the original text of an email, which is what most mail clients are set to do by default.)
- Sometimes it’s appropriate to change the subject rather than start a new thread.
-
Formating
-
-
- Plain text: Send your email as plain text only! Please, don’t send HTML emails.
- Line wrapping: Lines should be wrapped at 72 characters or fewer.
-
Replies
-
-
- Always use inline replies, i.e., break the original message by replying to each specific part of the message.
- When replying to long discussions, trim your message and leave only the relevant parts to the reply.
-
DON’Ts
-
-
- Avoid cross-posting, i.e., posting the same message to many mailing lists at the same time.
- Exceptions: The Linux Kernel maintains mailing lists for each subsystem, and patches are often sent to multiple mailing lists for review and discussion. However, avoid “top posting” on a Linux Kernel mailing list.
- Avoid cross-posting, i.e., posting the same message to many mailing lists at the same time.
-
-
-
- Avoid sending the wrong topic to the wrong mailing list. Make sure that your topic is the topic of the mailing list.
-
Setting up your email client
The Linux Kernel has great documentation on setting different email clients according to the rules mentioned above.
How to minimize the harm caused by conflicts?
Even if the code of conduct is applied, conflicts might exist. Many actions can be taken in case of dispute, and here are some examples:
Gather information about the situation
If someone has violated the code of conduct, you should carefully analyze the situation according to the experience working with that person [6]. It is essential to read the past comments and interactions with that person to have an unbiased perspective about what happened. Stephanie Zvan [7] has mentioned that the best way to avoid a conflict is not to get pulled into an argument. It is important to focus on what you need to do instead of getting sidetracked into dealing with others’ behaviors.
Take appropriate actions
Two ways to respond to the code of conduct violation is that the moderator of the community (i) in a thoughtful way explain in public how the person’s behavior affected the community, or (ii) privately reach out to the person and explain how that behavior was negative [6].
“A code of conduct that isn’t (or can’t be) enforced is worse than no code of conduct at all: it sends the message that the values in the code of conduct aren’t actually important or respected in your community.” Ada Initiative
General tips
- Open source projects are, in large part, successful due to the collaborative nature of projects. Thus, start conversations that lead to collaboration. That means, give feedback, support each other’s communication, and share your ideas.
- There is no additional cost to being transparent and authentic with your community. In that way, it is easy to keep your team informed, empowered, and focused on one specific goal or task.
About the author:
Isabella Ferreira is an Advocate at TARS Foundation, a cloud-native open-source microservice foundation under the Linux Foundation.
References:
[3] https://wiki.ubuntu.com/ContributeToUbuntu#Community_Communication
[4] https://thenewstack.io/how-to-begin-your-journey-as-a-contributor-to-the-linux-kernel/
[5] https://docs.openstack.org/project-team-guide/open-community.html
[7]
https://the-orbit.net/almostdiamonds/2014/04/10/so-youve-got-yourself-a-policy-now-what/
[10] Guzzi, Anja, et al. “Communication in open source software development mailing lists.” 2013 10th Working Conference on Mining Software Repositories (MSR). IEEE, 2013.
[11] Fogel, Karl. Producing open source software: How to run a successful free software project. ” O’Reilly Media, Inc.”, 2005.
[12] Gutwin, Carl, Reagan Penner, and Kevin Schneider. “Group awareness in distributed software development.” Proceedings of the 2004 ACM conference on Computer supported cooperative work. 2004.
[13] https://docs.openstack.org/project-team-guide/open-community.html#mailing-lists
This Linux Foundation Platinum Sponsor content was contributed by Tencent.
Logical Volume Manager (LVM) versus standard partitioning in Linux
Use this guide to integrate the flexibility, scalability, and increased features of LVM into your server storage strategies. Traditional partitioning is good, but LVM is better.
Read More at Enable Sysadmin
Aggregating Ansible Tower Logs to Splunk
Log forwarding from Ansible Tower helps you gain insights and a better view of Tower utilization and trends.
Read More at Enable Sysadmin
Consolidation of AI, ML and Date Projects at The Linux Foundation
The Linux Foundation consolidated its projects around AI, ML & Data by bringing them under the umbrella of the LF AI & Data Foundation. Swapnil Bhartiya, founder and host at TFiR.io, sat down with Ibrahim Haddad, Executive Director of LF AI & Data to discuss this consolidation.
Transcript of the discussion:
Swapnil Bhartiya: A lot of consolidation is happening within the Linux Foundation around AI/ML projects. Can you talk about what AI/ML & data projects are there under the Linux Foundation umbrella right now?
Ibrahim Haddad: So, if you think of Linux Foundation, it is kind of a foundation of foundations. There are multiple umbrella foundations. There’s the CNCF (Cloud Native Computing Foundation), there’s LF Edge, there’s the Hyperledger project, automotive, et cetera. And LF AI & Data is one of these umbrella foundations. We share the same goal, which is to accelerate the development of open-source projects and innovation. However, we each do it in our specific domains.
We’re focused on AI, machine learning, deep learning, and the data aspects of AI. The LF AI & Data Foundation was initially kicked off as LF Deep Learning in March of 2018. We grew a bit, and we started to host projects in other subdomains within the AI umbrella. And then we rebranded again to LF AI & Data to reflect the additional growth in our portfolio.
As of today, we host 22 projects across multiple domains of machine learning, deep learning, data models, and trusted AI. We have, I believe, 36 numbered companies that are involved in our foundation.
Swapnil Bhartiya: Within the Linux Foundation, there are a lot of projects that at times overlap, and then there are gaps as well. So, within the AI/ML space, where do you still see gaps that need to be bridged and overlaps that need consolidation?
Ibrahim Haddad: When a project is contributed to the foundation, we see under which umbrella it fits, however it’s the decision of the project where they want to go, we only offer guidance. If projects do overlap under the same umbrella, it’s their call to make. In terms of consolidation, we’re actually in the process of doing this at least in the AI space. We recently announced the formation of LF AI & Data, which consolidates two projects – LF AI Foundation and ODPi.
Swapnil Bhartiya: Can you also talk about what are the new goals or new areas that the Foundation is focusing on after this consolidation and merger?
Ibrahim Haddad: The first one is increasing the collaboration between the projects that are on the data side and the traditional open-source AI projects that we host. We host about seven projects that focus on the data and 15 projects in the general AI domain. One of the activities we launched, which we are going to accelerate in 2021, is creating integration across different projects so that companies see a tighter integration within projects inside the foundation.
The second area is trusted AI to build trust and a responsible AI system, which is really a hot topic across industry verticals including governments, NGOs and companies. They all are putting emphasis on building fair systems, systems that don’t create bias, systems that are transparent, systems that are robust. Building trust with the consumer of these systems is a very critical thing. So trusted and responsible AI would be a key area in addition to the integration and growing the data/AI collaborations.
How to audit permissions with the find command
You can audit permissions on your Linux system by using the find command with the -perm option. Plus four bonus permissions auditing methods.
Read More at Enable Sysadmin
How to setup WireGuard on Oracle Linux
Oracle Linux engineer William Kucharski provides an introduction to the VPN protocol WireGuard WireGuard has received a lot of attention of late as a new, easier to use VPN mechanism, and it has now been added to Unbreakable Enterprise Kernel 6 Update 1 as a technology preview. But what is it, and how do I use it? What is…
Click to Read More at Oracle Linux Kernel Development