Notes on BPF: Building applications using libbpf
Click to Read More at Oracle Linux Kernel Development
Notes on BPF: Building applications using libbpf
This article was originally posted at Major League Hacking
This Summer, Major League Hacking (MLH) is launching the Production Engineering Track of the MLH Fellowship, powered by Facebook. This is a 12-week educational program that will use industry leading curriculum from the Linux Foundation Training & Certification and hands-on project-based learning to teach students how to become Production Engineers. The program will provide opportunities to 100 aspiring software engineers to broaden their skills and career options, by learning important Production Engineering and DevOps skills. The program will run between June 7 – August 30, 2021 and will be open to United States, Mexico, and Canada based students who are enrolled in a 4 year degree granting program. Applications are open and will run until May 28, 2021.
What is Production Engineering anyway?
Early career software engineers are passionate and motivated to learn new skills and create a positive impact on the world, but many have not been exposed to the wider array of career options that are available to them. Production Engineering, also known as Site Reliability Engineering and DevOps, is one of the most in-demand skill sets that leading technology companies are hiring for, yet it is not widely available as an educational option in university settings.
Production Engineers (PEs) at Facebook are a hybrid between software and systems engineers and are core to engineering efforts that keep Facebook running smoothly and scaling efficiently. PEs work within Facebook’s product and infrastructure teams to make sure their products and services are reliable and scalable. This means writing code and debugging hard problems in production systems across Facebook services like Instagram, WhatsApp, and Oculus as well as backend services like Storage, Cache and Network.
What is the Production Engineering Track of the MLH Fellowship?
Initially launched in Summer 2020, the MLH Fellowship pairs early career software engineers with widely used open source projects like React, Jest, Docusaurus. This gives them the opportunity to apply their knowledge to real world projects, which helps them learn important concepts and patterns while also having production-level code to showcase in their portfolio. Through the Fellowship, MLH has created opportunities for hundreds of developers from around the world to level up and hone their skills in a 12-week cohort surrounded by peers and expert mentors. The Production Engineering Track takes this proven model and expands it to a broader range of technology disciplines, creating even more valuable opportunities for developers starting off their careers.
Program participants will gain practical skills thanks to educational content from Linux Foundation Training & Certification’s LFS201 – Essentials of System Administration training course, which covers how to administer, configure and upgrade Linux systems, along with the tools and concepts necessary to efficiently build and manage a production Linux infrastructure. By pairing this industry leading curriculum with hands-on project-based learning, students in the Production Engineering Track can build on their foundational software engineering knowledge to learn a broader array of technology skills and open the door to a variety of exciting and challenging new career options. Creating these types of non-traditional learning experiences helps empower developers from many diverse backgrounds and educational institutions to accelerate their careers and make an impact on the world.
Who is eligible?
The Production Engineering Track starts in June, and guarantees a $3,600 educational stipend to each participant. Applications are open now until May 28, 2021. Eligible students are rising sophomores or juniors who are United States, Mexico, and Canada based, enrolled in a 4 year degree granting program, and able to code. MLH invites and encourages people to apply who identify as women or non-binary. MLH also invites and encourages people to apply who identify as Black/African American or LatinX. In partnership with Facebook, MLH is committed to building a more diverse and inclusive tech industry and providing learning opportunities to under-represented technologists.
The post Introducing the Production Engineering Track of the MLH Fellowship, powered by Facebook appeared first on Linux Foundation – Training.
Bash scripting: Moving from backtick operator to $ parentheses
Are you hooked on backticks in your shell scripts? You should consider $ parens.
Mon, 4/26/2021 at 7:49pm
Image by Arek Socha from Pixabay
There are certain commands or tricks that you start using as a sysadmin, which you simply incorporate into your arsenal and never really stop to analyze in-depth all the options or alternatives to them.
Command line utilities
Read More at Enable Sysadmin
OPA, or Open Policy Agent, is one of the most popular open-source tools for administrators to have fine-grained control for their cloud-native environment. The project was created by Styra and contributed to CNCF. The project recently graduated from CNCF to join matured projects like Kubernetes. Torin Sandall, VP of Open Source at Styra, sat down with Swapnil Bhartiya, CEO of TFiR and host of video interviews at Linux.com, to talk about the significance of graduation for a project like OPA which is already being used in production, the importance of OPA in the cloud-native world and what are some of the most exciting use-cases of the project.
The OpenAPI Initiative (OAI), a consortium of forward-looking industry experts who focus on standardizing how APIs are categorized and described, released the OpenAPI Specification 3.1.0 in February. This new version introduces better support for webhooks and adds 100% compatibility with the latest draft (2020-12) of JSON Schema.
Complete information on the OpenAPI Specification (OAS) is available for immediate access here: https://spec.openapis.org/oas/v3.1.0 It includes Definitions, Specifications including Schema Objects, and much more.
Also, the OAI sponsored the creation of new documentation to make it easier to understand the structure of the specification and its benefits. It is available here: https://oai.github.io/Documentation. It is intended for HTTP-based API designers and writers wishing to benefit from having their API formalized in an OpenAPI Description document.
What is the OpenAPI Specification?
The OAS is the industry standard for describing modern APIs. It defines a standard, and programming language-agnostic interface description for HTTP APIs, which allows both humans and computers to discover and understand the capabilities of a service without requiring access to source code, additional documentation, or inspection of network traffic.
The OAS is used by organizations worldwide, including Atlassian, Bloomberg, eBay, Google, IBM, Microsoft, Oracle, Postman, SAP, SmartBear, Vonage, and many more.
JSON Schema Now Supported
The Schema Object defines everything inside the `schema` keyword in OpenAPI. Previously, this was loosely based on JSON Schema and referred to as a “subset superset” because it added some things and removed other things from JSON Schema. The OpenAPI and JSON Schema communities worked together to align the specifications to align tooling and approach.
Supporting modern JSON Schema is a significant step forward.
“The mismatch between OpenAPI JSON Schema-like structures and JSON Schema itself has long been a problem for users and implementers. Full alignment of OpenAPI 3.1.0 with JSON Schema draft 2020-12 will not only save users much pain but also ushers in a new standardized approach to schema extensions,” said Ben Hutton, JSON Schema project lead.
“We’ve spent the last few years (and release) making sure we can clearly hear and understand issues the community faces. With our time-limited volunteer-based effort, not only have we fixed many pain points and added new features, but JSON Schema vocabularies allows for standards to be defined which cater for use cases beyond validation, such as the generation of code, UI, and documentation.”
Major Changes in OpenAPI Specification 3.1.0
- JSON Schema vocabularies alignment
- New top-level element for describing Webhooks that are registered and managed out of band
- Support for identifying API licenses using the standard SPDX identifier
- PathItems object is now optional to make it simpler to create reusable libraries of components. Reusable PathItems can be described in the components object. There is also support for describing APIs secured using client certificates.
Get Involved in the OAI
To learn more about participating in the evolution of the OAS: https://www.openapis.org/participate/how-to-contribute
- Become a Member
- OpenAPI Specification Twitter
- OpenAPI Specification GitHub – Get started immediately!
- Share your OpenAPI Spec v3 Implementations
In order to achieve the impressive goal of full compatibility with modern JSON Schema, the OAS underwent important updates that make things a lot easier for tooling maintainers, as they no longer need to try and guess what draft a schema is by looking at where it is referenced from. Just this makes implementation of OAS 3.1.0 important to evaluate and strongly consider.
Jason Perlow, Editorial Director of the Linux Foundation, chats with Jory Burson, Community Director at the OpenJS Foundation about open standardization efforts and why it is important for open source projects.
JP: Jory, first of all, thanks for doing this interview. Many of us know you from your work at the OpenJS Foundation, the C2PA, and on open standards, and you’re also involved in many other open community collaborations. Can you tell us a bit about yourself and how you got into working on Open Standards at the LF?
The Joint Development Foundation is something I’m new to, but as part of that, I’m very excited to get to support the C2PA, which stands for Coalition for Content Provenance and Authenticity; it’s a new effort as well. They’re going to be working on standards related to media provenance and authenticity — to battle fakes and establish trustworthiness in media formats, so I’m very excited to get to support that project as it grows.
JB: There are the usual challenges that I think face any international or global team, such as coordination of meeting times and balancing the tension between asynchronously conducting business via email lists, GitHub, and that kind of thing. And then more synchronous forms of communication or work, like Slack and actual in-person meetings. Today, we don’t really worry as much about the in-person meetings, but still, there’s like, this considerable overhead of, you know, “human herding” problems that you have to overcome.
Another challenge is understanding the pace at which the organization you’re operating in really moves. This is a complaint we hear from many people new to standardization and are used to developing projects within their product team at a company. Even within an open source project, people are used to things moving perhaps a bit faster and don’t necessarily understand that there are actually built-in checks in the process — in some cases, to ensure that everybody has a chance to review, everybody has an opportunity to comment fairly, and that kind of thing.
Sometimes, because that process is something that’s institutional knowledge, it can be surprising to newcomers in the committees — so they have to learn that there’s this other system that operates at an intentionally different pace. And how does that intersect with your work product? What does that mean for the back timing of your deliverables? That’s another category of things that is “fun” to learn. It makes sense once you’ve experienced it, but maybe running into it for the first time isn’t quite as enjoyable.
JP: Why is it difficult to turn something like a programming language into an internationally accepted standard? In the past, we’ve seen countless flavors of C and Pascal and things like that.
JB: That’s a really good question. I would posit that programming languages are some of the easier types of standards to move forward today because the landscape of what that is and the use cases are fairly clear. Everybody is generally aware of the concept that languages are ideally standardized, and we all agree that this is how this language should work. We’re all going to benefit, and none of us are necessarily, outside of a few cases, trying to build a market in which we’re the dominant player based solely on a language. In my estimation, that tends to be an easier case to bring lots of different stakeholders to the table and get them to agree on how a language should proceed.
In some of the cases you mentioned, as with C, and Pascal, those are older languages. And I think that there’s been a shift in how we think about some of those things, where in the past it was much more challenging to put a new language out there and encourage adoption of that language, as well as a much higher bar and much more difficult sort of task in getting people information out about how that language worked.
JP: … such as for Augmented Reality APIs or some highly specialized 3D rendering thing. So what are the open standardization efforts you are actively working on at the LF now, at this moment?
JB: At this exact moment, I am working with the OpenJS Foundation standards working group, and we’ve got a couple of fun projects that we’re trying to get off the ground. One is creating a Learning Resource Center for people who want to learn more about what standardization activities really look like, what they mean, some of the terminologies, etc.
For example, many people say that getting involved in open source is overwhelming — it’s daunting because there’s a whole glossary of things you might not understand. Well, it’s the same for standardization work, which has its own entire new glossary of things. So we want to create a learning space for people who think they want to get involved. We’re also building out a feedback system for users, open source maintainers, and content authors. This will help them say, “here’s a piece of feedback I have about this specific proposal that may be in front of a committee right now.”
So those are two things. But as I mentioned earlier, I’m still very new to the Linux Foundation. And I’m excited to see what other awesome standardization activities come into the LF.
JP: Why do you feel that the Linux Foundation now needs to double down its open standards efforts?
JB: One of the things that I’ve learned over the last several years working with different international standards organizations is that they have a very firm command of their process. They understand the benefits of why and how a standard is made, why it should get made, those sorts of things. However, they don’t often have as strong a grasp as they ought to around how the software sausage is really made. And I think the Linux Foundation, with all of its amazing open source projects, is way closer to the average developer and the average software engineer and what their reality is like than some of these international standards developing boards because the SDOs are serving different purposes in this grander vision of ICT interoperability.
On the ground, we have, you know, the person who’s got to build the product to make sure it’s fit for purpose, make sure it’s conformant, and they’ve got to make it work for their customers. In the policy realm, we have these standardization folks who are really good at making sure that the policy fits within a regulatory framework, is fair and equitable and that everybody’s had a chance to bring concerns to the table — which the average developer may not have time to be thinking about privacy or security or whatever it might be. So the Linux Foundation and other open source organizations need to fit more of the role of a bridge-builder between these populations because they need to work together to make useful and interoperable technologies for the long term.
That’s not something that one group can do by themselves. Both groups want to make that happen. And I think it’s really important that the LF demonstrate some leadership here.
JP: Is it not enough to make open software projects and get organizations to use them? Or are open standards something distinctly different and separate from open source software?
JB: I think I’ll start by saying there are some pretty big philosophical differences in how we approach a standard versus an open source project. And I think the average developer is pretty comfortable with the idea that version 1.0 of an open source project may not look anything like version 2.0. There are often going to be cases and examples where there are breaking changes; there’s stuff that they shouldn’t necessarily rely on in perpetuity, and that there’s some sort of flex that they should plan for in that kind of thing.
Those are all concepts that are kind of getting baked into the idea of what makes a good standard. There’s plenty of standards out there that nobody has ever even implemented — people got together and agreed how something should work and then never did anything with it. And that’s not the kind of standard we want to make or the kind of thing we want to promote.
Those sorts of things really foster a sense of trustworthiness in a standard — it gives you a sense that it’s something that’s going to stick around for a while, perhaps longer than an open source project, which may be sort of the beginnings of a standardization activity. It may be a reference to implementing a standard, or some folks just sort of throwing spaghetti at a wall and trying to solve a problem together. And I think these are activities that are very complementary with each other. It’s another great reason why other open source projects and organizations should be getting involved and supporting standardization activities.
JP: Do open standardization efforts make a case for open source software even stronger?
I think so — I just see them as so mutually beneficial, right? Because in the case of an open standards activity, you may be working with some folks and saying, well, here’s what I’m trying to express what this would look like — if we take the prose — and most of the time, the standard is written in prose and a pseudocode sort of style. It’s not something you can feed into the machine and have it work. So the open source projects, and polyfills, and things of that sort can really help a community of folks working on a problem say, “Aha, I understand what you mean!” “This is how we interpreted this, but it’s producing some unintended behaviors”, or “we see that this will be hard to test, or we see that this creates a security issue.”
It’s a way of putting your ideas down on paper, understanding them together, and having a tool through which everybody can pull and say, Okay, let’s, let’s play with it and see if this is really working for what we need it for.”
Yes, I think they’re very compatible.
JP: Like peanut butter and jelly.
JB: Peanut butter and jelly. Yeah.
JP: I get why large organizations might want things like programming languages, APIs, and communications protocols to be open standards, but what are the practical benefits that average citizens get from establishing open standards?
JB: Open standards really help promote innovation and market activity for all players regardless of size. Now, granted, for the most part, a lot of the activities we’ve been talking about are funded by some bigger players. You know, when you look at the member lists of some of the standards bodies, it’s larger companies like the IBMs, Googles, and Microsofts of the world, the companies that provide a good deal more of the funding. Still, hundreds of small and midsize businesses are also benefiting from standards development.
You mentioned my work at Bocoup earlier — that’s another great example. We were a consulting firm, who heavily benefited from participating in and leveraging open standards to help build tools and software for our customers. So it is a system that I think helps create an equitable market playing field for all the parties. It’s one of those actual examples of rising tides, which lift all boats if we’re doing it in a genuinely open and pro-competitive way. Now, sometimes, that’s not always the case. In other types of standardization areas, that’s not always true. But certainly, in our web platform standards, that’s been the case. And it means that other companies and other content authors can build web applications, websites, services, digital products, that kind of thing. Everybody benefits — whether those people are also Microsoft customers, Google customers, and all that. So it’s an ecosystem.
JP: I think it’s great that we’ve seen companies like Microsoft that used to have much more closed systems embrace open standards over the last ten years or so. If you look at the first Internet Explorer they ever had out — there once were websites that only worked on that browser. Today, the very idea of a website that only works on one company’s web browser correctly is ridiculous, right? We now have open source engines that these browsers use that embrace open standards have become much more standardized. So I think that open standards have helped some of these big companies that were more closed become more open. We even see it happen at companies like Apple. They use the Bluetooth protocol to connect to their audio hardware and have adopted technologies such as the USB-C connector when previously, they were using weird proprietary connectors before. So they, too, understand that open standards are a good thing. So that helps the consumer, right? I can go out and buy a wireless headset, and I know it’ll work because it uses the Bluetooth protocol. Could you imagine if we had nine different types of wireless networking instead of WiFi? You wouldn’t be able to walk into a store and buy something and know that it would work on your network. It would be nuts. Right?
JB: Absolutely. You’re pointing to hardware and the standards for physical products and goods versus digital products and goods in your example. So in using that example, do you want to have seven different adapters for something? No, it causes confusion and frustration in the marketplace. And the market winner is the one who’s going to be able to provide a solution that simplifies things.
That’s kind of the same thing with the web. We want to simplify the solutions for web developers so they’re not having to say, “Okay, what am I going to target? Am I going to target Edge? Am I going to target Safari?”
JP: Or is my web app going to work correctly in six years or even six months from now?
JP: Besides web standards, are there other types of standardization you are passionate about, either inside the LF or in your spare time?
JB: It’s interesting because I think in my career, I’ve followed this journey of first getting involved because it was intellectually interesting to me. Then it was about getting involved because it was about making my job easier. Like, how does this help me do business more effectively? How does this help me make my immediate life, life as a developer, and my life as an internet consumer a little bit nicer?
Beyond that, you start to think of the order of magnitude: our standardization activities’ social impact. I often think about the role that standards have played in improving the lives of everyday people. For the last 100 years, we have had building standards, fire standards, and safety standards, all of these things. And because they developed, adopted, and implemented in global policy, they have saved people’s lives.
Apply that to tech — of course, it makes sense that you would have safety standards to prevent the building from burning down — so what is the version of that for technology? What’s the fire safety standard for the web? And how do we actually think about the standards that we make, impacting people and protecting them the way that those other standards did?
One of the things that have changed in the last few years is that the Technical Advisory Group group or “TAG” at the W3C are considering more of the social impact questions in their work. TAG is a group of architects elected by the W3C membership to take a horizontal/global view of the technologies that the W3C standardizes. These folks say, “okay, great; you’re proposing that we standardize this API, have you considered it from an accessibility standpoint? Have you considered it from, you know, ease of use, security?” and that sort of thing.
In the last few years, they started looking at it from an ethical standpoint, such as, “what are the questions of privacy?” How might this technology be used for the benefit of the average person? And also, perhaps, how could it potentially be used for evil? And can we prevent that reality?
So one of the thingsI think is most exciting, is the types of technologies that are advancing today that are less about can we make X and Y interoperable, but can we make X and Y interoperable in a safe, ethical, economical, and ecological fashion — the space around NFT’s right now as a case in point. And can we make technology beneficial in a way that goes above and beyond “okay, great, we made the website, quick click here.”
So C2PA, I think, is an excellent example of a standardization activity that the LF supports could benefit people. One of the big issues of the last several years is the authenticity of media that we consume things from — whether it was altered, or synthesized in some fashion, such as what we see with deepfakes. Now, the C2PA is not going to be able to and would not say if a media file is fake. Rather, it would allow an organization to ensure that the media they capture or publish can be analyzed for tampering between steps in the edit process or the time an end user consumes it. This would allow organizations and people to have more trust in the media they consume.
JP: If there was one thing you could change about open source and open standards communities, what would it be?
JB: So my M.O. is to try and make these spaces more human interoperable. With an open source project or open standards project, we’re talking about some kind of technical interoperability problem that we want to solve. But it’s not usually the technical issues that cause delays or serious issues — nine times out of ten; it comes down to some human interoperability problem. Maybe it’s language differences, cultural differences, or expectations — it’s process-oriented. There’s some other thing that may cause that activity to fail to launch.
So if there were something that I could do to change communities, I would love to make sure that everybody has resources for running great and effective meetings. One big problem with some of these activities is that their meetings could be run more effectively and more humanely. I would want humane meetings for everyone.
JP: Humane meetings for everyone! I’m pretty sure you could be elected to public office on that platform. <laughs>. What else do you like to do with your spare time, if you have any?
JB: I love to read; we’ve got a book club at OpenJS that we’re doing, and that’s fun. So, in my spare time, I like to take time to read or do a crossword puzzle or something on paper! I’m so sorry, but I still prefer paper books, paper magazines, and paper newspapers.
JP: Somebody just told me recently that they liked the smell of paper when reading a real book.
JB: I think I think they’re right; I think it feels better. I think it has a distinctive smell, but there’s also something very therapeutic and analog about it because I like to disconnect from my digital devices. So you know, doing something soothing like that. I also enjoy painting outdoors and going outside, spending time with my four-year-old, and that kind of thing.
JP: I think we all need to disconnect from the tech sometimes. Jory, thanks for the talk; it’s been great having you here.
The post Interview with Jory Burson, Community Director, OpenJS Foundation on Open Source Standards appeared first on Linux Foundation.