Istio, the open source service mesh that helps provide traffic management, observability, and security to microservices and distributed applications, is taking another step forward this week, as Google announces that it will be coming to Google Kubernetes Engine (GKE) next month in the form of a one-click integration.
Offered initially in beta, the integration will help GKE users by layering a service mesh onto existing GKE clusters and providing telemetry, logging, load balancing, routing and even security in the form of mTLS. All of this, the company notes, is turned on “by simply checking the ‘Enable Istio’ box in the GKE management console.”
According to Google Cloud VP of Engineering Eyal Manor, Istio is the natural next step in the evolution of enterprises adopting containers and microservices.
“Kubernetes has taken the world by storm. Istio is the second phase, built on top of it. Once you’re in production, Istio helps you automate it in a secure and visible way,” said Manor. “Istio is that bridge between the current state of the enterprise and the future state of the enterprise, which is cloud native.”
For a while now I’ve been working with a C++ project (Envoy), and sometimes I need to contribute to it, so my C++ skills have gone from “nonexistent” to “really minimal”. I’ve learned what an initializer list is and that a method starting with ~ is a destructor. I almost know what an lvalue and an rvalue are but not quite.
But the other day when writing some C++ code I figured out something exciting about how to use destructors that I hadn’t realized! (the tl;dr of this post for people who know C++ is “julia finally understands what RAII is and that it is useful” :))
what’s a destructor?
C++ has objects. When an C++ object goes out of scope, the compiler inserts a call to its destructor.
So if you have some code like
function do_thing() {
Thing x{}; // this calls the Thing constructor
return 2;
}
there will be a call to x’s destructor at the end of the do_thing function. so the code c++ generates looks something like:
As the use of open-source software continues to grow for businesses of all sizes, a shortage of qualified IT administrators and other staff members who are trained, experienced and certified in open-source technologies remains a serious challenge for IT departments and the channel.
Maja Kraljič, a web developer who works for the Association for Progressive Communications, lives in Berlin, but is from Slovenia, where it can be difficult to find exam sites for IT certifications depending on the location.
“While I was doing my LFCS and LFCE certifications I was living in Slovenia and there were no testing centers available in the city I lived in,” she told Channel Futures. “I wanted to gain more knowledge about Linux – as I was using Ubuntu – and wanted to know more how it works. I also wanted to teach others about Linux on workshops and having a certification from Linux Foundation for sure looks good on the CV.”
A real-world use case of eBPF tracing to understand file access patterns in the Linux kernel and optimize large applications.
By Gabriel Krisman Bertazi, Software Engineer at Collabora.
When Brendan Gregg gave his Performance Analysis superpowers with Linux BPF talk during the Open Source Summit in Los Angeles last year, he wasn’t messing around. Using the new eBPF tracing tools really feels like you gained some x-ray vision powers since, suddenly, opening the program’s hood is no longer necessary to see in details how it is behaving internally.
I had the chance of applying these new gained powers last month, when I was asked to develop a small tool to trace when and what parts of each file was being accessed for the first-time during a program initialization. This request came with a few constraints, like the information had to be available independently of the kind of buffered I/O method being used (synchronous, aio, memory-mapping). It also should be trivial to correlate the block data with which files were being accessed and, most importantly, the tracing code should not result in a large performance impact in the observed system.
What is the right level to install our probe?
I started by investigating where exactly we’d want to place our probe:
Tracing at the block layer level
A tracer at the Block Layer would examine requests in terms of disk blocks, which don’t directly correlate to filesystem files. A trace at this level gives insight on which areas of the disk are being read or write, and whether they are organized physically in a contiguous fashion. But it doesn’t give you a higher level view of the system in a file basis. Other tools already exist to trace block-level accesses, such as the EBF script biosnoop and the traditional blktrace.
Tracing at the filesystem level
A tracer at the filesystem level exposes data in terms of file blocks, which can resolve to one or more blocks of data in the disk. In an example scenario, an Ext4 filesystem with 4Kib file block sizes, one system page likely corresponds to 1 physical block (in 4K disks) or 4 physical blocks in disks with 512 sector size. The tracing at the filesystem level allows us to look at the file in terms of offsets, such that we ignore disk fragmentation. Different installations might fragment the disk differently, and from an application level perspective, we shouldn’t really be interested in the disk layout as much as what file blocks of data we need, in case we want to optimize by prefetching them.
Tracing at the Page Cache level
The page cache is a structure that sits in between the VFS/memory-mapping system and the filesystem layer, and is responsible for managing memory sections already read from the disk. By tracing the inclusion and removal pages in this cache we could gain the “First-access” behavior for free. When a new page is first accessed it is brought to the cache, and further accesses won’t need to go to the disk. If the page is eventually dropped from the cache because it is no longer needed, a new use will need to reach the disk, and a new access entry will be logged. In our Performance investigation scenario, was the exactly functionality we were looking for.
The probe
The probe we implemented traces Page Cache Misses inside the Kernel Page Cache handling functions to identify the first time a block is requested, before the request is even submitted to the disk. Further requests to the same memory area (as long as the data is still mapped) will return a hit in the cache, which we don’t care about, nor trace. This prevents our code from interfering with further accesses, severely diminishing the impact on performance our probe could have.
By tracing the page cache, we are also capable of differentiating blocks that were directly requested by the user program from blocks requested by the Read Ahead logic inside the kernel. Knowing which blocks were read ahead is very interesting information for application developers and system administrators, since it allows them to tune their systems or applications to prefetch the blocks they want in a sane manner.
Who wrote this code? Why? What changes led to this function’s current implementation?
These are typical questions that developers (and sometimes lawyers) ask during their work. Most software development projects use version control software (such as Git or Subversion) to track changes and use the “blame” feature of these systems to answer these questions.
Unfortunately, version control systems are only capable of tracking full lines of code. Imagine the following scenario: A simple file is created by developer A; later, it is changed by Developer B, and finally, by Developer C. The following figure depicts the contents of the files after each modification. The source code has been colored according to the developer who introduced it (blue for Developer A, green for Developer B, and red for Developer C; note that Developer B only changed whitespace –including merging some lines).
Blame tracks lines not tokens
If we were to use git to track these changes, and use git-blame with default parameters, its output will show that Developer B and C are mostly responsible for the contents of the file. However, if we were to instruct blame to ignore changes to whitespace, the results would be:
In general, one would expect to always ask blame to ignore whitespace. Unfortunately, this is not always possible (such as the “blame” view of GitHub, which is computed without ignoring whitespace).
Note that, even if we run blame with the ignore-whitespace option, the “blame” is incorrect. First, lines merged or split are not addressed properly by blame (the ignore-whitespace option does not ignore them). Second, lines that were mostly authored by Developer A are now assigned to Developer C because she was the last one to modify them.
If we consider the token as the indivisible unit of source code (i.e., a token cannot be modified, it can only be removed or inserted), then what we really want is to know who is responsible for introducing each token to the source code base. A blame-per-token for the file in our example would look like the figure below. Note how it correctly shows that the only changes made by C to the source code were the replacement of int with long in three places, and that B made no changes to the code:
cregit: improving blame of source code
We created cregit to do exactly this. The goal of cregit is to provide token-level blame for a software system whose history has been recorded using git. The details of cregit’s implementation can be found in this Working Paper (currently under review).
We have empirically evaluated cregit on several mature software systems. In our experiments, we found that blame-per-line tends to be accurate between 70% and 80% of the time. This highly depends on how much the code has been modified. The more modifications to existing code, the less likely that blame-per-line will be accurate. Cregit on the other hand is able to increase this accuracy to 95% (please see the paper mentioned above for details).
Blame-per-line is easy to implement, just put the blame information to the side; however, blame-per-token is significantly more complex, as its tokens might have different authors and/or commits responsible for them. Hence, we are currently rolling out an improved view of blame-per-token for kernel release 4.19 of Linux (older versions use an old view, and most of the information here does not apply).
cregit views: inspecting who changed what/when
Below is an example of the blame-per-token views of Linux 4.19, specifically for the file audit.c.html.
The top part gives us an overview of who the authors of the file are. The first 50 authors are individually colored. The source code is colored according to the person who last added the token. The right-hand side of the view shows an overview of the “ownership” of the source code.
While hovering over the source code, you will see a box displaying information about how that token got into the source code: the commit id, its author, and its commit timestamp and summary. If you click on the token, this information is enhanced with a link to the email thread that corresponds to the code review of the commit that inserted that token, as shown below:
The views are highly interactive. For example, one can select to highlight a commit (top middle combo box). In this case, all the code is grayed out, except for the tokens that were added by that commit, as shown below.
You can also click on an author’s name, and only that author’s code will be highlighted. For example, in the image below I have highlighted Eric Paris’s contributions.
cregit is also capable of highlighting the age of the code. The sliding bar at the top right allows to narrow the period of interest. Below I have selected to show changes during the last two years (note that the file was last modified in July 17, 2018.
It is also possible to focus on a specific function, which can be selected with theFunctions combo box at the top of the source code. In the example below I have selected the function audit_set_failure. The rest of the code has been hidden.
These features can be easily combined. You can select the age of the code by a specific author. And narrow it to a given function!
cregit views: improving the linkage of email code reviews
We are going to keep expanding the information shown in the commit panel. Currently, in addition to the metadata of the commit that is responsible for the token, it provides hyperlinks to the commit patch, and to any email discussions we have been able to find regarding this commit. We are working to match more and more commits.
cregit: where to get it
cregit is open source, and is accessible from https://github.com/cregit/cregit. It is capable of processing C, C++, Java, and go. We can probably add support for perl and python fairly easily. All we need to support a new language is a tokenizer.
cregit’s input is a git repository, and its output is another git repository that tracks the source code by token (see paper for details). From this repository we construct the blame views shown above. If you are interested to have your repository processed with cregit, email me.
Finally, I would like to acknowledge several people for their contributions:
Bram Adams. Bram and I are the creators of cregit.
Jason Lim. As part of his coursework at UVic he implemented the new cregit views, which have greatly improved their usefulness.
Alex Courouble. As part of his master’s at the Poly of Montreal he implemented the matching algorithms of commits to email discussions, based on earlier work of Yujuan Jiang during her PhD.
Kate Stewart. She has been instrumental to gather user requirements and to evaluate cregit and its views.
Isabella Ferreira. She is picking up where Alex left and continues to improve the matching of commits to emails.
This article was written by Daniel German (dmg@turingmachine.org) and originally appeared on GitHub.
If you’re looking to secure your Ubuntu Server, one of the first things you should do is install the fail2ban intrusion detection system. What fail2ban does is monitor specific log files (in /var/log) for failed login attempts or automated attacks on your server. When an attempted compromise is discovered from an IP address, fail2ban then blocks the IP address (by adding a new chain to iptables) from gaining entry (or attempting to further attack) the server.
Believe it or not, fail2ban is so easy to install and use, it should be considered a no-brainer for all Linux servers.
I want to walk you through the process of installing fail2ban on Ubuntu Server 18.04. I’ll then show you how to add a jail to monitor for failed SSH login attempts.
Major slowdowns caused by the new Linux 4.20 kernel have been traced to a mitigation for Spectre variant 2 that Linux founder Linus Torvalds now wants restricted.
As noted by Linux news site Phoronix, the sudden slowdowns have been caused by a newly implemented mitigation called Single Thread Indirect Branch Predictors (STIBP), which is on by default in the Linux 4.20 kernel for Intel systems with up-to-date microcode.
STIBP is one of three possible mitigations Intel added to its firmware updates in response to the Spectre v2 attacks. Others included Indirect Branch Restricted Speculation (IBRS), and Indirect Branch Predictor Barrier (IBPB), which could be enabled by operating-system makers.
And, you’ll get a free T-shirt with every purchase!
Through the limited-time Cyber Monday training sale, we’re offering prep course and certification exam bundles for just $179.
This offer includes the prep course and exam for the following certification options:
Linux Foundation Certified SysAdmin (LFCS) — This training is ideal for candidates looking to validate their Linux system administration skill set.
Linux Foundation Certified Engineer (LFCE) — This option is designed for the Linux engineer looking to demonstrate a more advanced level of Linux administration and engineering skill.
Cloud Foundry Certified Developer (CFCD) — This program will verify your expertise in using the Cloud Foundry platform and building cloud-native applications
Certified Kubernetes Administrator (CKA) — This program assures you have the skills and knowledge to perform the responsibilities of Kubernetes administrator.
Certified Kubernetes Application Developer (CKAD) — This option certifies that you can design, build, configure, and expose cloud native applications for Kubernetes.
Certified OpenStack Administrator (COA) — This program provides essential OpenStack and cloud infrastructure skills.
Change frequency is an indicator of time to create business value. In order to create value in a given amount of time, you need to be able to release your code a certain number of times and learn from those changes. The less frequently you release, the longer it can take to create value. Increase in rate of change shows that you’re reducing the time to create value, thus increasing team performance. Conversely, low change frequency indicates high time to create value and low team performance.
Those that develop and deliver quickly are better able to experiment with ways to increase customer adoption and satisfaction, pivot when necessary, and keep up with compliance and regulatory demands.
However, change frequency alone is not a sufficient measure of team performance. As the same State of DevOps report aptly captures, production stability is an equally important measure of team performance. What good is high change frequency if the production environment is falling apart often for long periods of time?
What to expect when you’re working with microservices and containers.
First of all, what are microservices? Microservices is a type of architecture that splits your application into multiple services that does a fine-grained function that’s a part of your application as a whole. Each of your microservices will have a different logical function for your application. Microservices is a more modern approach in an application’s architecture compared to a monolithic architecture where all your application’s components and functions are in a single instance. You can refer to a comparison of a monolithic to a microservices architecture on the diagram below.
