Home Blog Page 474

Redirecting Network Traffic: Part 2

In the previous article, I looked at how to use the clever redir utility to listen out for inbound traffic on a particular port on a host and then forward that traffic onward somewhere else. Here, I’ll briefly describe some other approaches to manipulating traffic that may suit your needs.

IPTables Local

You can, of course, use the mighty IPtables (the kernel-based firewall, Netfilter) to alter how your traffic is manipulated as it arrives at your server. Let’s consider a local port redirection and then we can have a quick a look receiving traffic to a port on one server and dutifully forwarding it onwards to another IP address.

Here are two examples for locally redirecting.

# iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 25 -j REDIRECT --to-port 2500
# iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 443

Here we use the “PREROUTING” functionality on IPtables. The first command redirects incoming traffic for SMTP to port 2500 and the second command intercepts HTTP port traffic and forwards it onto the SSL/TLS port. The syntax isn’t too hard to follow, thankfully.

If you get lost, you can easily look up any NAT (Network Address Translation) rules with this command:

# iptables -nvL -t nat

Should you feel your blood pressure rising, then just flush the problematic rules away like this:

# iptables -F; iptables -t nat -F

Adding these “-F” commands to a Bash alias is sometimes a good idea so you can recover quickly.

IPtables Remote

What about palming off traffic to another machine by using IPtables, along the same lines that we saw previously with the redir utility?

Needless to say you should know what you’re doing (and experiment on a test machine before trying this in production). To start off, we need to enable forwarding on our local machine (“forwarding” essentially equals “routing” for all intents and purposes, allowing traffic to move between network interfaces on a local machine). We can achieve that with this command:

# sysctl net.ipv4.ip_forward=1

If you remove the “sysctl” part and add the remainder of that command (“net.ipv4.ip_forward=1”) to the foot of the file “/etc/sysctl.conf” then that new config will survive a reboot.

Next, we simply declare our rule. Let’s use TCP port 80 again as our example:

# iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination 10.10.10.123:80

Finally, we add this line to enable masquerading:

# iptables -t nat -A POSTROUTING -j MASQUERADE

As you would expect, the “-p” switch allows us to change the protocol setting from “tcp” to “udp” or “icmp”. IPtables apparently supports all of these protocols should you have the need to expand that list:

tcp, udp, udplite, icmp, esp, ah, sctp or  all

Internal Redirects

Of course, you don’t need to rely on tools that are, admittedly, relatively complex when other alternatives will suffice.

We’ve already looked at a common redirect (which is required fairly frequently in my experience), namely those of web-based services and TCP ports 80 and 443, so we will briefly look at how redirects are handled internally using the world’s most popular web server, Apache’s httpd.

Once tested a little, these rules are relatively intuitive. Here is an example of what a simple redirect would look like. You can see below that we send all inbound traffic to the HTTP port on to the HTTPS port:

RewriteCond %{HTTPS} !=on

RewriteRule ^(.*)$ https://www.chrisbinnie.tld/$1

In the above example, if the traffic that hits this rule isn’t already using HTTPS (encrypted with SSL or TLS in other words) then the condition will assume it is unencrypted HTTP traffic and continue to the next rule beneath it. The exclamation and equals sign (!=) mean not equal to.

You might, for example, want all traffic, except that which is being sent by a particular IP address, to go a new location. Note the slightly obscure exclamation mark before the IP Address “10.10.10.10” which acts as a negatory condition again, if met. You could add a whole subnet here easily, too.

RewriteCond %{REMOTE_ADDR} !10.10.10.10

RewriteRule .* http://www.chrisbinnie.tld/newer_page.html [L]

This picks up all the external traffic to your virtual host which Apache is dutifully listening out for traffic to. If you’re curious, the “[L]” flag at the end of the second line means that “mod_rewrite”, the Apache module responsible for performing the redirects, stops at that “last” command. There are a mountain of flags which the super-slick Apache can use to process its rules, for Apache 2.4 have a look here: http://httpd.apache.org/docs/2.4/rewrite/flags.html

So that “nginx” web server users don’t feel left out, let’s have a quick look at one of its examples, too. The mighty nginx has gained massive traction amongst the web server market, if you’re interested in one of the reasons this highly performant piece of software took such a large bite out of Apache’s market share then look up the “c10k” problem using your favourite online search device.

A simple nginx example of forwarding TCP port 80 traffic to an encrypted connection would look something like this:

if ($host = 'www.chrisbinnie.tld' ) {
            rewrite  ^/(.*)$  https://secure.chrisbinnie.tld/$1  permanent;
     }

That’s a welcome, short piece of config hopefully you agree and it also includes a look at how nginx can employ “if” statements, which is highly useful at times, and more familiar to programmers than Apache config might be.

Incidentally, you need to place that config inside your “server { }” tag. There are different options to this config; I’ve seen other syntax used in nginx, so if it doesn’t work then you might need to look online so that your version’s needs are met or other config isn’t breaking things. This following example is how you might alter the above to catch multiple domain names for instance:

server {
 listen 80;
 server_name chrisbinnie.tld www.chris.tld;
 rewrite ^ $scheme://www.chrisbinnie.tld$request_uri permanent;

...

}

Here we are simply grabbing what you might consider as malformed HTTP traffic (it’s not really malformed, users have just typed the wrong domain names and URLs into the address bar of their browsers), and we are then forwarding it onto “www.chrisbinnie.tld” so that our precious brand remains intact.

EOF

The next time an unexpected issue arises, you will now be armed with the excellent redir utility along with a smattering of IPtables rules to solve your headaches. For my purposes (short-lived port redirections), the redir utility is usually my tool of choice.

This is thanks to the fact that I tend to have IPtables running on my machines and obviously prefer to avoid breaking something which I know that works correctly (especially if the ports that I want to redirect already have holes punched through via existing IPtables rules). I really enjoy the simplicity of the excellent redir utility, too; that means there’s much less chance of typos.

We’ve only looked at a few scenarios with which traffic redirection can assist. There are many other circumstances when it could be used, especially during migrations of servers between datacenters or internal upgrades that require the renumbering of machines.

If you test that the syntax works locally first then you can rest assured that you can’t break too many production services. After all, you are pointing at what should be an already running service and simply sending some more traffic its way. Ultimately, the traffic will either be served or rejected depending on it its suitability for the daemon listening.

Learn more about essential sysadmin skills: Download the Future Proof Your SysAdmin Career ebook now.

Chris Binnie’s latest book, Linux Server Security: Hack and Defend, shows how hackers launch sophisticated attacks to compromise servers, steal data, and crack complex passwords, so you can learn how to defend against these attacks. In the book, he also talks you through making your servers invisible, performing penetration testing, and mitigating unwelcome attacks. You can find out more about DevSecOps and Linux security via his website (http://www.devsecops.cc).

Out-of-Band Management with Redfish and Ansible

In this article, I’ll explain how Redfish and Ansible can be used together to fully automate, at large scale, systems management tasks from one central location, significantly reducing complexity and helping improve the productivity of IT administrators.

Redfish is an open industry-standard specification published by the Distributed Management Task Force (DMTF) designed for modern and secure management of platform hardware. On Dell EMC PowerEdge servers, the Redfish management APIs are available via the integrated Dell Remote Access Controller (iDRAC), an out-of-band management controller used to remotely manage all hardware components on a server. 

If you’d like to learn more, please join Jose Delarosa for Open Source Summit Europe 2017 on October 23, 2017. In his presentation, Automated Out-of-Band Management with Ansible and Redfish session, he’ll share more details on using open source tools and open industry standards to achieve scalable, automated out-of-band systems management.

Read more at OpenSource.com

APIStrat Conference Workshops Cover API Integration, Security, Testing, and More

The API Strategy & Practice conference (APIStrat) – taking place Oct. 31 through Nov. 2 in Portland – features three days of technical sessions, keynotes, and more, including several workshops providing hands-on learning opportunities. These sessions cover topics such as RESTful API integration, OpenID Connect, API security, and REST API testing.

Check out the following workshops happening at APIStrat:

Connect Your RESTful API to Hundreds of Others in Minutes (Zapier and other Integration Platforms) – Sean Matthews, Left Hook Digital

In this workshop, the Left Hook team will show how to connect your app to hundreds of others on Zapier’s platform in a matter of minutes. We’ll walk you through a quick integration, and then talk about the pros and cons of 30+ different integration platforms out there, as well as highlighting platforms upon which developers can build out their own API connectors today.

Read more at The Linux Foundation

The Role of API Gateways in Microservice Architectures

Despite their differences in nomenclature, newly emerging service meshesaren’t all that different that API Gateways, and the similarities between the two will continue to grow over time, so predicts Marco Palladino, Chief Technology Officer of API Gateway provider Mashape.

The two technologies actually offer quite similar functionality, Palladino noted. API Gateways, such as Amazon Web Services‘ API Gateway or Mashape’s own open source Kong, have been primarily used over the last decade or so for mapping external traffic to internal resources, whereas the more recently developed service meshes — such as Lyft’s Envoy or Uber’s Catylist— have been primarily been on brokering internal resources in a microservices architecture.

“When you think of gateways, you usually think of a centralized layer, an extra hop in the network that is processing additional features. But that doesn’t necessarily have to be true,” Palladino said, speaking at MesosCon 2017, held last week in Los Angeles. 

Read more at The New Stack

Mastering File Searches on Linux

There are many ways to search for files on Linux systems and the commands can be very easy or very specific — narrowing down your search criteria to find what just you’re looking for and nothing else. In today’s post, we’re going to examine some of the most useful commands and options for your file searches. We’re going to look into:

  • Quick finds
  • More complex search criteria
  • Combining conditions
  • Reversing criteria
  • Simple vs. detailed responses
  • Looking for duplicate files

There are actually several useful commands for searching for files. 

Read more at Network World

Unix to GitHub: 10 Key Events in Free and Open Source Software History

It’s easy to take open source software for granted today, but free and open source software as we know it is the product of a long series of developments that stretch back a half-century. Here’s a look at some of the big moments in free and open source history — from the heyday of free Unix, to the birth of GNU and Linux, to the GitHub’s democratization of coding, and everything in between.

1969: Birth of Unix

In 1969 programmers at AT&T’s Bell Labs began work on Unix. Unix was never a free or open source operating system. It was born before the concept of free or open source software even existed. Until the early 1980s, source code was almost always available to anyone who wanted it. In this respect, Unix played little role in promoting the idea of sharing source code.

Read more at The VAR Guy

Apache “Optionsbleed” Vulnerability – What You Need to Know

Remember Heartbleed?

That was a weird sort of bug, based on a feature in OpenSSL called “heartbeat”, whereby a visitor to your server can send it a short message, such as HELLO, and then wait a bit for the same short message to come back, thus proving that the connection is still alive.

The Heartbleed vulnerability was that you could sneakily tell the server to reply with more data than you originally sent in, and instead of ignoring your malformed request, the server would send back your data…

…plus whatever was lying around nearby in memory, even if that was personal data such as browsing history from someone else’s web session, or private data such as encryption keys from the web server itself.

No need for authenticated sessions, remotely injected executable commands, guessed passwords, or any other sort of sneakily labyrinthine sequence of hacking steps.

Read more at Naked Security by Sophos

Clouds and Puppies at Open Source Summit: Day 3 in 5 Minutes

Yes, there were Puppies on Day 3 at the Open Source Summit, and they called it Puppy Pawlooza.  In this five-minute video summary, I’m joined by Jono Bacon, leading community strategist and curator of the Open Community Conference.

View on YouTube

The Cloud Native Computing Foundation (CNCF) kicked things off with a bunch of announcements Wednesday morning.  Aside from Oracle and Ticketmaster joining the foundation, both Lyft and Uber announced projects entering the CNCF.  Lyft’s project is Envoy, an edge and service proxy, and Uber’s is Jaeger, a distributed tracing system. 

The remainder of the day was filled with fascinating talks about building community, the role of certifications, and some astounding stats on the adoption of both Docker and orchestration tools like Kubernetes.  Oh, and, of course, Puppy Pawlooza, or should I say Open Source Cuddles?

For more daily summaries, you can also watch Day 1 and Day 2, and if you dig this content, check out my Open Source Craft channel on YouTube.

The 7 Stages of Becoming a Go Programmer

One day at work, we were discussing the Go programming language in our work chatroom. At one point, I commented on a co-worker’s slide, saying something along the lines of:

“I think that’s like stage three in the seven stages of becoming a Go programmer.”

Naturally, my co-workers wanted to know the rest of the stages, so I briefly outlined them. Here, expanded with more context, are the seven stages of becoming a Go programmer; see if you can see yourself on this pathway.

Stage 1: You believe you can make Go do object oriented programming

After your initial run on A Tour of Go, you start thinking “Now, how can I make this language behave more like an object oriented language…?” After all, you are used to that stuff. You want to make robust code. You want polymorphism.

Read more at OpenSource.com

Future Proof Your SysAdmin Career: Advancing with Open Source

For today’s system administrators, the future holds tremendous promise. In this ebook, we have covered many technical skills that can be big differentiators for sysadmins looking to advance their careers. But, increasingly, open source skillsets can also open new doors.

A decade ago, Red Hat CEO Jim Whitehurst predicted that open source tools and platforms would become pervasive in IT. Today, that prediction has come true, with profound implications for the employment market. Participating in open source projects — through developing code, submitting a bug report, or contributing to documentation — is an important way to demonstrate open source skills to hiring managers.

future proof ebook

“Successful open source projects thrive on a wide variety of contributions from people with all levels of coding skills and commitment. If just one person fixes a compiler warning, closes a bug, or adds to the documentation, pretty soon you’re talking real progress,” according to this New Relic article by Andy Lester.  

Additionally, market researchers have pointed to the connection between open source skillsets and improved employment outcomes. Knowledge of open source best practices, licensing requirements, and project management experience are all important skills that can be gained through working with open source projects. However, the collaboration and communication skills acquired through such participation are equally valuable.

Collaboration is key

Collaboration “is an increasingly important skill in today’s job environment because software is being built outside of a firm,” said Zemlin, Executive Director at The Linux Foundation in an article in PCWorld. “Someone who can collaborate within their company and across different organizations is highly sought after.”

Sysadmins should take note of how they can improve job prospects by contributing to open source projects. As open source technology becomes more pervasive, tech and DevOps workers are building out and overseeing their own open source projects. From Google, to Netflix to Facebook, companies are also releasing their open source creations to the community. Sysadmins who contribute to open source projects can showcase their fluency and experience in this space.

More information on tools to help you understand and contribute to open source projects can be found in this post. The bottom line is that open source is now part of the essential playbook for sysadmins, and seeking training and making contributions can greatly advance your prospects.

Conclusion

A key takeaway from this ebook is that complacency is the enemy. You may be a Linux wizard or a Microsoft-certified admin with years of experience, but staying competitive and advancing your career requires continuous improvement.

We’ve covered some of the skills that are highly valued in the job market now, but emerging skillsets for sysadmins will always be a moving target. As the landscape shifts for sysadmins, adding new skills and acquiring experience is essential.

Learn more about essential sysadmin skills: Download the Future Proof Your SysAdmin Career ebook now.

Read more:

Future Proof Your SysAdmin Career: An Introduction to Essential Skills 

Future Proof Your SysAdmin Career: New Networking Essentials

Future Proof Your SysAdmin Career: Locking Down Security

Future Proof Your SysAdmin Career: Looking to the Cloud

Future Proof Your SysAdmin Career: Configuration and Automation

Future Proof Your SysAdmin Career: Embracing DevOps

Future Proof Your SysAdmin Career: Getting Certified

Future Proof Your SysAdmin Career: Communication and Collaboration

Future Proof Your SysAdmin Career: Advancing with Open Source