Good code is cheap; it’s operational knowledge that’s holding back big data from solving the great problems of our time.
Solving those operational difficulties with a modular, easy-to-use system was the solution Mark Shuttleworth laid out in his keynote entitled “More Fun, Less Friction” at Apache Big Data in Vancouver in May.
“If we take the friction out, we can unleash all sorts of creativity,” Shuttleworth said.
Shuttleworth is the founder of Canonical and is known for his work on the Linux distribution Ubuntu, which was created for the purpose of making the open operating system as easy as possible to use.
Now Canonical is working on a system called Juju, which models applications and services together to be deployed on any cloud database system. Those models are called charms, and companies from IBM to Cloudera to Couchbase to any number of Apache projects are creating them to run on Canonical’s system.
During his keynote, Shuttleworth spent the first half of his talk — about 8 minutes — spinning up a containerized Linux server on a laptop, an Amazon Web Services server, and a physical server rack using Juju, demonstrating how each environment handled the application cluster with the exact same commands.
“We’re using a common modeling language to describe these different applications, and then essentially binding all of those to the underlying resources needed in each of those environments,” Shuttleworth said. “Why is this interesting? It’s interesting because free software is becoming expensive… Scarcity, what’s rare, is not the code anymore. What’s scarce is the knowledge on how to operate that code.”
By reducing the complexity of spinning up servers to run dozens of combinations of different types of big data technology, and then getting those systems to play nice with each other, Shuttleworth and Canonical are trying to enable people working on big data to focus effort on making money, doing groundbreaking research into things like AI, or just making cool stuff.
“There are 33 big data projects in Apache. How many people know how to operate all of those?” Shuttleworth said. “And the complexity of those architectures is growing. It’s not two apps across two machines, it’s many apps across many machines. And architecture today is really a discussion about how we map those things together.
“We call this a phase change … because all of these pieces of software … they all have that same line crossing — the software is becoming free, but the operation is becoming expensive.”
Reducing complexity means reducing expense, and that means increased adoption and — hopefully — increased results.
Watch the complete presentation below:
As an applications developer, you are a problem solver. You design, implement and support next-generation applications that are utilized to meet company needs. You offer solutions to drive overall business performance and success. It should come as no surprise then that you are a hiring priority, as companies are in the market for professionals who can help them solve real-world problems.
While in popular demand, it doesn’t mean you can’t ignore what’s trending in tech. As companies look to be more “open,” they need applications developers who are not only familiar with emerging open source technologies, but who are experts in writing code and finding valuable insights through data mining.
With the bar set high in terms of what employers expect, it is important to educate yourself on what specific skillsets companies are looking for from an applications developer. This not only will allow you to grow and move up the ranks within your current company, but it will make you more marketable when looking to change employers.
Below are just a few of the most sought after skills employers are requesting on Dice when in the market for an open source applications developer.*
Big Data: Developing and running applications requires using large amounts of data. For this reason, companies need applications developers who have experience working with Big Data technologies, like Hadoop or Apache Spark, which work to help collect, process and analyze huge data sets. For those less acquainted with Big Data, it may be wise to get yourself up to speed on this skillset. Professionals with Big Data expertise earn $121,328 on average in the United States and are amongst the highest paid professionals, according to Dice’s latest annual salary survey.
Cloud: With the rise of cloud computing, you are seeing more and more companies looking for open source professionals with cloud expertise. In the 2016 Open Source Jobs Report, 51 percent of hiring managers surveyed said knowledge of cloud technologies has the biggest impact on their open source hiring decisions. OpenStack, in particular, is one cloud-related skill that is gaining momentum, with e-commerce and security companies alike looking for open source professionals with familiarity with this cloud-based operating system.
Mobile: For most companies today, understanding mobile isn’t so much an option as it is a requirement. Employers on Dice are looking for applications developers with a strong working knowledge of mobile coupled with UI/UX experience, solid programming background (i.e. Java and C/C++) and project management skills.
JavaScript: JavaScript is a core programming language that employers want in an applications developer. On any given day, there are over 13,000 JavaScript job postings on Dice, which represents approximately 15 percent of Dice’s overall job count. Expertise in JavaScript is also needed when working with open-source frameworks like AngularJS.
While these skillsets are a good jumping off point if you are looking to further develop yourself as an open source applications developer, there is always more you can learn. According to the 2016 Open Source Jobs Report, free online tutorials are the most common method used by open source professionals to keep their skills up-to-date, but there is also other formal training you can rely upon to gain additional skills expertise, such as meet-ups or instructor-led courses.
In the end, whichever method(s) you choose to brush up on or expand your skills base doesn’t really matter. What matters is that you are continuously learning and keeping current on what’s trending in tech. As a problem solver, you need to have a keen familiarity with the latest platforms and skills in order to offer up the best solutions.
* Applications Development is a broad job category that encompasses a wide range of job titles including but not limited to “applications developer,” “software development engineer,” “software developer,” “mobile developer” and “big data engineer.”
Yuri Bykov manages Data Science at Dice.
