Open Source and the Artificial Intelligence Frontier


The open source arena continues to rapidly converge with the field of artificial intelligence (AI). Not only are technology industry titans contributing meaningful tools to the community, but international players and billionaires are making contributions. Meanwhile, some of our smartest people are also laser-focused on keeping AI development open and safe.

Interest from China in AI has been ramping up quickly, and Chinese social media and gaming titan Tencent Holdings has just announced that it is opening an AI research center in Seattle, to be headed up by former Microsoft scientist Yu Dong. Reuters cites Tencent, which owns the WeChat messaging app, as Asia’s most valuable company with a market capitalization of almost $300 billion. The center’s leader, Yu Dong, was a speech recognition expert at Microsoft.

According to MIT Technology Review: “This may be the year in which China starts looking like a major player in the field of AI. The country’s tech industry is shifting away from copying Western companies, and it has identified AI and machine learning as the next big areas of innovation.”

Meanwhile, Elon Musk has stepped in with the formation of OpenAI, an organization that is focusing on anticipating the challenges that AI will present in the coming years, and keeping AI technology development open. It is a nonprofit organization with the dual mission of ensuring that AI stays safe, and its benefits are as broadly and equitably distributed as possible. Microsoft recently partnered with OpenAI to work on open tools and “democratize AI,” according to the company. You can follow the OpenAI Twitter feed to see just how rapidly the organization’s tools are being integrated with everything from open robotics tools to anti-spam filters.

Musk isn’t the only billionaire to be pointing to the promise of AI. In a recent series of tweets aimed at college graduates, Bill Gates said “it’s what I would do if starting out today.

Tech Giants Continue to Deliver Free, Open Tools

At leading companies in the technology sector, AI is a huge priority, and many of these companies are open sourcing important tools. Both Facebook CEO Mark Zuckerberg and Google CEO Sundar Pichai have been vocal about their contributions of open source artificial intelligence and machine learning tools.

In a Google Founders’ Letter to stockholders, Pichai said, “[Artificial Intelligence] can help us in everything from accomplishing our daily tasks and travels to eventually tackling even bigger challenges like climate change and cancer diagnosis.”

“The biggest thing that we’re focused on with artificial intelligence is building computer services that have better perception than people,” said Zuckerberg, on a conference call. “I think it’s possible to get to that point in the next five to 10 years.”

Just this month, Facebook discussed new, open technology that could revolutionize machine translation, resulting in translations that are far more accurate than what is available now. In addition, Facebook has open sourced its central machine learning system designed for artificial intelligence tasks at large scale. It’s a proven platform in use at Facebook. And, at the company’s recent F8 developer conference, Facebook also open sourced a new framework for deep learning and AI called Caffe2.

Google has open sourced a software framework called TensorFlow that it has spent years developing to support its AI software and other predictive and analytics programs. You can find out more about TensorFlow at its site, and it is the engine behind several Google tools you may already use, including Google Photos and the speech recognition found in the Google app.

Meanwhile, Yahoo has released its key artificial intelligence software under an open source license. Its CaffeOnSpark tool is based on deep learning, a branch of artificial intelligence particularly useful in helping machines recognize human speech, or the contents of a photo or video. And, IBM has announced that its proprietary machine learning program known as SystemML is freely available to share and modify through the Apache Software Foundation.

The Raspberry Pi community is waking up to the promise of AI as well. Some Raspberry Pi owners are building their own AI-driven voice assistants, and Google has open sourced an AI-driven “Voice Kit” for Pi devices.

Previously, I covered, formerly known as Oxdata, which has carved out a unique niche in the machine learning and artificial intelligence arena because its primary tools are free and open source.  You can get the main H2O platform and Sparkling Water, a package that works with Apache Spark, by simply downloading them.

These tools operate under the Apache 2.0 license, one of the most flexible open source licenses available, and you can even run them on clusters powered by Amazon Web Services (AWS) and others for just a few hundred dollars. Never before has this kind of data sifting power been so affordable and easy to deploy.’s Vinod Iyengar oversees product strategy at the company. In an interview, he said: “In the last five years the cost of storage has come down dramatically, as has the cost of memory,” he said. “Additionally, anyone can leverage an advanced computing cluster on, say, Amazon Web services, for a few hundred dollars. All of this means that organizations or individuals can take a whole lot of data and produce powerful predictions and insights from the large data sets without facing huge costs.”

To learn more about the convergence of open source with machine learning and artificial intelligence, watch a video featuring David Meyer, Chairman of the Board at OpenDaylight, a Collaborative Project at The Linux Foundation.

Are you interested in how organizations are bootstrapping their own open source programs internally? You can learn more in the Fundamentals of Professional Open Source Management training course from The Linux Foundation. Download a sample chapter now!