U.S. consumers are expected to drop a bundle this Black Friday on smart speakers and home hubs. A Nov. 15 Canalys report estimates that shipments of voice-assisted speakers grew 137 percent in Q3 2018 year-to-year and are on the way to 75 million-unit sales in 2018. At the recent Embedded Linux Conference and Open IoT Summit in Edinburgh, embedded Linux developer and Raspberry Pi HAT creator Leon Anavi of the Konsulko Group reported on the latest smart speaker trends.

As Anavi noted in his “Comparison of Voice Assistant SDKs for Embedded Linux Devices” talk, conversing with computers became a staple of science fiction over half a century ago. Voice technology is interesting “because it combines AI, big data, IoT, and application development,” said Anavi.

In Q3 2017, Amazon and Google owned the industry with 74.7 percent and 24.6 percent, respectively, said Canalys. A year later, the percentages were down to 31.9 and 29.8. China-based Alibaba and Xiaomi almost equally split another 21.8 percent share, followed by 17.4 percent for “others,” which mostly use Amazon Alexis, and increasingly, Google Assistant.

Despite the success of the mostly Linux-driven smart speaker market, Linux application developers have not jumped into voice app development in the numbers one might expect. In part, this is due to reservations about Google and Amazon privacy safeguards, as well as the proprietary nature of the hardware and cloud software.

“Privacy is a concern with smart speakers,” said Anavi. “You can’t fully trust a corporation if the product is not open source.”

Anavi summarized the Google and Amazon SDKs but spent more time on the fully open source Mycroft Mark. Although Anavi clearly prefers Mycroft, he encouraged developers to investigate all the platforms. “There is a huge demand in the market for these devices and a lot of opportunity for IoT integration, from writing new skills to integrating voice assistants in consumer electronics devices,” said Anavi.

Alexa/Echo

Amazon’s Alexa debuted in the Echo smart speaker four years ago. Amazon has since expanded to the Echo branded Dot, Spot, Tap, and Plus speakers, as well as the Echo Show and new Echo Show 2 display hubs.

The market leading Echo devices run on Amazon’s Linux- and Android-based Fire OS. The original Echo and Dot ran on the Cortex-A8-based TI DM3725 SoC while more recent devices have moved to an Armv8 MediaTek MT8163V SoC with 256MB RAM and 4GB flash.

Thanks to Amazon’s wise decision to release an Apache 2.0 licensed Alexa Voice Services (AVS) SDK, Alexa also runs on most third-party hubs. The SDK includes an Alexa Skills Kit for creating custom Skills. The cloud platform required to make Alexa devices work is not open source, however, and commercial vendors must sign an agreement and undergo a certification process.

Alexa runs on a variety of hardware including the Raspberry Pi, as well as smart devices ranging from the Ecobee4 Smart Thermostat to the LG Hub Robot. Microsoft recently began selling Echo devices, and earlier this year partnered with Amazon to integrate Alexa with its own Cortana voice agent in devices. This week, Microsoft announced that users can voice-activate Skype calls via Alexa on Echo devices.

Google Assistant/Home

The Google Assistant voice agent debuted on the Google Home smart speaker in 2016. It has since expanded to the Echo Dot-like Home Mini, which like the Home runs on a 1.2GHz dual-core Cortex-A7 Marvell Armada 1500 Mini Plus with 512MB RAM and 4GB flash. This year’s Home Max offered improved speakers and advanced to a 1.5GHz, quad-core Cortex-A53 processor. More recently, Google launched the touchscreen enabled Google Home Hub.

The Google Home devices run on a version of the Linux-based Google Cast OS. Like Alexa, the Python driven Google Assistant SDK lets you add the voice agent to third-party devices. However, it’s still in preview stage and lacks an open source license. Developers can create applications with Google Actions.

Last year, Google launched a version of its Google Assistant SDK for the Raspberry Pi 3 and began selling an AIY Voice Kit that runs on the Pi. There’s also a kit that runs on the Orange Pi, said Anavi.

This year, Google has aggressively courted hardware partners to produce home hub devices that combine Assistant with Google’s proprietary Android Things. The devices run on a variety of Arm-based SoCs led by the Qualcomm SD212 Home Hub Platform.

The SDK expansion has resulted in a variety of third-party devices running Assistant, including the Lenovo Smart Display and the just released LG XBOOM AI ThinQ WK9 touchscreen hubs. Sales of Google Home devices outpaced Echo earlier this year, although Amazon regained the lead in Q3, says Canalys.

Like Alexa, but unlike Mycroft, Google Assistant offers multilingual support. The latest version supports follow-up questions without having to repeat the activation word, and there’s a voice match feature that can recognize up to six users. A new Google Duplex feature accomplishes real-world tasks through natural phone conversations.

Mycroft/Mark

Anavi’s favorite smart speaker is the Linux-driven, open source (Apache 2.0 and CERN) Mycroft. The Raspberry Pi based Mycroft Mark 1 speaker was certified by the Open Source Hardware Association (OSHA).

The Mycroft Mark II launched on Kickstarter in January and has received $450,000 in funding. This Xilinx Zynq UltraScale+ MPSoC driven home hub integrates Aaware’s far-field Sound Capture technology. A Nov. 15 update post revealed that the Mark II will miss its December ship date.

Kansas City-based Mycroft has raised $2.5 million from institutional investors and is now seeking funding on StartEngine. Mycroft sees itself as a software company and is encouraging other companies to build the Mycroft Core platform and Mycroft AI voice agent into products. The company offers an enterprise server license to corporate customers for $1,500 a month, and there’s a free, Raspbian based Picroft application for the Raspberry Pi. A Picroft hardware kit is under consideration.

Mycroft promises that user data will never be saved without an opt-in (to improve machine learning algorithms), and that it will never be used for marketing purposes. Like Alexa and Assistant, however, it’s not available offline without a cloud service, a feature that would better ensure privacy. Anavi says the company is working on an offline option.

The Mycroft AI agent is enabled via a Python based Mycroft Pulse SDK, and a Mycroft Skills Manager is available for Skills development. Like Alexa and Assistant, Mycroft supports custom wake words. The new version uses its homegrown Precise wake-word listener technology in place of the earlier PocketSphinx. There’s also an optional device and account management stack called Mycroft Home.

For text-to-speech (TTS), Mycroft defaults to the open source Mimic, which is co-developed with VocaliD. It also supports eSpeak, MaryTTS, Google TTS, and FATTS.

Mycroft lacks its own speech to-text (STT) engine, which Anavi calls “the biggest challenge for an open source voice assistant.” Instead, it defaults to Google STT and supports IBM Watson STT and wit.ai.

Mycroft is collaborating with Mozilla on its open source DeepSpeech STT, an open source TensorFlow implementation of Baidu’s DeepSpeech platform. Baidu trails Alibaba and Xiaomi in the Chinese voice assistant market but is one of the fastest growing voice AI companies. Just as Alibaba uses its homegrown, Alexa-like AliGenie agent on its Tmall Genie speaker, Baidu loads its speakers with its DeepSpeech-driven DuerOS voice platform. Xiaomi has used Alexa and Cortana.

Mycroft is the most mature of several alternative voice AI projects that promise improved privacy safeguards. A recent VentureBeat article reported on emerging privacy-oriented technologies including Snips and SoundHound.

Anavi concluded with some demo videos showing off his soothing, Bulgarian AI whisperer vocal style. “I try to be polite with these things,” said Anavi. “Someday they may rule the world and I want to survive.”

Anavi’s video presentation can be seen here:

A number of home automation platforms support Python as an extension, but if you’re a real Python fiend, you’ll probably want Home Assistant, which places the programming language front and center. Paulus Schoutsen created Home Assistant in 2013 “as a simple script to turn on the lights when the sun was setting,” as he told attendees of his 2016 Embedded Linux Conference and Open IoT conference presentation. (You can watch the complete video below.)

Schoutsen, who works as a senior software engineer for AppFolio in San Diego, has attracted 20 active contributors to the project. Home Assistant is now fairly mature, with updates every two weeks and support for more than 240 different smart devices and services. The open source (MIT license) software runs on anything that can run Python 3, from desktop PCs to a Raspberry Pi, and counts thousands of users around the world.

Like most automation systems, Home Assistant offers mobile and desktop browser clients to control smart home devices from afar. It differs from most commercial offerings, however, in that it has no hub appliance, which means there are no built-in radios. You can add the precisely those radios you want, however, using USB sticks. There’s also no cloud component, but Schoutsen argues that any functionality you might sacrifice because of this is more than matched by better security, privacy, and resiliency.

“There is no dependency on a cloud provider,” said Schoutsen. “Even when the Internet goes down, the home doesn’t shut down, and your very private data stays in your home.”

Schoutsen did not offer much of a promo in his presentation, but quickly set to work explaining how the platform works. Since Home Assistant is not radically different from other IoT frameworks — one reason why it interfaces easily with platforms ranging from Nest to Arduino to Kodi — the presentation is a useful introduction to IoT concepts.

To get a better sense of Home Assistant’s strengths, I recently asked Schoutsen for his elevator pitch. He highlighted the free, open source nature of the software, as well as the privacy and security of a local solution. He also noted the ease of setup and discovery, and the strength of the underlying Python language.

Easy Extensions

“Python makes it very easy to extend the system,” Schoutsen told me. “As a dynamic language it allows a flexibility that Java developers can only dream off. It is very easy to test out and prototype new pieces on an existing installation without breaking things permanently. With the recent introduction of MicroPython, which runs on embedded systems as Arduino and ESP8266, we can offer a single language for all levels of IoT: from sensors to automation to integration with third-party services.”

In Schoutsen’s ELC 2016 presentation, he described how Home Assistant is an event-driven program that incorporates a state machine that keeps track of “entities” — all the selected devices and people you want to track. Each entity has an identifier, a state condition, and attributes. The latter describes more about the state, such as the color and intensity of the light on a Philips Hue smart bulb.

To integrate a Philips Hue into the system, for example, you would need to use a light “component,” which is aware of the bulb and how to read its state (off or on). Home Assistant offers components for every supported device or service, as well as easy access to component groups such as lights, thermostats, switches, and garage doors. Setup is eased with a network discovery component that scans the network and, if you have a supported device, sets it up automatically. 

The software is further equipped with a service registry, which provides services over the event bus. “We can register the turn-on command for a light, and have it send an email or SMS,” said Schoutsen. “A timer can send a time change event every second, and a component can ask to be notified at a particular time, or in intervals. Based on time change events, it will trigger the callback of the components.”

Each component writes its state to the state machine, emitting a state change event to the event bus. “The light component would register its turn on service inside the service registry so that anyone could fire an event to the event bus to turn on the light,” said Schoutsen.

You can easily integrate a light component with a motion detector component using an automation component. This would listen to the motion detector events, and fire a “turn light on” event to the event bus, which in turn would be forwarded to the service registry. The registry would then check to see that the light component can handle the event. “Automation components can listen for events, observe certain attribute states or triggers, and act on them,” explained Schoutsen.

Another component type handles presence detection. “The platform can check the router to see which phones are connected in order to see who is home,” said Schoutsen. “Other components are responsible for recording event and state history, or for entity organization — grouping multiple entities and summarizing their state.” Components are available for integrating third party services, such as MQTT or IFTTT, and other components export data to external databases and analysis tools.

Schoutsen went on to explain concepts such as a “platform” layer that sits above the entity components. Each platform integrates an “abstract base class,” which “acts as the glue between the real device and the one represented in Home Assistant,” said Schoutsen. Later, he ran through a code example for a basic switch and explored the use of trigger zones for geofencing.

As Schoutsen says, Home Assistant is “gaining a lot of traction.” Check out the complete video to see what happens when Python meets IoT.