Immersive Audio and Musical AI

The sound and music are 50% of the entertainment in a movie. George Lucas

As it is in games, I would also say the same for emerging mediums such as VR, AR, and MX.

I’ve been working recently on a couple of projects related to a few different areas, such as:

As part of this, last week I presented an introductory tutorial at the VRHubTokyo Meetup on Spatial Audio in Unity3D.

This is a really interesting area, with a lot of challenges still to be overcome.

This panel video of immersive audio experts is an excellent introduction to a lot of the issues:

We also had another Canvas Tokyo Musicians Meetup where we discussed a few of these topics, among others.

You can read the event report here: Musicians Meetup #2 Event Report – Mon 11th Sept @ Tacchi Studios

Check out the Spatial Music Visualizer which I also used to discuss a few of the issues around spatial music.

Stay tuned for more news soon!

Chatbot development with Azure Bot Service

Chatbot development and deployment

There’s a lot of hype at the moment about AI in general, and chatbot’s being the next big “app market”. Reality is the tech is still very immature and there are not a lot of use cases where the chatbot workflow is a better UX than simple point and click with a well defined menu and information architecture.

FAQ’s are an important corner of the customer support sector and is one area where they may have use so I decided to test out Microsoft’s bot service, which is still in preview mode: https://azure.microsoft.com/en-us/services/bot-service/

I created an FAQ chatbot for one of my own products as a practical test, which you can see running live here: http://www.harpninja.com/wp/faqhelp/
As you can see from the image there’s quite a bit to it, architecturally and technology wise, with a number of different services from NLU (Natural Language Understanding) to the bot development to connecting various user touchpoint services. There are a number of different types of bots you can implement and an FAQ bot is best implemented using another service they have called https://qnamaker.ai

Overall it’s a pretty nice developer workflow considering the different services they have hooked up to make the development and deployment a relatively painless experience. This FAQ chatbot is using their inbuilt web chat preview which made it a snap to integrate into WordPress without having to code up a plugin. The bot has also been deployed to Skype, Slack, and Facebook Messenger.

There’s a lot of different options for developing chatbots, which this article gives a great overview of: https://www.oreilly.com/ideas/bots-what-you-need-to-know , but I would definitely recommend Microsoft (the new Apple!).

Earthquake Data Scope

Interactive Data Visualization & Exploration

I recently attended the Rakuten Technology Conference 2016 and saw a very interesting presentation from Erik Berlow titled “Next-Generation Interfaces for Thinking with Data“, which was about data scope interfaces.

I’m familiar with Data Science, data mining, and data visualization and have been working with a few tools, from RapidMiner to cloud based ML platforms such as IBM Watson, Microsoft, Amazon, & Google.

Data Scope Interfaces

Erik opened my eyes to some new ways of looking at the problem of constructing visualization and analysis tools in an increasingly data saturated world. He introduced the concept of Data Scopes, interactive tools for exploring multidimensional data and inter-relationships using the human visual system as the pattern recognition engine. The data can be pre-processed with Machine Learning to slice, dice, connect and categorize to provide a rich content source for the visual interfaces to explore.

So I decided to do some prototyping and see what I could come up with to explore the Data Scope concept at a basic level. There are a wealth of great open source Javascript libraries for visualization around. I focused on DC.js which is a mashup of the well known D3 library for manipulating data documents and Crossfilter, a multivariate dataset explorer library originally released by Square.

Living in quake city (Tokyo) I decided to start with the NOAA Global Significant Earthquake Database, 2150 BC to present for this exercise.

As anyone who has worked with data science will tell you, massaging the data into shape is usually the most time consuming and painful part of the process. Garbage in, Garbage out as the old computer saying goes! I actually asked Erik in the Q&A what his processes were to clean the data, and he made the interesting comment that Data Scopes are also useful for rapidly detecting anomalous data. That’s why this earthquake data set is a great example because it has lots of missing data, in addition to lots of useful data.

Have a play with the Data Explorer below and see what patterns and relationships you can detect. See if you can spot where the data is missing, and see if you can spot where data looks a little strange (usually due to inaccuracy of records going back in time I suspect). It was a fun project and I learned a lot which I hope to apply moving forward to some more explorations.

Disclaimer: I did a small amount data manipulation for the purposes of this example . Many of the largest & oldest quake events are missing data in key categories, so I added a baseline 10000 to total injured in order to get some to display in the bubble chart.

Open the Quakescop in a new tab: https://sonicviz.github.io/quakescope/

Source code available on GitHub: https://github.com/sonicviz/quakescope

Internationalizing Meteor with Universe:i18n

What is Internationalization?

Internationalization: Internationalization is the design and development of a product, application or document content that enables easy localization for target audiences that vary in culture, region, or language.
Localization: Localization refers to the adaptation of a product, application or document content to meet the language, cultural and other requirements of a specific target market (a locale).
ref: W3C Localization vs. Internationalization

Why Internationalize your website or webapp?

If your target market(s) speak more than one language you need to consider how to internationalize (and localize) your meteor app.
The official meteor guide briefly explains some internationalization methods.

Why  Internationalize Meteor with Universe:i18n ?

I looked at a few of the different internationalization solutions and for one reason or another found them overly complex to implement. I wanted the simplest possible solution that made the code also easy to read. At this point in time Universe:i18n seems to do that. While the official meteor guide mentions Universe:i18n I felt it lacked some detail needed to get started, hence this short tutorial.

To demonstrate internationalization in the context of a working app I’m going to walk you through you a basic internationalization of the excellent Meteor Chef boilerplate Base project. The Base project is great because it follows the meteor structure guidelines, as well as implements React as the UI with React Router. We’ll refer to this as MC Base through the rest of this tutorial.

Here’s what MC Base looks like out of the box:

mcbase1

MC Base Internationalized, with English selected:

mcbase2

MC Base Internationalized, with Japanese selected:

mcbase3

MC Base Internationalized, with German selected:

mcbase4

The Process

So, how do we do it?

Step 1

The source code for this tutorial is based on MC Base 4.7.0 and is available on github here. I’m using SourceTree and GitFlow, so there are two branches in this repo:

  • master – containing the original MC Base 4.7.0 codebase
  • develop – containing the working internationalized codebase

Familiarize yourself with the MC Base file structure, which is explained here. The rest of the tutorial assumes you’re adding it to the original MC Base in master branch, but you can also just read through it to understand where to look in the develop branch.

Step 2

Add Universe:i18n to your meteor project with “meteor add universe:i18n“.

Step 3

Read about the universe:i18n translation file formats and options here.

Decide which languages you want to implement, then create the JSON files for them and add them to the meteor project in a directory where they can be accessed by the client and the server.

For the purposes of this tutorial we have created three JSON language files that have a namespace of “common” and they have been added  to directory “both/i18n”:

  • en.i18n.json
  • de.i18n.json
  • jp.i18n.json

Here’s the English file named “en.i18n.json”:

{
"_locale": "en-US",
"_namespace": "common",
"navbar": {
  "userName": "User name",
  "Language": "Language",
  "enUS": "English - U.S.",
  "jaJP": "Japanese",
  "deDE": "German",
  "Index": "Index",
  "Documents": "Documents",
  "Signup": "Sign Up",
  "Login": "Log In",
  "Logout": "Logout"
},
"signupform": {
  "Signup": "Sign Up",
  "Firstname": "First Name",
  "Lastname": "Last Name",
  "Email": "Email Address",
  "Password": "Password"
},
"loginform": {
  "Login": "Log In",
  "Email": "Email Address",
  "Password": "Password",
  "ForgotPassword": "Forgot Password?"
}
}

Here’s the Japanese jp.i18n.json :

{
"_locale": "ja-JP",
"_namespace": "common",
"navbar": {
  "userName": "ユーザー名",
  "Language": "言語",
  "enUS": "英語 - 米",
  "jaJP": "日本語",
  "deDE": "ドイツ人",
  "Index": "インデックス",
  "Documents": "ドキュメント",
  "Signup": "サインアップ",
  "Login": "ログイン",
  "Logout": "ログアウト"
},
"signupform": {
  "Signup": "サインアップ",
  "Firstname": "ファーストネーム",
  "Lastname": "苗字",
  "Email": "電子メールアドレス",
  "Password": "パスワード"
},
"loginform": {
  "Login": "ログイン",
  "Email": "電子メールアドレス",
  "Password": "パスワード",
  "ForgotPassword": "パスワードをお忘れですか"
}
}

The strings are simply accessed as keys by references such as namespace.JSONelement1.JSONelement2 etc. For example, common.navbar.Signup or common.loginform.Email , where common is the namespace 2nd line of the JSON files. You don’t have to have a namespace, but for more complex localization efforts it may be needed so this demonstrates how to use it.

Step 4

Identify the files you need to change to add the internationalization dropdown. MC Base demonstrates public routing and authenticated routing. If you wish to have the localization menu available on the navbar of both routes you will need to implement it on both.

  • imports/ui/components/public-navigation.js
  • imports/ui/components/authenticated-navigation.js

Identify any other files that need to be localized. Since we are also localizing the login and signup pages you will need to modify them as well.

  • imports/ui/pages/login.js
  • imports/ui/pages/signup.js

Step 5

Open the file imports/ui/components/public-navigation.js and modify it as follows:

You need to import universe:i18n, so add the following after the main imports in the file:

import i18n from 'meteor/universe:i18n';

You need to add a dropdown menu with menu items, and as these are not in the original import you need to add them:

import { MenuItem, Nav, NavDropdown, NavItem } from 'react-bootstrap';

There’s a couple of ways to translate strings with Universe:i18n, but since we’re using React it comes with a very neat inline react component which you can read about here. All you need to do is create an instance of the react translation component and feed it the keys from our translation files.

//instance of translate component with top-level context
const T = i18n.createComponent();

The following code from public-navigation.js shows how that is implemented on the main navigation bar.
Note the language switching is done simply by changing the locale eg: “i18n.setLocale(‘ja-JP’);” and everything else is automatically handled by the string references in the NavItems to the JSON language files.

export const PublicNavigation = React.createClass({

handleSelect(eventKey) {
event.preventDefault();
switch (eventKey) {
  case 5.1:
    i18n.setLocale('en-US');
    break;
  case 5.2:
    i18n.setLocale('ja-JP');
    break;
  case 5.3:
    i18n.setLocale('de-DE');
    break;
  default:
    i18n.setLocale('en-US');
    break;
}
},
render() {
return (
<div>
<Nav pullRight>
<NavDropdown eventKey={5} title={<T>common.navbar.Language</T>} id="nav-dropdown" onSelect={this.handleSelect}>
  <MenuItem eventKey={5.1}><T>common.navbar.enUS</T></MenuItem>
  <MenuItem eventKey={5.2} ><T>common.navbar.jaJP</T></MenuItem>
  <MenuItem eventKey={5.3} ><T>common.navbar.deDE</T></MenuItem>
</NavDropdown>
<LinkContainer to="signup">
 <NavItem eventKey={3} href="/signup"><T>common.navbar.Signup</T></NavItem>
</LinkContainer>
<LinkContainer to="login">
 <NavItem eventKey={4} href="/login"><T>common.navbar.Login</T></NavItem>
</LinkContainer>
</Nav>
</div>
);
}
});

Step 6

Modify the imports/ui/components/authenticated-navigation.js file to reflect the same structure as Step 5. Note it has two extra menu items (index, documents) for the authenticated route.

export const AuthenticatedNavigation = React.createClass({

handleSelect(eventKey) {
event.preventDefault();
  switch (eventKey) {
  case 5.1:
    i18n.setLocale('en-US');
    break;
  case 5.2: 
    i18n.setLocale('ja-JP');
    break;
  case 5.3:
    i18n.setLocale('de-DE');
    break;
  default:
    i18n.setLocale('en-US');
    break;
}
},
render() {
return (
<div>
<Nav pullLeft>
<IndexLinkContainer to="/">
  <NavItem eventKey={1} href="/"><T>common.navbar.Index</T></NavItem>
</IndexLinkContainer>
<LinkContainer to="documents">
  <NavItem eventKey={2} href="/documents"><T>common.navbar.Documents</T></NavItem>
</LinkContainer>
</Nav>
<Nav pullRight>
<NavDropdown eventKey={5} title={<T>common.navbar.Language</T>} id="nav-dropdown" onSelect={this.handleSelect}>
  <MenuItem eventKey={5.1}><T>common.navbar.enUS</T></MenuItem>
  <MenuItem eventKey={5.2} ><T>common.navbar.jaJP</T></MenuItem>
  <MenuItem eventKey={5.3} ><T>common.navbar.deDE</T></MenuItem>
</NavDropdown>
<NavDropdown eventKey={ 3 } title={ userName() } id="basic-nav-dropdown">
  <MenuItem eventKey={ 3.1 } onClick={ handleLogout }><T>common.navbar.Logout</T></MenuItem>
</NavDropdown>
</Nav>
</div>
);
}
});

Step 7

Modify the signup & login forms as per step 4 and 5.

Step 8

That’s it.

Further Improvements

It would be nice to add flag icons instead of text for languages but I’ll leave that as an exercise for you.
Add it to the github and send me a PR;-)

Thanks for reading, if you got this far!

Cognitive Technology – a Quickstart Guide Infographic

“I’m sorry Dave, I’m afraid I can’t do that” HAL 9000

Cognitive Technology, Smart Apps, Data Science, Artificial Intelligence…the list of new technology buzz goes on and on.

But what do they actually mean?

Cognitive Technology Infographic

Here’s an infographic to help explain what Cognitive Technology is and how it can be applied for business use.

The use cases are many and varied, depending on your business needs.

  • Expert systems
  • Autonomous vehicle control on Land. Sea, and Air
  • Robotics
  • Sales and inventory forecasting
  • Price modeling
  • Image, video, and handwriting recognition
  • Text and document analysis for classification, concept, and sentiment extraction
  • Customer segmentation and market analysis
  • Social network analysis for brand and product affinity
  • Anomaly detection for fraud and other behavior patterns

There are also many vendors in this space ranging from the big 4

  • IBM
  • Microsoft
  • Google
  • Amazon

to many smaller players focused on providing different levels of service.

Enjoy!
Image Credit: Evolution by Jakob Vogel from the Noun Project

Cognitive Technology Quickstart Infographic

Cognitive Technology Quickstart Infographic

Pin It on Pinterest