Author: aguisande@decisionscience.bi

El otro día me plantearon un caso de uso que me pareció interesante explorar, y debido a que los resultados me fascinaron quiero compartirlo con Uds.

En esta época de anuncios acerca de “…apriete un botón en su computadora y un modelo de Machine Learning o Inteligencia Artificial hará todo lo que Uds. Quiera…” al CMO de un cliente nuestro le pareció súper natural el planteo del siguiente caso:

Transcribir a archivos de texto horas y horas de grabaciones del sistema de atención a clientes.

Ok my friend, desafío aceptado.

Los insumos

Íbamos a utilizar los clips que nuestro amigo menciona, pero como no podíamos acceder todavía a sus archivos, preparé una serie de clips de audio para entrenar los modelos y poder presentar una prueba de concepto completa a nuestro cliente.

Para esto utilicé porciones de video que extraje de algunos webinars que he dictado a través del tiempo y que tenía en el disco de mi laptop. Algunos en español, otros en inglés.

Para el ejemplo que describiré ahora, pueden ver el video completo acá: https://www.decisionscience.bi/2020/09/20/demistifying-price-elasticity/

Subí el video a AWS (específicamente a un bucket de S3), para agilizar el proceso y tenerlo siempre disponible.

Armado del Modelo

Con una simple interface drag and drop, podemos definir el flujo de procesamiento del modelo.

En nuestro primer nodo, debemos definir los datos que utilizaremos, lo que se hace fácilmente con un simple JSON cuya estructura la podemos tomar de la documentación disponible:

En nuestro caso el Id del video, el email a notificar cuando el job se complete y la url conteniendo el activo de datos a utilizar (en este caso nuestro video).

Asignamos las variables definidas en nuestro JSON al payload del modelo, seleccionamos el motor de procesamiento y capturamos los errores (en caso de que surgiera alguno).

Salvamos, hacemos deploy y ya nuestro modelo está listo para utilizar.

Así que pasamos a Alteryx.

Para ejecutar nuestros modelos, existen dos etapas:

• La primera es disparar el job de ejecución al servidor
• La segunda es leer los resultados.

Para ello, tenemos en el Gallery dos herramientas (https://gallery.alteryx.com/#!app/Veritone-aiWARE-Tools/5ff7444e0462d71998d0c4be), una para cada proceso.

Paso 1: Lanzamos el job al servidor:

Seleccionamos desde dónde estaremos alimentando los datos, en este caso desde S3, pero podríamos estar subiendo archivos desde nuestra máquina, tomar datos desde una herramienta conectada al conector de input, o seleccionar archivos subidos anteriormente.

Luego le damos un nombre a nuestro proyecto (en este caso “Webinar Transcription”) y como seleccioné que iba a utilizar un video alojado en S3, debo colocar las credenciales de acceso.

Por último, en la parte inferior de la herramienta, seleccionaremos qué modelo vamos a utilizar. Disponemos de la opción de flujos standard o custom (creados por nosotros, como el que les comenté más arriba)

Y eso es todo lo que necesitamos, así que a ejecutar el workflow.

Paso 2: La segunda parte del ejemplo involucra el acceso a los resultados del modelo de transcripción, donde esperamos leer la transcripción del contenido.

Para eso, en un flujo nuevo, vamos a utilizar la segunda herramienta mencionada más arriba (Query aiWARE Data):

Esta vez, vamos a seleccionar de nuestras opciones “List of Assets”, que corresponde a los resultados de los modelos ejecutados por nosotros previamente.

En caso de que exista algún proceso nuestro, todavía ejecutándose en el servidor, lo veríamos debajo de la sección “Pending AI Jobs”.

Un poco de parsing a los resultados recibidos y podemos acceder a toda la transcripción del audio.

Todo el texto en un campo único, o

Palabra por palabra, con los datos en milisegundos donde aparece en el audio, y el % de confianza de su traducción.

Un caso de uso simple, pero de altísimo valor para las empresas. Pensemos en la aplicación de estas soluciones a transcripción de llamadas de Call Center, testimonios judiciales, generación de manuales en base a dictado, subtitulado de videos, en fin, sus aplicaciones son infinitas.

En breve les estaré compartiendo otros casos de uso (reconocimiento facial en videos, reconocimiento de logotipos en fotos y videos y otros más) en los que estoy trabajando.

Espero que les sea útil y no duden en contactarse si necesitan ayuda/apoyo.

Saludos

A couple weeks ago, I was presented with a solution that was in development, and I volunteered to be a beta tester. I always say YES!  to these opportunities, so I embarked in learning a new (yes, again!) platform to perform ML & AI. Read more about the Alteryx + Veritone partnership and the aiWARE tools.

So, why did I do it? Well, first I’m very curious by nature. Second, the demo I saw explained some ML & AI underworld to a basic guy like me. Finally, because it was fully integrated to Alteryx (so I thought I had that part covered). Very nice starting point to this journey.

The example I saw was a part of an Alteryx webinar where @MarqueeCrew  and @LeahK  (two very well known Alteryx superheroes) were talking, with the video switching faces as they talked. Four or five photos of each were uploaded to train the model (downloaded from Google Images).

Once the model was trained, it ran and voilà! The results show some very curious information:

• It recognized the time where each identified speaker was talking
• It recognized a speaker that was not trained (another Alteryx superhero @CharlieS) 
• The model made the transcription of who said what at what time
• What was the expression in the voice when they said it

For those with me so far, let me tell you… if your head is not exploding right now, it will be in a few paragraphs.

My First Test

So far, that was my experience with the demo, so I decided to do it myself (thinking I wasn’t capable of pulling this off myself). Big surprise! I received two Alteryx macros in advance: the aiWARE Input tool and the Run aiWARE tool (accessible on the Gallery).

As an Alteryx enthusiast, the first thing I did was to drop the Run aiWARE tool on to the canvas. I was pleasantly surprised that I was able to sign up for a free account from within the macro, so I did; yes, you can even do it from within the macro, without leaving Alteryx, and receive a free trial, too!

Now that I have my credentials, I was able to log in, and the main interface of the tool was presented:

Select Data: Lists options to retrieve your data:

Upload Type: If you’re going to be uploading a single file or a folder contents:

Project Name: Just a name to keep your projects organized.

Local File: Since I chose to upload a local file.

Flow Options:

Here starts the amazing journey for me. You can use standard workflows, published by Company, or you can use yours, through the Custom Flows Option.

For the Standard Flows, here is a partial list of those available to Alteryx right now:

I decided to try the Facial Recognition first, based on two previous disappointing experiences I had with two other vendors and this kind of model. When I selected Facial Recognition, I was presented with a new drop down, Select Library:

And a very suggestive option to Train my First Library. Obviously I clicked the link.

In the same panel, I was presented with the Libraries section of the solution, where you can build, edit/update/expand or delete libraries for model training.

I started to look into my work laptop and found some pictures of my kids, so I fed the library with them. The process is simple, easy and straightforward, with instructions at every step. It took me more time to find the pictures than training the model with them.

So finally, I selected the library for this test, and ran the workflow (I am a Browse tool guy in development, so I added one first).

The workflow throws a job to the server to start performing the Facial Recognitions. A very important thing to notice is there’s no need to wait until the task finishes, so meanwhile you can work on another things… (yes, I know, bummer, but your boss will love this!)

The execution results:

If you’ve been reading carefully, I said I received two tools and now is the turn of the second one to appear in a new canvas. The aiWARE Input tool asked me to log in again, and returned the following:

Selection Option: It allows you to choose from where you’ll get results.

List of Assets: In this case, all the jobs I previously ran.

Pending AI Jobs: Shows what jobs are still running on the server

I ran the second workflow (a pretty basic one, btw). 2.9 seconds for the results; I went directly to the results. It’s JSON, and I’m in Alteryx, parsing is no problem!

JSON Parse to the rescue and after a little simple Alteryx magic, I have readable results:

I’m not going to enter in much detail on the results, because I think you need to test this and be amazed yourself, but I will comment on some of them:

Emotion detection (up to seven emotions tested by frame) and Confidence probabilities of each one:

Twenty-nine face landmarks evaluated per subject and their probability score:

Each person in the library recognized, based on a set of four pictures of each (the one that is blank is my goddaughter, whose picture where not included in the training set):

Part of the video where they were recognized:

And a lot of other results, like if the model detected beards, sunglasses, poses, etc.

Conclusion? It took me 10 minutes from scratch to get the experiment done, starting with picture gathering, video uploading, model training and face recognition completed.

A more complex experiment

Since I was already hooked up with these cool new tools, I decided to get into a real business case. A couple of years ago, a potential customer asked us about feature recognition on shopping mall footage.

What a better way to test it right now!

I got a video online (a walk through the Dolphin mall in Miami that somebody posted on the internet), and decided to test the Logo Recognition Engine.

I strongly recommend that you give this a test, I’m sure it’ll blow your mind as it did to me. If you have any questions, please reply to this article. Read more about the Alteryx + Veritone partnership and the aiWARE tools.