Fabiano Luz
Artificial Intelligence Scientist
About Me
Fabiano has a passion for Artificial Intelligence, and everything related to it. He has a long-established experience in researching and developing applications for Automatic Speech Recognition (ASR) since 2006, when he began his studies on CMU Sphinx. In 2013, influenced by other researchers, he started studying and developing AI systems related to Kaldi ASR. From 2015, he started working with End-to-End approaches and Deep Learning architectures to ASR and Speech Synthesis. In 2019, he was awarded his PhD in Neural Semantic Parsing from the most respected South American University: Universidade de São Paulo (USP), Brazil.
Fabiano has over 10 years of experience in developing commercial AI applications. He has stood out as a leader in managing projects and teams in AI projects specially with agile methodologies. Among other projects, Fabiano has worked with more than 10 topics: Text classification, Named Entity Recognition, Semantic Parsing, Automatic Part-of-speech tagging, Text to Speech, Speech to Text, Voice conversion, Voice cloning, Voice detection, Voice identification, Data Prediction, Object Detection in Image.
Research Interests
In general, my interest is in applying advanced AI techniques to voice-based solutions. I'm particularly interested in speech synthesizing with dirigible prosody and emotion, as well as voice identification and recognition.
In the past I worked on
- Chatbots and knowledge representation;
- Logical and statistical inference;
- Machine translation;
- Voice biometrics;
- Text to speech;
- Speech to Text.
- Voice cloning;
- Voice identification and recognition;
- Emotional Text to Speech;
- Voice separation;
- Deep fake;
- Capacity and complexity of different neural network architectures;
- AI for social good.
Education
| Finished | Degree | Institute | Thesis | Advisor |
|---|---|---|---|---|
|
Feb 2019 |
Deep neural semantic parsing: translating from natural language into SPARQL (Link) |
|||
|
Oct 2013 |
Querying ontologies using controlled natural language |
|||
|
Jul 2011 |
Use of technical Natural Language Processing and ontologies in the development of a cognitive QA system |
Research Experience
- Company: Mutant LTDA
- Title: Black Bolt
- Midia: link
Short Description: More than a high quality voice synthesizer, Blackbolt proposes to be a virtual speaker and the production of audios is fully configurable maintaining the quality and humanization standards of the company. The project is being developed based on recent technologies such as: tacotron and wavenet;
- Company: Mutant LTDA
- Title: Xavier
Short description: Xavier is a basic natural language processing tool from Mutant. In addition to extracting text and named entities, Xavier also does some formatting and transformations. The tool was tested and compared with others in the same segment on the market obtaining the best result for Brazilian Portuguese.
- Company: Alabia LTDA
- Title: Alabia Speech Recognition
- Midia: link
Short Description: Short description: Alabia’s speech recognizer was developed to perform well in critical situations, in a telephony environment where the standard format for Brazil is 8 bit and 8KHz, there are different noises and different environments, where intermittence maybe occur. In addition, the acoustic model and the language model are fully customizable for the business to be applied.
Work Experience
- Company: Mutant LTDA
- Main Tasks: Lead people and projects in the context of artificial intelligence (AI) laboratory, choose technologies to be used by the company and support any other AI subject for the company.
Main projects:
- Development of virtual assistant (Claro, Sky, Telefonica);
- Development of Sky's synthesized voice;
- Development of mutant's first speech synthesizer;
- Development of robot debt collector (Claro, Sky).
Important events:
- In 2017, Mutant's artificial intelligence area and laboratory was created, led by me;
- In 2019, Mutant’s artificial intelligence laboratory already had 10 scientists who assisted in both research, development and operation;
- From 2016 to 2020 the company practically doubled in size.
- Company: Finch Soluções
- Main tasks: Design and prototype AI models to be used in the company’s products
Main projects:
- Chatbots;
- Named Entity Recognition.
- Company: Genesys Prime
- Main tasks: develop products using AI
Main projects:
- Question answering systems;
- Chatbots;
- Automatic answering to SMS messages.
Some Presentations
| Date | Event | Title |
|---|---|---|
|
Mar 2020 |
Invited by Interaxa and Atos. Buenos Aires, Argentina |
Natural Language Processing using tensorflow and keras |
|
Jun 2019 |
Mutant's Meetup. São Paulo - SP, Brazil |
Meetup about data culture and artificial intelligence |
|
Oct 2017 |
IME (USP), series of seminars the group of logic and artificial intelligence |
Semantic Parsing Natural Language into SPARQL: an LSTM Enconder-Decoder Neural Net Approach. |
|
Feb 2015 |
Campus Party. São Paulo - SP, Brazil |
Talking with robots |
|
Oct 2014 |
IEEE eScience Conference. Guarujá - SP, Brazil |
Querying Ontologies Using Natural Language |
|
Oct 2013 |
ENIAC. Fortaleza - CE, Brazil |
Ontologies and Chatbots |
Speech Synthesis Samples
Black Bolt (Male voice)
The Black Bolt project from Mutant Company started in 2019 with the goal to develop a virtual speaker. Unlike a conventional synthesizer, a virtual speaker can be easily RUN through natural language. The following audios are from the first phase of the project, which focuses only on the prosodic quality.Black Bolt (Sky voice)
The second result related to Black Bolt was synthesizing audios from a hired speaker. That was the first synthesized voice to be deployed. It's worth telling the story of why it came into production so quickly. The hired speaker got sick and we had to upload the audios urgently. So, we synthesized it with the Black Bolt and we sent it for approval which happened immediately. The first audio is in the original quality at 22KHz, while the second audio has been converted to the URA quality and standards. It's worth to say that customers didn't even notice the difference.Black Bolt (Female voice)
This is Black Bolt's standard female voice, it has been developed in early 2021.Voice Cloning Samples
Cloning (El Chavo del ocho)
Marcelo Gastaldi, a Brazilian voice actor, died in 1995. He is mainly known for dubbing the main character El Chavo from the classic Mexican TV show "El Chavo del Ocho". Due to the great success of the show in Brazil, Marcelo Gastaldi's voice became very popular among Brazilians. In Alabia, we have 'cloned' Marcelo Gastaldi's voice using some audios samples extracted from El Chavo show dubbed in the '80s. The challenge here was to bring back such a popular voice with just a few examples of how Marcelo Gastaldi dubbed El Chavo using audios in bad quality.Cloning (Luciano do Valle)
Another very beloved voice in Brazil is that of sports announcer Luciano do Valle. The announcer passed away in 2014 leaving a vast curriculum and a legion of fans throughout Brazil. The voice generated below was obtained with approximately 5 minutes of excerpts extracted from an interview given to the program Roda Viva.Emotional Synthesis Samples
Black Bolt (Emotional and Transfer Learning)
Another feature we've started to explore in Mutant's laboratories was the ability to add emotion to voices generated by Black Bolt. This example is just a rough draft: we have used 10 minutes of audio from Alex simulating anger, and another 10 minutes of audio simulating happiness. We have used a pre-trained model as a basis.Artificial Voice Samples
Artificial voice (Darth Vader)
This is an example of artificial voice produced, we used a pre-trained model and from this model, we pre-established new characteristics for the artificial voice.Prosodic Control Samples
Black Bolt (Emphasis)
Here we explore the ability to direct emphasis during synthesizing.Example A:"Bom dia FABIANO, como vai você ?" Example B:"Bom dia Fabiano, como vai VOCÊ ?" Example C:"Bom DIA Fabiano, como vai você ?"