With the goal of launching a new research collaboration, for the development of Adaptive Conversational Engines, a research team led by Shinji Watanabe, Associate Professor at the Language Technologies Institute (CMU) traveled to Lisbon from June 15 to 18. This falls under the scope of the CMU Portugal supported project, “ACE-FM: Adaptive Conversational Engine powered by Foundation Models”, “which aims to enhance the capability of speech foundation models to understand long-form, multi-party conversations toward conversational AI. Besides, it supports the collaboration between CMU and PRR Mobilizing Agenda Accelerat.AI, led at INESC-ID by Human Language Technologies researcher Alberto Abad, Associate Professor at Instituto Superior Técnico (Técnico).”

As part of the visit, INESC-ID hosted an intensive two-day workshop, kicking off the collaboration while exchanging knowledge between teams. The workshop featured presentations from both teams and a keynote by Shinji Watanabe, titled “Open Whisper-Style Speech Models: Transparency, Scalability, and Advancing Explainability”. The  first day provided an overview of the models developed at CMU, whilehands-on tutorials by CMU students Masao Someki, MLT student, and William Chen, PhD student, empowered researchers based in Portugal to apply them effectively. The second day focused on practical sessions in model fine-tuning with custom data and introduction to ESPnet3, CMU’s next-generation speech modeling platform. of what was a very productive workshop.

Along with fellow INESC-ID Human Language Technologies researcher, Isabel Trancoso, Alberto Abad shared insights into their team structure and highlighted successful past collaborations within the CMU Portugal program. According to project leads Aberto Abad and Shinji Watanabe, the visit helped both teams to gain a deeper understanding of each other’s expertise, research styles and focus areas, and also strengthen connections with fellow language technologies’ researchers.

Both teams expect that this collaboration will lay the foundation for future projects and long-term academic exchanges.  “The outcomes of this collaboration will benefit not only the Accelerat.ai project, but also other PRR-supported initiatives, including the development of a national Portuguese large language model”, shared Alberto Abad.

The 2024 CMU Portugal PRR Call for Collaborative Research Grants aims to strengthen collaboration between CMU and Portuguese research institutions aligned with the Portuguese Recovery and Resilience Plan (PRR) Mobilizing Agendas. The call funded 12 projects, representing a significant step in fostering impactful research between Portugal and CMU.

Read the full original article here