我爱自然语言处理 » 语音识别

追忆大师贾里尼克

52nlp — Tue, 21 Sep 2010 12:55:09 +0000

　　这是Google Research Blog上的一篇文章，英文原名“Remembering Fred Jelinek”。在Google Reader上读到的，是Google的这些研究员们对于贾里尼克老先生的一些追忆和缅怀，觉得写得很好，就转载到这里了。

Remembering Fred Jelinek

Posted by Ciprian Chelba, Research Team

It is with great sadness that we note the passing of Fred Jelinek, teacher and colleague to many of us here at Google. His seminal contributions to statistical modeling of speech and language influenced not only us, but many more members of the research community.

Several of us at Google remember Fred:

Ciprian Chelba:
Fred was my thesis advisor at CLSP. My ten years of work in the field after graduation led me to increasingly appreciate the values that Fred instilled by personal example: work on the hard problem because it simply cannot be avoided, bring fundamental and original contributions that steer clear of incrementalism, exercise your creativity despite the risks entailed, and pursue your ideas with determination.

I recently heard a comment from a colleague, “A natural born leader is someone you follow even if only out of curiosity.” I immediately thought of Fred. Working with him marked a turning point in my life, and his influential role will be remembered.

Bob Moore:
I first met Fred Jelinek in 1984 at an IBM-sponsored workshop on natural-language processing. Fred’s talk was my first exposure to the application of statistical ideas to language, and about the only thing I understood was the basic idea of N-gram language modeling: estimate the probability of the next word in a sequence based on a small fixed number of immediately preceding words. At the time, I was so steeped in the tradition of linguistically-based formal grammars that I was sure Fred’s approach could not possibly be useful.

Starting about five years later, however, I began to interact with Fred often at speech and language technology meetings organized by DARPA, as well as events affiliated with the Association for Computational Linguistics. Gradually, I (along with much of the computational linguistics community) began to understand and appreciate the statistical approach to language technology that Fred and his colleagues were developing, to the point that it now dominates the field of computational linguistics, including my own research. The importance of Fred’s technical contributions and visionary leadership in bringing about this revolution in language technology cannot be overstated. The field is greatly diminished by his passing.

Fernando Pereira:
I met Fred first at a DARPA-organized workshop where one of the main topics was how to put natural language processing research on a more empirical, data-driven path. Fred was leading the charge for the move, drawing from his successes in speech recognition. Although I had already started exploring those ideas, I was not fully convinced by Fred’s vision. Nevertheless, Fred’s program raised many interesting research questions, and I could not resist some of them. Working on search for speech recognition at AT&T, I was part of the small team that invented the finite-state transducer representation of recognition models. I gave what I think was the first public talk on the approach at a workshop session that Fred chaired. It was Fred’s turn to be skeptical, and we had a spirited exchange in the discussion period. At the time, I was disappointed that I had failed to interest Fred in the work, but later I was delighted when Fred became a strong supporter of our work after a JHU Summer workshop where Michael Riley led the use of our software tools in successful experiments with a team of JHU researchers and students. Indeed, in hindsight, Fred was right to be skeptical before we had empirical validation for the approach, and his strong support when the results started coming in was thus much more meaningful and gratifying. Through these experiences and much more, I came to respect immensely Fred’s pioneer spirit, vision, and sharp mind. Many of my most successful projects benefited directly or indirectly from his ideas, his criticism, and his building of thriving institutions, from CLSP to links with the research team at Charles University in Prague. I saw Fred last at ACL in Uppsala. He was in great form, and we had a good discussion on funding for the summer workshops. I am very sad that he will not be with us to continue these conversations.

Shankar Kumar:
Fred was my academic advisor at CLSP/JHU and I interacted with him throughout my Ph.D. program. I had the privilege of having him on my thesis committee. My very first exposure to research in speech and NLP was through an independent study that I did under him. A few years later, I was his teaching assistant for the speech recognition class. Fred’s energy and passion for research made a strong impression on me back then and continues to influence my work to this day. I remember Fred carefully writing up his ideas and sending them out as a starting point to our discussions. While I found this curiously amusing at the time, I now think this was his unique approach to ensure clarity of thought and to steer the discussion without distractions. Fred’s enthusiasm for learning new concepts was infectious! I attended several classes and guest lectures with him – graphical models, NLP, and many more. His insightful questions and his active participation in each one of these classes made them memorable for me. He epitomized what a life-long learner should be. I will always recall Fred’s advice on sharing credit generously. In his own words, “The contribution of a research paper does not get divided by the number of authors”. By his passing, we have lost a role model who dedicated his life to research and whose contributions will continue to impact and shape the field for years to come.

Michael Riley:
I got to know Fred pretty well having attended two of the CLSP six-week summer workshops, working on a few joint grants, and visiting CLSP in between. If there is a ‘father of speech recognition’, its got to be Fred Jelinek – he led the IBM team that invented and popularized many of the key methods used today. His intellect, wide knowledge, and force of will served him well later as the leader of the JHU Center for Language and Speech Processing – a sort of academic hearth where countless speech/NLP researchers and students interacted over the years in seminars and workshops. I was impressed that at an age when many retired and after which most of his IBM colleagues had gone into (very lucrative) financial engineering, he remained a vigorous, leading academic. Fernando mentioned the initial skepticism he had for our work on weighted FSTs for ASR. Some years later though I heard that he praised the work to my lab director, Larry Rabiner, on a plane ride that likely helped my promotion shortly thereafter. And no discussion of Fred would be complete without a mention of his inimitable humor, delivered in that loud Czech-accented voice:

Riley [at workshop planning meeting]: “Could they hold the summer workshop in some nicer place than Baltimore to help attract people?”
Fred: “Riley, we’ll hold it in Rome next year and get better people than you!”

Seminar presenter: [fumbling with Windows configuration for minutes].
Fred [very loud]: “How long do we have to endure this high-tech torture?”

The website of The Johns Hopkins University’s Center for Language and Speech Processing links to Fred’s own descriptions of his life and technical achievements.

英文原文链接见：
http://googleresearch.blogspot.com/2010/09/remembering-fred-jelinek.html

注：转载请注明出处“我爱自然语言处理”：www.52nlp.cn

本文链接地址：http://www.52nlp.cn/追忆大师贾里尼克

语音识别和自然语言处理大师贾里尼克去世

52nlp — Fri, 17 Sep 2010 11:50:51 +0000

　　中午在CSDN看到这个不幸的消息：[逝者]自然语言处理大师Fred Jelinek，之后水木自然语言处理版也有nlper转载了英文的相关信息。我读了一下Language Log里的文章，印象比较深刻的是： Jason adds that “He was in fine health and spirits and no one expected this. Those of us who are here are still trying to absorb the loss ourselves”. 　　　　愿老人家一路走好，在天堂里能继续他的“语音识别和自然语言处理”研究！相关文章: recruiting Ph.D. students 追忆大师贾里尼克神奇的约翰霍普金斯夏季研讨会 … 继续阅读 → 相关文章:

SMT经典再回首之Brown90:统计机器翻译与语音识别

52nlp — Sat, 04 Apr 2009 00:00:14 +0000

　　今天我们谈一谈统计机器翻译与语音识别的关系。吴军在《数学之美系列八：贾里尼克的故事和现代语言处理》中提到：
　　“七十年代的IBM 有点像九十年代的微软和今天的Google, 给予杰出科学家作任何有兴趣研究的自由。在那种宽松的环境里，贾里尼克等人提出了统计语音识别的框架结构。在贾里尼克以前，科学家们把语音识别问题当作人工智能问题和模式匹配问题。而贾里尼克把它当成通信问题，并用两个隐含马尔可夫模型（声学模型和语言模型）把语音识别概括得清清楚楚。这个框架结构对至今的语音和语言处理有着深远的影响，它从根本上使得语音识别有实用的可能。贾里尼克本人后来也因此当选美国工程院院士。”
　　贾里尼克用在语音识别上的这个框架，其影响之一就是统计机器翻译。众所周知，Brown90提出的基于信源信道模型的统计机器翻译框架，其基本思想是把机器翻译看成是一个信息传输的过程，用一种信源信道模型对机器翻译进行解释，可以看出，这个框架基本上是学习和借鉴了贾里尼克将语音识别看成通信问题的思想。以下我们具体看看Brown90中所探讨的基本问题。
　　在Brown90中，机器翻译的问题视作如下的过程：已知目标语言中的一个句子T，寻找翻译机(translator)在产生T时所使用的句子S，因此，选择的句子S应能尽可能的使Pr(S|T)最大。利用贝叶斯定理，可以写成：
　　　　　　　
　　在这个等式中，右边的分母Pr(T)并不依赖于S，因此，这个问题也等价于选择合适的S使Pr(S)Pr(T|S)最大，其中Pr(S)被称为源语言S的语言模型概率（语言模型），Pr(T|S)被称为给定S后到T的翻译概率（翻译模型），如下图所示：
　　　　　　
　　相应的，实际的翻译过程由解码器(Decoder)来执行，如下图所示：
　　　　　　　
　　其目标是给定目标语言句子T的情况下，选择一个源语言句子S，使：
　　　　　　　
　　这个公式，也被称为统计机器翻译的基本公式。如果了解语音识别，应该知道语音识别的基本公式：
　　　　　　　
　　其也是由贝叶斯公式推导而来。另外统计机器翻译被分解为三大问题：
　　1、语言模型Pr(S)的参数估计；
　　2、翻译模型Pr(T|S)的参数估计；
　　3、搜索（解码）问题：寻找最优的译文；
　　这三大问题也一一对应着语音识别中的语言模型，声学模型和解码问题。事实上，Brown90在具体描述这三大问题时，每一部分都与语音识别息息相关，如直接采语音识别中广泛使用的n-gram语言模型，在进行翻译模型参数估计时使用语音识别中采用的EM算法，而其搜索算法则采用语音识别中的“stack search”算法。
　　毫不夸张的说，Brown90中的统计机器翻译方法完全脱胎于语音识别的基本框架，语音识别这个保姆在早期统计机器翻译诞生和成长的过程中给予了SMT无微不至的关怀和照顾。
　　之所以有这层亲密的关系，我们可以大制了解一下Brown本人的工作环境，事实上当时他就工作在贾里尼克所在的IBM语音识别实验室里，这个实验室的研究阵容被吴军称之为空前绝后，而Brown90中的作者阵容自然也无比强大了，这个我们下一篇文章里再聊。

注：原创文章，转载请注明出处“我爱自然语言处理”：www.52nlp.cn

本文链接地址：http://www.52nlp.cn/statistical-machine-translation-and-speech-recognition-of-smt-classic-brown90/

神奇的约翰霍普金斯夏季研讨会

52nlp — Tue, 03 Mar 2009 00:00:42 +0000

　　Google吴军在《数学之美系列八》里讲贾里尼克(Frederick Jelinek)的故事时，说他离开IBM后去约翰霍普金斯大学建立了世界著名的CLSP实验室，每年夏天都会邀请世界上20-30名顶级的科学家和学生到CLSP一起工作，使得CLSP成为世界上语音和语言处理的中心之一。
　　CLSP全称约翰霍普金斯语言和语音处理中心（The Johns Hopkins Center for Language and Speech Processing），建立于1992年，其目标是推动语言和语音识别的研究和教育，由美国政府资助。而贾里尼克每年夏天的邀请活动则成就了大名鼎鼎的约翰霍普金斯夏季研讨会（Johns Hopkins Summer Workshop），简称JHU Workshop，著名的开源工具SRILM，Giza++, Moses都是这个研讨会的产物！
　　每年夏季，CLSP都会组织和主办一届侧重于语音和语言工程的研讨会（JHU Workshop）。多年来每届研讨会的研究成果对于大词汇量连续语音识别（LVCSR），自然语言处理（NLP）及对话等领域产生了广泛的影响。
　　JHU Workshop对于那些很有希望但是由于缺乏资源没有得到充分研究的领域提供检验的机会。但是JHU Workshop最大的贡献应该是那些无形资产，如研究小组间思维的撞击和相互促进，或者由著名的专家将知识传授给研讨会的参加者。
　　许多情况下研讨会的研究成果对于语言和语音领域产生很重要的影响并提供解决问题的方案，这些对于政府，工业界和学术界来说也是非常有吸引力的。其他情况下研讨会为许多不同的研究项目埋下了种子，在会议结束后其得到很好的发展，如SRILM, Moses。
　　JHU Workshop对于语音识别及自然语言处理领域做出的另一个重要贡献是：通过训练学生，促进研究者之间的相互学习及对所有的研讨会参与者提供培训等方法造就了一批训练有素的专家。
　　截止目前，JHU Workshop已成功举办了14届（1995—2008），2009年的Workshop将从6月22日到7月31日，为期6周，其主题是“面向语言工程的机器学习(Machine Learning for Language Engineering)”，大家有兴趣和条件可以考虑参加。

　JHU Workshop的主页见：http://www.clsp.jhu.edu/workshops/

注：原创文章，转载请注明出处“我爱自然语言处理”：www.52nlp.cn

本文链接地址：http://www.52nlp.cn/the-magic-of-johns-hopkins-summer-workshop/