Automatic Speech Recognition (ASR) is a key technology for a variety of industrial and IT applications; and it extends the reach of IT across people as well as applications. Automatic Speech Recognition (ASR) is gaining a growing role in a variety of applications, such as hands-free operation and control (as in cars and air-
planes), automatic query answering, telephone communication with information systems, au-tomatic dictation (speech-to-text transcription), government information systems, etc. In fact, speech communication with computers, PCs, and household appliances is envisioned to be the dominant human-machine interface in the near future. In spite of the tangible success of this technology for the English language and other languages, there are still many issues for Arabic language that need to be addressed by
Arabic Phonetic dictionaries for Speech recognition
Mohamed Ali, King Fahd University of Petroleum and Minerals, Saudi Arabia
Moustafa Elshafei, King Fahd University of Petroleum and Minerals, Saudi Arabia
Mansour Al-Ghamdi, King Abdulaziz City of Science and Technology, Saudi Arabia
Husni Al-Muhtaseb, King Fahd University of Petroleum and Minerals, Saudi Arabia
Atef Al-Najjar, King Fahd University of Petroleum and Minerals, Saudi Arabia
AbSTrACTPhonetic dictionaries are essential components of large-vocabulary speaker-independent speech recognition systems. This paper presents a rule-based technique to generate phonetic dictionaries for a large vocabulary Arabic speech recognition system. The system used conventional Arabic pronunciation rules, common pro-nunciation rules of Modern Standard Arabic, as well as some common dialectal cases. The paper gives in detail an explanation of these rules as well as their formal mathematical presentation. The rules were used to generate a dictionary for a 5.4 hour corpus of broadcast news. The rules and the phone set were tested and evaluated on an Arabic speech recognition system. The system was trained on 4.3 hours of the 5.4 hours of Arabic broadcast news corpus and tested on the remaining 1.1 hours. The phonetic dictionary contains 23,841 definitions corresponding to about 14232 words. The language model contains both bi-grams and tri-grams. The Word Error Rate (WER) came to 9.0%.
researchers to catch up with the progress of the ASR technology in the other languages.
One of the key components of the modern large-vocabulary speech recognition systems is the pronunciation or phonetic dictionary. This dictionary serves as an intermediary between the Acoustic Model and the Language Model in speech recognition systems. It contains a subset of the words available in the language and the pronunciation of each word in terms of the phonemes or the allophones available in the acoustic model.
For instance, the CMU dictionary for North American English contains over 125,000 words and their transcriptions (CMU, 2008). The format of this dictionary is particularly useful for speech recognition and synthesis, as it has mappings from words to their pronunciations in the given phoneme set The current phoneme set contains 39 English phonemes, for which the vowels may also carry lexical stress. Because of the large number of pronunciation exceptions in English, this dictionary was essentially built manually by experts over many years.
On the other hand, pronunciation of Arabic text follows specific rules when the text is fully diacritized. Many of these pronunciation rules can be found in Elshafei (1991), and Alghamdi el. al. (2004).
The statistical approach for speech recogni-tion (Huang etal, 2001; Jelinek, 1998; Rabiner & Juang, 1993) has virtually dominated Automatic Speech Recognition (ASR) research over the last few decades, leading to a number of suc-cesses (Lee, 1988; Soltau et al, 2007; Stallard et al., 2008; Young, 1997; Zhou et .al, 2003). The statistical approach is dominated by the powerful statistical technique called Hidden Markov Model (HMM) (Rabiner 1989). The HMM-based ASR technique allowed to build many successful applications that depend on large vocabulary speaker-independent continu-ous speech recognition.
The HMM-based technique essentially consists of recognizing speech by estimating the likelihood of each phoneme at contiguous, small frames of the speech signal (Huang et al., 2001; Rabiner & Juang, 1993). Words in the
target vocabulary are modeled into a sequence of phonemes, and then a search procedure is used to find, amongst the words in the vocabulary list, the phoneme sequence that best matches the sequence of phonemes of the spoken word.
Two notable successes in the academic community in developing high performance large vocabulary speaker independent speech recognition systems are the HMM tools, known as the HTK tool kit, developed at Cambridge University, (HTK, 2007), and the Sphinx sys-tem developed at Carnegie Mellon University (Huang et al., 1993; Lamere et al, 2003; Noa-many et al., 2007; Placeway et al., 1997).
Development of an Arabic speech recogni-tion is a multi-discipline effort, which requires integration of Arabic phonetic (Alghamdi, 2000; Alghamdi et al, 2004), Arabic speech processing techniques (Elshafei, 1991; Elshafei et al., 2002), and Natural languages processing (Elshafei et al., 2006).
Development of Arabic speech recogni-tion systems has recently been addressed by a number of researchers (Elshafei. et al., 2008; Hiyassat, 2007; Noamany et al, 2007; Soltau et al., 2007). Saroti et al. (2007) used Sphinx tools for Arabic speech recognition. They dem-onstrated the use of the tools for recognition of isolated Arabic digits. The data was recorded from 6 speakers. They achieved digits recogni-tion accuracy of 86.66%. Hiyassat (2007), in his Ph.D. thesis, developed a tool to generate Arabic pronunciation dictionaries. The gener-ated dictionaries are based on a small MSA speech corpus consisting of digits or command and control vocabulary.
A workshop was held in 2002 at John Hopkins University (Kirchhoff et al., 2003) to define and address the challenges in developing a speech recognition system using Egyptian dialectic Arabic for telephone conversations. They proposed to use Romanization method for transcription of the speech corpus.
Billa et al. (2002) addressed the problems of indexing of Arabic news broadcast, and dis-cussed a number of research issues for Arabic speech recognition.
Journal of Information Technology Research, 2(4), 67-80, October-December 2009 69
Further research in Arabic morphology was performed by Krichhoff et al (2006). They represented four different approaches for Arabic language modeling and introduced a novel tech-nique called factored language models. Xiang et al. (2006) also investigated algorithms to separate words and affixes in language models. Afify et al. (2006) proposed a word decomposi-tion morphological language model to improve recognition rates for Iraqi dialect.
Messaoudi et al. (2006) investigated the problem of generating phonetic dictionaries and the effect of using morphological rules to generate pronunciations for huge databases of more than 1 million words. Gales et al. (2007) studied the problem of generating phonetic dictionaries, while focusing on the effect of multiple pronunciations on recognition quality. Their research emphasizes on the inclusion of unsupervised training data as a way to improve the overall system accuracy. An enhancement to that effort was done in (Diehl et al, 2008) where a multi-phase pronunciation generation is performed, with expert rules that cover cases that can’t be captured with morphological analyzers.
Due to the significant increase in avail-able Arabic speech data, recent research on developing complete Automatic Arabic Speech Recognition (AASR) systems has become sig-nificant, with efforts from IBM (Soltau et al, 2007) and CMU/Interact group (Noamany et al, 2007). Both projects are parts of the GALE program (Gale, 2008) supported by DARPA. Both research teams highlighted the importance of speaker adaptation in improving recognition quality.
This paper addresses the phonetic dic-tionary component of Arabic speech recogni-tion systems. We provided detailed rules for automatic generation of the Arabic phonetic dictionaries and described the evaluation of these rules using an Arabic broadcast news speech recognition system.
In Section 2, we discuss the Arabic pho-neme set of choice. Then, in Section 3 we de-scribe methodology and formulation of the rules for generating the phonetic dictionary. Section
4 discusses in details the set of developed rules and their implications. Finally, in Section 5, we present an evaluation of the rule set by generat-ing various test cases of the phonetic dictionary and compare the recognition results.
ArAbIC PhoNEmE SET
Table 1 shows the listing of the phoneme set used in training and the corresponding phoneme symbols. The table also shows illustrative ex-amples of the vowel usage. The chosen phoneme set is based on our previous experimentation with Arabic text-to-speech systems (Alghamdi et al. 2002; Alghamdi, 2003; Elshafei, 1991), and the corresponding phoneme set which is successfully used in the English ASR (CMU, 2008). The English ASR phoneme set is also given in Table 2 for quick reference. Roman names for Arabic letters are the same as the ones used in Unicode standard (Unicode Con-sortium, 2006).
The regular Arabic short vowels /AE/, /IH/, and /UH/ correspond to the Arabic diacritical marks Fatha, Damma, and Kasra respectively. The /AA/ is the pharyngealized allophone of /AE/, which appears after an emphatic letter. Similarly, the /IX/ and /UX/ are the pharyngeal-ized allophones of /IH/ and /UH/ respectively. When /AE/ appears before an emphatic letter, its allophone /AH/ is used instead. When a short vowel is located between two nasal letters in the same syllable it is likely to be nasalized. The allophones /AN/, /IN/, and /UN/ are the nasalized versions of /AE/, /IH/, and /UH/ re-spectively, however, they were not considered in this reported work.
The regular Arabic long vowel allophones are /AE:/, /IY/ and /UW/ respectively. The length of a long vowel is normally equal to two short vowels. The allophones /AY/ and /AW/ are actu-ally two vowel sounds in which the articulators move from one post to another. These vowels are called Diphthongs. The allophone /AY/ appears when a Fatha comes before an unvowelled Yeh. Similarly, /AW/ appears when a Fatha comes before an undiacritized Waw.
70 Journal of Information Technology Research, 2(4), 67-80, October-December 2009
The Arabic voiced stops phonemes /B/ and /D/ are similar to their English counter parts. /DD/ corresponds to the sound of the Arabic Dhad letter.
The Arabic voiceless stops /T/ and /K/ are basically similar to their English counter parts.
The sound of the Arabic emphatic let-ter Qaf is represented by the phone /Q/. The Hamza plosive sound is represented by the phone /E/.
The normal allophone of Jeem is /JH/. The Arabic affricative sound /JH/ is similar to the corresponding one in English, while /ZH/ is a concatenation of a voiced stop followed by a fricative sound. The /ZH/ allophone is more common in the unvoweled positions, but could also replace /JH/ in some dialects. The third version of Jeem is the /G/ allophone which is commonly used in the Egyptian dialect. The /G/ allophone replaces the Qaf letter in many Arabic dialects.
The voiceless fricatives are produced with no vibration of the voice cords. The sound is produced by the turbulence flow of air through a constriction. The Arabic voiceless fricatives /F/, /S/, /TH/, /SH/, and /H/ are basically similar to their English twins. In addition, the Arabic
phones /SS/, /HH/, and /KH/ are the sounds of the Arabic letters Sad, Hah, and Khah re-spectively.
Voiced Fricatives are generated with simul-taneous vibration of the vocal cords. The Arabic voiced fricative phones are /AI/, /GH/, /Z/, and /DH/ corresponding to the sound of the Arabic letters: Ain, Ghain, Zain, and Thal.
The Arabic resonants are similar to the English resonant phones. These are /Y/ for Yeh, /W/ for Waw, /L/ for Lam, and /R/ for Reh.
ArAbIC PhoNETIC dICTIoNAry
Using the selected phoneme set, we developed a set of rules that are used to automatically generate the phonetic pronunciations for Arabic words. We also created a set of tools that process the given Arabic text and generate all possible pronunciations for every word in the text.
Rules are provided for each Arabic letter available in the Unicode listing (45 letters). Each rule tries to match certain conditions on the context of the letter and provide a replace-ment from the phoneme list. Replacements can be one or more phonemes. Some letters don’t have an effect on pronunciation or, depending
Phoneme Arabic Letter
Example Description
/AI/ ع Arabic Voiced Pharyngeal Fricative Consonant AIN
/GH/ غ Arabic Voiced Velar Fricative Consonant GHAIN, also a variation of QAF in many dialects.
<P>: Prefix letters (WAW, BEH, FEH, • KAF, and LAM).<E>: Emphatic letters (TAH, SAD, DAD, • and ZAH).<PH>: Pharyngeal letters, or semi-em-• phatic letters (QAF, GHAIN, KHAH, and REH).
The pre-condition has one of the follow-ing formats:
• (?<=pattern): context before the current position matches the pattern.
• (?<!pattern): context before the current position does not match the pattern.
In the same way, the post-condition has one of the following formats:
• (?=pattern): context after the current po-sition matches the pattern.
• (?!pattern): context after the current posi-tion does not match the pattern.
Patterns use the following operators to define expressions:
• Alternation: A vertical bar (|) is used to separate alternatives.
• Grouping: Parentheses () are used to define groups that determine scope and precedence of the operators and to build complex expressions.
• Optional Matching: A question mark (?) is used to mark parts of the expression that may or may not exist.
The right hand side of the rule defines the replacement, which can either be a phoneme or a sequence of phonemes from the phoneme list, or the letter might not have a matching phoneme and will be omitted from pronuncia-tion. This case is marked with an asterisk (*) on the right hand side.
We define a rule set that covers all possible Arabic letters that are used in typing. Many of
the rules are straight forward; they match the Arabic letters to their corresponding phonemes as explained in Table 1. Vowels require more elaborate rules to cover all possibilities. Special attention is required for nasalized consonants (Meem and Noon) and a few more exceptions that will be explained in the following sec-tions.
The following section lists the proposed rules with explanation of the meaning of each rule or rule group.
ThE rulES
The Arabic pronunciation rules are listed in Table 3. The left column states the rule, while the right column provides brief explanation of the rules.
The following is a sample from the gener-ated phonetic dictionary:
E AE: B AE: R IX N ٍراَبآE AE: B AA: R IX N ٍراَبآ(2)E AE: KH AA R رَخآE AE: KH AA R AA َرَخآE AE: KH AA R UW N AE َنْوُرَخآE AE: KH AA R IX: N AE َنْيِرَخآE AE: KH AA R IX: N ْنْيِرَخآE AE: KH AA R ْرَخآE AE: KH IX DH AE T UH N ٌةَذِخآE AE: KH IX R AA َرِخآE AE: KH IX R ْرِخآE AE: DH AE: R ْراَذآ:E AE: S Y AE اَيْسآE AE: S Y AE: N ْناَيْسآE AE: S Y AE W IH Y AE H ْةَّيِوَيْسآE AE: S Y AE W IH Y AE T (2)ْةَّيِوَيْسآE AE: F AE: Q IX ِقاَفآE AE: F AE: Q ْقاَفآE AE: L لآE AE: L AE: F IH N ٍفالآE AE: L AE: F ْفالآE AE: L AE: F IH ِفَالآ
74 Journal of Information Technology Research, 2(4), 67-80, October-December 2009
Pronunciation Rules DescriptionHAMZA: .-> E ALEF_WITH_MADDA_ABOVE: .(?!FATHA)-> E AE: .(?=FATHA)-> E ALEF_WITH_HAMZA_ABOVE: .-> E WAW_WITH_HAMZA_ABOVE: .-> E ALEF_WITH_HAMZA_BELOW: .-> E .(?!KASRA|KASRATAN)-> E IH YEH_WITH_HAMZA_ABOVE: .-> E
A HAMZA or other letters that are combined with a HAMZA will always be replaced by the phoneme /E/. The rules take in consideration cases where the user might have forgotten to put the proper dicritization over the (آ) and (إ). Both symbols have implicit vowels that must be included even if the user misses them. This is not the case, however, for (أ), (ئ) and (ؤ). The user must explicitly define the vowels.
ALEF: (?<=<S>).-> E .-> *
The ALEF (ا) is always omitted in pronunciation. The only exception is when the letter comes at the beginning of the word, in which case it might be pronounced as glottal stop /E/; we cannot be certain which pronuncia-tion the user will use, so both alternatives are used in that case.
BEH: .-> B
The letter BEH (ب) is always matched with the phoneme /B/.
TEH_MARBUTA: .(?=<T>)-> H .-> T
The letter TEH_MARBUTA is formally pronounced as an /H/ if the speaker stops on it. However, some dialects will pronounce it as a /T/ regardless of stopping or not. The dictionary will include both pronunciation possibili-ties.
TEH: .-> T
The letter TEH (ت) is always matched with the phoneme /T/.
THEH: .-> TH .-> S
The standard pronunciation for the letter THEH is /TH/. However, in many dialects, such as the Egyptian dialect, it is pronounced as /S/.
JEEM: .-> J .-> JH .-> G .-> ZH
JEEM is another letter that has multiple pronunciations depending on dialectal differences. Also, foreign names (in the Egyptian standard) that have either G or J are always transliterated to a JEEM.
HAH: .-> HH KHAH: .-> KH
These letters are always matched to their corresponding phonemes.
DAL: .(?=TEH<V>)-> * .(?!TEH<V>)-> D
If the letter DAL is followed by a vowelled TEH then it is omitted in pronunciation. For example, the word .”مُتدرأ“
THAL: .-> DH .-> Z
Multiple pronunciations are caused by different dialects.
REH: .-> R ZAIN: . -> Z SEEN: .-> S SHEEN: .-> SH SAD: .-> SS
These letters are always matched to their corresponding phonemes.
Journal of Information Technology Research, 2(4), 67-80, October-December 2009 75
If the letter DAD (ض) is followed by a vowelled TEH or TAH then it is omitted in pronunciation. For example, the words “مُتضفأ” and “متررُطضا”. (germination taken care of by the acoustic model)
TAH: .-> TT ZAH: .-> DH2 .-> Z AIN: .-> AI GHAIN: .-> GH TATWEEL: .-> * FEH: .-> F QAF: .-> Q .-> G .-> GH .-> E KAF: .-> K
These rules match the letters to their corresponding pho-nemes while taking account of dialectal differences.
LAM: (?<=(<P><V>)?ALEF FATHA?).(?= <SH>)-> * .-> L
The first rule takes care of the case of the Shamsi Lam. If the LAM is part of the (ـلا) followed by a letter from the Shamsi group then it is possibly omitted from pronunciation. However, not every speaker will abide to this rule all the time and exceptions might happen. That’s why another rule will always create an alternative pronunciation that has the /L/ phoneme included.
MEEM: .-> M NOON: .(?=BEH)-> M .-> N
The letter MEEM (م) is always pronounced as an /M/. The letter NOON (ن) might also be pronounced as an /M/ if it was followed by a BEH. But again the speaker might not abide to that rule and pronounce it as an /N/.
HEH: .-> H
HEH is always replaced by /H/.
WAW: (?<=(FATHA|DAMMA)).(?!<V>)-> * (?<=(FATHA|DAMMA)).(?=<V>)-> W (?<!(FATHA|DAMMA)).-> W
The letter WAW (و) is sometimes treated as semi-consonants /W/ or /AW/ and other times it is treated as a long vowel, depending on its context. If the letter WAW is not vowelled and is preceded by a DAMMA then it is considered to be a long vowel. Examples include “موُجُن”. The case of the semi-vowel /AW/ is similar to the long vowel, except it is then preceded by a FATHA. In this case the WAW is omitted. The insertion of the /AW/ phoneme is handled by the FATHA rules as it will follow shortly. In the rest of the cases the WAW is converted to the semi-vowel /W/.
ALEF_MAKSURA: .-> *
The ALEF MAKSURA (ى) is always omitted.
YEH: (?<=(FATHA|KASRA)).(?!<V>)-> * (?<=(FATHA|KASRA)).(?=<V>)-> Y (?<!(FATHA|KASRA)).-> Y
As with the WAW, YEH (ي) is sometimes treated as the semi-consonants /Y/ and /AY/ and sometimes treated as a long vowel. Rules here follow the same logic for the WAW.
Table 3. Arabic pronunciation rules
76 Journal of Information Technology Research, 2(4), 67-80, October-December 2009
The Dictionary was tested on a large vocabulary speaker independent Arabic speech recognition system (Elshafei et al., 2008). The system was built using CMU Sphinx tools. The System was trained and tested on 5.4 hours of Arabic broadcast news corpus in Modern standard Arabic.
To test and validate the proposed phoneme set and the rules we split the audio recordings into training and testing sets. The training set contained around 4.3 hours of audio while the testing set contained the remaining 1.1 hours. We used the CMU language toolkit to build a statistical language model from the transcription of the full 5.4 hours of audio.
Several test cases were built to validate our choice of the phoneme set. The main focus was on vowels since they impose most of the complexity and variety to the rules. For each of these cases the recognition model is rebuilt with the modified dictionary, and performance of the speech recognition is evaluated on the test set of the speech utterances (1144 voice files).
First, we tested the AASR system using the phone set and the rules outlines in the previous sections as the base system (Elshafei et al, 2008). We then developed four test cases.
The first test case studies the effect of re-moving the emphatic and pharyngeal vowels. In this model, the emphatic and pharyngeal vowels were removed from the phoneme set, which reduces the size of the model. The rules that apply to these vowels were also disabled.
Pronunciation Rules DescriptionFATHATAN: (?<!<E>|<PH>).-> AE N (?<=<E>).-> AH N (?<=<PH>).-> AA N DAMMATAN: (?<!<PH>).-> UH N (?<=<PH>).-> UX N KASRATAN: (?<!<PH>).-> IH N (?<=<PH>).-> IX N
These rules differentiate between the emphatic and or pharyngeal versions of the vowels. Each rule appends an /N/ sound to the pronunciation. In order to reduce the complexity of the dictionary, we omit the case where the speaker stops at the end of the word and doesn’t proceed to pronounce the /N/ sound. We assume that whenever the symbols for Tanween are present in the corpus that the user has actually pronounced them.
FATHA: (?<!<E>|<PH>).(?!ALEF|((WAW|YEH) (<L>|<T>)))-> AE (?<!<E>|<PH>).(?=ALEF)-> AE: (?<=ALEF_WITH_MADDA_ABOVE).-> AE: (?<=<E>).(?!ALEF|((WAW|YEH)(<L>| <T>)))-> AH (?<=<E>).(?=ALEF)-> AH: (?<=<PH>).(?!ALEF|((WAW|YEH)(<L>|<T>)))-> AA (?<=<PH>).(?=ALEF)-> AA: .(?=WAW (<L>|<T>))-> AW .(?=YEH (<L>|<T>))-> AY
First two rules are responsible for the long and short versions of the normal vowels. Third rule is also for the long vowel where the FATHA is followed by an (آ). Rules 4-7 are for the emphatic and pharyngeal versions of the vowel. Rules 8-9 takes care of the semi-vowels AW and AY as mentioned in the rules for the WAW and YEH.
DAMMA: (?<!<PH>).(?!WAW)-> UH (?<=<PH>).(?!WAW)-> UX (?<!<PH>).(?=WAW<V>)-> UH (?<=<PH>).(?=WAW<V>)-> UX .(?=WAW(<L>|<T>))-> UW KASRA: (?<!<PH>).(?!YEH)-> IH (?<=<PH>).(?!YEH)-> IX (?<!<PH>).(?=YEH<V>)-> IH (?<=<PH>).(?=YEH<V>)-> IX (?<!<PH>).(?=YEH (<L>|<T>))-> IY (?<=<PH>).(?=YEH (<L>|<T>))-> IX:
Rules for DAMMA and KASRA follow the same logic for the FATHA.
SHADDA: .-> * SUKUN: .-> *
Shadda and Sukun symbols have no matching pho-nemes.
Table 3. Arabic pronunciation rules
Journal of Information Technology Research, 2(4), 67-80, October-December 2009 77
In the second test case we examined the effect of merging the long and short versions of vowels into a single phoneme (for example /AE/ and /AE:/) to test whether the tri-phone acoustic models in the speech recognition engine would be capable of handling these vowels without the need to introduce additional phonemes.
In the third case we examined alternative rules for handling the vowels preceding the definite article AL ALTA’REEF (ـلا), and some special cases for the short vowel /AE/ and the long vowel /AE:/.
In the forth case we examined alternative rules for dealing with the gemination (Shaddah), and for co-articulation effects of the emphatic consonants on the preceding vowels.
Results for these test cases are shown in Table 4.
Further analysis indicates that many of the word substitution errors are due to slight differences (deletion/substitution) of diacritical marks, especially the end cases. Since MSA text is written without diacritical marks, the error analysis was carried out once more after remov-ing all the diacritical marks. The percentage of the correctly recognized words was 92.84%. The WER dropped to 9.0%.
The results of these tests lead us to conclude that it is necessary to include both emphatic and pharyngeal vowels and maintain separate phonemes for short and long vowels. The tests clearly validate the proposed phoneme set and
the proposed rules for automatic generation of the Arabic pronunciation dictionaries for Arabic speech recognition applications. However, we believe that more research work is still needed to achieve better accuracy results.
The rules we proposed were based on the assumption that the triphone model used in HMMs will be able to capture gemination cases (double consonants). To validate that, we built an additional test case where we replace geminated letters with double phonemes, for example, a Beh followed by Shaddah will be replaced by (/B/ /B/). However, this didn’t improve the accuracy of the model.
CoNCluSIoN
The paper provides a comprehensive set of rules for automatic generation of Arabic phonetic dictionary. This result was part of an on-going research towards achieving large vocabulary, speaker independent, natural Arabic automatic speech recognition system. The generated dic-tionary was based on about 14 K vocabulary in a 5.4 Arabic broadcast news corpus. The Dictionary was tested on a large vocabulary speaker independent Arabic speech recogni-tion system. The speech recognition system achieves a comparable accuracy to English ASR system for the same vocabulary size. The rules presented here focused only on the Arabic phonetic rules, which address the mapping from
Table 4. Summary of the performance of the AASR system for different phone/rules test cases.
Test case accuracy I D S WER
Base system 90.1 168 82 838 11.71
1- Emphatic and pharyngeal versions of vowels were removed 89.8 168 89 858 12
2- Long versions of Vowels were removed 87.96 179 99 872 12.33
3- Alternative rules for FATHA and for /AE/ and /AE:/ in the definite articles 88.47 214 80 991 13.84
4- Alternative rules for co-articulation effects of the emphatic consonants 89.81 157 77 869 11.88
* I: Word Insertion errors; D: word deletion errors; S: word substitution errors; WER: % word error rate, based on 9288 words in a 1144 sentence test corpus.
78 Journal of Information Technology Research, 2(4), 67-80, October-December 2009
the grapheme to phonemes. Generation of the vocabulary using the Arabic morphological pat-terns was not addressed here, and it will appear in a subsequent publication. Further enhancement will be carried out during the next phase of this research work, including extending the corpus size and enhancing the rule based phonetic dictionary by morphologically driven patterns and dialectal pronunciation rules.
ACkNowlEdGmENT
This work was supported by a grant #AT-24-94 from King Abdulaziz City of Science and Technology (KACST). The authors would like also to thank King Fahd University of Petro-leum and Minerals for its support in carrying out this project.
rEFErENCES
Afify, M., Sarikaya, R., Kuo, H. J., Besacier, L., & Gao, Y. (2006). On the Use of Morphological Analysis for Dialectal Arabic Speech Recognition. Interspeech-2006, Pittsburg, PA, USA.
Algamdi, M. (2000). Arabic Phonetics. Riyadh, Saudi Arabia: Attaoobah.
Algamdi, M. (2003). KACST Arabic Phonetics Data-base. Fifteenth International Congress of Phonetics Science, Barcelona (pp. 3109-3112).
Algamdi, M., Almuhtasib, H., & Elshafei, M. (2004). Arabic Phonological Rules. [King Saud University.]. Journal of Computer Sciences and Information, 16, 1–25.
Algamdi, M., Elshafei, M., & Almuhtasib, H. (2002)., Speech Units for Arabic Text-to-speech. Fourth Workshop on Computer and Inforamtion Sciences (pp. 199-212).
Billa, J., Noamany, M., Srivastava, A., Liu, D., Stone, R., Xu, J., et al. (2002). Audio indexing of Arabic broadcast news. IEEE International Confer-ence on Acoustics, Speech, and Signal Processing ICASSP’02, 1, 5-8.
Diehl, F., Gales, M. J. F., Tomalin, M., & Woodland, P. C. (2008). Phonetic pronunciations for Arabic speech-to-text systems. IEEE International Confer-ence on Acoustics, Speech and Signal Processing (pp. 1573-1576).
Elshafei, M., Al-Muhtaseb, H., & Alghamd, M. (2002). Techniques for High Quality Text-to-speech. Information Science, 140(3-4), 255–267. doi:10.1016/S0020-0255(01)00175-X
Elshafei, M., Al-Muhtaseb, H., & Alghamd, M. (2006). Machine Generation of Arabic Diacritical Marks. 2006 International Conference on Machine Learning; Models, Technologies, and Applications (MLMTA’06), USA.
Elshafei, M., Al-Muhtaseb, H., & Alghamd, M. (2006). Statistical Methods for Automatic Diacri-tization of Arabic text. 18th National computer Conference NCC’18, Riyadh.
Elshafei, M., Al-Muhtaseb, H., & Alghamd, M. (2008). Speaker-independent Natural Arabic Lan-guage Speech Recognition System. International Conference on Intelligent Systems (ICIS 2008), Bahrain.
Elshafei, M., Al-Muhtaseb, H., & Alghamd, M. (2008). Development of a Recognition Engine for Natural Arabic Speech. KACST final project report #AT-24-94-2008.
Elshafei-Ahmed, M. (1991). Toward an Arabic Text-to-Speech System. The Arabian Journal of Science and Engineering, 16(4B), 565–583.
English Pronouncing DictionaryC. M. U. (2008). http://www.speech.cs.cmu.edu/cgi-bin/cmudict
GALE project at NIST. (2008). http://www.nist.gov/speech/tests/gale/
Gales, M. J. F., Diehl, F., Raut, C. K., Tomalin, M., Woodland, P. C., & Yu, K. (2007). Development of a phonetic system for large vocabulary Arabic speech recognition. IEEE Workshop on Automatic Speech Recognition & Understanding (pp. 24-29).
Hiyassat, H. A. R. (2007). Automatic Pronunciation Dictionary Toolkit for Arabic Speech Recognition Us-ing SPHINX Engine. Ph.D. thesis, Arab Academy for Banking and Financial Sciences, Amman, Jordan.
Huang, X., Alleva, F., Hon, H. W., Hwang, M. Y., & Rosenfeld, R. (1993). The SPHINX-II speech recognition system: an overview. Computer Speech & Language, 7(2), 137–148. doi:10.1006/csla.1993.1007
Jelinek, F. (1998). Statistical Methods for Speech Recognition. Cambridge: MA: MIT Press.
Kirchhoff, K., Bilmes, J., Das, S., Duta, N., & Egan, M. Gang Ji, Feng He, Henderson, J., Daben Liu, Noamany, M., Schone, P., Schwartz, R., & Vergyri, D. (2003). Novel Approaches to Arabic Speech Recogni-tion: Report from the 2002 John-Hopkins Summer Workshop. International Conference on Acoustics, Speech and Signal Processing, ICASSP’03 (pp. 344-347).
Kirchhoff, K., Vergyri, D., Bilmes, J., Duh, K., & Stolcke, A. (2006). Morphology-based language modeling for conversational Arabic speech recogni-tion. Computer Speech & Language, 20, 589–608. doi:10.1016/j.csl.2005.10.001
Lamere, P., Kwok, P., Walker, W., Gouvea, E., Singh, R., Raj, B., & Wolf, P. (2003). Design of the CMU sphinx-4 decoder. “, In the 8th European Confer-ence on Speech Communication and Technology, EUROSPEECH-2003 (pp. 1181-1184), Geneve, Switzerland.
Lee, K. F. (1988). Large Vocabulary Speaker Indepen-dent Continuous Speech Recognition: The SPHINX System. PhD Thesis, Carnegie Mellon University.
Messaoudi, A., Gauvain, J. L., & Lamel, L. (2006). Arabic Broadcast News Transcription Using a One Million Word Vocalized Vocabulary. International Conference on Acoustics, Speech and Signal Pro-cessing, ICASSP’06.
Noamany, M., Schaaf, T., & Schultz, T. (2007). Ad-vances in the CMU-InterACT Arabic Gale Transcrip-tion System. HLT/NAACL, Rochester, NY, US.
Placeway, P., Chen, S., Eskenazi, M., Jain, U., Parikh, V., Raj, B., et al. (1997). The 1996 HUB-4 Sphinx-3 system. DARPA Speech Recognition Workshop, Chantilly, VA. Morgan Kaufmann Publishers.
Rabiner, L. (1989). A Tutorial on Hidden Markov Models and Selected Applications in Speech Rec-ognition. Proceedings of the IEEE.
Rabiner, L., & Juang, B. H. (1993). Fundamentals of Speech Recognition. Prentice Hall.
Satori, H., Harti, M., & Chenfour, N. (2007). In-troduction to Arabic Speech Recognition Using CMU Sphinx System. International Symposium on Information and Communication Technologies, ICTIS07.
Soltau, H., Saon, G., Povey, D., Mangu, L., Kuo, J., Omar, M., & Zweig, G. (2007). The IBM 2006 Gale Arabic ASR System. IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2007 (pp. 349-352).
Stallard, D., Chia-lin, K., Krstovski, K., Liu, D., Natarajan, P., Prasad, R., et al. (2008). Recent im-provements and performance analysis of ASR and MT in a speech-to-speech translation system. IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2008 (pp. 4973-4976).
The Unicode Consortium. (2006). http://unicode.org/
Xiang, B., Nguyen, K., Nguyen, L., Schwartz, R., & Makhoul, J. (2006). Morphological Decomposition for Arabic Broadcast News Transcription. IEEE International Conference on Acoustics, Speech and Signal Processing.
Young, S. (1996). Large vocabulary continuous rec-ognition [review]. IEEE Signal Processing Magazine, 13(5), 45–57. doi:10.1109/79.536824
Zhou, B., Gao, Y., Sorensen, J., Dechelotte, D., & Picheny, M. (2003). A hand-held speech-to-speech translation system. IEEE Workshop on Automatic Speech Recognition and Understanding (pp. 664-669).
80 Journal of Information Technology Research, 2(4), 67-80, October-December 2009
Mohamed Ali received his BSc in 2005 in computer science from King Fahd University of Petro-leum and Minerals, Saudi Arabia, with GPA of 3.92 on 4.0 scale. He has diverse programming skills in Java, C#, Php, C++, o Java SE and EE using NetBeans, LINQ using Visual Studio.net, Scripting Languages: Python, Perl, Photoshop scripting, Microsoft Power Shell and Shell scripting. He participated in a number of research projects e.g., a Speech recognition system based on Gaussian Neural Networks, an Online Handwriting Recognition system using support vector machines, and a study of protein sequence analysis in Bioinformatics. Ali is doing his MSc in computer science at King Fahd University of Petroleum and Minerals, Saudi Arabia.
Elshafei received his PhD (with Dean List) from McGill University, Canada, in electrical engineer-ing in 1982. Since then, he has accumulated a unique blend of 9 years of industrial experience and over 17 years of academic experience. He is co-inventor/sole inventor of several US patents and international patents. He has over 120 publications in international journals, conferences, and technical reports. He was the PI/CI of many funded projects exceeding 4 million SR, and he was also involved in many internally funded or industry funded projects. His research interest includes Arabic speech processing, digital signal processing, and Intelligent Instrumentation. Elshafei is a member of IEEE, ISA, and SPE.
Al-ghamdi received his PhD degree in 1990 in experimental phonetics, from Reading University, UK. Since then, he held several positions, the latest is the Deputy Director of Computer and Electronics Research Institute at KACST since 2004. Alghamdi published more than 30 scientific books and papers related to speech and Arabic language, and was the principle investigator of 10 research projects, and participated in many research and studies.
Al-ghamdi research interest includes Phonetics, Phonology, Linguistics, Speech Processing (Text-to-speech, Speech-to-text, Speaker verification, Voiceprint) and Speech Therapy.
Husni Al-Muhtaseb received his MSc in computer science and engineering from KFUPM in 1988 and the B.E. in electrical engineering, computer option, from Yarmouk University, Jordan in 1984.
He is currently an instructor of computer science at KFUPM. He worked as a technical consultant for the dean of admissions and registration for 10 years. His research interests include software development, Arabic Computing, computer Arabization, Arabic OCR, e-learning & online tu-toring and natural Arabic understanding. Al-Muhtaseb has participated in several industrial projects and worked as a consultant with different institutes/ organizations. Al-Muhtaseb has more than 50 research publications.
Atef received the BSc in electrical engineering from King Saud University with First Honor, and was ranked 1st on the School of Engineering. He obtained his M.Sc. Degree in Electrical Engineering from King Fahd University of Petroleum & Minerals with First Honor (4.00 GPA). He then obtained the PhD in computer engineering from Purdue University. Al–Najjar was a Member, Eta Kappa Nu (HKN) Honor Society. He taught and participated in many projects in the areas of signal processing, multi-media, encryption, and artificial intelligence.
