[Speech Technology] What is Vector Quantization?

The size of training data should be large enough for reliable values to be driven for all

the parameters. However, the larger the number of feature vectors is, the more possible

values for each feature are. Not only is this memory-inefficient, but also this is problematic

because many feature vectors will not occur at all in the training data. One solution for these

problems is using Vector Quantization (VQ).

VQ is a data compression technique. It does not deal with all the feature vectors, but only

some centroids of them, which can be obtained through Euclidean distance.

As a simple example, if we want to represent 0 to 7 in one dimension. 3 bits (by 23 = 8)

are needed to do so. However, if we apply VQ to this, only 2 bits (by 22 = 4) are needed

(4 centroids in 4 clusters: 1 in 0 to 1.99; 2 in 2 to 3.99; 5 in 4 to 5.99; and 7 in 6 to 7.99).

So, if “41371512” is the target, 24 bits (by 3 * 8) are needed without VQ while 16 bits (by 2 * 8)

with VQ. However, they are represented in different ways: VQ version is

with centroids “51271512” while non-VQ version is as it is. VQ also can be applied

to more than one dimensions. For example, if there are 16 dots in a two-dimensional space

(plane), 4 bits (by 24 =16) per data value are needed, but if 16 dots are represented

with 4 centroids, only 2 bits (by 22 = 4) per data value are needed.

To be more specific on how this works, let’s take another example.

If there are 4,096 dots in a plane, 12 bits (by 212 = 4,096) per data value are needed,

but if these are represented with 16 centroids through such clustering techniques as k-means,

only 4 bits (24 = 16) per data value are needed. All these centroid values (e.g., (2, 5)) are to be

saved as vector values and will be assigned vector numbers

(i.e., 0000, 0001, 0010, … 1111 in this case). These values are further to be saved

in the codebook, with vector values as codebook values and vector numbers as codebook entries.

If a codebook is established with a training data set as such, a sequence of acoustic data

in the form of vectors can be represented with codebook entries. For instance, if (2, 5) is saved

as entry 3 in the codebook and one of the vectors in the new acoustic data is (2, 4.9), it is going

to be represented as the entry number of 3.

In short, VQ is a data compression method, by which only some representative vectors of clusters

are dealt with. As the number of feature vectors is increasing, the effect of VQ is also increasing.

[Phonetics, Speech Technology] What are Fundamental Frequency, Harmonics, and Formants?

The human vocal tract is a kind of resonator, with one fixed-end (i.e., vocal folds) and the other open-end (i.e., lips),

which is acting as a filter in terms of source-filter theory. Regarding sound source, there are largely two types of it:

glottal source and supra-glottal source. Glottal source can be periodic (i.e., voiced), aperiodic (i.e., whisper),

or mixed (i.e., breathy) while supra-glottal source is almost always aperiodic (i.e., noise).

Of the two sources, glottal source, in particular vocal folds, is pertinent to the fundamental frequency, harmonics, and formants,

because they are essentially periodic.

Among them, the fundamental frequency and harmonics are properties of the source,

exactly vocal folds while formants are those of the filter, exactly the vocal tract.

First of all, a complex wave, according to Fourier theorem, consists of simple waves, and

each simple wave is referred to as harmonic. The first harmonic is referred to as t

he fundamental frequency, which is determined by the number of the vocal folds' vibration

in one second, and the following harmonics, also called overtones, are multiple integers of it.

For instance, if the fundamental frequency, the first harmonic, is 100 Hz, the second harmonic

should be 200 Hz, the third harmonic should be 300 Hz, and so on.

When those harmonics are filtered through the vocal tract, some are going to be dampen out

while others are going to be amplified by the length and shape of the vocal tract. The harmonics

filtered through the vocal tract are called formant frequencies, essentially resonant frequencies.

The first three formant frequencies, each called the first formant (F1), the second formant (F2),

and the third formant (F3), are widely used by the hearer to categorize speech sounds,

in particular vowels.

In short, overtones are multiple integers of the fundamental frequency at the vocal cord,

harmonics are embracing the fundamental frequency and overtones, and formants are

resonant frequencies, viz. harmonics filtered through the vocal tract.

Lastly, formants are determined by the length and shape of the vocal tract and can be

calculated by Fn = (2n-1)*c/4L. 'n' represents the number of the resonance,

'c' speed of sound (mostly in air), which is around 35,000 cm/sec, and 'L' the length

of the resonator, here the length of the vocal tract. If one's vocal tract is 17.5cm long and

one is going to produce a schwa, which does not much affect the shape of the vocal tract,

the first three formants can be calculated as in the following table.

The number of the resonance | Formant frequencies

:----------:|:---------------------

1 (F1)|35,000cm/sec / 17.5cm*4 = 500 Hz

2 (F2)|35,000cm/sec / 17.5cm*4/3 = 1,500 Hz

3 (F3)|35,000cm/sec / 17.5cm*4/5 = 2,500 Hz

[Paper Review - Psycholinguistics] Supporting Information (Bergelson & Aslin, 2017)

 Supporting Information

Bergelson & Aslin, 2017

• - 0;6 looked significantly more at named target images when the competitor images were semantically unrelated (e.g., milk and foot) than when they are related (e.g., milk and juice), just as older learners do. 
• - Home-lab links: common noun "copresence" correlated with in-lab comprehension. -> cross-word relations are formed early and the home learning environment measurably helps shape the lexicon from the outset. 
•  

1. 1. Introduction 

• It is relatively easy to examine infants' phonetic development 

1. • 0;6 infants have begun understanding nouns, suggesting they form word-referent links 
   • 0;6 sensitivity to vowels / 0;12 to consonants  

• Motivation: "While toddlers are sensitive to visual similarity, shape, and semantic category, little is known about nascent semantic representations." 
• Some literature 

1. • Arias-Trejo & Plunkett: visual similarity & category membership contribute to semantic competition 
   • Bergelson & Aslin: 1;0-1;8 infants' semantic specificity increases (e.g., 1;0 infants' labeling a cookie as either "cookie" or "banana" while the older ones do less)  
   • Adults consider semantic and perceptual relations among words in both the visual world paradigm and lexical decision tasks. 

• Research question: knowledge of how words are related goes hand in hand with knowledge of what words mean; is this true for infants' earliest words, or whether initial words are more like "islands," unrelated to other emerging lexical entries? 
• Method: gathering real-time processing data through in-lab eye tracking & home recordings 

1. • Question 1: Does semantic relatedness between visually available referents influence word comprehension in novice word learners? -> presented infants with image pairs that were either semantically related or unrelated (e.g., car-stroller or car-juice); one image was named aloud (e.g., car)
   • Question 2: Do readily measurable aspects of infants' home life account for those same infants' variability in word comprehension? -> gathered daylong audio and hour-long video recordings from infants in their homes and examined those time slices when concrete nouns were directed to the infant 
   • Referential transparency: infants learn nonnouns later than common nouns /. Referential transparency boosts learnability? 

• Hypothesis: if young infants are influenced by semantic relatedness, we predict better performance in the unrelated trials. 

2. Results

• Eye-tracking results 
• Measurement: where the child was looking for each 20-ms bin during the test trials (i.e., the target or distractor interest areas, or neither) 
• Two-time windows: a pretarget baseline from trial start to target word onset & a posttarget window from 367 ms to trial end 
• The standard baseline-corrected target looking metric: calculate target / (target+distractor) in the posttarget window and subtracts this same proportion from the baseline window  

• Two-tailed t-test: performance was significantly above chance on unrelated trials  

  -> Infants looked more at the labeled image on unrelated trials than on related trials. 

• Home-recording results

• Annotation: utterance type (syntactic & prosodic features into 7 categories), object copresence (yes, no, & unclear), & speaker 

  -> Although infants heard more object word input in the audio recordings in an absolute sense, they heard relatively more input in the videos.  

3. Discussion

• 0;6 infants understand words more readily when shown two semantically unrelated referents than when shown two related ones. 
• In-lab word comprehension is linked with referential transparency in the home, but not with measures of talker or utterance-type, and only marginally with input quantity. 
• [eye-tracking] even first words are not unconnected islands of meanings; they already contain semantic structure 
• Infants may know enough about a word's meaning to tell it apart from the unrelated referent but not the related one. 
• Object copresence significantly correlated with in-lab comprehension. 
• Referential transparency for a given infant may map onto that same child's comprehension, within the already referentially transparent noun class.