Accessing Festival

The instructions assume you are using the installation of Festival on the computers in the PPLS Appleton Tower (AT) labs. You can work on the computers in the actual physical lab or you can use the the remote desktop service (see Module 0 for instructions).

Note: The PPLS AT Lab computers/remote desktop we are using for this course are completely separate from Informatics remote desktop/DICE!   Importantly, you won’t have access to the voice setup we will be using on DICE. 

It is possible to install Festival directly onto your computer.  However, this will require a unix like environment (Linux or MacOS, or a Linux style terminal runnning on Windows) and require you to compile some code (see guidance here: Install Festival). If you’ve never compiled code before, and don’t have much experience with the unix command line, your best bet is to use the PPLS AT Lab computers.

Accessing Festival Remotely

You can use the installation of Festival on the Appleton Tower lab servers using the remote desktop service. To connect using the remote desktop, follow the instructions here: Module 0 – computing requirements

Once you’ve started the remote desktop and logged in (with your UUN and EASE password), you can open the Terminal app by going to the menu bar at the top and clicking Applications > System Tools > Terminal. You may want to drag the Terminal icon to the desktop to make it easier for you to find it.

If you accidentally close VNC before you log out, you can reconnect by double clicking on the machine you previously logged onto in VNC viewer.

When you are finished, remember to log out: in the menu at the top of the screen > System > Log out

Assignment Data

If you are using the remote desktop to access the AT lab computers (or are physically in the lab), all the relevant data is already there for you on the linux machines.

If you have installed Festival on your own computer, you will need to get the voice database and dictionaries used to run the voice (voice:cstr_edi_awb_arctic_multisyn, dictionary:unilex). You can find instructions by following this link.

Start Festival

Festival has a command line interface runs through the terminal (i.e. the unix bash shell).  To do this in the PPLS AT lab, you’ll need to:

1. Make sure the computer is booted into Linux (if it is in windows restart the machine and select the penguin (the Linux mascot!) when presented with the choice);
2. open a terminal via Applications > System Tools > Terminal from the menu bar at the top left of the screen.   You can drag the Terminal icon from the menu to the desktop if you want to make a shortcut.

So, now, open a Terminal and run Festival by typing in festival at the prompt $:

$ festival

You should see some text about the version of Festival we are using (Festival 2.5.0):

Festival Speech Synthesis System 2.5.0:release December 2017
Copyright (C) University of Edinburgh, 1996-2010. All rights reserved.

..etc

and the prompt will also change to show the following:

festival>

This new prompt means that Festival is running; any commands that you type will now be interpreted by Festival rather than by the shell.

You will be pleased to know that Festival’s command-line interface is the same as the bash shell (e.g., TAB completion, ctrl-a, ctrl-e, ctrl-p, ctrl-n, up/down/left/right cursor keys, etc.). Here’s a nice cheat sheet for common bash commands.  For a comprehensive list of these shortcuts, see the Wikipedia entry for GNU Readline.

If you get into trouble at any point and need to exit the current command, use ctrl-c. This applies to both Festival and the bash shell.

It’s really worth learning these keyboard shortcuts because they also apply to the bash shell and will save you a lot of time.

Make Festival speak

Synthesise some sentences to become familiar with to the Festival command line.

Festival contains a number of different synthesis engines and for each of these, several voices are available: the quality of synthesis is highly dependent on the particular engine and voice that is being used.

Using the SayText command

By default, Festival will start with a rather old diphone voice, which does not sound great, but is fast and intelligble:

festival> (set! myutt (SayText "Welcome to Festival"))

This command combines a bunch of different things:  It converts the input text “Welcome to Festival” to a linguistic specification and uses that specification to generate speech by selecting appropriate diphones.  The set! myutt part of the comment tells Festival to store all the information relating to how the utterance was synthesized in the variable called myutt.

You can set the voice to the one we will use in the assignment by typing the following after starting festival:

festival> (voice_cstr_edi_awb_arctic_multisyn)

You’ll see some “EST warning” messages printed to the screen, but you can ignore those.

Now try getting generating a sentence as you did before with the SayText command.  Can you hear a difference between the two voices?

Generating speech without playing it

To generate an utterance without playing it, use the following steps instead of SayText:

festival> (set! myutt (Utterance Text "Hello"))
festival> (utt.synth myutt)


Then you can save the utterance myutt a wave file as “myutt.wav” with the following command:

festival> (utt.save.wave myutt "myutt.wav" 'riff)


This will save a file called myutt.wav in whatever directory you started festival in.  If you just opened a terminal and started festival without changing directories, you will be in your ‘home’ directory.  You can check the folder on the desktop called [your username]’s home and see if a new file wav file has appeared there.   Otherwise you can exit festival by pressing ctrl-d  and typing the command pwd: this will tell you where you currently are – your “present working directory”.

Note: When you issue a command to Festival you must put it in round brackets (...) – if you do not, it will generate an error. You are using a language called Scheme.

Scheme, and lots of brackets

Scheme is a LISP-like language used as the interface to Festival. When you run Festival in interactive mode, you talk to Festival in Scheme. Fortunately, we’re not going to have to learn too much Scheme. All you need to know for now is that the basic syntax is (function_name argument1 argument2 ...).

In Scheme, all functions return a value, which by default is printed after the function completes. The SayText function returns an Utterance structure, so that is why # is printed after the completion of the function. A variable (myutt in this case) can be set to capture this return value, which will allow us to examine the utterance after processing. This is done using the set! command (note the two sets of brackets):

festival> (set! myutt (SayText "Welcome to Festival"))
#

The TTS process

We can now examine the contents of the myutt variable. The SayText function is a high level function which calls a number of other functions in a chain. Each of these functions performs a specific job, such as deciding the pronunciation of a word, or how long a phone should be. We’ll be running these step-by-step later on.

The TTS process in Festival is a pipeline of sub-processes, which build up an Utterance structure in stages. This building process takes the original text as input and adds more and more information, which is stored in the utterance structure. In Festival, a unified mechanism for representing all types of data needed by the system has been developed: this is called the Heterogeneous Relation Graph system, or HRG for short.

Each Relation in an HRG is a structure that links items of a particular linguistic type. For example, we have a Word relation which is a list linking all the words, and a Segment relation which links all the phones etc. Relations can take different forms: the most common types are linear lists and trees.

Each module in Festival takes a number of relations as input and either creates new relations as output, or modifies the input ones. The vast majority of modules only write new information, leaving all information in the input untouched (there are a few exceptions, such as post-lexical processing). Because of this, examining the contents of the relations in an utterance after processing gives an insight into the history of the TTS process.

Different configurations of Festival can use vary with respect to their use of HRGs, and which modules they call.

Examining a saved object

Once you have synthesised an utterance you can do lots of things with it. Here are a few examples.

festival> (utt.play myutt)
festival> (utt.relationnames myutt)
festival> (utt.relation.print myutt 'Word)
festival> (utt.relation.print myutt 'Segment)

You can get a list of the relations that are present in a synthesised utterance by using the utt.relationnames command.

Relations that are lists can easily be printed to the screen with the utt.relation.print command. Try this with all of the relations in an utterance. Some of them won’t reveal useful information, others will.

The output from (utt.relation.print myutt 'Word) may look like this:

()
id _3 ; name hello ; pos_index 16 ; pos_index_score 0 ; pos uh ;
        phr_pos uh ; phrase_score -13.43 ; pbreak_index 1 ;
        pbreak_index_score 0 ; pbreak NB ;
id _4 ; name world ; pos_index 8 ; pos_index_score 0 ; pos nn ;
        phr_pos n ; pbreak_index 0 ; pbreak_index_score 0 ;
        pbreak B ; blevel 3 ;
nil

Each data line starts with an id number like id _3 then a series of features follow separated by semicolons. Each feature has a name and a value, e.g., feature name: pos, feature value: uh.

Examining the processing steps

Tokens – First the text is split into Tokens. Look at the Token relation, where an item is created for each component of the text you input. The Token relation will still have digits and abbreviations in it.

Words – The Tokens are then converted to Words, abbreviations and digits are processed and expanded. Look for this in the Word relation.

Part of Speech Tagging – Each word is tagged with its part of speech, which is added as a feature to the Word relation.

Pronunciation – The pronunciation of each word is determined and the Syllable and Segment relations created. Examine these: the syllable relation is not very interesting as there is very little information here, just a count of the syllables.

You can look up the pronunciation of a word yourself with the function lex.lookup

festival> (lex.lookup "caterpillar")
("caterpillar" nil (((k ae t) 1) ((ax p) 0) ((ih l) 1) ((er) 0)))

The actual pronunciation returned depends on which lexicon a particular voice uses, and whether the word is in the lexicon or if Festival has to predict the pronunciation using letter-to-sound rules.

Try looking up the pronunciation of some real words, and some made up ones.

Accent Prediction – An intonation module assigns pitch accents (and other intonational events) to syllables. A number of different modules exist within Festival, operating with a number of intonation models including Tobi and Tilt. The assignment voice (awb) doesn’t actually do accent prediction, but you can see what this would look like by trying the older diphone synthesis voice, kal, which does:

To switch to the kal voice, enter the following in festival:

festival> (voice_kal_diphone)

Now, look at the IntEvent relation to see which pitch events have been assigned. From the pitch events and the predicted durations, a pitch contour is generated. This contour is a list of numbers which specify the pitch at each point for the resulting waveform. There is no easy way to view the pitch contour.

You can use the following command to change back to the assignment voice:

festival> (voice_cstr_edi_awb_arctic_multisyn)

Waveform generation – The Unit relation is created by making a list of diphones from the segments and the information about the speech needed for synthesis is copied in. The Unit relation contains features with values in square brackets [...] These are references to the actual speech used to synthesise these units.

Quit Festival

festival> (quit)

or use ctrl-D, just like in the bash shell. Festival remembers your command history inbetween sessions (again, just like bash). Next time you start Festival you can use the up cursor key to find previous command, and then hit ‘Enter’ to execute them again. Of course, Festival does not remember the values of variables (e.g., myutt in the above example) between sessions.

Transferring data from the AT lab servers

To get your data (e.g. generated wav files) from the AT lab servers (e.g. remote desktop) you can either use a terminal based command like rsync or an SFTP client like FileZilla or WinSCP (graphical interface). For example, the following copies the file myutt.wav that’s in ~/Documents/sp/assignment1 to the directory where you’re running the rsync command from on your own computer:

rsync -avz s1234567@scp1.ppls.ed.ac.uk:Documents/sp/assignment1/myutt.wav ./

Note [24/10/23]: The file transfer server scp1.ppls.ed.ac.uk doesn’t appear to be working right now.  We are investigating, but for now you can still transfer files by replacing scp1.ppls.ed.ac.uk with one of the remote desktop addresses, e.g. ppls-atl-1079.ppls.ed.ac.uk (and similarly for references to scp1.ppls.ed.ac.uk below).  You’ll need to have the university VPN on to do this. You can see the list of PPLS AT lab remote desktop addresses here: https://resource.ppls.ed.ac.uk/whoson/atlab.php

Note: The previous command will only work if you’ve already made the directories Documents/sp/assignment1 in your home directory on the AT lab servers. If you haven’t done this, you can skip this for now and try it after you’ve created some files.

You can share your files with yourself by copying them to OneDrive (or Google Drive) via a browser.

You can also use a file transfer app like FileZilla. In this case, you need to set the remote host to scp1.ppls.ed.ac.uk. For FileZilla, go to File > Site Manager, then set the protocol to SFTP, the host as scp1.ppls.ed.ac.uk, and use your UUN as username and EASE password as the password. After connecting you should see your home directory on the AT lab servers as the remote site. You can then drag files from remote site side to the appropriate place in the local site side.

What you should now be able to do

• start Festival and make it speak using SayText
• capture the Utterance structure returned by SayText
• look inside the Utterance structure at the Relations
• have an initial understanding of what Relations are, but not yet the full picture
• use some of the keyboard shortcuts that are common to Festival and the bash shell
• save a synthesized utterance as a .wav file and transfer it to your own computer.

We are going to examine the speech synthesis process in detail. You will examine the sequence of processes Festival uses to perform text-to-speech (TTS) and relate these processes to what you have learnt in the Speech Processing course. Make notes in a lab book as you work! Remember that Festival is just one example of a TTS system – there are generally many other ways of implementing each step.

Festival stores information like words, phonemes, F0 targets, syntax, etc. in things called Relations. These approximately correspond to the levels of linguistic representation mentioned in lectures. As each stage in the process is performed, more information is added to the utterance structure, and you can examine this inside Festival. So, each step in the pipeline will either add a new relation, or modify an existing one. Note, however, not all TTS voice configurations generate and use all types of relations (e.g., the awb voice is definitly missing some that the kal voice has).

Your task in this part of the exercise is to explore the synthesis process and discover what Festival does in each step of processing, when converting text to speech. If you notice errors in the prediction of phrases, pronunciation, the processing of numbers or anything else, make a note of it, as this will be useful for the next part of the exercise.

Hints

Three hints for this practical exercise:

1. Read the instructions through completely before you start.
2. Use Festival’s tab-completion and command-line history (which is kept even if you quit and restart Festival) to save typing and avoid mistakes.
3. If things go wrong (either with Festival, or with you), quit Festival and restart it.

Festival help

Festival can make your life easier in a number of ways.

Command history

You can access commands you have previously typed using the arrow keys. Press the up arrow a number of times to see the previous commands you entered, then use the left and right arrow keys to edit them. Press ENTER to run the edited command.

TAB completion

If you start to type a command name into Festival, then press TAB, it will either complete the command for you or give you a list of possible completions. For example, to get a list of all of the commands that work on an utterance type

festival> (utt.

and then press TAB once or twice.

Getting help

Most commands in Festival have built-in help. Type the name of the command (including the initial open bracket) and the press ⟨ALT⟩-h (Hold the ALT key down and press h), or alternatively, press (and release) ESC and then press h. Festival will print some help for that command, including a list of arguments that it expects.

Make an assignment working directory

Before you start, make a folder to work in, so you can keep your data organised.  On the PPLS AT lab computers/remote desktop, you can make a directory in ~/Documents/sp/assignment1. We will assume this is your working directory in the rest of these instructions.  You can do this on the command line (i.e., in a terminal) with the following commands.

cd ~
mkdir -p ~/Documents/sp/assignment1

Change into this directory, e.g.:

cd ~/Documents/sp/assignment1

You can use the pwd command to see which directory you are currently in and the ls command to see which files are in the current directory.

Starting Festival with a specific voice configuration

Previously, you started Festival in the default mode.  But for this assignment we want to use a specific voice in a specific configuration.

We are going to use a unit selection voice, called cstr_edi_awb_arctic_multisyn in this exercise.

We’re going to use a simple configuration file to tell Festival to load the correct voice, and to add a few extra commands (file location of the AT lab servers):

cp /Volumes/Network/courses/sp/assignment1/config.scm .
chmod ugo+r config.scm

If you followed this gist to install festival on your own computer, you probably already downloaded the config.scm file (by default the “assignment1” directory next to the “tools” directory you installed festival (see lines 82-85). In that case, you can start festival from there or move the config.scm file to the directory you want to work in.

Remember that if you come back later, you only need to cd to your working direcotry (e.g. ~/Documents/sp/assignment1 following the remote desktop instructions). You don’t need to copy the file again as long as you start festival from a directory that contains config.scm. Now, every time you start Festival during the rest of this exercise, do it like this.

festival config.scm

Compared to the earlier exercises using a diphone voice, Festival will take longer to start when loading this unit selection voice. Why?

In the following festival> at the beginning of the line, just recreates the festival command line prompt – you just need to type in the bit in parentheses.

Once Festival is running, check that it speaks:

festival> (SayText "hello world")

You should hear a reasonably good quality Scottish male voice. If not, you probably forgot to start festival using the config.scm file.

Synthesising an utterance step-by-step

Read this section carefully before trying any of the commands.

So far we have only synthesised utterances from start to finish in one go, using SayText. Now we are going to do it step-by-step.
First you need to create a new utterance object. The following command creates a new utterance object with the text supplied and sets the variable myutt to point to it. In the following festival> at the beginning of the line, just recreates the festival command line prompt – you just need to type in the bit in parentheses.

festival> (set! myutt (Utterance Text "Put your own text here."))

Now you can manually run each step of the text-to-speech pipeline – don’t skip any steps (what would happen if you did?). Use a single short utterance of your own when performing this part – make it interesting (e.g., containing some text that needs normalising, a word that is unlikely to be in the dictionary, and so on).

festival> (Initialize myutt)
festival> (Text myutt)
festival> (Token_POS myutt)
festival> (Token myutt)
festival> (POS myutt)
festival> (Phrasify myutt)
festival> (Word myutt)
festival> (Pauses myutt)
festival> (PostLex myutt)
festival> (Wave_Synth myutt)
festival> (utt.play myutt)

If you get an error, you will have to start again by creating a new utterance with the set! command. If you get confused, quit Festival and start from the beginning again.

Note that running the synthesis pipeline step-by-step is just to help you understand what is happening. You might need it to diagnose some mistakes later on, but most of the time, you will just use SayText.

Commands for examining utterances

You should pause to examine the contents of myutt between each step.

To determine which relations are now present:

festival> (utt.relationnames myutt)

and to examine a particular Relation (if it exists):

festival> (utt.relation.print myutt 'Phrase)
festival> (utt.relation.print myutt 'Word)
festival> (utt.relation.print myutt 'Segment)

and so on for any other Relations that exist.

You can also use the following command to see the overall structure of the utterance:

festival> (print_sylstructure myutt)

This will show you how the different relations tie together. It will show you Words, Syllables as lists of segments, and the presence of stress.

Concentrate on discovering which commands create or modify Relations in the utterance structure, and what information is stored in those Relations. Note: the initialize command will not reveal anything interesting, and it may be difficult to see what the Pauses and PostLex commands do.

What you should now be able to do

• start Festival and load a configuration file (which is just a sequence of Scheme commands to run after startup)
• Make full use of keyboard shortcuts including: TAB completion, ctrl-A, ctrl-E, ctrl-P, ctrl-N, ctrl-R, up/down cursor keys to navigate the command history, left/right cursor keys to edit a command.
• run the pipeline step-by-step
• describe which Relations are added, or modified, by each step
• understand that Relations are composed of Items
• understand that Items are data structures containing an unordered set of key-value pairs
• have an initial understanding of what some (but not all) of the keys and values mean (e.g., POS tags in the Word relation)

Starting Festival

Just a reminder that every time you start Festival during this exercise, make sure you use the right voice config file so that you’re finding errors in the right voice (awb).  Remember to change to the directory where you placed the config.scm file:

$ festival config.scm

Saving waveforms from Festival

You’ll want to save at least some of your synthesised utterances for further analysis, e.g. in Praat.  Since you have a fully synthesised utterance object in Festival, it is possible to extract the waveform to a file as follows:

festival>(utt.save.wave myutt "myutt.wav" 'riff)

myutt should be the name of the utterance object, myutt.wav is the filename, which you can choose; if you save more than one waveform, then give them different names. You can now view and analyse the waveform in Praat or Wavesurfer.

Explaining mistakes made by Festival

Using what you have learned about Festival, you can now find some examples of it making errors for English text-to-speech. Find examples in each of the following categories:

• Text normalisation
• POS tagging/homographs
• Phrase break prediction
• Pronunciation (dictionary or letter-to-sound)
• Waveform generation

Aim for a variety of different types of errors, with different underlying causes: 1 error for each of the front-end categories, (see the assignment overview). Don’t report lots of errors of the same type.

Be sure that you understand the differences between these various types of error. For example, when Festival says a word incorrectly, it might not be a problem with the pronunciation components (dictionary + letter-to-sound) – it could be a problem earlier or later in the pipeline. You need to play detective and be precise about the underlying cause of every error you report.

You might discover errors in other categories too. That’s fine: you may wish to incorporate some discussion of this in the last section of your report (discussing the TTS pipeline as a whole) .

Use the SayText command to synthesise text that you think will produce errors given what you’ve already found out about the awb voice and the TTS pipeline it uses.   If you’re stuck you might try generating sentences from an external text for inspiration (e.g., from a news website, or a novel).

For each utterance you analyse, store the results in a variable. You will need to examine the contents of this utterance structure to decide what type each error is.

You will often be able to identify the source of the error from the relations generated by SayText directly (i.e. without running the pipeline step-by-step).  However, in some cases (but not all – it will be too slow), you may also need to run Festival step-by-step (as in the previous part of the exercise).  The crucial thing for the write-up is that you provide evidence of where and why the error occured.

Skills to develop in this assignment

• use SayText to synthesise lots of different sentences
• precisely pinpoint audible errors in the output (e.g., which word, syllable or phone)
• understand that errors made by later steps in the pipeline might be caused by erroneous input; in other words, the mistake actually happened earlier in the pipeline
• understand that mistakes can happen in both the front end and the waveform generator
• make a hypothesis about the cause of the mistake
• trace back through the pipeline to find the earliest step at which something went wrong
• obtain evidence to test your hypothesis, by inspecting the Utterance and/or the waveform and/or the spectrogram