Project Complete

After a year, I have completed my Honours Project. Today I recorded my original score, functioning within Fmod Studio, then edited that with the original gameplay video I chose and exported it.

Fmod Studio running with my score:

https://vimeo.com/album/2764789/video/94858443

With captions:

Without captions:

https://vimeo.com/album/2764789/video/94887339

The Video

The video depicts my original score for the piece of gameplay, which I also recorded, in Tomb Raider (2013) working within Fmod Studio so that it’s a piece of dynamic game audio. The Tomb Raider footage is to make you pretend that it’s working with an actual game, rather than a recording, like the gameplay actually is.

The method of creating dynamic game audio that I used is called ‘Horizontal Re-Sequencing’ and was my preferred method as it’s one of the easiest methods to create game audio and one of the ones I found the most research and writing on.

Horizontal Re-sequencing involves a piece of music being split into smaller sections, or ‘cues’, that can be played in middleware, such as Fmod Studio, to then be looped and marked with makers and transition regions so that it can seamlessly play along with a piece of gameplay. (Phillips, 2014.) Usually, the music will have to be composed in the same time signature (in this instance it was 120 beats per minute) and the same key. This is because, if a cue was moving to another, it would sound jarring and sudden if it wasn’t within the same key, which would prevent people feeling the flow of immersion from the gameplay, which is one of the main points of game audio  being dynamic or interactive.

Also, by composing your music within the same BPM, it’s easier to edit it within Fmod Studio as Fmod parameters can be set by beats per bar upon the ruler on the ‘timeline’. If it wasn’t within the same time signature, that would make the editing process extremely difficult. This method was also used  when scoring Red Dead Redemption Composed by by Bill Elm and Woody Jackson, it was written entirely in the scale of A minor and in 130 beats per minute.

My Project

The aim of my project has changed drastically from first semester. Originally, it was to:  critically analyze the use dynamic audio in modern action/adventure games in order to develop a framework of creating dynamic audio that can be implemented to a piece of gameplay. This project would involve three main objectives in order to be achieved; the selection of action/adventure game titles to be played and studied in order to see the ways in which the composers approach and create their own dynamic audio. The studying of these games would involve transcribing the music and relating that to the triggers and in-game events that occur with it. 

This was when I assumed that I’d easily grasp how to work Fmod Studio, however, with my purely creative background, I found it a much more difficult task and by the end, I was only able to learn the absolute basics. However, I personally feel that’s still a great accomplishment, especially when I think of my background being mainly in the study of singing.

My aim of my project then became: to critically analyze the use dynamic audio in games in order to develop dynamic audio that can be implemented to a piece of gameplay. This project would involve two main objectives in order to be achieved; the study and analysis of existing theories and written works on dynamic audio and by interviewing game composers who are currently creating it.
The videos above are the result of reading written works on game audio and dynamic game audio, by talking to and interviewing other composers and by researching how to use middleware, such as Fmod Studio.

Concluding Thoughts

 

Despite my project not having the same outcome as I had imagined, I’m still happy and proud of the work I’ve done. While at Abertay, I felt like I wouldn’t succeed at all as most of the other students were familiar with middleware like Fmod and how to use it despite not being composers. I felt like I was completely inexperienced and that I shouldn’t be doing an honours degree at all.

I always felt that my composing skills weren’t up to scratch, and that anything I did compose I really did not want to show at a graduate showcase for fellow students and those in the industry to see. However, upon just putting myself out there and acknowledging that yes, there are gaps in my knowledge, yes, I still have a long way to go and yes, I find learning new software to do with music an extremely difficult and daunting process, I just went for it.

Originally, I wanted to compose for an actual piece of gameplay from an actual game. But with having to commute from Edinburgh and while having a part time job and my deteriorating health in the last year, I knew that wasn’t possible. Then, I wanted to compose to 3 pieces of gameplay from different games, Tomb Raider, Uncharted and The Last of Us. However, this was when I was under the assumption that I’d get to grips with Fmod in semester one, but that didn’t pan out either. Instead I composed to one piece of gameplay from Tomb Raider, and only understood the basics of one method of creating dynamic audio within it.

Though that may not seem like an accomplishment to others, it’s one to me. Upon first downloading Fmod, I never thought I’d understand it at all. But now, I could again compose original music and implement it into Fmod. Infact, after university, I want to continue my Fmod studies as I know that being able to advertise myself as a composer who can work middleware will make me more hirable for game projects in the future, also, as I love game audio so much, I love to learn everything I can about it.

By running my own original score in Fmod Studio and creating a piece of dynamic audio, I believe I have shown a good approach to  research and development. I talked to and learned from those currently within the industry as I was determined to understand what’s really important about game audio and that the fact that it can be dynamic.

I have gone from being ashamed to show my work at the graduate showcase to being proud, and the fact that I want to carry on learning about Fmod, is all I could of asked for as a result of all the years work as it’ll help me becoming the best game composer that I can be.

Advertisements

Fmod: QTE Problems

Not all game composers know how to use middleware such as Fmod studio, Wwise or more, however there are clear benefits from being able to use them, or by being able to come to terms with just he grasps of how it’s used to implement music to a game, and that is: being able to have your original music be properly prepared.

When creating a piece of original music that will eventually become non-linear, your score needs to be cut into smaller sections that can be looped, etc, depending on the players decisions or the game states. If you are aware of how to use, for example, Fmod Studio, you will have that in mind during composing.

When creating my score for the Tomb Raider (2013) gameplay, I did have this in mind, however, as I wasn’t yet to grips with the basic functions of Fmod Studio,  and so I didn’t properly prepare my cues.

This was especially the case when I created the cues for the quick time event (QTE) section of the score. Originally, for the QTE section, I had 3 cues. The first was the first QTE section of the gameplay scored at its length until you can die. The second contained the score if you succeeded once, but then died. And the 3rd contained the score if you succeed the entire sequence.

How I scored and cut those cues defeats the entire purpose of my project. As by then importing them into Fmod, I was just playing the cues in order, to make up the score. That’s not dynamic.

Here are those original cues I composed:

QTE1.

QTE2.

QTE3.
From discussion with my project supervisor and from experience of trying to then get Fmod to work, I realised I had to go back into Logic Pro 9 and amend the score for the QTE sections. For this, I made one QTE cue that lasted 2 bars. I then also created that same cue, but which contained more instrumentation to represent a higher intensity.

When this was then imported into Fmod, a loop section was added to the 2 bar nee QTE cue. If the player (Lara) fails the first QTE section, it’ll transition to the Game Over cue, BUT, if not, it’ll continue to loop. If the player then progresses further, Fmod will automate to the higher intensity version of the QTE cue, to represent they were close to the end of the QTE sequence altogether.

This was what the case was with the final project.

If I was a composer sending my work to actual game developers, and I had no idea what Fmod Studio was, nevermind how to use it, my score and my cues may be unprepared and need to be amended.  But by doing the basic fundamentals of Fmod, or other middleware for that matter, composers can make a programmer’s life easier, and their scores better.

Fmod: Studies Continued

Previously, I was working with tutorials created by Stephen Schutze, on how to use Fmod to create dynamic audio. However, the tutorials soon disappeared close to Game Developers Conference, and to now get them, you have to pay a large fee. Due to this, I then had to find other sources to learn how to incorporate my originally composed score into Fmod Studio.

Despite software like Fmod Studio being recently made free to indie developers and students alike, there is still a lack of information available on how to actually use it. Winifred Phillips even notes in her book, A Composer’s Guide to Game Audio, that “manuals and tutorials focus first on the implementation of sound effects and environmental ambiance before addressing music.” (2014.) She later discusses that the reason there lacks information on how to implement original music to Fmod Studio is because it’s assumed that you’ve learned on how to use it with the information available on implementing sound. It doesn’t take into account that composers aren’t always sound designers.

“Fortunately, the game audio community has produced some excellent video tutorials on some of those middleware applications, which we can find on such video streaming sites as YouTube or Vimeo” – Winifred Phillips.

Fortunately is right, as I was able to be directed to the YouTube tutorials by Matthew Pablo. Which, along with being shown a quick demo within a supervisor meeting, helped me grasp the basics of Fmod Studio that I needed in order to develop and finish my project.

Screenshot 2014-05-11 20.43.40

The demo and test which was put together within a meeting was saved, and later referred to it and messed around with it to get a feel of how Fmod really worked. Suddenly, I had a lightbulb moment where I realised that the fundamentals of Fmod began to make sense to me.

I began to slowly import my own cues of music, taken from my original score, and applied them to what Pablo was doing within his videos.

Exposition: The Importance of Marketing

1907329_10154101798870693_1973524580779277591_n
An honours degree isn’t just about the work load, but it’s about preparing you for the professional future in your chosen career path. By allowing you to create and develop your own project, us students will be better equipped to complete projects that we may take on in the future (for example, composing for a game.)

After the initial deadlines, we have the graduate showcase, which is our shot as advertising not only our work through the year, but showing that we can present and sell ourselves at a professional industry standard.

For this, I commissioned new artwork from an artist that I’ve used previously for my website and for business cards. Business cards are really important to those, especially working in the creative industries. My cards will be available at my space at the showcase and, I bought a matching pink card holder to have around with me at all times.

The cards will also come in handy for the second Edinburgh Game Symposium! Which is a perfect place to network.

Completing the Score: The importance of Setting the Tone

Though the focus of my project is on creating and studying dynamic game audio, and not the worth of the composition itself, part of what makes a dynamic score successful is not just on how well it changes with the players input and game states, but how well it fits the mood of the scenes and the gameplay as a whole.

Within the re-score for the Tomb Raider gameplay which I composed, I created a short ‘theme’ for the emotional cut-scene sequence where Lara kills for the first time. This was because, from being an experienced gamer, I felt that that is what worked with the scene.

The premise of the Tomb Raider reboot, was to not have Lara be a hero, but a survivor. This was most likely done to be a fresher take on video game protagonists (compared to say the likes of Link, Super Mario, etc) and as it can able players to empathise with Lara more on an emotional level. For not everyone in the world has saved a princess or rescued all their friends, but a lot of people have dealt with highly emotional and difficult situations with involved them being strong and getting through, or ‘surviving’ it.  I personally have dealt with struggling with mental health problems for most of my life and when playing the Tomb Raider reboot, I saw the reinvented Lara as a great symbol and inspiration to continue through life and give it my all.

The notion of Lara being a survivor is very important, and in this scene which I composed an emotional piece of music to, Lara kills for the first time. From the scene, you can see that she’s truly devastated that she’s done that. Not something you often see in games where killing enemies is commonplace. This moment for Lara is where she knows that what’s ahead of her isn’t going to be easy, that she’s staring death literally in the face, but she needs to carry on for her friends sake and her own.

I tried to capture this tone for that scene, which can be heard here;

Through my research on dynamic audio, I discovered that composers put a big importance on setting the tone and creative themes when creating their own dynamic audio. When I asked Robin Beanland, head of music at Rare Ltd about how he created dynamic audio, he told me that his, “approach is to set the tone for the game first and then look at the game and design docs to see if there are any opportunities for implementing some dynamic music.”

A transcription of theme from that piece of music above, can be found here. Due to the score being needed to be composed with it later being interactive in mind, I had to keep the changes and length very short and minimal but, as I mentioned earlier, I also had to evoke the tone and nature of the scene as a whole.  The theme itself, consisting of just the notes, G, B, E, F#.

 

TR Theme

After putting this piece of music on soundcloud, I got good feedback from others saying that it was very emotional and evocative, even though it was such a short piece. This re-assured me that I was able, and able to capture emotion within my music which is very important to creating dynamic music as the purpose of dynamic audio is really about making the music seem seamless so that the player feels more a part of this virtual reality.

 

Completing the Score: The Score

This is my completed score for a scene I recorded myself from Tomb Raider (2013.) I chose this scene because of its importance to the game as a whole and because it contained multiple types of gameplay which would be perfect to demonstrate the act of interactive music to.

Originally, when I first started my honours, I wanted to score to an actual level of a game in actual development. Though I have done work for an actual game before, it was hard to find time to work with developers and programmers when I have to commute for university and have a job to support that. Because of that I made the decision to compose to a piece of pre-existing gameplay, despite that being linear and defeating the nature of my entire project. To get around that, by cutting my score into mini-scores or cues, and then importing them to Fmod to re-create the seamless score using interactive audio creation methods and recording the result along with the gameplay, I believed that would be the way to get around the initial problem of composing to pre-existing gameplay.

I composed the score as I compose generally, I watch the medium over and over until I can start to form an idea of the audio in my head, I pick VSTS I believe I’ll need, for example, as this was primarily orchestral, I already selected varying string instruments that would be appropriate. I then, using my MIDI keyboard which is connected to Logic Pro 9, start to record in layers which then develop over time.

I decided on orchestral instruments as it better suited the mood of the game. In order to compose for games, I believe that you have to be an avid gamer. To know what’s expected or what suits the mood, you need to play games and see what others have done to suit moods in certain scene and game types. I’ve completed Tomb Raider before and am very familiar with Jason Graves score. I didn’t want to copy it, and don’t believe I did, but I did study closely how he interpreted the scene musically which I then re-scored. I took a more dramatic approach, especially for the QTE segments, but I followed his approach of what sections he deemed should be dramatic and what ones shouldn’t.

For the gameplay section at the start, I took a ‘march/sneeking’ approach to the tone of the music. I also tried to ensure the music sounded a bit dissonant to make a would be player alert of Lara’s doomed and very frightening situation.

For the QTE section, I took a very dramatic approach, as I’ve already stated. This was because I haven’t ever really composed pieces of music like that before, and believed that though it was perhaps a bit too over the top for the situation, that it still fit. To get the right feel for the work, I listened to Corbeil’s soundtrack for Quantic Dream’s Heavy Rain.

In Heavy Rain, each character has a ‘suspense theme’. A theme which plays when the character may soon be facing their doom. In Heavy Rain, a character can die at given times in the game that can alter the story and what ending you produce, and so the music replicated this. From studying pieces such as the one below, I learned the importance of using brass swells, accented on the beat to create a suspenseful like tone to a scene. And so this is what I did for the QTE segments.

 

 

For the QTE segments, I had to compose knowing they’d be cut into cues. This meant, that the score could stop at any given time, switch to the game over music, as Lara can die if the QTE fails, then switch back again. Knowing this, I decided to make it so that the sequence I composed, if Lara/the player passed the first QTE mark would continue, then if they succeeded the second, it’d continue again, and if they succeeded again, strings and more instruments would appear to try and create more dread to lead up to the final action of that piece of gameplay.

The scene which follows the QTE section, which is discussed in this post here, was composed to be emotional because of its importance to the game over all. Lara has just killed someone for the first time. Though common in the previous line of Tomb Raider games, this game is a reboot and the start of Lara’s story, before she becomes confident enough to not allow anyone to get in her way. Lara killing someone for the first time, resonates with her what she now has to deal with, what her life will now be. Though faced with dramatic events already in the game, I believe this is what truly hits her to make her aware that she is in grave danger and that life is fleeting.

The entire game is about not being a hero, but a survivor. This scene is what makes Lara realise this.

And that’s why I tried to compose something that was emotional. I also composed my own take on a ‘theme’, as I believe that as this scene is important to the story and to Lara overall, that having a theme here would be extremely relevant. A theme is what represents Lara throughout the score, and this is scene is all about her life changing so having the theme play, or my version of it, works.

The score then relates back to gameplay, which I decided to be the same as previously as Lara is still in danger, still surrounded by NPC’s (non playable characters) and it meant that in Fmod, I could tell it to resort back to the gameplay marker.

Overall, I’m not 100% pleased with my score. But when is a composer ever truly happy with their work?

If I were to re-do it, I’d spend time sourcing better VSTS as I believe the fake nature of the sounds of the strings take away from the emotion of the score over-all. I would also thicken up the score as a whole as I believe it’s lacking in a lot of flourishes of percussion, which are really important to  games of that genre.

I am happy with my QTE sections however. One of the cons of horizontal re-sequencing is that a lot of your music has to be composed with the same BPM. My score was composed in 120 BPM, which isn’t that fast but dramatic suspenseful sequences in games are usually faster paced, like the Heavy Rain example. After plating some games and fully listening to how the brass is utilised, I believe I managed to capture the basis of how to create the sense of dread within those QTE sections.

Completing the Score: Horizontal Re-Sequencing

In my previous post, I talked about how the score had been cut into mixes which varied in intensity. And that these would be imported into Fmod to create the interactivity of the score. The other method of creating interactive audio is called Horizontal Re-Sequencing. This involves the completed score, being cut into mini scores or sections. For example, my score has been cut into these sections, which you can see below, and altogether they make up the entire score.

 

Screenshot 2014-05-05 19.18.03

In Fmod Studio, I took these mini-scores and aligned them in order of how they appear in the score. I then set up parameters from 1 – 10, and assigned one to each of the scores. A tempo marking of 120 BPM was set, as that’s what the audio was composed to within Logic Pro 9, and I assigned transition marking to the mini-scores so that the sections of music would switch to play the score in full.

Screenshot 2014-05-06 01.33.29
As deadlines draw a close end, I am in the process of finalising this method to create my representation of the practice of horizontal re-sequencing. The final project will feature the gameplay playing, while showing Fmod running the score using the horizontal re-sequencing method and the vertical re-mixing method.