I’m going to write some posts about personal projects.
I’ve been undertaking lots of work lately for other people, or for my master’s degree course, which has been incredibly valuable for learning and establishing working relationships with other creative producers.
Because of this, my own areas of interest, as explored right here in this blog, have been slightly shelved during this time, even if the other projects feed into ideas and technical skills – so I felt it would be helpful to articulate some of what I want to have a go at, to ensure the ideas don’t ‘drift‘.
I have been thinking on about sensory subjectivity and autism, which continues to interest me, as at the heart of it, really, is the same old question ‘what is consciousness?’ (I just started reading this book, which promises to look askance at the same question, from a different perspective, and might fuel some interesting ideas for further work, though I note the reader reviews are pretty mixed.)
I had an online conversation with an autistic adult a few years ago, during which he told me a little of his childhood. He was saying that his parents used to worry like crazy about what looked to them like a cognitive absence as he was visually ‘stimming’. He wished to reassure other parents of children with autism that it is OK, the stimming is a useful thing, that autistic children will turn into interesting, capable autistic adults, but that they must be allowed to regulate their sensory processes. It was a very kind comment. But it was the description of what stimming looked and felt like that captured me. He said that if he stared long enough at certain patterns, they took on a kind of iridescent quality that eventually pulled him into a peaceful, beautiful landscape, a place of mental repose for him – and also a source of delight.
One idea for a final major project for the degree is to produce my own animation, a highly-coloured artistic representation of this effect, and an accompanying sound-world that draws in natural sounds and transforms them, to try and replicate the neurological journey. It would be flanked by two monochrome sections where a parent is trying and failing to engage a child in communication, failing to see that the child is clearly communicating that s/he needs right now to be doing exactly what s/he is doing and that this is no cause for sadness.
I was put in contact by a mutual friend with American artist Angela Weddle, who produced this digital sketch for me, from her own projection of what such a world might ‘feel’ like, drawing partly on her own experience.
Here is a video showing how the image was built:
And here is the finished image:
As I am keen to be involved in producing the animation as far as my limited visual creative prowess allows, I am doing my best not to anticipate how it will sound too early in the process, tempting though it is! But I already known the incremental layering of strands seen here must be present in the soundtrack, and that it must use the naturally occurring sound design and Foley with added sounds representing the child’s own internal processes, in order to present the child’s entire neural construct of the world, with all the elements that build it.
I made an eight-minute ‘behind the scenes’ documentary about how I created some of the more idiosyncratic sounds that are part of the fabric of the series.
If you are interested in this creative process that has been dubbed ‘beautiful lies,’ then have a listen to see how I told my particular lies!
You can download it from the iTunes link above, or press play here:
Following on from this post, I’ve been going through some of these short films and noting any trends. There is a bit of variety and one or two of them do some more interesting things. Here’s a few I picked out.
The Shout
I was impressed enough with the use of sound and the choice and editing of music in this one to contact the production company, who have been very friendly and referred me on to the composer and an editor they recommend. I really like the opening with acousmatic wave sounds against the title on a black bacground, introducing the sense of danger and power in the sea at night and immediately drawing us in. I think the choice of music works well to get the heartbeat going – even, maybe, simulating heartbeat. Ben Winters’ article in the journal Music, Sound and the Moving Image, Corporeality, Musical Heartbeats, and Cinematic Emotion explores this phenomenon in a very interesting way. (Winters 2008)
In addition, the way the ‘story’ the interviewees tell is structured and punctuated by the editing of the music. Sound design and foley also brought us closer to the ‘action’ in our bodies. It had me imagining what it is like to work as a volunteer in sea rescue and got my pulse raised as a result (while sitting in my cozy living room with a cup of tea…).
Found Something
This is an awareness video for men, who, it is well known, find it harder than women do to discuss physical problems or to visit the doctor, especially if they are of an intimate nature. The orgnisation use a light, colloquial tone (you know, that ‘laddish’ thing…) and humour as a way to broach the subject of being aware of testicular changes.
No music is used in here – which I think is a good choice, personally; it replicates the casual, ‘down the pub’ kind of mood and keeps the soundtrack clean, neutral and unthreatened by anything remotely resembling ‘suspicious’ subtext.
The two choices they do make are the choice of the narrator – someone who, by his accent and delivery of the matching script, could be someone we, the audience, know personally – and colourful, slightly comedic sound effects throughout to enhance key moments in the animation, a key part of the identity of the film.
What seems to be the typical model in this genre?
Many of the films’ soundtracks were simply confined to edited interviews or voice-overs with some very generic ‘background’ music behind them. This music is generally looped figures in a major key, not more than four chords, extremely repetitive, typically featuring piano and perhaps sustained synths or strings behind it, maybe a little light percussion – drum kit or higher-pitched instruments on repeated ostinati. So it is extremely generic, ‘feel good’, and the definition of wallpaper music. If a more sombre mood is required, then simple, three-four chord ’emotional piano’ is often used.
Here’s a good example of a bland underscore:
And here is one where pure interview + ’emotional piano’ + strings is used, and in this case I think it is absolutely right for the subject matter. It actually made me a bit teary, so it did its job.
I think we have to be careful in this genre not to let our egos intervene too much or to look like we’re trying to manipulate anyone. However, equally, these films are competing for attention on social media. So I wonder if the soundtracks could be a bit more innovative sometimes. This is something I would like to look into further.
(1) Winters, B., 2008. Corporeality, Musical Heartbeats, and Cinematic Emotion. Music, Sound, and the Moving Image 2, 3–25.
I have been putting in the work this week on our Soundtrack inVideo Art assignment. During the course of this, I had reason to find a recording of a newborn’s cry. (Not having a baby to hand, I did sadly have to source this from elsewhere – though I’m going to be an auntie again soon, so if my new nephew obligingly arrives before my video art deadline, I will grab my own recording and substitute it in!)
One experiment I did as part of this was to take all my significant symbolic sounds (of which this is one) and try dropping them into the sampler Kontakt‘s mapping editor.
The idea was to give myself some way of creating a ‘tail’ for the original sounds, perhaps a continuation of them absorbed into the wider ambiance, or mutated over time beyond recognition. I expected to find that once I’d played with the waveform a little, I could isolate and make musical one particular, recognisable quality from the original sound.
But when I placed the sound of the baby’s cry in, a phenomenon happened that I found both fascinating and disturbing in equal measure.
Here is the original sound:
Now, I am a mother. I know that babies cry – a lot, at first, some of them. It is not easy, being a baby. Being born is a terrible shock and the human brain is hopelessly immature at birth. I read once that if humans were born when elephants are in their equivalent relative points of brain development, we would be giving birth to fully verbal, running, playing, food-eating pre-schoolers. However, the huge apprenticeship that is early infancy, childhood and adolescence is also vital to us being so clever as a species!
The cry of a baby is its one big weapon. It is relying on that sound – well, that and looking cute to our nurture-programmed brains – to stop us from leaving it under a bush somewhere to perish. And as such, it is not easy to hear.
On the other hand, we become rather accustomed to hearing it, don’t we? And we assume that because they cry all the time, babies don’t really mean it the way adults do. And no doubt in some ways this is true, but we must also not forget that immature neurology. When a baby’s needs are met through their relationship with their carers, time and time again, as nature expects them to be, and when loving adults respond positively to their attempts to communicate, the brain starts to tailor its development towards becoming a well-regulated adult. Although a layman, this seems to me to be pretty much the most important process in human experience and functioning that there is out there. But before this happens, the brain is all over the place and it must be terrifying when, even for a minute, this absolutely dependent being finds itself without the response it needs. The parenting book What Every Parent Needs to Know… by child psychotherapist Margot Sunderland is one book I have read that discusses this process in clear language. (Sunderland 2007)
So, in that moment (and how else do the very young live but in the moment?), for that un-refined, un-developed being, I think the distress is perhaps as genuine as any more sophisticated person would feel – even if it’s over quickly and perhaps better understood retrospectively as the person learns, with adults to guide them.
Now I get back to my cry. Here is how it sounded, one octave down (so the frequency is halved and the wavelength doubled):
What fascinated and disturbed me was that this cry became a man. A man is crying. You can hear the infancy in the cry, I think – it could be a someone with a lower mental than chronological age – but still, the sound of human distress (adult human distress, to my ears) is unmistakable.
So I took it down another octave…
This to me, sounded like an animal in distress. A new character, but what was consistent was the feeling behind it.
So I took it down another octave…
This to me sounded like a fictional animal, some kind of giant, perhaps, in pain. But still, the universal sound of distress.
Next I tried it up an octave (twice the frequency and half the wavelength). It sounded like this:
To me this sounded like a small animal, a bird, perhaps, or a rodent, defending their young from a predator. The faster speed added an anxiety to the distress.
Finally, I put it up one more octave…
The sound of protest and fear was even more apparent to me in this version.
It could be that, because I have small children, my senses are more attuned to this. But I think there is a strong possibility that the animal communication, the affective, the messages in a sound, are a universal thing.
SUNDERLAND, M. (2007) What Every Parent Needs to Know: The incredible effects of love, nurture and play on your child’s development. DK: London.
This is an article by Patrick Bury in The British Journal of Sociology (Bury, 2017). I came across it half by accident, but found it interesting.
It is also relevant to my work on world-view in that it helped me identify another area of diverse outlook – the closed military life versus civilian incomprehension! (I’m in the latter camp, by the way, which is perhaps why I found the research so diverting.)
There were some astute insights into a very specific social environment, one defined by tradition and hierarchy, and into how behaviours are governed by it.
This work focused on an unusual breakdown of habitual adherence to roles and discipline and the conditions that precipitated this. The author concludes that a recent stressful tour of duty in Afghanistan served to flatten the authority gradient by placing junior officers in positions of great responsibility. It also picked out certain agitating factors on the night, like the inclusion to a senior officer outside of the regimental ‘family’ which seemed to serve the ambitions of the commanding officer and breach and therefore disrupt the firm etiquette of this traditional social occasion.
How this looks to civilian eyes and how this narrative can be reproduced succinctly in story to help us civilians enter into the military mindset is where I may apply this work. I believe soundtrack could be highly referential in this case, referencing:
Sounds associated with this tradition – with drinking, with chants and traditional forms of words, with the banging on tables and the scraping of chairs in rising for the big toast.
(This is the toast that didn’t happen this time when it went wrong – could the chair scrapes sound different, as the junior officer whose speech went wrong moves his chair back to run from the room, as the other chairs squeak when the remaining company turn round to watch him go? Could the new scrapes evoke the old ones, imagined in daydreams as the man prepared the big speech in days leading up?)
Sounds associated with battle and junior officers granted agency beyond that to which they are accustomed in drills.
Sounds associated with disorder.
The ambient environment would be rather claustrophobic, close and dry. This could be used to good effect in conveying the sense of an out of control situation that one cannot escape from, or the uncomfortable proximity of people who suddenly feel different as order breaks down.
I note also that while no doubt there were some woman officers present, the predominate sound of the room would be male.
It is all speculation, as no such project is planned! But interesting speculation all the same.
BURY, P. (2017) Barossa Night: cohesion in the British Army officer corps. The British Journal of Sociology 68, 314–335. doi:10.1111/1468-4446.12236
More rats! I hope this isn’t a recurring theme as I’m not so keen on them…!
This article, Sonic Subjectivity and Auditory Perspective in Ratatouille (Collins, 2013), goes into satisfying depth about sound representations of individual perspective and the techniques used to achieve them.
It discusses:
Microphone placement. For example a scene where two rats are in dialogue alters the perspective mid-scene to a visually distant point while not altering the microphone placement. This apparent contradiction actually serves to keep us psychologically close to the characters while providing physical perspective to enhance our understanding of the story.
Loudspeaker placement and ‘proxemics’ – the study of distance between people as they interact. This can be replicated by careful placing of sound within an auditorium and create natural sympathies by mimicking the various zones of intimacy interpreted by the human psyche from the space between ourselves and another person. It occurred to me, as I read this, that this technique could equally be used to extend feelings of intimidation or violation when a person behaves aggressively and therefore could also build up feelings of antipathy towards a character.
Signal processing effects. She gives the example of the use of a low-pass filter to replicate the experience of being underwater, which is effective in communicating the sensation even when the camera is not following the character in the water 100% of the time. To me this suggests that what we hear can have – at the subconscious level at least – more dominance over our experience of the world than what we see, despite being such a visually-oriented culture. I’m remembering here Walter Murch’s musings on hearing being the first sense of life – indeed, the only one, for the first nine months.
(Murch, W. – foreword to Chion, M., Gorbman, C., 1994. Audio-vision: sound on screen. Columbia University Press, New York.)
To identify these techniques and name them explicitly, rather than relying on artistic intuition to happen upon them, is incredibly useful. I don’t want the success of a project to rely on chance and depend on the presence of inspiration!
COLLINS, K. (2013) Sonic Subjectivity and Auditory Perspective in Ratatouille. Animation 8, 283–299. doi:10.1177/1746847713507164
There are so many areas and disciplines that could feed into my discussions of cuing an audience into subjectivity, world-view (or sensory world-experience) and hearing with another person’s point of audition, that it’s hard to know where to focus.
I have read articles (for example!) on:
How playing a single frequency to young rats enlarges the area of the auditory cortex associated with that frequency, making the rat less sensitive to that one, but more sensitive to neighbouring frequencies.
An analysis of the use of sound to create identification with characters in the animated film Ratatouille.
A write up of a series of tests to probe how autistic adolescents process sound as compared to peers with more typical neurology.
A fascinating analysis of why discipline broke down during a traditional ceremonial officers’ dinner in a British Army corps soon after a stressful tour of duty in Afghanistan.
An article putting forth some interesting theories about gender in the soundtrack to Forbidden Planet.
All of this is relevant. Lets take them one by one.
The Baby Rats! (Han et al, 2007)
The evidence that exposure to a specific noise can alter cortical sensory neurons in early life raises all sorts of interesting speculation for me.
If a child grows up in a noisy, environment (for example, if the telly is on constantly, or the family habitually speaks loudly or aggressively) and if, like the rats, the neurons arrange themselves so that the child hears less of that kind of sound, how does that affect future responses to environment and where they may choose to be (where they settle, where they ‘hang out’, etc).
It seems to me it would have an impact on how they conduct interactions or even how, and with whom, they form relationships?
And could this be reflected in a film by making an audience accustomed to a background noise and then abruptly losing it as we switch?
Just as an aside before I plunge back into With The Light and points of audition, my helpful tutor set me on a trail that led me here.
To me, it seems to be a meditation on the constant observing, self-observation and being observed that is a staple of existence as a woman, and on the feedback loop between these observations and our projected identity.
I have often thought about how the lines get blurred in our heads between this projected identity and the autonomous identity that we would otherwise derive from our core of personality. I think Tai Shani might be looking at how the make and break of female identity depends on how we are observed – particularly the conclusions people draw solely from our appearance – and how confusing it can be to try to identify our authentic responses to what we experience.
In the end, the withdrawal of male scrutiny, the lack of identity bestowed, ‘kills’ the real being – and the watchers too! And yet the scrutiny continues.
Difference in male and female experiences of identity, how we are respectively perceived and how we respond accordingly are definitely an area I could explore!
Strange and dream-like though this art is, I picked up a few concrete technical ideas that I really liked.
One is the way this artist deliberately blurs lip-syncs. This to me conveys quite nicely the interruption of language communication by social anxiety, poor communication, disrupted social expectations, emotive states, people communicating with opposing agendas, or any number of interesting phenomena within dialogues.
Similarly, she often ‘mis-points’ the camera away from speaker to confuse who is the speaker and who is the hearer, whether the voice is external or internal.
Sometimes, the wrong person is issuing the words. It is as though the man looking on the corpse (for example) has become a voice in the woman’s head indistinguishable from her own – another phenomenon I thoroughly recognise from personal experience.
Another device I liked is the alteration of the sound quality of the opening music as we switch between points of view.
I have decided to start by looking afresh at my recent work in the light of our studies at University of Hertfordshire.
We have been given an overview of the varied and interdependent functions of soundtrack in the context of a film in the past fortnight.
A few ideas leaped out as being techniques I have used in some way before, and that might be worth developing.
I’m going to take them one by one and give them a post each.
So, the first one that I started thinking about was…
‘Leitmotif of Noise’
We looked in our first week at the creatively fruitful phenomenon in film that happens when diegetic sound fulfills the traditional role of non-diegetic music. We were shown a little of Forbidden Planet, with its ground-breaking electronic soundtrack by Louis and Bebe Barron. We noted that the soundtrack seems to occur in the narrative space and also to comment on the characters’ states of mind simultaneously.
As we examined this, I thought about traditional film scoring’s heavy use of leitmotif to cue audiences into the big themes of the story and the characterisation. However, contemporary scoring seems to be rejecting this, focusing on acting as the general subconscious of the story. I wondered if, as a kind of substitute signpost, diegetic sounds crossing into the non-diegetic realm could essentially act as a ‘leitmotif of noise’. So, associating a certain recurring sonic event with a theme or character.
This would, I thought, need a conscious and deliberate focussing of the use of sound.
Trains!
I tried to think of examples of where this has happened.
I decided as a starting point, I would have a look at Brief Encounter. Because the film-makers had chosen to use Rachmaninoff’s 2nd piano concerto, there was no purpose-built film score to manipulate melody for its own purposes, even though the Rachmaninoff is used effectively to reflect the general mood and emotional arc of the story. I felt this alone might make it easier to pick out the role of the non-musical, designed soundtrack.
Could this ‘leitmotif gap’ have been filled by the equally iconic train sounds that we all associate as closely with the film as the use of the piano concerto?
(I would have to add, I’m sure this has been examined before, so I will also go on a quest for relevant writing on the subject. All suggestions gratefully received! However, it was an interesting exercise for me to do.)
I watched the film again and documented each time we hear the high-pitched, frantic whistle of the express train. Every time we hear it, seems to represent a moment of psychological danger and/or disarray in Laura’s mind.
Here are the key moments where I spotted it (times are approximate):
Just as the music of the opening credits climaxes (at about 1’45); the music is lost in the sound and this moment segues into the opening dialogue. It seems to serve as an introduction to this particular sound and the music tells us subconsciously, perhaps, to associate it with high feeling.
6’05 – we are told the ‘express’ is going through and the alarming noise obliterates the inconsequential dialogue about chocolate purchasing among secondary characters. This is the first time we walk through this moment – the second time, at 1’23’30, we know Laura comes close to committing suicide by throwing herself under the train here.
12’11 – Fred makes a joke about the children and Laura starts to cry. Interestingly, this time we hear it outside of the place within the story where the sound originates – the station. Although it is possible we are meant to assume that it could be overheard from the nearby railway, this use seems really deliberately non-diegetic to me. Laura senses the danger of losing control of herself and inadvertently doing irreparable damage to her marriage and home. This use of it at this early stage in the narrative cements it in my head as being something we are to associate with ‘danger’ – especially in Laura’s mind.
17’53 – we see Alec in the background and it is the first time within the storyline we see him (because I’m not counting the first account of the final scene that we see at the beginning). Alec looks up and noticesLaura just as the whistle sounds faintly. It is easy to miss but looks very carefully placed when you are paying attention!
18’12 – Laura gets the grit in her eye, thus providing the vehicle for their meeting – fate providing the next step towards infidelity.
59’48 – we hear it quietly as (it turns out, moments later) Alec is making the decision to return to Stephen’s flat with the expectation that she will follow.
1’00’07 – Alec announces his intention baldly – Laura refuses to follow, but we know her tortured ambivalence. This time, the whistle is long, loud and alarming; the danger of infidelity is now so great.
1’03’39 – after apparently escaping temptation, she changes her mind at the last minute and gets off the train again to go back to the flat.
1’23’30 – we hear the ‘suicide’ whistle for the second time and this time we are fully aware of its meaning. It is the climax and I believe the longest and loudest instance of the whistle. The camerawork backs it up this time around. We do not hear any more train noises after this last, final, greatest danger to her and her family.
So – in conclusion! – I do feel I can justifiably describe this as leitmotif of noise.
As to where this takes my work?
I’m really interested in how different people experience the world differently and how this can be portrayed through visual media with the help of sound.
The ‘dream’ is to use visual + aural media to counteract the polarisation of views and tribalism I’m seeing more and more on social media and to help people identify more strongly with people who seem different to them.
Autism is an obvious way into this, as there are sometimes dramatic differences in an autistic’s perceptions to that of a neurotypical brain, partly due to sensory processing differences. Autism is also close to me as there are several important people in my personal life who are on the autistic spectrum. However, I should like to refine these ideas to look at more subtle differences too, in due course.
Earlier this year, as an exercise for myself in storytelling with music and sound, I wrote a soundtrack to accompany a series of stills. The pictures were rudely snatched from a manga book about raising an autistic child, called ‘With the Light,’ by Keiko Tobe and I combined very traditional orchestral writing for the neurotypical protagonist with sound design for the autistic child protagonist’s point of view.
I was attempting to use music to retell the story along with my own response to it, so the music is in the foreground, and the sound has both diegetic and non-diegetic functions.
Here it is:
For this project, I had some really informative and interesting conversations with adults on the autistic spectrum about what sensory overload feels like, in order to represent little Hikaru’s meltdown – but I’ll discuss that more next time.)
For now, here are the things I did in this soundtrack that could at least partially be described as leitmotif of noise, if further developed in a longer piece.
Hikaru’s Stones
The sound of little Hikaru playing with the sound of stones on a drain in the final segment of the piece is significant.
The book is mostly told from Sachiko’s point of view, but there are odd moments in the illustrations where we observe Hikaru observing. Many autistic people will tell you that as children – and in some cases into adulthood – they are well used to perceiving a lot more than people around them understand that they do.
I didn’t want Hikaru’s own loneliness and his observations of his mother’s unhappiness to go without note; I didn’t want him just to be a vehicle for Sachiko’s story.
The stones interrupt the music that represents Sachiko’s crisis periodically to help us remember that Hikaru is there – and listening, and, through the corner of his eye, watching.
The stones are the sound of a mother and child impeded in communication with one another, and they persist into the fade-out as the relationship continues not quite to function.
The stones work in conjunction with Hikaru’s own actual leitmotif, which is a diminished fifth resolving inwardly to a major third repeatedly.
(This is something I stole from a real-life little boy with autism, who had discovered and play this figure over and over again at a piano. It had interested me greatly at the time, having studied diatonic function, as it appeared from this that the dominant-tonic relationship might be something intrinsically meaningful to this child’s ear at least.)
The musical motif is used throughout the piece including as an intrusion in the more sound-based middle section – and represents Hikaru himself.
The knock of the mother-in-law
Perhaps rather ‘cheesily’ I appropriated the unfeeling mother-in-law’s imperious knock on the car window and turned the rhythm of the knock into a rhythmic motif of sorts, signifying the character. So what began as diegetic sound indicated an unwelcome intrusion, a harsh judgment, incomprehension of the difficult journey, then continued to indicate that meaning throughout.
Future uses?
What is relevant about this? Why does it matter that I’ve happened upon this device?
Well, if I do go with my initial plan and research and help portray in sound what affects cognitive and sensory world view, as a long-term device this would seem quite useful.
So often formative experiences determine how we respond in later life. I wonder if this device might be useful to signify this? A character behaving unreasonably could retain the sympathy of the audience if a sound associated with a formative experience recurs to remind them of the context for that behaviour.
In the example above, the books do continue to chart the progress of Sachiko and her family and (as is usually the case) life gets easier as the parents learn how to support the child’s development, and as the child gets older.
However, there are many difficult times along the way. What if the sound of the stones returned whenever Sachiko feels as though she and Hikaru are failing to relate to one another, when he feels lonely or misunderstood, or when they are both feeling bruised by an episode of sensory distress?
What if they returned when it seems her behaviour is overly defensive or touchy and she as a character becomes harder to like, helping the audience to recall the sadness of the early days and how it changed her?
I will look for examples of film or animation where this is used and will think about how I might like to use it in due course.
In my next post, I should like to refer again to With the Light, but looking more at the concept of ‘point of audition’ and how it might be exploited to examine different experiences of the world (due to neurology, perhaps, or other differences) some more, and how I can build on this with more subtlety.
Katharine Seaton 