To create the Synesthetic dreamscape, I needed to collect data on what the world of music looks like through the eyes of a synesthete. Because the experience of each person is wildly different, I decided to gather data first and then work through the result. I reached out on instagram and reddit (which had a subreddit of Synesthetes) and the response was overwhelming!
Outcomes
RESEARCH, RESEARCH AND MORE RESEARCH!!!!!
I found myself going back to the start and asking myself, what is color? and because it is a perception of the eye, it isn’t physical. To trigger a memory, the colour needs to actively interact with the senses – taste, touch, smell, sound and sight.
That’s when I came across the term ‘synesthesia’. A neurological condition where a person has multiple senses cross over. Ex : The color purple can make a person feel bitter, the letter A is yellow. A synesthete can taste, smell and hear colours. I found this extremely intriguing and exciting and was massively jealous. This is when I decided that I would base my VR experience on synesthesia.
Miro Board : Link to open the board
I started off by looking at similar experiences which have been created on this topic, especially in VR and came across an interesting websites that have been made by synesthetes themselves.
Website : https://www.synesthesia.world/en is an interactive website that lets users visualise the world through 4 synesthetes who see colours and shapes in music. They have further translated it into a VR cardboard experience and what is exciting about it, is the fact that each individual witnesses different type of visuals for different music or different colours for different alphabets. There is no universal depiction. This was a challege because the
The Kadinsky Effect
I was able to think of 4 ideas for the experience
Idea 1 : A social experiment to see if the participant have synesthesia because most of the cases occur with creative people, I thought having this experience at LCC would be an ideal testing ground and have students partcipate, explore and be educated about it.
Idea 2 : Person enters a synestheatic dreamscape with interactive elements where one can tap a button and melodies play, and from the melodies, shapes and colours emerge.
Idea 3 : There is a scene from Ratatouille where Remy is eating cheese and grapes and you can see an explosion of shapes and colour happening nehind him. The director based this on synesthesia of taste which he experiences. I imagined the experience where the player enters the scene as Remy the mouse living in the kitchen and each action depicts one form of synesthesia – Remy painting a picture, eating food, playing music etc triggers a certain visual
Idea 4 : The player enters the scene and witnesses different types of Synesthesia, unaware of the fact that it exists, he starts to go crazy and later realises it is a gift.
Finally went ahead with idea 2 and narrowed down the experience to 3 elements – Shapes, Color + Music
For my AR project I was thinking of having a vinyl record of Frank Ocean’s ‘Channel Orange’ be the base for the AR experience where when the person points their phone over the vinyl, you can see the orange on the cover animating and each of his music plays and when the person taps the screen different shapes and colours start to form to depict synesthesia. I also like the idea of adding tactility to the project, because everything in the modern world is so digital that we lose the essence of textures and the feeling of touch.
The entire first week was an adrenaline rollercoaster of new information and imagery.
We got introduced to our first brief which was to create a VR & AR experience, reflecting design principles for immersive media and XR concepts. While learning about coherence and presence in VR, it was interesting to learn about how we can either use the same principles or break the rules and create experiences against it.
“If VR can make people feel comfortable in experiences, how can I make it uncomfortable on purpose?”
IDEATION
While thinking about the same for my project, I got the idea of – A simple game of confusion
It’s a simple game where there are different colored shapes coming at you, with completely different color names. Ex : A red circle with green written on it, A blue circle with red written on it and the prompt saying “Slash the red circle”. Because our brain sees the color before reading text, the player will slash a red circle.
THE FEELING: As the player progresses to different spaces, the elements and space get more confusing making the player frustrated and annoyed and seriously testing their patience.
Example
– Stairs that go up when you start walking
– You enter the space and it shifts upside down
– The walls start caving in
– The walls are spinning around you
– Soft floor & Ice floor
– Sudden cliffs, falling
RESEARCH
Looking at various works from James Turrel, I was inspired by the way he uses only light as a medium where he creates an experience of “Ganzfeld”: a German word to describe the phenomenon of the total loss of depth perception as in the experience of a white-out.
If smell can trigger a memory, can light?
Keeping this question in my mind, I started looking at the effects of color on the brain
Can the interior color of space, change the mood of the visitor?
Can color induce psychedelic effects on the user? (Yes 😉 )
Can we trick the brain into making a person feel warm in the winter?
What are the different types of confusion?