Keram Malicki-Sanchez, Founder of the VRTO Conference & the FIVARS Festival

88 Names: For those people who are not familiar with you and what you do, do you mind telling us who you are and what you do and how your work intersects with virtual reality technology?

Keram: My name is Keram Malicki-Sanchez and I am the founder of the VRTO Immersive Media & Technologies World Conference launched in 2016 and the founder and executive director of the FIVARS Festival of International Virtual and Augmented Reality Stories that has been now running for 7 years and moving towards our 8th show in the fall of 2021. I am the editor-in-chief of indiegamereviewer.com since 2007 that tracked the rise and golden age of independently developed video games, and also CEO of Constant Change Media Group, Inc., which is an umbrella corporation that handles event production, both on location and for WebXR events, and a virtual reality and game developer.

I began a professional acting career when I was 7 years old and grew up doing theater and film, later studying directing, producing and cinematography at Ryerson University and UCLA — so I am intimately familiar with every part of the production process — all of which features in putting on a virtual reality production and in particular the new movement towards performing in immersive environments. 

Most recently I contributed a 13,000-word essay to the global handbook of immersive Media edited by Jackie Morie and Kate McCallum titled “Out of Our Minds: Ontology and Embodied Media in a Post-Human Paradigm” and another essay titled “Grab Your Helmet For the New Space Age” for a book anthology called Dyscorpia 2 edited by Marilene Oliver and published by the University of Alberta.

88 Names: Of the many potential applications of VR, AR, and other immersive technologies, which do you think will have the biggest impact on society? Which application excites or interests you the most personally?

Keram: Immersive media and technologies are, like all tools, about leverage; they give us a way to activate things far greater than we could do on our own. Whether it’s a process via which we can better understand ourselves or others, the world around us, our cognitive biases and blind spots develop new mental frameworks — they will continue to evolve with us as we will with them. Of course, this is the most optimistic view and we could easily descend into a quagmire of important concerns concerning privacy, surveillance, ethics, and control. Like any technology, in the wrong hands these inventions are capable of doing great harm, or in the right hands triggering significant breakthroughs that have not been achievable through other means.

To be more specific: one application will be how we can create an infinite number of layers over consensus reality where we can look at the past and future of any space or event, that may be familiar, alien, design or art-oriented, industrial and enterprise-driven, archival and educational, pre-visionary, escapist and solo, inclusive and multi-user and so on.

There is an onus on us right now to be awake to this future and be capturing the world as it is now, which due to our old friend entropy, is always becoming irretrievable.

The very nature of time can be exposed to us through these media in ways that few other methods can expose.

88 Names: How do you see VR/AR as a positive/progressive medium on both individual and systemic levels? Do you think it can actually change the way whole groups or countries view the world and then act on those new perceptions?

Keram: Of course. Running my international festival, I’m exposed to the activation of this idea where we see either spherical video content or interactive CG-based content through the lens of the local culture, community, or industry whence it was created. This of course can also be said of cinema, television, radio, painting, dance, and literature, but the power of these spatialized media is in how they expose us to embodied expression and considerations around interpersonal space — and I don’t mean in how they are different, so much, as in how they, and we, are the same. Also, the ‘frameless’ nature of the medium, can, in the case of videographic spherical media, afford new contextual cues that typical foveated media may not afford us.

One of the most powerful things about this medium is how it requires full attention by virtue of its design. Augmented Reality will be an overlay to the real world doing precisely as its name states: augmenting that reality offering new information additional information, whereas Virtual Reality is, by definition, an encapsulated experience. All of this requires qualification of course because we are also moving towards interoperability between virtual reality worlds and systems where you are nesting one dimension inside of another and hopping between them at will. We will also see user interfaces that will transmit additional data attached to your virtual wrist or in a heads-up display or as a window into a parallel virtual dimension, so the advantage of singular focus is diluted by that. Nonetheless, you may not, at present, be aware that there is a robber in your house or that the oatmeal is catching fire. This will change, though, as new best practices will include alerts when someone enters your personal VR space. And it couldn’t come sooner — in the early days of our shows we had to discourage and then enforce preventing attendees from mocking, startling, photographing, or otherwise exploiting the prone and wholly distracted state of the user.

We’re always hoping to move towards a more holistic method for communication that can be done through symbols and auditory cues that’s non-language-based, and VR and AR will be excellent in this pursuit, but that is not to say that language is something to be overcome. Language carries with it very important cultural cues, sentiments, concepts and frameworks. I have spoken four different languages since I was a child and there are certain ideas I have difficulty expressing in another language. At the same time, I am a board game designer, and this parlays more than you’d expect into building diegetic instructional cues in VR. So we want to discover the best way to communicate these ideas — whether they are instructional, emotional, or otherwise through a multimodal process. 

Perhaps through these processes, we can learn more about those cues and frameworks. As you can tell I have no definitive answers I have more questions and curiosities and that is a large part of the essence of my work in this field.

88 Names: Are there potential negative impacts of this technology that you are especially worried about?

Keram: I don’t have any specific concerns that I wouldn’t have about any technology (though I may convince myself otherwise). 

Any tool in the wrong hands can cause harm. I don’t believe that people will be so addicted to Virtual Reality that they’ll never leave it, any more than people were so addicted to radio or television, in fact, even less so. I think of Virtual Reality as more akin to a toaster oven where you occasionally make bagels. 

Of course, hardware will become lighter and intrinsic to our daily outfits or bodies, but when that happens it will be how we’ve evolved. You can put the genie back in the bottle. It will just turn into something different and I’m certainly not here to stop that. 

This week (April 1st 2021) Microsoft finally secured its 22billion dollar contract with the US army to implement augmented reality headsets for soldiers. Some people were up in arms about this, but VR and AR have always been about the industrial-military complex. AR started with HUDs for the air force. It’s people like Jaron Lanier, Brenda Laurel, Char Davies, Android Jones, and, dare I include myself and my contemporaries that aim to draw out, foster, promote and enrich the creative aspect, and I have to say that the immersive community has been deeply creative, always checking itself to do better and promote more inclusivity and concerned with creating better society and community. It didn’t have to go that way, but it has, and I think that it has to do with people who have seen these sorts of rises of technological tides before, wanting, insisting that we do better this time. 

The real concern comes from matters of privacy and monolithic panopticons like Facebook exploiting the data, which of course are deeply personal, and turning that back on us as they have historically. I won’t be the first person to say this, and I won’t be the last person to say this, but that is the gravest concern; a monopoly by a bad actor that sees us as so much krill to siphon through the whale that will exploit our every whim and biological exploit.

And there is another dark side there too — which is that with these sort of monoliths, come powerful gatekeepers who decide who gets in and who is left out. It is the indie developers who have done the lion’s share of ideation and grunt work in figuring out what experiences to create, and that has proven successful, but many feel stepped on, squeezed out, disenfranchised by not only gatekeeping but actual nefarious practices by the big distributors, with Oculus under Facebook’s rule being particularly egregious. And look, today it is Facebook — tomorrow Facebook may just be like MySpace, Friendster, or Colecovision in our rearview mirror but the threat will be the same and it is essential that we not only create but support alternate channels for distribution. I was a part of the indie DIY Gen X record label movement of the 90s as a response to the major record labels controlling who and what we could hear. The same applies here. And it is why I do what I do.

88 Names: One of the themes of 88 Names is that making and enforcing good rules is hard, because people who want to misbehave are clever about finding loopholes, and the internet lets them crowdsource the problem. To the extent that VR may have negative impacts, do you have any thoughts about the best way to deal with these?

Keram: I guess first we have to define what are things that need to be enforced, and I presume on the user side that include bad behavior, abuse, harassment, bullying, extortion, impersonation. There’s a constant conversation about how to police and to moderate virtual communities (or gaming communities for that matter) without essentially becoming helicopter parents and killing the experience and killing the ability to communicate openly and freely and still afford that essential experimentation and curiosity that leads users to come up with incredible feedback that makes experiences so much better. 

The problem comes from both sides; it comes from the members of the public who are perpetrators and then those who are trying to enforce it that are equally problematic and the methods by which do so. This is a challenge of any society. One of the solutions in Virtual Reality, however, is having the ability to filter whom you see: whether you can see only people that you know, see only people that you don’t know, see people that are on a frequency, and so on. Furthermore, you can also do things in 3D spaces like becoming invisible to those who you don’t want to see you or create a bubble around you. I wish we had those powers in real life. I guess, now we do if you have seen what a Flaming Lips concert looks like during the pandemic where every member of the audience is nested inside a giant plastic bubble they can roll around it (and I won’t digress in the fascinating ways in which consensus reality meatspace is now starting to reflect Virtual Reality design concepts).

I suppose another side to this is impersonation — someone pretending to be you inside a virtual world. As general adversarial networks (GANs/deepfakes), LiDAR, photogrammetry volumetric capture, all become faster, more affordable, and ubiquitous, we have to be concerned with the appropriation of our likeness and others operating on our behalf. Or being digitally exhumed for that matter. Heck, our friends at Microsoft patented chatbot technology for reanimating our deceased loved ones. (I write about digital ghosts and exhumation and posthumous rights extensively in my chapter for the global handbook. My chapter is available for digital purchase here: Out of Our Minds: Ontology and Embodied Media in a Post-Human Paradigm: Media & Communications Book Chapter | IGI Global (igi-global.com).

ID theft is not a virtual reality problem exclusively — it is a real-life problem and some safeguards and methodologies can be used to ensure that a person is who they say they are. 

Now, as we enter the world of the blockchain we see another double-edged sword, which is that we can create an immutable ledger that will record the inception of a work, of a statement, of an action, and, equally, nefarious: we are creating an immutable ledger that will record an action, or inception of work, or statement. In otherwise for better or worse we are creating a system of accountability and tracking for everything we do and generate.

And I suppose a third vector would be the concern that people could make others become addicted by exploiting their pleasure receptors, their weaknesses, triggering their dopamine, trying to keep them in the game, spending money they don’t have, spending time they don’t have, manipulating their emotional triggers. I saw this happen in my early days with Everquest and World of Warcraft, which was akin to being addicted to a hard drug. It was harder to quit than smoking. 

But I learned a lot from it, and I developed lifelong relationships that were stronger for having no consideration of the person other than their character (ASL was irrelevant) and learned hard lessons about my inability to control my boundaries and I came out stronger for it. It’s like with smokers, alcoholics, drug addicts, gamblers, or sex addicts: they ultimately have to decide to come to the possible paths to escape the cycle and find the solution on their own.

I do think that Virtual Reality will offer us something sometimes more beautiful than the real world, and I think it was Edward Castronova who wrote, in his seminal treatise of valuation of digital assets “Synthetic Worlds,” that games offer us the rewards for those quests we work so hard far that real life cannot guarantee. I am actually more concerned about what smartphones (and ultimately AR) have done and will do to us than VR. However, I feel that the onus is on us to ensure that humanity remains inside of these finger traps that we are creating because we can now deploy tools that can outthink us, outlast us, outdo us, and that are faster than us and our little biological wetsuits. Remember that these are weaponized to bend our will and override our better judgment. So we must regulate and monitor these things like we do fentanyl or gambling or similar addictive substances and activities. We are, after all, only human.

88 Names: Is there such a thing as a failsafe when it comes to handling these potential negative outcomes?

Keram: Insofar as content is concerned — it is different to kill someone in VR than in a traditional game. Though saying so can get some people very upset. But studies have shown time and again that our visceral reaction to violence in VR is more acute than when we have it contained within a visual frame abstracted from embodiment, like on a traditional computer screen.

I think that we can consider that these experiences are more potent on that level, but we have to be cautious lest we return to a Tipper Gore era of radical vigilance over cultural and creative output. 

We see how the opioid pandemic was mishandled by pushing these pills out like a panacea and then decimating communities. An awareness that the product is addictive can be overwhelming and override our better judgment should be present in how it is distributed and disseminated. Some solutions can be to have a timer, reminder, or use any sort of biometric data to caution the user that their heart rate or blood pressure is too high. We could have a necessary visual component as part of a visual resting frame that can fade in and remind you or always be reminding you that you are in an artificial world. 

I remember after too much World of Warcraft or Everquest my dreams fusing those experiences as though they were true memories and trying to reconcile them, and I would wake up very troubled by these hyperactive MMORPG dreams. My eyesight got worse because I would stare at a screen up close for hours for 15-20 hours at a stretch. I became agoraphobic and developed acute insomnia. So we want to always respect the fact that these things are not simply passive media when used at length, that these are tools, and are like drugs and hyper simulators whose mission it is to convince you that you have been transported somewhere else if they are considered to be successful. Again, I am not advocating for excessive regulation so much as affording the technology the gravity that it demands of itself and creating best practices and commonly applied standards and best practices for treating it as such.

88 Names: These days, anyone with a smartphone can shoot and edit high-quality video, which makes it easier than ever to get started in filmmaking. Do similar tools exist yet for getting creative with VR? What advice would you give to an amateur artist who wanted to explore the medium?

Keram: At the time of this writing, I can state that it is far easier to create spherical video content with all-in-one cameras that do the stitching intrinsically, and editing software that allows you to natively edit that footage. Even 5 years ago that was not the case. It used to cost thousands of dollars per finished minute of stitched spherical video. Now you can do that natively in Adobe Premiere or dozens of other video editing applications. 

That problem is effectively solved, though the allure of 360 filmings has unfortunately been left by the wayside, and I’m still here championing it because I think it is an incredible and powerful evolution of audiovisual media. But if it had been better managed by the distributors and retailers, and were sexier in the tech industry then we would see it trickling down to more consumer-grade products and not still stuck in the semi-pro and pro levels. 

But also at the time of this writing, we are seeing LiDAR scanning, photogrammetry and volumetric capture tools appear in smartphones like the iPhone 12 and the iPad Pro and soon undoubtedly the capture tools will spread to everything. That will take a little while though because I do respect the enormous amount of investment that Apple put into making it possible to do volcap this well at this price point so rapidly. (Note that Google’s Tango phone had this functionality several years ago but it was considered a niche product and was purchased primarily by enthusiasts).

It’s only a matter of time till we have volcap in our glasses and contact lenses, store windows and bus shelters, front ends of our cars, all of us collectively creating the mirror-world. 

And as far as interactive VR development these are the salad days for young or incumbent developers with free off the shelf tools like Unity, Unreal Engine, Godot, Blender, GIMP, Audacity and more. There are countless free resources to use for experimentation and creation and equally countless purchasable assets from royalty-free music and 3D models to scripts and functions to make your experience work.

Even code has improved vastly — and Javascript is now much more than a way to create dynamic forms on web pages. It has libraries that form the basis of the 3D web and VR and AR web.

If I were to speak to an amateur who wants to make immersive cinema, I would say that you can pick up a tool like a Theta camera or a Samsung Gear 360 (or whatever super low-cost spherical camera exists when you are reading this) and start capturing the world around you and figuring out this new visual grammar. 

I would encourage you to explore spatial sound design and to come at this from the lens of accessibility — what I mean by that is explore the medium as a visual medium and then explore the medium as a wholly auditory medium, and in VR you can even explore it as a tactile or haptic medium. These multimodal elements comprise the nature of this “immersive” format and it will improve and empower your work to consider accessibility in your design from the outset.

Don’t be intimidated by the plethora of information and elements and moving parts. Think of something you want to make and then make one thing at a time. The period during which I learned and retained the most was when I stopped watching general tutorials and created a real deadline with real consequences for myself to produce something. I learned how to modify a 3D object in Blender, and then add a material to it, and then make some parts of it emissive or glowy. Or for 360 creation, I wanted to capture the one-person play written by my friend before other affairs prevented us from doing so — so we pulled out my Vuze camera and shot it a few times, learned some stuff and then came back and did it some more. Just jump in and start doing something that matters to you.

It’s far easier than you might think to get going and we want to see your stories whether out of present-day curiosity or for future cultural anthropological concerns.

88 Names: Are there ways to make the technology available to all classes of society? What kind of formal or informal guidelines, standards and policies should be placed on the technology and the companies that develop it in order to make it available to everyone regardless of their annual income?

Keram: It’s a huge question and it’s really difficult to answer. In fact, my team and our events are part of a consortium right now called Codes of Contact up in Canada where several festivals, schools, arts organizations, and community leaders are working together to do discovery on this very challenge. It applies to every part of the process from how to access an online form or to even reach members of communities that may not have digital tools or platforms at their disposal. For example, they may need to learn about networking, educational, recreational or professional development events through a phone call or a flyer or a newspaper article that’s in print. And beyond that, it’s about never assuming that what you know is what everybody knows; this is a core tenet of user experience and interface design. Ask questions, and encourage questions and feedback from users in every walk of life, demographic and experience level and then iterate, test, fail, ask, iterate, test, fail, improve.

There’s a very steep learning curve and it takes a while to climb it and you have to take the time to slow down and put yourself in a position far outside of your understanding to help people come on board and teach you all that they know, which is likely far richer and more valuable than what you do. The onus is on us to make these tools and their interfaces accessible and available to everyone. And all of these organizations and developers and schools who move on to the next phase of technology should make their current technology available to anybody else who can come in and should be investing in those extended communities so that they can teach us their skills, their stories, and their perspectives and make this technology worth creating in the first place.

But again it requires that we understand that there are intrinsic disadvantages for certain groups and that we have to make an explicit effort to ensure that no person is left behind and that no one is afraid to ask a question or reach out for help. This should never be the exception but the norm.

88 Names: You are a bit of a renaissance man, yourself, what specific ways do you see VR/AR technology really improving education and cultural awareness and why do you think the technology holds an advantage over more tried and reliable methods?

Keram: One of the most powerful aspects to spatialized media is how you can layer information and use the full array of somatosensory interfaces and planes to activate the transmission of ideas. You can have text that hovers in front of something and you can walk through it or around it so that you can have contextual relationships to data, feelings and moods, informational entities that you can explore dimensionally.

You’re also creating an imprint on the user that is akin to, if not wholly a true phenomenological moment. In cinema, we are looking at a frame and we understand that that frame has a grammar so we aren’t any longer surprised when we cut from a wide to a close-up. And in a stereoscopic film, we are titillated by the illusion of depth. But in spatialized media when there’s a context for our position within that data there’s a different level or means to access how we are exposed to it. We can have lateral information, tertiary influencing stimulating information, and we will perceive it, retain it, and remember it differently.

The simplest example of what I’m talking about is in a 360 video where you are in the place of residence of a family and, while you may be paying attention to the action between the family members, you’re also able to look around at the mise en scene, or at those things that may not necessarily need to be in center of frame and infer more information from those things that can provide a deeper context for not having been edited out. In his book Poetics of Space, Gaston Bachelard approaches architecture from a phenomenological approach, more concerned with the emotional and psychological effects the design of a space will engender versus a practical or aesthetic one. I remember Werner Herzog, in our Rogue Film School class, insisting that there is no better way to understand a city than by walking it on foot. And just tonight I took a walk through Hollywood and the smell of orange blossoms and jasmine and honeysuckle that pervaded my senses as I walked past one building to the smell of exhaust, cigarette smoke in the next 10 ft, the cool ocean air of one corner of the block to the hot pungent nature of the next, the sound of cicadas and birds and crickets and then sirens and coyotes. The way that light falls from halogen lights and sodium vapor street lamps and tungsten illuminated windows and the daylight color pall of LEDs slicing through the branches of a kumquat tree at dusk from which I pluck those fruit and crunch into the tangy sour and sweet citrusy flesh. All of these things that affect the sensorium can be available to us in immersive media.

And beyond those aspects, there is the thing that VR does best, which is scale and how it creates a persistent horizon and a sense of relationship to scale so that when we observe let’s say a building or a small creature, or a diorama or an epic capture of a wonder of the world, we feel proportional to it, and it has an impressive emotional effect. So when you are teaching children, or anyone for that matter, about the pyramids at Giza, or the Angel waterfalls in Venezuela, there is an ocean of difference between a small black and white photo in a textbook versus beholding the glory and complexity of that space by being put inside of, or next to it. And it doesn’t take all that much to be convincing. If I have any concern it would be that we may become increasingly inoculated to that effect, so the onus is on us to define those things whose nuances can continue to create that impact for us.

88 Names: Finally, do you have any upcoming projects that you’d like people to know about?

Keram: I’m currently developing worlds in WebXR, which is a new coding base that evolved from WebVR and comprises a series of libraries to create three-dimensional spaces inside your internet browser. I think that the spatial web is important in that it is available to anybody on any device anywhere anytime without requiring proprietary downloads, excessive bandwidth, or storage.

With that, comes the challenge of optimization; realizing your ideas in a way that can be delivered efficiently over the internet. WebXR mostly relies on graphics processors, though, and some RAM, but it is important to understand what can and can’t be successfully rendered via this method. It’s an important exercise to explore how to create these experiences through a distilled process where you have a limited number of polygons and textures and dynamic lights that you can use to convey these ideas powerfully. I think it’s a discipline that everyone should learn so that everyone can use these readily available tools to enact their visions.

I used the time of the pandemic, over a year in isolation and lockdown, to convert my terrestrial events — VRTO and the FIVARS festival — into virtual ones and learned an awful lot about what makes that special and better than a wall of video heads staring at each other. We can eschew the traditional orthogonal designs in exchange for open space designs where budget is not an object and that the spirit of play, emergent approaches, should always be engendered and encouraged to create a successful interaction.

I am looking forward to exploring new forms of film festivals, new forms of arts, play, theater and communication, and exploration of archiving the incredible world and spaces around us that are being lost or transformed, and auditioning wonderful new possible futures and how they play out through these media.