{"id":936,"date":"2015-07-11T17:16:26","date_gmt":"2015-07-11T17:16:26","guid":{"rendered":"http:\/\/ico.bukvic.net\/main\/?p=936"},"modified":"2019-05-10T12:26:45","modified_gmt":"2019-05-10T12:26:45","slug":"operacraft","status":"publish","type":"post","link":"https:\/\/ico.bukvic.net\/main\/operacraft\/","title":{"rendered":"OPERAcraft"},"content":{"rendered":"<p><iframe loading=\"lazy\" title=\"OPERAcraft\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/cwzBHym281o?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>If you are a gamer and\/or have kids, there is practically zero chance you&#8217;ve never heard of <a href=\"http:\/\/www.minecraft.net\">Minecraft<\/a>. This truly unique sandbox\/game hybrid has taken the world by storm, becoming an integral part of our 21st century digital culture, and spawning an ever-growing army of clones. In 2013, when my colleague Ariana Wyatt told me she was \u00a0looking\u00a0for an exciting way to introduce\u00a0kids to the opera genre, I suggested to her producing a virtual opera that could be viewed simultaneously by thousands of viewers all across the globe using a gaming engine. Given its unprecedented popularity among the target population, I suggested\u00a0modding\u00a0Minecraft and making it into a comprehensive real-time <a href=\"https:\/\/en.wikipedia.org\/wiki\/Machinima\">machinima<\/a> production environment that goes beyond Minecraft&#8217;s default affordances. That same year, the research team\u00a0received a $25,000 <a href=\"http:\/\/ico.bukvic.net\/main\/icat\/\">ICAT<\/a> seed grant to pursue this idea.<\/p>\n<p><em>OPERAcraft\u00a0<\/em>is a Minecraft mod designed to interface with <a href=\"http:\/\/ico.bukvic.net\/main\/pd-l2ork\/\">pd-l2ork<\/a>, so as to allow for real-time control of various aspects of opera production. Children engaged in the project\u00a0were given an opportunity to craft their own story and libretto, make their own costumes and build an\u00a0entire world with predetermined locations where the action would take place, as well as control various aspects of the real-time theatrical production. My colleague and project collaborator Tracy Cowden focused on retrofitting famous Mozart arias with the newfound libretto and composing\u00a0connecting passages to smooth transitions between scenes.<\/p>\n<p><a href=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/operacraft_in_performance.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-946\" title=\"OPERAcraft in performance\" src=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/operacraft_in_performance-1024x576.jpg\" alt=\"OPERAcraft in performance\" width=\"730\" height=\"411\" srcset=\"https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/operacraft_in_performance-1024x576.jpg 1024w, https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/operacraft_in_performance-300x169.jpg 300w, https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/operacraft_in_performance.jpg 1280w\" sizes=\"auto, (max-width: 730px) 100vw, 730px\" \/><\/a><\/p>\n<p>Considering opera called for a real-time production, there was a need for a number of features that were unavailable in Minecraft and, to the best of my knowledge to this day remain absent in any of the machinima-friendly gaming engines. These include:<\/p>\n<ul>\n<li>A reasonably accurate real-time mouth animation matching that of the speaking human, or in this case opera singer;<\/li>\n<li>A reasonably flexible body language, in this case independent arm movement and postures commonly associated with opera acting;<\/li>\n<li>Ability to cross-fade between scenes and instantly teleport actors to predetermined locations, so as to\u00a0allow for uninterrupted scene changes;<\/li>\n<li>Ability to use different camera angles\u00a0and seamlessly transition between them, akin to that of a\u00a0movie production, that are visible to actors and appropriately marked to reflect the current active camera but invisible to the audience;<\/li>\n<li>Ability to communicate among actors and production staff without making such communication visible to audience, and<\/li>\n<li>Subtitles accompanying opera singer to make drama easier to follow.<\/li>\n<\/ul>\n<p>Below is a diagram of how we went about making these possible. <a href=\"http:\/\/ico.bukvic.net\/main\/pd-l2ork\/\">Pd-l2ork<\/a>\u00a0served as our audio capture mechanism that used a simple algorithm designed to monitor for melismas and transients from singer&#8217;s audio feeds. These were broadcast to a custom socket inside Minecraft that controlled &#8220;helmets&#8221; on players heads, each helmet offering a different mouth position. Pd-l2ork was also used to broadcast production staff cues, subtitles, and control scene and camera changes. To make Minecraft capable of handling these new features,\u00a0my undergraduate student researcher Cody Cahoon and I set out to decompile, deobfuscate, and modify Minecraft code base, which at the time was version 1.5.2, using tools like Modloader, Forge, and Mod Coder Pack. The newfound mod offered all the aforesaid features, plus a reasonably simple configuration, while also leaving room for future expansion where we could have audience members as spectators inside the game experiencing the drama from their preferred vantage point, while remaining invisible ghosts unable to affect the drama or the world (so, no, you could not blow up the key scene with a TNT block spam).<\/p>\n<p><a href=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/OPERAcraft-diagram.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-939\" title=\"OPERAcraft technical setup diagram\" src=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/OPERAcraft-diagram.png\" alt=\"OPERAcraft technical setup diagram\" width=\"730\" height=\"723\" srcset=\"https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/OPERAcraft-diagram.png 695w, https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/OPERAcraft-diagram-150x150.png 150w, https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/07\/OPERAcraft-diagram-300x297.png 300w\" sizes=\"auto, (max-width: 730px) 100vw, 730px\" \/><\/a><\/p>\n<p>In the final production, there were five soloists accompanied by piano (performed by Tracy Cowden). There were five characters and two camera\u00a0operators, all of whom were middle- and high-school students. Additional professional staff assisted with various technical aspects of the production.\u00a0The premiere took place in December 2014 in\u00a0<a href=\"http:\/\/www.vt.edu\">Virginia Tech<\/a>\u00a0<a href=\"http:\/\/Center for the Arts\">Center for the Arts<\/a>&#8216;\u00a0<a href=\"http:\/\/ico.bukvic.net\/main\/icat\/\">Cube<\/a>\u00a0in front of a standing room only audience. Perhaps an even more exciting aspect of the premiere were over 1,500 viewers who experienced OPERAcraft via a live stream with an unexpectedly high\u00a0viewership from Germany (it was approximately 230am German time when the performance took place). Due to demand, the performance was repeated several days later and over the next month an additional 30,000 visitors viewed the archived video stream that has since been mirrored on\u00a0<a href=\"https:\/\/www.youtube.com\/watch?v=BCFKgffSdwM\">YouTube<\/a>. The 20-minute opera was seen as a resounding\u00a0success and\u00a0we were literally showered with requests\u00a0for a sequel.<\/p>\n<p>Although there was no way of anticipating this at the time, the supporting tools we used for decompiling and deobfuscating Minecraft code have since been deprecated in favor of better and more portable approaches, which unfortunately made our changes forward incompatible. Nonetheless, in December 2015, OPERAcraft is scheduled to\u00a0present its long awaited sequel as part of Center for the Arts&#8217; 2015-6 season engaging a whole new generation of kids. As I look forward to reworking the mod in a more forward-compatible fashion, this is something that is unlikely to make the December deadline. I hope kids\u00a0won&#8217;t mind creating an opera\u00a0that lacks cats and ponies.<\/p>\n<p>Research team:<br \/>\nIvica Ico Bukvic (Co-PI)<br \/>\nTracy Cowden (VT)<br \/>\nKatie Dredger (JMU)<br \/>\nKelly Parkes (VT)<br \/>\nAriana Wyatt (VT, PI)<\/p>\n<p>Research Publications and Published Interviews:<\/p>\n<ul>\n<li>K. Dredger, A. Wyatt, T. Cowden, I. Bukvic, and K. Parkes, \u201cMinecraft opera: Intersections of creative narrative, music, and video games,\u201d In Writing in the Performing and Visual Arts: Creating, Performing, and Teaching, S. Corbett, T. Decker, and B. Cooper, Eds., Across the Disciplines (ATD publishers), 2018.<\/li>\n<li><a href=\"http:\/\/ico.bukvic.net\/PDF\/Operacraft_Journal_2018.pdf\">A. Wyatt, T. Cowden, I. Bukvic, K. and Dredger, \u201cOPERAcraft: Opera outreach for the 21st Century,\u201d National Opera Association Opera Journal. 2018.<\/a><\/li>\n<li><a href=\"http:\/\/www.amazon.com\/dp\/1137279575\/\">Toppo, Greg, &#8220;Epilogue&#8221; in <em>The Game Believes in You: How Digital Play Can Make Our Kids Smarter<\/em>, New York, NY,\u00a0St. Martin&#8217;s Press, 2015.<\/a><\/li>\n<li><a href=\"http:\/\/hdl.handle.net\/2027\/spo.bbp2372.2014.036\">Bukvic, I., *Cahoon, C., Wyatt, A., Cowden, T., Dredger, K. (2014). \u201cOPERAcraft: Blurring the Lines between Real and Virtual. \u201c<em>International Computer Music Conference<\/em>, Athens, Greece.<\/a><\/li>\n<\/ul>\n<p>Software:<\/p>\n<p><a href=\"http:\/\/disis.music.vt.edu\/OPERAcraft\/\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-237 size-full\" title=\"OPERAcraft website\" src=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/06\/Download-icon-2.png\" alt=\"OPERAcraft website\" width=\"65\" height=\"48\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you are a gamer and\/or have kids, there is practically zero chance you&#8217;ve never heard of Minecraft. This truly unique sandbox\/game hybrid has taken the world by storm, becoming an integral part of our 21st century digital culture, and spawning an ever-growing army of clones. In 2013, when my colleague Ariana Wyatt told me [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":334,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[95,34,3,96,97],"tags":[175,131,359,9,357,362,355,77,358,354,361,356,12,277,360],"class_list":["post-936","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-art","category-collaboration","category-portfolio","category-research","category-software","tag-175","tag-131","tag-ariana-wyatt","tag-art","tag-education","tag-forge","tag-gaming","tag-k12","tag-machinima","tag-minecraft","tag-modloader","tag-opera","tag-pd-l2ork","tag-performance","tag-tracy-cowden"],"_links":{"self":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts\/936","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/comments?post=936"}],"version-history":[{"count":11,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts\/936\/revisions"}],"predecessor-version":[{"id":1672,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts\/936\/revisions\/1672"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/media\/334"}],"wp:attachment":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/media?parent=936"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/categories?post=936"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/tags?post=936"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}