{"id":1325,"date":"2017-05-04T22:40:51","date_gmt":"2017-05-04T22:40:51","guid":{"rendered":"http:\/\/ico.bukvic.net\/main\/?p=1325"},"modified":"2019-04-22T20:40:15","modified_gmt":"2019-04-22T20:40:15","slug":"cinemacraft","status":"publish","type":"post","link":"https:\/\/ico.bukvic.net\/main\/cinemacraft\/","title":{"rendered":"Cinemacraft"},"content":{"rendered":"<p><iframe loading=\"lazy\" title=\"Cinemacraft -- a real-time machinima using Minecraft and Kinect\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/i84DGnfitfc?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>In preparation for Virginia Tech&#8217;s inaugural presence at\u00a0<a href=\"https:\/\/www.sxsw.com\/\">SXSW<\/a> 2016 expo in Austin, Texas, I was invited to serve on the University-level task force whose goals was to identify content for a generously-sized booth. As a result, I introduced a University-wide survey to collect appropriately developed and engaging content. Through this process I became familiar with a large number of projects and initiatives with a high visibility potential. More importantly, I was introduced to the work of a faculty and\u00a0soon-to-be colleague and collaborator Dr. Lynn Abbott whose research in part focuses on recognizing and categorizing human facial expressions with specific focus on helping autistic individuals master recognition of\u00a0emotions through facial expressions.\u00a0When I learned that the\u00a0<a href=\"http:\/\/ico.bukvic.net\/main\/operacraft\/\">OPERAcraft<\/a>\u00a0was one of the selected projects to be highlighted at SXSW, I approached Lynn to see if there was a way to pursue collaborative convergence and research consolidation.<\/p>\n<p>The end result was a new project titled <em>Cinemacraft<\/em>\u00a0and its installation-centric variant\u00a0<a href=\"http:\/\/ico.bukvic.net\/main\/mirror-worlds\/\"><em>Mirrorcraft<\/em><\/a>.<em>\u00a0<\/em>It\u00a0builds on the success of the <a href=\"http:\/\/ico.bukvic.net\/main\/operacraft\/\">OPERAcraft<\/a>\u00a0and pushes the immersion further by incorporating Kinect HD feed designed to track both body and facial expressions and map them onto the <a href=\"https:\/\/minecraft.net\/\">Minecraft<\/a> avatar. During the intensive 3-month development, the project engaged 4 undergraduate students, one masters student, and one post-doc. The above video showcases a January 2017 iteration based on Minecraft.<\/p>\n<p><a href=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2017\/05\/DSC_0804.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1327 size-full\" title=\"Virginia Tech President Sands explores Mirrorcraft at SXSW 2016\" src=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2017\/05\/DSC_0804.jpg\" alt=\"Virginia Tech President Sands explores Mirrorcraft at SXSW 2016\" width=\"1600\" height=\"1060\" srcset=\"https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2017\/05\/DSC_0804.jpg 1600w, https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2017\/05\/DSC_0804-300x199.jpg 300w, https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2017\/05\/DSC_0804-768x509.jpg 768w, https:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2017\/05\/DSC_0804-1024x678.jpg 1024w\" sizes=\"auto, (max-width: 1600px) 100vw, 1600px\" \/><\/a><\/p>\n<p>The expo was a great success with thousands of visitors having an opportunity to explore <em>Mirrorcraft<\/em> and other amazing creations by the Virginia Tech scholars. Even Virginia Tech President Tim Sands took a moment to play with the installation. Yet, due to complex licensing stipulations associated with reverse engineering the Minecraft source code, the project has since transitioned to the free and open-source alternative titled\u00a0<a href=\"http:\/\/www.minetest.net\/\">Minetest<\/a>\u00a0as its new platform of choice. In addition to porting the <a href=\"http:\/\/ico.bukvic.net\/main\/operacraft\/\">OPERAcraft<\/a>&#8216;s features, the new system is also designed to further enhance the facial tracking and lip syncing by employing contextual sensory fusion. For instance, if a person&#8217;s mouth is open but they are making no sound, the Kinect HD&#8217;s feed takes precedence over the audio-based lip-sync algorithm. On the other hand, if a person is making sound and the mouth is moving, then the lip-sync\u00a0takes precedence over the Kinect HD observed mouth position due to its significantly lower framerate capacity, etc.<\/p>\n<p>The newfound platform is one step closer to an immersive real-time collaborative <a href=\"http:\/\/www.machinima.com\/\">machinima<\/a> that conveniently side-steps the uncanny valley using tried Minecraft-like aesthetics. And while this project is unlikely to reach the capabilities of the recent impressive advances in real-time machinima, the likes of GDC 2016&#8217;s <a href=\"https:\/\/youtu.be\/JbQSpfWUs4I?t=6m15s\">Hellblade<\/a>, I see it carving its own niche, much like\u00a0Minecraft\u00a0did, catering to an entirely different audience with different needs and expectations. Once again, I find myself particularly curious about the project&#8217;s outreach and K-12 education potential I am eager to explore further.<\/p>\n<p>Shows:<\/p>\n<ul>\n<li>SXSW 2016<\/li>\n<li>2016 ICAT day<\/li>\n<li>Science Museum of Western Virginia (February 2017-present)<\/li>\n<li>Virginia Tech Visitor Center (pending)<\/li>\n<\/ul>\n<p>Research Publications:<\/p>\n<ul>\n<li><a href=\"https:\/\/quod.lib.umich.edu\/i\/icmc\/bbp2372.2017.065\/--cinemacraft-immersive-live-machinima-as-an-empathetic?view=image\">*S. Narayanan, and I. Bukvic, \u201cCinemacraft: Immersive Live Machinima as an Empathetic Musical Storytelling Platform,\u201d International Computer Music Conference, Shanghai, China, 2017, pp. 384-389.<\/a><\/li>\n<li>\n<div class=\"csl-bib-body\">\n<div class=\"csl-entry\">\n<div class=\"csl-left-margin\"><a href=\"https:\/\/link.springer.com\/article\/10.1007\/s10055-019-00382-0\">S. Narayanan, N. Polys, and I. I. Bukvic, \u201cCinemacraft: exploring fidelity cues in collaborative virtual world interactions,\u201d <i>Virtual Reality<\/i>, Apr. 2019.<\/a><\/div>\n<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<p>Software:<\/p>\n<p><a href=\"http:\/\/disis.music.vt.edu\/cinemacraft\/\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-237 size-full\" title=\"Learn more about the Cinemacraft platform\" src=\"http:\/\/ico.bukvic.net\/main\/wp-content\/uploads\/2015\/06\/Download-icon-2.png\" alt=\"Learn more about the Cinemacraft platform\" width=\"65\" height=\"48\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In preparation for Virginia Tech&#8217;s inaugural presence at\u00a0SXSW 2016 expo in Austin, Texas, I was invited to serve on the University-level task force whose goals was to identify content for a generously-sized booth. As a result, I introduced a University-wide survey to collect appropriately developed and engaging content. Through this process I became familiar with [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1328,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[49,95,34,3,96,97],"tags":[380,381,390,396,56,398,460,393,395,358,354,392,400,391,394,399,389,397,307],"class_list":["post-1325","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administration","category-art","category-collaboration","category-portfolio","category-research","category-software","tag-380","tag-381","tag-cinemacraft","tag-facial-expression","tag-icat","tag-immersion","tag-irb","tag-kinect-hd","tag-kinematic","tag-machinima","tag-minecraft","tag-minetest","tag-mirrorcraft","tag-operacraft","tag-sensory-fusion","tag-smwv","tag-sxsw","tag-tracking","tag-virginia-tech"],"_links":{"self":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts\/1325","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/comments?post=1325"}],"version-history":[{"count":9,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts\/1325\/revisions"}],"predecessor-version":[{"id":1663,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/posts\/1325\/revisions\/1663"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/media\/1328"}],"wp:attachment":[{"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/media?parent=1325"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/categories?post=1325"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ico.bukvic.net\/main\/wp-json\/wp\/v2\/tags?post=1325"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}