Magic Windows: Final Project

Screen Shot 2018-12-20 at 2.18.33 PM

There are hundreds of parks in New York City alone and each park adds a bit of reprise from the concrete jungle and a sense of community in whichever neighborhood it is located.

For my final project I wanted to utilize ARKit to create an app that would encourage users to go outside, visit their local parks and interact with the space virtually.  The how of the app was to allow users to grow virtual gardens in the parks. in order to do this I needed to accomplish a couple of objectives.

  1. Determine user’s location and whether or not they are in a park
  2. Create AR functionality that allows users to plant crops when they are in the park

User Location

In order to find the user’s location, I used Unity’s Location Services to return the current longitude and latitude of the user’s device. Secondly, I used Google’s Places API to conduct a nearby search to determine whether or not the user is nearby a park.

Screen Shot 2018-12-20 at 2.33.37 PM.png
Here is a screenshot of the app where I manually entered the longitude and latitude of Central Park which was sent to Google Places API.

 

 

AR Functionality

In order to build the functioning app I used Unity’s ARKit Plugin which allowed me to find planes on the floor or any other flat surface.  I imported crop models from the Unity asset store and built the app so that when the API indicated that the user was nearby a park, the user could touch the screen and a plant would appear in AR space.  Additionally, I made it so that the crop changed for different park the user was next to.

 

The X-Code build and scripts I created can be found here.

Advertisements

ICM/PCOMP: Final Project “Deep Dive” (Formerly: “Escape the Cave”)

3D Model of Setup

Project Summary: “Deep Dive” is an interactive “escape” game that combines both physical inputs and digital visuals.  Players must walk around a 24 square-foot mat to discover the buttons hidden underneath as well as use a handheld button in order to navigate through an underwater cave and make it to the surface before the time runs out.

Project Components:

  • Software:
    • Unity Engine
    • Arduino
  • Hardware:
    • 24 sqft foam mat
    • x4 aluminum switches
    • x1 push-button attached to goggles
    • Projector
    • Wiring and perf-board
    • Arduino Uno

Development:

Initially, we developed the basic functionality of the project and prototyped it.  In this example testers would step on a yoga mat and I would manually press the left or right buttons on my computer if they stepped on the right spot:

Further Development and Objectives: ICM Presentation

By this point the primary functionality had been programmed and additional code was written to add supplementary elements such as a UI and audio.

Challenges;

  • Navigating Unity. Using Unity as a game engine is not new to me, but this was my first full-fledged game and a lot of the functionality needed to be learned outside of class. This included:
    • First-Person Character Controllers
    • Serial Communication in Unity
    • Texturing
    • IENumerators
    • External Script referencing
    • Animation
    • Audio
    • Timers

Code:

The code for the game can be found here: https://github.com/dylandawk/PCOMP-ICM-Final/tree/master/Scripts

The raw code may be a bit difficult to decipher because of the lack of commenting on my part (sorry) and because of the nature of how all the parts connected within the project is omitted.

Updated Demo:

Updated screen recording includes On-screen instructions, timer, and final 3D models (audio not available for screen recording).

 

 

 

 

PCOMP/ICM Final: Escape the Cave (Code)

Source Code
Player Controller:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.IO.Ports;

/// <summary>
/// THe script defines how the user serial input affects the game main references
/// for this code com efrom Unity Documentation and Unity Forum
/// link: https://docs.unity3d.com/ScriptReference/
/// link: https://answers.unity.com/index.html
/// </summary>

public class CameraControl : MonoBehaviour
{

    public Camera cam1;
    public Camera cam2;

    private static string port = "/dev/cu.usbmodem146401";
    private static int baudRate = 9600;
    SerialPort sp = new SerialPort(port, baudRate);
    public IsVisible lightVis;
    public IsVisible tankVis;
    public IsVisible dolphinVis;
    public GameObject dolphin;
    public bool interactPressed;
    private float t;
    private float speed = 2f;
    private bool gameStarted;
    private Rigidbody player;

    public LightControl light;
    public LightControl tank;
    public GameObject spotlight;
    public Transform spotlightPrefab;
    private Quaternion spotlightRot; 
    private Vector3 centerPos;
    private Vector3 lastPos;
    private Quaternion lastRot;
    private bool lightAcquired;
    private bool airAcquired;
    private bool dolphinInstantiated;

    private bool isRunning;


    IEnumerator FallThrough(){
        var fromPos = transform.position;
        var toPos = new Vector3(0.012f, -7f, -9f);
        for (var i = 0f; i = closeEnough)
        {
            // Confirm that it's moving
            //Debug.Log("Executing Movement");

            // Move a bit then  wait until next  frame
            transform.position = Vector3.Slerp(transform.position, targetPos, delta);
            transform.rotation = Quaternion.Lerp(transform.rotation, targetRot, delta);
            yield return wait;

            // Check if we should repeat
            distance = (transform.position - targetPos).magnitude;
        }

        // Complete the motion to prevent negligible sliding
        transform.position = targetPos;
        transform.rotation = targetRot;



        // Confirm  it's ended
        isRunning = false;
        //Debug.Log("Movement Complete");
    }




    // Use this for initialization
    void Start()
    {

        cam1.enabled = true;
        cam2.enabled = false;
        interactPressed = false;
        isRunning = false;
        gameStarted = false;
        lightAcquired = false;
        airAcquired = false;
        dolphinInstantiated = false;
        dolphin.SetActive(false);
        centerPos = new Vector3(0.09f, -7f, -9.5f);
        spotlightRot = Quaternion.Euler(.62f, 80.22f, 0);

        Physics.gravity = new Vector3(0f, -1f, 0f);
        player = GetComponent();

        sp.Open();
        sp.ReadTimeout = 1;


    }

    // Update is called once per frame
    void Update()
    {

        if (sp.IsOpen)
        {

            try
            {
                RotateView(sp.ReadByte());

            }
            catch (System.Exception)
            {

            }
        }

        KeyInput();

    }

    private void KeyInput(){

        if (Input.GetKey(KeyCode.DownArrow))
        {
            transform.position -= transform.forward * speed * Time.deltaTime;
            Debug.Log("Move Backwards");
        }
        if (Input.GetKey(KeyCode.UpArrow))
        {
            transform.position += transform.forward * speed * Time.deltaTime;
            Debug.Log("Move Forwards");

        }
        if (Input.GetKey(KeyCode.Space))
        {
            cam1.enabled = false;
            cam2.enabled = true;
            sp.Close();
        }
    }

    void RotateView(int button)
    {

        Vector3 byAngles;
        float inTime = 4f;
        if(gameStarted == false){
            player.useGravity = true;
            gameStarted = true;

        }

        if (button == 0 &amp;&amp; !interactPressed)
        {
            byAngles = Vector3.up * 5;
            var fromAngle = transform.rotation;
            var toAngle = Quaternion.Euler(transform.eulerAngles + byAngles);
         
            transform.rotation = Quaternion.Slerp(fromAngle, toAngle, Time.deltaTime * inTime);

            //Debug.Log("right came through");

        }

        if (button == 1 &amp;&amp; !interactPressed)
        {
            byAngles = Vector3.up * -5;
            var fromAngle = transform.rotation;
            var toAngle = Quaternion.Euler(transform.eulerAngles + byAngles);

            transform.rotation = Quaternion.Slerp(fromAngle, toAngle, Time.deltaTime * inTime);

            //Debug.Log("left came through");

        }

        if (button == 2 &amp;&amp; !interactPressed &amp;&amp; !isRunning) {

            lastPos = transform.position;
            lastRot = transform.rotation;
            float delta = .005f;

            if (lightVis.isVis)
            {
                Vector3 toSpotlight = new Vector3(1.63f, -7f, -10.2f);
                StartCoroutine(MoveToObject(toSpotlight, spotlightRot, delta));
            }

            if (tankVis.isVis &amp;&amp; lightAcquired)
            {
                Vector3 toTank = new Vector3(-2f, -6.9f, -9.6f);
                Quaternion tankRot = Quaternion.Euler(0f, 322.35f, 0f);
                StartCoroutine(MoveToObject(toTank, tankRot, delta));
            }

            interactPressed = true;
            Debug.Log("interact pressed = true");

        
        }

        if (button == 3 &amp;&amp; interactPressed &amp;&amp; !isRunning)
        {
            float delta = .005f;
            StartCoroutine(MoveToObject(lastPos, lastRot, delta));
            interactPressed = false;
            Debug.Log("interact pressed = false");


        }


        if (button == 4)
        {

            Debug.Log("button 5 pressed");
            Debug.Log(dolphinVis.isVis);

            if (lightVis.isVis &amp;&amp; light.inRange &amp;&amp; !lightAcquired)
            {
                GameObject newlight = Instantiate(spotlight, transform.position, transform.rotation);
                newlight.transform.parent = this.transform;
                lightAcquired = true;
            } 
            else if (tankVis.isVis &amp;&amp; tank.inRange &amp;&amp; lightAcquired &amp;&amp; !dolphinInstantiated)
            {
                Debug.Log("Oxygen Acquired");
                airAcquired = true;
                //.Instantiate(dolphin,new Vector3 (-0.25f,-6.92f,-7.22f), Quaternion.identity);
                dolphin.SetActive(true);
                dolphinInstantiated = true;
            }
            else if (dolphinVis.isVis == true)
            {
                cam1.enabled = false;
                cam2.enabled = true;
                sp.Close();
                Debug.Log("Dolphin is Vis");
            }

        }



    }
}

Underwater Fog

using UnityEngine;
using System.Collections;

public class Underwater : MonoBehaviour
{
    public float waterHeight;
    private bool isUnderwater;
    private Color normalColor;
    private Color underwaterColor;

    // Use this for initialization


    void Start()
    {
        normalColor = new Color(0.5f, 0.5f, 0.5f, 0.5f);
        underwaterColor = new Color(.165f,0.2f,0.29f, .5f);
        waterHeight = 0;

    }

    // Update is called once per frame
    void Update()
    {

        if ((transform.position.y < waterHeight) != isUnderwater)
        {
            isUnderwater = transform.position.y < waterHeight;
            if (isUnderwater) SetUnderwater();
            if (!isUnderwater) SetNormal();

        }


    }

    void SetNormal()
    {
        RenderSettings.fogColor = normalColor;
        RenderSettings.fogDensity = 0.01f;
        Physics.gravity = new Vector3(0f, -1f, 0f);

    }

    void SetUnderwater()
    {
        RenderSettings.fogColor = underwaterColor;
        RenderSettings.fogDensity = 0.7f;
        Debug.Log("true"); 
        Physics.gravity = new Vector3(0f, -.2f, 0f);

    }


}

Object Visible:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class IsVisible : MonoBehaviour
{
Renderer m_Renderer;
public bool isVis;

// Use this for initialization
void Start()
{
m_Renderer = GetComponent();
isVis = false;
}

// Update is called once per frame
void Update()
{
if (m_Renderer.isVisible)
{
//Debug.Log(“Object is visible”);
isVis = true;

}
else if(!m_Renderer.isVisible) {

//Debug.Log(“Object is no longer visible”);
isVis = false;

}
}
}

Light Control

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class LightControl : MonoBehaviour {

    public bool inRange;


	// Use this for initialization
	void Start () {

        inRange = false;
		
	}
	
	// Update is called once per frame
	void Update () {
		
	}

    public void OnTriggerEnter(Collider other) {
        if(other.tag == "Player") {
            Debug.Log("I is here!");
            inRange = true;

        }
    }

    public void OnTriggerExit(Collider other)
    {
        if (other.tag == "Player")
        {

            Debug.Log("I is gone!");
            inRange = false;
           

        }
    }
}

Dolphin Character

using UnityEngine;
using System.Collections;


/// <summary>
/// This code defines the motions and animations of the dolphin 
/// The source original source code comes from the purchased asset
/// link: https://assetstore.unity.com/packages/3d/characters/animals/bottlenose-dolphin-128643
/// </summary>

public class DolphinCharacter : MonoBehaviour
{
    Animator dolphinAnimator;
    Rigidbody dolphinRigid;

    public bool isLived = true;

    public float forwardSpeed = 1f;
    public float turnSpeed = .3f;
    public float upDownSpeed = 0f;
    public float rollSpeed = 0f;


    public float maxForwardSpeed = 1f;
    public float maxTurnSpeed = 100f;
    public float maxUpDownSpeed = 100f;
    public float maxRollSpeed = 100f;



    void Start()
    {
        dolphinAnimator = GetComponent();
        dolphinRigid = GetComponent();

    }

    void FixedUpdate()
    {
        Move();
    }


    public void Hit()
    {
        dolphinAnimator.SetTrigger("Hit");
    }

    public void Bite()
    {
        dolphinAnimator.SetTrigger("Bite");
    }


    public void Death()
    {
        dolphinAnimator.SetTrigger("Death");
        isLived = false;
    }

    public void Rebirth()
    {
        dolphinAnimator.SetTrigger("Rebirth");
        isLived = true;
    }





    public void Move()
    {
        if (isLived)
        {
            //dolphinRigid.velocity = transform.forward * forwardSpeed * maxForwardSpeed + transform.up * upDownSpeed * maxUpDownSpeed;

            //if(forwardSpeed&gt;.1f){
            dolphinRigid.velocity = transform.forward * forwardSpeed * maxForwardSpeed;
            //	if(Mathf.Abs(rollSpeed)&lt;.1f){
            transform.RotateAround(transform.position, -transform.right, upDownSpeed * Time.deltaTime * maxUpDownSpeed);
            transform.RotateAround(transform.position, transform.up, turnSpeed * Time.deltaTime * maxTurnSpeed);
            //	}
            transform.RotateAround(transform.position, transform.forward, rollSpeed * Time.deltaTime * maxRollSpeed);
            //}

            dolphinAnimator.SetFloat(&quot;Forward&quot;, forwardSpeed);
            dolphinAnimator.SetFloat(&quot;Turn&quot;, turnSpeed);
            dolphinAnimator.SetFloat(&quot;UpDown&quot;, upDownSpeed);
            dolphinAnimator.SetFloat(&quot;Roll&quot;, rollSpeed);
        }
    }
}

Dolphin Controller

using UnityEngine;
using System.Collections;
using System.IO.Ports;

///

/// This code specifies how the user input affects the dolphin’s motion
/// The source original source code comes from the purchased asset and includes
/// the first line in the Start() function and the FixedUpdate() function
/// link: https://assetstore.unity.com/packages/3d/characters/animals/bottlenose-dolphin-128643
///

public class DolphinUserController : MonoBehaviour
{
DolphinCharacter dolphinCharacter;
private static string port = “/dev/cu.usbmodem146401”;
private static int baudRate = 9600;
SerialPort sp = new SerialPort(port, baudRate);
public Camera cam2;
private bool cam2Enabled = false;

void Start()
{
dolphinCharacter = GetComponent();
StartCoroutine(“DolphinCircles”);

}

void Update()
{
if (!cam2Enabled && cam2.enabled)
{
cam2Enabled = true;
sp.Open();
sp.ReadTimeout = 1;
StopCoroutine(“DolphinCircles”);

}

Introduction to Computational Media: Week 09

Final Project Proposal:

Generative Model of Schooling Behavior in Fish

Presentation

For my project final I want to create a digital model that simulates the schooling behavior in fish and other aquatic creatures. This is obviously inspired by my love of the ocean and the water but also by my desire to create and represent mathematical models in p5.  Biomimicry is a really cool and exciting space in which technology can grow and I would love to bring that into my project now, and I also think this model could be useful in other projects I have going forward.  My model will be informed by the research congregated in “Patterns and Mechanisms of Schooling Behavior in Fish: A Review” by D. S. Pavlov and A. O. Kasumyan.  This article gives valuable information on the distribution, position, and other relationships involved in schooling fish.

Screen Shot 2018-11-06 at 3.09.05 PM.png
Abstract of the research article I will be basing much of my modeling on.

Magic Windows: Midterm Project

Midterm Concept:  From my last blog, I wrote about creating a Voodoo-like AR experience in which the user has the power to inflict harm on to my avatar.  Essentially, there is one target (my driver’s license) that is augmented and all the other targets are triggers that affect the main target.

The latest iteration:

Development

My Avatar:

The avatar I used was a 3D scan of myself that I brought into Maya to clean up and morphed with a base mesh in Wrap3D.  The avatar uses 5 different Mixamo animations in order to respond to the other objects and idle. The image target is my drivers license.

Fire particle effect:

I created the particle effect for the fire instead of downloading it.  I created the texture in photoshop and brought it over to Unity. It changes size and color over its lifetime and is composed of three different particle systems: Base, Add (saturation and intensity), and Glow (Luminance). The image target is a person holding a voodoo doll over a fire.

Car:

The car was the easiest as it was just downloaded from the asset store.  The image target was a bunch of hotwheels.

The Code:

Creating the C# scripts was by far the most difficult part of the project because I have very little experience coding in C# and I was tackling some logical challenges that I did not know how to accomplish.  Figuring all of this out required extensive googling and all the parts of my project have been individually done before but not many people have put it together in the way I did.

The biggest breakthrough came when I realized the best way to affect the digital models was not to attach them as children of the image target in the scene, but rather add them in the script (default Image target script) when I needed them.

Test of the fire sequence

The rest of the logic was simple enough and the time consuming part was just researching the Unity documentation and googling how to accomplish that logic. The different animations were triggered using if statements, public/private booleans and animator controllers.

This is the code attached to the Driver’s License target (adapted from Default Trackable Event Handler Script by Vuforia):

using UnityEngine;

using Vuforia;

///

/// A custom handler that implements the ITrackableEventHandler interface.

///

/// Changes made to this file could be overwritten when upgrading the Vuforia version.

/// When implementing custom event handler behavior, consider inheriting from this class instead.

///

 

public class SCRIPT_DYLAN : MonoBehaviour, ITrackableEventHandler

{

   public GameObject dylan;

   private GameObject newDylan;

   Animator hit_anim;

   public GameObject fire;

   private GameObject newFire;

   private bool fire_exists;

   Animator fire_anim;

   public GameObject car;

   private GameObject newCar;

   private bool car_exists;

   Animator car_anim;

   public SCRIPT_BOOL_HOLDER boolboy;

   #region PROTECTED_MEMBER_VARIABLES

   protected TrackableBehaviour mTrackableBehaviour;

   #endregion // PROTECTED_MEMBER_VARIABLES

   #region UNITY_MONOBEHAVIOUR_METHODS

   protected virtual void Start()

   {

       fire_exists = false;

       car_exists = false;

       mTrackableBehaviour = GetComponent();

       if (mTrackableBehaviour)

           mTrackableBehaviour.RegisterTrackableEventHandler(this);

   }

   protected virtual void Update()

   {

       if (boolboy.Fire_Active && !fire_exists)

       {

           newFire = Instantiate(fire, transform.position, transform.rotation);

           newFire.transform.parent = this.transform;

           fire_exists = true;

           fire_anim = newDylan.GetComponent();

           fire_anim.SetTrigger(“Make_Fire”);

       }

       if (!boolboy.Fire_Active && fire_exists){

           Destroy(newFire);

           fire_exists = false;

       }

       if (boolboy.Car_Active && !car_exists)

       {

           newCar = Instantiate(car, transform.position, transform.rotation);

           newCar.transform.parent = this.transform;

           newCar.transform.Rotate(0f, 180f, 0f);

           car_anim = newCar.GetComponent();

           car_anim.SetTrigger(“Move_Car”);

           car_exists = true;

           hit_anim = newDylan.GetComponent();

           hit_anim.SetTrigger(“Hit_Car”);

       }

       if (!boolboy.Car_Active && car_exists)

       {

           Destroy(newCar);

           car_exists = false;

       }

   }

   protected virtual void OnDestroy()

   {

       if (mTrackableBehaviour)

           mTrackableBehaviour.UnregisterTrackableEventHandler(this);

   }

   #endregion // UNITY_MONOBEHAVIOUR_METHODS

   #region PUBLIC_METHODS

   ///

   ///     Implementation of the ITrackableEventHandler function called when the

   ///     tracking state changes.

   ///

 

   public void OnTrackableStateChanged(

       TrackableBehaviour.Status previousStatus,

       TrackableBehaviour.Status newStatus)

   {

       if (newStatus == TrackableBehaviour.Status.DETECTED ||

           newStatus == TrackableBehaviour.Status.TRACKED ||

           newStatus == TrackableBehaviour.Status.EXTENDED_TRACKED)

       {

           Debug.Log(“Trackable ” + mTrackableBehaviour.TrackableName + ” found”);

           OnTrackingFound();

       }

       else if (previousStatus == TrackableBehaviour.Status.TRACKED &&

                newStatus == TrackableBehaviour.Status.NO_POSE)

       {

           Debug.Log(“Trackable ” + mTrackableBehaviour.TrackableName + ” lost”);

           OnTrackingLost();

       }

       else

       {

           // For combo of previousStatus=UNKNOWN + newStatus=UNKNOWN|NOT_FOUND

           // Vuforia is starting, but tracking has not been lost or found yet

           // Call OnTrackingLost() to hide the augmentations

           OnTrackingLost();

       }

   }

   #endregion // PUBLIC_METHODS

   #region PROTECTED_METHODS

   protected virtual void OnTrackingFound()

   {

       newDylan = Instantiate(dylan, transform.position, transform.rotation);

       newDylan.transform.parent = this.transform;

       newDylan.transform.Rotate(0f, 180f, 0f);

   }

   protected virtual void OnTrackingLost()

   {

       Destroy(newDylan);

       Destroy(newFire);

       boolboy.Fire_Active = false;

       fire_exists = false;

   }

   #endregion // PROTECTED_METHODS

}