Augmented Reality: My experiments with AR in Oncology

Wikipedia defines AR as “an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information, sometimes across multiple sensory modalities, including visualauditoryhapticsomatosensory and olfactory

AR demo. It appears impressive but I have included it here to show the possibilities.

I had first noticed AR in BlackBerry Z10 (way back) when BlackBerry dabbled in the consumer technology. It has long withered from human recollection, but I was alive to its potential in healthcare. You place the camera in one direction and they superimposed a complex overlay of images. For example, if I desired to go to a restaurant, the arrows would guide me to the exact spot. Since the computation took place on the device itself, it was explicitly private. However, it needed extensive data in the background and was next to inconceivable to employ it in early 3G era.

AR could never make it to the mainstream and one reason is that there was no “profitable use case” beyond overlaying advertisements or corporations seeking to accomplish self goals.

After I picked up Samsung Galaxy Note, I was instantly taken to “AR doodle” wherein, instead of a generated overlay, I was an active participant in generating the overlay myself. In many reports, they have marketed AR Doodle as a gimmick but for me, it was an opportunity to explore the intriguing insights into its potential.

Here’s a small vide0 (and my clumsy attempt to define the overlay):

The purpose of this is to show how the AR can be utilised in defining the evolution of brain tumours (glioblastoma) in a patient formerly treated with the standard radiation and chemotherapy (TMZ). I afterward made other videos for presentation- overlays to highlight essential aspects of the radiation therapy department. We are merely constrained by our imagination!

Here’s another video which I made to show how the radiation works (and can be applied in patient education). In Galaxy Note, they call this feature as “live messages”. Again, this was my first attempt at the feature and I am still getting used to the process- using the SPen that comes attached with it.

While not strictly AR (since it is made on a static image); still it highlights its potential.

(The idea behind the smartphones is that we should be able to use them smartly. Formerly this wouldn’t have been possible except by using clever optimization of the onboard operating system. (Incidentally, BlackBerry 10 was way ahead of its time and iOS adopted the unique gestural interface that it pioneered, and then Android 10- nearly 6 years after its debut).

The videos may appear jarring or have excessive “colours” but it is only because of the way it has been processed by WordPress.