Bonestagram - DICOM visualization made fun

Rationale

While the cost of production of medical images such as Computed Tomographies, Magnetic Resonances and Ultrasound is decreasing, the number of studies that deal with these images is steadily increasing. These medical images are important especially for diagnosis of bone or internal tissue injuries. So, this makes it possible that many individuals have multiple scans of different parts of their bodies, which they can take home (Figure 1).

Fig 1. Examples of two medical images

The problem we want to tackle is: a vast majority of individuals do not collect their medical images produced for diagnosis, however it has an important advantage to do so. For example, in case of a new injury, historic data about previous existing conditions can help physicians in their treatment plan.

To encourage individuals to collect their medical images, we want to create a visualization app that is easy to use and through an interaction with the user’s web cam can visualize the medical image together with the injured part of the body (e.g. it can lay image of a bone over user’s arm or an image of the brain over user’s head). By moving the body part, the overlaid medical image should also be moved. Finally, the video or a snapshot (picture) produced by the app can for example be shared with family and friends (Figure 2).

Fig 2. Gamification - overlaying a medical image over the image of a user through the web cam

The gamification, i.e. allowing users to overlay the images or their renderings to webcam images or selfies, can for example be done similarly to Face Swap apps.

Approach

The first part of this project is represented by data collection. Different DICOM images (modalities and object of interest) need to be obtained in order to understand the diversity and nature of the data.
The second part of the project will be represented by the formulation of an intuitive, fast DICOM viewer that can be run in any browser.
The third stage of the project is represented by adding gamifying features to the viewer that encourage the user to use the software and share his/her renderings e.g. in the social context.

Challenges

  • Privacy
  • Heterogeneous data
  • Complete JavaScript frontend, platform-independent solution
  • Simple, non-expert UI design for highly non-trivial data

Involved toolkits or projects

Sample Data sources:

Open Source related projects:

Needed skills

Fluency in JavaScript and Git

  • Knowledge in Computer Vision
  • Knowledge in best practices in UI/UX
  • Knowledge about medical images is beneficial but not required
  • Very high motivation and good independent worker
Mentors

Christian Dallago (TU Munich, RostLab), Hesam Rabeti (TU Munich), Guy Yachdav (BioSof)