### Welcome! Thank you for participating in this very short and anonymous on-line experiment; please follow the instructions below …

**Background:** Interactive Machine Learning (iML) can be defined as “algorithms that can interact with agents and can optimize their learning behavior through these interactions, where the agents can also be human [1], [2].” This “human-in-the-loop” can be beneficial in solving computationally hard problems [3], where human expertise can help to reduce an exponential search space through heuristic selection of samples. Therefore, what would otherwise be an NP-hard problem, reduces greatly in complexity through the input and the assistance of an human agent involved into the learning phase.

**Experiment:** In this on-line experiment we evaluate the effectiveness of the iML-”human-in-the-loop” approach, particularly in opening the ”black box”, thereby enabling a human to directly manipulating and interacting with an algorithm. For this purpose, we selected the Ant Colony Optimization (ACO) framework, and apply it on the Traveling Salesman Problem (TSP) [4], [5] which is an NP-hard problem and is of high importance in solving many practical problems in health informatics, e.g. in the study of proteins etc.; however, you can see the nodes (vertices) as cities and may ask the following question: *“Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once?”*

**Instruction – please follow these for all three data sets:**

Your goal is to find the shortest connection between all cities, and you can adjust with “Change Weight to …” the probability that a path is taken. Remark: It works with *any* Browser, but the use of Chrome brings faster results. It is recommended to watch the short instruction video below (2 minutes) first to fully understand what to do.

- 1) Press “Start” to initialize the algorithm
- 2a) Click “Pause/Resume” to pause the algorithm and to do your changes
- 2b) You may now select two cities by clicking on the respective nodes
- 2c) With “Change Weight to max/min” you can adjust the importance of the path between these two selected cities
- 3) Click on “Pause/Resume” to continue the algorithm
- 4) The steps above can be repeated as often as you wish
- 5) After 250 Iteration the algorithm automatically stops
- 6) Please continue with the next dataset

Thank you very much for your participation. If you wish to be informed about the outcome

of the experiment, please contact a.holzinger@hci-kdd.org

**References:**

[1] Holzinger, A. 2016. Interactive Machine Learning for Health Informatics: When do we need the human-in-the-loop? Springer Brain Informatics (BRIN), 3, (2), 119-131, doi:10.1007/s40708-016-0042-6.

[2] Holzinger, A. 2016. Interactive Machine Learning (iML). Informatik Spektrum, 39, (1), 64-68, doi:10.1007/s00287-015-0941-6.

[3] Dossier: Interactive Machine Learning for Health Informatics

[4] Wikipedia: Travelling salesman problem (last visited: 01.08.2016, 18:00 CET)

[5] Google Scholar: Traveling salesman problem (last visited: 01.08.2016, 18:05 CET – 46,800 results)