Supervised Visual System for Recognition of Erythema Migrans, an Early Skin Manifestation of Lyme Borreliosis

For machine-supported detection of segments of interest in medical images and understanding the content with learning algorithms [1] and [2], neural networks [3] and [4], naive Bayes, support vector machine (SVM) and others [5] are among key diagnostic tools of modern medicine. In several tasks, processing of visual content consists of image processing operations which include acquisition of image, pre-processing, segmentation procedure (performed with GrowCut (GW), random walker (RW) and other methods), attribute (feature) extraction and classification [2]. Improvement of existing analyses of medical images leads to better medical diagnosis [6]. This article represents a new effort to establish machine-supported analysis of medical images related to erythema migrans, an early manifestation of Lyme borreliosis. Lyme borreliosis is a multisystem disease [7], caused by Borrelia burgdorferi sensu lato and transmitted by a tick bite. The early course of Lyme borreliosis is characterized by an expanding skin lesion named erythema migrans. The lesion typically appears about one week after a tick bite (range, 3 to 30 days; median 7 to 10 days) as a small redness at the site of the bite and expands over a period of days to weeks, often to an oval lesion with central clearing [8] and [9]. In Europe, about one third of patients with erythema migrans have a concomitant viral-like illness (the " summer flu ") characterized by myalgias, arthralgias, headache, and fatigue. Fever is rarely present. Without treatment, erythema migrans skin lesion resolves itself within several weeks to months but the infection may progress and affect skin, nervous system and/or joints, and less frequently eyes and/or heart [7], [8] and [10]. Early recognition and proper antibiotic treatment can successfully prevent harmful effects of Lyme borreliosis and enable faster disappearance of erythema migrans. Since visual recognition is part of the clinical diagnosis of erythema migrans, we decided to design a visual system that would serve as a complement in the diagnosis of erythema migrans either at the medical exam or at home through smartphones. However, execution of erythema migrans edge detection is a challenging task since the colours of erythema migrans typically vary from very pale to very intensive red, less frequently blue or brown shades or even a combination of two colours. Furthermore, the diameters of erythema migrans lesions in our database vary from 2 to 30 cm, less frequently more than 30 cm. Most literature on visual detection and recognition of skin patterns relates …


INTRODUCTION
For machine-supported detection of segments of interest in medical images and understanding the content with learning algorithms [1] and [2], neural networks [3] and [4], naive Bayes, support vector machine (SVM) and others [5] are among key diagnostic tools of modern medicine.In several tasks, processing of visual content consists of image processing operations which include acquisition of image, pre-processing, segmentation procedure (performed with GrowCut (GW), random walker (RW) and other methods), attribute (feature) extraction and classification [2].Improvement of existing analyses of medical images leads to better medical diagnosis [6].This article represents a new effort to establish machine-supported analysis of medical images related to erythema migrans, an early manifestation of Lyme borreliosis.
Lyme borreliosis is a multisystem disease [7], caused by Borrelia burgdorferi sensu lato and transmitted by a tick bite.The early course of Lyme borreliosis is characterized by an expanding skin lesion named erythema migrans.The lesion typically appears about one week after a tick bite (range, 3 to 30 days; median 7 to 10 days) as a small redness at the site of the bite and expands over a period of days to weeks, often to an oval lesion with central clearing [8] and [9].In Europe, about one third of patients with erythema migrans have a concomitant viral-like illness (the "summer flu") characterized by myalgias, arthralgias, headache, and fatigue.Fever is rarely present.Without treatment, erythema migrans skin lesion resolves itself within several weeks to months but the infection may progress and affect skin, nervous system and/or joints, and less frequently eyes and/or heart [7], [8] and [10].Early recognition and proper antibiotic treatment can successfully prevent harmful effects of Lyme borreliosis and enable faster disappearance of erythema migrans.
Since visual recognition is part of the clinical diagnosis of erythema migrans, we decided to design a visual system that would serve as a complement in the diagnosis of erythema migrans either at the medical exam or at home through smartphones.However, execution of erythema migrans edge detection is a challenging task since the colours of erythema migrans typically vary from very pale to very intensive red, less frequently blue or brown shades or even a combination of two colours.Furthermore, the diameters of erythema migrans lesions in our database vary from 2 to 30 cm, less frequently more than 30 cm.
Most literature on visual detection and recognition of skin patterns relates to skin cancer, but there are no journal articles related to visual pattern recognition of erythema migrans.Therefore we analysed related domain articles for computer supported pattern recognition which is a two step procedure.In the first step, the object of interest is identified based on edge detection.As a result of rapid development in the image processing of skin lesions, a modified skin cancer segmentation of colour-texture regions in images and video (JSEG) [11] was proposed.The basic idea behind the algorithm is a division of the segmentation process into two independent stages: colour quantization and spatial distribution.Tang [12] propose a multi directional gradient vector flow algorithm.They modified the algorithm in a way that anisotropic diffusion (AD) filter is less sensitive to noise.The proposed AD filter uses adaptive threshold and a new way of calculating the gradient, which can effectively remove hair.Another very accurate multimodal segmentation technique suggested by Yuan et al. [13] is based on the region fusion and graph division of narrow energy band.The method is effective on complex characteristics of skin lesion changes such as topology, weak or false edges, and asymmetry.In segmentation of tumour, Schmid [14] proposed a colour based segmentation method Fuzzy c-means.Furthermore, Gomez et al. [15] developed an unsupervised algorithm called independent histogram pursuit, which can be easily combined with the majority of classification algorithms, while Zhou et al. [16] developed another segmentation method mean shift based on fuzzy c-means, which consumes less computing time with respect to other methods, while the detection of skin lesion edge remains precise and effective.
Edge detection is often obstructed by skin lines and hair.Xie et al. [17] similarly to Abbas et al. [18] removed hair using line detection in combination with exemplar-based inpainting.
After successful edge detection and extraction of skin lesion area, the second step is performed in order to recognize the lesion on the bases of attributes calculated from the first step.SVM [19], Neural network [20] and AdaBoost [21] classifiers were applied for recognition of a skin cancer lesion [19] to [21].
Another related visual detection and recognition system is classification of colour and texture features of human tongue that reflects the health status of a patient [22].A segmentation method maximal similarity based region merging (MSRM) [22] and [23] contains a built-in initial flood segmentation method.Results are reflected in the precise segmentation of the human tongue.Furthermore, Shi et al. [24] proposed a novel approach for tongue image segmentation called C 2 G 2 FSnake.They combine a geometrical snake model with a parameterized gradient vector flow (GVF) snake model.For the health status analysis of a patient, a classification transductive SVM method is proposed by Zhang et al. [25].
Roullot et al. [26] proposed semiautomatic measurement which was used for detecting allergic rash.Semiautomatic measurements are useful for different kinds of allergic reactions that need to be visually detected and classified.Huttunen et al. [27] also proposed segmentation where user has to point the position of an allergic rash.
In this article, a supervised visual system for recognition of erythema migrans is presented.The article is divided into several sections.The first section describes the use and composition of the system, while the second section contains a description of image database, user marker inputs, segmentation and "finger draw 1" (FD1) user marker input, shape properties and attribute calculation and classification methods used in the experiment.This section also describes the methods for evaluation of segmentation and classification.In the third section experimental results and discussion for the segmentation and classification of skin lesions are presented.In the last, conclusion section, our findings are summarized.

SYSTEM FOR VISUAL RECOGNITION OF ERYTHEMA MIGRANS
The primary targeted application of our system for visual recognition of erythema migrans is a mobilemarket system for smartphones with touchscreen display as a marker input.

User Input -Touchscreen
Touchscreen interfaces are used in a wide range of technologies, from mobile phones to in-car systems [28].A smartphone touchscreen is an electronic visual display that can recognize finger touch either with responding to a mechanical force applied to the material, infrared-based approaches or others [29].In our experiments, touchscreen display was simulated with laptop touchpad that allows the user to draw a marker input with a finger.

Image Analysis
The process of skin lesion image analysis [19] and [20] consists of two steps: edge detection and recognition of skin lesion.Since edge detection affects the accuracy of subsequent step mentioned above, the process of edge detection is of vital importance [18].Many features such as asymmetry, border irregularity and other shape properties are calculated from the edge of the skin lesion [19].
Most commonly, skin lesion edge detection is divided into three steps.The first step is called image scaling and is essential for execution of the following two steps.Image pre-processing is the second step in which data is processed in order to improve contrast [30] and remove noise.The third step in edge detection is segmentation.
Typically, segmentation methods need a user marker input that defines the position of seed points for extraction of skin lesion area.This is a step where skin lesion is extracted from the captured image: the image is divided into foreground that includes the skin lesion and the background that includes all other parts of the image such as skin without the lesion, different clothing and the surrounding environment.The background is then excluded from further analysis.The most commonly used segmentation methods in medical imaging [31], and therefore the most suitable for segmentation of skin lesions are contour tracking methods, region growing method [26] and [32], GCmethod [33], RW method [34], active contours [35] and [36], level sets, geodesic active contours, Graph Cut method [37] and [38], watershed algorithms [39], scale multiplication algorithms [40] and [41], generic segmentation models and deformable segmentation models [42] and [43].
Similar to the edge detection, recognition of skin lesion is divided into steps.The first step in recognition of skin lesion is shape properties and attribute calculation where attributes are identified.The second step is called classification of skin lesion.
Fig. 1 illustrates edge detection and recog-nition of erythema migrans.The structure of our system in Fig. 1 corresponds to a modern visual recognition system.The novelty lies in methods and input type.The methods for segmentation and classification used in the experiment are described in the next section.

User Marker Inputs
Four different user marker inputs were used in the experiment.In the user marker input (FD1), the user has to draw a curve marker around the skin lesion with a finger.In the case of "finger draw 2" (FD2), the inside curve marker is also needed.In the "point marker 6" (PM6) user marker input, the user clicks six evenly distributed points around skin lesion.In the case of "point marker 2" (PM2), the user has to click one inlier and one outlier marker point.These marker inputs were used as inputs to segmentation methods GC k > 6 (GC-FD1), GC k = 6 (GC-PM6), MSRM k > 6 (MSRM-FD2), RW k = 2 (RW-PM2) and GC k = 2 (GC-PM2).Variable k is the number of marker points.

Segmentation and FD1 User Marker Input
Our approach is based on the GC method introduced by Vezhnevets and Konouchine [44] that is also used for melanoma detection proposed by Ayoub et al. [33].
In order to successfully detect potential erythema migrans skin lesion, segmentation methods need several evenly distributed inlier and outlier points.We improved the functioning of GC method with our FD1 marker input in a way that the user draws a curve around potential erythema migrans skin lesion.The type of input was enabled with introduction of multimedia interactive terminals.From the outlier curve or the outlier points, we automatically calculate inlier points by uniformly decreasing outlier curve using Matlab function maketform [45] and affine transformation.Matrix for affine scaling [46] transformation is as follows: where experimentally obtained scaling factor values for our image database are v x = v y = 0.3.
During our experiments we compared the influence of segmentation methods: GC-FD1 with GC-PM6, MSRM-FD2 [22] based on initial watershed segmentation, Grady's RW-PM2 [47] and GC-PM2.In the third section besides segmentation results for all methods, we also present the results in combination with MSRM-FD2 and GC-FD1 segmentation methods related to erythema migrans computer-aided diagnosis.

Shape Properties and Attribute Calculation
Generally, second or third order splines [48] are used for shape description where the number of spline anchors has to be minimized.Due to the fact that the analysed skin lesions are of oval shape, we used basic shape properties.These shape properties of segmented potential erythema migrans skin lesion were calculated with Matlab regionprops [49] function.The properties were: minor axes b, major axes a, orientation o, eccentricity ε and circumference c properties.Attributes, calculated from the shape properties and used as an input for classification algorithms were: • eccentricity: Matlab regionprops function, • small and large axis ratio: b/a , • ellipse focus: • ellipse angular ε: where c is circumference, r a = a / 2 and r b = b / 2, • orientation: Matlab regionprops function.

Classification
We classified potential erythema migrans skin lesions with two types of experiments.In the first type the default class was ellipse, where the ground truth was determined by the expert.The second type includes the determination of class erythema migrans skin lesion and the ground truth was also determined by the expert.Table 1 shows the number of true and negative cases for the two types of experiments.Attributes were calculated with equations in shape properties and attribute calculation section which were used as inputs to Matlab implementations of classification algorithms naive Bayes, SVM, AdaBoost, random forest and neural network.
Accuracy (Acc) [50] was calculated to estimate the performance of classifiers and is defined as: where numerator (true positive and true negative) are correctly classified skin lesions to class ellipse or erythema migrans and denominator (true positive, true negative, false positive and false negative) represents all skin lesions, i.e. 143.

Evaluation of Segmentation and Classification
For comparison between segmentation of image lesions where the MSRM, RW and GC segmentation methods using different user marker inputs were used on one side, and manually labelled image lesions where these labels were used as ground truth defined by an expert on the other side, we used a known method for segmentation evaluation [23].Afterwards, automatically segmented labels and ground truth were applied for computation of true positive rate (TPR) and false positive rate (FPR).The TPR is defined as: where the numerator is the number of correctly classified object pixels and the denominator the number of total object pixels in the ground truth.Equation for FPR is defined as: where the numerator is the number of background pixels classified as object pixels and the denominator the number of background pixels in the ground truth.Moreover, we calculated the mean value of true positive rate (TPRA) and the mean false positive rate (FPRA) for all 143 images.The higher the TPRA and the lower FPRA get, the better the method.We also calculated the corresponding standard deviation of TPRA (σ TPRA ) and FPRA (σ FPRA ).Classification to class ellipse and erythema migrans was evaluated with K-fold cross-validation [50] with K = 10 for estimation of predictive model performance in practice.For every fold of both types of experiments, a paired student's 't test' [50] was performed between classifiers combined with GC-FD1 and classifiers combined with MSRM-FD2.In addition, a standard deviation for the Acc results (σ Acc ) was calculated for the second type of the experiment.

EXPERIMENTAL RESULTS AND DISCUSSION
Table 2 shows segmentation results for five different approaches with k marker points for all 143 skin lesions.We tested GC-PM2 and RW-PM2 in the way where a non-expert computer specialist clicked one inlier and outlier marker point placed on the same position for both methods.For GC-PM2, TPRA was 36.64% higher than for RW-PM2.GC-PM2 has a higher FPRA value but lower σ FPRA .Our GC-FD1 yielded comparable results to the state of the art MSRM-FD2.In case of GC-PM6 the goal was to estimate how many marker points are needed to achieve results that are comparable to GC-FD1.In addition to Table 2, Fig. 2 presents the performance of the first three best segmentation approaches for the three skin lesions.Manually drawn ground truth (first line on Fig. 2) was compared with segments computed by GC-PM6 (second line on Fig. 2) , GC-FD1 (third line on Fig. 2) and MSRM-FD2 (fourth line on Fig. 2).The marker points or lines for the three methods are presented in the first column.
Fig. 2 shows that our GC-FD1 approach satisfactorily detects the proper shape for all three segmented skin lesions, where results obtained with GC-FD1 in Table 5 additionally support this claim.For better understanding of segmented lesions in Fig. 2, TPR in Table 3 and FPR in Table 4 are presented.
In the last part of this section, experiments for classification of skin lesions are presented.Tables 5  The results in Table 5 show that best classification Acc results for ellipse recognition, which is the most common shape of erythema migrans, are the ones where classifiers were combined with GC-FD1.We also performed paired student's 't test' and confirmed that results for all classification algorithms in combination with GC-FD1 for the class ellipse are significantly better than the results obtained with the classification algorithms combined with MSRM-FD2.Table 6 shows that GC-FD1 in combination with all classifiers achieves best Acc and its standard deviation (σAcc) results for class erythema migrans.Calculation of paired student's 't test' for the Acc for every fold showed that results between GC-FD1 and MSRM-FD2 in combination with naive Bayes, SVM and AdaBoost are significantly better.However, the test did not prove significance for the random forest and neural network classifier.
For the automatic edge detection of a human tongue, the MSRM method needs a prior knowledge that the tongue body usually locates in the centre of the tongue image.In this case a circle marker is positioned in the centre of the tongue image.Such a scenario of marker positioning is not possible for potential erythema migrans edge detection, as potential erythema migrans are of different sizes.In addition, potential erythema migrans have different shades of colours which make the task even more challenging and require precise placement of marker points.Although the results for the class erythema migrans are satisfactory, shape recognition does not give the final diagnosis of erythema migrans, since the diagnosis also demands answers from the patients.Our image database was acquired with equipment with better technical capabilities than that available in smartphones since the quality of the lens in our Canon EOS 600D is better than the quality of the lens available in smartphones.However, it is worth noting that smartphone external lens can already substantially improve the quality of image.Regarding the fact that the development of smart devices, especially smartphones, is evolving quickly we expect that smartphone technology will reach a satisfactory image quality level in a few years.Currently, the best smartphone camera available on the market has a 41 megapixel resolution which is approximately 23 megapixel higher compared to Canon EOS 600D.

CONCLUSION
The novelty of our approach stems from the use of pattern recognition methods in the field of infectious diseases including erythema migrans, development of new features that can be used for medical purposes, and from the mass introduction of multimedia interactive terminal, available also on smartphones.This type of input enables simple finger drawing to improve visual detection.However, as experimentally shown, different inputs and different methods give different results for segmentation and classification.From experimental results we can conclude that GC-PM2 performed better than RW-PM2 segmentation method and that GC-FD1 was comparable to the MSRM-FD2 (see Table 2) segmentation method.For satisfactory segmentation results of potential erythema migrans six or more evenly distributed marker points around potential erythema migrans are needed.From a practical point of view it is easier and faster for a user to draw a curve around the potential erythema migrans than clicking six or more evenly distributed marker points using the multimedia interactive terminal technology.The results of our experiments indicate that this technical innovation improves the diagnostic results in a reasonable manner.The experiments showed that the GC-FD1 approach is robust to various shapes and colours of potential erythema migrans.Moreover, our approach gives better classification Acc results for the ellipse class and better Acc and σAcc results for erythema migrans class (see Tables 5 and  6).
In near future we intend to improve the Acc of the existing visual system for recognition of erythema migrans with colour and Gabor based attributes.However, for efficient diagnosis of erythema migrans, hybrid method will be needed.We plan to combine medical image analysis with patient textual data involving machine learning methods.The reason for combining image analysis with text description usage is based on current best practices in treatment of erythema migrans.

ACKNOWLEDGMENTS
The authors would like to thank the rest of Department of Infectious Diseases of the University Medical Centre Ljubljana personnel for helping with their expertise on Lyme borreliosis diagnosis and helping create an image database for visual recognition of erythema migrans.Our work was partly financed by European Union.The study was approved by the Medical Ethics Committee of the Ministry of Health of Republic of Slovenia, No 130/05/12, which assesses the compliance with the Helsinki declaration.

Fig. 2 .
Fig. 2. Segmentation methods comparison for the three skin lesions

Table 1 .
True and negative cases for ellipse and erythema migrans class

Table 2 .
Comparison of segmentation methods for 143 skin lesions and k marker points

Table 3 .
TPR segmentation evaluation for all three segmented skin lesions in Fig.2

Table 4 .
FPR segmentation evaluation for all three segmented skin lesions in Fig.2

Table 5 .
Classification to class ellipse for 143 skin lesions

Table 6 .
Classification to class erythema migrans for 143 skin lesions