Handheld Optical System for Skin Topography Measurement Using Fourier Transform

Until recently, skin surface evaluations were done visually with no universally consistent measuring procedure. One example of this requires standardized digital photographs of the skin to be taken and visually assessed by experienced professionals [1]. In dermatology and cosmetics a need has arisen for an accurate, fast, broadly applicable, simple and cheap in-vivo skin surface measuring system. This kind of optical device would provide accurate and consistent information about the skin surface topography and colour. Its most significant new feature would be the introduction of depth measurement without any contact or invasive procedure. Indeed great advancements have been made in skin surface measuring techniques and in the development of such systems and devices. The most practical and therefore most widely used technique for in-vivo skin surface analysis is fringe projection areal topography [2]. It is done by projecting a fringe pattern onto the measured surface and recording its image. The image is then processed by special computer software which calculates the topography of the surface [3]. Many commercial 3D skin surface measuring systems have been invented, some examples of which can be found in the literature: handheld skin surface measuring device [4], handheld body scanner [5], stationary face measuring device [6], to name a few. Most of these use non-coherent light sources, mainly light emitting diodes (LED). They operate with precision from 0.05 mm to 0.5 mm in ranges from 18 mm × 13 mm × 10 mm to 380 mm × 380 mm × 250 mm with data acquisition times from 0.001 s to 0.1 s. Some experimental skin topography measuring systems may be different from commercial ones. They may use a flashtube as a light source [7], for example. Such devices can be used in medicine to study wounds [8]. A similar principle can also be used in three dimensional foot shape measurements [9], chest wall monitoring during breathing [10], head-to-trunk orientation determination [11] and the study of tissue ablation depths during laser treatments [12]. Similar devices are used for measuring numerous types of objects in science and industry where their adverse health and safety effects are not of concern [13]. This article presents a new optical system that uses the Fourier transform profilometry method [14] for skin surface topography measurement. It was developed primarily for precise and objective monitoring of the skin surface appearance after various laser based skin treatments [15]. In the interest of simplicity, accessibility and costeffectiveness, this skin topography measuring system Handheld Optical System for Skin Topography Measurement Using Fourier Transform Profilometry Laloš, J. – Mrak, M. – Pavlovčič, U. – Jezeršek, M. Jernej Laloš* – Marko Mrak – Urban Pavlovčič – Matija Jezeršek University of Ljubljana, Faculty of Mechanical Engineering, Slovenia


INTRODUCTION
Until recently, skin surface evaluations were done visually with no universally consistent measuring procedure.One example of this requires standardized digital photographs of the skin to be taken and visually assessed by experienced professionals [1].
In dermatology and cosmetics a need has arisen for an accurate, fast, broadly applicable, simple and cheap in-vivo skin surface measuring system.This kind of optical device would provide accurate and consistent information about the skin surface topography and colour.Its most significant new feature would be the introduction of depth measurement without any contact or invasive procedure.
Indeed great advancements have been made in skin surface measuring techniques and in the development of such systems and devices.The most practical and therefore most widely used technique for in-vivo skin surface analysis is fringe projection areal topography [2].It is done by projecting a fringe pattern onto the measured surface and recording its image.The image is then processed by special computer software which calculates the topography of the surface [3].
Many commercial 3D skin surface measuring systems have been invented, some examples of which can be found in the literature: handheld skin surface measuring device [4], handheld body scanner [5], stationary face measuring device [6], to name a few.Most of these use non-coherent light sources, mainly light emitting diodes (LED).They operate with precision from 0.05 mm to 0.5 mm in ranges from 18 mm × 13 mm × 10 mm to 380 mm × 380 mm × 250 mm with data acquisition times from 0.001 s to 0.1 s.Some experimental skin topography measuring systems may be different from commercial ones.They may use a flashtube as a light source [7], for example.Such devices can be used in medicine to study wounds [8].A similar principle can also be used in three dimensional foot shape measurements [9], chest wall monitoring during breathing [10], head-to-trunk orientation determination [11] and the study of tissue ablation depths during laser treatments [12].Similar devices are used for measuring numerous types of objects in science and industry where their adverse health and safety effects are not of concern [13].
This article presents a new optical system that uses the Fourier transform profilometry method [14] for skin surface topography measurement.It was developed primarily for precise and objective monitoring of the skin surface appearance after various laser based skin treatments [15].
In the interest of simplicity, accessibility and costeffectiveness, this skin topography measuring system Highlights • A new handheld optical skin surface topography measuring system has been developed.• The device is based on a DSLR camera with a newly designed flash light grating projector and image recording optics.
• The Fourier transform profilometry method is used for surface topography measurements.• Surface topography is measured and reconstructed quickly and accurately from a single image.
• The surface roughness parameters are used to show the effects of laser skin treatment therapy on several people.
was to be constructed out of as many commercially available products as possible.It was chosen to be based on a digital single-lens reflex (DSLR) camera with a non-coherent flashtube light source around which a camera-specific optical system was to be constructed.

FOURIER TRANSFORM PROFILOMETRY
Fourier transform profilometry is a type of fringe projection areal topography method in which the Fourier series expansion and Fourier transform of a single line grating pattern image are used together with optical triangulation to determine the topography of the measured surface [14].
The basics of crossed-optical-axes-with-paralleloptical-planes geometry is presented in Fig. 1.In this setup, the optical axes of the projector lens and camera lens intersect at a certain point.At that same time their respective optical planes coincide as well and they effectively form one single plane.This is useful because it ensures the proper orthogonal grating pattern projection without unnecessary distortions.The coincidence of both planes is achieved by using the Scheimpflug principle [16].It determines at what angle θ an object plane transmitted through a lens with magnification M p has to be tilted in order to produce an image plane tilted at an angle α 0 relative to the normal plane in the other direction: Therefore, the projection grating has to be tilted at the angle θ before it can be properly projected.
The spatial period of the line grating is p 0 .However, when it is projected through a projector lens with a magnification M p, the projected grating pattern period becomes p = M p p 0 which makes its projected spatial frequency ξ p = 1 / p. Fig. 2 shows the principles of this kind of optical setup.The optical axes of the projector and camera lenses intersect at point O at angle α 0 and their optical planes coincide in plane I.It lies at a distance l 0 away from the camera lens and is perpendicular to its optical axis.A point H(x, y) on the measured object's surface is in the focus of interest.

Fig. 2. Optical schematics in crossed-optical-axes-with-paralleloptical-planes geometry; H(x, y) is a point of interest on the object surface; the optical axes of the projector lens (P) and the camera lens (C) intersect at point O and their optical planes coincide in plane I; (S) represents the camera sensor and (G) the projection grating
A measurement takes the form of a recorded image of a grating pattern projected onto a measured surface.Such an image can be interpreted as multiple rows of light intensity signals modulated in phase ϕ(x, y) and amplitude r(x, y) with the main spatial carrier frequency of ξ p .
This kind of light intensity signal of a single row, as shown in Fig. 3a, can be expanded into the Fourier series in the x dimension which is perpendicular to the grating lines orientation: Here, r(x, y) represents the light distribution on the diffuse surface, ϕ(x, y) the phase shift and the A n a Fourier coefficient.
The Fourier series expansion from Eq. ( 2) is then Fourier transformed and filtered.The Fourier transform is performed only in the x direction, perpendicular to the orientation of the grating lines, while y is kept fixed.
Frequency filtering filters out all frequencies except the fundamental one ξ p and some of its closest neighbours.A Hanning window function centred at ξ p is used [17].Width w of the window function has to be carefully chosen -too wide and the signal may carry too much noise; too narrow and it may lose the sharp edge and discontinuity details [7].Original spatial frequency distribution and its filtering is shown in Fig. 3b.
The filtered Fourier transform is inversely Fourier transformed back to a complex signal: The argument of this complex signal Φ(x, y) = 2πξ p x + ϕ(x, y) contains phase shift information ϕ(x, y) and has to be expressed: This kind of calculation gives principal values of the argument wrapped into an interval of [−π, π].This creates discontinuity phase jumps of 2π for variations greater than 2π, as shown in Fig. 3c.Therefore, a phase unwrapping algorithm is necessary in order to reconstruct the correct continuous phase distribution.The one used here is two-dimensional and is presented in the literature [18].This algorithm provides an optimal balance between quality and speed as it is designed to give priority to pixels with a higher reliability in order to prevent error propagation.
Once phase Φ(x, y) is obtained, trigonometry is used to determine the spatial coordinates of a surface point H(x, y).The coordinates x and y are calculated from their respective pixel positions on the camera sensor u and v. Coordinate z(x, y), however, is calculated from: where a rctan( ( , )), x y x y = + 0 (6) and Here, d g is a half-width of the grating and s p is its mean distance from the projector lens.
Further details can be found in the literature [19] and [20]. of the first signal in the wrapped (black) and unwrapped (grey) state

MEASURING SYSTEM
The core of this skin topography measuring system is a custom-made handheld device based on a commercially available DSLR camera with its builtin flash as a light source to which other optical components are added.The whole system is shown in Fig. 4 and its most important optical dimensions are shown in Fig. 5.
The structural body (B) is made out of a solid plastic block and has the function of rigidly holding the camera lens, the projector lens, the object spacer and the camera in precise relative positions to each other.The camera (S) used here is Nikon D90 DSLR camera [21].Its image sensor is a complementary metal-oxide-semiconductor (CMOS) active pixel sensor of Nikon DX format with dimensions of 23.6 mm × 15.8 mm.It has approximately 12.3 effective megapixels arranged in a 4288 × 2848 array.The camera has a sensitivity from 200 up to 3200 according to the International Organization for Standardization (ISO) standard [22].It is attached to the structural body behind the camera lens on the same optical axis using a Samyang's lens mount adapter for T2 Lenses on a Nikon F mount.
The built-in pop-up flash (F) produces short white light projection illumination.When open, it is in line with the projector lens and projection grating.
The camera lens (C) is positioned at the base of the structural body and in line with the camera sensor and object spacer.It is an Edmund Optics' #32724 MgF 2 coated achromatic doublet lens with an effective focal length of f c = 60 mm and an overall diameter of d c = 25.0 mm.The camera lens setup has a designed optical magnification of M c = 1.0, as seen on Fig. 5.
Its diaphragm is positioned at a c = 20 mm behind the doublet.
The projector lens (P) is similar in design to the camera lens.It is positioned above the camera lens at an angle α 0 = 22.6° against it and in line with the projection grating and flash.The lens is an Edmund Optics' #45-136 MgF 2 coated achromatic doublet lens with an effective focal length of f p = 45 mm and a diameter of d p = 12.5 mm.The projector lens setup was designed to have an optical magnification of M p = 2.5.The diaphragm of the projector lens is positioned at a p = 20 mm behind the doublet.
The projection grating (G) is a horizontal line pattern Ronchi grating.The Edmund Optics' #58-776 1''×1'' (10 lp/mm) Ronchi ruling with a pattern period of p 0 = 0.1 mm was used.It sits in the object plane of the projector lens between the lens and the flash and in line with them.The grating is inclined towards the projector lens at an angle θ = 9.45° in accordance with the Scheimpflug principle so that its projected image falls correctly on the measured object.
The object spacer (L) is an open half-tube attached to the structural frame in line with the camera lens.Its open end is meant to lean against the measured object -skin in this case.The length of the spacer is chosen such that the distance between the open end of the spacer and the camera lens doublet is equal to the designed object plane distance of the camera lens l 0 = 120 mm, thus making the measured area lie in the object plane of the camera lens and in the image plane of the projector lens (I).
The whole optical device weighs 1500 g and measures 270 mm × 135 mm × 160 mm overall.It is robust, light and compact, making it convenient to be used as a portable handheld device.
Experimentally it was found that the flash powered with half of its maximum power provides an appropriate lighting for the purposes of Fourier transform profilometry.Due to a relatively short duration of flash illumination compared to the exposure time, the best possible setting is a shutter speed of 1/200 s at an ISO sensitivity of 200.
Special computer software was developed for the processing of the images.Its code was written in Visual C++ with the aid of the OpenCV and OpenGL libraries.In a few seconds the software analyses the image using the Fourier transform profilometry method.With it, each pixel row is transformed and filtered separately.The pixel rows are then combined into an array for the phase unwrapping and final calculations.This entire process determines the spatial positions of a large number of discrete points on the measured surface -as described in Section 1.
The calibration of the measuring system and corresponding computer software was done using a precisely manufactured reference surface with a known geometry which was measured at incremental distances between 115.5 mm and 125.5 mm from the camera lens.The captured images were imported into the software for the transformation parameter optimization.The parameters were optimized using the Powell's conjugate gradient descent method.This resulted in the standard deviation between the measurements and the reference surface being 0.05 mm.Further information about such a calibration can be found in the literature [7].
This optical system has an effective measuring range of 24 mm × 15 mm × 7 mm in which a precision of 0.05 mm is achieved.The computer software provides data about the position of the discrete points (x i , y i , z i ) on the surface of the measured object which correspond to each pixel in a rectangular grid of the camera sensor.The most interesting aspect of these data for the purposes of surface roughness determination is of course the vertical component in relation to its planar position z i (x i , y i ).
A topography analysis of the measured surfaces is made using four surface roughness parameters: the mean absolute deviation R a , the root-meansquare deviation R q , the skewness R sk and kurtosis R ku [23].In order to calculate these parameters, discrete reference points of a smooth reference plane had to be determined as an average value of all the measured points which lie in the neighbourhood of the corresponding measured point.

MEASUREMENTS
For experimental purposes, five facial skin areas, together belonging to three people, were measured.Thus the areas A1 and A2 belong to person A, areas B1 and B2 belong to person B and area C1 belongs to person C. Each skin surface area was measured twice -before and after the laser skin treatment therapy.While the purpose of this therapy was to smooth the skin, its effects were measured and evaluated here.
To provide a better diffuse reflection of light and a higher line pattern contrast on the semi-translucent and non-homogeneous skin surface, face powder was applied to the measured areas.Three-dimensional skin surface reconstructions from the captured images of the selected samples B2Be (before the therapy) and B2Af (after the therapy) are shown in Fig. 6.The surface roughness parameters were calculated from the measured data points.For the establishment of reference plane points, the neighbourhood O i was chosen in the shape of an octagon with a diagonal of 2 mm.The calculated surface roughness parameters for each of the measured samples are presented in Table 1.
Table 1.The calculated surface roughness parameters for the measured skin samples before (index .Be) and after (index .Af) the laser skin treatment therapy and the relative changes Δ in their absolute values; the parameters are: the mean absolute deviation R a , the root-meansquare deviation R q , the skewness R sk and kurtosis R ku ; χ represents the average value for all the samples and σ χ their standard deviation These statistically calculated surface roughness parameters show that most of the facial skin samples in general have a mean absolute deviation R a between 45 μm and 80 μm and a root-mean-square deviation R q between 55 μm and 110 μm.The parameters also show that the skin samples have low peaks and deep valleys (R sk < 0) and that, for the most part, those peaks and valleys are relatively pointed (R ku > 0).
Regarding the laser skin treatment therapy and its effect on skin surface roughness, it can be clearly seen from Table 1 that almost all of the samples measured after the therapy have lower parameters R a and R q compared to those measured before the therapy.This would suggest that the skin is indeed less rough after the therapy.However, the comparison of the parameters R sk and R ku before and after the therapy from all the samples together is quite inconclusive.

CONCLUSIONS
The measurement principle and design of a new optical three-dimensional skin topography measurement system is described.Its measuring range is 24 mm × 15 mm × 7 mm and its precision is 0.05 mm.Its small size, light weight, as well as its design and shape make it compact and easy-handling -a very convenient portable handheld device.In-vivo measurements and their analyses are done quickly and precisely.
The surface roughness parameters provide a simple way to quantify the surface roughness information of a skin sample.They help compare the surface roughness of different samples or that of a single sample at different times and circumstances.Here, the mean deviation R a and the root-mean square deviation R q parameters show a noticeable reduction in the height and depth of the skin profile after the laser skin treatment in almost all samples.

Fig. 3 .
Fig. 3. Typical processing signals of Fourier transform profilometry method; a) light intensity signal g(x, y) segment of one pixel row of the grating image taken before (black) and after frequency filtering (grey); b) spatial frequency distribution of the first signal (grey) and the window function for frequency filtering (black); c) phase Φ(x, y)of the first signal in the wrapped (black) and unwrapped (grey) state

Fig. 6 .
Fig. 6.Experimentally measured areas of person B's facial skin (top) and three-dimensional skin surface reconstructions from the measurements of the area B2, before (sample B2Be) and after (sample B2Af) laser skin treatment therapy