Surface generation from point cloud.pdf

Proceedings of COBEM 2009 20th International Congress of Mechanical Engineering
Copyright © 2009 by ABCM November 15-20, 2009, Gramado, RS, Brazil
3D SURFACE GENERATION FROM POINT CLOUDS ACQUIRED FROM
A VISION-BASED LASER SCANNING SENSOR
Gerardo Antonio Idrobo Pizo, gerardo_idrobo@unb.br
José Mauricio S. T. Motta, jmmotta@unb.br
Universidade de Brasília, Faculty of Technology, Department of Mechanical Engineering
70.910-900, Brasilia – DF, Brazil
Abstract. This paper presents special algorithms to be used with a specially developed vision-based laser scanning
optical sensor to generate 3D surface digital models aiming at repairing defects on turbine blade geometric profiles
with welding robots. Currently, most of the systems for 3D surface modeling based on vision make use of angular
position sensors for measuring the rotation of a source of light. These sensors impose restrictions in the volume and
range of work of the scanner and the reconstruction is typically based on equations of triangulation which produce
measurement errors due to the non-linear relationship between angle and distance, increasing severely the errors as a
function of the distance from the object targeted. To overcome these difficulties, specialized computer vision algorithms
were developed in this research to eliminate the use of angular measuring sensors and positioning systems, reducing
costs and errors and also allowing larger measurement distances. The system is also self-calibrated with no need of
other measurement systems for camera and laser calibrations. The system developed so far capture object target points
and calculates their 3D position producing a cloud of points. To generate a 3D surface with sufficient accuracy from
the cloud of points, triangular meshes are generated to be used with NURBS modeling and these parametric surfaces
are optimized by using adaptive filters. The system is compact, easy to transport and it is shown the complete system
architecture with two laser diodes and a single camera. Experimental procedures have been specially developed to
estimate the accuracy of the measurement system by locating the center of spherical targets from the surface generated
from the sphere, with promising results. Theoretical basis, hardware implementation and practical results obtained are
also discussed.
Keywords: 3D Vision Sensor; Robot Vision; Surface Modeling; Laser Scanning
1. INTRODUCTION
This article presents and discusses an ongoing research on 3D surface modeling from point clouds acquired from a
special vision sensor system embedded on a robot arm dedicated to repair surface flaws on hydraulic turbine blades
capable to make weld beads in layers. For the robot to have the 3D surface map of the blade it is necessary to develop a
computer system to transform a 3D cloud of points in a 3D surface digital model.
The exact representation of the original surface is not always possible, but only an approximation of it. This
approximation should keep not only the geometric information of the real object, but also needs to offer a realistic and
precise map of the surface.
Surface modeling can be performed by interpolation or by approximation routines. Interpolation requires that the
surface passes exactly through all the points, but if the data contain some noise added during the acquisition process, an
approximated surface is more appropriate. In this case, it is only required that the surface passes close to the control
points.
To assess the quality of a 3D object model is not easy when the relations of connectivity between points that form
the model are not known, and mostly, when the cloud of points used to model the object is contaminated with noise
from the image acquisition process. Such noise can be eliminated or attenuated through filtering. The main objective of
filtering is to improve the quality of the signal according to a performance criterion (Chui, 2008).
This article discusses adaptive filters as one way to tackle the problem of image noise filtering, since many of the
noise that pollute the cloud of points appear as a stochastic process that cannot be attenuated successfully through
common digital filters (low pass, high pass, band pass). (Manolakias, 2000). This article also addresses the combination
of two techniques for 3D surface modeling of objects: reconstruction of surfaces by triangular mesh grids and
parametric functions - NURBS (Non-Uniform Rational B-Splines) (Farin, 2001, Anderson, 2000, Roth, 1998). The
algorithm performance is evaluated experimentally by comparing the geometric dimensions of a sphere model
constructed by the system to the actual sphere previously measured with standard measurement devices, showing that
the system can be used to construct 3D object surface models with known geometric accuracy.
2. 3D VISON SENSOR AND IMAGE FILTERING
The 3D vision sensor constructed and used to acquire cloud of points is a laser scanner based system with a high
resolution camera and two laser diodes to project two light planes on the object and uses computer vision algorithms to
extract information directly from the image. With this arrangement it is possible to remove sensors to measure angular
positions of the laser diode, improving accuracy. Some details of this system can be found in Ginani and Motta (2007).
From the geometric distortions of the projected light in the object and knowing the intrinsic camera parameters it is
ABCM Symposium Series in Mechatronics - Vol. 4 - pp.44-53
Copyright © 2010 by ABCM

possible to calculate the 3D coordinates of the object surface points, generating point clouds from the light strips
scanning the object surface. However, as in any other vision system, the camera and the environment generate noise.
Figura 1. Vision sensor hardware VISSCAN-3D
The cloud of points acquired can be modeled as the output of an input image x(n) which passes through a FIR
(finite-duration impulse response) filter (corresponding to the image processing) with impulse response (Fig. 2)
(Haykyn,1991), with finite length N. Random noise s(n) can be added to the output of the impulse
response and
(1)
is the cloud of points contaminated by noise.
The attenuation of the random noise signal s(n) is proposed here to be achieved as in Fig. (2). The output of the
adaptive filter w is y(n)
(2)
Where . This estimative is subtracted from the contaminated signal, generating the error
(3)
In an ideal case w = h and e(n) = s(n), such that the noise would be completely eliminated. However, the approximation
of h by a filter w of finite length restricts full noise elimination.
Figure 2. Block diagram showing an adaptive filter to attenuate noise in point clouds
2.1. Adaptive Filtering
The choice of an adaptive filter (Haykin, 1991; Diniz, 1997) is often based on speed of convergence and accuracy,
numerical complexity and structure stability.
The speed of convergence is defined as the number of iterations required for the algorithm to be close enough to the
optimal solution with stationary signals. The higher the speed of convergence, the faster the algorithm adapts to an
environment with unknown stationary characteristics.
Numerical complexity is the number of operations to perform a complete iteration of the algorithm. Depending on
the adaptive algorithm, there may be numerical instability in the filter. The accumulation of errors due to accuracy
restrictions can make the algorithm to move away from the ideal state. It is possible that the deviation is of such a nature
that the errors will accumulate indefinitely and the filter will be left to overflow and instability.

The chosen adaptive algorithm was the LMS (Least Mean Square) as it exhibits a higher convergence speed (about
10 times higher) than other known algorithms such as Kalman and Wiener filters and slightly lower than RLS
(Recursive Least Square), mainly due to its simplicity.
With the LMS algorithm, the vector of coefficients is updated in the opposite direction to the gradient vector
obtained from the instantaneous quadratic error, i.e. (Fisher et. al.,2000):
(4)
, where ' is the step of the algorithm. Thus, the equation for updating the coefficients of the LMS algorithm is given
by:
(5)
with
3. ORGANIZING POINT CLOUDS WITHIN A TRIANGULAR MESH
Over the years, several techniques have been proposed to solve the problem of surface reconstruction from point
clouds, using concepts from computational geometry to models based on partial differential equations (Roth,1998).
Most of the methods can be classified as implicit or parametric (Hoppe et al,1992;Yvart ,2005). More recently,
Boissnoat and Cazalis (2002) classified the methods into four groups: 1) Methods based on local projections:
approximation is performed treating the surface as a function defined in a local reference domain; 2) Sculpting
Methods: the methods are based on the removal of triangles from a spatial arrangement, as the Delaunay triangulation;
3) Implicit Methods: the methods are based on an implicit definition of a function of distance with signal f and in the
realization of a triangulation approximation of the isosurface f (x) = c. This process of approximation of the function f is
performed using techniques of extraction of isosurfaces; 4) Methods based on deformable models: from an initial shell
(membrane) deformations are applied to minimize a function of energy and get closer to surface.
All kinds of methods listed above are based on the functional characteristics of the algorithm. However, other types
of classification are possible, such as divisions based on computational complexity or quality demanded by the
sampling algorithms.
Among the previous methods, a method based on deformable models was chosen because it offers more speed in
the process of organizing the point clouds within a mesh. Also, it minimizes the computational cost as the 3D surface is
modeled.
The algorithm is divided into three steps, and each step is divided into several sub-steps (Chui, 2008): (a) Projection
of the cloud of points on the mesh grid, (b) Nodal point repositioning, (c) Reconstruction of the 3D triangular mesh
from the 2D triangular mesh.
3.1. Projection of the cloud of points on the mesh grid
1. A cloud of points is projected from R3
to R2
on the plane (Fig. 3a and Fig. 3b).
2. Adjust the cloud of points inside a parallelogram (angle of 60 ° (Fig. 3c). This polygon involves all points of
the sample that are to be analyzed in all subsequent steps of the algorithm.
3. Divide the polygon into horizontal and vertical lines separated of a distance t. This separation influences the
resolution of the 3D surface (Figure 3 (d) and Figure 3 (e)).
4. Diagonal lines are projected from the left to the right of the polygon, forming equilateral triangles (Fig. 3f).
(a) (b)

(c) (d) (e) (f)
Figure 3. Projection of the cloud of points on the mesh grid
3.2. Nodal Point Repositioning
Nodal point repositioning is a essential step to adjust a mesh nodal point to the a cloud point. In each mesh nodal
point a circle is drawn connecting the adjacent nodal points, in order to select only one point among many that are
within the area enclosed by the circle. The point selected is the one that is closer to the node of the mesh inscribed in the
corresponding circle (Fig. 4a). The procedure is repeated for the other node points, and those points that have not been
selected will be deleted.
(a) (b)
(c) (d)
Figure 4: Repositioning of nodal points. (a) Point selection approach, (b) An example case for choosing projected points
using Critirion A, (c) A projected point is selected as the new nodal point position using Critirion B, (d) Projected point
being chosen using Critirion C.
For example, in Fig. 4a, the nearest projected point will be considered as the new position of the nodal point Pi, j.
However, if there are two or more points satisfying closer distances, the following criteria should be established:
Criterion A. If two or more projected points are equal in distance to P(i, j) but are located in different triangles, the
projected point at the top (i.e., higher y) will be chosen. For example, in Fig. 4b, five points are projected in equal
distances from the nodal point P(i, j). Items found in the upper nodal point will be selected. The choice of the new
position of the nodal point will be explained using Criterion B.
Criterion B. If two or more projected points are equal in distance in the same horizontal line (Fig. 4c), but in
several different triangles, the projected point in the triangle is chosen as the new position of the nodal point Pi, j.
Criterion C. If two or more projected points are equal in distance in the same horizontal line and same triangle, the
projected point with higher value x is chosen (Fig. 4d).

3.3 Reconstruction of the 3D triangular mesh from the 2D triangular mesh.
The 3D triangular mesh can be reconstructed next to the process of displacing the nodes of the 2D mesh (Fig. 5a)
by using the depth values of each displaced nodal point (criterion A, criterion B or criterion C ) (Fig. 5b).
(a) (b)
Figure 5. (a) 2D triangular mesh constructed from the projected points next to nodal points repositioning. (b) 3D
triangular mesh is constructed based on its corresponding 2D triangular mesh.
4. NURBS SURFACE REPRESENTATION
The construction of a 3D model using parametric surfaces is a process that requires an adjustment of approximation
between the geometrical structures of a structured mesh in the space R3
(Eck and Hoppe, 1996).
NURBS (Non Uniform Rational Basis Spline) is an industrial standard tool for modeling and design of simple and
complex geometries. The reasons for such widespread use are: a) based on common mathematical methods to represent
free form objects; b) provides a high flexibility in the design of shapes; c) the processing time is reasonably small; d)
are generalizations of curves and surfaces of B-spline and Bezier.
The NURBS surfaces are curve sets of the same type that can be defined as (Park. and Sang, 1999).:
!
" " # $ % & ' ( )# &*# &
+
&,-
.
#,-
" " # $ % & ' ( )# &
+
&,-
.
#,-
(6)
where Di, j are the control points; Wi, j are the weights defining how significant will be the control point on the curve; Ti,p
and Tj,q are the B-spline functions defined in two parametric directions. The number of control points is defined by the
surface degree in each parametric direction ("u" and "v"). Therefore, in each direction there is a number of control
points. The B-spline functions are functions of the nodes (ti) that are increasing sequences. So, one can define a B-spline
function as:
/0 1 2
30 4 4 305
6 7
8 (7)
with
/0 1
% 9#
9#:;<= 9#
> /0 ?
9#:; %
9#:; 9#:=
> /05 ? (8)
The nodes are represented by a list of numbers that is commonly called the node vector. The node vector needs to
be a sequence of numbers equal or increasing (uniformly or not).
4.1. Problem Formulation
Given a sequence of samples 0 @0 with (i = 0, ..., m), a surface A ! A) * ! is to be found that best
fits BC DEFGHID @0, minimizing the mean square error:
JK
L M /
" NA) * 0 !0 @0N
O
0P (9)
, where W are the weights, D are the control points and T is the node parametric vector.
Randrianarivony and Brunet (2002) addresses the Eq. (9) as a function of the node parametric vector T, since there
is no linear dependence between X and the parameters W, D, T. They give solution to the problem, simplifying the task
and adding some penalties that guarantee the mathematical formulation of NURBS functions (Loop, 2002). The

solution of Eq. (9) is subject to the restrictions of a solution with a sequence T of no decreasing nodes. Thus, the
problem can be considered a typical reduction by least squares:
A ! QA R ! A S ! A T ! U K3 @0 @0 R @0 S @0 T (10)
To linearize the equations, a matrix vector is defined as
V
W0 / A R ! @0R
W05 / A S ! @0S
W05 / A T ! @0T
X (11)
Replacing S in Eq. (11), one obtains
JK
/
" 0
WO5
0P / (12)
The solution of Eq. (12) should guarantee a sequence of increasing T nodes, for which a function is defined as
Y 2
6 Z 6
W 4 6
[ (13)
In addition, the penalty function can be formulated as
JK
/
" 0 / Y /
WO5
0P (14)
where is a very large positive number ( 1 x 1010
). Finally, ]0 / 0 / Y / in
JK
/
" ]0 /
WO5
0P (15)
Equation (15) can be solved by an optimization method based on nonlinear least squares and the Levenberg-
Marquardt algorithm (Michael., 1996), which is not addressed in this article.
4. RESULTS
Figure 6 presents several steps of the 3D surfaces fitting process from cloud of points. The steps are described below:
1. Figure (6a) shows the object 3D surface to be reconstructed, a metal plate with a weld bead on it. The object
dimensions were measured by means of a caliper guide as: ^_ ` 6a6 mm by b_ ` 6a6 mm and a maximum
height of ` 6a6 mm.
2. Figure (6.b) shows the light stripe on the object of the object, projected by the laser scanning system.
3. Figures (6.c) and (6.d) shows the processed images and the range images as point clouds. Some noise can be
noted visually.
4. Figures (6.e) and (6.f) shows a plot of a set of points from the light stripe on the object surface (Figure 6b)
before and after adaptive filtering with the LMS algorithm.
5. Figures (6.g), (6.h) and (6.i) present the construction of a 3D model using meshes and parametric surfaces. It
was assigned a square grid (4mm) as in figures (3.e) and (3. f) for filtering the data. The initial number of
points in the point clouds was 99,278 and at the end of the process of assigning the 3D mesh nodes (Fig. 5.b)
the number of points decreased to 2,350 points.
(a) (b)

(c) (d)
(e) (f)
(g) (h)
(i) (j)
Figure 6. Several steps for surface fitting in point clouds: (a) Weld bead; (b) Laser scanning on the weld bead; (c) Point
cloud acquired from the weld bead image; (d) Noise detected visually in the point cloud; (e) Plot of a section of the
weld bead in Z coordinates; (f) Plot of the section filtered with the adaptive LMS algorithm; (g) Point cloud projected
on the 2D plane; (h) 2D triangular mesh; (i) 3D triangular mesh; (j) Parametric surface fitted in the 3D triangular mesh
6. EXPERIMENTAL EVALUATION
It is extremely important to assess the resolution of a vision sensor as a function of the distance from the target
object. Thus, depending on the details needed in the object image, the distance can be chosen properly. For an
assessment of that resolution and measurement accuracy, a spherical target object was used, with 36.10±0.05
mm of
diameter (a roller ball of an electric generator bearing). The ball was scanned by the sensor and point clouds were
generated at different distances from a reference (Fig. 7a).

(a) (b)
Figure 7. (a) Acquisition of point clouds at different distances between point P0 and the sphere; (b) Point cloud
acquired from the sphere image.
Next to scanning the metal sphere at different distances, the sphere radius was calculated at each distance (Fig. 7b)
by using point triangulation (Forsyth and Ponce, 2003), explained next.
The experimental assessment was conducted using the tools developed to generate the meshes (section 6). For each
surface patch connected to a node the average normal vector was calculated as shown graphically in Fig. 8a. Then, the
normal vectors to each adjacent and opposite surface patches of a pair connected to the node are projected to intersect at
the center of the sphere (Fig. 8b).
Due to the finite precision of the system and to noise, parameters and image locations are known only
approximately and the two rays will not actually intersect in space; their intersection can only be estimated as the point
of minimum distance from both rays. So, the intersection point of the prolonged normal vectors arising from the
centroid of each of the two patches, p1 and p2, can be estimated as the midpoint of a segment perpendicular to both rays
(Fig. 9).
(e)
Figure 8. Several steps to estimate the sphere radius experimentaly.
The vectors shown in Fig. 9 can be expressed in the same coordinate system as
cd e d f Y d / gY d (16)
, where T is the translation vector between p1 and p2 and R the rotation matrix between the two patch planes π1 and π2

; a, b and c are the parameters to be solved to find the endpoints of the segment s. The intersection point is the midpoint
of the segment s.
Figura 9. Triangulação dos pontos d , d h
The radius associated to each pair of patches is the distance from the midpoint of s to the node point. This process is
repeated for the other two pair of patches connected to the same node, as shown in Fig. 8.b. The radius associated to
each node is calculated as the average radius of the three pairs of patches, as shown in Fig. 8.c. The sphere radius was
estimated as
i
j
" k&
.
&,=
l
(17)
, where n is the number of nodes. The standard deviation is calculated from
m
l
" in i
j
l
0P
o
(18)
Table (1) presents the results of the sphere radius values calculated at different distances. The distances between the
sensor and the sphere were measured using a breadboard with holes spaced by 25±0.5
mm as shown in Fig. 8e. The
sphere was positioned on a metal post. The initial distance was estimated using a manual caliper.
Table 1. Dimension errors of the vision system associated to surface modeling from point clouds
Distances (mm)
Sphere-Sensor
i
j (mm)
Estimated Average Radius
p q i
j p
Error (mm)
(mm)
Standard Deviation
Number of Nodes
225.00 18.640 0.59 1.2 472
250.00 18.831 0.78 1.3 452
275.05 18.754 0.70 1.4 428
300.05 18.822 0.77 1.8 436
325.50 18.847 0.80 1.9 450
350.20 18.901 0.85 2.2 472
375.00 18.975 0.93 2.4 448
400.05 19.095 1.05 2.8 426
425.15 19.185 1.14 2.9 388
450.05 19.155 1.11 3.1 324
475.10 18.937 0.88 3.0 326
500.30 18.984 0.93 3.0 332
r = measured sphere radius
The results show that the average errors found are within the range of 1mm at distances of 500mm. The results, of
course, are dependent on the lens size, since the image size is the parameter to influence the object image physical
resolution. It can be observed that the average error did not increase at the same ratio as the distance sensor-sphere, but
the standard deviation. So, with the lens used (focal length = 8.5mm), the vision system dimensional average accuracy
was evaluated as approximately 1:500.

7. CONCLUSIONS
This article presented the development of a system model to fit 3D surfaces to point clouds acquired by a special
vision sensor used in robot applications where surface details have to be mapped for welding tasks. It is presented and
discussed a technique to generate 3D surfaces from point clouds by using parametric surfaces – NURBS – with control
points defined as nodal points of 3D triangular meshes. The triangular meshes are constructed from the projection of
point clouds onto 2D meshes, next cloud points are selected when closer to the nodal points of the 2D mesh, then the
2D mesh is deformed to match the nodal points with the selected cloud points and finally the 2D mesh is projected back
into the 3D space. Parametric surfaces are then approximated to the control points to fit the 3D surface to the mesh
grids. A formulation of adaptive filters for point clouds in images is discussed and the model used presented.
The results of the constructed model are shown by means of digital representation from range images along all the
surface fitting steps. Experimental results were presented to assess the 3D accuracy of the vision system by comparing
3D data points from the surface fitted with the measured dimensions of a metal sphere at several distances from the
vision sensor. Results showed that the system accuracy is fairly enough to guarantee weld quality with sub-millimeter
positioning errors.
8. ACKNOWLEDGEMENTS
ELETRONORTE (Electrical Power Plants of the North of Brazil) and FINATEC (Scientific and Technological
Enterprises Foundation) are thanked for their financial support and collaboration. Our gratitude is especially conveyed
to the staff of ELETRONORTE for their technical support and interest.
9. REFERENCES
Anderson, C.W.,Crawford-Hines, 2000, “Fast generation of NURBS surfaces from polygonal mesh models of human
anatomy”. Technical Report CS-99-101, Fort Collins, Colorado State University, USA
Loop, C., 1990, “Generalized B-spline surfaces of arbitrary topology”, Proceedings of the 17th
Annual Conference on
Computer Graphics and Interactive Techniques, Dallas, TX, USA pp. 347 - 356
Boissonnat, J.-D. and Cazals, F., 2002,”Smooth surface reconstruction via natural neighbour interpolation of distance
functions”, Proceedings of the 16th
Annual Symposium on Computational Geometry, Clear Water Bay, Kowloon,
Hong Kong, pp. 223 – 232.
Chui, K.L., Chiu, W.K. and Yu, K. M., 2008, “Direct 5-axis tool-path generation from point cloud input using 3D biarc
fitting”, Vol. 24 , no. 2, p.p. 270-286.
Diniz, P.S.R., 2002, “Adaptative filter theory: Algorithm and practical implementations”, Kluwer Academic Publishers,
USA, 559 p.
Haykyn,S., 2001, “Adaptive filter theory”, 4th
ed., Prentice Hall, USA, 978 p.
Hoppe, H., 1994, “Piecewise smooth surface reconstruction”, Proceedings of the 21st
Annual Conference on Computer
Graphics and Interactive Techniques, Orlando, Florida, pp. 295 – 302.
Farin, G., 1996, “Curves and Surfaces for Computer-Aided Geometric Design: A Practical Guide”, 4th
Ed., USA, 421 p.
Fisher, M, Mandic, D, Bangham, J.A. and Harvey, R., 2000.“Visualizing error surfaces for adaptive filters and other
purposes”. IEEE International Conference on Acoustic, Speech and Signal Processing, Istanbul, Turkey, 4 p.
Forsyth, D and Ponce, J., 2003, “Computer Vision: A Modern Approach”, Prentice-Hall, New Jersey, USA.
Ginani, L. and Motta, J. M S. T.., 2007, “A Laser Scanning System for 3D Modeling of Industrial Objects Based on
Computer Vision”, Proceedings of 19th
International Congress of Mechanical Engineering, Vol. 1, Brasilia-DF,
Brazil, 10 p.
Manolakis, D. G., 1999, “Statistical and Adaptive Signal Processing: Spectral Estimation, Signal Modeling , Adaptative
Filtering and Array Processing ”. McGraw-Hill, USA, 819 p.
Park. K., Yun, I. D. and Lee, S. U., 1999, “Constructing NURBS surface model from scattered and unorganized range
data ”. Proceedings of the Second International Conference on 3D Digital Imaging and Modeling, pp. 312-320.
Peters, J. , 1987, “Constructing c.l surfaces of arbitrary topology using bicuadric and bicubic splines’, Designing Fair
Curves and Surfaces, Edited by Nickolas S. Sapidis, Cambridge, pp. 277–293., 330 p.
Randrianarivony, M. e Brunnett, G., 2002, “Parallel implementation of surface reconstruction from noisy samples”,
Preprintreihe des Chemnitzer SFB 393, 02-16.
Roth, G. and Boulanger, P. 1998, “Cad model building from multiple range images”, Proceedings of Vision Interface,
Vancuver, BC, pp. 431-438.
Yvart, A., Hahmann, S. and Bonneau, G.-P., 2005, “Smooth adaptive fitting of 3D models using hierarchical triangular
splines ”, Proceedings of the International Conference on Shape Modeling and Applications 2005, pp. 13-22.
5. RESPONSIBILITY NOTICE
The authors are the only responsible for the printed material included in this paper.

Surface generation from point cloud.pdf

More Related Content

Similar to Surface generation from point cloud.pdf (20)

Recently uploaded (20)

Surface generation from point cloud.pdf