DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the "right to exclude" granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Langi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 12205264 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because they can read on to each other, see the following mapping table.
Current Application
1
2
3
4
5
6
7
8
Patent application
1
2
3
4
5
6
7
8
Current Application
9
10
11
12
13
14
15
16
17
Patent application
11
12
13
14
15
16
17
18
20
Also, shown below is a mapping between the limitations of independent claims of current application U.S. Patent Application 19032355 and independent claims of U.S. Patent Application 12205264 B2.
Claims
Current Application
Claims
Patent Application
1
A system for detecting and measuring features relative to curved surfaces from a 3D point cloud imaged by a 3D vision system camera assembly comprising:
1
A system for detecting features relative to curved surfaces from a 3D point cloud imaged by a 3D vision system camera assembly comprising:
a vision system processor configured to carry out processes comprising:an extraction process that identifies the curved surfacein 3D space and provides a reference model relative to the curved surface, the reference model comprising a reference surface;a flattening process that flattens the curved surface into a flattened, planar shapein accordance with the reference model;a feature location process that locates features on the planar shape by:generating a height image, each pixel of the height image,representing a height relative to the reference surface, wherein eachpixel of the height image is classified as positive type, negative type, or unchanged type based on whether the pixel is above, below, or on the referenc surface,forming 2D blobs from the planar shape and the height image bygrouping neighboring pixels of the same type, andcreating 3D blobs from the 2D blobs;
an extraction process that identifies a curved surface in 3D space and provides a reference model relative to the curved surface; a flattening process that flattens the curved surface, including a feature thereof, into a flattened, planar shape in accordance with the reference model, wherein a portion of the curved surface is represented as a height image having values representative of distances of the points on the curved surface from a central element of the reference model, wherein the curved surface comprises one or more regions that are monotonic with respect to an outward radial direction from the central element of the reference model; a feature location process that locates the feature on the planar shape represented in the height image by processing the planar shape produced from the flattening process with respect to the height image;
a mapping process that converts the located features into the 3D space;and a feature output process that reports the features relative to the curved surface.
a mapping process that converts the feature located by processing the planar shape into the 3D space; and a feature output process that reports, relative to the curved surface, the feature converted to the 3D space.
9
A method for detecting and measuring features relative to curved surfaces from a 3D point cloud imaged by a 3D vision system camera assembly, the method performed by a vision system processor, the method comprising:
11
A method for detecting features relative to curved surfaces from a 3D point cloud imaged by a 3D vision system camera assembly comprising:
identifying the curved surface in 3D space and providing a reference modelrelative to the curved surface, the reference model comprising a reference surface;flattening the curved surface into a flattened, planar shape in accordance with thereference model;locating features on the planar shape by:generating a height image, each pixel of the height image representing aheight relative to the reference surface, wherein each pixel of the height imageis classified as positive type, negative type, or unchanged type based onwhether the pixel is above, below, or on the reference surface,forming 2D blobs from the planar shape and the height image bygrouping neighboring pixels of the same type, andcreating 3D blobs from the 2D blobs;
identifying a curved surface in 3D space and providing a reference model relative to the curved surface; flattening the curved surface into a flattened, planar shape in accordance with the reference model, wherein a portion of the curved surface is represented as a height image having values representative of distances of points on the curved surface from a central element of the reference model, wherein the curved surface comprises one or more regions that are monotonic with respect to an outward radial direction from the central element of the reference model; locating a feature on the planar shape represented in the height image by processing the planar shape produced from the flattening process with respect to the height image;
mapping the located features into the 3D space; andreporting the features relative to the curved surface.
mapping the located feature into the 3D space; and reporting the located feature relative to the curved surface.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 – 17 are rejected under 35 U.S.C. 103 as being unpatentable over Ben-Haim et al. (Publication: US 2020/0286225 A1) in view of Raskar et al. (Publication: US 2003/0222868 A1).
Regarding claim 1, Ben-Haim discloses discloses a system for detecting and measuring features relative to curved surfaces from a 3D point cloud imaged by a 3D vision system camera assembly comprising ([0303], A computer processor with memory, probe with camera [0276]. [0152] – measure of the flattening, detect curvature in 3-D point cloud representation [0365].):
a vision system processor configured to carry out processes comprising ([0303], A computer processor with memory, probe with camera [0276]. [0152] – measure of the flattening, detect curvature in 3-D point cloud representation [0365].):
an extraction process that identifies the curved surface in 3D space and provides a reference model relative to the curved surface, the reference model comprising a reference surface ([0308] - The choice of global curvature is not limited to circles (or spheres in 3-D), and a different choice can lead to a different residual result of preserved relief features. Ellipse 1404 of FIG. 14D illustrates a different function which could be used to model a global curvature of path 1401. The resulting flattened curve would suppress relief features such as the pattern of long peaks 1410 and valleys 1412 which superimposes on the shorter peaks 1412 and valleys 1413 of FIG. 14B. As shown in Fig. 14B, it extract the long peaks and flattend it, “extraction process”.
[0195] - an image, surfaces, is produced using the flattened reconstruction.
[0194] - a flattened reconstruction is produced from the source reconstruction. The flattened reconstruction is produced so that a global curvature (that is, a curve defined over the area of the curved surface, but not following all its details) is reduced. The global curvature is the curvature of a curve defined over the area of the curved surface, but not following all its details, “the reference model comprising a reference surface”. For example, it may be the curvature of a sphere or of an ellipsoid, best-fitting the curved surface. The global curvature is implicit, e.g., in the choice of coordinate systems used in a flattening transformation.);
a flattening process that flattens the curved surface into a flattened, planar shape in accordance with the reference model ([0308] - The resulting flattened curve would suppress relief features such as the pattern of long peaks 1410 and valleys 1412 which superimposes on the shorter peaks 1412 and valleys 1413 of FIG. 14B. As shown in Fig. 14B, it extract the long peaks and flattend it based on the global curvature, “shape… the reference model”.
The choice of global curvature is not limited to circles (or spheres in 3-D), and a different choice can lead to a different residual result of preserved relief features. Ellipse 1404 of FIG. 14D illustrates a different function which could be used to model a global curvature of path 1401.);
a feature location process that locates features on the planar shape by ([0308] - The resulting flattened curve would suppress relief features such as the pattern of long peaks 1410 and valleys 1412 which superimposes on the shorter peaks 1412 and valleys 1413 of FIG. 14B. As shown in Fig. 14B, it locates the long peaks, extract the long peaks and flattend it based on the global curvature. ):
generating a height image ( [0444] - As shown in Fig. 23, curvatures, 12, 18, 20, height image.
PNG
media_image1.png
520
850
media_image1.png
Greyscale
), each pixel of the height image, representing a height relative to the reference surface, wherein each pixel of the height image is classified as positive type, negative type, or unchanged type based on whether the pixel is above, below, or on the reference surface ( [0306] to [0309] –
The long peaks 1410 is the pixels of height images, relatives to global curvature, “reference surface” .
The long peaks 1410 is the pixels of height image , and the long peaks would be repressed based on the classification of global curvatures such as curve 1401. Based on the global curvatures “reference surface”, the long peak “high” can be flattened “positive” or not flattened “negative” on the global curvatures, “reference surface”
FIG. 14C represents a different flattened version of FIG. 14A, with some of the global curvature represented by circle 1402 remaining in flattened circle 1402B and flattened curve 1401B. Equivalently, a different choice of global curvature such as curve 1404 could be used as the basis of flattening (and then flattened completely, for example line 1404B) to result in a shape like that of 1401B. For example, ellipse 1404 of FIG. 14D illustrates a different function which could be used to model a global curvature of path 1401. The resulting flattened curve would suppress relief features, pixels, such as the pattern of long peaks 1410 and valleys 1412 which superimposes on the shorter peaks 1412 and valleys 1413 of FIG. 14B .
PNG
media_image2.png
736
464
media_image2.png
Greyscale
);
a mapping process that converts the located features into the 3D space ([0189] In the flattening transformation used in producing the reconstruction schematically indicated in FIG. 1B, it is approximately as though the left atrium wall was slit partially up the center of the view of FIG. 1A, 3D Sphere, on two sides (e.g., along the lines extending upward from reference points 22 and 23), and unwrapped for viewing. Arrows 11A-11B of FIGS. 1A-1B represent spherical angle coordinates of FIG. 1A mapped to Cartesian axes of FIG. 1B.).
Ben-Haim does not disclose; however, Raskar discloses
forming 2D blobs from the planar shape and the image by grouping neighboring pixels of the same type ([0059], [0061] - Using the mouse, user input is acquired as a collection of 2D screen-space coordinates. The user begins the stroke 501 by clicking, and then dragging a cursor over a path of the desired stroke, “2D blobs from the planar shape”, project the sequence of 2D points of the stroke onto the z=0, “height”, and finishes by releasing the mouse button.
[0059], [0061] As shown in FIG. 6, we first project the sequence of 2D points of the stroke onto the z=0 plane 601. The resulting points are used as zero-points for the implicit function we want to generate. We refer to the path defined by these zero-points as the contour, “grouping neighboring pixels, see Fig. 7”.
We re-sample these points so that the points are substantially even spaced. If there are too many points close together in the sequence, then we simply ignore some samples. For example, if the distance between a previous point and a current point is less than 15 pixels, then we do not add the current point. We then "close" the stroke by joining the last point to the first to make a closed curve, “by grouping neighboring pixels of the same type”.
PNG
media_image3.png
318
330
media_image3.png
Greyscale
), and
creating 3D blobs from the 2D blobs ([0057] - converting a 2D input stroke, blob, to a 3D blob.);
a feature output process that reports the features relative to the curved surface ([0082], [0085] -
As shown in FIGS. 12b, the implicit surface constructed from the remaining vertices and corresponding normals is a smooth merge between the two surfaces, “output process that reports”.
two blobs are automatically merged “output process that reports” with implicit functions f and g, eliminate all vertices v, and their corresponding normal points that lie inside the intersection of the two blobs, “features”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Ben-Haim with forming 2D blobs from the planar shape and the image by grouping neighboring pixels of the same type ; creating 3D blobs from the 2D blobs; a feature output process that reports the features relative to the curved surface as taught by Raskar. The motivation for doing is to enable the functions to be applied to the surfaces thus maintain a smooth surface as taught by Raskar.
Regarding claim 2, Ben-Haim in view of Raskar disclose all the limitation of claim 1.
Ben-Haim discloses wherein the curved surface is a cylindrical surface or a spherical surface and the reference model, respectively, comprises a cylinder or a sphere (
[0152] - compared to the best-fit sphere for the source 3-D representation of the surface. The radius increase is determined for substantially unchanged sizes of surface features (e.g., the same on average). this radius increase is at least a factor of 2, and preferably at least a factor of 5. The best-fit sphere for the source 3-D representation is considered to define the global curvature which is relatively flattened.
[0189] - As shown in Fig. 1A, the reference points 22, are in a sphere . ).
Regarding claim 3, Ben-Haim in view of Raskar disclose all the limitation of claim 1.
Ben-Haim discloses wherein the curved surface comprises one or more regions that appear monotonic when viewed from an outward radial direction of an axis of the cylinder or the a center of the sphere superimposed thereon ([0152] – Zero curvature.
a measure of the flattening may be expressed as an increase in the radius of a sphere which best fits (e.g., minimizes average distance to) the flattened reconstruction, compared to the best-fit sphere for the source 3-D representation of the surface. this radius increase is at least a factor of 2, and preferably at least a factor of 5 thus outward radial direction of an axis can be read on.
The radius increase is determined for substantially unchanged sizes of surface features thus Monotonic can be read on because monotonic means a way that it either never decreases or never increases ).
Regarding claim 4, Ben-Haim in view of Raskar disclose all the limitation of claim 1.
Ben-Haim discloses wherein the flattening process flattens the curved surface along surface tangent direction ([0308] The choice of global curvature is not limited to circles (or spheres in 3-D), and a different choice can lead to a different residual result of preserved relief features. Ellipse 1404 of FIG. 14D illustrates a different function which could be used to model a global curvature of path 1401. The resulting flattened curve would suppress relief features such as the pattern of long peaks 1410 and valleys 1412 which superimposes on the shorter peaks 1412 and valleys 1413 of FIG. 14B.
Surface tangent direction is flattened along the cylindrical or spherical surface, see Spec, so the flattened curve suppress relief peak to make it flat along the spherical surface reads on.).
Regarding claim 5, Ben-Haim in view of Raskar disclose all the limitation of claim 1.
Ben-Haim discloses divides the planar surface, wherein 3D points of the curved surface reside, into a grid based upon orthogonal axes defining azimuth and height on a cylinder or azimuth and tilt on a sphere (
[0094] - The 3-D sphere model of curved surface is divided with details, grids.
[0430] , Fig. 22B - The unfolding transformation causes a plate Carrée or other cartographic projection of the azimuth and inclination coordinates, angle, of the model onto a flat surface, such that the unfolded model becomes a flat model that includes the relief details of the 3-D model of the surface, “azimuth and tilt on a sphere”.)
Regarding claim 6, Ben-Haim in view of Raskar disclose all the limitation of claim 1.
Ben-Haim discloses a feature identification process that identifies features differing from a normal surface geometry of the curved surface ([0308] - The choice of global curvature is not limited to circles (or spheres in 3-D), and a different choice can lead to a different residual result of preserved relief features. Ellipse 1404 of FIG. 14D illustrates a different function which could be used to model a global curvature of path 1401. The resulting flattened curve would suppress relief features such as the pattern of long peaks 1410 and valleys 1412 which superimposes on the shorter peaks 1412 and valleys 1413 of FIG. 14B.).
Regarding claim 7, Ben-Haim in view of Raskar disclose all the limitation of claim 1.
Ben-Haim discloses wherein the differing features comprise object defects ([0308] The choice of global curvature is not limited to circles (or spheres in 3-D), and a different choice can lead to a different residual result of preserved relief features. Ellipse 1404 of FIG. 14D illustrates a different function which could be used to model a global curvature of path 1401. The resulting flattened curve would suppress relief features such as the pattern of long peaks 1410 and valleys 1412 which superimposes on the shorter peaks 1412 and valleys 1413 of FIG. 14B.).
Regarding claim 8, Ben-Haim in view of Raskar disclose all the limitation of claim 1.
Ben-Haim discloses a reference surface refinement process that adapts the reference surface to 3D points representing the planar surface (
[0195] - an image, surfaces, is produced using the flattened reconstruction.
[0194] - a flattened reconstruction is produced from the source reconstruction. The flattened reconstruction is produced so that a global curvature (that is, a curve defined over the area of the curved surface, but not following all its details) is reduced. The global curvature is the curvature of a curve defined over the area of the curved surface, but not following all its details. For example, it may be the curvature of a sphere or of an ellipsoid, best-fitting the curved surface. The global curvature is implicit, e.g., in the choice of coordinate systems used in a flattening transformation. ).
Regarding claim 9, see rejection on claim 1.
Regarding claim 10, see rejection on claim 2.
Regarding claim 11, see rejection on claim 2.
Regarding claim 12, see rejection on claim 3.
Regarding claim 13, see rejection on claim 4.
Regarding claim 14, see rejection on claim 5.
Regarding claim 15, see rejection on claim 6.
Regarding claim 16, see rejection on claim 7.
Regarding claim 17, see rejection on claim 8.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ming Wu whose telephone number is (571)270-0724. The examiner can normally be reached on Monday - Friday: 9:30am - 6:00pm EST .
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached on 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MING WU/
Primary Examiner, Art Unit 2618
Prosecution Insights