Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-5, 10-11, 14-15, 18, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Klug (US 20220256135) in view of Yamamoto (US 20230222753) and further in view of Braunecker (US 20070064246).
Regarding claim 18, Klug teaches a light scanner system for scanning an object, comprising:
A first camera positioned at a first viewpoint configured to acquire a first image of the illuminated object (Paragraph 6, Special cameras or measurement cameras are generally used to generate the spatially representative image data. These measurement cameras are set up at different positions located remotely from one another and thus each have a different perspective);
A second camera positioned at a first viewpoint configured to acquire a second image of the illuminated object (Paragraph 6, Special cameras or measurement cameras are generally used to generate the spatially representative image data. These measurement cameras are set up at different positions located remotely from one another and thus each have a different perspective); and
An image processing device including circuitry configured to:
Obtain first image data representing the first image, obtain second image data representing the first image, project the first and the second image data in the illumination plane representing a projected first and second image, respectively, for merging the first and second image data in the illumination plane (Paragraph 3, For this purpose, in general at least two individual image data sets having photographs from different perspectives are generated of a desired detail of the environment. Two two-dimensional recordings, which each show the environmental detail from a different perspective, can thus be provided by these data sets. Upon the visual superposition of the two individual image data sets, the spatially representative image data result. The resulting stereoscopic image thus shows the superposition of the two two-dimensional recordings).
While Klug fails to disclose the following, Yamamoto teaches:
A light source configured to illuminate an object with a line of light traveling in a first direction and extending across a second direction in an illumination plane defined by the first direction and the second direction (Paragraph 47, device that irradiates the sample S with planar light L2; Figure 1 (L2)).
Yamamoto and Klug are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Klug to incorporate the teachings of Yamamoto and use a light source to illuminate an object with a line of light traveling in a first direction and extending across a second direction in an illumination plane. Doing so would allow for using a known method of selective plane illumination microscopy (SPIM) to scan a three-dimensional object (Yamamoto, Paragraph 2).
While the combination of Klug and Yamamoto fails to disclose the following, Braunecker teaches:
Determine the first viewpoint and the second viewpoint based on positions and orientations of surface reflections visible on a plurality of reference objects located in the illumination plane (Abstract, determine the actual position and/or actual orientation of a measuring appliance; Paragraph 25, owing to the centre of gravity of the reflection on the surface of the sphere or of the sphere segment, the position of the reference point can also be determined for angle measurements with sufficient accuracy).
Braunecker and the combination of Klug and Yamamoto are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug and Yamamoto to incorporate the teachings of Braunecker and determine the viewpoints based on surface reflections in the illumination plane. Doing so would allow for using a known accurate way of determining the position of the measuring device.
Apparatus claim 1 and method claim 11 correspond to apparatus claim 18. Therefore, claims 1 and 11 are rejected for the same reasons as used above.
Regarding claim 4, the combination of Klug, Yamamoto, and Braunecker teaches the image processing device according to claim 1, wherein the first image is acquired by a first camera positioned at the first viewpoint and the second image is acquired by a second camera positioned at the second viewpoint (Klug, Paragraph 6, These measurement cameras are set up at different positions located remotely from one another and thus each have a different perspective, thus a viewing direction of the desired object in the environment dependent on the respective position).
Method claim 14 corresponds to apparatus claim 4. Therefore, claim 14 is rejected for the same reasons as used above.
Regarding claim 5, the combination of Klug, Yamamoto, and Braunecker teaches the image processing device according to claim 1, wherein the first image is acquired by a camera positioned at the first viewpoint and the second image is acquired by the camera moved from the first viewpoint to the second viewpoint (Klug, Paragraph 7, Images of the desired object in the environment are then recorded using this single measurement camera from different positions, thus from different perspectives).
Method claim 15 corresponds to apparatus claim 5. Therefore, claim 15 is rejected for the same reasons as used above.
Regarding claim 10, the combination of Klug, Yamamoto, and Braunecker teaches the image processing device according to claim 1, wherein the circuitry is further configured to:
Obtain a third image data representing a third image of the illuminated object, wherein the third image is acquired at a third viewpoint being different from the first and the second viewpoint (Klug, Paragraph 48, the environment or the object in the environment can be acquired from three different perspectives); and
Project the third image data in the illumination plane representing a projected third image for merging the first, second and third image data in the illumination plane (Klug, Paragraph 65, In combination, the individual image data B1, B2, B3, when brought together to form a common image data set, then result in the spatially representative image data. The spatially representative image data thus represent a common image of the superimposed individual images, namely the photogrammetric image RB of the object O).
Regarding claim 21, the combination of Klug, Yamamoto, and Braunecker teaches the image processing device according to claim 1. While the combination as presented previously fails to disclose the following, Yamamoto further teaches:
Wherein the line of light has a spatial distribution that is narrower in a direction perpendicular to the illumination plane than within the illumination plane (Paragraph 51, the planar light L2 is preferably a thin planar light having a thickness of 2 mm or less in consideration of a resolution).
Yamamoto and the combination of Klug and Braunecker are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug and Braunecker to incorporate the teachings of Yamamoto and use a line of light that is narrower in a direction perpendicular to the illumination plane. Doing so would allow for using a known method of selective plane illumination microscopy (SPIM) to scan a three-dimensional object (Yamamoto, Paragraph 2) and allow for scanning of cross-sections.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Davidson (WO 03023684) and Jin (US 20220051379).
Regarding claim 2, the combination of Klug, Yamamoto, and Braunecker teaches the image processing device according to claim 1. While the combination fails to disclose the following, Davidson teaches:
Wherein the circuitry is further configured to merge the first and second image data in the illumination plane representing a merged image by calculating a product of the projected first and the second image data (Page 32, Paragraph 4, the obtained images are combined in computer memory using a multiplication operation on each pixel appearing in the images).
Note: The claimed invention specification recites “Then the image contrast maybe enhanced by calculating the product of the pixel values in this region, since high intensity regions are enhanced and low intensity regions are suppressed.” Davidson teaches “Combining the images in this fashion results in dark areas getting darker and light areas staying the same” (Page 33, Paragraph 3).
Davidson and the combination of Klug, Yamamoto, and Braunecker are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, and Braunecker to incorporate the teachings of Davidson and merge the first and second image by calculating a product. Doing so would enhance the high intensity areas and create a resulting image free of ambient light reflection (Page 33, Paragraph 3).
While the combination of Klug, Yamamoto, Braunecker, and Davidson fails to disclose the following, Jin teaches:
Representing a merged image by calculating a product of the project first and the second image data for identical parts in the projected first and second image (Paragraph 8, multiply a pixel value of each pixel of the partial region and a pixel value of each pixel of the first reference image that corresponds to the each pixel of the partial region).
Jin and the combination of Klug, Yamamoto, Braunecker, and Davidson are both considered analogous to the claimed invention because they are in the same field of image processing and segmentation. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, and Davidson to incorporate the teachings of Jin and calculate the product of identical parts of an image. Doing so would have further enhanced the high intensity areas of only the matching regions of the two images.
Method claim 12 corresponds to apparatus claim 2. Therefore, claim 12 is rejected for the same reasons as used above.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Heidemann (US 20180321383).
Regarding claim 3, the combination of Klug, Yamamoto, and Braunecker teaches the image processing device according to claim 1. While the combination fails to disclose the following, Heidemann teaches:
Wherein the circuitry is further configured to calibrate the projection of the first and the second image data in the illumination plane based on the plurality of reference objects in the illumination plane (Paragraph 95, background objects may be used to assist in registering successive scans).
Heidemann and the combination of Klug, Yamamoto, and Braunecker are both considered analogous to the claimed invention because they are in the same field of image processing and segmentation. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, and Braunecker to incorporate the teachings of Heidemann and use reference objects to calibrate the first and second image data. Doing so would have allowed for efficiently matching parts of different images taken from different viewpoints.
Method claim 13 corresponds to apparatus claim 3. Therefore, claim 13 is rejected for the same reasons as used above.
Claims 6-7 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over on Klug in view of Yamamoto and further in view of Braunecker, and further in view of Davidson, and further in view of Jin as applied to claims 2 and 12 above and further in view of Fisker (US 10835361).
Regarding claim 6, the combination of Klug, Yamamoto, Braunecker, Davidson, and Jin teaches the image processing device according to claim 2. While the combination fails to disclose the following, Fisker teaches:
Wherein the circuitry is further configured to perform image segmentation on the merged image for determining a cross-section of the object in the illumination plane (Column 12, Lines 61-63, FIG. 4A shows cross sections of a first 3D tooth model 451 and a second 3D tooth model 451 segmented from a first and a second digital 3D representation, respectively).
Fisker and the combination of Klug, Yamamoto, Braunecker, Davidson, and Jin are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, Davidson, and Jin to incorporate the teachings of Fisker and perform image segmentation on a merged image to determine a cross-section of an object. Doing so would allow for readily and precisely identifying specific segments of the merged image (Fisker, Column 13, Lines 6-8).
Method claim 16 corresponds to apparatus claim 6. Therefore, claim 16 is rejected for the same reasons as above.
Regarding claim 7, the combination of Klug, Yamamoto, Braunecker, Davidson, Jin, and Fisker teaches the image processing device according to claim 6, wherein the image segmentation is performed based on a set of predetermined cross-section criteria (Fisker, Column 2, Lines 42-44, The segmentation provides that the identified teeth and the gingiva can be separated and treated as independent 3D models of the individual teeth and the gingiva). Note: The predetermined cross-section criteria includes separating the individual teeth and the gingiva into different segments.
Fisker and the combination of Klug, Yamamoto, Braunecker, Davidson, and Jin are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, Davidson, and Jin to incorporate the teachings of Fisker and use pre-determined cross-section criteria. Doing so would allow for readily and precisely identifying specific segments of the merged image (Fisker, Column 13, Lines 6-8).
Method claim 17 corresponds to apparatus claim 7. Therefore, claim 17 is rejected for the same reasons as above.
Claims 8-9 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Nichol (WO 2019090139).
Regarding claim 8, the combination of Klug, Yamamoto, and Braunecker teaches the image processing device according to claim 1. While the combination fails to disclose the following, Nichol teaches:
Wherein the object is further illuminated with a second line of light in the illumination plane or in a second illumination plane different from the illumination plane (Paragraph 290, second light source emitting light).
Nichol and the combination of Klug, Yamamoto, and Braunecker are both considered analogous to the claimed invention because they are in the same field of illumination planes. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, and Braunecker to incorporate the teachings of Nichol and use a second line of light in the illumination plane. Doing so would improve the luminance uniformity, improve the illuminance uniformity, improve the color uniformity, increase the luminance of the light emitting region, or provide a back-up light emitting region when component failure causes light from the first lightguide to fall below specification (such as color uniformity, luminance uniformity, or luminance) in the overlapping region (Nichol, Paragraph 193).
Method claim 19 corresponds to apparatus claim 8. Therefore, claim 19 is rejected for the same reasons as above.
Regarding claim 9, the combination of Klug, Yamamoto, Braunecker, and Nichol teaches the image processing device according to claim 8. While the combination as presented previously fails to disclose the following, Nichol further teaches:
Wherein the second line of light has a center wavelength different from a center wavelength of the line of light (Paragraph 290, second light source emitting light with a second peak wavelength different from than the first peak wavelength).
Nichol and the combination of Klug, Yamamoto, and Braunecker are both considered analogous to the claimed invention because they are in the same field of illumination planes. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, and Braunecker to incorporate the teachings of Nichol and use a second line of light that has a different center wavelength from the line of light. Doing so would improve the luminance uniformity, improve the illuminance uniformity, improve the color uniformity, increase the luminance of the light emitting region, or provide a back-up light emitting region when component failure causes light from the first lightguide to fall below specification (such as color uniformity, luminance uniformity, or luminance) in the overlapping region (Nichol, Paragraph 193).
Method claim 20 corresponds to apparatus claim 9. Therefore, claim 20 is rejected for the same reasons as used above.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Valdes (WO 20200249341).
Regarding claim 22, the combination of Klug, Yamamoto, Braunecker teaches the image processing device according to claim 1. While the combination fails to disclose the following, Valdes teaches:
Merge the first image data and the second image data in the illumination plane representing a merged image by calculating a ratio or a difference between the projected first and second image data for identical parts in the projected first and second image (Paragraph 4, determining a positional shift based on a photogrammetrical difference between the first camera image and the second camera image; determining a first position of the imager in the environment at the first time and a second position of the imager in the environment at the second time based on the trajectory and the photogrammetrical difference; merging the first radio image with the second radio image based on the first position and the second position to produce a synthetic aperture radar image of the object).
Valdes and the combination of Klug, Yamamoto, and Braunecker are both considered analogous to the claimed invention because they are in the same field of illumination planes. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, and Braunecker to incorporate the teachings of Valdes and calculate the difference between a first image and a second image and merge the images. Doing so would allow for using a known method to combine similar images to produce a more detailed and accurate view of the desired object.
Response to Amendment
Applicant’s arguments with respect to claims 1, 11, and 18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Braunecker addresses the subject matter of determining the first viewpoint and second viewpoint based on positions and orientations of surface reflections visible on a plurality of reference objects located in the illumination plane.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SNIGDHA SINHA whose telephone number is (571)272-6618. The examiner can normally be reached Mon-Fri. 12pm-8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SNIGDHA SINHA/Examiner, Art Unit 2619
/JASON CHAN/Supervisory Patent Examiner, Art Unit 2619