Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02 June 2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-5, 10-11, 14-15, 18, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Klug (US 20220256135) in view of Yamamoto (US 20230222753) and further in view of Braunecker (US 20070064246) and further in view of Aiki (US 20050224582).
Regarding claim 18, Klug teaches a light scanner system for scanning an object, comprising:
A first camera positioned at a first viewpoint configured to acquire a first image of the illuminated object (Paragraph 6, Special cameras or measurement cameras are generally used to generate the spatially representative image data. These measurement cameras are set up at different positions located remotely from one another and thus each have a different perspective);
A second camera positioned at a first viewpoint configured to acquire a second image of the illuminated object (Paragraph 6, Special cameras or measurement cameras are generally used to generate the spatially representative image data. These measurement cameras are set up at different positions located remotely from one another and thus each have a different perspective); and
An image processing device including circuitry configured to:
Obtain first image data representing the first image, obtain second image data representing the first image, project the first and the second image data in the illumination plane representing a projected first and second image, respectively, for merging the first and second image data in the illumination plane (Paragraph 3, For this purpose, in general at least two individual image data sets having photographs from different perspectives are generated of a desired detail of the environment. Two two-dimensional recordings, which each show the environmental detail from a different perspective, can thus be provided by these data sets. Upon the visual superposition of the two individual image data sets, the spatially representative image data result. The resulting stereoscopic image thus shows the superposition of the two two-dimensional recordings).
While Klug fails to disclose the following, Yamamoto teaches:
A light source configured to illuminate an object with a line of light traveling in a first direction and extending across a second direction in an illumination plane defined by the first direction and the second direction (Paragraph 47, device that irradiates the sample S with planar light L2; Figure 1 (L2)).
The illumination plane defines an illumination coordinate system (Paragraph 69, In this case, the analysis unit 10 extracts the luminance value corresponding to the pixel value (the coordinate value) of the top position XY image data).
Yamamoto and Klug are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Klug to incorporate the teachings of Yamamoto and use a light source to illuminate an object with a line of light traveling in a first direction and extending across a second direction in an illumination plane and define an illumination coordinate system. Doing so would allow for using a known method of selective plane illumination microscopy (SPIM) to scan a three-dimensional object (Yamamoto, Paragraph 2).
While the combination of Klug and Yamamoto fails to disclose the following, Braunecker teaches:
Determine the first viewpoint and the second viewpoint based on positions and orientations of surface reflections visible along the first direction and the second direction on a plurality of reference objects located in the illumination plane (Abstract, determine the actual position and/or actual orientation of a measuring appliance; Paragraph 25, owing to the centre of gravity of the reflection on the surface of the sphere or of the sphere segment, the position of the reference point can also be determined for angle measurements with sufficient accuracy). Note: Yamamoto teaches the first direction and second direction in the illumination plane.
Braunecker and the combination of Klug and Yamamoto are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug and Yamamoto to incorporate the teachings of Braunecker and determine the viewpoints based on surface reflections in the illumination plane. Doing so would allow for using a known accurate way of determining the position of the measuring device.
While the combination of Klug, Yamamoto, and Braunecker fails to disclose the following, Aiki teaches:
The circuitry is further configured to determine the first viewpoint and the second viewpoint relative to the illumination coordinate system during the scanning of the object by the line of light at predetermined scanning positions along a scanning direction (Paragraph 93, calculating the pointing position on the surface to be detected, based on a time from start of scanning of the line-shaped scan beam that scans in a first direction of the surface to be detected until the light-receiving section detects the line-shaped scan beam, and a time from start of scanning of the line-shaped scan beam that scans in a second direction of the surface to be detected until the light-receiving section detects the line-shaped scan beam).
Aiki and the combination of Klug, Yamamoto, and Braunecker are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, and Braunecker to incorporate the teachings of Aiki and determine the viewpoints relative to the illumination coordinate system during scanning of the object by the line of light at predetermined scanning positions. Doing so would allow for using a known accurate way of determining the position of the measuring device.
Apparatus claim 1 and method claim 11 correspond to apparatus claim 18. Therefore, claims 1 and 11 are rejected for the same reasons as used above.
Regarding claim 4, the combination of Klug, Yamamoto, Braunecker, and Aiki teaches the image processing device according to claim 1, wherein the first image is acquired by a first camera positioned at the first viewpoint and the second image is acquired by a second camera positioned at the second viewpoint (Klug, Paragraph 6, These measurement cameras are set up at different positions located remotely from one another and thus each have a different perspective, thus a viewing direction of the desired object in the environment dependent on the respective position).
Method claim 14 corresponds to apparatus claim 4. Therefore, claim 14 is rejected for the same reasons as used above.
Regarding claim 5, the combination of Klug, Yamamoto, Braunecker, and Aiki teaches the image processing device according to claim 1, wherein the first image is acquired by a camera positioned at the first viewpoint and the second image is acquired by the camera moved from the first viewpoint to the second viewpoint (Klug, Paragraph 7, Images of the desired object in the environment are then recorded using this single measurement camera from different positions, thus from different perspectives).
Method claim 15 corresponds to apparatus claim 5. Therefore, claim 15 is rejected for the same reasons as used above.
Regarding claim 10, the combination of Klug, Yamamoto, Braunecker, and Aiki teaches the image processing device according to claim 1, wherein the circuitry is further configured to:
Obtain a third image data representing a third image of the illuminated object, wherein the third image is acquired at a third viewpoint being different from the first and the second viewpoint (Klug, Paragraph 48, the environment or the object in the environment can be acquired from three different perspectives); and
Project the third image data in the illumination plane representing a projected third image for merging the first, second and third image data in the illumination plane (Klug, Paragraph 65, In combination, the individual image data B1, B2, B3, when brought together to form a common image data set, then result in the spatially representative image data. The spatially representative image data thus represent a common image of the superimposed individual images, namely the photogrammetric image RB of the object O).
Regarding claim 21, the combination of Klug, Yamamoto, Braunecker, and Aiki teaches the image processing device according to claim 1. While the combination as presented previously fails to disclose the following, Yamamoto further teaches:
Wherein the line of light has a spatial distribution that is narrower in a direction perpendicular to the illumination plane than within the illumination plane (Paragraph 51, the planar light L2 is preferably a thin planar light having a thickness of 2 mm or less in consideration of a resolution).
Yamamoto and the combination of Klug, Braunecker, and Aiki are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Braunecker, and Aiki to incorporate the teachings of Yamamoto and use a line of light that is narrower in a direction perpendicular to the illumination plane. Doing so would allow for using a known method of selective plane illumination microscopy (SPIM) to scan a three-dimensional object (Yamamoto, Paragraph 2) and allow for scanning of cross-sections.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Davidson (WO 03023684) and Jin (US 20220051379).
Regarding claim 2, the combination of Klug, Yamamoto, Braunecker, Aiki teaches the image processing device according to claim 1. While the combination fails to disclose the following, Davidson teaches:
Wherein the circuitry is further configured to merge the first and second image data in the illumination plane representing a merged image by calculating a product of the projected first and the second image data (Page 32, Paragraph 4, the obtained images are combined in computer memory using a multiplication operation on each pixel appearing in the images).
Note: The claimed invention specification recites “Then the image contrast maybe enhanced by calculating the product of the pixel values in this region, since high intensity regions are enhanced and low intensity regions are suppressed.” Davidson teaches “Combining the images in this fashion results in dark areas getting darker and light areas staying the same” (Page 33, Paragraph 3).
Davidson and the combination of Klug, Yamamoto, Braunecker, and Aiki are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, and Aiki to incorporate the teachings of Davidson and merge the first and second image by calculating a product. Doing so would enhance the high intensity areas and create a resulting image free of ambient light reflection (Page 33, Paragraph 3).
While the combination of Klug, Yamamoto, Braunecker, Aiki and Davidson fails to disclose the following, Jin teaches:
Representing a merged image by calculating a product of the project first and the second image data for identical parts in the projected first and second image (Paragraph 8, multiply a pixel value of each pixel of the partial region and a pixel value of each pixel of the first reference image that corresponds to the each pixel of the partial region).
Associate pixel values of the first image data and the second image data with coordinates in the illumination coordinate system such that first coordinates associated with first pixel values generated in response to detection of light originating from points which lie in the illumination plane are the same as second coordinates associated with second pixel values generated in response to detection of light originating from the same points which lie in the illumination plane (Paragraph 17, determining whether or not a partial region matching the first reference image exists in the first segmented image processed by the reference value, based on calculation results for the first pixels of the first segmented image processed by the reference value and second pixels of the first reference image corresponding to the first pixels; upon determining that the partial region exists).
Jin and the combination of Klug, Yamamoto, Braunecker, Aiki, and Davidson are both considered analogous to the claimed invention because they are in the same field of image processing and segmentation. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, Aiki, and Davidson to incorporate the teachings of Jin and calculate the product of identical parts of an image and associate first pixels with second pixels originating from the same points. Doing so would have further enhanced the high intensity areas of only the matching regions of the two images.
Method claim 12 corresponds to apparatus claim 2. Therefore, claim 12 is rejected for the same reasons as used above.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker and further in view of Aiki as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Heidemann (US 20180321383).
Regarding claim 3, the combination of Klug, Yamamoto, Braunecker, and Aiki teaches the image processing device according to claim 1. While the combination fails to disclose the following, Heidemann teaches:
Wherein the circuitry is further configured to calibrate the projection of the first and the second image data in the illumination plane based on the plurality of reference objects in the illumination plane (Paragraph 95, background objects may be used to assist in registering successive scans).
Heidemann and the combination of Klug, Yamamoto, Braunecker, and Aiki are both considered analogous to the claimed invention because they are in the same field of image processing and segmentation. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, and Braunecker to incorporate the teachings of Heidemann and use reference objects to calibrate the first and second image data. Doing so would have allowed for efficiently matching parts of different images taken from different viewpoints.
Method claim 13 corresponds to apparatus claim 3. Therefore, claim 13 is rejected for the same reasons as used above.
Claims 6-7 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over on Klug in view of Yamamoto and further in view of Braunecker and further in view of AIi and further in view of Davidson and further in view of Jin as applied to claims 2 and 12 above and further in view of Fisker (US 10835361).
Regarding claim 6, the combination of Klug, Yamamoto, Braunecker, Aiki, Davidson, and Jin teaches the image processing device according to claim 2. While the combination fails to disclose the following, Fisker teaches:
Wherein the circuitry is further configured to perform image segmentation on the merged image for determining a cross-section of the object in the illumination plane (Column 12, Lines 61-63, FIG. 4A shows cross sections of a first 3D tooth model 451 and a second 3D tooth model 451 segmented from a first and a second digital 3D representation, respectively).
Fisker and the combination of Klug, Yamamoto, Braunecker, Aiki, Davidson, and Jin are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, Aiki, Davidson, and Jin to incorporate the teachings of Fisker and perform image segmentation on a merged image to determine a cross-section of an object. Doing so would allow for readily and precisely identifying specific segments of the merged image (Fisker, Column 13, Lines 6-8).
Method claim 16 corresponds to apparatus claim 6. Therefore, claim 16 is rejected for the same reasons as above.
Regarding claim 7, the combination of Klug, Yamamoto, Braunecker, Aiki, Davidson, Jin, and Fisker teaches the image processing device according to claim 6, wherein the image segmentation is performed based on a set of predetermined cross-section criteria (Fisker, Column 2, Lines 42-44, The segmentation provides that the identified teeth and the gingiva can be separated and treated as independent 3D models of the individual teeth and the gingiva). Note: The predetermined cross-section criteria includes separating the individual teeth and the gingiva into different segments.
Fisker and the combination of Klug, Yamamoto, Braunecker, Aiki, Davidson, and Jin are both considered analogous to the claimed invention because they are in the same field of photogrammetry. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, Aiki, Davidson, and Jin to incorporate the teachings of Fisker and use pre-determined cross-section criteria. Doing so would allow for readily and precisely identifying specific segments of the merged image (Fisker, Column 13, Lines 6-8).
Method claim 17 corresponds to apparatus claim 7. Therefore, claim 17 is rejected for the same reasons as above.
Claims 8-9 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker and further in view of Aiki as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Nichol (WO 2019090139).
Regarding claim 8, the combination of Klug, Yamamoto, Braunecker, and Aiki teaches the image processing device according to claim 1. While the combination fails to disclose the following, Nichol teaches:
Wherein the object is further illuminated with a second line of light in the illumination plane or in a second illumination plane different from the illumination plane (Paragraph 290, second light source emitting light).
Nichol and the combination of Klug, Yamamoto, Braunecker, and Aiki are both considered analogous to the claimed invention because they are in the same field of illumination planes. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, and Aiki to incorporate the teachings of Nichol and use a second line of light in the illumination plane. Doing so would improve the luminance uniformity, improve the illuminance uniformity, improve the color uniformity, increase the luminance of the light emitting region, or provide a back-up light emitting region when component failure causes light from the first lightguide to fall below specification (such as color uniformity, luminance uniformity, or luminance) in the overlapping region (Nichol, Paragraph 193).
Method claim 19 corresponds to apparatus claim 8. Therefore, claim 19 is rejected for the same reasons as above.
Regarding claim 9, the combination of Klug, Yamamoto, Braunecker, Aiki, and Nichol teaches the image processing device according to claim 8. While the combination as presented previously fails to disclose the following, Nichol further teaches:
Wherein the second line of light has a center wavelength different from a center wavelength of the line of light (Paragraph 290, second light source emitting light with a second peak wavelength different from than the first peak wavelength).
Nichol and the combination of Klug, Yamamoto, Braunecker, and Aiki are both considered analogous to the claimed invention because they are in the same field of illumination planes. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, and Aiki to incorporate the teachings of Nichol and use a second line of light that has a different center wavelength from the line of light. Doing so would improve the luminance uniformity, improve the illuminance uniformity, improve the color uniformity, increase the luminance of the light emitting region, or provide a back-up light emitting region when component failure causes light from the first lightguide to fall below specification (such as color uniformity, luminance uniformity, or luminance) in the overlapping region (Nichol, Paragraph 193).
Method claim 20 corresponds to apparatus claim 9. Therefore, claim 20 is rejected for the same reasons as used above.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Klug in view of Yamamoto and further in view of Braunecker and further in view of Aiki as applied to claims 1, 4-5, 10-11, 14-15, 18, and 21 above and further in view of Valdes (US 20200249341) and Sempere (WO 2014187474).
Regarding claim 22, the combination of Klug, Yamamoto, Braunecker, and AIki teaches the image processing device according to claim 1. While the combination fails to disclose the following, Valdes teaches:
Merge the first image data and the second image data in the illumination plane representing a merged image (Paragraph 4, determining a positional shift based on a photogrammetrical difference between the first camera image and the second camera image; determining a first position of the imager in the environment at the first time and a second position of the imager in the environment at the second time based on the trajectory and the photogrammetrical difference; merging the first radio image with the second radio image based on the first position and the second position to produce a synthetic aperture radar image of the object).
Valdes and the combination of Klug, Yamamoto, Braunecker, and Aiki are both considered analogous to the claimed invention because they are in the same field of illumination planes. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, and Aiki to incorporate the teachings of Valdes and merge images. Doing so would allow for using a known method to combine similar images to produce a more detailed and accurate view of the desired object.
While the combination of Klug, Yamamoto, Braunecker, Aiki, and Valdes fails to disclose the following, Sempere teaches:
Calculating a ratio between the projected first and second image data for identical parts in the projected first and second image (Page 7, last paragraph, a mean value of the pixel by pixel ratio between first image data and second image data (i.e. the average of Imageli,.sub.j / Image2.sub.1 j)). Note:
Sempere and the combination of Klug, Yamamoto, Braunecker, Aiki, and Valdes are both considered analogous to the claimed invention because they are in the same field of illumination planes. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Klug, Yamamoto, Braunecker, Aiki, and Valdes to incorporate the teachings of Sempere and calculate the ratio between a first image and a second image. Doing so would allow for using a known method to combine similar images to produce a more detailed and accurate view of the desired object.
Response to Arguments
Applicant’s arguments with respect to claims 1, 2, and 22 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument
Regarding claim 1, Yamamoto teaches the illumination coordinate system along with the first and second direction of the illumination plane. Aiki teaches determining the first viewpoint and second viewpoint based on scanning an object with a line of light from predetermined scanning directions. Additionally, applicant argues that Yamamoto scans samples with only one image acquisition unit. However, it would have been obvious a person of ordinary skill in the art to use the multiple image acquisition units of Klug to scan from multiple viewpoints. Additionally, Braunecker teaches detecting objects while scanning. It would have been obvious to a person of ordinary skill in the art to detect objects while scanning with a line of light from different viewpoints.
Regarding claim 2, Jin teaches associating the pixel values of the first image data and second image data with coordinates such that the coordinates first pixel values generated in response to detection of light are the same as second pixel values. Jin detects matching pixels of different images or regions of images. Therefore, it would have been obvious to a person of ordinary skill in the art to use the teachings of Jin and detect matching portions of images.
Regarding claim 22, Sempere calculates a ratio between two images. Therefore, it would have been obvious to a person of ordinary skill in the art to use the teachings of Sempere and calculate a ratio of two images for merging.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SNIGDHA SINHA whose telephone number is (571)272-6618. The examiner can normally be reached Mon-Fri. 12pm-8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SNIGDHA SINHA/Examiner, Art Unit 2619
/JASON CHAN/Supervisory Patent Examiner, Art Unit 2619
Prosecution Insights