DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Amendments
Acknowledgment of receiving amendments to the claims, which were received by the Office on 06/04/2026.
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the arguments do not apply to the same combination of references being used in the current rejection. Applicant’s arguments are directed solely to the claimed invention as amended 06/04/2026, which has been rejected under new ground of rejection necessitated by amendment. See rejection below for full detail.
Claim Objections
Claim 8 objected to because of the following informalities:
In claim 8, lines 23-24, change “the plurality of second image” to “the plurality of second images”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 8-13 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 8 recites “generate, by learning using the learning data, a learned model configured to output the depth map indicating a distance from the optical system to a subject as an output parameter with the plurality of first images, the plurality of second images, and the plurality of third images as input parameters”.
However, applicant’s specification does not appear to teach this limitation. In claim 8, the plurality of first images, the plurality of second images, and the plurality of third images are used to generate/train the learning model. The learning model is then used to generate depth maps using a different set of first, second and third images as input parameters. Applicant’s specification does not appear to describe using the plurality of first images, the plurality of second images, and the plurality of third images to generate the learning model and then also use the plurality of first images, the plurality of second images, and the plurality of third images as inputs to the learning model to generate a depth map.
Claims 9-13 are rejected as being dependent on claim 8.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeon et al. (US 2023/0377177 A1) in view of Tagawa et al. (US 2012/0200673 A1).
Regarding claim 1, Jeon et al. (hereafter referred as Jeon) teaches a control device (Jeon, Paragraph 0060) comprising:
one or more processors comprising hardware (Jeon, Paragraph 0060), the one or more processors being configured to:
acquire a focal stack images including a set of a plurality of images taken at different focuses (Jeon, Fig. 2, aligned focal stack 20, Paragraph 0062) from an imaging device (Jeon, Fig. 2, photographing device 1, Paragraph 0062)
input the focal stack images to a learned model configured to generate a depth map of a subject indicating a distance from the optical system to the subject (Jeon, Figs. 2 and 9-11, depth estimation neural network 200, Paragraphs 0062 and 0113-0114), and
output the depth map (Jeon Fig. 2, depth map 30, Paragraphs 0113-0114).
However, Jeon does not teach the one or more processors being configured to: acquire a first image, a second image and a third image from an imaging device; control a focus adjustment mechanism of an optical system to position the optical system at a near point end position, with the optical system positioned at the near point end position, control the imaging device to generate the first image by imaging a subject with an image sensor, control the focus adjustment mechanism to move the optical system from the near point end position toward a far point end position, while the optical system is moving from the near point end position toward the far point end position, control the imaging device to generate a second image by imaging the subject with the image sensor control the focus adjustment mechanism to position the optical system at the far point end position, with the optical system positioned at the far point end position, control the imaging device to generate the third image by imaging the subject with the image sensor, the second image being an image acquired while a focus position of the optical system is changing, and the first image and the third image being images acquired in a state in which there is no change in the focus position of the optical system, input the first image, the second image, and the third image images to the learned model.
In reference to Tagawa et al. (hereafter referred as Tagawa), Tagawa teaches one or more processors being configured to:
acquire a first image, a second image and a third image (Tagawa, Fig. 4, Image A-C, Paragraphs 0044-0046) from an imaging device (Tagawa, Fig. 1, image sensor 11, Paragraph 0038);
control a focus adjustment mechanism of an optical system to position the optical system at a near point end position (Tagawa, Fig. 4, Image A, Paragraphs 0044 and 0069),
with the optical system positioned at the near point end position, control the imaging device to generate the first image by imaging a subject with an image sensor (Tagawa, Fig. 4, Image A, Paragraphs 0044 and 0069),
control the focus adjustment mechanism to move the optical system from the near point end position toward a far point end position (Tagawa, Fig. 4, Image B, Paragraphs 0045 and 0069),
while the optical system is moving from the near point end position toward the far point end position, control the imaging device to generate a second image by imaging the subject with the image sensor (Tagawa, Fig. 4, Image B, Paragraphs 0045 and 0069),
control the focus adjustment mechanism to position the optical system at the far point end position (Tagawa, Fig. 4, Image C, Paragraphs 0046 and 0069),
with the optical system positioned at the far point end position, control the imaging device to generate the third image by imaging the subject with the image sensor (Tagawa, Fig. 4, Image C, Paragraphs 0046 and 0069),
the second image being an image acquired while a focus position of the optical system is changing, and the first image and the third image being images acquired in a state in which there is no change in the focus position of the optical system (Tagawa, Fig. 4, Images A-C, Paragraphs 0044-0046 and 0069),
input the first image, the second image, and the third image images to a depth map generation unit (Tagawa, Fig. 1, Program (Elements 15-17), Paragraph 0039).
These arts are analogous since they are both related to capturing images at different focal lengths to produce a depth map. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Jeon with the method of capturing images as seen in Tagawa since it is a known method of capturing images at different focal lengths and would produce similar and expected results.
Regarding claim 20, the combination of Jeon and Tagawa teaches the control device according to claim 1 (see claim 1 analysis), wherein the first image and the third image are images acquired when the focus position of the optical system is stationary (Tagawa, Fig. 4).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeon et al. (US 2023/0377177 A1) in view of Tagawa et al. (US 2012/0200673 A1) in view of Isogai et al. (US 2012/0300114 A1).
Regarding claim 2, the combination of Jeon and Tagawa teaches the control device according to claim 1 (see claim 1 analysis). However, the combination of Jeon and Tagawa does not teach wherein the second image is at least one frame image in a video including an image group which is temporally continuous, the image group being generated by the imaging device during a period in which the focus adjustment mechanism changes a focal length between the near point end and the far point end.
In reference to Isogai et al. (hereafter referred as Isogai), Isogai teaches wherein the second image is at least one frame image in a video including an image group which is temporally continuous, the image group being generated by the imaging device during a period in which the focus adjustment mechanism changes a focal length between the near point end and the far point end (Isogai, Fig. 4, Paragraph 0078).
These arts are analogous since they are both related to capturing images at different focal lengths to produce a depth/distance map. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Jeon and Tagawa with the teaching of the second image being at least one frame image in an image group generated by the imaging device during a period in which the focus adjustment mechanism changes the focal length as seen in Isogai to improve accuracy in distance calculation by making the noise in each image equal (Isogai, Paragraph 0019-0020) and since it is a known alternative method of capturing images while changing the focal length and would produce similar and expected results.
Claim(s) 3-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeon et al. (US 2023/0377177 A1) in view of Tagawa et al. (US 2012/0200673 A1) in view of Isogai et al. (US 2012/0300114 A1) in view of Wadhwa et al. (US 2021/0183089 A1).
Regarding claim 3, the combination of Jeon, Tagawa and Isogai teaches the control device according to claim 2 (see claim 2 analysis), wherein the one or more processors being further configured to: acquire a camera parameter of the optical system (Jeon, Paragraphs 0065-0066 and 0069-0070); wherein the depth map is based on the camera parameter (Jeon, Paragraphs 0069-0070 and 0113-0114, Camera parameters are used to align the images which are then used to create the depth map.)
However, the combination of Jeon, Tagawa and Isogai does not teach estimate a shape of the subject based on the camera parameter and the depth map of the subject.
In reference to Wadhwa et al. (hereafter referred as Wadhwa) estimating a shape of the subject based on a depth map of the subject (Wadhwa, Paragraph 0027).
These arts are analogous since they are all related to imaging devices generating depth maps. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Jeon, Tagawa and Isogai with the method of determining object shapes as seen in Wadhwa to allow the device to perform depth-aware image processing or some other image processing on the image (Wadhwa, Paragraph 0027). Further, the limitation “estimate a shape of the subject based on the camera parameter and the depth map of the subject” is met since the depth mad is based on the camera parameter.
Regarding claim 4, the combination of Jeon, Tagawa, Isogai and Wadhwa teaches the control device according to claim 3 (see claim 3 analysis), wherein the one or more processors being configured to control the focus adjustment mechanism so as to stop the optical system at a near point end and a far point end (Tagawa, Fig. 4).
Regarding claim 5, the combination of Jeon, Tagawa, Isogai and Wadhwa teaches the control device according to claim 3 (see claim 3 analysis), wherein the one or more processors being configured to control the focus adjustment mechanism so as to linearly move the optical system from a near point end to a far point end (Tagawa, Fig. 4).
Regarding claim 6, the combination of Jeon, Tagawa, Isogai and Wadhwa teaches the control device according to claim 3 (see claim 3 analysis), wherein the one or more processors being configured to control the focus adjustment mechanism so as to stop the optical system at each of a near point end and a far point end within a predetermined period of time (Tagawa, Fig. 4).
Claim(s) 7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeon et al. (US 2023/0377177 A1) in view of Tagawa et al. (US 2012/0200673 A1) in view of Nishide (US 2022/0293268 A1).
Regarding claim 7, the combination of Jeon and Tagawa teaches the control device according to claim 1 (see claim 1 analysis). However, the combination of Jeon and Tagawa does not teach an endoscope system comprising: an endoscope configured to image an inside of a body of a subject; and the control device according to claim 1 connected to the endoscope, wherein the endoscope comprises the optical system, the imaging device, and the focus adjustment mechanism.
In reference to Nishide, Nishide teaches an endoscope system (Nishide, Fig. 1) comprising:
an endoscope configured to image an inside of a body of a subject (Nishide, Fig. 1, endoscope 10, Paragraph 0026); and
a control device connected to the endoscope (Nishide, Fig. 1, processor 20, Paragraph 0025-0026),
wherein the endoscope comprises an optical system, an imaging device, and a focus adjustment mechanism (Nishide, Fig. 2, imaging unit 12, Paragraph 0033).
These arts are analogous since they are all related to capturing images at different focal lengths to produce a depth/distance map (Nishide, Fig. 8, Steps S101 and S108, Paragraphs 0092 and 0099). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Jeon and Tagawa with its use for an endoscope as seen in Nishide to allow the device to generate depth maps for endoscopic images.
Regarding claim 14, Jeon et al. (hereafter referred as Jeon) teaches a control device (Jeon, Paragraph 0060) comprising:
one or more processors comprising hardware (Jeon, Paragraph 0060), the one or more processors being configured to:
acquiring, by the one or more processors, a focal stack images including a set of a plurality of images taken at different focuses (Jeon, Fig. 2, aligned focal stack 20, Paragraph 0062) from an imaging device (Jeon, Fig. 2, photographing device 1, Paragraph 0062)
inputting, by the one or more processors, the focal stack images to a learned model configured to generate a depth map of the subject indicating a distance from the optical system to the subject (Jeon, Figs. 2 and 9-11, depth estimation neural network 200, Paragraphs 0062 and 0113-0114), and
outputting, by the one or more processors, the depth map of the subject(Jeon Fig. 2, depth map 30, Paragraphs 0113-0114).
However, Jeon does not teach a medical assistant method executed by a control device including one or more processors, the medical assistant method comprising: acquiring, by the one or more processors, a first image, a second image and a third image from an imaging device; controlling a focus adjustment mechanism of an optical system to position the optical system at a near point end position, with the optical system positioned at the near point end position, controlling the imaging device to generate the first image by imaging a subject with an image sensor, controlling the focus adjustment mechanism to move the optical system from the near point end position toward a far point end position, while the optical system is moving from the near point end position toward the far point end position, controlling the imaging device to generate a second image by imaging the subject with the image sensor, controlling the focus adjustment mechanism to position the optical system at the far point end position, with the optical system positioned at the far point end position, controlling the imaging device to generate the third image by imaging the subject with the image sensor, the second image being an image acquired while a focus position of the optical system is changing, and the first image and the third image being images acquired in a state in which there is no change in the focus position of the optical system, input the first image, the second image, and the third image images to the learned model.
In reference to Tagawa et al. (hereafter referred as Tagawa), Tagawa teaches one or more processors being configured to:
acquire a first image, a second image and a third image (Tagawa, Fig. 4, Image A-C, Paragraphs 0044-0046) from an imaging device (Tagawa, Fig. 1, image sensor 11, Paragraph 0038);
control a focus adjustment mechanism of an optical system to position the optical system at a near point end position (Tagawa, Fig. 4, Image A, Paragraphs 0044 and 0069),
with the optical system positioned at the near point end position, control the imaging device to generate the first image by imaging a subject with an image sensor (Tagawa, Fig. 4, Image A, Paragraphs 0044 and 0069),
control the focus adjustment mechanism to move the optical system from the near point end position toward a far point end position (Tagawa, Fig. 4, Image B, Paragraphs 0045 and 0069),
while the optical system is moving from the near point end position toward the far point end position, control the imaging device to generate a second image by imaging the subject with the image sensor (Tagawa, Fig. 4, Image B, Paragraphs 0045 and 0069),
control the focus adjustment mechanism to position the optical system at the far point end position (Tagawa, Fig. 4, Image C, Paragraphs 0046 and 0069),
with the optical system positioned at the far point end position, control the imaging device to generate the third image by imaging the subject with the image sensor (Tagawa, Fig. 4, Image C, Paragraphs 0046 and 0069),
the second image being an image acquired while a focus position of the optical system is changing, and the first image and the third image being images acquired in a state in which there is no change in the focus position of the optical system (Tagawa, Fig. 4, Images A-C, Paragraphs 0044-0046 and 0069),
input the first image, the second image, and the third image images to a depth map generation unit (Tagawa, Fig. 1, Program (Elements 15-17), Paragraph 0039).
These arts are analogous since they are both related to capturing images at different focal lengths to produce a depth map. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Jeon with the method of capturing images as seen in Tagawa since it is a known method of capturing images at different focal lengths and would produce similar and expected results.
However, the combination of Jeon and Tagawa does not teach a medical assistant method executed by a control device including one or more processors.
In reference to Nishide, Nishide teaches a medical assistant method executed by a control device including one or more processors (Nishide, Fig. 1)
These arts are analogous since they are all related to capturing images at different focal lengths to produce a depth/distance map (Nishide, Fig. 8, Steps S101 and S108, Paragraphs 0092 and 0099). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Jeon and Tagawa with its use for an endoscope as seen in Nishide to allow the device to generate depth maps for endoscopic images.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeon et al. (US 2023/0377177 A1) in view of Tagawa et al. (US 2012/0200673 A1) in view of Nishide (US 2022/0293268 A1) in view of Isogai et al. (US 2012/0300114 A1).
Regarding claim 15, the combination of Jeon, Tagawa and Nishide teaches the medical assistant method according to claim 14 (see claim 14 analysis). However, the combination of Jeon, Tagawa and Nishide does not teach wherein the second image is at least one frame image in a video including an image group which is temporally continuous, the image group being generated by the imaging device during a period in which the focus adjustment mechanism changes a focal length between the near point end and the far point end.
In reference to Isogai et al. (hereafter referred as Isogai), Isogai teaches wherein the second image is at least one frame image in a video including an image group which is temporally continuous, the image group being generated by the imaging device during a period in which the focus adjustment mechanism changes a focal length between the near point end and the far point end (Isogai, Fig. 4, Paragraph 0078).
These arts are analogous since they are both related to capturing images at different focal lengths to produce a depth/distance map. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Jeon, Tagawa and Nishide with the teaching of the second image being at least one frame image in an image group generated by the imaging device during a period in which the focus adjustment mechanism changes the focal length as seen in Isogai to improve accuracy in distance calculation by making the noise in each image equal (Isogai, Paragraph 0019-0020) and since it is a known alternative method of capturing images while changing the focal length and would produce similar and expected results.
Claim(s) 16-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeon et al. (US 2023/0377177 A1) in view of Tagawa et al. (US 2012/0200673 A1) in view of Nishide (US 2022/0293268 A1) in view of Isogai et al. (US 2012/0300114 A1) in view of Wadhwa et al. (US 2021/0183089 A1).
Regarding claim 16, the combination of Jeon, Tagawa, Nishide and Isogai teaches the medical assistant method according to claim 15 (see claim 15 analysis), further comprising, acquiring a camera parameter of the optical system (Jeon, Paragraphs 0065-0066 and 0069-0070); wherein the depth map is based on the camera parameter (Jeon, Paragraphs 0069-0070 and 0113-0114, Camera parameters are used to align the images which are then used to create the depth map.)
However, the combination of Jeon, Tagawa, Nishide and Isogai does not teach estimating a shape of the subject based on the camera parameter and the depth map of the subject.
In reference to Wadhwa et al. (hereafter referred as Wadhwa) estimating a shape of the subject based on a depth map of the subject (Wadhwa, Paragraph 0027).
These arts are analogous since they are all related to imaging devices generating depth maps. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Jeon, Tagawa, Nishide and Isogai with the method of determining object shapes as seen in Wadhwa to allow the device to perform depth-aware image processing or some other image processing on the image (Wadhwa, Paragraph 0027). Further, the limitation “estimating a shape of the subject based on the camera parameter and the depth map of the subject” is met since the depth mad is based on the camera parameter.
Regarding claim 17, the combination of Jeon, Tagawa, Nishide, Isogai and Wadhwa teaches the medical assistant method according to claim 16 (see claim 16 analysis), further comprising controlling the focus adjustment mechanism so as to stop the optical system at a near point end and a far point end (Tagawa, Fig. 4).
Regarding claim 18, the combination of Jeon, Tagawa, Nishide, Isogai and Wadhwa teaches the medical assistant method according to claim 16 (see claim 16 analysis), further comprising controlling the focus adjustment mechanism so as to linearly move the optical system from a near point end to a far point end (Tagawa, Fig. 4).
Regarding claim 19, the combination of Jeon, Tagawa, Nishide, Isogai and Wadhwa teaches the medical assistant method according to claim 16 (see claim 16 analysis), further comprising controlling the focus adjustment mechanism so as to stop the optical system at each of a near point end and a far point end within a predetermined period of time (Tagawa, Fig. 4).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sabato et al. (US 2022/0383525 A1) in view of Tagawa et al. (US 2012/0200673 A1).
Regarding claim 8, Sabato et al. (hereafter referred as Sabato) teaches a learned model generator (Sabato, Fig. 6) comprising:
one or more processors comprising hardware (Sabato, Paragraph 0108), the one or more processors being configured to:
acquire learning data obtained by combining a plurality of first images, a plurality of second images (Sabato, Fig. 6, sequence 600, Paragraph 0235, A first and second image are images of different focus positions for a first scene. A plurality of different scenes are imaged at the different focus positions.), and a correct value of a depth map with each of a plurality of targets (Sabato, Fig. 6, ground truth depth maps 610, Paragraph 0239), from an imaging device (Sabato, Paragraphs 0236-0237),
the correct value of the depth map being related to a distance from the optical system to each of the plurality of targets (Sabato, Paragraphs 0238-0243),
generate, by learning using the learning data, a learned model configured to output a depth map indicating a distance from the optical system to a subject as an output parameter (Sabato, Fig. 6, machine learning algorithm 630, Paragraphs 0238-0243).
However, Sabato does not explicitly state a plurality of third images, and does not teach control a focus adjustment mechanism of an optical system to position the optical system at a near point end position, with the optical system positioned at the near point end position, control the imaging device to generate the plurality of first images by imaging a subject with an image sensor, control the focus adjustment mechanism to move the optical system from the near point end position toward a far point end position, while the optical system is moving from the near point end position toward the far point end position, control the imaging device to generate the plurality of second images by imaging the subject with the image sensor, control the focus adjustment mechanism to position the optical system at the far point end position, with the optical system positioned at the far point end position, control the imaging device to generate the plurality of third images by imaging the subject with the image sensor, the plurality of second images being an image acquired while a focus position of the optical system is changing, and the plurality of first images and the plurality of third images being images acquired in a state in which there is no change in the focus position of the optical system, and the learned model configured to output the depth map with the plurality of first images, the plurality of second images, and the plurality of third images as input parameters.
In reference to Tagawa et al. (hereafter referred as Tagawa), Tagawa teaches one or more processors being configured to:
acquire a first image, a second image and a third image (Tagawa, Fig. 4, Image A-C, Paragraphs 0044-0046) from an imaging device (Tagawa, Fig. 1, image sensor 11, Paragraph 0038);
control a focus adjustment mechanism of an optical system to position the optical system at a near point end position (Tagawa, Fig. 4, Image A, Paragraphs 0044 and 0069),
with the optical system positioned at the near point end position, control the imaging device to generate the first image by imaging a subject with an image sensor (Tagawa, Fig. 4, Image A, Paragraphs 0044 and 0069),
control the focus adjustment mechanism to move the optical system from the near point end position toward a far point end position (Tagawa, Fig. 4, Image B, Paragraphs 0045 and 0069),
while the optical system is moving from the near point end position toward the far point end position, control the imaging device to generate a second image by imaging the subject with the image sensor (Tagawa, Fig. 4, Image B, Paragraphs 0045 and 0069),
control the focus adjustment mechanism to position the optical system at the far point end position (Tagawa, Fig. 4, Image C, Paragraphs 0046 and 0069),
with the optical system positioned at the far point end position, control the imaging device to generate the third image by imaging the subject with the image sensor (Tagawa, Fig. 4, Image C, Paragraphs 0046 and 0069),
the second image being an image acquired while a focus position of the optical system is changing, and the first image and the third image being images acquired in a state in which there is no change in the focus position of the optical system (Tagawa, Fig. 4, Images A-C, Paragraphs 0044-0046 and 0069),
output a depth map with the first, second, and third images as input parameters (Tagawa, Fig. 1, Program (Elements 15-17), Paragraph 0039).
These arts are analogous since they are both related to capturing images at different focal lengths to produce a depth map. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Jeon with the method of capturing images as seen in Tagawa since it is a known method of capturing images at different focal lengths and would produce similar and expected results. Further, the limitations “a plurality of third images, control a focus adjustment mechanism of an optical system to position the optical system at a near point end position, with the optical system positioned at the near point end position, control the imaging device to generate the plurality of first images by imaging a subject with an image sensor, control the focus adjustment mechanism to move the optical system from the near point end position toward a far point end position, while the optical system is moving from the near point end position toward the far point end position, control the imaging device to generate the plurality of second images by imaging the subject with the image sensor, control the focus adjustment mechanism to position the optical system at the far point end position, with the optical system positioned at the far point end position, control the imaging device to generate the plurality of third images by imaging the subject with the image sensor, the plurality of second images being an image acquired while a focus position of the optical system is changing, and the plurality of first images and the plurality of third images being images acquired in a state in which there is no change in the focus position of the optical system” are met by performing the method of image capture of Tagawa at the plurality of different scenes of Sabato.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sabato et al. (US 2022/0383525 A1) in view of Tagawa et al. (US 2012/0200673 A1) in view of Isogai et al. (US 2012/0300114 A1).
Regarding claim 9, the combination of Sabato and Tagawa teaches the control device according to claim 8 (see claim 8 analysis). However, the combination of Sabato and Tagawa does not teach wherein the plurality of second images comprises at least one frame image in a video including an image group which is temporally continuous, the image group being generated by the imaging device during a period in which the focus adjustment mechanism changes a focal length between the near point end position and the far point end position.
In reference to Isogai et al. (hereafter referred as Isogai), Isogai teaches wherein the second image is at least one frame image in a video including an image group which is temporally continuous, the image group being generated by the imaging device during a period in which the focus adjustment mechanism changes a focal length between the near point end and the far point end (Isogai, Fig. 4, Paragraph 0078).
These arts are analogous since they are both related to capturing images at different focal lengths to produce a depth/distance map. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Sabato and Tagawa with the teaching of the second image being at least one frame image in an image group generated by the imaging device during a period in which the focus adjustment mechanism changes the focal length as seen in Isogai to improve accuracy in distance calculation by making the noise in each image equal (Isogai, Paragraph 0019-0020) and since it is a known alternative method of capturing images while changing the focal length and would produce similar and expected results.
Claim(s) 10-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sabato et al. (US 2022/0383525 A1) in view of in view of Tagawa et al. (US 2012/0200673 A1) in view of Isogai et al. (US 2012/0300114 A1) in view of Wadhwa et al. (US 2021/0183089 A1).
Regarding claim 10, the combination of Sabato, Tagawa and Isogai teaches the learned model generator according to claim 9 (see claim 9 analysis), wherein the one or more processors being further configured to: acquire a camera parameter of the optical system (Sabato, Fig. 6, Focus schedule 620, Paragraph 022); wherein the depth map is based on the camera parameter (Sabato, Fig. 6, Paragraphs 0238)
However, the combination of Sabato, Tagawa and Isogai does not teach estimate a shape of the subject based on the camera parameter and the depth map of the subject.
In reference to Wadhwa et al. (hereafter referred as Wadhwa) estimating a shape of the subject based on a depth map of the subject (Wadhwa, Paragraph 0027).
These arts are analogous since they are all related to imaging devices generating depth maps. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Sabato, Tagawa and Isogai with the method of determining object shapes as seen in Wadhwa to allow the device to perform depth-aware image processing or some other image processing on the image (Wadhwa, Paragraph 0027). Further, the limitation “estimate a shape of the subject based on the camera parameter and the depth map of the subject” is met since the depth mad is based on the camera parameter.
Regarding claim 11, the combination of Sabato, Tagawa, Isogai and Wadhwa teaches the learned model generator according to claim 10 (see claim 10 analysis), wherein the one or more processors being configured to control the focus adjustment mechanism so as to stop the optical system at a near point end and a far point end (Tagawa, Fig. 4).
Regarding claim 12, the combination of Sabato, Tagawa, Isogai and Wadhwa teaches the learned model generator according to claim 10 (see claim 10 analysis), wherein the one or more processors being configured to control the focus adjustment mechanism so as to linearly move the optical system from a near point end to a far point end (Tagawa, Fig. 4).
Regarding claim 13, the combination of Sabato, Tagawa, Isogai and Wadhwa teaches the learned model generator according to claim 10 (see claim 10 analysis), wherein the one or more processors being configured to control the focus adjustment mechanism so as to stop the optical system at each of a near point end and a far point end within a predetermined period of time (Tagawa, Fig. 4).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WESLEY JASON CHIU whose telephone number is (571)270-1312. The examiner can normally be reached Mon-Fri: 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Twyler Haskins can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WESLEY J CHIU/Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights