DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Regarding Claims 1-10, 12-14, 16-18 and 20, are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable respectively over claims of U.S. Patent No. 10,785,401 B2 as presented in the table below. Although the conflicting claims are not identical, they are not patentably distinct from each other because claims of the instant application are broader in scope over claims of U.S. Patent 10,785,401 B2 as presented in the table below. Claims 1-10,12-14, 16-18 and 20 of the instant application are therefore not patently distinct from the earlier U.S. Patent claims and such are unpatentable for obvious-type double patenting.
Instant application
US 10785401 B2
1. A device, comprising: a non-transitory memory storing instructions; and one or more processors in communication with the non-transitory memory, wherein the one or more processors execute the instructions to:
sample a preliminary image as image data, using a camera module; identify a first focus region, using an image classifier; identify a second focus region, using the image classifier; determine a first focus target information corresponding to the first focus region; determine a second focus target information corresponding to the second focus region; adjust a focus, based on the first focus target information, using the camera module; capture at least one first image based on the first focus target information, using the camera module; adjust the focus, based on the second focus target information, using the camera module; capture at least one second image based on the second focus target information, using the camera module; and save the at least one first image and the at least one second image to an image stack.
1. A device, comprising: a camera module, comprising: an image sensor, configured to convert a first optical scene information to a first electrical representation and convert a second optical scene information to a second electrical representation, wherein the first electrical representation includes first focus pixel information and the second electrical representation includes second focus pixel information; a lens assembly, configured to adjust focus of the first optical scene information based on a first focus control signal and adjust focus of the second optical scene information based on a second focus control signal; a controller, coupled to the image sensor and the lens assembly, and configured to continuously generate the first focus control signal based on the first focus pixel information and a first focus target information, and the second focus control signal based on the second focus pixel information and a second focus target information, and to transmit the first focus control signal and the second focus control signal to the lens assembly; a camera interconnect, coupled to the controller and to an application processor, wherein the application processor is in communication with at least one non-transitory memory, wherein the application processor executes the instructions to: identify a first focus region of the first focus pixel information, using an image classifier, wherein the image classifier identifies or tracks a first set of objects within the first optical scene information; identify a second focus region of the second focus pixel information, using the image classifier, wherein the image classifier identifies or tracks a second set of objects within the second optical scene information; generate the first focus target information corresponding to the first focus region; transmit the first focus target information to the controller through the camera interconnect; generate the second focus target information corresponding to the second focus region; transmit the second focus target information to the controller through the camera interconnect.
2. The device of claim 1, wherein the one or more processors further execute the instructions to: align the at least one first image and the at least one second image; and blend the at least one first image and the at least one second image to create a first blended image.
3. The device of claim 1, wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a radius and weight profile.
4. The device of claim 1, wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a weight mask.
5. The device of claim 4, wherein the device is configured such that one or more weights associated with the weight mask are used to establish a focus estimate.
6. The device of claim 1, wherein the device is configured such that first information from one or more focus pixels associated with the first focus region is aggregated to generate a first focus estimate for the first focus region, the first focus estimate based on the first focus target information, and second information from one or more focus pixels associated with the second focus region is aggregated to generate a second focus estimate for the second focus region, the second focus estimate based on the second focus target information.
7. The device of claim 1, wherein the device is configured such that the first focus target information and the second focus target information each include a 2D mask of values.
8. The device of claim 1, wherein the device is configured such that the first focus target information is saved as metadata associated with the at least one first image, and the second focus target information is saved as metadata associated with the at least one second image.
9. The device of claim 1, wherein the device is configured such that the first focus target information and the second focus target information each includes at least one coordinate corresponding to one or more pixel points.
10. The device of claim 1, wherein the device is configured such that at least one of the first focus target information and the second focus target information is selected by a user.
12. The device of claim 1, wherein the one or more processors further execute the instructions to: capture at least one third image based on third focus target information, the third focus target information found between the first focus target information and the second focus target information.
13. The device of claim 12, wherein the device is configured such that the first focus target information corresponds with a first focus setting, the second focus target information corresponds with a second focus setting, and the third focus target information corresponds with a third focus setting.
14. The device of claim 13, wherein the device is configured such that the first focus setting is used to capture the at least one first image, the second focus setting is used to capture the at least one second image, and the third focus setting is used to capture a third image, wherein the at least one first image, the at least one second image, and the third image are blended to create a second blended image.
16. The device of claim 3, wherein the device is configured such that the weight profile includes a start point and an end point, with a predefined function to be applied between the start point and the end point.
17. The device of claim 2, wherein the device is configured such that blending the at least one first image and the at least on second image is based on at least one of focus stacking, focal plane merging, z-stacking, and focus blending.
18. The device of claim 2, wherein the device is configured such that the first blended image preserves the first focus target information and the second focus target information such that the first focus region or the second focus region may be later selected to readjust a point of focus.
2. The device of claim 1, wherein the application processor is configured to: align a first image associated the first optical scene information and a second image associated with the second optical scene information; and blend the first image and the second image to create a first blended image.
11. The device of claim 1, wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a weight mask.
9. The device of claim 1, wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a radius and weight profile.
12. The device of claim 11, wherein the device is configured such that one or more weights associated with the weight mask are used to establish a focus estimate.
13. The device of claim 1, wherein the device is configured such that first information from one or more focus pixels associated with the first focus region is aggregated to generate a first focus estimate for the first focus region, the first focus estimate based on the first focus target information, and second information from one or more focus pixels associated with the second focus region is aggregated to generate a second focus estimate for the second focus region, the second focus estimate based on the second focus target information.
14. The device of claim 1, wherein the device is configured such that the first focus target information and the second focus target information each include a 2D mask of values.
3. The device of claim 2, wherein the device is configured such that the first focus target information is saved as metadata associated with the first image, and the second focus target information is saved as metadata associated with the second image.
15. The device of claim 1, wherein the device is configured such that the first focus target information and the second focus target information each includes at least one coordinate corresponding to one or more pixel points.
16. The device of claim 1, wherein the device is configured such that at least one of the first focus target information and the second focus target information is selected by a user.
4. The device of claim 2, wherein the controller is configured to receive a third focus target information, the third focus target information found between the first focus target information and the second focus target information.
5. The device of claim 4, wherein the device is configured such that the first focus target information corresponds with a first focus setting, the second focus target information corresponds with a second focus setting, and the third focus target information corresponds with a third focus setting.
6. The device of claim 5, wherein the device is configured such that the first focus setting is used to capture the first image, the second focus setting is used to capture the second image, and the third focus setting is used to capture a third image, wherein the at least one first image, the at least one second image, and the third image are blended to create a second blended image.
10. The device of claim 9, wherein the device is configured such that the weight profile includes a start point and an end point, with a predefined function to be applied between the start point and the end point.
7. The device of claim 2, wherein the device is configured such that blending the first image and the second image is based on at least one of focus stacking, focal plane merging, z-stacking, and focus blending.
8. The device of claim 2, wherein the device is configured such that the first blended image preserves the first focus target information and the second focus target information such that the first focus region or the second focus region may be later selected to readjust a point of focus.
20. A computer-implemented method, comprising: sampling an initial image as image data, using a camera module; identifying a first focus region, using an image classifier; identifying a second focus region, using the image classifier; determining a first focus target information corresponding to the first focus region, using a processing unit; determining a second focus target information corresponding to the second focus region, using the processing unit; adjusting a focus, based on the first focus target information, using the camera module; capturing at least one first image based on the first focus target information, using the camera module; adjusting the focus, based on the second focus target information, using the camera module; capturing at least one second image based on the second focus target information, using the camera module; and save the at least one first image and the at least one second image to an image stack, using the processing unit.
20. A computer-implemented method, comprising: receiving, using a controller coupled to an image sensor and a lens assembly, a first focus target information, wherein the image sensor is configured to convert a first optical scene information to a first electrical representation and convert a second optical scene information to a second electrical representation, the first electrical representation includes first focus pixel information and the second electrical representation includes second focus pixel information, and wherein the lens assembly is configured to adjust focus of the first optical scene information based on a first focus control signal and adjust focus of the second optical scene information based on a second focus control signal; continuously generating, by the controller, the first focus control signal based on the first focus target information and the first focus pixel information; transmitting, using the controller, the first focus control signal to the lens assembly; receiving, using the controller, a second focus target information; continuously generating, using the controller, the second focus control signal based on the second focus target information and the second focus pixel information; transmitting, using the controller, the second focus control signal to the lens assembly; identifying, using an application processor, a first focus region of the first focus pixel information using an image classifier, wherein the image classifier identifies or tracks a first set of objects within the first optical scene information; generating, using the application processor, the first focus target information corresponding to the first focus region; transmitting, using the application processor, the first focus target information to the controller through a camera interconnect; identifying, using the application processor, a second focus region of the second focus pixel information using the image classifier, wherein the image classifier identifies or tracks a second set of objects within the second optical scene information; generating, using the application processor, the second focus target corresponding to the second focus region; and transmitting, using the application processor, the second focus target information to the controller through the camera interconnect.
Statement of Statutory Basis, 35 U.S.C. §101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
claim 20 is rejected under 35 U.S.C. § 101, because the claimed invention is directed to nonstatutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because claim 20 is directed to a program per se.
Claim 20 explicitly recites
“20. A computer-implemented method: …”
Thus, only a “program” per se is actually claimed with the “computer” being intended use.
Since a computer program per se “has no physical or tangible form, and thus does not fall within any statutory category” (MPEP 2106.03), claim 12 is nonstatutory under 35 U.S.C. §101.
No claim depends on claim 20.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 10, 12-14 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Omori et al. (US 2016/0028948 A1, hereafter “Omori”), in view of Yano (US 2012/0057786 A1, hereafter “Yano”).
Regarding claim 1, Omori teaches a device (Fig. 2), comprising: a non-transitory memory storing instructions; and one or more processors in communication with the non-transitory memory (Fig. 1, [0110]-[0112]: program for realizing functions that are described in the embodiments recorded in a computer-readable recording medium, the program that is recorded in the recording medium may be read and executed by a computer system, and a process of each unit may thereby be performed. A storage apparatus 105), wherein the one or more processors execute the instructions (As illustrated in Fig.2, [0051]: control unit 103 performs control of a drive of the lens (not illustrated) of the image capturing unit 101, reception of input signals from input apparatuses (not illustrated) such as a power button and a shutter button, image display on the image display unit 102, and so forth. The control unit 103 is realized by executing a program by hardware such as a central processing unit (CPU) or a graphics processing unit (GPU) that is included in the image capturing apparatus 10) to:
sample a preliminary image as image data, using a camera module (Fig. 3, [0054]: an image photographed by the image capturing unit 101 is displayed on the image display unit 102 as a preview screen (S201));
identify a first focus region (Figs. 1-3, [0054]: operator to specify the object that the operator desires to photograph is output to the image display unit 102 (S202)),
identify a second focus region (Figs. 1-3, [0054]: the object selection reception unit 103-2 determines whether or not a different object from the object selected in S203 is further selected (S205)),
determine a first focus target information corresponding to the first focus region (Figs. 1-3, [0054]: When an input about object selection from the operator is detected (S203), the object selection reception unit 103-2 of the control unit 103 causes a focus control signal to be output from the image capturing control unit 103-3 of the control unit 103 to the image capturing unit 101 so that the focus is set on the specified object. Then, focal position information that provides the best focus, which is detected by control by the focus control unit 103-1 of the control unit 103, is recorded in the storage apparatus 105 (S204));
determine a second focus target information corresponding to the second focus region (Figs. 1-3, [0053]: S203 to S205 are repeated until the object selection by the operator is finished. Other than the configuration that repeats S203 to S205, a configuration may be used in which all the objects whose image the operator desires to capture are selected in S203, the respective focal positions of the objects are thereafter calculated in S204, and the process then progresses to S205);
adjust a focus, based on the first focus target information, using the camera module; capture at least one first image based on the first focus target information, using the camera module (Figs. 1-3, [0056]: an image is captured while the focus is set on the focal position that is decided based on the focal position information recorded in the storage apparatus 105. Adjustment of the focal position and image capturing are repeated until capturing all the images is finished);
adjust the focus, based on the second focus target information, using the camera module; capture at least one second image based on the second focus target information, using the camera module (Figs. 1-3, [0056]: an image is captured while the focus is set on the focal position that is decided based on the focal position information recorded in the storage apparatus 105. Adjustment of the focal position and image capturing are repeated until capturing all the images is finished.);
and save the at least one first image and the at least one second image to an image stack ([0070]: images of the selected objects in the respective focal positions are saved in the storage apparatus 105).
Omori does not teach identify the first focus region, using an image classifier; identify the second focus region, using the image classifier.
However, Yano discloses identify the first focus region, using an image classifier; identify the second focus region, using the image classifier (Figs. 8-9, [0106]-[0114]: determination target regions (region 1, region 2, . . . , region 6) extracted by a determination target region extraction unit 104A. Focus adjustment is then performed with priority on the determination target region determined to have great importance).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the steps of identify the first focus region, using an image classifier; identify the second focus region, using the image classifier as taught by Yano into Omori device. The suggestion/ motivation for doing so would be such that the importance of the subject can be determined more accurately, and therefore an image reflecting the intended composition of the user can be obtained (Yano: [0121]).
Regarding claim 2, the Omori and Yano combination teaches the device of Claim 1, in addition Omori discloses wherein the one or more processors further execute the instructions to: align the at least one first image and the at least one second image (Figs. 3&15, [0073]-[0075]: corresponding points between plural images used in the synthesis have to be searched for); and blend the at least one first image and the at least one second image to create a first blended image (Figs. 2&21, [0082]-[0083]: based on the focusing degree calculated in S2102, pixels (area) with the highest focusing degree in corresponding pixels (corresponding areas) are selected from the images, and the all in-focus image is thereby synthesized.).
Regarding claim 10, the Omori and Yano combination teaches the device of Claim 1, in addition Omori discloses wherein the device is configured such that at least one of the first focus target information and the second focus target information is selected by a user (Figs. 1-3, [0054]&[0083]: When an input about object selection from the operator is detected (S203), the object selection reception unit 103-2 of the control unit 103 causes a focus control signal to be output from the image capturing control unit 103-3 of the control unit 103 to the image capturing unit 101 so that the focus is set on the specified object. Then, focal position information that provides the best focus, which is detected by control by the focus control unit 103-1 of the control unit 103, is recorded in the storage apparatus 105 (S204). the plural focal positions are configured based on the selection by the operator, images are successively captured in the configured focal positions, images of all the objects whose images the operator desires to capture are clearly captured in the images that are successively captured).
Regarding claim 12, the Omori and Yano combination teaches the device of Claim 1, in addition Omori discloses wherein the one or more processors further execute the instructions to cause the camera module to capture at least one third image based on third focus target information, the third focus target information found between the first focus target information and the second focus target information ([0073]-[0075]&[0083]: plural focal positions are configured based on the selection by the operator, images are successively captured in the configured focal positions, images of all the objects whose images the operator desires to capture are clearly captured in the images that are successively captured. When images are captured in the order of the objects H3, B3, and M3 from the closer side, the respective images are an image 1501, an image 1502, and an image 1503, which are illustrated in Fig. 15).
Regarding claim 13, the Omori and Yano combination teaches the device of Claim 12, in addition Omori discloses wherein the device is configured such that the first focus target information corresponds with a first focus setting, the second focus targetApplication No.: 15/643,311Attorney's Docket No.: DUELPO25A/DLO28A Page : 5 of 17 information corresponds with a second focus setting, and the third focus target information corresponds with a third focus setting ([0072]: capturing images while changing the focus (claimed “focus setting”), capturing images from the image with the closer focal position to the image with the farther focal position provides the following effects. In a case where images are captured from the image with the closer focal position, the change in the focal position occurs in the same direction from the first image capturing to the final image capturing. Thus, an effect of reducing a focus driving time is obtained, compared to a case where the focal position is changed in a random order. That is, the order that needs the shortest time for successive image capturing may be obtained).
Regarding claim 14, the Omori and Yano combination teaches the device of Claim 13, in addition Omori discloses wherein the device is configured such that the first focus setting is used to capture the at least one first image, the second focus setting is used to capture the at least one second image, and the third focus setting is used to capture the at least one third image ([0072]: capturing images while changing the focus (claimed “focus setting”), capturing images from the image with the closer focal position to the image with the farther focal position provides the following effects. In a case where images are captured from the image with the closer focal position, the change in the focal position occurs in the same direction from the first image capturing to the final image capturing. Thus, an effect of reducing a focus driving time is obtained, compared to a case where the focal position is changed in a random order. That is, the order that needs the shortest time for successive image capturing may be obtained), wherein the at least one first image, the at least one second image and the at least one third image are blended to create a second blended image (Figs. 2&21,[0058], [0073]& [0082]-[0083]: the images are successively captured while the focal position is decided based on the selection information, and a preferable synthesized image that conforms to the intention of the operator is thereby synthesized).
Regarding claim 17, the Omori and Yano combination teaches the device of Claim 2, in addition Omori discloses wherein the device is configured such that blending the at least one first image and the at least on second image is based on at least one of focus stacking, focal plane merging, z-stacking, and focus blending ((Figs. 2&21, [0082]-[0083]: based on the focusing degree calculated in S2102, pixels (area) with the highest focusing degree in corresponding pixels (corresponding areas) are selected from the images, and the all in-focus image is thereby synthesized).
Regarding claim 18, the Omori and Yano combination teaches the device of Claim 2, in addition Omori discloses wherein the device is configured such that the first blended image preserves the first focus target information and the second focus targetApplication No.: 15/643,311Attorney's Docket No.: DUELPO25A/DLO28A Page : 6 of 17 information such that the first focus region or the second focus region may be later selected to readjust a point of focus (Figs. 2&21,[0058], [0073]& [0082]-[0083]: based on the focusing degree calculated in S2102, pixels (area) with the highest focusing degree in corresponding pixels (corresponding areas) are selected from the images, and the all in-focus image is thereby synthesized. The images are successively captured while the focal position is decided based on the selection information, and a preferable synthesized image that conforms to the intention of the operator is thereby synthesized).
Regarding claim 19, claim 19 has been analyzed and rejected with regard to claim 1 and in accordance Omori 's further teaching on: computer program product comprising computer executable instructions stored on a non-transitory computer readable medium that when executed by a processor instruct the processor [0110]-[0112]).
Regarding claim 20, Method claim 20 is drawn to the method of using the corresponding device claimed in claim 1. Therefore, method claim 20 corresponds to device claim 1 and is rejected for the same reasons of anticipation as used above.
Claims 4-7 are rejected under 35 U.S.C. 103 as being unpatentable over Omori and Yano combination as applied above to claim 1, in view of Chen (US 2008/0151097 A1, hereafter “Chen”).
Regarding claim 4, the Omori and Yano combination teaches the device of Claim 1, except wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a weight mask.
However, Chen discloses wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a weight mask (Fig. 3, [0022]: a cross mask 20 is used to perform a convolution calculation on an image 21, and the convolution sum is a definition value of the image 21 that serves as a focus value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a weight mask as taught by Chen into Omori and Yano combination. The suggestion/ motivation for doing so would be such that the time required for focus searching is greatly reduced, and focus searching efficiency is improved (Chen: [0029]).
Regarding claim 5, the Omori, Yano and Chen combination teaches the device of Claim 4, in addition Chen discloses wherein the device is configured such that one or more weights associated with the weight mask are used to establish a focus estimate ([0022]-[0024]: After the results of all the operations are added together, the definition value of the image 21 is obtained. during the initializing process, the image contrast 41, the image definition 42, and the image intensity 43 of the image 21 obtained through the above processes are normalized, and are added together to serve as a focus value 44, and the focus position 45 is searched according to the focus value 44).. The suggestion/ motivation for doing so would be such that the time required for focus searching is greatly reduced, and focus searching efficiency is improved (Chen: [0029]).
Regarding claim 6, the Omori and Yano combination teaches the device of Claim 1, except wherein the device is configured such that first information from one or more focus pixels associated with the first focus region is aggregated to generate a first focus estimate for the first focus region, the first focus estimate based on the first focus target information, and second information from one or more focus pixels associated with the second focus region is aggregated to generate a second focus estimate for the second focus region, the second focus estimate based on the second focus target information.
However, Chen discloses wherein the device is configured such that first information from one or more focus pixels associated with the first focus region is aggregated to generate a first focus estimate for the first focus region, the first focus estimate based on the first focus target information, and second information from one or more focus pixels associated with the second focus region is aggregated to generate a second focus estimate for the second focus region, the second focus estimate based on the second focus target information ([0022]&[0024]: a cross mask 20 is used to perform a convolution calculation on an image 21, and the convolution sum is a definition value of the image 21 that serves as a focus value. After the results of all the operations are added together, the definition value of the image 21 is obtained. during the initializing process, the image contrast 41, the image definition 42, and the image intensity 43 of the image 21 obtained through the above processes are normalized, and are added together to serve as a focus value 44, and the focus position 45 is searched according to the focus value 44).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the device is configured such that first information from one or more focus pixels associated with the first focus region is aggregated to generate a first focus estimate for the first focus region, the first focus estimate based on the first focus target information, and second information from one or more focus pixels associated with the second focus region is aggregated to generate a second focus estimate for the second focus region, the second focus estimate based on the second focus target information as taught by Chen into Omori and Yano combination. The suggestion/ motivation for doing so would be such that the time required for focus searching is greatly reduced, and focus searching efficiency is improved (Chen: [0029]).
Regarding claim 7, the Omori and Yano combination teaches the device of Claim 1, except wherein the device is configured such that the first focus target information and the second focus target information each include a 2D mask of values.
However, Chen discloses wherein the device is configured such that the first focus target information and the second focus target information each include a 2D mask of values ([0022]: the shape of the mask 20 is symmetric, and weight values a1 to a8 and weight positions are mainly configured. The weight on the mask 20 is multiplied by the pixel value corresponding to each weight position on the image 21 through the convolution process).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the device is configured such that the first focus target information and the second focus target information each include a 2D mask of values as taught by Chen into Omori and Yano combination. The suggestion/ motivation for doing so would be such that the time required for focus searching is greatly reduced, and focus searching efficiency is improved (Chen: [0029]).
Claims 3 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Omori and Yano combination as applied above to claim 1, Matsuyama (US 2014/0354781A1, hereafter “Matsuyama”).
Regarding claim 3, the Omori and Yano combination teaches the device of Claim 1, except wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a radius and weight profile.
However, Matsuyama discloses wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a radius and weight profile (Figs. 14-15, [0115],[0118]: in a captured image 1211, focus distance is generated based on a square frame 610 (radius) indicating the subject of interest, and weighted area of the detected subject of interest).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the device is configured such that the first focus region and the second focus region each include one or more pixel points with a radius and weight profile as taught by Matsuyama into Omori and Yano combination. The suggestion/ motivation for doing so would be to enhance focus accuracy (Matsuyama: [0007]).
Regarding claim 15, the Omori and Yano combination teaches the device of Claim 1, except wherein the one or more processors further execute the instructions to: receive updated focus target information corresponding to the identified first focus region or the identified second focus region; in response to receiving the updated focus target information, adjust the focus, based on the updated focus target information.
However, Matsuyama discloses wherein the one or more processors further execute the instructions to: receive updated focus target information corresponding to the identified first focus region or the identified second focus region; in response to receiving the updated focus target information, adjust the focus, based on the updated focus target information ([0108]-[0109]&[0112]: a functional block for detecting a subject of interest may be provided separately to the image processing unit 107. In the case where the area of a subject of interest is detected, the system control unit 104 then detects the amount of shift in focus with respect to the subject of interest, from the area of the range image corresponding to the detected area of the subject of interest. The system control unit 104 then controls the focus distance of the image capture optical system 101, so as to focus on the area of the subject of interest. Since the subject of interest is likely to be important to the photographer, the chances of being able to obtain a more desirable combined image can be enhanced).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the one or more processors further execute the instructions to: receive updated focus target information corresponding to the identified first focus region or the identified second focus region; in response to receiving the updated focus target information, adjust the focus, based on the updated focus target information as taught by Matsuyama into Omori and Yano combination. The suggestion/ motivation for doing so would be to obtain a more desirable combined image could be enhanced (Matsuyama: [0112]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Omori and Yano combination as applied above to claim 1, in view of Lee et al. (US 2014/0009636 A1, hereafter “Lee”).
Regarding claim 8, the Omori and Yano combination teaches the device of Claim 1, in addition Omori discloses focal position information that provides the best focus, which is detected by control by the focus control unit 103-1 of the control unit 103, is recorded in the storage apparatus 105 (S204);except wherein the device is configured such that the first focus target information is saved as metadata associated with the at least one first image, and the second focus target information is saved as metadata associated with the at least one second image.
However, Lee discloses wherein the device is configured such that the first focus target information is saved as metadata associated with the at least one first image, and the second focus target information is saved as metadata associated with the at least one second image ([0029]: Focus values of each focus area are measured by using a focus window of an image signal processor (ISP), and stored in a metadata form with an image. The metadata is used for searching the maximum focus value in a selected focus area).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the device is configured such that the first focus target information is saved as metadata associated with the at least one first image, and the second focus target information is saved as metadata associated with the at least one second image as taught by Lee into Omori and Yano combination. The suggestion/ motivation for doing so would be to allow searching the maximum focus value in a selected focus area (Lee: [0029]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Omori and Yano combination as applied above to claim 1, in view of Ng et al. (US 2013/0342526 A1 , hereafter “Ng”).
Regarding claim 9, the Omori and Yano combination teaches the device of Claim 1, except wherein the device is configured such that the first focus target information and the second focus target information each includes at least one coordinate corresponding to one or more pixel points.
However, Ng discloses wherein the device is configured such that the first focus target information and the second focus target information each includes at least one coordinate corresponding to one or more pixel points ([0015]: Depth-assigned content can be associated with XYZ coordinates, and/or it can be associated with objects, regions, and/or zones within the image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the device is configured such that the first focus target information and the second focus target information each includes at least one coordinate corresponding to one or more pixel points as taught by Ng into Omori and Yano combination. The suggestion/ motivation for doing so would be to allow the content item to be processed and/or displayed in connection with images (Ng: [0013]).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Omori and Yano combination as applied above to claim 1, in view of Van Dalen et al. (US 20130176458 A1 , hereafter “Van Dalen”).
Regarding claim 11, the Omori and Yano combination teaches the device of Claim 1, except wherein capturing the at least one first image includes capturing two or more first images and capturing the at least one second image includes capturing two or more second images.
However, Van Dalen discloses wherein capturing the at least one first image includes capturing two or more first images (Figs 1-3&6, [0014], [0024]-[0025], [0035]: a burst sequence may be taken of a scene including moving objects by performing a series of image captures while adjusting an exposure of the image sensor between each image capture and generate an image file).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate performing a series of image captures while adjusting an exposure of the image sensor between each image capture and generate an image file as taught by Van Dalen into Omori and Yano combination such that wherein capturing the at least one first image includes capturing two or more first images and capturing the at least one second image includes capturing two or more second images would have yielded predictable results and resulted in an improved focusing technique. The suggestion/ motivation for doing so would be to allow capturing of a shot with the best exposure (Van Dalen: [0003]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over the Omori, Yano and Matsuyama combination as applied to claim 3 above, in view of Matsuyama (US 2013/0070145 A1, hereafter “Matsuyama’145”).
Regarding claim 16 the Omori, Yano and Matsuyama combination teaches the device of Claim 3, except wherein the device is configured such that the weight profile includes a start point and an end point, with a predefined function to be applied between the start point and the end point.
However, Matsuyama’145 discloses wherein the device is configured such that the weight profile includes a start point and an end point, with a predefined function to be applied between the start point and the end point ([0047] In Fig. 3C: a first touch position designated by the user's index finger comes close to the second touch position designated by the thumb. A frame 304 is displayed at the first touch position designated by the moving index finger, indicating that the control unit 108 is tracking the touch position designated by the user. The control unit 108 changes the focus area in accordance with the positional relationship between the first touch position and the second touch position. More specifically, the focus area is gradually moved in accordance with the speed of the change in the distance between the first area and the second area. In addition, an arrow 303 is displayed as a guide from the first touch position first designated by the index finger to the frame 302 indicating the second focusing target).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the device is configured such that the weight profile includes a start point and an end point, with a predefined function to be applied between the start point and the end point as taught by Matsuyama’145 into the Omori, Yano and Matsuyama combination. The suggestion/ motivation for doing so would be to allow a user to designate a desired focus area without temporarily hiding the focusing target object with a finger or the like and missing a motion or facial expression (Matsuyama’145: [0111]).
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDELAAZIZ TISSIRE whose telephone number is (571)270-7204. The examiner can normally be reached on Monday through Friday from 8 AM to 5 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, YE LIN can be reached on 571-272-8597. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDELAAZIZ TISSIRE/ Primary Examiner, Art Unit 2638
Prosecution Insights