DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 18/836,931 (the instant application), filed on 8 August 2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 8 August 2024 was received and the information disclosure statement has been considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Image Processing Device, Method, and Program for Determining Image Composition and Subject Motion
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
"composition determination unit", "region of interest determination unit", and "motion information calculation unit" of claim 1
"deblurring unit" of claim 2
"image output unit" of claim 4
"image recognition unit" of claim 9
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Because these claim limitations are computer-implemented means-plus-function limitations, they are being interpreted as a combination of the structure performing the task (i.e. a general purpose computer) as well as the algorithm for performing the task which transforms the general purpose computer into a special purpose computer, see MPEP 2181.II.B. "The corresponding structure is not simply a general purpose computer by itself but the special purpose computer as programmed to perform the disclosed algorithm. Aristocrat, 521 F.3d at 1333, 86 USPQ2d at 1239. Thus, the specification must sufficiently disclose an algorithm to transform a general purpose microprocessor to the special purpose computer."
The claim limitations are interpreted as follows:
"composition determination unit"
Structure: a CPU, see [0028] and Fig. 3.
Algorithm: execute an image recognition process to detect and recognize a subject in the image; determine a rule that depends on the subject; determine a composition in accordance with the rule; see [0030]-[0031] and Fig. 4
"region of interest determination unit"
Structure: a CPU, see [0028] and Fig. 3.
Algorithm: determine a rule that depends on the subject; determine a region of interest around the subject in accordance with the rule; see [0033] and Fig. 4
"motion information calculation unit"
Structure: a CPU, see [0028] and Fig. 3.
Algorithm: calculate a point spread function (PSF) of the region of interest to determine motion information; see [0034] and Fig. 4
"deblurring unit"
Structure: a CPU, see [0028] and Fig. 3.
Algorithm: perform deconvolution using the PSF on the entire cropped image to remove blurring from the region of interest; see [0035] and Fig. 4.
Image output unit
Structure: a CPU, see [0028] and Fig. 3.
Algorithm: a specific algorithm is not required as the examiner understands outputting data as a coextensive function of a processor, see MPEP 2181.II.B. "the Federal Circuit has stated that "a microprocessor can serve as structure for a computer-implemented function only where the claimed function is ‘coextensive’ with a microprocessor itself." EON Corp. IP Holdings LLC v. AT&T Mobility LLC, 785 F.3d 616, 622, 114 USPQ2d 1711, 1714 (Fed. Cir. 2015), citing In re Katz Interactive Call Processing Patent Litigation, 639 F.3d 1303, 1316, 97 USPQ2d 1737, 1747 (Fed. Cir. 2011)."
"image recognition unit"
Structure: a CPU, see [0028] and Fig. 3.
Algorithm: detect and recognize the subject in the image using a process known in the technology; see [0029] and Fig. 4
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-8, 12, and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 6, line 6 recites "a plurality of the subjects". It is unclear if this plurality of subjects is the same or different from the plurality of subjects introduced in line 3-4. For the purpose of examination, the examiner interprets line 6 to read "the plurality of the subjects". Claims 7-8 are dependent on claim 6 and are rejected for failing to remedy the ambiguity of claim 6.
Regarding claim 7, lines 6-7 recites "a plurality of the regions of interest". It is unclear if this plurality of regions of interest is the same or different from the plurality of regions of interest introduced in line 3-4. For the purpose of examination, the examiner interprets line 6-7 to read "the plurality of the regions of interest".
Regarding claim 8, lines 7-8 recites "a plurality of the regions of interest". It is unclear if this plurality of regions of interest is the same or different from the plurality of regions of interest introduced in line 3-4. For the purpose of examination, the examiner interprets line 7-8 to read "the plurality of the regions of interest".
Regarding claim 12, line 4 recites "a rule specified by the user". It is unclear if this is the same rule as taught in claim 1 or if it is a new rule. For the purpose of examination, the examiner understands this to be the rule of claim 1 with a further limitation that it is specified by the user.
Regarding claim 14, line 6-7 recites "a rule specified by the user". It is unclear if this is the same rule as taught in claim 1 or if it is a new rule. For the purpose of examination, the examiner understands this to be the rule of claim 1 with a further limitation that it is specified by the user.
Claim Rejections - 35 USC § 112(d)
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 12 and 14 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Regarding claim 12, line 4 recites "a rule specified by the user". It is unclear if this is the same rule as taught in claim 1 or if it is a new rule. If the claim is understood to introduce a new rule then the claim fails to include all the limitations of the claim upon which it depends because it replaces and no longer uses the rule of claim 1. For the purpose of examination, the examiner understands this to be the rule of claim 1 with a further limitation that it is specified by the user. Appropriate language to correct this may be "on a basis of [[a]]the rule, wherein the rule is specified by a user in a manner that depends on the subject."
Regarding claim 14, line 6-7 recites "a rule specified by the user". It is unclear if this is the same rule as taught in claim 1 or if it is a new rule. If the claim is understood to introduce a new rule then the claim fails to include all the limitations of the claim upon which it depends because it replaces and no longer uses the rule of claim 1. For the purpose of examination, the examiner understands this to be the rule of claim 1 with a further limitation that it is specified by the user. "on a basis of [[a]]the rule, wherein the rule is specified by a user in a manner that depends on the subject."
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to a product that does not have a physical or tangible form by claiming a program, i.e. "software per se" see MPEP 2106.03.I., as a product without any structural recitations. The examiner notes that claim 20 says "a computer to execute processing" but notes that the computer itself if not claimed. Rather, the program is claimed as being capable of "causing a computer to execute processing". Therefore, the claim is not directed to one of the four categories of patent eligible subject matter and the claim is rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-5, 9-12, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (WO 2020054241 A1; US 20210352221 A1 is being used as an English translation of this patent document and all references made in this action will be to US 20210352221 A1; hereafter, Ikeda) in view of Toyohara (JP 2017108243 A) in further view of Ono et al. (US 20160112627 A1; hereafter, Ono).
Regarding claim 1, some limitations of claim 1 are being interpreted under 35 U.S.C. 112(f). Those limitations are notated here in italics text.
Ikeda discloses:
An image processing device comprising: a composition determination unit Structure: a CPU ([0146] the functions of the method may be performed by a CPU);
Algorithm: execute an image recognition process to detect and recognize a subject in the image ([0086] a recognition unit detects a subject in the image);
determine a rule that depends on the subject ([0087]-[0088] a first rule and a second rule are considered based on the subject region and [0090] it is determined which rule, i.e. shot, to use);
determine a composition in accordance with the rule ([0091] a composition is set based on the selected shot, i.e. rule);
that determines a composition in an image ([0091] a composition is set based on the selected shot, i.e. rule) in accordance with a rule predetermined in a manner that depends on a subject ([0087]-[0088] a first rule and a second rule are considered based on the subject region and [0090] it is determined which rule to use by selecting a shot based on the first or second rule);
a cropped image corresponding to the composition in the image ([0093] a cropping is performed corresponding to the composition);
Ikeda does not disclose expressly to determine a region of interest and a motion information calculation unit that calculated motion information regarding motion of the subject in the region of interest.
Toyohara discloses:
determines a region of interest (pg. 5 para. last, step 401 a main subject region is detected which is understood as a region of interest)
and a motion information calculation unit Structure: a CPU (pg. 15 para. 3, the system include a computer with processors and memory which is understood as a CPU);
Algorithm: calculate a point spread function (PSF) of the region of interest to determine motion information (pg. 10 para. 3, step 408 calculate a PSF which is understood as motion information);
that calculates motion information regarding motion of the subject predetermined in the region of interest (pg. 10 para. 3, step 408 calculate motion information i.e. PSF).
Toyohara is combinable with Ikeda because it is from the related field of endeavor of improving image quality by corrected motion blur (Toyohara pg. 1 para. last; Ikeda performs a process which also improves quality of an image, see [0007]-[0009]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the motion determination of Toyohara with the invention of Ikeda.
The motivation for doing so would have been to use that information to correct blur in the image (see Toyohara pg. 11 para. 5).
Therefore, it would have been obvious to combine Toyohara with Ikeda.
Ikeda in view of Toyohara is not being depended on to disclose a region of interest determination unit that determines a region of interest in accordance with a rule depending on the subject.
Ono discloses:
a region-of-interest determination unit Structure: a CPU ([0091] a CPU stores and operates the method);
Algorithm: determine a rule that depends on the subject ([0094] an object is recognized in the image. [0097] an area is determined based on the object information which is understood as applying a rule);
determine a region of interest around the subject in accordance with the rule ([0097] an attention area is determined, i.e. a region of interest, based on the object information which is understood as based on a rule depending on the object information);
Ono is combinable with Toyohara in view of Ikeda because it is from the related field of endeavor of improving the quality of a captured image (Ono [0008]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the region of interest determination of Ono with the invention of Ikeda in view of Toyohara.
The motivation for doing so would have been to determine "an area in which a thing that is considered as a principal object in the image by a user, who captures the image, such as a person, an animal, a plant, and a building, exists" (Ono, [0096]).
Therefore, it would have been obvious to combine Ono with Ikeda in view of Toyohara to obtain the invention as specified in claim 1.
Regarding claim 2, some limitations of claim 2 are being interpreted under 35 U.S.C. 112(f). Those limitations are notated here in italics text.
Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda is not being relied on to disclose a deblurring unit that executes a deblurring process on all of the cropped image according to the motion information.
Toyohara discloses:
The image processing device according to claim 1, further comprising a deblurring unit Structure: a CPU (pg. 15 para. 3, the system include a computer with processors and memory which is understood as a CPU);
Algorithm: perform deconvolution using the PSF on the entire cropped image to remove blurring from the region of interest (pg. 11 para. 5, step 410 correct the blur by deconvolution);
that executes a deblurring process on all of the cropped image (pg. 11 para. 5, step 410 correct the blur by deconvolution) in accordance with the motion information (pg. 11 para. 5, step 410 the correction is based on the PSF information).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the motion based blur correction of Toyohara with the invention of Ikeda.
The motivation for doing so would have been to correct blur in the image and generate an image clean of blur (see Toyohara pg. 11 para. 5).
Therefore, it would have been obvious to combine Toyohara with Ikeda to obtain the invention as specified in claim 2.
Regarding claim 4, some limitations of claim 4 are being interpreted under 35 U.S.C. 112(f). Those limitations are notated here in italics text.
Ikeda in view of Toyohara in further view of Ono disclose the subject matter of claim 2.
Ikeda further discloses:
The image processing device according to claim 2, further comprising an image output unit Structure: a CPU ([0146] the functions of the method may be performed by a CPU) that outputs the cropped image subjected to the deblurring process ([0059] cropped images are output. A person of ordinary skill in the art would understand that in combination with Toyohara a deblurred cropped image is output).
Regarding claim 5, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda in view of Toyohara is not relied upon to disclose that the region of interest is determined in accordance with a rule.
Ono discloses:
The image processing device according to claim 1, wherein the region-of-interest determination unit determines the region of interest in accordance with the rule used when the composition is determined ([0097] an attention area is determined, i.e. a region of interest, based on the object information which is understood as based on a rule depending on the object information, i.e. subject).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the region of interest determination of Ono with the invention of Ikeda in view of Toyohara.
The motivation for doing so would have been to determine "an area in which a thing that is considered as a principal object in the image by a user, who captures the image, such as a person, an animal, a plant, and a building, exists" (Ono, [0096]).
Therefore, it would have been obvious to combine Ono with Ikeda in view of Toyohara to obtain the invention as specified in claim 5.
Regarding claim 9, some limitations of claim 9 are being interpreted under 35 U.S.C. 112(f). Those limitations are notated here in italics text.
Ikeda in view of Toyohara in further view of Ono disclose the subject matter of claim 1.
Ikeda further discloses:
The image processing device according to claim 1, further comprising an image recognition unit Structure: a CPU ([0146] the functions of the method may be performed by a CPU);
Algorithm: detect and recognize the subject in the image using a process known in the technology ([0086] a recognition unit detects a subject in the image);
that executes an image recognition process of recognizing the subject from the image ([0086] a recognition unit detects a subject in the image).
Regarding claim 10, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 9.
Ikeda further discloses:
The image processing device according to claim 9, wherein the composition determination unit determines the composition on a basis of a result of the image recognition process ([0087]-[0088] a first rule and a second rule are considered based on the subject region and [0090] it is determined which rule, i.e. shot, to use. [0091] a composition is set based on the selected shot, i.e. rule).
Regarding claim 11, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 9.
Ikeda further discloses:
The image processing device according to claim 9, wherein the composition determination unit determines a number of the compositions on a basis of a result of the image recognition process ([0087]-[0088] a first rule and a second rule are considered based on the subject region and [0090] it is determined which rule, i.e. shot, to use. [0091] a composition is set based on the selected shot, i.e. rule. Determining a single composition is understood as a number of compositions as one is a number).
Regarding claim 12, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda further discloses:
The image processing device according to claim 1, wherein the composition determination unit determines the composition on a basis of a rule specified by a user ([0072] a used designates "base shots". [0087] "A base shot is an example of a first rule," The composition frame setting of [0091] is based on the base shot and first rule, therefore the composition is based on a rule specified by the user as the user may specify the base shot).
Regarding claim 18, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda further discloses:
The image processing device according to claim 1, wherein the image is captured by a PTZ camera ([0067] the camera performs panning and zooming operations and is understood as a PTZ camera).
Claims 3, 6, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (WO 2020054241 A1; US 20210352221 A1 is being used as an English translation of this patent document and all references made in this action will be to US 20210352221 A1; hereafter, Ikeda) in view of Toyohara (JP 2017108243 A) in further view of Ono et al. (US 20160112627 A1; hereafter, Ono) and of Maeda et al. (US 20130336597 A1; hereafter, Maeda).
Regarding claim 3, Ikeda in view Toyohara in further view of Ono discloses the subject matter of claim 2.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly to deblur all of the cropped image to reduce blurring of the subject in the cropped image and increase blurring of a region other than the subject.
Maeda discloses:
The image processing device according to claim 2, wherein in a case where the subject in the cropped image is moving, the deblurring unit executes the deblurring process on all of the cropped image in accordance with the motion information ([0075] step 110 deblurring is performed on the whole image which when in combination with Ikeda may be understood as the cropped image) to reduce blurring of the subject in the cropped image ([0075] blurring is corrected) and increase blurring of a region other than the subject ([0150] "However, depending on the conditions of photographing, the image may blur partially differently." Therefore, regions experience different blurring. A person of ordinary skill in the art would understand that deblurring an image that does not have uniform blur with a function that deblurs in a uniform manner will produce blurring in areas of the image).
Maeda is combinable with Ikeda in view of Toyohara in further view of Ono is from the related field of endeavor of reducing blurring in an image (Maeda, [0001]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the deblurring of Maeda with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been that applying a single blur correction involves less processing than applying blur corrections to individual regions (see Maeda Fig. 2 determines one PSF to stabilize an image versus Fig. 24 which determines three PSF and performs a cutting and a merging operation).
Therefore, it would have been obvious to combine Maeda with Ikeda in view of Toyohara in further view of Ono to obtain the invention as specified in claim 3.
Regarding claim 6, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly that when a plurality of subjects moves differently to generate a plurality of regions of interest.
Maeda discloses:
The image processing device according to claim 1, wherein in a case where there is a plurality of the subjects that moves differently in the cropped image ([0150] different regions blur differently which is understood as being equivalent to moving differently), the region-of-interest determination unit determines each of a plurality of the subjects as the region of interest ([0152] and Fig. 24 the image is divided into regions with each section as its own region).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the plural regions of interest of Maeda with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been "a clear image is obtained by correcting blurring using the corresponding PSF for each separation region and then synthesizing the image after correction" (Maeda, [0152]).
Therefore, it would have been obvious to combine Maeda with Ikeda in view of Toyohara in further view of Ono to obtain the invention as specified in claim 6.
Regarding claim 17, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly that the image is captured by a fixed camera.
Maeda discloses:
The image processing device according to claim 1, wherein the image is captured by a fixed camera ([0012] patent literature 1 discloses fixing a camera).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the fixed camera of Maeda with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been to fix a positional relationship between the subject and the camera for assistance in determining camera shake (see Maeda [0012]).
Therefore, it would have been obvious to combine Maeda with Ikeda in view of Toyohara in further view of Ono to obtain the invention as specified in claim 17.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (WO 2020054241 A1; US 20210352221 A1 is being used as an English translation of this patent document and all references made in this action will be to US 20210352221 A1; hereafter, Ikeda) in view of Toyohara (JP 2017108243 A) in further view of Ono et al. (US 20160112627 A1; hereafter, Ono) and of Maeda et al. (US 20130336597 A1; hereafter, Maeda) and of Fukunishi (US 20100034428 A1).
Regarding claim 7, Ikeda in view of Toyohara in further view of Ono and Maeda discloses the subject matter of claim 6.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly a plurality of regions of interest.
Maeda discloses:
The image processing device according to claim 6, wherein in a case where a plurality of the regions of interest is determined in the cropped image ([0152] and Fig. 24 the image is divided into regions with each section as its own region),
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the plural regions of interest of Maeda with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been "a clear image is obtained by correcting blurring using the corresponding PSF for each separation region and then synthesizing the image after correction" (Maeda, [0152]).
Therefore, it would have been obvious to combine Maeda with Ikeda in view of Toyohara in further view of Ono.
Ikeda in view of Toyohara in further view of Ono and Maeda does not disclose expressly to average pieces of motion information of the plurality of regions of interest as the motion information of the image.
Fukunishi discloses:
the motion information calculation unit calculates a value obtained by averaging pieces of the motion information regarding a plurality of the regions of interest as the motion information regarding the cropped image ([0134] "the positional displacement amount may be determined by selecting highly reliable motion vectors, and then calculating an average vector of the selected plurality of motion vectors", the "highly reliable motion vectors" come from measurement areas and measurement areas with highly reliable motion vectors may be understood as regions of interest. Therefore, the average motion information of regions of interest is determined).
Fukunishi is combinable with Ikeda in view of Toyohara in further view of Ono because it is from the related field of endeavor of correcting the blur of an image (Fukunishi [0001]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the averaging of motion information of Fukunishi with the invention of Ikeda in view of Toyohara in further view of Ono and Maeda.
The motivation for doing so would have been "the inter-image positional displacement amount is determined after excluding the effects of distortion in the optical system, and therefore double lines and image blurring do not occur following positioning" (Fukunishi [0129]).
Therefore, it would have been obvious to combine Fukunishi with Ikeda in view of Toyohara in further view of Ono and of Maeda to obtain the invention as specified in claim 7.
Regarding claim 8, Ikeda in view of Toyohara in further view of Ono and Maeda discloses the subject matter of claim 6.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly a plurality of regions of interest.
Maeda discloses:
The image processing device according to claim 6, wherein in a case where a plurality of the regions of interest is determined in the cropped image ([0152] and Fig. 24 the image is divided into regions with each section as its own region),
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the plural regions of interest of Maeda with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been "a clear image is obtained by correcting blurring using the corresponding PSF for each separation region and then synthesizing the image after correction" (Maeda, [0152]).
Therefore, it would have been obvious to combine Maeda with Ikeda in view of Toyohara in further view of Ono.
Ikeda in view of Toyohara in further view of Ono and Maeda does not disclose expressly to do a weighted sum of pieces of motion information of the plurality of regions of interest as the motion information of the image.
Fukunishi discloses:
the motion information calculation unit calculates a value obtained by giving predetermined weights to and adding up pieces of the motion information regarding a plurality of the regions of interest as the motion information regarding the cropped image ([0134] "the positional displacement amount may be determined by selecting highly reliable motion vectors, and then . . . adding the plurality of selected motion vectors weighted in accordance with the reliability", the "highly reliable motion vectors" come from measurement areas and measurement areas with highly reliable motion vectors may be understood as regions of interest. Therefore, the weighted sum of motion information of regions of interest is determined).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the weighted sum of motion information of Fukunishi with the invention of Ikeda in view of Toyohara in further view of Ono and Maeda.
The motivation for doing so would have been "the inter-image positional displacement amount is determined after excluding the effects of distortion in the optical system, and therefore double lines and image blurring do not occur following positioning" (Fukunishi [0129]).
Therefore, it would have been obvious to combine Fukunishi with Ikeda in view of Toyohara in further view of Ono and of Maeda to obtain the invention as specified in claim 8.
Claims 13 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (WO 2020054241 A1; US 20210352221 A1 is being used as an English translation of this patent document and all references made in this action will be to US 20210352221 A1; hereafter, Ikeda) in view of Toyohara (JP 2017108243 A) in further view of Ono et al. (US 20160112627 A1; hereafter, Ono) and of Kitaura et al. (US 20200314309 A1; hereafter, Kitaura).
Regarding claim 13, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly that the composition unit determines a number of composition specified by the user.
Kitaura discloses:
The image processing device according to claim 1, wherein the composition determination unit determines a number of the compositions specified by a user ([0055] a user selects a composition from a plurality of compositions. Selecting a composition is understood as specifying one composition to be determined which is a number of compositions).
Kitaura is combinable with Ikeda in view of Toyohara in further view of Ono because it is from the related field of endeavor of assisting a user in generating an ideal composition image (Kitaura [0008]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the number selection by a user of Kitaura with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been to allow the user to make the composition selection when it is difficult for the system to determine a best composition (see Kitaura [0055]).
Therefore it would have been obvious to combine Kitaura with Ikeda in view of Toyohara in further view of Ono to obtain the invention as specified in claim 13.
Regarding claim 15, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly to cause a user to select a composition from a plurality of candidate compositions.
Kitaura discloses:
The image processing device according to claim 1, wherein the composition determination unit determines a plurality of candidates for the composition in the image ([0055] two or more compositions are determined), causes a user to select one of the plurality of candidates determined ([0055] a user selects a composition), and finally determines the composition selected ([0060] the composition is output which is understood as a final determining).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to the composition selection of Kitaura with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been to allow the user to make the composition selection when it is difficult for the system to determine a best composition (see Kitaura [0055]).
Therefore, it would have been obvious to combine Kitaura with Ikeda in view of Toyohara in further view of Ono to obtain the invention as specified in claim 15.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (WO 2020054241 A1; US 20210352221 A1 is being used as an English translation of this patent document and all references made in this action will be to US 20210352221 A1; hereafter, Ikeda) in view of Toyohara (JP 2017108243 A) in further view of Ono et al. (US 20160112627 A1; hereafter, Ono) and of Fukunishi (US 20100034428 A1) and of Kitaura et al. (US 20200314309 A1; hereafter, Kitaura).
Regarding claim 16, Ikeda in view of Toyohara in further view of Ono discloses the subject matter of claim 1.
Ikeda in view of Toyohara in further view of Ono does not disclose expressly to determine a plurality of regions of interest.
Fukunishi discloses:
The image processing device according to claim 1, wherein the region-of-interest determination unit determines a plurality of candidates for the region of interest in the cropped image ([0134] "highly reliable motion vectors" come from measurement areas and measurement areas with highly reliable motion vectors may be understood as regions of interest),
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the plurality of regions of interest of Fukunishi with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been "the inter-image positional displacement amount is determined after excluding the effects of distortion in the optical system, and therefore double lines and image blurring do not occur following positioning" (Fukunishi [0129]).
Therefore, it would have been obvious to combine Fukunishi with Ikeda in view of Toyohara in further view of Ono.
Ikeda in view of Toyohara in further view of Ono and Fukunishi does not disclose expressly to select a region of interest from a plurality of regions of interest.
Kitaura discloses:
causes a user to select one of a plurality of the regions of interest determined ([0055] a user selects a composition), and finally determines the region of interest selected ([0060] the composition is output which is understood as a final determining).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the user selection of Kitaura with the invention of Ikeda in view of Toyohara in further view of Ono.
The motivation for doing so would have been combining prior art elements according to known methods to yield predictable results. The used of multiple regions of interest and of user selection in the face of multiple options is known in the art as shown above by Fukunishi and Kitaura. A person of ordinary skill in the art would be enabled to combine them as the user selection requires only the additional element of a user to make the decision. Further, in combination, each element performs the same function as it does separately, namely plural regions of interest showing various options for regions of interest and user selection allowing a user to pick the preferred option from among a series of options. A person or ordinary skill in the art would have recognized that combining these elements would yield a predictable result of a user being enable to select a preferred region of interest from among plural regions of interest.
Therefore, it would have been obvious to combine Kitaura with Ikeda in view of Toyohara in further view of Ono and Fukunishi to obtain the invention as specified in claim 16.
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (WO 2020054241 A1; US 20210352221 A1 is being used as an English translation of this patent document and all references made in this action will be to US 20210352221 A1; hereafter, Ikeda) in view of Toyohara (JP 2017108243 A).
Regarding claim 19, Ikeda discloses:
An image processing method comprising: determining a composition in an image ([0091] a composition is set based on the selected shot, i.e. rule) in accordance with a rule predetermined in a manner that depends on a subject ([0087]-[0088] a first rule and a second rule are considered based on the subject region and [0090] it is determined which rule, i.e. shot, to use);
a cropped image corresponding to the composition in the image ([0093] a cropping is performed corresponding to the composition);
Ikeda does not disclose expressly determining a region of interest and calculating motion information for the subject in the region of interest.
Toyohara discloses:
determining a region of interest (pg. 5 para. last, step 401 a main subject region is detected which is understood as a region of interest)
calculating motion information regarding motion of the subject predetermined in the region of interest (pg. 10 para. 3, step 408 calculate motion information i.e. PSF).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the motion determination of Toyohara with the invention of Ikeda.
The motivation for doing so would have been to use that information to correct blur in the image (see Toyohara pg. 11 para. 5).
Therefore, it would have been obvious to combine Toyohara with Ikeda to obtain the invention as specified in claim 19.
Regarding claim 20, claim 20 recites a program with elements corresponding to the steps recited in claim 19. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim, claim 19. Additionally, the rationale and motivation to combine Ikeda with Toyohara, presented in rejection of claim 19, apply to this claim. Finally, Toyohara discloses:
A program for causing a computer to execute processing, the processing comprising ([0146] a program stored on a memory for causing a computer to perform the disclosed functions):
Allowable Subject Matter
Claim 14 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, and 35 U.S.C. 112(d) or 35 U.S.C. 112 (pre-AIA ), 4th paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 14, the closest prior art Ikeda et al. (WO 2020054241 A1; US 20210352221 A1 is being used as an English translation of this patent document and all references made in this action will be to US 20210352221 A1; hereafter, Ikeda) in view of Toyohara (JP 2017108243 A) in further view of Ono et al. (US 20160112627 A1; hereafter, Ono) disclose the subject matter of claim 9. Kitaura et al. (US 20200314309 A1; hereafter, Kitaura), as shown in the teaching of claim 13, discloses determining a number of compositions as specified by a user which user input may be understood as a user defined rule. However, the prior art does not disclose or reasonably suggest switching between a determination of the composition based on an image recognition process and a determination of the composition based on a rule specified by a user. Specifically, the prior art does not disclose in an obvious manner a switching between those modes of composition determination.
The claim as a whole is found non-obvious over the prior art including:
switching between determination of the composition based on a result of the image recognition process and determination of the composition based on a rule specified by a user.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lin et al., US 20150116350 A1, discloses a system which applies machine learning models to determine multiple composition options of an image and to crop the image according to a composition.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA B CROCKETT whose telephone number is (571)270-7989. The examiner can normally be reached Monday-Thursday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSHUA B. CROCKETT/Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights