Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7 and 9-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
MPEP 2106 III provides a flowchart for the subject matter eligibility test for product and
processes. The claim analysis following the flowchart is as follows:
Regarding claim 1:
Step 1: Is the claim to a process, machine, manufacture, or composition of matter?
Yes, it recites an apparatus, which is a machine.
Step 2A, Prong One: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes. Claim 1 recites:
An image processing apparatus comprising:
one or more hardware processors; and
one or more memories storing one or more programs configured to be executed by the one or more hardware processors, the one or more programs including instructions for:
[1] obtaining data on at least one input image;
[2] obtaining data on a background image used to extract a foreground region from the input image;
[3] obtaining data on at least one three-dimensional model generated based on the foreground region extracted from the input image; and
[4] updating the background image based on the input image and the three-dimensional model.
The limitations of [1], [2], and [3] as drafted, are processes that under its broadest reasonable interpretation, is directed to a data gathering process without significantly more. Obtaining data of two images and the foreground as a 3D model is information that can be manually gathered and visualized by a person as a mental process.
The limitation of [4] “updating the background image based on the input image and the three-dimensional model.” is a process that under its broadest reasonable interpretation, is directed to a mental process that a person is capable of acting in their mind. More specifically, “updating the background image” is further recited to “replace the background image”, as recited in claim 2, in which a person can mentally visualize another image as a background image in the mind.
If a claim limitation, under its broadest reasonable interpretation covers performance of the limitation in the mind but for the recitation of generic instructions, then it falls within the “mental processes” grouping of the abstract idea. Accordingly, the claim recites an abstract idea.
Step 2A, Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application?
Yes, claim 1 recites the additional elements “one or more hardware processors”, and
“one or more memories storing one or more programs”.
Step 2B: Does the claim recite additional elements that amount to significantly more than the
judicial exception?
No. Processors and memories are general computer components that do not integrate with the abstract ideas recited in these claims in to practical application or amount to significantly more (see MPEP 2016.05(a), (b), and (f)).
Therefore, claim 1 is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 2, the claim depends on claim 1 with the additional limitation “wherein the one or more programs further include instructions for: in a case where the three-dimensional model is projected on the input images, updating the background image by using the input image on which the three-dimensional model is not projected to replace the background image.” This limitation further defines the mental process of “updating the background image” as “replac[ing] the background image”, as discussed previously, therefore, not amounting to significantly more than the judicial exception.
Therefore, claim 2 is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 3, the claim depends on claim 1 with the additional limitation “wherein the one or more programs further include instructions for: in a case where the three-dimensional model is projected on the input image, updating the background image by using a region of the input image on which the three-dimensional model is not projected to replace a region of the background image corresponding to the region.” Similar to claim 2, this limitation further defines the mental process of “updating the background image” as replacing the background image, therefore, not amounting to significantly more than the judicial exception.
Therefore, claim 3 is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 4, the claim depends on claim 1 with the additional limitation “wherein the one or more programs further include instructions for: obtaining object information concerning a target object; specifying a target three-dimensional model corresponding to the target object from the three-dimensional models based on the object information”. This limitation further recites data being gathered as input without significantly more, where identifying and classifying an object and its information in the foreground can be done as a mental process.
Therefore, claim 4 is not eligible subject matter under 35 U.S.C. 101.
Claims 5 and 6 have substantially similar limitations to claim 2 and 3 respectively, therefore, recite additional limitations do not amount to significantly more than the judicial exception. Therefore, claims 5 and 6 are not eligible subject matter under 35 U.S.C. 101.
Regarding claim 7, the claim depends on claim 1 with the additional limitation “wherein the one or more programs further include instructions for: obtaining data on an image obtained through image capturing by an image capturing apparatus as the data on the input image.” Even though claim 7 has the additional element of “an image capturing apparatus”, obtaining an image is directed towards a data gathering process without significantly more.
Therefore, claim 7 is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 9, the claim depends on claim 1 with the additional limitation “wherein the one or more programs further include instructions for: extracting the foreground region from the input image using the background image.” This limitation further recites data being gathered as input without significantly more, where separating the foreground to the background can be done as a mental process.
Therefore, claim 9 is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 10, the claim depends on claim 1 with the additional limitation “wherein the one or more programs further include instructions for: obtaining the data on the three-dimensional model by generating the three-dimensional model based on the foreground region.” This limitation further recites data gathering of a 3D model from a foreground region, where the 3D object in the foreground region can be visualized as a mental process.
Therefore, claim 10 is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 11, claim 11 recites substantially similar limitations to claim 1 as discussed above, but as a method, therefore, recites limitations directed to a mental process without amounting to significantly more than the judicial exception.
Therefore, claim 11 is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 12, claim 12 recites substantially similar limitations to claim 1 as discussed above, but with the additional element of storage media and instructions. Storage media and instructions are generic computer component that do not integrate with the abstract ideas recited in these claims into practical application or amount to significantly more (see MPEP 2016.05(a), (b), and (f)).
Therefore, claim 12 is not eligible subject matter under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-7 and 9-12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yamauchi (JP 2023013540 A, see English Translation).
Regarding claim 1, Yamauchi teaches an image processing apparatus comprising: (image processing device 100 [Fig. 1 and Fig. 2], where “The image processing apparatus according to this disclosure includes:” [0006])
one or more hardware processors; (CPU 211 [Fig. 2], where “The CPU 211 is a processor that controls the computer using programs or data stored in the ROM 212 or RAM 213, thereby enabling the computer to function as one of the components of the image processing device 100 shown in Figure 1” [0016])
and one or more memories storing one or more programs configured to be executed by the one or more hardware processors, (RAM 213 [Fig. 2], where “The CPU 211 is a processor that controls the computer using programs or data stored in the ROM 212 or RAM 213, thereby enabling the computer to function as one of the components of the image processing device 100 shown in Figure 1” [0016])
the one or more programs including instructions for: ([0074] “The present disclosure can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program.”)
obtaining data on at least one input image; ([0012] “The image processing device 100 takes captured image data, which is the target of foreground region extraction, as input”, where “The image acquisition unit 110 acquires the captured image.” [0019])
obtaining data on a background image used to extract a foreground region from the input image; ([0020] “The foreground acquisition unit 120 separates and acquires the foreground and background regions in the captured image based on the difference between the captured image and the background image acquired by the image acquisition unit 110”)
obtaining data on at least one three-dimensional model generated based on the foreground region extracted from the input image; ([0023] “The shape generation unit 160 generates shape data (hereinafter referred to as "3D model") that represents the three-dimensional shape corresponding to the foreground object, based on the foreground silhouette images generated by each foreground separation unit 170, which is possessed by each of the multiple foreground-background separation units 101”)
and updating the background image based on the input image and the three-dimensional model (background update control unit 150 [Fig. 1], where “and update control means for controlling the updating of the background image based on the amount of movement of the foreground region corresponding to an object acquired by the foreground movement amount acquisition means and the amount of movement of the 3D model in virtual space acquired by the model movement amount acquisition means.” [0006]).
Regarding claim 2, Yamauchi teaches the image processing apparatus according to claim 1, wherein the one or more programs further include instructions for: in a case where the three-dimensional model is projected on the input images, updating the background image by using the input image on which the three-dimensional model is not projected to replace the background image ([0051] “the background update control unit 150 controls the background image so that it does not update the image region corresponding to the foreground region”, where “the background update control unit 150 may control the background image to update image regions corresponding to the foreground region, the region obtained by applying an expansion process to the foreground region, the region obtained by reprojecting the 3D model, or the region composed of these regions.” [0051]).
Regarding claim 3, Yamauchi teaches the image processing apparatus according to claim 1, wherein the one or more programs further include instructions for: in a case where the three-dimensional model is projected on the input image, updating the background image by using a region of the input image on which the three-dimensional model is not projected to replace a region of the background image corresponding to the region. ([0051] “the background update control unit 150 controls the background image so that it does not update the image region corresponding to the foreground region”, where “the background update control unit 150 may control the background image to update image regions corresponding to the foreground region, the region obtained by applying an expansion process to the foreground region, the region obtained by reprojecting the 3D model, or the region composed of these regions.” [0051]).
Regarding claim 4, Yamauchi teaches the image processing apparatus according to claim 1, wherein the one or more programs further include instructions for: obtaining object information concerning a target object; ([0043] “Next, in S604, the model movement amount acquisition unit 140 calculates the movement amount (model movement amount) of each 3D model for each 3D model corresponding to the foreground object using the centroid coordinates of each 3D model.”, where “the model movement amount acquisition unit 140 performs labeling processing on each of the acquired 3D shape models, thereby assigning identification information (hereinafter referred to as "model ID") to each 3D model.” [0042]. Note: the “centroid coordinates of each 3D model” is mapped to object information, and “each 3D model” with “model ID” is mapped to target object.)
specifying a target three-dimensional model corresponding to the target object from the three-dimensional models based on the object information; ([0042] “the model movement amount acquisition unit 140 performs labeling processing on each of the acquired 3D shape models, thereby assigning identification information (hereinafter referred to as "model ID") to each 3D model.”, where “Model IDs can be assigned using the centroid coordinates of each 3D model” [0042]).
and updating the background image based on the input image and the target three- dimensional model. ([0049] “After S712, in S713, the background update control unit 150 associates the foreground area with the 3D model, thereby obtaining the foreground ID and model ID as information for identifying foreground objects (hereinafter referred to as "object ID").”)
Regarding claim 5, claim 5 recites substantially similar limitations to claim 2, therefore, rejected under the same rationale as claim 3.
Regarding claim 6, claim 6, recites substantially similar limitations to claim 3, therefore, rejected under the same rationale as claim 3.
Regarding claim 7, Yamauchi teaches the image processing apparatus according to claim 1, wherein the one or more programs further include instructions for: obtaining data on an image obtained through image capturing by an image capturing apparatus as the data on the input image ([0019] “Specifically, the image acquisition unit 110 acquires captured images by acquiring captured image data output by a corresponding imaging device”).
Regarding claim 9, Yamauchi teaches the image processing apparatus according to claim 1, wherein the one or more programs further include instructions for: extracting the foreground region from the input image using the background image. ([0020] “The foreground acquisition unit 120 separates and acquires the foreground and background regions in the captured image based on the difference between the captured image and the background image acquired by the image acquisition unit 110, using a general foreground-background separation method based on background subtraction.”).
Regarding claim 10, Yamauchi teaches the image processing apparatus according to claim 1, wherein the one or more programs further include instructions for: obtaining the data on the three-dimensional model by generating the three-dimensional model based on the foreground region. ([0023] “The shape generation unit 160 generates shape data (hereinafter referred to as "3D model") that represents the three-dimensional shape corresponding to the foreground object, based on the foreground silhouette images generated by each foreground separation unit 170, which is possessed by each of the multiple foreground-background separation units 101”)
Regarding claim 11, claim 11 recites substantially similar limitations to claim 1, but in a method form. Yamauchi further teaches an image processing method comprising the steps of: ([0001] “This disclosure relates to a technique for extracting the foreground region in an image.”).
Regarding claim 12, claim 12 recites substantially similar limitations to claim 1, but in a medium form. Yamauchi further teaches a non-transitory computer readable storage medium storing a program for causing a computer to perform a control method of an image processing apparatus, the control method comprising the steps of: ([0074] “The present disclosure can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program.”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 8 are rejected under 35 U.S.C. 103 as being unpatentable over Yamauchi, in view of Handa (US 2023/0291865 A1).
Yamauchi teaches the image processing apparatus according to claim 7, but fails to teach wherein the one or more programs further include instructions for: executing projection processing of the three-dimensional model on the input image based on a camera parameter of the image capturing apparatus. However, this is known in the art as taught by Handa.
Handa teaches wherein the one or more programs further include instructions for: executing projection processing of the three-dimensional model on the input image based on a camera parameter of the image capturing apparatus. ([0030] “The foreground model generation unit 203 generates a 3D model (foreground model), which is shape data representing the three-dimensional shape of the object, based on the silhouette image generated by the foreground extraction unit 202 and the camera parameters of each of the actual cameras 101 and 102.”).
Handa is analogous to the claimed invention, as both relates to generating a three-dimensional model from foreground extracted from an image. Handa further teaches that “according to the present embodiment, it is made possible to provide an image in which the image quality is secured to a viewer without the need to considerably increase the number of actual cameras or augment the network.” [0065]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Handa to Yamauchi in order to output an image without having to greatly increase the number of cameras or network resources.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALICIA HA whose telephone number is (571)272-3601. The examiner can normally be reached Mon-Thurs 9:30 AM - 6:30 PM, and Fri 9:30 AM - 1:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALICIA HA/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights