Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
RESPONSE TO ARGUMENTS
Claim Status
The examiner acknowledges the amendment of claims 1, 4, 6-10, 12-13 & 16 and the addition of claims 17-20. As to new claims 17-20, see rejections on 101 and prior art below.
112f INTERPRETATION
The examiner acknowledges the amendment of claims 1, 4, 6-10, 12-13 & 16 and the addition of claims 17-20. After carefully reviewing applicant amendments, 35 USC 112f guidance and applicant arguments, amendments are sufficient to overcome 112(f) interpretation.
PRIOR ART REJECTION
Argument 1
The examiner acknowledges the amendment of claims 1, 4, 6-10, 12-13 & 16 and the addition of claims 17-20. After carefully reviewing applicant amendments, prior art guidance and claim limitations, examiner respectfully disagrees.
Applicant submits applied reference Heo is silent to situational information indicating (1) state of the predetermined object and/or (2) an image capture situation of the captured image, and (B) power consumption as claimed. In response, examiner respectfully disagrees. [0047-0048] & Fig. 2 discloses 260, Fig. 2 wherein when reliability of the second type input image (i.e. image capture situation of the captured image) is lower than the second threshold operations 270, 280 and 290 may be performed.
In response to applicant's argument that the references fail to show certain features of applicant’s invention, it is noted that the features upon which applicant relies (i.e., As to situational information indicating (B)) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
In view of above arguments, rejection on prior art grounds is sufficient and respectfully maintained.
101 Alice
The examiner acknowledges the amendment of claims 1, 4, 6-10, 12-13 & 16 and the addition of claims 17-20. After carefully reviewing applicant amendments, 35 USC 101 Alice guidance and claim limitations, examiner respectfully disagrees.
Applicant submits "the matching method used by the first tracking unit has a lower power consumption associated with executing the tracking processing than the matching method used by the second tracking unit," with power consumption being something completely different from a mental processor and indeed being something that arises only in connection with a tangible construction of matter.
In response, examiner submits matching as disclosed is viewed as mathematical image processing implemented on generic computer components with data gathering/output with insignificant extra-solution activity.
Applicant respectfully traverses the statements in the Official Action that the claimed features "do not include additional elements that are sufficient to amount to significantly more than the judicial exception" and do "not integrate the abstract idea into a practical application or constitute significantly more than the abstract," especially in the absence of any cited documents providing evidence on this point.
In response, examiner submits matching as disclosed is viewed as mathematical image processing implemented on generic computer components with data gathering/output with insignificant extra-solution activity.
In view of above arguments, rejection on prior art grounds is sufficient and respectfully maintained.
CLAIM REJECTIONS - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to as ineligible under subject eligibility test. In the Subject Matter Eligibility Test for Products and Processes (Federal Register, Vol. 79, No. 241, dated Tuesday, December 16, 2014, page 74621), The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional device elements, which are recited at a high level of generality, provide conventional computer functions that do not add meaningful limits to practicing the abstract idea.
Claims 1 & 15
Step 1
This step inquires “is the claim to a process, machine, manufacture or composition of matter?” Yes,
Claim 1 - “Apparatuses” are machines.
Claim 15 – “Method” is a process.
Step 2A - Prong 1
This step inquires “does the claim recite an abstract idea, law or natural phenomenon”. This claim appears to directed to an abstract idea.
The limitation of “(a) at least one memory storing a computer program and at least one processor executing the computer program and/or (b) at least one circuit, that function as plurality of units comprising: {1) a first acquiring unit configured to acquire the captured image; {2) a tracking unit configured to execute tracking processing to identify a position of the image of the predetermined object included in the captured image, the tracking unit including a first tracking unit and a second tracking unit configured to execute the tracking processing using different matching methods; (3) a second acquiring unit configured to acquire situation information indicating a state of the predetermined object and/or an image capture situation of the captured image; and (4) a control unit configured to, on a basis of the situation information, switch to executing the tracking processing using either the first tracking unit or the second tracking unit on a basis of the captured image acquired by the first acquiring unit, wherein the matching method used by the first tracking unit has a lower power consumption associated with executing the tracking processing than the matching method used by the second tracking unit.”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind (e.g. mathematical concepts, mental processes or certain methods of organizing human activity) but for the recitation of generic computer components. That is, other than reciting “at least one memory storing a computer program and at least one processor executing the computer program and/or (b) at least one circuit” nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “at least one memory storing a computer program and at least one processor executing the computer program and/or (b) at least one circuit” language, “acquiring, executing & switching” in the context of this claim encompasses covers performance of the limitation in the mind (e.g. mathematical concepts, mental processes or certain methods of organizing human activity).
STEP 2A – PRONG 1 - CONCLUSION
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A - Prong 2
This step inquires “does the claim recite additional elements that integrate the judicial exception into a practical application”. This judicial exception is not integrated into a practical application. In particular, the claim recites three additional element – using a “at least one memory storing a computer program and at least one processor executing the computer program and/or (b) at least one circuit” to perform “acquiring, executing & switching” steps. The “at least one processor and/or circuit, first acquiring unit, tracking unit, second acquiring unit, control unit,” are recited at a high-level of generality (i.e., as a generic processor) “at least one processor and/or circuit configured to function as following units: a first acquiring unit configured to acquire the captured image; a tracking unit configured to execute tracking processing to identify a position of the image of the predetermined object included in the captured image, the tracking unit including a first tracking unit and a second tracking unit configured to execute the tracking processing using different matching methods; a second acquiring unit configured to acquire situation information indicating a state of the predetermined object and/or an image capture situation of the captured image; and a control unit configured to, on a basis of the situation information, switch to executing the tracking processing using either the first tracking unit or the second tracking unit on a basis of the captured image acquired by the first acquiring unit, wherein the matching method used by the first tracking unit has a lower power consumption associated with executing the tracking processing than the matching method used by the second tracking unit.” such that it amounts no more than mere instructions to apply the exception using a generic computer component.
STEP 2A – PRONG 2 - CONCLUSION
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B
The critical inquiry here is does the claim recite additional elements that amount to “significantly more” than the judicial exception? The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a “at least one processor and/or circuit, first acquiring unit, tracking unit, second acquiring unit, control unit,” to perform “acquiring, executing & switching” steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Dependent Claims
As to claim 2, this claim is directed to generic computer components (“first and second tracking unit”), and insignificant extra-solution activity (“negative recitation of “not using ML””). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 3, this claim is directed to generic computer components (“first tracking unit”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 4, this claim is directed to generic computer components (“control unit & first and second tracking unit”), mental process (“simple “if object-type = x then…else… decision”) and insignificant extra-solution activity (“type-based switching”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 5, this claim is directed to mental process (“classify “shape-changing” object”) and insignificant extra-solution activity (“adds only a domain label”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 6, this claim is directed to generic computer components (“control unit & first and second tracking unit”), mental process (“compare size to threshold”) and insignificant extra-solution activity (“threshold checking”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 7, this claim is directed to generic computer components (“control unit & first and second tracking unit”), mental process (“compare movement amount to threshold”) and insignificant extra-solution activity (“route parameter check”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 8, this claim is directed to generic computer components (“control unit & first and second tracking unit”), mental process (“check autofocus mode, decide accordingly”) and insignificant extra-solution activity (“mode based branching”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 9, this claim is directed to generic computer components (“control unit & first and second tracking unit”), mental process (“compare shutter speed to threshold”) and insignificant extra-solution activity (“metadata-based decision”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 10, this claim is directed to generic computer components (“control unit & first tracking unit”), mental process (“brightness comparison”) and insignificant extra-solution activity (“environmental check”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 11, this claim is directed to generic computer components (“second tracking unit”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 12, this claim is directed to generic computer components (“control unit & first and second tracking unit”), mental process (“compare object count to threshold”) and insignificant extra-solution activity (“basic count-based branching”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 13, this claim is directed to generic computer components (“control unit”), mental process (“detect panning, then skip tracking”) and insignificant extra-solution activity (“simple on/off switch based on shooting mode”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 14, this claim is directed to generic computer components (“image capture unit”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 16, this claim is directed to generic computer components (“a computer readable storage medium”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 17, this claim is directed to mental process (“determining state” of an object”) and insignificant extra-solution activity (“acquiring image is data gathering”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 18, this claim is directed to mental process (“deciding “is it a dog/cat/car/person?”) and insignificant extra-solution activity (“using camera image”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 19, this claim is directed to this claim is directed to mental process (“determining lighting/blur/exposure is abstract at this level”) and insignificant extra-solution activity (“simple on/off switch based on shooting mode”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 20, this claim is directed to mental process (“threshold logic”) and insignificant extra-solution activity (“brightness/motion extraction”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
CLAIM REJECTIONS - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 14-16 & 19 are rejected under 35 U.S.C. 102(a) (1) as being anticipated by
HEO et al. (U.S. Publication 2019/0130579)
As to claims 1 & 15, HEO discloses an image processing apparatus for tracking an image of a predetermined object included in an input captured image, the image processing apparatus comprising: at least one processor (110, Fig. 1 & [0031, 0034] discloses an image processing apparatus ) and/or circuit configured to function as following units: a first acquiring unit (120, Fig. 1 & [0029, 0034] discloses a camera) configured to acquire the captured image (S210, Fig. 2 & [0043] discloses detecting a target object from an input image based on a current modality); a tracking unit configured to execute tracking processing to identify a position of the image of the predetermined object included in the captured image, the tracking unit including a first tracking unit and a second tracking unit configured to execute the tracking processing using different matching methods (See In Detection Mode (upper half of Fig. 2) block 210 performs a first visible light match (Tracking 1); if the object is not found, block 240 switches modality and reruns detection in infrared using a more compute intensive match, [0043-0056]); a second acquiring unit configured to acquire situation information indicating a state of the predetermined object and/or an image capture situation of the captured image (260, Fig. 2 discloses in tracking mode (lower half of Fig. 2), the system computes reliability score or “situational information” by comparing the tracked visible light results against stored reference data); and a control unit configured to, on a basis of the situation information, switch to executing the tracking processing using either the first tracking unit or the second tracking unit on a basis of the captured image acquired by the first acquiring unit (Decision block 260, Fig. 2 tests if the reliability metric is above a threshold; if NO, control flows to block 290 to switch modality and invoke the infrared tracker 2; if Yes, it loops back to block 250 to continue using Tracker 1.), wherein the matching method used by the first tracking unit has a lower power consumption associated with executing the tracking processing than the matching method used by the second tracking unit (Blocks 210/250 uses a light weight correlation on visible light frames (low compute/power), whereas track 2 at block 240/290, Fig. 2 uses dense feature matching on infrared frames (higher compute/power)).
As to claim 2, HEO discloses everything as disclosed in claim 1. In addition, HEO discloses wherein the matching method used by the first tracking unit and the matching method used by the second tracking unit are both matching methods not using a machine learning model.
(Both Tracker 1 (210/250) and Tracker 2 (240/290) employ conventional correlation and feature matching pipelines – no machine learning model used.)
As to claim 14, HEO discloses everything as disclosed in claim 1. In addition, HEO discloses an image capturing apparatus comprising: an image capture unit (120, Fig. 1 & [0029]) configured to output a captured image; and the image processing apparatus (110, Fig. 1 & [0029]) according to claim 1. (See Claim 1 Rationale)
As to claim 16, HEO discloses everything as disclosed in claim 1. In addition, HEO discloses a computer-readable storage medium storing a program configured to cause a computer to function as the units of the image processing apparatus according to claim 1. (See Claim 1 Rationale)
As to claim 19, HEO discloses everything as disclosed in claim 1. In addition, HEO discloses wherein the situation information indicates the image capture situation of the captured image. (260, Fig. 2 discloses in tracking mode (lower half of Fig. 2), the system computes reliability score or “situational information” by comparing the tracked visible light results against stored reference data)
CLAIM REJECTIONS - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over HEO et al. (U.S. Publication 2019/0130579) in view of KIM et al. (U.S. Publication 2010/0119109)
As to claim 3, HEO discloses everything as disclosed in claim 2 but is silent to wherein the first tracking unit executes the tracking processing using a feature point matching method.
However, KIM discloses wherein the first tracking unit executes the tracking processing using a feature point matching method.
(See Abstract & Summary of Invention, See Feature Tracking Method)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HEO’s disclosure to include the above limitations in order to achieve necessary throughput and/or accelerate object tracking.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over HEO et al. (U.S. Publication 2019/0130579) in view of Sultana et al. (U.S. Publication 2012/0106790)
As to claim 4, HEO discloses everything as disclosed in claim 3 but is silent to wherein the situation information includes information indicating a type of the predetermined object, and wherein the control unit switches to executing the tracking processing using the first tracking unit when the type of the predetermined object is a first type, and switches to executing the tracking processing using the second tracking unit when the type of the predetermined object is a second type different from the first type.
However, Sultana’s [0012] discloses wherein the situation information includes information indicating a type of the predetermined object, and wherein the control unit switches to executing the tracking processing using the first tracking unit when the type of the predetermined object is a first type, and switches to executing the tracking processing using the second tracking unit when the type of the predetermined object is a second type different from the first type. ([0012] discloses a template matching module that is configured to process the processor to apply multiple differently tuned object detection classifier sets in parallel to the digital image to determine one or more of an object type…and to dynamically switch between object detection templates to match a determined object type)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HEO’s disclosure to include the above limitations in order to select a tracking first or second tracking unit based on object category rather than just a statistical score.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over HEO et al. (U.S. Publication 2019/0130579) in view of FAN (U.S. Publication 2011/0311100)
As to claim 11, HEO discloses everything as disclosed in claim 3 but is silent to wherein the second tracking unit executes the tracking processing using a template matching method
However, FAN’s [0041] discloses wherein the second tracking unit executes the tracking processing using a template matching method. ([0041] discloses template matching)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HEO’s disclosure to include the above limitations in order to provide reliable tracking spectrum of real-world conditions.
Claims 17-18 & 20 are rejected under 35 U.S.C. 103 as being unpatentable over HEO et al. (U.S. Publication 2019/0130579) in view of Carranza et al. (U.S. Publication 2019/0043207)
As to claim 17, HEO discloses everything as disclosed in claim 1 but is silent to wherein the situation information indicates the state of the predetermined object.
However, Carranza discloses wherein the situation information indicates the state of the predetermined object. (706, Fig. 7 & [0075] discloses generating metadata associated with an initial state of the object. For example, metadata may indicate the object type, identity, physical characteristics, behavior characteristics, orientation, distance from certain cameras, position, speed, direction of travel, path, trajectory)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HEO’s disclosure to include the above limitations in order to improve robustness of tracker selection and camera control under varying object conditions.
As to claim 18, HEO discloses everything as disclosed in claim 1 but is silent to wherein situation information relates to any one or more of (a) a determination that the predetermined object is a dog, (b) a determination that the predetermined object is a cat, (c) a determination that the predetermined object is a bird, (d) a determination that the predetermined object is a train, (e) a determination that the predetermined object is a car, (f) a determination that the predetermined object is a person, (g) a determination that the predetermined object is a face, or (h) a determination that the predetermined object is a pupil.
However, Carranza discloses wherein situation information relates to any one or more of (a) a determination that the predetermined object is a dog, (b) a determination that the predetermined object is a cat, (c) a determination that the predetermined object is a bird, (d) a determination that the predetermined object is a train, (e) a determination that the predetermined object is a car, (f) a determination that the predetermined object is a person, (g) a determination that the predetermined object is a face, or (h) a determination that the predetermined object is a pupil. (706, Fig. 7 & [0075] discloses generating metadata associated with an initial state of the object. For example, metadata may indicate the object type (e.g. person, car), identity, physical characteristics, behavior characteristics, orientation, distance from certain cameras, position, speed, direction of travel, path, trajectory)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HEO’s disclosure to include the above limitations in order to tailor tracking and control strategies to object types.
As to claim 20, HEO discloses everything as disclosed in claim 1 but is silent to wherein the situation information relates to any one or more of (a) a determination that the predetermined object is of atype expected to have a change in shape during tracking, (b) a determination that an area of the predetermined object in the captured image is greater than a predetermined size, (c) a determination that a movement amount of the predetermined object is greater than a predetermined value, (d) a determination of an autofocus mode, (e) a determination of whether a shutter speed is greater than a predetermined value, (f) a determination of a number of objects in the captured image being greater than a predetermined value, or (g) a determination that a brightness of an image capture environment is lower than a predetermined value.
However, Carranza discloses wherein the situation information relates to any one or more of (a) a determination that the predetermined object is of a type expected to have a change in shape during tracking, (b) a determination that an area of the predetermined object in the captured image is greater than a predetermined size, (c) a determination that a movement amount of the predetermined object is greater than a predetermined value, (d) a determination of an autofocus mode, (e) a determination of whether a shutter speed is greater than a predetermined value, (f) a determination of a number of objects in the captured image being greater than a predetermined value, or (g) a determination that a brightness of an image capture environment is lower than a predetermined value. (706, Fig. 7 & [0075] discloses generating metadata associated with an initial state of the object. For example, metadata may indicate the object type (e.g. person, car), identity, physical characteristics, behavior characteristics, orientation, distance from certain cameras, position, speed, direction of travel, path, trajectory)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HEO’s disclosure to include the above limitations in order to trigger tracker/capture adjustments under strict circumstances and filter out uninteresting objects.
CONCLUSION
No prior art has been found for claims 5-13 in their current form. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stephen P Coleman whose telephone number is (571)270-5931. The examiner can normally be reached Monday-Thursday 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Stephen P. Coleman
Primary Examiner
Art Unit 2675
/STEPHEN P COLEMAN/Primary Examiner, Art Unit 2675
Prosecution Insights