Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: “Estimation Apparatus for selecting a model for subject tracking, drive method, and program”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-3, 7-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “factor information” recited in claim 1 is unclear and confusing. The specification fails to provide a clear, defined definition for this term and one of ordinary skill in the art would not be able to determine the scope of the claim. For example, paragraph [0084] of the specification merely indicates some examples of what may be considered as factor information but does not limit the factor information to those examples.
Claims 10 and 11 similarly recite the term “factor information” and are thus similarly rejected.
Claims 2-3, 7-9 are dependent on claim 1 and are thus similarly rejected.
However, dependent claims 4 and 5 each recite limitations which further clarify the scope of the term “factor information”; thus, claims 4 and 5 are not rejected under 35 U.S.C. 112(b). Claim 6 is dependent on claim 5 and is thus similarly not rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 8, 10, 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yoon (US 20220301188 A1) in view of Grill (US 20210383225 A1).
Regarding claim 1, Yoon discloses an estimation apparatus comprising: a memory that stores a first model and a second model that have been trained through machine learning for subject tracking (Fig. 1, [0047] apparatus including a memory; [0056]-[0057] multiple neural network models may be stored in the memory, the neural networks being trained to track/determine location of target objects within images); and
a processor that receives an imaging signal from an imaging element (Fig. 1, [0047] a processor; [0050] the processor receiving an image signal; Fig. 9, [0133], [0136] the processor receiving the input image which is generated by a camera), wherein the processor is configured to execute:
a decision process of deciding on a tracking subject of a tracking target (Fig. 2, [0072] a decision of an ROI object to be tracked is made);
a first creation process of creating a first reference image for the first model including the tracking subject (Fig. 2, [0074] the detection in the first frame is used to create a reference image which will be used in detecting in the second frame; [0080]-[0081] the reference image is used by the CNN for object tracking in the next frame);
a selection process of selecting one of the first model or the second model as a selected model based on factor information (Fig. 3, [0094]-[0095] one of the available neural network models is chosen based on factor information (e.g. object feature values, aspect ratio values, network training) to perform object tracking on the present object);
an input process of inputting a captured image represented by the imaging signal into the selected model (Fig. 3, [0092] an image is input into the selected neural network model); and
an estimation process of estimating a position of the tracking subject from within the captured image by using the selected model and a reference image for the selected model out of the first reference image and the second reference image (Fig. 4, [0099] using the reference created in a first frame, [0101] and using the selected neural network model, [0103] to generate an estimation of the location of the target object in the current image frame).
Yoon fails to disclose a second reference image for the second model including the tracking subject based on the imaging signal.
Grill, in a related system from the same field of endeavor of image processing using machine learning models for purposes including subject tracking (Abstract, [0002]), discloses a first creation process of creating a first reference image for the first model including the tracking subject and a second reference image for the second model including the tracking subject based on the imaging signal (Fig. 2, [0075] first and second transformations are applied to a data item to generate first and second views (i.e. reference images) of the data item, the first and second views (i.e. reference images) are each generated in order to be input to two different neural networks (e.g. target NN and online NN); [0046] wherein the data item is images; [0047] wherein the system is performing object tracking across images/frames).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to combine Grill with Yoon and create a first reference image for the first model including the tracking subject and a second reference image for the second model including the tracking subject based on the imaging signal, as disclosed by Grill, as part of an estimation apparatus for subject tracking, as disclosed by Yoon, for the purposes of achieving high accuracy and efficiency of the trained machine-learning system for object tracking (see Grill: [0099], [0031]-[0032]).
Regarding claim 8, Yoon in view of Grill discloses the estimation apparatus according to claim 1 as applied above. Yoon further discloses wherein the processor is configured to: execute a second update process of updating the first reference image and the second reference image based on a change in size of the tracking subject within an angle of view of the captured image (Fig. 2, [0084], [0087] updating the reference image based on a new frame including updated the size of the object).
Regarding claim 10, Yoon in view of Grill discloses everything claimed as applied above (see rejection of claim 1).
Regarding claim 11, Yoon in view of Grill discloses everything claimed as applied above (see rejection of claim 1). Yoon further discloses a non-transitory computer-readable storage medium storing a program ([0069] non-transitory computer-readable medium storing instructions executable by a processor).
Claim(s) 2, 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yoon (US 20220301188 A1) in view of Grill (US 20210383225 A1) in further view of Kim (US 20210073945 A1).
Regarding claim 2, Yoon in view of Grill discloses the estimation apparatus according to claim 1 as applied above.
Yoon fails to disclose wherein the second model has a larger number of layers or a larger layer size than that of the first model.
Kim, in a related system from the same field of image processing including selecting from among neural networks (Abstract), discloses wherein the second model has a larger number of layers or a larger layer size than that of the first model (Fig. 5, Fig. 7, [0211], [0213]-[0214] some of the neural network models which are available to be selected have a higher complexity than others; [0168] wherein complexity corresponds to the number of layers in the network or other indicators of complexity).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to combine Kim with Yoon in view of Grill and include a second model which has a larger number of layers of a larger layer size than that of the first model, as disclosed by Kim, as part of an estimation apparatus for subject tracking, as disclosed by Yoon in view of Grill, for the purposes of extracting specific information from images and improving efficiency (See Kim: [0049], [0003]).
Regarding claim 9, Yoon in view of Grill discloses the estimation apparatus of claim 8 as applied above.
Yoon fails to disclose wherein the processor is configured to: execute the second update process based on a change in imaging magnification of an imaging apparatus including the imaging element.
Kim, in a related system from the same field of image processing including selecting from among neural networks (Abstract), discloses wherein the processor is configured to: execute the second update process based on a change in imaging magnification of an imaging apparatus including the imaging element (Fig. 10, [0232], [0235] a level of magnification of an image determined by a user of an apparatus is used to update the reference image and network selection).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to combine Kim with Yoon in view of Grill and execute a second update based on a change in imaging magnification, as disclosed by Kim, as part of an estimation apparatus for subject tracking, as disclosed by Yoon in view of Grill, for the purposes of extracting specific information from images and improving efficiency (See Kim: [0049], [0003]).
Allowable Subject Matter
Claims 3, 7 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claims 4-6 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 3, Yoon in view of Grill and Kim discloses the estimation apparatus according to claim 2 as applied above. However, neither Yoon nor any obvious combination of the closest known prior art discloses wherein the second reference image has a higher resolution than that of the first reference image.
Claims 4-6 are dependent on claim 3 and thus recite similarly allowable subject matter.
Regarding claim 7, Yoon in view of Grill discloses the estimation apparatus according to claim 1 as applied above. However, neither Yoon nor any obvious combination of the closest known prior art discloses wherein the processor is configured to: execute a first update process of updating the first reference image and the second reference image in a case where the selected model is switched from one of the first model or the second model to the other in the selection process.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kirchhoff (US 20190340524 A1) discloses selecting a specific AI model for a specific task, the task including image processing and detecting objects.
Lee (US 20220138493 A1) discloses adaptive tracking of a target object including generating dynamic reference images of the object for use in subsequent frames.
Shimada (SHIMADA Nobutaka et al., “Shape Estimation of Quickly Moving Hand under Complex Backgrounds for Gesture Recognition”, IEICE Trans. on Info. and Sys. Vol. J90-D No. 3 617-627, 2007.) discloses selecting from two models for tracking a target subject in images based on the movement and features of the subject.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLINE DEPALMA whose telephone number is (571)270-0769. The examiner can normally be reached Mon-Thurs 9:00am-4pm Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at 571-272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLINE E. DEPALMA/Examiner, Art Unit 2675
/ANDREW M MOYER/Supervisory Patent Examiner, Art Unit 2675