DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of priority from Foreign Application No. JP2023-192121, filed November 10, 2023.
Information Disclosure Statement
The information disclosure statement (“IDS”) filed on October 7, 2024 was reviewed and the listed references were noted.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a system, method, and non-transitory computer-readable medium for object detection. Consider method claim 6:
Step 1:
With regard to Step 1, the instant claim is directed to a method or a process; and therefore, the claim is directed to one of the statutory categories of invention.
Step 2A, Prong One:
With regard to 2A, Prong One, the limitations “predicting a position of an object in a new input image based on a position of the object detected in a past input image”,
“determining an object detection target area in the new input image based on the prediction result” and “performing object detection on the determined object detection target area” as drafted, recite an abstract idea, such as a process that, under its broadest reasonable interpretation, covers performance of the limitations manually and in the mind of a person. That is, a user or person skilled in the art may predict the location of an object in an image based on the location of the object in an image taken immediately before, determine a target area to detect an object based on the prediction, and perform object detection in that area. This is the concept that falls under the grouping of abstract ideas mental processes, i.e., a concept performed in the human mind, evaluation, judgement, and/or opinion of the user.
Step 2A, Prong Two:
The 2019 PEG defines the phrase “integration into a practical application” to require an additional step or a combination of additional steps in the claim to apply, rely on, or use the judicial exception. In addition, with respect to the device claims of claims 1-5 and 16-18 and computer-readable medium claims of claims 11-15, the mere recitation of a generic processor, memory, or storage medium to perform/store programming instructions of the recited/identified abstract idea does not integrate the identified abstract idea into a practical application. Accordingly, the above-mentioned additional elements/limitations do not integrate the abstract idea into a practical application; and therefore, the independent claims recite an abstract idea.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B. The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception, because as discussed above with respect to integration of the abstract idea into practical application, the additional elements/limitations to perform the recited steps, amount to no more than insignificant extra-solution activity. Mere instructions to apply an exception using a generic component cannot provide an inventive concept. Therefore, independent claims 1, 6 and 11 are not patent eligible. In addition, claims 2-5, 7-10 and 12-18 of the instant application provide limitations that both individually or in combination do not integrate the identified abstract idea into a practical application or provide significantly more than the identified abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 6 and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kim et al. (US 12,524,987 B2, filed December 29, 2022).
Regarding claim 1, Kim teaches an image processing device comprising:
a memory storing software instructions (Col. 5, lines 53-58, the multi-scale object detection device may include a memory); and
one or more processors configured to execute the software instructions (Col. 5, lines 53-58, the multi-scale object detection device may include a processor) to:
predict a position of an object in a new input image based on a position of the object detected in a past input image (Col. 7, lines 10-21, the critical region extractor may extract at least one second critical region from a current image frame based on at least one first critical region extracted from a previous image frame among consecutive image frames. Here, the critical region may correspond to a region in which an object appears to exist in the image frame, and the first and second critical regions may correspond to regions in which objects appear to exist in the previous and current image frames, respectively);
determine an object detection target area in the new input image based on the prediction result (Col. 7, lines 10-21, the critical region extractor may extract at least one second critical region from a current image frame (i.e., object detection target area) based on at least one first critical region extracted from a previous image frame among consecutive image frames. Here, the critical region may correspond to a region in which an object appears to exist in the image frame, and the first and second critical regions may correspond to regions in which objects appear to exist in the previous and current image frames, respectively) and
perform object detection on the determined object detection target area (Col. 8, lines 34-49, the multi-scale object detector may undergo a first object detection process for the current image and a second object detection process for at least one second critical region (i.e., object detection on the determined object detection target area).
Claim 6 recites a method with steps corresponding to the elements of the system recited in Claim 1. Therefore, the recited steps of this claim are mapped to the proposed reference in the same manner as the corresponding elements in its corresponding system claim.
Claim 11 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 6. Therefore, the recited programming instructions of this claim are mapped to the proposed reference in the same manner as the corresponding steps in its corresponding method claim. Additionally, the Kim reference discloses a computer readable storage medium (Col. 4, lines 39-48, computer-readable code on a computer-readable recording medium).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-4, 7-9 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 12,524,987 B2, filed December 29, 2022) in view of Sundaresan et al. (US 10,628,961 B2).
Regarding claim 2, Kim teaches the image processing device according to claim 1, wherein the one or more processors are further configured to execute the software instructions to:
perform a second object detection on the new input image (Kim, Col. 8, lines 34-49, the multi-scale object detector may undergo a first object detection process for the current image (i.e., perform object detection on new input image) and a second object detection process for at least one second critical region).
Although Kim teaches performing a second object detection on the input image (Kim, Col. 8, lines 34-49), Kim does not explicitly teach to “update the object detection target area based on the object detection result of the second object detection”. However, in an analogous field of endeavor, Sundaresan teaches the first step is to predict a tracker's state, and the second step is to use measurements to correct or update the state. In this case, the tracker from the last frame can predict its location in the current frame. When the current frame is received, the tracker can use the measurement of the object in the current frame (i.e., object detection result of the second object detection) to correct its location in the current frame, and then can predict its location in the next frame (Sundaresan, Col. 13, lines 35-52).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the device of Kim with the teachings of Sundaresan by including updating the object detection target area (i.e., tracker) based on the object detection result of the second object detection. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for detecting and tracking objects in images using neural network based detection, as recognized by Sundaresan. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 3, Kim in view of Sundaresan teaches the image processing device according to claim 2, wherein
the one or more processors identify a new object that is not included in the object detection target area determined based on the prediction result, among the objects detected by the second object detection, and update the object detection target area to include the identified new object (Sundaresan, Col. 16, lines 18-34, the trackers maintained by the lightweight tracker can be updated after the lightweight tracker has been initialized. For example, when new objects are detected by the neural network detection system and the newly detected objects are tracked by the strong object tracker, the tracked objects are input (e.g., as bounding boxes) to the lightweight tracker thread. The lightweight tracker thread can update the tracked objects maintained by the lightweight object tracker and/or can associate (or merge) the newly tracked objects (based on the new object detection results) with the tracked objects currently being maintained by the lightweight object tracker).
The proposed combination as well as the motivation for combining the Kim and Sundaresan references presented in the rejection of Claim 2, apply to Claim 3 and are incorporated herein by reference. Thus, the device recited in Claim 3 is met by Kim in view of Sundaresan.
Regarding claim 4, Kim in view of Sundaresan teaches the image processing device according to claim 3, wherein
the one or more processors update the object detection target area to include the new object whose position information has been updated based on the object detection result of the object detection (Sundaresan, Col. 16, lines 18-34, the trackers maintained by the lightweight tracker can be updated after the lightweight tracker has been initialized. For example, when new objects are detected by the neural network detection system and the newly detected objects are tracked by the strong object tracker, the tracked objects are input (e.g., as bounding boxes) to the lightweight tracker thread. The lightweight tracker thread can update the tracked objects maintained by the lightweight object tracker and/or can associate (or merge) the newly tracked objects (based on the new object detection results) with the tracked objects currently being maintained by the lightweight object tracker).
The proposed combination as well as the motivation for combining the Kim and Sundaresan references presented in the rejection of Claim 2, apply to Claim 4 and are incorporated herein by reference. Thus, the device recited in Claim 4 is met by Kim in view of Sundaresan.
Claims 7-9 recite methods with steps corresponding to the elements of the devices recited in Claims 2-4, respectively. Therefore, the recited steps of these claims are mapped to the proposed combination in the same manner as the corresponding elements in their corresponding system claims. Additionally, the rationale and motivation to combine the Kim and Sundaresan references, presented in rejection of Claim 2, apply to this claim.
Claims 12-14 recite computer-readable storage mediums storing programs with instructions corresponding to the steps recited in Claims 7-9, respectively. Therefore, the recited programming instructions of these claims are mapped to the proposed combination in the same manner as the corresponding steps in their corresponding method claims. Additionally, the rationale and motivation to combine the Kim and Sundaresan references, presented in rejection of Claim 2, apply to these claims. Finally, the combination of the Kim and Sundaresan references discloses a computer readable storage medium (Kim, Col. 4, lines 39-48, computer-readable code on a computer-readable recording medium).
Claims 5, 10 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 12,524,987 B2, filed December 29, 2022) in view of Jong Hu Jeong (US 11,869,212 B1, filed February 7, 2023).
Regarding claim 5, Kim teaches the image processing device according to claim 1, as described above.
Although Kim teaches determining an object detection target area based on a previous frame (Kim, Col. 7, lines 10-21), Kim does not explicitly teach “the one or more processors use independent information for each time point when using position information of objects detected in a plurality of input images generated at different times as input for prediction”. However, in an analogous field of endeavor, Jeong teaches performing object detection using a current frame image and the prior information, which is the temporal information related to the location information of the bounding boxes according to object detection information detected from the previous frame image (Jeong, Col. 17, lines 24-40).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the device of Kim with the teachings of Jeong by including using independent information for each time point (i.e., prior information which is the temporal information of bounding boxes from the previous frame) when using position information of objects detecting in input images generated at different times as input for prediction (i.e., object detection). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for improving robustness of video object detection, as recognized by Jeong. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 10 recites a method with steps corresponding to the elements of the system recited in Claim 5. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Kim and Jeong references, presented in rejection of Claim 5, apply to this claim.
Claim 15 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 10. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Kim and Jeong references, presented in rejection of Claim 5, apply to this claim. Finally, the combination of the Kim and Jeong references discloses a computer readable storage medium (Kim, Col. 4, lines 39-48, computer-readable code on a computer-readable recording medium).
Claims 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 12,524,987 B2, filed December 29, 2022) in view of Sundaresan et al. (US 10,628,961 B2), as applied to claims 2-4, 7-9 and 12-14 above, and further in view of Jong Hu Jeong (US 11,869,212 B1, filed February 7, 2023).
Regarding claim 16, Kim in view of Sundaresan teaches the image processing device according to claim 2, as described above.
Although Kim in view of Sundaresan teaches determining an object detection target area based on a previous frame (Kim, Col. 7, lines 10-21), they do not explicitly teach “the one or more processors use independent information for each time point when using position information of objects detected in a plurality of input images generated at different times as input for prediction”. However, in an analogous field of endeavor, Jeong teaches performing object detection using a current frame image and the prior information, which is the temporal information related to the location information of the bounding boxes according to object detection information detected from the previous frame image (Jeong, Col. 17, lines 24-40).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the device of Kim in view of Sundaresan with the teachings of Jeong by including using independent information for each time point (i.e., prior information which is the temporal information of bounding boxes from the previous frame) when using position information of objects detecting in input images generated at different times as input for prediction (i.e., object detection). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for improving robustness of video object detection, as recognized by Jeong. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 17, Kim in view of Sundaresan teaches the image processing device according to claim 3, as described above.
Although Kim in view of Sundaresan teaches determining an object detection target area based on a previous frame (Kim, Col. 7, lines 10-21), they do not explicitly teach “the one or more processors use independent information for each time point when using position information of objects detected in a plurality of input images generated at different times as input for prediction”. However, in an analogous field of endeavor, Jeong teaches performing object detection using a current frame image and the prior information, which is the temporal information related to the location information of the bounding boxes according to object detection information detected from the previous frame image (Jeong, Col. 17, lines 24-40).
The proposed combination as well as the motivation for combining the Kim, Sundaresan and Jeong references presented in the rejection of Claim 16, apply to Claim 17 and are incorporated herein by reference. Thus, the device recited in Claim 17 is met by Kim in view of Sundaresan further in view of Jeong.
Regarding claim 18, Kim in view if Sundaresan teaches the image processing device according to claim 4, as described above.
Although Kim in view of Sundaresan teaches determining an object detection target area based on a previous frame (Kim, Col. 7, lines 10-21), they do not explicitly teach “the one or more processors use independent information for each time point when using position information of objects detected in a plurality of input images generated at different times as input for prediction”. However, in an analogous field of endeavor, Jeong teaches performing object detection using a current frame image and the prior information, which is the temporal information related to the location information of the bounding boxes according to object detection information detected from the previous frame image (Jeong, Col. 17, lines 24-40).
The proposed combination as well as the motivation for combining the Kim, Sundaresan and Jeong references presented in the rejection of Claim 16, apply to Claim 18 and are incorporated herein by reference. Thus, the device recited in Claim 18 is met by Kim in view of Sundaresan further in view of Jeong.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights