DETAILED ACTION
Response to Amendment
Applicant’s response to the last Office Action, filed on 6/9/2026 has been entered and made of record.
Applicant’s amendments necessitated the new ground of rejection set forth herein; therefore, this action is made Final.
Rejections under 35 USC 112(a) are added in view of amendments.
Response to Arguments
Applicant's arguments filed on 6/9/2026 have been fully considered but they are not persuasive.
The Kolouri reference (US Pat. No. 12,008,079) is added in view of amendments, see detailed analysis below. As such arguments directed to earlier prior art are moot.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 8-14, 16-17, and 19-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention:
Claim 8 recites, “information processing device according to claim 1, wherein the predetermined condition is satisfied when the first detection result indicates an object of a first type and the second detection result indicates an object of a second type that is different from the first type for a substantially overlapping region in the image.” There is no disclosure in the originally filed specification for this indicating an error based on object type like this.
Claim 9 recites, “information processing device according to claim 4, wherein the plurality of frames constitutes a moving image, and the predetermined condition is satisfied when the difference between the first detection result and the second detection result occurs continuously across a predetermined number of consecutive frames of the moving image.” There is no disclosure in the originally filed specification for satisfying the condition based on a result occurring continuously across a predetermined number of consecutive frames.
Claim 10 recites, “information processing device according to claim 1, wherein the notification includes erroneous detection notification information comprising a cropped image of the region where the object was detected by the second machine learning model and a robustness characteristic identifier associated with the second machine learning model.” There is no disclosure in the originally filed specification for notification information comprising a cropped image of the region where the object was detected by the second machine learning model and a robustness characteristic identifier. Claim 19 is rejected similarly.
Claim 11 recites, “information processing device according to claim 4, wherein the second machine learning model processes the second number of images at a constant frequency to perform a parallel audit of the first detection result, wherein the constant frequency is maintained regardless of a confidence score of the first detection result.” There is no disclosure in the originally filed specification for a constant frequency maintained regardless of a confidence score of the first detection result.
Claim 12 recites, “information processing device according to claim 1, wherein the first detection result identifies the object as a human-like region and the second detection result identifies the object as an insect, and the predetermined condition is satisfied when the second machine learning model detects the insect in a region where the first machine learning model erroneously detected the human-like region.” There is no disclosure in the originally filed specification for this. The only reference to insect versus human detections comes strictly from a brief description of a related art of a different system at ¶ 0002-0004, as files, not of the instant invention. Claim 17 is rejected similarly.
Claim 13 recites, “information processing device according to claim 1, wherein the notification includes erroneous detection notification information comprising a cropped image of the region where the object was detected by the second machine learning model and an identifier associated with the second machine learning model.” There is no disclosure in the originally filed specification for notification information comprising a cropped image of the region where the object was detected by the second machine learning model and a characteristic identifier.
Claim 14 recites, “information processing device according to claim 1, wherein the notification includes erroneous detection notification information comprising a cropped image of the region where the object was detected by the second machine learning model and an identifier associated with the second machine learning model.” There is no disclosure in the originally filed specification for notification information comprising a cropped image of the region where the object was detected by the second machine learning model and a characteristic identifier.
Claim 16 recites, “information processing device according to claim 5, wherein the robust second predetermined characteristic of the third machine learning model is different from the robust predetermined characteristic of the second machine learning model, such that each model is robust against different types of adversarial samples.” There is no disclosure in the originally filed specification for different models robust against different types of adversarial samples. Claim 20 is rejected similarly.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-4, 7-10, 12-15 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xiong (US PGPub 2022/0414382) in view of Kolouri (US Pat. No. 12,008,079).
Regarding claim 1, Xiong teaches an information processing device that detects an object in an image, the information processing device comprising: (Xiong teaches a method for video surveillance object detection in which an initial object detection algorithm model is checked by a more computationally intensive object verification detection model.)
one or more processors; and (¶ 0024)
a memory storing one or more instructions, wherein the one or more instructions, when executed by the one or more processors, cause the information processing device to: (¶ 0024)
output a first detection result indicating a region where an object has been detected in a region in an image by using a first machine learning model that detects the object in the image; (See ¶ 0008 and 0049 teaching the field object detector as well as the machine learning architectures for the object detection at ¶ 0061.)
output a second detection result indicating a region where an object has been detected in a region in the image by using a second machine learning model having a different predetermined characteristic related to detection of the object from that of the first machine learning model; and (See verification object detector at Figs. 5 and 6 as well as ¶ 0060 and 0093 which teach that the verification object detector is a different heavier weight algorithm that is more computationally intensive.)
make a notification of presence of erroneous detection regarding detection of an object by the first machine learning model when a difference between the first detection result and the second detection result satisfies a predetermined condition. (¶ 0093 teaches a comparator to test the match of the bounding boxes between the two object detections and determine a false detection result. Also see Fig. 5, numeral 526 and ¶ 0124 for notification of presence of erroneous detection.)
wherein the second machine learning model has a robust predetermined characteristic related to detection of an object compared to the first machine learning model. (As above, see verification object detector at Figs. 5 and 6 as well as ¶ 0060 and 0093 which teach that the verification object detector is a different more robust heavier weight algorithm that is more computationally intensive.)
In the field of object detection verification Kolouri teaches that the robust predetermined characteristic related to detection of the object is a characteristic with which an object is capable of being detected in an image including an attack by an adversarial sample. (Kolouri teaches a system in which object detection is performed using an initial model and object detection verification is performed using a secondary model which is robust to adversarial attacks, see Abstract. Col. 10, last paragraph through col. 12, ¶ 3, describes the system of the unsupervised part decomposition for object detection verification which is robust to an adversarial attack. A part-based model is built to verify the initial hypothesis of the first model by existence of object parts and their relations via a ‘foveated hypothesis verification’.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Xiong’s dual model object detection and verification strategy with Kolouri’s dual model object detection and verification strategy. Xiong teaches a simple ensemble detection strategy with a primary detector and a verification detector. Kolouri teaches a system in which object detection is performed using an initial model and object detection verification is performed using a secondary model which is robust to adversarial attacks. The combination constitutes the repeatable and predictable result of simply applying Kolouri’s teaching here for extending Xiong’s object detection verification to be robust to adversarial attacks. Simply applying this existing technique cannot be considered a non-obvious improvement in view of the relevant prior art here. Using known engineering design, no “fundamental” operating principle of the teachings are changed; they continue to perform the same functions as originally taught prior to being combined.
Regarding claim 3, the above combination teaches the information processing device according to claim 2, wherein the making a notification includes determining that the predetermined condition is satisfied in a case where an object in a region having a predetermined size or more is detected in the second detection result, in a region where no object is detected in the first detection result. (As above, see Xiong Fig. 5, numeral 526 and ¶ 0124 for notification of presence of erroneous detection due to a conflict between the field object detector and the verification object detector. That a test for predetermined size or more of an object is used is taught at ¶ 0086-0087.)
Regarding claim 4, the above combination teaches the information processing device according to claim 1, wherein the one or more instructions further cause the information processing device to acquire images of a plurality of consecutive frames, the first machine learning model outputs the first detection result to a first number of images per second among the images of the plurality of frames, and the second machine learning model outputs the second detection result to a second number of images per second that is smaller than the first number, among the images of the plurality of frames. (Xiong Fig. 5, numeral 510, a video stream of consecutive frames is input, all of which are processed by the field object detection. Fig. 6 and ¶ 130-0135 show selective verification sending frames segments only when accuracy is below a threshold confidence for example, leading to a lower fraction of the total video stream frames sent to verification (lower frames per second of the total stream frames.)
Regarding claim 7, the above combination teaches the information processing device according to claim 1, wherein the second machine learning model has a characteristic with which a specific type of an object is capable of being robustly detected compared to the first machine learning model. (As above, see verification object detector at Xiong Figs. 5 and 6 as well as ¶ 0060 and 0093 which teach that the verification object detector is a different more robust heavier weight algorithm that is more computationally intensive. See ¶ 0094 regarding specific face object types.)
Regarding claim 8, the above combination teaches the information processing device according to claim 1, wherein the predetermined condition is satisfied when the first detection result indicates an object of a first type and the second detection result indicates an object of a second type that is different from the first type for a substantially overlapping region in the image. (Xiong, ¶ 0094)
Regarding claim 9, the above combination teaches the information processing device according to claim 4, wherein the plurality of frames constitutes a moving image, and the predetermined condition is satisfied when the difference between the first detection result and the second detection result occurs continuously across a predetermined number of consecutive frames of the moving image. (Xiong, ¶ 0067)
Regarding claim 10, the above combination teaches the information processing device according to claim 1, wherein the notification includes erroneous detection notification information comprising a cropped image of the region where the object was detected by the second machine learning model and a robustness characteristic identifier associated with the second machine learning model. (Xiong, see Fig. 3 and ¶ 0093-0096 which teach outputting missed detections and associated data. Fig. 6 numeral 630 and Fig. 7 teach outputting the object image data for calibration and associated robustness identification conditions. ¶ 0140 teaches cropping object image data out of the video stream for this purpose.)
Regarding claim 12, the above combination teaches the information processing device according to claim 1, wherein the first detection result identifies the object as a human-like region and the second detection result identifies the object as an insect, and the predetermined condition is satisfied when the second machine learning model detects the insect in a region where the first machine learning model erroneously detected the human-like region. (Xiong, ¶ 0094)
Regarding claim 13, the above combination teaches the information processing device according to claim 1, wherein the notification includes erroneous detection notification information comprising a cropped image of the region where the object was detected by the second machine learning model and an identifier associated with the second machine learning model. (As above, see Xiong Fig. 3 and ¶ 0093-0096 which teach outputting missed detections and associated data. Fig. 6 numeral 630 and Fig. 7 teach outputting the object image data for calibration and associated robustness identification conditions. ¶ 0140 teaches cropping object image data out of the video stream for this purpose.)
Regarding claim 14, the above combination teaches the information processing device according to claim 4, wherein the plurality of frames constitutes a moving image, and the predetermined condition is satisfied when the difference between the first detection result and the second detection result occurs continuously across a predetermined number of consecutive frames of the moving image. (Xiong, ¶ 0067)
Regarding claim 15, the above combination teaches the information processing device according to claim 1, wherein the second detection result indicates at least a part of an object of interest in a region of the image where the first machine learning model failed to detect the object due to the adversarial sample. (Xiong, see Fig. 3 and ¶ 0093-0096 which teach outputting missed detections and associated data. Fig. 6 numeral 630 and Fig. 7 teach outputting the object image data for calibration and associated robustness identification conditions. ¶ 0140 teaches cropping object image data out of the video stream for this purpose. As above, Kolouri teaches a system in which object detection is performed using an initial model and object detection verification is performed using a secondary model which is robust to adversarial attacks, see Abstract. Col. 10, last paragraph through col. 12, ¶ 3.)
Regarding claim 17, the above combination teaches the information processing device according to claim 1, wherein the first detection result identifies a region in the image as a human-like region and the second detection result identifies the region as an insect, and the predetermined condition is satisfied when the second machine learning model determines that the region is the insect. (Xiong, ¶ 0094)
Regarding claim 18, the above combination teaches the information processing device according to claim 1, wherein the second detection result indicates at least a part of an object of interest in a region of the image where the first machine learning model failed to detect the object of interest due to the adversarial sample. (As above, Kolouri teaches a system in which object detection is performed using an initial model and object detection verification is performed using a secondary model which is robust to adversarial attacks, see Abstract. Col. 10, last paragraph through col. 12, ¶ 3.)
Regarding claim 19, the above combination teaches the information processing device according to claim 1, wherein the notification comprises erroneous detection notification information including a cropped image of the region where the object was detected by the second machine learning model and a robustness characteristic identifier associated with the second machine learning model. (As above, see Xiong Fig. 3 and ¶ 0093-0096 which teach outputting missed detections and associated data. Fig. 6 numeral 630 and Fig. 7 teach outputting the object image data for calibration and associated robustness identification conditions. ¶ 0140 teaches cropping object image data out of the video stream for this purpose.)
Claim(s) 5-6, 16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xiong (US PGPub 2022/0414382) in view of Kolouri (US Pat. No. 12,008,079) and Casado-Garcia (“Ensemble Methods for Object Detection”)
Regarding claim 5, the above combination teaches the information processing device according to claim 3, wherein the one or more instructions further cause the information processing device to output a detection result indicating a region in which an object is detected in a region in the image by using a machine learning model having a robust predetermined characteristic related to detection of the object compared to the first machine learning model, and the making a notification includes making a notification of presence of erroneous detection for detection of an object by the first machine learning model in a case where an object in a region of the predetermined size or more is detected in at least one of the second detection result. (See detailed analysis above.)
In the field of object detection Casado-Garcia teaches using a third detection result from a third machine learning model with a second predetermined characteristic and in a region where no object is detected in the first detection result. (Casado-Garcia teaches a technique for using multiple ensemble objectors in order to generate a more robust overall object detection. A few algorithmic strategies are disclosed involving three object detectors. Section 3.1 on pg. 2-3 teach the different strategies for multiple object detection models for ensemble detection such as the consensus/majority and the unanimous strategy. These algorithms meet the claim requirements. Table 1 teaches an example of applying 5 models and then takes top 3 and applying 3 best consensus/ unanimous models.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Xiong’s dual object detector strategy with Casado-Garcia’s triple object detector strategy. Xiong teaches a simple ensemble detection strategy with a primary detector and a verification detector. Casado-Garcia teaches a technique for using a triple ensemble object detector in order to generate a more robust overall object detection. Three object detectors are used. The combination constitutes the repeatable and predictable result of simply applying Casado-Garcia’s teaching here and cannot be considered a non-obvious improvement in view of the relevant prior art here. Using known engineering design, no “fundamental” operating principle of the teachings are changed; they continue to perform the same functions as originally taught prior to being combined.
Regarding claim 6, the above combination teaches the information processing device according to claim 5, wherein the making a notification includes making a notification of information regarding an object related to erroneous detection. (Xiong Fig. 5, numeral 526 and ¶ 0124 teaches notification of presence of erroneous detection.)
Regarding claim 16, the above combination teaches the information processing device according to claim 5, wherein the robust second predetermined characteristic of the third machine learning model is different from the robust predetermined characteristic of the second machine learning model, such that each model is robust against different types of adversarial samples. (Casado-Garcia teaches a technique for using multiple ensemble objectors in order to generate a more robust overall object detection. A few algorithmic strategies are disclosed involving three object detectors. Section 3.1 on pg. 2-3 teach the different strategies for multiple object detection models for ensemble detection such as the consensus/majority and the unanimous strategy. See combination with Kolouri in rejection of claim 1.)
Regarding claim 20, the above combination teaches the information processing device according to claim 5, wherein the robust second predetermined characteristic of the third machine learning model is different from the robust predetermined characteristic of the second machine learning model, such that the second and third machine learning models are robust against different types of adversarial samples. (See rejection of claim 16.)
Conclusion
Based on these facts, THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Raphael Schwartz whose telephone number is (571)270-3822. The examiner can normally be reached Monday to Friday 9am-5pm CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571) 272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAPHAEL SCHWARTZ/ Examiner, Art Unit 2671
Prosecution Insights