DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/23/2026 has been entered.
Response to Amendments
The amendments to claims 1, 9, 13, 17, and 23 have been accepted and entered.
Claims 4-5, 11, 16, and 20 are cancelled.
Claims 1-3, 6-10, 12-15, 17-19, and 21-25 are pending regarding this application.
Response to Arguments
Applicant’s arguments, see Remarks, filed 11/11/2025, with respect to the Claim Objections of claims 1, 13, and 17 have been fully considered and are persuasive. The Claim Objections of claims 1, 13, and 17 have been withdrawn.
Applicant’s arguments with respect to claim(s) 1, 13, and 17 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Please see the combination of Koushik et al. (U.S. Publication No. 2020/0019819 A1) in view of Cinnamon et al. (U.S. Publication No. 2020/0193666 A1) in the 103 rejection below regarding this matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 6, 9, 13, 14, 17-18, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Koushik et al. (U.S. Publication No. 2020/0019819 A1), hereinafter Koushik in view of Cinnamon et al. (U.S. Publication No. 2020/0193666 A1), hereinafter Cinnamon.
Regarding claim 1, Koushik teaches a computer-implemented method (Faster R-CNN algorithm to perform feature extraction of a computer component contained within image/video input; Figs 2, 3, 6 and ¶ [0036]-[0043]) comprising:
identifying one or more objects in image data associated with at least one device by processing at least a portion of the image data using at least one region-based convolutional neural network, wherein processing the at least a portion of the image data using at least one region-based convolutional neural network comprises (Koushik teaches an input image 204, 302 is used to identify an object of a computing device using one or more anchor boxes 206 for a region proposal network 210 via CNN of an object detection model 202, 304; Figs 2, 3 and ¶ [0038]-[0040]; see also para. [0036] wherein Koushik teaches “identification of server type and server components is achieved through implementation of a CNN, such as, for example, a faster region-based CNN (Faster R-CNN) algorithm”):
extracting one or more features from at least a portion of the one or more object- related region candidates (Koushik teaches “a Faster R-CNN algorithm is implemented to perform feature extraction over an image or video input, wherein such features can include component color, component scaling, component rotation/orientation, component illumination, component edge detail, etc” in para. [0037]; this process occurs in order to confirm specific component regions as further shown in para. [0037]. Koushik teaches that the CNN (304) (which can be a Faster R-CNN as shown above), produced convolutional feature maps to locate objects in the input image in para. [0040]);
classifying at least a portion of the one or more object-related region candidates as at least one predetermined device-related component category by processing at least a portion of the one or more features using at least one classifier model (Koushik teaches a classifier algorithm (220) that can identify one or more server components in the input image based on the proposal region in para. [0039]; the process of identifying and classifying the server components is equivalent to the claimed categorizing process. See FIG. 2 #220. See also para. [0046] and FIG. 5);
performing a comparison of at least a portion of the one or more identified objects to a predetermined set of one or more objects associated with the at least one device (Koushik teaches the output of the proposed region 212, 310 of the computer component (object) is input to a spatial pooling algorithm 216, 312, regression algorithm 218, 316 and classifier algorithm 220, 314 to perform an identification and output of the computer component 318, wherein “a variety of images of individual server components 502, [] are compared to a pre-trained model 504 of objects for purposes of identification” as shown in para. [0046] and FIG. 5; Figs 2, 3 and ¶ [0039], [0042]-[0043]);
determining internal state information attributed to the at least one device (Koushik teaches “dynamically identifying server components from video and/or image input (live and/or static video or image input) and displaying the identified components and related information (such as from a manual, for example) through an interface” in para. [0035]; the information gathered in the above citation(s) can be interpreted as internal state information as Koushik teaches identifying internal components of a server. See also that the embodiment as described in FIG. 6 may be utilized to “detect session-based access anomalies and undertake appropriate remediation actions” as shown in para. [0053]);
wherein the method is performed by at least one processing device comprising a processor coupled to a memory (Koushik teaches the object detection model 202, 130 is executed using a processor 120 from instructions stored in the memory 122; Figs 1-3 and ¶[0031], [0036]-[0043]).
Koushik does not teach determining one or more object-related region candidates in the image data using one or more bounding boxes; determining internal state information attributed to the at least one device based at least in part on results of the comparison, wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device; and performing one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information.
However, Cinnamon teaches determining one or more object-related region candidates in the image data using one or more bounding boxes (Cinnamon teaches “once the segmenter has generated scores for each position and anchor box, the classification engine may identify a set of candidate bounding boxes which have the highest position scores values. The segmenter may output the identified set of highest scoring candidate bounding boxes to a classifier, which may comprise a convolutional network, for further analysis to determine whether a given identified candidate bounding boxes may contain classifiable items” as shown in para. [0048]);
determining internal state information attributed to the at least one device based at least in part on results of the comparison (Cinnamon teaches determining an internal state of a device wherein a “classification engine may compare the material characteristic data for a given classified item or component to previously-determined material characteristic data for the item or component, which may be stored, e.g. in item database 106” as shown in para. [0204]), wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device (Cinnamon teaches “if classification engine 104 determines that the material data differs from the previously-determined material characteristic data by a threshold margin, classification engine 104 may determine that the given item or component has been modified or has a missing component or sub-item, and is therefore anomalous” in para. [0204]; here the modified or missing component is interpreted as equivalent to the claimed misplaced component(s) within the device); and
performing one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information (Cinnamon teaches that the “graphical output generator 108 may generate a graphical representation 500 based on the determined anomaly” in para. [0213] and FIG. 5, wherein “the graphical representation may cue security personnel to look for other components of the set that have not been identified, and/or may include one or more indications of missing components of the set” as shown in para. [0294] and “graphical representation 112 may also comprise representations of identified items, and may emphasize certain potentially-identified items or sub-items, and regions associated with threats for closer inspection by a security operator” in para. [0142]).
Koushik and Cinnamon are both considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of this application to combine the teachings of Koushik with Cinnamon and include “determining one or more object-related region candidates in the image data using one or more bounding boxes; determining internal state information attributed to the at least one device based at least in part on results of the comparison, wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device; and performing one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information”. The motivation for doing so would have been to “significantly improve the classification of items within a scanned object or scene” and “determine[] that the material data differs from the previously-determined material characteristic data by a threshold margin, [such that] classification engine 104 may determine that the given item or component has been modified or has a missing component or sub-item, and is therefore anomalous”, as suggested by Cinnamon in para. [0183] and para. [0204], respectively. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik with Cinnamon to obtain the invention specified in claim 1.
Regarding claim 2, Koushik and Cinnamon teach the computer-implemented method of claim 1,
wherein the image data associated with the at least one device comprises X-ray image data associated with the at least one device (Cinnamon teaches that “detection devices may comprise x-ray scanners” wherein “the images captured by detection devices 102 may represent the captured data using various representations” as shown in para. [0105]. See para. [0107] wherein the “detection devices 102 may capture one or more images for each object being scanned” as shown in para. [0107]. See also para. [0209], [0200], and [0038] wherein the object may be a device such as a laptop or other electronic). Similar motivations as applied to claim 1 can be applied here to claim 2.
Regarding claim 6, Koushik and Cinnamon teach the computer-implemented method of claim 1,
wherein extracting one or more features from at least a portion of the one or more object-related region candidates comprises extracting the one or more features from the at least a portion of the one or more object-related region candidates (see Koushik’s teaching of this in claim 1) using at least one deep convolutional neural network (Koushik teaches the usage of a Faster R-CNN in the process of feature extraction in para. [0036]-[0037]. Here, the Faster R-CNN is interpreted as equivalent to an example of a deep convolutional neural network).
Regarding claim 9, Koushik and Cinnamon teach the computer-implemented method of claim 1,
wherein determining internal state information further comprises identifying at least one of one or more missing components within the at least one device and one or more missing connections within the at least one device (Cinnamon teaches “classification engine 104 may attempt to identify anomalies associated with an item. Some examples of anomalies may comprise an item, sub-item, or component that has been modified, or an item that has a missing sub-item or component” in para. [0201]). Similar motivations as applied to claim 1 can be applied here to claim 9.
Regarding claim 13, Koushik teaches a non-transitory processor-readable storage medium (Koushik teaches “a computer-readable storage media” wherein “the term “article of manufacture” as used herein should be understood to exclude transitory, propagating signals” in para. [0027]) having stored therein program code of one or more software programs (Koushik teaches “program code” in para. [0026]; and a “storage device” in para. [0027]), wherein the program code when executed by at least one processing device causes the at least one processing device:
to identify one or more objects in image data associated with at least one device by processing at least a portion of the image data using at least one region-based convolutional neural network, wherein processing the at least a portion of the image data using at least one region-based convolutional neural network comprises (Koushik teaches an input image 204, 302 is used to identify an object of a computing device using one or more anchor boxes 206 for a region proposal network 210 via CNN of an object detection model 202, 304; Figs 2, 3 and ¶ [0038]-[0040]; see also para. [0036] wherein Koushik teaches “identification of server type and server components is achieved through implementation of a CNN, such as, for example, a faster region-based CNN (Faster R-CNN) algorithm”):
extracting one or more features from at least a portion of the one or more object- related region candidates (Koushik teaches “a Faster R-CNN algorithm is implemented to perform feature extraction over an image or video input, wherein such features can include component color, component scaling, component rotation/orientation, component illumination, component edge detail, etc” in para. [0037]; this process occurs in order to confirm specific component regions as further shown in para. [0037]. Koushik teaches that the CNN (304) (which can be a Faster R-CNN as shown above), produced convolutional feature maps to locate objects in the input image in para. [0040]);
classifying at least a portion of the one or more object-related region candidates as at least one predetermined device-related component category by processing at least a portion of the one or more features using at least one classifier model (Koushik teaches a classifier algorithm (220) that can identify one or more server components in the input image based on the proposal region in para. [0039]; the process of identifying and classifying the server components is equivalent to the claimed categorizing process. See FIG. 2 #220. See also para. [0046] and FIG. 5);
to perform a comparison of at least a portion of the one or more identified objects to a predetermined set of one or more objects associated with the at least one device (Koushik teaches the output of the proposed region 212, 310 of the computer component (object) is input to a spatial pooling algorithm 216, 312, regression algorithm 218, 316 and classifier algorithm 220, 314 to perform an identification and output of the computer component 318, wherein “a variety of images of individual server components 502, [] are compared to a pre-trained model 504 of objects for purposes of identification” as shown in para. [0046] and FIG. 5; Figs 2, 3 and ¶ [0039], [0042]-[0043]);
to determine internal state information attributed to the at least one device (Koushik teaches “dynamically identifying server components from video and/or image input (live and/or static video or image input) and displaying the identified components and related information (such as from a manual, for example) through an interface” in para. [0035]; the information gathered in the above citation(s) can be interpreted as internal state information as Koushik teaches identifying internal components of a server. See also that the embodiment as described in FIG. 6 may be utilized to “detect session-based access anomalies and undertake appropriate remediation actions” as shown in para. [0053]).
Koushik does not teach determining one or more object-related region candidates in the image data using one or more bounding boxes; to determine internal state information attributed to the at least one device based at least in part on results of the comparison, wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device; and to perform one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information.
However, Cinnamon teaches determining one or more object-related region candidates in the image data using one or more bounding boxes (Cinnamon teaches “once the segmenter has generated scores for each position and anchor box, the classification engine may identify a set of candidate bounding boxes which have the highest position scores values. The segmenter may output the identified set of highest scoring candidate bounding boxes to a classifier, which may comprise a convolutional network, for further analysis to determine whether a given identified candidate bounding boxes may contain classifiable items” as shown in para. [0048]);
to determine internal state information attributed to the at least one device based at least in part on results of the comparison (Cinnamon teaches determining an internal state of a device wherein a “classification engine may compare the material characteristic data for a given classified item or component to previously-determined material characteristic data for the item or component, which may be stored, e.g. in item database 106” as shown in para. [0204]), wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device (Cinnamon teaches “if classification engine 104 determines that the material data differs from the previously-determined material characteristic data by a threshold margin, classification engine 104 may determine that the given item or component has been modified or has a missing component or sub-item, and is therefore anomalous” in para. [0204]; here the modified or missing component is interpreted as equivalent to the claimed misplaced component(s) within the device); and
to perform one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information (Cinnamon teaches that the “graphical output generator 108 may generate a graphical representation 500 based on the determined anomaly” in para. [0213] and FIG. 5, wherein “the graphical representation may cue security personnel to look for other components of the set that have not been identified, and/or may include one or more indications of missing components of the set” as shown in para. [0294] and “graphical representation 112 may also comprise representations of identified items, and may emphasize certain potentially-identified items or sub-items, and regions associated with threats for closer inspection by a security operator” in para. [0142]).
Koushik and Cinnamon are both considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of this application to combine the teachings of Koushik with Cinnamon and include “determining one or more object-related region candidates in the image data using one or more bounding boxes; determining internal state information attributed to the at least one device based at least in part on results of the comparison, wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device; and performing one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information”. The motivation for doing so would have been to “significantly improve the classification of items within a scanned object or scene” and “determine[] that the material data differs from the previously-determined material characteristic data by a threshold margin, [such that] classification engine 104 may determine that the given item or component has been modified or has a missing component or sub-item, and is therefore anomalous”, as suggested by Cinnamon in para. [0183] and para. [0204], respectively. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik with Cinnamon to obtain the invention specified in claim 13.
Regarding claim 14, Koushik and Cinnamon teach the non-transitory processor-readable storage medium of claim 13,
wherein the image data associated with the at least one device comprises X-ray image data associated with the at least one device (Cinnamon teaches that “detection devices may comprise x-ray scanners” wherein “the images captured by detection devices 102 may represent the captured data using various representations” as shown in para. [0105]. See para. [0107] wherein the “detection devices 102 may capture one or more images for each object being scanned” as shown in para. [0107]. See also para. [0209], [0200], and [0038] wherein the object may be a device such as a laptop or other electronic). Similar motivations as applied to claim 13 can be applied here to claim 14.
Regarding claim 17, Koushik teaches an apparatus comprising:
at least one processing device comprising a processor coupled to a memory (Koushik teaches a processor and memory in para. [0026] and [0027]);
the at least one processing device being configured:
to identify one or more objects in image data associated with at least one device by processing at least a portion of the image data using at least one region-based convolutional neural network, wherein processing the at least a portion of the image data using at least one region-based convolutional neural network comprises (Koushik teaches an input image 204, 302 is used to identify an object of a computing device using one or more anchor boxes 206 for a region proposal network 210 via CNN of an object detection model 202, 304; Figs 2, 3 and ¶ [0038]-[0040]; see also para. [0036] wherein Koushik teaches “identification of server type and server components is achieved through implementation of a CNN, such as, for example, a faster region-based CNN (Faster R-CNN) algorithm”):
extracting one or more features from at least a portion of the one or more object- related region candidates (Koushik teaches “a Faster R-CNN algorithm is implemented to perform feature extraction over an image or video input, wherein such features can include component color, component scaling, component rotation/orientation, component illumination, component edge detail, etc” in para. [0037]; this process occurs in order to confirm specific component regions as further shown in para. [0037]. Koushik teaches that the CNN (304) (which can be a Faster R-CNN as shown above), produced convolutional feature maps to locate objects in the input image in para. [0040]);
classifying at least a portion of the one or more object-related region candidates as at least one predetermined device-related component category by processing at least a portion of the one or more features using at least one classifier model (Koushik teaches a classifier algorithm (220) that can identify one or more server components in the input image based on the proposal region in para. [0039]; the process of identifying and classifying the server components is equivalent to the claimed categorizing process. See FIG. 2 #220. See also para. [0046] and FIG. 5);
to perform a comparison of at least a portion of the one or more identified objects to a predetermined set of one or more objects associated with the at least one device (Koushik teaches the output of the proposed region 212, 310 of the computer component (object) is input to a spatial pooling algorithm 216, 312, regression algorithm 218, 316 and classifier algorithm 220, 314 to perform an identification and output of the computer component 318, wherein “a variety of images of individual server components 502, [] are compared to a pre-trained model 504 of objects for purposes of identification” as shown in para. [0046] and FIG. 5; Figs 2, 3 and ¶ [0039], [0042]-[0043]);
to determine internal state information attributed to the at least one device (Koushik teaches “dynamically identifying server components from video and/or image input (live and/or static video or image input) and displaying the identified components and related information (such as from a manual, for example) through an interface” in para. [0035]; the information gathered in the above citation(s) can be interpreted as internal state information as Koushik teaches identifying internal components of a server. See also that the embodiment as described in FIG. 6 may be utilized to “detect session-based access anomalies and undertake appropriate remediation actions” as shown in para. [0053]).
Koushik does not teach determining one or more object-related region candidates in the image data using one or more bounding boxes; to determine internal state information attributed to the at least one device based at least in part on results of the comparison, wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device; and to perform one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information.
However, Cinnamon teaches determining one or more object-related region candidates in the image data using one or more bounding boxes (Cinnamon teaches “once the segmenter has generated scores for each position and anchor box, the classification engine may identify a set of candidate bounding boxes which have the highest position scores values. The segmenter may output the identified set of highest scoring candidate bounding boxes to a classifier, which may comprise a convolutional network, for further analysis to determine whether a given identified candidate bounding boxes may contain classifiable items” as shown in para. [0048]);
to determine internal state information attributed to the at least one device based at least in part on results of the comparison (Cinnamon teaches determining an internal state of a device wherein a “classification engine may compare the material characteristic data for a given classified item or component to previously-determined material characteristic data for the item or component, which may be stored, e.g. in item database 106” as shown in para. [0204]), wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device (Cinnamon teaches “if classification engine 104 determines that the material data differs from the previously-determined material characteristic data by a threshold margin, classification engine 104 may determine that the given item or component has been modified or has a missing component or sub-item, and is therefore anomalous” in para. [0204]; here the modified or missing component is interpreted as equivalent to the claimed misplaced component(s) within the device); and
to perform one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information (Cinnamon teaches that the “graphical output generator 108 may generate a graphical representation 500 based on the determined anomaly” in para. [0213] and FIG. 5, wherein “the graphical representation may cue security personnel to look for other components of the set that have not been identified, and/or may include one or more indications of missing components of the set” as shown in para. [0294] and “graphical representation 112 may also comprise representations of identified items, and may emphasize certain potentially-identified items or sub-items, and regions associated with threats for closer inspection by a security operator” in para. [0142]).
Koushik and Cinnamon are both considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of this application to combine the teachings of Koushik with Cinnamon and include “determining one or more object-related region candidates in the image data using one or more bounding boxes; to determine internal state information attributed to the at least one device based at least in part on results of the comparison, wherein determining internal state information comprises identifying at least one of one or more misplaced components within the at least one device, and one or more misplaced connections within the at least one device; and to perform one or more automated actions based at least in part on the internal state information, wherein performing one or more automated actions comprises generating at least one recommendation for one or more responsive actions to be taken in connection with the internal state information”. The motivation for doing so would have been to “significantly improve the classification of items within a scanned object or scene” and “determine[] that the material data differs from the previously-determined material characteristic data by a threshold margin, [such that] classification engine 104 may determine that the given item or component has been modified or has a missing component or sub-item, and is therefore anomalous”, as suggested by Cinnamon in para. [0183] and para. [0204], respectively. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik with Cinnamon to obtain the invention specified in claim 17.
Regarding claim 18, Koushik and Cinnamon teach the apparatus of claim 17,
wherein the image data associated with the at least one device comprises X-ray image data associated with the at least one device (Cinnamon teaches that “detection devices may comprise x-ray scanners” wherein “the images captured by detection devices 102 may represent the captured data using various representations” as shown in para. [0105]. See para. [0107] wherein the “detection devices 102 may capture one or more images for each object being scanned” as shown in para. [0107]. See also para. [0209], [0200], and [0038] wherein the object may be a device such as a laptop or other electronic). Similar motivations as applied to claim 17 can be applied here to claim 18.
Regarding claim 23, Koushik and Cinnamon teach the apparatus of claim 17,
wherein determining internal state information further comprises identifying at least one of one or more missing components within the at least one device and one or more missing connections within the at least one device (Cinnamon teaches “classification engine 104 may attempt to identify anomalies associated with an item. Some examples of anomalies may comprise an item, sub-item, or component that has been modified, or an item that has a missing sub-item or component” in para. [0201]). Similar motivations as applied to claim 1 can be applied here to claim 9.
Claims 3, 12, 15, 19, and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Koushik et al. (U.S. Publication No. 2020/0019819 A1), hereinafter Koushik in view of Cinnamon et al. (U.S. Publication No. 2020/0193666 A1), hereinafter Cinnamon and Wang et al. (U.S. Publication No. 2020/0118365 A1), hereinafter Wang.
Regarding claim 3, Koushik and Cinnamon teach the computer-implemented method of claim 1.
Koushik and Cinnamon fail to teach wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device.
However, Wang teaches wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device (Wang teaches that “Although vehicle models are different, manufacturers usually retain configuration information of each factory vehicle. Therefore, in the one or more embodiments of this specification, automotive part list information of a vehicle for damage assessment can be obtained by using a unique vehicle identification code of the vehicle. Then the automotive part list information is combined with an image recognition algorithm, so that the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced” para. [0026]; here, the configuration information is interpreted as the set of accessories present before the vehicle was shipped and/or sold by the manufacturer).
Koushik, Cinnamon, and Wang are all considered to be analogous to the claimed invention because they are in the same field of analyzing images of devices (in the case of Wang, the car and its parts is the device). Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Wang and include “wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device”. The motivation for doing so would have been that “the automotive part list information is combined with an image recognition algorithm, so that the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced”, as suggested by Wang in para. [0026]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik and Cinnamon with Wang to obtain the invention specified in claim 3.
Regarding claim 12, Koushik and Cinnamon teach the computer-implemented method of claim 1.
Koushik and Cinnamon fail to teach wherein performing one or more automated actions comprises automatically training at least a portion of the one or more artificial intelligence techniques based at least in part on feedback related to the internal state information.
However, Wang teaches wherein performing one or more automated actions comprises automatically training at least a portion of the one or more artificial intelligence techniques based at least in part on feedback related to the internal state information (Wang teaches “after early-stage sample training, the damaged automotive part recognition model may recognize a damage location and a damage type of an automotive part in the image of the part” para. [0032]; here, the damaged automotive part recognition model is being trained on the damage type of an automotive part wherein the damage type is interpreted as equivalent to the claimed feedback related to internal state information).
Koushik, Cinnamon, and Wang are all considered to be analogous to the claimed invention because they are in the same field of analyzing images of devices (in the case of Wang, the car and its parts is the device). Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Wang and include “wherein performing one or more automated actions comprises automatically training at least a portion of the one or more artificial intelligence techniques based at least in part on feedback related to the internal state information”. The motivation for doing so would have been so that “the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced”, as suggested by Wang in para. [0026]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik and Cinnamon with Wang to obtain the invention specified in claim 12.
Regarding claim 15, Koushik and Cinnamon teach the non-transitory processor-readable storage medium of claim 13.
Koushik and Cinnamon fail to teach wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device.
However, Wang teaches wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device (Wang teaches that “Although vehicle models are different, manufacturers usually retain configuration information of each factory vehicle. Therefore, in the one or more embodiments of this specification, automotive part list information of a vehicle for damage assessment can be obtained by using a unique vehicle identification code of the vehicle. Then the automotive part list information is combined with an image recognition algorithm, so that the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced” para. [0026]; here, the configuration information is interpreted as the set of accessories present before the vehicle was shipped and/or sold by the manufacturer).
Koushik, Cinnamon, and Wang are all considered to be analogous to the claimed invention because they are in the same field of analyzing images of devices (in the case of Wang, the car and its parts is the device). Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Wang and include “wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device”. The motivation for doing so would have been that “the automotive part list information is combined with an image recognition algorithm, so that the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced”, as suggested by Wang in para. [0026]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik and Cinnamon with Wang to obtain the invention specified in claim 15.
Regarding claim 19, Koushik and Cinnamon teach the apparatus of claim 17.
Koushik and Cinnamon fail to teach wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device.
However, Wang teaches wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device (Wang teaches that “Although vehicle models are different, manufacturers usually retain configuration information of each factory vehicle. Therefore, in the one or more embodiments of this specification, automotive part list information of a vehicle for damage assessment can be obtained by using a unique vehicle identification code of the vehicle. Then the automotive part list information is combined with an image recognition algorithm, so that the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced” para. [0026]; here, the configuration information is interpreted as the set of accessories present before the vehicle was shipped and/or sold by the manufacturer).
Koushik, Cinnamon, and Wang are all considered to be analogous to the claimed invention because they are in the same field of analyzing images of devices (in the case of Wang, the car and its parts is the device). Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Wang and include “wherein performing the comparison comprises comparing the at least a portion of the one or more identified objects to a predetermined set of one or more objects present in a pre-shipped version of the at least one device”. The motivation for doing so would have been that “the automotive part list information is combined with an image recognition algorithm, so that the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced”, as suggested by Wang in para. [0026]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik and Cinnamon with Wang to obtain the invention specified in claim 19.
Regarding claim 25, Koushik and Cinnamon teach the apparatus of claim 17.
Koushik and Cinnamon fail to teach wherein performing one or more automated actions comprises automatically training at least a portion of the one or more artificial intelligence techniques based at least in part on feedback related to the internal state information.
However, Wang teaches wherein performing one or more automated actions comprises automatically training at least a portion of the one or more artificial intelligence techniques based at least in part on feedback related to the internal state information (Wang teaches “after early-stage sample training, the damaged automotive part recognition model may recognize a damage location and a damage type of an automotive part in the image of the part” para. [0032]; here, the damaged automotive part recognition model is being trained on the damage type of an automotive part wherein the damage type is interpreted as equivalent to the claimed feedback related to internal state information).
Koushik, Cinnamon, and Wang are all considered to be analogous to the claimed invention because they are in the same field of analyzing images of devices (in the case of Wang, the car and its parts is the device). Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Wang and include “wherein performing one or more automated actions comprises automatically training at least a portion of the one or more artificial intelligence techniques based at least in part on feedback related to the internal state information”. The motivation for doing so would have been so that “the accuracy of a damaged automotive part recognition result using a damage assessment image algorithm can be significantly improved, and additional learning costs and a learning period of an image recognition algorithm/module can be greatly reduced”, as suggested by Wang in para. [0026]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Koushik and Cinnamon with Wang to obtain the invention specified in claim 25.
Claims 7 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Koushik et al. (U.S. Publication No. 2020/0019819 A1), hereinafter Koushik in view of Cinnamon et al. (U.S. Publication No. 2020/0193666 A1), hereinafter Cinnamon and Huang et al. (CN 104217225 B, see English translation for citations), hereinafter Huang.
Regarding claim 7, Koushik and Cinnamon teach the computer-implemented method of claim 1.
Koushik further teaches processing at least a portion of the one or more features using at least one classifier model (Koushik teaches “a classifier algorithm 220, resulting in an identification of one or more server components (from the input image 204)” para. [0039]).
Koushik and Cinnamon fail to teach wherein processing at least a portion of the one or more features using at least one classifier model comprises processing the at least a portion of the one or more features using at least one linear support vector machine classifier model.
However, Huang teaches wherein processing the at least a portion of the one or more features using at least one linear support vector machine classifier model (Huang “training an object appearance model by using a multi-example linear support vector machine based on the feature description of the candidate region” para. [0025])
Koushik, Cinnamon, and Huang are all considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Huang and include “processing the at least a portion of the one or more features using at least one linear support vector machine classifier model”. The motivation for doing so would have been that “the invention provides a multi-example linear support vector machine optimization algorithm based on a credible domain Newton method, which is more efficient than L-BFGS”, as suggested by Huang in para. [0087]. Therefore, it would have been obvious to combine Koushik and Cinnamon with Huang to obtain the invention specified in claim 7.
Regarding claim 21, Koushik and Cinnamon teach the apparatus of claim 17.
Koushik further teaches processing at least a portion of the one or more features using at least one classifier model (Koushik teaches “a classifier algorithm 220, resulting in an identification of one or more server components (from the input image 204)” para. [0039]).
Koushik and Cinnamon fail to teach wherein processing at least a portion of the one or more features using at least one classifier model comprises processing the at least a portion of the one or more features using at least one linear support vector machine classifier model.
However, Huang teaches wherein processing the at least a portion of the one or more features using at least one linear support vector machine classifier model (Huang “training an object appearance model by using a multi-example linear support vector machine based on the feature description of the candidate region” para. [0025])
Koushik, Cinnamon, and Huang are all considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Huang and include “processing the at least a portion of the one or more features using at least one linear support vector machine classifier model”. The motivation for doing so would have been that “the invention provides a multi-example linear support vector machine optimization algorithm based on a credible domain Newton method, which is more efficient than L-BFGS”, as suggested by Huang in para. [0087]. Therefore, it would have been obvious to combine Koushik and Cinnamon with Huang to obtain the invention specified in claim 21.
Claims 8 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Koushik et al. (U.S. Publication No. 2020/0019819 A1), hereinafter Koushik in view of Cinnamon et al. (U.S. Publication No. 2020/0193666 A1), hereinafter Cinnamon and Dion et al. (U.S. Publication No. 2021/0110440 A1), hereinafter Dion.
Regarding claim 8, Koushik and Cinnamon teach the computer-implemented method of claim 1.
While Koushik teaches wherein identifying one or more objects in image data associated with at least one device comprises identifying one or more objects in image data associated with at least one device in connection with one or more device-related issues (see claim 1), Koushik and Cinnamon fail to teach wherein the device is returned by at least one user.
However, Dion teaches wherein the device is returned by at least one user (Dion “users who are purchasing a new smartphone may be interested in trading in their current smartphone in exchange for value, to obtain a protection plan for their new smartphone or the current smartphone, to obtain repair recommendations for their current smartphone, etc.” para. [0040]).
Koushik, Cinnamon, and Baier are all considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Dion and include “wherein the device is returned by at least one user”. The motivation for doing so would have been “to determine which parts need to be replaced and apply adequate pricing rules in order to estimate the cost required to repair the POD with improved accuracy compared to estimation many techniques”, as suggested by Dion in para. [0214]. Therefore, it would have been obvious to combine Koushik and Cinnamon with Dion to obtain the invention specified in claim 8.
Regarding claim 22, Koushik and Cinnamon teach the apparatus of claim 17.
While Koushik teaches wherein identifying one or more objects in image data associated with at least one device comprises identifying one or more objects in image data associated with at least one device in connection with one or more device-related issues (see claim 1), Koushik and Cinnamon fail to teach wherein the device is returned by at least one user.
However, Dion teaches wherein the device is returned by at least one user (Dion “users who are purchasing a new smartphone may be interested in trading in their current smartphone in exchange for value, to obtain a protection plan for their new smartphone or the current smartphone, to obtain repair recommendations for their current smartphone, etc.” para. [0040]).
Koushik, Cinnamon, and Dion are all considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Dion and include “wherein the device is returned by at least one user”. The motivation for doing so would have been “to determine which parts need to be replaced and apply adequate pricing rules in order to estimate the cost required to repair the POD with improved accuracy compared to estimation many techniques”, as suggested by Dion in para. [0214]. Therefore, it would have been obvious to combine Koushik and Cinnamon with Dion to obtain the invention specified in claim 22.
Claims 10 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Koushik et al. (U.S. Publication No. 2020/0019819 A1), hereinafter Koushik in view of Cinnamon et al. (U.S. Publication No. 2020/0193666 A1), hereinafter Cinnamon and Tan et al. (U.S. Publication No. 2009/0254209 A1), hereinafter Tan.
Regarding claim 10, Koushik and Cinnamon fail to teach the computer-implemented method of claim 1.
Koushik and Cinnamon fail to teach wherein performing one or more automated actions comprises determining at least one root cause of at least one issue associated with the at least one device based at least in part on the internal state information.
However, Tan teaches wherein performing one or more automated actions comprises determining at least one root cause of at least one issue associated with the at least one device based at least in part on the internal state information (Tan “The defect inspection device moves along the shortest path, and takes a picture for each defect. These defect images can be used for the analyses of the causes of the defects by the officials in the Quality Assurance Department to improve the production processes” para. [0049]… “If the defect is caused by a stain or a short circuit, the laser repairing technique can be used. If the defect is caused by the broken circuit, the laser CVD technique can be used” para. [0050]; here, the defect (internal state information) is identified and the cause of the defect is analyzed in order to perform an action based on the causation of the defect).
Koushik, Cinnamon, and Tan are all considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Tan and include “wherein performing one or more automated actions comprises determining at least one root cause of at least one issue associated with the at least one device based at least in part on the internal state information”. The motivation for doing so would have been to automate different repair actions until all the defects are repaired, as suggested by Tan in para. [0050] and to improve production processes, as suggested by Tan in para. [0049]. Therefore, it would have been obvious to combine Koushik and Cinnamon with Tan to obtain the invention specified in claim 10.
Regarding claim 24, Koushik and Cinnamon teach the apparatus of claim 17.
Koushik and Cinnamon fail to teach wherein performing one or more automated actions comprises determining at least one root cause of at least one issue associated with the at least one device based at least in part on the internal state information.
However, Tan teaches wherein performing one or more automated actions comprises determining at least one root cause of at least one issue associated with the at least one device based at least in part on the internal state information (Tan “The defect inspection device moves along the shortest path, and takes a picture for each defect. These defect images can be used for the analyses of the causes of the defects by the officials in the Quality Assurance Department to improve the production processes” para. [0049]… “If the defect is caused by a stain or a short circuit, the laser repairing technique can be used. If the defect is caused by the broken circuit, the laser CVD technique can be used” para. [0050]; here, the defect (internal state information) is identified and the cause of the defect is analyzed in order to perform an action based on the causation of the defect).
Koushik, Cinnamon, and Tan are all considered to be analogous to the claimed invention because they are in the same field of identifying objects through image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Koushik (as modified by Cinnamon) to incorporate the teachings of Tan and include “wherein performing one or more automated actions comprises determining at least one root cause of at least one issue associated with the at least one device based at least in part on the internal state information”. The motivation for doing so would have been to automate different repair actions until all the defects are repaired, as suggested by Tan in para. [0050] and to improve production processes, as suggested by Tan in para. [0049]. Therefore, it would have been obvious to combine Koushik and Cinnamon with Tan to obtain the invention specified in claim 24.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can
normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/ Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights