DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/22/2026 has been considered by the examiner and placed in applicant file.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20220392210 A1), hereinafter referenced as KIM in view of LEE et al. (US 20210326638 A1), hereinafter referenced as LEE.
Regarding claim 1, KIM explicitly teaches a method (Fig. 1 and 4 Paragraph [0054]-KIM discloses FIG. 1 is a diagram for explaining an electronic device 100 assessing a video quality score, and outputting an image having a processed quality on a screen. In paragraph [0131]-KIM discloses FIG. 4 is a block diagram of an internal structure of the processor 210 of FIG. 2. In paragraph [0132]-KIM discloses referring to FIG. 4, the processor 210 may further include a high-complexity feature information obtainment unit 219 in addition to the subjective assessment score obtainment unit 211, the location weight obtainment unit 213, the weighted assessment score obtainment unit 215, and the final quality score obtainment unit 217) comprising:
receiving one or more recognition results over multiple image frames for an object over multiple image frames (Fig. 1, #120 called input video. Paragraph [0115]-KIM discloses the electronic device 100a may receive the video and may divide each frame of the received video into the plurality of sub-regions. In paragraph [0135]-KIM discloses the high-complexity feature information obtainment unit 219 may obtain the high-complexity feature information from an input frame (wherein high-complexity information may include object classifications/detections, semantic segmentations, saliency mapping and weighting assignments). Further in paragraph [0150]-KIM discloses the first neural network 511 may be a model trained to extract a quality assessment score of an image from data input by analyzing and classifying the input data. Please also read paragraph [0165-0167]);
generating a final recognition result for the object based on the one or more weights (Fig. 1. Paragraph [0165]-KIM discloses the final quality score obtainment unit 517 may receive the weighted assessment score matrix from the second neural network 512. The final quality score obtainment unit 513 may obtain a final quality score for the entire frame by averaging the weighted assessment scores included in the weighted assessment score matrix. Please also read paragraph [0150, 0163 and 0166]); and
transmitting the final recognition result (Fig. 1. Paragraph [0164]-KIM discloses the second neural network 512 may transmit a weighted assessment score matrix including the weighted assessment score of each of the plurality of sub-regions to the final quality score obtainment unit 513. In paragraph [0166]-KIM discloses the electronic device 100a may further include a high-complexity feature information obtainment unit. The final quality score obtainment unit 513 may receive a high-complexity weight indicating high-complexity feature information from the high-complexity feature information obtainment unit, and may apply the high-complexity weight to the weighted assessment score for each sub-region. The final quality score obtainment unit 513 may obtain a final quality score for the entire frame, based on the weighted assessment score for each sub-region to which the high-complexity weight has been applied).
Although KIM explicitly teaches assigning one or more weights to the one or more recognition results (Fig. 4. Paragraph [0087]-KIM discloses pieces of information included in the high-complexity information may be assigned different importances with different weights, respectively. In paragraph [0163]-KIM discloses the second neural network 512 may apply the location weight to the subjective assessment score for each of the plurality of sub-regions received from the first neural network 511. The second neural network 512 may obtain the weighted assessment score for each sub-region by multiplying the subjective assessment score for each sub-region by the location weight for each sub-region. Please also read paragraph [0140-0141 and 0150-151]) based on:
a stability of the one or more recognition results over the multiple image frames (Fig. 1. Paragraph [0167]-KIM discloses the final quality score obtainment unit 513 may obtain a final quality score for the entire video by using the final quality score for each frame. The final quality score obtainment unit 513 may consider a temporal influence or temporal dependence related to video recognition, by using the quality scores of frames accumulated over time. The final quality score obtainment unit 513 may obtain a final quality score for the entire video by smoothing time-series data. The final quality score obtainment unit 513 may use a simple heuristic rule or a neural network model to smooth the time-series data. The final quality score obtainment unit 513 may obtain a final quality score for the entire video in consideration of an effect over time with respect to accumulated time-series data. Please also read paragraph [0174-0177]);
KIM fails to explicitly teach relevance scoring of one or more neighboring objects.
However, LEE explicitly teaches relevance scoring of one or more neighboring objects (Fig. 5. Paragraph [0071]-LEE discloses FIG. 5 shows an apparatus for panoptic video segmentation. The example shown includes encoder 500, fusion component 505, track head 515, semantic head 520, bounding box head 525, mask head 530, and segmentation component 535. In paragraph [0058]-LEE discloses FIG. 4 shows a span of video frames 410. Multiple frame span 405 may include one or more frames 410. Different span lengths may be used to determine the quality of panoptic segmentation information (e.g., whether objects are correctly classified, and whether instances are consistent between frames). In paragraph [0038]-LEE discloses a Spatial Attention Network (SAN) is a variant of a CNN designed to exploit the spatial context of images. A SAN utilizes attention weights for clustered regional features. The attention weights indicate the value of the contribution of different regions to the overall classification. SAN uses a weighted sum of regional features as discriminative features. Thus, an SAN draws attention to important contents by giving them a higher attention weight).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM of having a method comprising: receiving one or more recognition results for an object over multiple image frames; assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames, generating a final recognition result for the object based on the one or more weights; and transmitting the final recognition result, with the teachings of LEE of having and relevance scoring of one or more neighboring objects
Wherein having KIM’s method having assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames and relevance scoring of one or more neighboring objects;
The motivation behind the modification would have been to obtain a method that improves the quality and quality assessment of both images and object recognition/tracking, since both KIM and LEE concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LEE provides systems and methods that improve scene labeling and per-frame panoptic quality (PQ) by properly utilizing spatial-temporal features. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LEE et al. (US 20210326638 A1), Abstract and paragraph [0051 and 0049].
Regarding claim 8, KIM in view of LIU explicitly teaches a device (Fig. 1, #100 called an electronic device. Paragraph [0054]-KIM discloses FIG. 1 is a diagram for explaining an electronic device 100 assessing a video quality score, and outputting an image having a processed quality on a screen. In paragraph [0132]-KIM discloses referring to FIG. 4, the processor 210 may further include a high-complexity feature information obtainment unit 219 in addition to the subjective assessment score obtainment unit 211, the location weight obtainment unit 213, the weighted assessment score obtainment unit 215, and the final quality score obtainment unit 217) comprising:
a processor (Fig. 1, #210 called a processor. Paragraph [0132]); and
a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations (Fig. 9. Paragraph [0214]-KIM discloses referring to FIG. 9, the electronic device 100b may include the processor 210, the memory 220, and an image quality processing unit 920. In paragraph [0267]-KIM discloses video quality assessment methods and apparatuses can be embodied as a storage medium including instruction codes executable by a computer such as a program module executed by the computer) to:
receive one or more recognition results for an object over multiple image frames (Fig. 1, #120 called input video. Paragraph [0115]-KIM discloses the electronic device 100a may receive the video and may divide each frame of the received video into the plurality of sub-regions. In paragraph [0135]-KIM discloses the high-complexity feature information obtainment unit 219 may obtain the high-complexity feature information from an input frame (wherein high-complexity information may include object classifications/detections, semantic segmentations, saliency mapping and weighting assignments). Further in paragraph [0150]-KIM discloses the first neural network 511 may be a model trained to extract a quality assessment score of an image from data input by analyzing and classifying the input data. Please also read paragraph [0165-0167]);
generate a final recognition result for the object based on the one or more weights (Fig. 1. Paragraph [0165]-KIM discloses the final quality score obtainment unit 517 may receive the weighted assessment score matrix from the second neural network 512. The final quality score obtainment unit 513 may obtain a final quality score for the entire frame by averaging the weighted assessment scores included in the weighted assessment score matrix. Please also read paragraph [0150, 0163 and 0166]); and
transmit the final recognition result (Fig. 1. Paragraph [0164]-KIM discloses the second neural network 512 may transmit a weighted assessment score matrix including the weighted assessment score of each of the plurality of sub-regions to the final quality score obtainment unit 513. In paragraph [0166]-KIM discloses the electronic device 100a may further include a high-complexity feature information obtainment unit. The final quality score obtainment unit 513 may receive a high-complexity weight indicating high-complexity feature information from the high-complexity feature information obtainment unit, and may apply the high-complexity weight to the weighted assessment score for each sub-region. The final quality score obtainment unit 513 may obtain a final quality score for the entire frame, based on the weighted assessment score for each sub-region to which the high-complexity weight has been applied).
Although KIM explicitly teaches assign one or more weights to the one or more recognition results (Fig. 4. Paragraph [0087]-KIM discloses pieces of information included in the high-complexity information may be assigned different importances with different weights, respectively. In paragraph [0163]-KIM discloses the second neural network 512 may apply the location weight to the subjective assessment score for each of the plurality of sub-regions received from the first neural network 511. The second neural network 512 may obtain the weighted assessment score for each sub-region by multiplying the subjective assessment score for each sub-region by the location weight for each sub-region. Please also read paragraph [0140-0141 and 0150-151]) based on:
a stability of the one or more recognition results over the multiple image frames (Fig. 1. Paragraph [0167]-KIM discloses the final quality score obtainment unit 513 may obtain a final quality score for the entire video by using the final quality score for each frame. The final quality score obtainment unit 513 may consider a temporal influence or temporal dependence related to video recognition, by using the quality scores of frames accumulated over time. The final quality score obtainment unit 513 may obtain a final quality score for the entire video by smoothing time-series data. The final quality score obtainment unit 513 may use a simple heuristic rule or a neural network model to smooth the time-series data. The final quality score obtainment unit 513 may obtain a final quality score for the entire video in consideration of an effect over time with respect to accumulated time-series data. Please also read paragraph [0174-0177]).
KIM fails to explicitly teach and relevance score of one or more neighboring objects;
However, LEE explicitly teaches and relevance score of one or more neighboring objects (Fig. 5. Paragraph [0071]-LEE discloses FIG. 5 shows an apparatus for panoptic video segmentation. The example shown includes encoder 500, fusion component 505, track head 515, semantic head 520, bounding box head 525, mask head 530, and segmentation component 535. In paragraph [0058]-LEE discloses FIG. 4 shows a span of video frames 410. Multiple frame span 405 may include one or more frames 410. Different span lengths may be used to determine the quality of panoptic segmentation information (e.g., whether objects are correctly classified, and whether instances are consistent between frames). In paragraph [0038]-LEE discloses a Spatial Attention Network (SAN) is a variant of a CNN designed to exploit the spatial context of images. A SAN utilizes attention weights for clustered regional features. The attention weights indicate the value of the contribution of different regions to the overall classification. SAN uses a weighted sum of regional features as discriminative features. Thus, an SAN draws attention to important contents by giving them a higher attention weight);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM of having a device comprising: a processor; and a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations to: receive one or more recognition results for an object over multiple image frames; assign one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames; generate a final recognition result for the object based on the one or more weights; and transmit the final recognition result, with the teachings of LEE of having and relevance score of one or more neighboring objects.
Wherein having KIM’s device having assign one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames, and relevance score of one or more neighboring objects.
The motivation behind the modification would have been to obtain a device that improves the quality and quality assessment of both images and object recognition/tracking, since both KIM and LEE concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LEE provides systems and methods that improve scene labeling and per-frame panoptic quality (PQ) by properly utilizing spatial-temporal features. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LEE et al. (US 20210326638 A1), Abstract and paragraph [0051 and 0049].
Regarding claim 15, KIM explicitly teaches a non-transitory computer readable storage medium storing computer executable instructions that when executed by a computing device cause the computing device to effectuate operations (Fig. 1. Paragraph [0267]-KIM discloses video quality assessment methods and apparatuses according to some embodiments can be embodied as a storage medium including instruction codes executable by a computer such as a program module executed by the computer. A computer readable medium can be any available medium which can be accessed by the computer and includes all volatile/non-volatile and removable/non-removable media. Further, the computer readable medium may include both computer storage and communication media. The computer storage medium includes all volatile/non-volatile and removable/non-removable media embodied by a certain method or technology for storing information such as computer readable instruction code, a data structure, a program module or other data. Please also read paragraph [0054 and 0131]) comprising:
receive one or more recognition results for an object over multiple image frames (Fig. 1, #120 called input video. Paragraph [0115]-KIM discloses the electronic device 100a may receive the video and may divide each frame of the received video into the plurality of sub-regions. In paragraph [0135]-KIM discloses the high-complexity feature information obtainment unit 219 may obtain the high-complexity feature information from an input frame (wherein high-complexity information may include object classifications/detections, semantic segmentations, saliency mapping and weighting assignments). Further in paragraph [0150]-KIM discloses the first neural network 511 may be a model trained to extract a quality assessment score of an image from data input by analyzing and classifying the input data. Please also read paragraph [0165-0167]);
generate a final recognition result for the object based on the one or more weights (Fig. 1. Paragraph [0165]-KIM discloses the final quality score obtainment unit 517 may receive the weighted assessment score matrix from the second neural network 512. The final quality score obtainment unit 513 may obtain a final quality score for the entire frame by averaging the weighted assessment scores included in the weighted assessment score matrix. Please also read paragraph [0150, 0163 and 0166]); and
transmit the final recognition result (Fig. 1. Paragraph [0164]-KIM discloses the second neural network 512 may transmit a weighted assessment score matrix including the weighted assessment score of each of the plurality of sub-regions to the final quality score obtainment unit 513. In paragraph [0166]-KIM discloses the electronic device 100a may further include a high-complexity feature information obtainment unit. The final quality score obtainment unit 513 may receive a high-complexity weight indicating high-complexity feature information from the high-complexity feature information obtainment unit, and may apply the high-complexity weight to the weighted assessment score for each sub-region. The final quality score obtainment unit 513 may obtain a final quality score for the entire frame, based on the weighted assessment score for each sub-region to which the high-complexity weight has been applied).
Although KIM explicitly teaches assign one or more weights to the one or more recognition results (Fig. 4. Paragraph [0087]-KIM discloses pieces of information included in the high-complexity information may be assigned different importances with different weights, respectively. In paragraph [0163]-KIM discloses the second neural network 512 may apply the location weight to the subjective assessment score for each of the plurality of sub-regions received from the first neural network 511. The second neural network 512 may obtain the weighted assessment score for each sub-region by multiplying the subjective assessment score for each sub-region by the location weight for each sub-region. Please also read paragraph [0140-0141 and 0150-151]) based on:
a stability of the one or more recognition results over the multiple image frames (Fig. 1. Paragraph [0167]-KIM discloses the final quality score obtainment unit 513 may obtain a final quality score for the entire video by using the final quality score for each frame. The final quality score obtainment unit 513 may consider a temporal influence or temporal dependence related to video recognition, by using the quality scores of frames accumulated over time. The final quality score obtainment unit 513 may obtain a final quality score for the entire video by smoothing time-series data. The final quality score obtainment unit 513 may use a simple heuristic rule or a neural network model to smooth the time-series data. The final quality score obtainment unit 513 may obtain a final quality score for the entire video in consideration of an effect over time with respect to accumulated time-series data. Please also read paragraph [0174-0177]).
KIM fails to explicitly teach and relevance score of one or more neighboring objects;
However, LEE explicitly teaches and relevance score of one or more neighboring objects (Fig. 5. Paragraph [0071]-LEE discloses FIG. 5 shows an apparatus for panoptic video segmentation. The example shown includes encoder 500, fusion component 505, track head 515, semantic head 520, bounding box head 525, mask head 530, and segmentation component 535. In paragraph [0058]-LEE discloses FIG. 4 shows a span of video frames 410. Multiple frame span 405 may include one or more frames 410. Different span lengths may be used to determine the quality of panoptic segmentation information (e.g., whether objects are correctly classified, and whether instances are consistent between frames). In paragraph [0038]-LEE discloses a Spatial Attention Network (SAN) is a variant of a CNN designed to exploit the spatial context of images. A SAN utilizes attention weights for clustered regional features. The attention weights indicate the value of the contribution of different regions to the overall classification. SAN uses a weighted sum of regional features as discriminative features. Thus, an SAN draws attention to important contents by giving them a higher attention weight).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM of having a non-transitory computer readable storage medium storing computer executable instructions that when executed by a computing device cause the computing device to effectuate operations comprising: receive one or more recognition results for an object over multiple image frames; assign one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames; generate a final recognition result for the object based on the one or more weights; and transmit the final recognition result., with the teachings of LEE of having and relevance score of one or more neighboring objects.
Wherein having KIM’s non-transitory computer readable storage medium having assign one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames, and relevance score of one or more neighboring objects.
The motivation behind the modification would have been to obtain a non-transitory computer readable storage medium that improves the quality and quality assessment of both images and object recognition/tracking, since both KIM and LEE concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LEE provides systems and methods that improve scene labeling and per-frame panoptic quality (PQ) by properly utilizing spatial-temporal features. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LEE et al. (US 20210326638 A1), Abstract and paragraph [0051 and 0049].
Claims 2, 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20220392210 A1), hereinafter referenced as KIM in view of LEE et al. (US 20210326638 A1), hereinafter referenced as LEE and in further view of KARAVADI et al. (US 20180268556 A1), hereinafter referenced as KARAVADI.
Regarding claim 2, KIM in view of LEE explicitly teach the method of claim 1, KIM fails to explicitly teach wherein the assigning of the one or more weights based on the stability comprises assigning a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
However, KARAVADI explicitly teaches wherein the assigning of the one or more weights based on the stability (Fig. 1. Abstract-KARAVADI discloses weights based upon spatio-temporal statistical properties of the extracted foreground blobs and blob edge overlap are used to identify and track with bounding boxes for one or more true objects. In paragraph [0018]-KARAVADI discloses referring to FIG. 1 that illustrates a flow diagram of a method for robust tracking of an object. At step 102, the video stream is received the corresponding one or more images are retrieved. In paragraph [0026]-KARAVADI discloses statistical weights are assigned for size/area of the extracted foreground blobs, intensity, distance, displacement vectors, and dilated edges measures across frames for the extracted one or more blobs. Please also read paragraph [0032 and 0034]) comprises assigning a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames (Fig. 1. Paragraph [0046]-KARAVADI discloses the statistical weight assignment module upon being executed by the processor may identify potential targets by assigning a weight to the map from one or more blobs in the current frame to the one or more potential correspondences in the next frame. Identifying potential targets may include selecting the blob with the highest total positive weight as the blob corresponding to the one or more blobs in the current frame. The tracking objects detection module upon being executed by the processor may track one or more potential targets and display the bounding box for one or more true objects by selecting one or more potential blobs whose weights of the mapping exceeds a predetermined threshold for more than predefined minimum frames continuously. Therefore, it would have been obvious to a person of ordinary skill in the art to assign a first weight to the recognition results that are within a threshold range of consistency over a threshold number of image frames. KARAVADI teaches assigning weight based on temporal consistency and designating recognition results as true when a threshold weight has been reached for a minimum number of frames. Thus, it would be obvious to assign a weight to recognition results when a threshold level of consistency has been met for a certain period given this would improve future determinations where there is still uncertainty).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a method comprising: receiving one or more recognition results for an object over multiple image frames; assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames and relevance scoring of one or more neighboring objects, with the teachings of KARAVADI of having wherein the assigning of the one or more weights based on the stability comprises assigning a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
Wherein having KIM’s method having wherein the assigning of the one or more weights based on the stability comprises assigning a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a method that improves the quality and quality assessment of images and object recognition, since both KIM and KARAVADI concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while KARAVADI provides systems and methods that improves object tracking by weighting based upon spatio-temporal statistical properties to identify and track with bounding boxes. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and KARAVADI et al. (US 20180268556 A1), Abstract and paragraph [0051 and 0049].
Regarding claim 9, KIM in view of LEE explicitly teach the device of claim 8, KIM in view of LEE fails to explicitly teach wherein the one or more processors, when the assigning of the one or more weights based on the stability, are configured to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
However, KARAVADI explicitly teaches wherein the one or more processors, when the assigning of the one or more weights based on the stability (Fig. 1. Abstract-KARAVADI discloses weights based upon spatio-temporal statistical properties of the extracted foreground blobs and blob edge overlap are used to identify and track with bounding boxes for one or more true objects. In paragraph [0018]-KARAVADI discloses referring to FIG. 1 that illustrates a flow diagram of a method for robust tracking of an object. At step 102, the video stream is received the corresponding one or more images are retrieved. In paragraph [0026]-KARAVADI discloses statistical weights are assigned for size/area of the extracted foreground blobs, intensity, distance, displacement vectors, and dilated edges measures across frames for the extracted one or more blobs. Please also read paragraph [0032 and 0034]), are configured to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames (Fig. 1. Paragraph [0046]-KARAVADI discloses the statistical weight assignment module upon being executed by the processor may identify potential targets by assigning a weight to the map from one or more blobs in the current frame to the one or more potential correspondences in the next frame. Identifying potential targets may include selecting the blob with the highest total positive weight as the blob corresponding to the one or more blobs in the current frame. The tracking objects detection module upon being executed by the processor may track one or more potential targets and display the bounding box for one or more true objects by selecting one or more potential blobs whose weights of the mapping exceeds a predetermined threshold for more than predefined minimum frames continuously. Therefore, it would have been obvious to a person of ordinary skill in the art to assign a first weight to the recognition results that are within a threshold range of consistency over a threshold number of image frames. KARAVADI teaches assigning weight based on temporal consistency and designating recognition results as true when a threshold weight has been reached for a minimum number of frames. Thus, it would be obvious to assign a weight to recognition results when a threshold level of consistency has been met for a certain period given this would improve future determinations where there is still uncertainty).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a device comprising: a processor; and a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations to: receive one or more recognition results for an object over multiple image frames, with the teachings of KARAVADI of having wherein the one or more processors, when the assigning of the one or more weights based on the stability, are configured to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
Wherein having KIM’s device having wherein the one or more processors, when the assigning of the one or more weights based on the stability, are configured to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a device that improves the quality and quality assessment of images and object recognition, since both KIM and KARAVADI concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while KARAVADI provides systems and methods that improves object tracking by weighting based upon spatio-temporal statistical properties to identify and track with bounding boxes. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and KARAVADI et al. (US 20180268556 A1), Abstract and paragraph [0051 and 0049].
Regarding claim 16, KIM in view of LEE explicitly teach the non-transitory computer-readable medium of claim 15, KIM in view of LEE fails to explicitly teach wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability, cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
However, KARAVADI explicitly teaches wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability (Fig. 1. Abstract-KARAVADI discloses weights based upon spatio-temporal statistical properties of the extracted foreground blobs and blob edge overlap are used to identify and track with bounding boxes for one or more true objects. In paragraph [0018]-KARAVADI discloses referring to FIG. 1 that illustrates a flow diagram of a method for robust tracking of an object. At step 102, the video stream is received the corresponding one or more images are retrieved. In paragraph [0026]-KARAVADI discloses statistical weights are assigned for size/area of the extracted foreground blobs, intensity, distance, displacement vectors, and dilated edges measures across frames for the extracted one or more blobs. Please also read paragraph [0032 and 0034]), cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames (Fig. 1. Paragraph [0046]-KARAVADI discloses the statistical weight assignment module upon being executed by the processor may identify potential targets by assigning a weight to the map from one or more blobs in the current frame to the one or more potential correspondences in the next frame. Identifying potential targets may include selecting the blob with the highest total positive weight as the blob corresponding to the one or more blobs in the current frame. The tracking objects detection module upon being executed by the processor may track one or more potential targets and display the bounding box for one or more true objects by selecting one or more potential blobs whose weights of the mapping exceeds a predetermined threshold for more than predefined minimum frames continuously. Therefore, it would have been obvious to a person of ordinary skill in the art to assign a first weight to the recognition results that are within a threshold range of consistency over a threshold number of image frames. KARAVADI teaches assigning weight based on temporal consistency and designating recognition results as true when a threshold weight has been reached for a minimum number of frames. Thus, it would be obvious to assign a weight to recognition results when a threshold level of consistency has been met for a certain period given this would improve future determinations where there is still uncertainty).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a non-transitory computer readable storage medium storing computer executable instructions that when executed by a computing device cause the computing device to effectuate operations comprising: receive one or more recognition results for an object over multiple image frames, with the teachings of KARAVADI of having wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability, cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
Wherein having KIM’s non-transitory computer readable storage medium having wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability, cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a non-transitory computer readable storage medium that improves the quality and quality assessment of images and object recognition, since both KIM and KARAVADI concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while KARAVADI provides systems and methods that improves object tracking by weighting based upon spatio-temporal statistical properties to identify and track with bounding boxes. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and KARAVADI et al. (US 20180268556 A1), Abstract and paragraph [0051 and 0049].
Claims 3, 6, 10, 13, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20220392210 A1), hereinafter referenced as KIM in view of LEE et al. (US 20210326638 A1), hereinafter referenced as LEE and in further view of LIU et al. (US 20230360396 A1), hereinafter referenced as LIU.
Regarding claim 3, KIM in view of LEE explicitly teach the method of claim 1, KIM in view of LEE fails to explicitly teach wherein the assigning of the one or more weights based on relevance scoring comprises assigning a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity.
However, LIU explicitly teaches wherein the assigning of the one or more weights based on relevance scoring (Fig. 1. Paragraph [0064]-LIU discloses at 250, the inference module 330 uses the segmentation map 320, importance weights associated with the various classes, and the locations of objects within the image to compute class scores 340 (wherein objects or regions in images may include weighting for class importance and spatial importance). In paragraph [0088]-LIU discloses the temporally filtered area ratios may then be supplied to a ranking module 640 to rank the classes detected in the image based on the weighted area ratios of the classes in order to compute ranked labels 642) comprises assigning a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity (Fig. 1. Paragraph [0081]-LIU discloses another factor is that a scene class may be very small in spatial size in the image, but may be very important to the scene. For example, the text class has a higher weight than the others classes in the same group because high quality (e.g., in focus and high contrast) appearance of text may be of particular importance when capturing images (e.g., for optical character recognition), but may also make up a very small part of the total area of the image).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a method comprising: receiving one or more recognition results for an object over multiple image frames; assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames and relevance scoring of one or more neighboring objects, with the teachings of LIU of having wherein the assigning of the one or more weights based on relevance scoring comprises assigning a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity.
Wherein having KIM’s method having wherein the assigning of the one or more weights based on relevance scoring comprises assigning a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity.
The motivation behind the modification would have been to obtain a method that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Regarding claim 6, KIM in view of LEE explicitly teaches the method of claim 1, KIM fails to explicitly teach wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames.
However, LIU explicitly teaches wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames (Fig. 4. Paragraph [0047]-LIU discloses the present disclosure utilize semantic segmentation to perform the dominant scene classification. The present disclosure relates to assigning importance weights to objects detected in a scene, where the importance weights may be calculated based on the class of the object, a location of the object within the image, and an area ratio of the object in the image (wherein semantic segmentation and class scores are based on spatial, temporal and class importance based inferences). Please also read paragraph [0108-0112]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a method comprising: receiving one or more recognition results for an object over multiple image frames; assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames and relevance scoring of one or more neighboring objects, with the teachings of KARAVADI of having wherein the assigning of the one or more weights based on the stability comprises assigning a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
Wherein having KIM’s method having wherein the assigning of the one or more weights based on the stability comprises assigning a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a method that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Regarding claim 10, KIM in view of LEE explicitly teach the device of claim 8, KIM in view of LEE fails to explicitly teach wherein the one or more processors, when the assigning of the one or more weights based on relevance scoring, are configured to assign a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity.
However, LIU explicitly teaches wherein the one or more processors, when the assigning of the one or more weights based on relevance scoring (Fig. 4. Paragraph [0064]-LIU discloses at 250, the inference module 330 uses the segmentation map 320, importance weights associated with the various classes, and the locations of objects within the image to compute class scores 340 (wherein objects or regions in images may include weighting for class importance and spatial importance). In paragraph [0088]-LIU discloses the temporally filtered area ratios may then be supplied to a ranking module 640 to rank the classes detected in the image based on the weighted area ratios of the classes in order to compute ranked labels 642), are configured to assign a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity (Fig. 4. Paragraph [0081]-LIU discloses another factor is that a scene class may be very small in spatial size in the image, but may be very important to the scene. For example, the text class has a higher weight than the others classes in the same group because high quality (e.g., in focus and high contrast) appearance of text may be of particular importance when capturing images (e.g., for optical character recognition), but may also make up a very small part of the total area of the image).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a device comprising: a processor; and a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations to: receive one or more recognition results for an object over multiple image frames, with the teachings of LIU of having wherein the one or more processors, when the assigning of the one or more weights based on relevance scoring, are configured to assign a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity.
Wherein having KIM’s device having wherein the one or more processors, when the assigning of the one or more weights based on relevance scoring, are configured to assign a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity.
The motivation behind the modification would have been to obtain a device that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Regarding claim 13, KIM in view of LEE explicitly teaches the device of claim 8, KIM fails to explicitly teach wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames.
However, LIU explicitly teaches wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames (Fig. 4. Paragraph [0047]-LIU discloses the present disclosure utilize semantic segmentation to perform the dominant scene classification. The present disclosure relates to assigning importance weights to objects detected in a scene, where the importance weights may be calculated based on the class of the object, a location of the object within the image, and an area ratio of the object in the image (wherein semantic segmentation and class scores are based on spatial, temporal and class importance based inferences). Please also read paragraph [0108-0112]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a device comprising: a processor; and a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations to: receive one or more recognition results for an object over multiple image frames, with the teachings of LIU of having wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames.
Wherein having KIM’s device having wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames.
The motivation behind the modification would have been to obtain a device that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Regarding claim 17, KIM in view of LEE explicitly teach the non-transitory computer-readable medium of claim 15, KIM in view of LEE fails to explicitly teach wherein the one or more instructions that cause the device to the assigning of the one or more weights based on relevance score, cause the device to assign a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity.
However, LIU explicitly teaches wherein the one or more instructions that cause the device to the assigning of the one or more weights based on relevance score (Fig. 4. Paragraph [0064]-LIU discloses at 250, the inference module 330 uses the segmentation map 320, importance weights associated with the various classes, and the locations of objects within the image to compute class scores 340 (wherein objects or regions in images may include weighting for class importance and spatial importance). In paragraph [0088]-LIU discloses the temporally filtered area ratios may then be supplied to a ranking module 640 to rank the classes detected in the image based on the weighted area ratios of the classes in order to compute ranked labels 642), cause the device to assign a first weight to the one or more recognition results in one or more areas of an image frame based on a threshold level of clarity (Fig. 4. Paragraph [0081]-LIU discloses another factor is that a scene class may be very small in spatial size in the image, but may be very important to the scene. For example, the text class has a higher weight than the others classes in the same group because high quality (e.g., in focus and high contrast) appearance of text may be of particular importance when capturing images (e.g., for optical character recognition), but may also make up a very small part of the total area of the image).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a non-transitory computer readable storage medium storing computer executable instructions that when executed by a computing device cause the computing device to effectuate operations comprising: receive one or more recognition results for an object over multiple image frames, with the teachings of KARAVADI of having wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability, cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
Wherein having KIM’s non-transitory computer readable storage medium having wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability, cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a non-transitory computer readable storage medium that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Regarding claim 20, KIM in view of LEE explicitly teaches the non-transitory computer-readable medium of claim 15, KIM fails to explicitly teach wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames.
However, LIU explicitly teaches wherein the relevance scoring of one or more neighboring objects is based on semantic context of one or more image regions of the multiple image frames (Fig. 4. Paragraph [0047]-LIU discloses the present disclosure utilize semantic segmentation to perform the dominant scene classification. The present disclosure relates to assigning importance weights to objects detected in a scene, where the importance weights may be calculated based on the class of the object, a location of the object within the image, and an area ratio of the object in the image (wherein semantic segmentation and class scores are based on spatial, temporal and class importance based inferences). Please also read paragraph [0108-0112]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a non-transitory computer readable storage medium storing computer executable instructions that when executed by a computing device cause the computing device to effectuate operations comprising: receive one or more recognition results for an object over multiple image frames, with the teachings of KARAVADI of having wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability, cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
Wherein having KIM’s non-transitory computer readable storage medium having wherein the one or more instructions that cause the computing device to the assigning of the one or more weights based on the stability, cause the device to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a non-transitory computer readable storage medium that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Claims 4, 7, 11, 14, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20220392210 A1), hereinafter referenced as KIM in view of LEE et al. (US 20210326638 A1), hereinafter referenced as LEE and in further view of YUAN et al. (US 20240104760 A1), hereinafter referenced as YUAN.
Regarding claim 4, KIM in view of LEE explicitly teaches the method of claim 1, KIM in view of LEE fails to explicitly teaches further comprising: determining a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
However, YUAN explicitly teaches further comprising: determining a number of possible object candidates (Fig. 4d, #431 and #432 called objects or parts. Paragraph [0064]) based on a level of confidence (Fig. 4d. Paragraph [0065]-YUAN discloses FIG. 5 illustrates a flowchart describing a method for determining a probability value indicating a probability that an object captured in a stream of image frames 422 belongs to an object type. In paragraph [0071]-YUAN discloses the determining may be performed by weighting probability values for several parts of the first object 431. In paragraph [0073]-YUAN discloses an object may be counted as specific pre-defined object when the first probability value is above the first threshold value. The first threshold value may be a masking threshold value, a counting threshold value or another threshold value associated with some other function to be performed on the object as a consequence of the first probability value being above the first threshold value) in one or more areas (Fig. 4d, #441 and #442 called areas. Paragraph [0069 and 0074]) of an image frame (Fig. 4d, #421 and #422 called image frames. Paragraph [0064]-YUAN discloses FIG. 4b illustrates a first stream of image frames 421 captured by the first camera 401 of the multi-camera system 400 and a second stream of image frames 422 captured by the second camera 402 of the camera system 400. The second camera 402 is different from the first camera 401. The first stream of image frames 421 comprises an image frame 421_2 capturing a first object 431. The first object 431 may comprise a first part 431a. The second stream of image frames 422 comprises an image frame 422_2 capturing a second object 432. The second object 432 may comprise a second part 432a. The first object 431 may correspond to the second object 432 in that it is the same real object that has been captured. Also, the first part 431a may correspond to the second part 432a) based on a level of clarity (Fig. 4d. Paragraph [0056]-YUAN discloses the detection information may comprise a detection score or a detection probability. Different cameras may have different pre-conditions to detect people due to camera resolution, installation positions of different cameras, and their processing power, which is turn impacts characteristics of a video network (wherein an example of these characteristics is a size of the image, visibility/occlusion, comparative pixel density) too low pixel density to be detected in one camera may be detectable in another camera where the pixel density of the person is higher. Please also read paragraph [0046-0047]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a method comprising: receiving one or more recognition results for an object over multiple image frames; assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames and relevance scoring of one or more neighboring objects, with the teachings of YUAN of having further comprising: determining a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
Wherein having KIM’s method having further comprising: determining a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
The motivation behind the modification would have been to obtain a method that improves the quality and quality assessment of images and object recognition, since both KIM and YUAN concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while YUAN provides systems and methods that improves determining a probability value indicating whether an object captured in a stream of image frames belongs to an object type. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and YUAN et al. (US 20240104760 A1), Abstract and paragraph [0013].
Regarding claim 7, KIM in view of LEE explicitly teaches the method of claim 1, KIM further teaches further comprising: tracking one or more objects over time (Fig. 1. Paragraph [0167]-KIM discloses the final quality score obtainment unit 513 may obtain a final quality score for the entire video by using the final quality score for each frame. The final quality score obtainment unit 513 may consider a temporal influence or temporal dependence related to video recognition, by using the quality scores of frames accumulated over time. The final quality score obtainment unit 513 may obtain a final quality score for the entire video by smoothing time-series data. The final quality score obtainment unit 513 may use a simple heuristic rule or a neural network model to smooth the time-series data. The final quality score obtainment unit 513 may obtain a final quality score for the entire video in consideration of an effect over time with respect to accumulated time-series data. Please also read paragraph [0174-0177]); and
KIM in view of LEE fails to explicitly teach sending an indication of an increase in a level of confidence associated with a first recognition result based on a frequency of one or more recognition results.
However, YUAN explicitly teaches sending an indication of an increase in a level of confidence associated with a first recognition result (Fig. 4d. Paragraph [0063]-YUAN discloses cameras may assign a detection score, e.g., a probability value indicating that the detected object belongs to an object type. In paragraph [0071]-YUAN discloses the determining may be performed by weighting probability values for several parts of the first object 431. In paragraph [0073]-YUAN discloses an object may be counted as specific pre-defined object when the first probability value is above the first threshold value. The first threshold value may be a masking threshold value, a counting threshold value or another threshold value associated with some other function to be performed on the object as a consequence of the first probability value being above the first threshold value) based on a frequency of one or more recognition results (Fig. 4d. Paragraph [0101]-YUAN discloses the object type is an object type to be counted. Then the first and second threshold values may be for counting objects or parts of objects of the object type to be counted. Then the method further comprises: when the updated second probability value is above the second threshold value, increasing a counter value associated with the second stream of image frames 422. The counter value may be for the object or the object part. In paragraph [0124]-YUAN discloses the image-processing device 600 and/or the processing module 601 and/or the counting module 640 may further be configured to increase the counter value associated with the second stream of image frames 422 when the updated second probability value is above the second threshold value.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a method comprising: receiving one or more recognition results for an object over multiple image frames; assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames and relevance scoring of one or more neighboring objects, with the teachings of YUAN of having sending an indication of an increase in a level of confidence associated with a first recognition result based on a frequency of one or more recognition results.
Wherein having KIM’s method having sending an indication of an increase in a level of confidence associated with a first recognition result based on a frequency of one or more recognition results.
The motivation behind the modification would have been to obtain a method that improves the quality and quality assessment of images and object recognition, since both KIM and YUAN concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while YUAN provides systems and methods that improves determining a probability value indicating whether an object captured in a stream of image frames belongs to an object type. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and YUAN et al. (US 20240104760 A1), Abstract and paragraph [0013].
Regarding claim 11, KIM in view of LEE explicitly teaches the device of claim 8, KIM in view of LEE fails to explicitly teaches wherein the one or more processors are further configured to: determine a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
However, YUAN explicitly teaches wherein the one or more processors are further configured to: determine a number of possible object candidates (Fig. 4d, #431 and #432 called objects or parts. Paragraph [0064]) based on a level of confidence (Fig. 5. Paragraph [0065]-YUAN discloses FIG. 5 illustrates a flowchart describing a method for determining a probability value indicating a probability that an object captured in a stream of image frames 422 belongs to an object type. In paragraph [0071]-YUAN discloses the determining may be performed by weighting probability values for several parts of the first object 431. In paragraph [0073]-YUAN discloses an object may be counted as specific pre-defined object when the first probability value is above the first threshold value. The first threshold value may be a masking threshold value, a counting threshold value or another threshold value associated with some other function to be performed on the object as a consequence of the first probability value being above the first threshold value) in one or more areas (Fig. 4d, #441 and #442 called areas. Paragraph [0069 and 0074]) of an image frame (Fig. 4d, #421 and #422 called image frames. Paragraph [0064]-YUAN discloses FIG. 4b illustrates a first stream of image frames 421 captured by the first camera 401 of the multi-camera system 400 and a second stream of image frames 422 captured by the second camera 402 of the camera system 400. The second camera 402 is different from the first camera 401. The first stream of image frames 421 comprises an image frame 421_2 capturing a first object 431. The first object 431 may comprise a first part 431a. The second stream of image frames 422 comprises an image frame 422_2 capturing a second object 432. The second object 432 may comprise a second part 432a. The first object 431 may correspond to the second object 432 in that it is the same real object that has been captured. Also, the first part 431a may correspond to the second part 432a) based on a level of clarity (Fig. 4d. Paragraph [0056]-YUAN discloses the detection information may comprise a detection score or a detection probability. Different cameras may have different pre-conditions to detect people due to camera resolution, installation positions of different cameras, and their processing power, which is turn impacts characteristics of a video network (wherein an example of these characteristics is a size of the image, visibility/occlusion, comparative pixel density) too low pixel density to be detected in one camera may be detectable in another camera where the pixel density of the person is higher. Please also read paragraph [0046-0047]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a device comprising: a processor; and a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations to: receive one or more recognition results for an object over multiple image frames, with the teachings of YUAN of having wherein the one or more processors are further configured to: determine a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
Wherein having KIM’s device having wherein the one or more processors are further configured to: determine a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
The motivation behind the modification would have been to obtain a device that improves the quality and quality assessment of images and object recognition, since both KIM and YUAN concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while YUAN provides systems and methods that improves determining a probability value indicating whether an object captured in a stream of image frames belongs to an object type. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and YUAN et al. (US 20240104760 A1), Abstract and paragraph [0013].
Regarding claim 14, KIM in view of LEE explicitly teaches he device of claim 8, KIM further teaches wherein the one or more processors are further configured to: track one or more objects over time (Fig. 1. Paragraph [0167]-KIM discloses the final quality score obtainment unit 513 may obtain a final quality score for the entire video by using the final quality score for each frame. The final quality score obtainment unit 513 may consider a temporal influence or temporal dependence related to video recognition, by using the quality scores of frames accumulated over time. The final quality score obtainment unit 513 may obtain a final quality score for the entire video by smoothing time-series data. The final quality score obtainment unit 513 may use a simple heuristic rule or a neural network model to smooth the time-series data. The final quality score obtainment unit 513 may obtain a final quality score for the entire video in consideration of an effect over time with respect to accumulated time-series data. Please also read paragraph [0174-0177]);
KIM in view of LEE fails to explicitly teach and send an indication of an increase in a level of confidence associated with a recognition result based on a frequency of one or more recognition results.
However, YUAN explicitly teaches and send an indication of an increase in a level of confidence associated with a recognition result (Fig. 4d. Paragraph [0063]-YUAN discloses cameras may assign a detection score, e.g., a probability value indicating that the detected object belongs to an object type. In paragraph [0071]-YUAN discloses the determining may be performed by weighting probability values for several parts of the first object 431. In paragraph [0073]-YUAN discloses an object may be counted as specific pre-defined object when the first probability value is above the first threshold value. The first threshold value may be a masking threshold value, a counting threshold value or another threshold value associated with some other function to be performed on the object as a consequence of the first probability value being above the first threshold value) based on a frequency of one or more recognition results (Fig. 4d. Paragraph [0101]-YUAN discloses the object type is an object type to be counted. Then the first and second threshold values may be for counting objects or parts of objects of the object type to be counted. Then the method further comprises: when the updated second probability value is above the second threshold value, increasing a counter value associated with the second stream of image frames 422. The counter value may be for the object or the object part. In paragraph [0124]-YUAN discloses the image-processing device 600 and/or the processing module 601 and/or the counting module 640 may further be configured to increase the counter value associated with the second stream of image frames 422 when the updated second probability value is above the second threshold value.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a device comprising: a processor; and a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations to: receive one or more recognition results for an object over multiple image frames, with the teachings of KARAVADI of having wherein the one or more processors, when the assigning of the one or more weights based on the stability, are configured to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
Wherein having KIM’s device having wherein the one or more processors, when the assigning of the one or more weights based on the stability, are configured to assign a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a device that improves the quality and quality assessment of images and object recognition, since both KIM and YUAN concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while YUAN provides systems and methods that improves determining a probability value indicating whether an object captured in a stream of image frames belongs to an object type. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and YUAN et al. (US 20240104760 A1), Abstract and paragraph [0013].
Regarding claim 18, KIM in view of LEE explicitly teaches the non-transitory computer-readable medium of claim 15, KIM in view of LEE fails to explicitly teaches wherein the one or more instructions further cause the device to: determine a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
However, YUAN explicitly teaches wherein the one or more instructions further cause the device to: determine a number of possible object candidates (Fig. 4d, #431 and #432 called objects or parts. Paragraph [0064]) based on a level of confidence (Fig. 5. Paragraph [0065]-YUAN discloses FIG. 5 illustrates a flowchart describing a method for determining a probability value indicating a probability that an object captured in a stream of image frames 422 belongs to an object type. In paragraph [0071]-YUAN discloses the determining may be performed by weighting probability values for several parts of the first object 431. In paragraph [0073]-YUAN discloses an object may be counted as specific pre-defined object when the first probability value is above the first threshold value. The first threshold value may be a masking threshold value, a counting threshold value or another threshold value associated with some other function to be performed on the object as a consequence of the first probability value being above the first threshold value) in one or more areas (Fig. 4d, #441 and #442 called areas. Paragraph [0069 and 0074]) of an image frame (Fig. 4d, #421 and #422 called image frames. Paragraph [0064]-YUAN discloses FIG. 4b illustrates a first stream of image frames 421 captured by the first camera 401 of the multi-camera system 400 and a second stream of image frames 422 captured by the second camera 402 of the camera system 400. The second camera 402 is different from the first camera 401. The first stream of image frames 421 comprises an image frame 421_2 capturing a first object 431. The first object 431 may comprise a first part 431a. The second stream of image frames 422 comprises an image frame 422_2 capturing a second object 432. The second object 432 may comprise a second part 432a. The first object 431 may correspond to the second object 432 in that it is the same real object that has been captured. Also, the first part 431a may correspond to the second part 432a) based on a level of clarity (Fig. 4d. Paragraph [0056]-YUAN discloses the detection information may comprise a detection score or a detection probability. Different cameras may have different pre-conditions to detect people due to camera resolution, installation positions of different cameras, and their processing power, which is turn impacts characteristics of a video network (wherein an example of these characteristics is a size of the image, visibility/occlusion, comparative pixel density) too low pixel density to be detected in one camera may be detectable in another camera where the pixel density of the person is higher. Please also read paragraph [0046-0047]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE of having a non-transitory computer readable storage medium storing computer executable instructions that when executed by a computing device cause the computing device to effectuate operations comprising: receive one or more recognition results for an object over multiple image frames, with the teachings of YUAN of having wherein the one or more instructions further cause the device to: determine a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
Wherein having KIM’s non-transitory computer readable storage medium having wherein the one or more instructions further cause the device to: determine a number of possible object candidates based on a level of confidence in one or more areas of an image frame based on a level of clarity.
The motivation behind the modification would have been to obtain a non-transitory computer readable storage medium that improves the quality and quality assessment of images and object recognition, since both KIM and YUAN concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while YUAN provides systems and methods that improves determining a probability value indicating whether an object captured in a stream of image frames belongs to an object type. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and YUAN et al. (US 20240104760 A1), Abstract and paragraph [0013].
Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20220392210 A1), hereinafter referenced as KIM in view of LEE et al. (US 20210326638 A1), hereinafter referenced as LEE and in further view of YUAN et al. (US 20240104760 A1), hereinafter referenced as YUAN and in further view of LIU et al. (US 20230360396 A1), hereinafter referenced as LIU.
Regarding claim 5, KIM in view of LEE and in further view of YUAN explicitly teaches the method of claim 4, KIM in view of LEE fails to explicitly teach further comprising: using different thresholding approaches based on an indication of the level of confidence.
However, LIU explicitly teaches further comprising: using different thresholding approaches based on an indication of the level of confidence (Fig. 4. Paragraph [0096]-LIU discloses at 720, the dominant scene classifier 300 applies per class thresholds [t.sub.1, t.sub.2, . . . t.sub.k] to each element of the segmentation map s.sub.m, to obtain a thresholded segmentation map s′.sub.m in accordance with the confidence map. In paragraph [0097]-LIU discloses qualitatively, each location or pixel of the thresholded segmentation map has the class value c of the segmentation map when the confidence value of that classification is greater than a threshold t.sub.c for that class c. In paragraph [0098]-LIU discloses class importance is also applied when computing the thresholded segmentation map, giving more weight to important classes. Please also read paragraph [0092 and 0094]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE and in further view of YUAN of having a method comprising: receiving one or more recognition results for an object over multiple image frames; assigning one or more weights to the one or more recognition results based on: a stability of the one or more recognition results over the multiple image frames and relevance scoring of one or more neighboring objects, with the teachings of LIU of having further comprising: using different thresholding approaches based on an indication of the level of confidence.
Wherein having KIM’s method having wherein the assigning of the one or more weights based on the stability comprises assigning a first weight to the one or more recognition results that are within a threshold range of consistency over a threshold number of image frames.
The motivation behind the modification would have been to obtain a method that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Regarding claim 12, KIM in view of LEE and in further view of YUAN explicitly teaches he device of claim 11, KIM in view of LEE fails to explicitly teach wherein the one or more processors are further configured to: use different thresholding approaches based on an indication of the level of confidence.
However, LIU explicitly teaches wherein the one or more processors are further configured to: use different thresholding approaches based on an indication of the level of confidence (Fig. 4. Paragraph [0096]-LIU discloses at 720, the dominant scene classifier 300 applies per class thresholds [t.sub.1, t.sub.2, . . . t.sub.k] to each element of the segmentation map s.sub.m, to obtain a thresholded segmentation map s′.sub.m in accordance with the confidence map. In paragraph [0097]-LIU discloses qualitatively, each location or pixel of the thresholded segmentation map has the class value c of the segmentation map when the confidence value of that classification is greater than a threshold t.sub.c for that class c. In paragraph [0098]-LIU discloses class importance is also applied when computing the thresholded segmentation map, giving more weight to important classes. Please also read paragraph [0092 and 0094]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE and in further view of YUAN of having a device comprising: a processor; and a memory coupled with the processor, the memory storing executable instructions that when executed by the processor cause the processor to effectuate operations to: receive one or more recognition results for an object over multiple image frames, with the teachings of LIU of having wherein the one or more processors are further configured to: use different thresholding approaches based on an indication of the level of confidence.
Wherein having KIM’s device having wherein the one or more processors are further configured to: use different thresholding approaches based on an indication of the level of confidence.
The motivation behind the modification would have been to obtain a device that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Regarding claim 19, KIM in view of LEE and in further view of YUAN explicitly teaches the non-transitory computer-readable medium of claim 18, KIM in view of LEE fails to explicitly teach wherein the one or more instructions further cause the device to: use different thresholding approaches based on an indication of the level of confidence.
However, LIU explicitly teaches wherein the one or more instructions further cause the device to: use different thresholding approaches based on an indication of the level of confidence (Fig. 4. Paragraph [0096]-LIU discloses at 720, the dominant scene classifier 300 applies per class thresholds [t.sub.1, t.sub.2, . . . t.sub.k] to each element of the segmentation map s.sub.m, to obtain a thresholded segmentation map s′.sub.m in accordance with the confidence map. In paragraph [0097]-LIU discloses qualitatively, each location or pixel of the thresholded segmentation map has the class value c of the segmentation map when the confidence value of that classification is greater than a threshold t.sub.c for that class c. In paragraph [0098]-LIU discloses class importance is also applied when computing the thresholded segmentation map, giving more weight to important classes. Please also read paragraph [0092 and 0094]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIM in view of LEE and in further view of YUAN of having a non-transitory computer readable storage medium storing computer executable instructions that when executed by a computing device cause the computing device to effectuate operations comprising: receive one or more recognition results for an object over multiple image frames, with the teachings of LIU of having wherein the one or more instructions further cause the device to: use different thresholding approaches based on an indication of the level of confidence..
Wherein having KIM’s non-transitory computer readable storage medium having wherein the one or more instructions further cause the device to: use different thresholding approaches based on an indication of the level of confidence.
The motivation behind the modification would have been to obtain a non-transitory computer readable storage medium that improves the quality of images and object recognition, since both KIM and LIU concern systems and methods for video analysis. Wherein KIM provides systems and methods that improve the quality scores for frames, while LIU provides systems and methods that improves segmentation accuracy and performance. Please see KIM et al. (US 20220392210 A1), Abstract and Paragraph [0088] and LIU et al. (US 20230360396 A1), Abstract and paragraph [0051, 0105, and 0110].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant`s disclosure.
Chellappan et al. (US 20220104474 A1)- An insect trap includes a combination of one or more components used to classify the insect according to a genus and species. The trap includes an imaging device, a digital microphone, and passive infrared sensors at the entrance of the trap to sense wing-beat frequencies and size of the insect (to identify entry of a mosquito). A lamb-skin membrane, filled with an insect attractant such as carbon dioxide mixed with gas air inside, mimics human skin so that the insect can rest on the membrane and even pierce the membrane as if a blood meal is available. An imaging device such as a passive infrared sensor or a camera gathers image data of the insect. The insect may be a mosquito................................. Please see Fig. 2-8. Abstract
HASHIMOTO et al. (US 20210042592 A1)- An object detection device calculates, for each of images acquired in time-series, a confidence score of each of a plurality of types of an object to be detected for each of a plurality of regions on the image, detects the type of the object corresponding to the confidence score in a region in which the confidence score is equal to or higher than a confidence threshold of the type for each of the images, tracks the detected object, counts a frequency of occurrence for each type of the detected object in a period in which the detected object is tracked, and updates the confidence threshold in such a way that the confidence threshold of a type having a higher frequency of occurrence is lower than the confidence threshold of a type having a lower frequency of occurrence................................ Please see Fig. 3-7. Abstract
Bitouk et al. (US 20160277645 A1)- Techniques related to detecting local change in video are discussed. Such techniques may include determining inlier and outlier keypoints for a current frame of a video sequence based on inlier keypoints from previous frames, detecting a region of local change based on outlier keypoints of the current and previous frames, and providing an indicator of local change based on the detected region of local change................................ Please see Fig. 2-7. Abstract
CHEN et al. (US 20200175279 A1)- An object recognition method and system thereof are provided. A recognition result of a first object of a (i−1).sup.th frame of a video stream is obtained. A i.sup.th frame is received, and a second object is detected from the i.sup.th frame. Whether the first object and the second object are corresponding to the same target object is determined according to a position of the first object in the (i−1).sup.th frame and a position of the second object in the i.sup.th frame. If the first object and the second object are corresponding to the same target object, whether a recognition confidence level is greater than a predetermined threshold is determined so as to perform the object recognition on the second object or assign the recognition result of the first object to the second object................................ Please see Fig. 1-3 and para. [0036-0039, 0042-0048, and 0067]. Abstract
Chandrasekhar et al. (US 20230082197 A1)- A method and a sports analytics system (SAS) for analyzing a live video broadcast stream (LVBS) of a sporting event are provided. The SAS splits the LVBS into a real time messaging protocol (RTMP) stream and a hypertext transfer protocol live stream (HLS) and analyses the RTMP stream using a phase difference between the RTMP stream and the HLS. The SAS detects persons present in a frame of the RTMP stream using a first set of cues and tracks the detected persons by analyzing preceding frames. The SAS recognizes the tracked persons using a second set of cues, assigns individual weights to each of the second set of cues, and compares the assigned weights of each of the recognized persons with pre-existing data of all players to identify the players in the frame. The SAS transmits the HLS and contextual interactive content of the identified players to a user device.............................. Please see Fig. 3 and paragraph [0035-0040]. Abstract
Nomura et al. (US 20130322696 A1)- The present invention enables detection of a local differing region between images. Inter-image difference information indicating a difference in feature amounts for each subregion between first and second images is generated based on a first feature amount vector that is a set of feature amounts respectively corresponding to a plurality of subregions in the first image and a second feature amount vector that is a set of feature amounts respectively corresponding to a plurality of subregions in the second image, a differing region that is an image region that differs between the first and second images is detected based on differences in the respective subregions indicated by the inter-image difference information, and detection information that indicates a result of the detection is outputted............................ Please see Fig. 3-7 and Para. [0050 and 0083]. Abstract.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to Aaron Bonansinga whose telephone number is (703) 756-5380 The examiner can normally be reached on Monday-Friday, 9:00 a.m. - 6:00 p.m. ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Chineyere Wills-Burns can be reached by phone at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON TIMOTHY BONANSINGA/Examiner, Art Unit 2673 /CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights