Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 16-19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over CN 111010606 A to Zhang et al., hereinafter, “Zhang” in view of US 11158344 B1 to Townsend et al., hereinafter, “Townsend”.
Claim 1. Zhang teaches A video detection method, performed by a computer device, the method comprising: [0134] S204 the condition present in the video frame image containing the mark, dividing frame in the video segment that corresponds to the mark
acquiring a video frame sequence corresponding to a target video; [0135] video is composed of the continuous multi-frame image, a frame image that is one frame picture.
sequentially performing a frosted glass detection on a plurality of video frames in the video frame sequence by using a trained frosted glass region detection model [0135] in the video, frame fragment includes one frame image containing the mark, or comprises a plurality of successive frame image containing the mark.
[0136] …it can by deep learning algorithm such as the method for detecting the video, determining in each frame image of the video whether there is to be removed flag is set in flag and the determination flag area occupied in the frame picture… also can adopt other method for frame-by-frame video detection.
and obtaining a target video frame that includes a frosted glass region in the video frame sequence and a position of the frosted glass region in the target video frame; [0035] a center coordinate of the target area
[0135] in the video, frame fragment includes one frame image containing the mark, or comprises a plurality of successive frame image containing the mark.
clustering consecutive target video frames in the target video according to an overlapping degree between positions of frosted glass regions, to obtain a plurality of consecutive target video clips; [0142] …then the previous frame is divided to a frame segment and the flag corresponding to the next frame to the other frame segment corresponding to the flag. of the two areas and the ratio is the intersection of the two regions and the two regions of the union, intersection over union is larger, it indicates that the overlap of these two areas is higher, the intersection over union becomes lower, the overlap of these two areas is lower. Examiner interprets union to be clustering.
[0143] The shown in FIG. 3, according to the second method to obtain the 5 frame fragment j-th frame itself constitutes one frame segment corresponding to the flag. region B1 and region B2 of intersection over union is greater than the preset threshold, the crosslinking of area B2 and area B3 and is greater than the predetermined threshold, the j-th 1 frame +, the j-th + 2 frame, + 3 j-th frame the three successive frames together form a frame segment corresponding to the mark B. area C1 and area C2 of intersection over union is more than the pre-set threshold, the area C2 and the area C3 and is less than or equal to the preset threshold, the j-th + 1 frame and j + 2 together form a frame segment and the mark C corresponding to the j-th + 2 frame and the j-th + 3 together form another C corresponding to mark of the frame segment. + 3 j-th frame itself constitutes one frame segment corresponding to the mark D. Examiner interprets union to be clustering.
[0144] when the frame segment comprises a plurality of frame pictures, any adjacent target area of the two frame picture of intersection over union are greater than the preset threshold.
Zhang fails to explicitly teach outputting respective start and stop time of the plurality of consecutive target video clips in the target video and the positions of the frosted glass regions. Townsend, in the field of video clip creation, teaches and outputting respective start and stop time of the plurality of consecutive target video clips in the target video and the positions of the frosted glass regions. [col. 13, lines 64-67] a video clip may include one or several moments associated with a region of interest (e.g., position within the video frame, object/person within the video frame, etc.)
[col. 34, lines 24-50] the rendering information may indicate an order of the selected video clip(s), the begin point and end point associated with the individual video clip(s)… a first video tag may indicate the order of the selected video clip(s), a second video tag may indicate the begin point and the end point associated with a single video clip, etc
Zhang is in the field of video processing. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang with the teachings of Townsend [col. 2, lines 29-47] to improve processing time and improve video editing.
Claim 16. Zhang teaches further comprising: acquiring a ratio of an intersection area to a union area of the frosted glass regions of the consecutive target video frames; and using the ratio as the overlapping degree between the positions of the frosted glass regions in the consecutive target video frames. [0144] if both preceding and following adjacent frames contain a certain flag and the flag occupies a region in the frame pictures of both frames
Claim 17. Zhang teaches wherein the frosted glass region detection model is trained by: performing a supervised training on a frosted glass region detection model by using an annotated training sample set to obtain an initial model; acquiring an unannotated training sample set, performing a prediction on an unannotated training sample in the unannotated training sample set and a corresponding augmented training sample by using the initial model respectively, acquiring respective prediction results, and obtaining a consistency loss based on a difference between the respective prediction results of the unannotated training sample and the corresponding augmented training sample; and performing a joint training on the initial model based on a labeled training loss of an annotated training sample and the consistency loss, to obtain a trained frosted glass region detection model. [0136] Frame-by-frame detection of a video can be performed by a method such as a deep learning algorithm, determining if a flag from the set of flags to be eliminated is contained in each frame picture of the video and determining the area occupied by the flag in the frame picture (frame-by-frame detection by deep learning algorithms necessarily requires training of the model first, i.e. Implicitly publicly leads to a trained detection model).
Claim 18. Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim 20. Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over CN 111010606 A to Zhang et al., hereinafter, “Zhang” in view of US 11158344 B1 to Townsend et al., hereinafter, “Townsend” in view of CN110852177 A to Chen et al., hereinafter, “Chen”.
Claim 5. Zhang teaches wherein the method further comprises: acquiring an annotated training sample set configured for training the frosted glass region detection model; Zhang [0136] …it can by deep learning algorithm such as the method for detecting the video, determining in each frame image of the video whether there is to be removed flag is set in flag and the determination flag area occupied in the frame picture… also can adopt other method for frame-by-frame video detection.
Zhang fails to explicitly teach determining, according to annotation data of each annotated training sample in the annotated training sample set, aspect ratios of frosted glass regions in the annotated training sample. Chen, in the field of object detection, teaches determining, according to annotation data of each annotated training sample in the annotated training sample set, aspect ratios of frosted glass regions in the annotated training sample; step S2: clustering the training set by a k-means method to obtain the aspect ratio of the obstacle target
[0112] clustering the training set by a k-means method to obtain the aspect ratio of the obstacle target;
clustering the aspect ratios of the frosted glass regions in the annotated training sample, to obtain a plurality of cluster centers; step S22: setting a first clustering center for each category of obstacle targets, calculating the maximum intersection ratio of a rectangular frame formed by each point and the first clustering center in the training set, obtaining the intersection ratio sum according to the first clustering center with the maximum intersection ratio sum, and obtaining a plurality of second clustering centers according to the intersection ratio sum;
and after the aspect ratios represented by the cluster centers are used as hyperparameters for training the frosted glass region detection model, step S32: setting a plurality of hyper-parameters of the YOLOv3 network model according to the verification set, wherein one of the hyper-parameters is the aspect ratio
performing a supervised training on the frosted glass region detection model by using the annotated training sample. step S3: training a Yolov3 network model according to the plurality of data sets and the aspect ratio of the obstacle target;
Zhang is in the field of video processing. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang with the teachings of Chen [Abstract] for real time performance and accuracy.
Claim(s) 10 and 12-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over CN 111010606 A to Zhang et al., hereinafter, “Zhang” in view of US 11158344 B1 to Townsend et al., hereinafter, “Townsend” in view of Unsupervised Data Augmentation for Consistency Training to Xie et al., hereinafter, “Xie”.
Claim 10. Zhang fails to explicitly teach acquiring an unannotated training sample set. Xie, in the field of Data Augmentation, teaches wherein the method further comprises: acquiring an unannotated training sample set, [2.2 Unsupervised Data Augmentation] utilizing unlabeled examples to enforce smoothness of the model.
Figure 1: Training objective for UDA, where M is a model that predicts a distribution of y given x.
performing a data augmentation on an unannotated training sample in the unannotated training sample set, [2.2 Unsupervised Data Augmentation]
and obtaining an unannotated sample similarity pair based on the unannotated training sample and the augmented training sample; Given an input x, compute the output distribution pθ(y | x) given x and a noised version pθ(y | x, ) by injecting a small noise .
using a frosted glass region detection model obtained by performing a supervised training by using an annotated training sample set as an initial model, Figure 1: Training objective for UDA, where M is a model that predicts a distribution of y given x.
[2.2 Unsupervised Data Augmentation]
performing a prediction on the training samples comprised in the unannotated sample similarity pair respectively by using the initial model, Figure 1: Training objective for UDA, where M is a model that predicts a distribution of y given x.
and acquiring respective prediction results of the training samples comprised in the unannotated sample similarity pair; Figure 1: Training objective for UDA, where M is a model that predicts a distribution of y given x.
obtaining a consistency loss of the unannotated sample similarity pair based on a difference between the respective prediction results of the training samples comprised in the unannotated sample similarity pair; [2.2 Unsupervised Data Augmentation] minimizing the consistency loss gradually propagates label information from labeled examples to unlabeled ones.
and obtaining a joint loss based on the consistency loss of the unannotated sample similarity pair and a labeled training loss of an annotated training sample, [2.2 Unsupervised Data Augmentation] in each iteration, we compute the supervised loss on a mini-batch of labeled examples and compute the consistency loss on a mini-batch of unlabeled data. The two losses are then summed for the final loss. We use a larger batch size for the consistency loss.
and adjusting model parameters of the initial model by using the joint loss, [2.2 Unsupervised Data Augmentation]
to obtain the trained frosted glass region detection model. [2.2 Unsupervised Data Augmentation]
Zhang is in the field of video processing. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang with the teachings of Xie [Introduction] improve deep learning training.
Claim 12. Xie teaches wherein the obtaining a joint loss based on the consistency loss of the unannotated sample similarity pair and a labeled training loss of an annotated training sample comprises: acquiring, according to a prediction result of the annotated training sample by the initial model, predicted confidence of whether a frosted glass region exists in the annotated training sample; using an annotated training sample with the predicted confidence being less than or equal to a threshold as a target training sample; and obtaining the joint loss based on the consistency loss of the unannotated sample similarity pair and a labeled training loss of the target training sample. [A.1. Training Signal Annealing for Low-Data Regime] To prevent such overfitting, the article proposes a method of Training Signal Annealing (TSA) that only targets labeled data. Overfitting is prevented by dynamically changing the threshold. The specific operation thereof is as follows: where K represents the number of training data categories, t is the total number of training steps, t is the current training step count, and when the prediction value is above a threshold value t, the loss of that sample is removed from the total loss
Claim 13. Xie teaches wherein the obtaining a consistency loss of the unannotated sample similarity pair based on a difference between the respective prediction results of the training samples comprised in the unannotated sample similarity pair comprises: sharpening the respective prediction results of the training samples comprised in the unannotated sample similarity pair, and calculating the consistency loss of the unannotated sample similarity pair according to the sharpened prediction results. [page 5 Sharpening Predictions] Unlabeled data and predicted distribution of unlabeled data would be flat when labeled data is very small. The following three sharpening schemes are proposed: Confidence Based Mask: not performing well on model predictions, labels whose predicted probability is less than a certain threshold, not computing a loss of consistency
Claim 14. Xie teaches wherein the sharpening the respective prediction results of the training samples comprised in the unannotated sample similarity pair comprises: in a case that predicted confidence in the prediction results of the training samples comprised in the unannotated sample similarity pair is greater than a threshold, keeping the unannotated sample similarity pair to participate in the calculation of the consistency loss; and when the predicted confidence in the prediction results of the training samples comprised in the unannotated sample similarity pair is less than the threshold, eliminating the unannotated sample similarity pair to keep the unannotated sample similarity pair from participating in the calculation of the consistency loss. [page 5 Sharpening Predictions]
Allowable Subject Matter
Claims 2-4 and 6-9 and 11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DELOMIA L GILLIARD whose telephone number is (571)272-1681. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DELOMIA L GILLIARD/Primary Examiner, Art Unit 2661
Prosecution Insights