CTNF 18/895,161 CTNF 87169 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. DETAILED ACTION CLAIM REJECTIONS - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to as ineligible under subject eligibility test. In the Subject Matter Eligibility Test for Products and Processes (Federal Register, Vol. 79, No. 241, dated Tuesday, December 16, 2014, page 74621) , The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional device elements, which are recited at a high level of generality, provide conventional computer functions that do not add meaningful limits to practicing the abstract idea. Claims 1 & 12 Step 1 This step inquires “is the claim to a process, article of machine, manufacture or composition of matter?” Yes, Claim 1 - “Apparatuses” are machines. Claim 12 – “Method” is a process. Step 2A - Prong 1 This step inquires “does the claim recite an abstract idea, law or natural phenomenon”. This claim appears to directed to an abstract idea. Abstract ideas fall under three categories: (1) Mathematical Concepts; (2) Certain Methods of Organizing Human Activity; (3) Mental Processes Mental Processes The limitation of “ identifying a keyframe from a received sequence of frames based on differences between features of frames of the received sequence of frames; receiving an annotation for the keyframe; identifying a first set of frames from the received sequence of frames based on differences between features of frames in the received sequence of frames and features of the keyframe; and generating annotated frames by extrapolating the annotation for the keyframe to the first set of frames. ” , as drafted, is a process that, under its broadest reasonable interpretation, covers mental processes (e.g. observation, evaluation, judgment, opinion/subjective determination . That is, other than reciting “ at least one memory; and at least one processor coupled to the at least one memory, the at least one processor ” nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the at least one memory; and at least one processor coupled to the at least one memory, the at least one processor ” language, “ identifying, receiving, generating ” in the context of this claim encompasses covers mental processes (e.g. observation, evaluation, judgment, opinion/subjective determination . If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas . Accordingly, the claim recites an abstract idea . Step 2A - Prong 2 This step inquires “does the claim recite additional elements that integrate the judicial exception into a practical application”. This judicial exception is not integrated into a practical application. In particular, the claim recites two additional element – using at least one memory; and at least one processor coupled to the at least one memory, the at least one processor ” to perform “ identifying, receiving, generating ” steps . The at least one memory; and at least one processor coupled to the at least one memory, the at least one processor ” are recited at a high-level of generality (i.e., as a generic processor) “ identifying a keyframe from a received sequence of frames based on differences between features of frames of the received sequence of frames; receiving an annotation for the keyframe; identifying a first set of frames from the received sequence of frames based on differences between features of frames in the received sequence of frames and features of the keyframe; and generating annotated frames by extrapolating the annotation for the keyframe to the first set of frames. ” such that it amounts no more than mere instructions to apply the exception using a generic computer component. STEP 2A – PRONG 2 - CONCLUSION Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. Step 2B The critical inquiry here is does the claim recite additional elements that amount to “significantly more” than the judicial exception? The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a “ at least one memory; and at least one processor coupled to the at least one memory, the at least one processor ” to perform “ identifying, receiving, generating ” steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. Dependent Claims As to claims 2 & 13 , this claim is directed to generic computer components (“generic processor”) , mental process (“detecting and extracting”) and insignificant extra-solution activity (“extracting for manual annotation is routing/outputting items for later for human review is post/extra solution activity.”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claims 3 & 14 , this claim is directed to mathematical comparison (“detecting frames for potential errors based on an intersection over union for annotations between consecutive frames.”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claims 4 & 15 , this claim is directed to generic computer components (“generic processor”) , mental process (“abstract workflow: choosing a reference frame, find similar frames, copy/propagate annotations.”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claims 5 & 16 , this claim is directed to mental process (“ wherein the annotated neutral frame is independent of the received sequence of frames. ”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claims 6 & 17 , this claim is directed to generic computer components (“generic processor”) , mental process (“ wherein the at least one processor is configured to detect a set of features for each frame of the received sequence of frames. ”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claims 7 & 18 , this claim is directed to mental process (“yes but more mathematically”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claims 8 & 19 , this claim is directed to mental process (“yes but more mathematically”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claims 9 & 20 , this claim is directed to generic computer components (“generic processor”) , mental process (“a human can identify a manually annotated frame, group similar frames, and create/review a segment”) and insignificant extra-solution activity (“generating a video segment to review annotations is extra solution activity”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claim 10 , this claim is directed to insignificant extra-solution activity (“no particular tracking architecture, training, parameters or technical improvement is claimed”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. As to claim 11 , this claim is directed to mental process (“manual annotation is a human labeling step”) and insignificant extra-solution activity (“human activity/data input feeding the abstract process, not a technical improvement”) . Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract. CLAIM REJECTIONS - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-3, 6, 11-13 & 17 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Publication 2021/0089783) in view of Goldenberg et al. (U.S. Patent 10,733,450) & Datar et al. (U.S. Publication 2021/0081603) As to claims 1 & 12 , Liu discloses a method for annotating data ([0003] discloses fast visual data annotation includes automatic detection using an automatic detector to detect subjects and joints in video frames.) , comprising: identifying a keyframe from a received sequence of frames based on differences between features of frames of the received sequence of frames ([0004] discloses receiving video content on a device and selecting samples of the video content to manually review, wherein the samples are frames selected based on a difference amount between one or more joints of a first frame and a second frame. [0022] discloses the frames are able to be selected for sampling based on the automatic detector result or a difference between frames is above or below a threshold. ) ; receiving an annotation for the keyframe ([0016] discloses the annotator is asked to manually correct the annotations of those specific subjects in the suggested frame. The corrected annotations of a subject in a frame is called a sample. [0018] discloses a user manually reviews the automatically generated annotations to verify and correct the annotations. ) ; and generating annotated frames by extrapolating the annotation for the keyframe to the first set of frames ([0004] discloses interpolating joint information for non-selected frames. [0022] discloses for the frames between the selected frames, interpolation is used as described therein.) . Liu is silent to identifying a first set of frames from the received sequence of frames based on differences between features of frames in the received sequence of frames and features of the keyframe; identifying the claimed first set of frames specifically based on differences between features of candidate frames and features of the keyframe. However, Goldenberg discloses annotation entered on a frame and tracked/extrapolated to other frames. (Abstract discloses an operator may enter an annotation to an image frame via a user interface and annotations may be updated by the operator and/or tracked subsequent image frames. Predicted locations of the annotations in subsequent image frames within each of the video files may be determined e.g. by a tracker. Column 4 Lines 29 thru 65 discloses the interior contents of each of the portions may be analyzed to recognize any colors, textures, outlines and/or other aspects of objects within the portions and that such information may then be recorded and used to search for colors, textures, outlines or aspects that corresponds to the right hand. The right hand may be tracked within the visual image frames e.g. frame to frame using any number of tracking algorithms or systems. Col. 9 Lines 19-65 discloses the set of pixels may be readily extrapolated to another visual image frame. Each of the annotations may be tracked within successive video image frames by one or more tracking algorithms or systems.) It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Liu’s disclosure to include the above limitations in order to reduce manual annotation burden while improving propagation of annotations from selected frames to other frames using known tracking extrapolation techniques. Liu in view of Goldenberg is silent to identifying a first set of frames from the received sequence of frames based on differences between features of frames in the received sequence of frames and features of the keyframe. However, Datar discloses identifying a first set of frames from the received sequence of frames based on differences between features of frames in the received sequence of frames and features of the keyframe. ([0006] discloses a mapping is created between the first instance of the media file and a second instance of the media file by comparing features of the two instances. The annotation can be indexed to the second instance using the mapping between the first and second instances. [0052] discloses the annotation server 110 maps the client instance of video 408 to a canonical instance of video 406. [0053] discloses the annotation server 110 can compute one feature set for all frames that belong to the same scene and a client can request annotations. [0054] discloses a distance function is defined over the universe of features that captures the closeness of the underlying sets of frames. For all of the frames in the candidate set, the annotation server 110 computes the distance between the feature set of the frame from the client 408 and feature set of the frame from canonical video 406. The frame from the canonical video 406 with the shortest distance is termed as the matching frame. [0055] discloses annotation server 110 may also search for features by defining a distance metric over the feature set and selecting the candidate frame with the smallest distance.) It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Liu in view of Goldenberg’s disclosure to include the above limitations in order to identify the specific frames to which a selected keyframe annotation should be propagated based on objective feature similarity distance, thereby avoiding propagation of annotations to non-corresponding or insufficiently similar frames. As to claims 2 & 13 , Liu in view of Goldenberg & Datar discloses everything as disclosed in claims 1 & 12 , In addition, Liu discloses wherein the at least one processor is configured to: detect frames with potential errors from the annotated frames; and extract the detected frames for manual annotation. ([0022] discloses the next frame position is able to be compared with linear prediction and if the difference is large then there is a bigger change in the trajectory of the joint than expected or the joint has disappeared, appeared, or was wrong. In some embodiments, when the difference is above the threshold, then a human is alerted to make a final decision. In other words, a frame with a difference above threshold is indicated/marked as a sample.) As to claims 6 & 17 , Liu in view of Goldenberg & Datar discloses everything as disclosed in claims 1 & 12 , In addition, Liu discloses wherein the at least one processor is configured to detect a set of features for each frame of the received sequence of frames. ([0020] discloses the automatic detector detects targets and joints for each target automatically.) As to claims 11 , Liu in view of Goldenberg & Datar discloses everything as disclosed in claim 1 , In addition, Liu discloses wherein the identified keyframe is manually annotated. ([0016] discloses the annotator is asked to manually correct the annotations of those specific subjects in the suggested frame. The corrected annotations of a subject in a frame is called a sample. [0018] discloses a user manually reviews the automatically generated annotations to verify and correct the annotations.) CONCLUSION No prior art has been found for claims 4-5, 7-10, 14-16 & 18-20 in their current form. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stephen P Coleman whose telephone number is (571)270-5931. The examiner can normally be reached Monday-Thursday 8AM-5PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. Stephen P. Coleman Primary Examiner Art Unit 2675 /STEPHEN P COLEMAN/Primary Examiner, Art Unit 2675 Application/Control Number: 18/895,161 Page 2 Art Unit: 2675 Application/Control Number: 18/895,161 Page 3 Art Unit: 2675 Application/Control Number: 18/895,161 Page 4 Art Unit: 2675 Application/Control Number: 18/895,161 Page 5 Art Unit: 2675 Application/Control Number: 18/895,161 Page 6 Art Unit: 2675 Application/Control Number: 18/895,161 Page 7 Art Unit: 2675 Application/Control Number: 18/895,161 Page 8 Art Unit: 2675 Application/Control Number: 18/895,161 Page 9 Art Unit: 2675 Application/Control Number: 18/895,161 Page 10 Art Unit: 2675 Application/Control Number: 18/895,161 Page 11 Art Unit: 2675 Application/Control Number: 18/895,161 Page 12 Art Unit: 2675
Prosecution Insights