CTNF 18/545,055 CTNF 94442 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Examiner’s Note For clarification and to avoid possible interpretations under 35 U.S.C. 112(f) or possible rejections under 35 USC § 101 (e.g., software per se), it is advised that in claim 19, “a memory device ” be amended to “a memory”, and “a processor device ” be amended to “a processor” or “a hardware processor”. Providing supporting paragraph(s) for each limitation of amended/new claim(s) in Remarks is strongly requested for clear and definite claim interpretations by Examiner (e.g., to avoid rejections under 35 U.S.C § 112(a) “Lack of written description”) Applicant can schedule interviews (via Automated Interview Request (AIR)) at any stage of the prosecution (e.g., Non-Final, Final, and After-Final ) to discuss any issues related to, for example, rejections under 35 U.S.C § 101 and § 102/103, for moving toward allowance. Priority Acknowledgment is made of applicant's claim for the PRO application filed on 02/09/2022 . Claim Objections Claim(s) 5 is/are objected to because of the following informalities: it appears that “the k-1 classifiers” (line 2) needs to read “the k-1 binary classifiers” or something else. Appropriate correction is required. In addition, claim(s) 14 is/are objected to for the same reason. Claim(s) 7 is/are objected to because of the following informalities: for consistency, simplicity and clarity, it appears that “wherein said identifying step searches” (line 1) needs to read “wherein the identifying searches” or something else. Appropriate correction is required. In addition, claim(s) 16 is/are objected to for the same reason. Claim(s) 8 is/are objected to because of the following informalities: for consistency, simplicity and clarity, it appears that “wherein said correcting step removes” (line 1) needs to read “wherein the correcting removes” or something else. Appropriate correction is required. In addition, claim(s) 17 is/are objected to for the same reason. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claim(s) 1-20 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim(s) 1 recite(s) the limitation “the labeled space” (line 5). There is insufficient antecedent basis for this limitation in the claim. It is not clear what it is referring to. It appears it may need to read “a labeled space”, or something else. For the purposes of examination, “a labeled space” is used. In addition, claim(s) 10, 19 is/are rejected for the same reason. The term “high” ( claim 2, line 3 ) is a relative term which renders the claim indefinite. The term “high” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. In addition, claims 11, 20 is/are rejected for the same reason. 07-34-03 The term “low” ( claim 2, line 3 ) is a relative term which renders the claim indefinite. The term “low” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. In addition, claims 11, 20 is/are rejected for the same reason. Claim(s) 2 recite(s) the limitation “from a high dimensional space above x dimensions into a low dimensional space below y dimensions, where x and y are integers, and x > y”. However, based on par 44 “We first use neural networks to transform time series segments from high dimensional to low dimensional latent space”, it is not clear why the high dimensional space is “ above x dimensions” and the low dimensional space is “ below y dimensions” since it renders a gap of 2 dimensions between the “high dimensional space” and the “low dimensional space”. It appears that it may need to read “from an x-dimensional space into a y-dimensional space, where x and y are integers, and x > y”. For the purposes of examination, “from an x-dimensional space into a y-dimensional space, where x and y are integers, and x > y” is used. In addition, claims 11, 20 is/are rejected for the same reason. Claim(s) 6 recite(s) the limitation “the nominal loss” (line 1). There is insufficient antecedent basis for this limitation in the claim. It is not clear if it means “a nominal loss” or indicates “a nominal loss” (claim 5), or something else. It appears “The computer-implemented method of claim 1” (claim 6) may need to read “The computer-implemented method of claim 5 ” or something else. For the purposes of examination, “The computer-implemented method of claim 5” is used. In addition, claim(s) 15 is/are rejected for the same reason. Claim(s) 1-2, 6, 10-11, 15, 19-20 each recite(s) limitations that raise issues of indefiniteness as set forth above, and their dependent claims are rejected at least based on their direct and/or indirect dependency from the claims listed above . Appropriate explanation and/or amendment is required. Double Patenting 08-33 AIA The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg , 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman , 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi , 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum , 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel , 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington , 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b). The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA/25, or PTO/AIA/26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp. Claim(s) 1, 3-9 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1, 3-9 of copending Application No. 18/152,238 (reference application) in view of Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Zhuang et al. (US 20180188727 A1). This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Instant application Reference 1. A computer-implemented method for ordinal prediction of [multiple objects, each having their own locations and trajectories] , comprising: encoding time series data with a temporal encoder to obtain latent space representations; optimizing the temporal encoder using semi-supervised learning to distinguish different classes in the [labeled] space using labeled data, and augment the latent space representations using unlabeled training data, to obtain semi-supervised representations; training k-1 binary classifiers on top of the semi-supervised representations to obtain k-1 binary predictions; identifying and correcting inconsistent ones of the k-1 binary predictions by matching the inconsistent ones to consistent ones of the k-1 binary predictions; and aggregating the k-1 binary predictions to obtain an ordinal prediction of [multiple objects, each having their own locations and trajectories; and navigate around the objects to avoid potential collisions there between]. 1. A computer-implemented method for ordinal prediction, comprising: encoding time series data with a temporal encoder to obtain latent space representations; optimizing the temporal encoder using semi-supervised learning to distinguish different classes in the latent space using labeled data, and augment the latent space representations using unlabeled training data, to obtain semi-supervised representations; discarding a linear layer after the temporal encoder and fixing the temporal encoder 1 ; training k-1 binary classifiers on top of the semi-supervised representations to obtain k-1 binary predictions; identifying and correcting inconsistent ones of the k-1 binary predictions by matching the inconsistent ones to consistent ones of the k-1 binary predictions; and aggregating the k-1 binary predictions to obtain an ordinal prediction 2. The computer-implemented method of claim 1, wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data [from a high dimensional space above x dimensions into a low dimensional space below y dimensions, where x and y are integers, and x > y] . 2. (Currently amended) The computer-implemented method of claim 1, wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data from a first dimensional space into a second dimensional space. 3. The computer-implemented method of claim 1, further comprising discarding a linear layer after the temporal encoder and fixing the temporal encoder 1 , wherein the linear layer is a classifier on the latent space representations. 3. (Original) The computer-implemented method of claim 1, wherein the linear layer is a classifier on the latent space representations. 4. The computer-implemented method of claim 1, wherein the temporal encoder and the linear layer are both trainable. 4. (Original) The computer-implemented method of claim 1, wherein the temporal encoder and the linear layer are both trainable. 5. The computer-implemented method of claim 1, further comprising training the k-1 classifiers using a nominal loss. 5. (Original) The computer-implemented method of claim 1, further comprising training the k-1 classifiers using a nominal loss. 6. The computer-implemented method of claim 1, wherein the nominal loss is selected from the group consisting of a softmax cross-entropy loss and a binary cross-entropy loss. 6. (Currently amended) The computer-implemented method of claim 1, wherein a nominal loss is selected from the group consisting of a softmax cross-entropy loss and a binary cross-entropy loss. 7. The computer-implemented method of claim 1, wherein said identifying step searches for sequences of ones having an unexpected zero therein and sequences of zeros having an unexpected one therein. 7. (Original) The computer-implemented method of claim 1, wherein said identifying step searches for sequences of ones having an unexpected zero therein and sequences of zeros having an unexpected one therein. 8. The computer-implemented method of claim 7, wherein said correcting step removes unexpected zeros and unexpected ones from the sequence of ones and the sequences of zeros, respectively. 8. (Original) The computer-implemented method of claim 7, wherein said correcting step removes unexpected zeros and unexpected ones from the sequence of ones and the sequences of zeros, respectively. 9. The computer-implementing method of claim 1, further comprising automatically controlling a vehicle system for collision avoidance responsive to the ordinal prediction predicting an impending collision. 9. (Original) The computer-implementing method of claim 1, further comprising automatically controlling a vehicle system for collision avoidance responsive to the ordinal prediction predicting an impending collision. * The copending application does not teach the bracketed claim languages of the instant application. The superscripts are used for indicating corresponding subject matter between the instant application and the reference application. ( Note : Hereinafter, if a limitation has bold brackets (i.e. [·] ) around claim languages, the bracketed claim languages indicate that they have not been taught yet by the current prior art reference but they will be taught by another prior art reference afterwards.) Regarding claim 1 A computer-implemented method for ordinal prediction of [multiple objects, each having their own locations and trajectories] , comprising: optimizing the temporal encoder using semi-supervised learning to distinguish different classes in the [labeled] space using labeled data, and augment the latent space representations using unlabeled training data, to obtain semi-supervised representations; aggregating the k-1 binary predictions to obtain an ordinal prediction of [multiple objects, each having their own locations and trajectories; and navigate around the objects to avoid potential collisions there between]. ( Note : Hereinafter, if a limitation has one or more bold underlines, the one or more underlined claim languages indicate that they are taught by the current prior art reference, while the one or more non-underlined claim languages indicate that they have been taught already by one or more previous art references.) Cao teaches optimizing the temporal encoder using semi-supervised learning to distinguish different classes in the labeled space using labeled data, and augment the latent space representations using unlabeled training data, to obtain semi-supervised representations; (Cao [fig(s) 2] “ResNet-34” [sec(s) 3] “Given a training dataset D = {x i , y i } N i=1 , a rank y i is first extended into K − 1 binary labels y (1) i , . . ., y (K−1) i such that y (k) i ∈ {0, 1} indicates whether y i exceeds rank r k , for instance, y (k) i = 1{y i > r k }. The indicator function 1{·} is 1 if the inner condition is true and 0 otherwise. Using the extended binary labels during model training , we train a single CNN with K − 1 binary classifiers in the output layer , which is illustrated in Fig. 2. Based on the binary task responses, the predicted rank label for an input x i is obtained via h(x i ) = r q . The rank index 1 q is given by PNG media_image1.png 137 340 media_image1.png Greyscale , (1) where f k (x i ) ∈ {0, 1} is the prediction of the kth binary classifier in the output layer. We require that {f k } K−1 k=1 reflect the ordinal information and are rank-monotonic, f 1 (x i ) ≥ f 2 (x i ) ≥ . . . ≥ f K−1 (x i ), which guarantees consistent predictions. To achieve rank-monotonicity and guarantee binary classifier consistency (Theorem 1), the K − 1 binary tasks share the same weight parameters 2 but have independent bias units (Fig. 2). … which is the weighted cross-entropy of K − 1 binary classifiers. For rank prediction (Eq. (1)), the binary labels are obtained via PNG media_image2.png 86 750 media_image2.png Greyscale (5)” [sec(s) 4.2] “To evaluate the performance of CORAL for age estimation from face images, we chose the ResNet-34 architecture [9], which is a modern CNN architecture that achieves good performance on a variety of image classification tasks [8]. For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN.” ; ) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of the copending application with the labeled space of Cao. Doing so would lead to determining a criterion to split at a node to substantially improve the predictive performance of CNNs for age estimation on three independent age estimation datasets. (Cao [sec 6] “The experimental results showed that the CORAL framework substantially improved the predictive performance of CNNs for age estimation on three independent age estimation datasets. Our method can be readily generalized to other ordinal regression problems and different types of neural network architectures, including multilayer perceptrons and recurrent neural networks.”) However, the combination of the copending application, Cao does not appear to explicitly teach: A computer-implemented method for ordinal prediction of [multiple objects, each having their own locations and trajectories] , comprising: aggregating the k-1 binary predictions to obtain an ordinal prediction of [multiple objects, each having their own locations and trajectories; and navigate around the objects to avoid potential collisions there between]. Zhuang teaches A computer-implemented method for ordinal prediction of multiple objects, each having their own locations and trajectories , comprising: (Zhuang [par(s) 60] “In one embodiment, a motion plane would consider the vehicle chassis information (e.g., speed, acceleration/deceleration, heading, location, etc.), perception information (e.g., obstacles' location, speed, heading and its motion trajectory ), detection information (e.g., dependency module's status) and its history information. The history information can be a history list of planned trajectory and its corresponding states. The output of a motion plan would be a vector of points, which include not only the planned positions, but also its speed, heading and station.” [par(s) 32] “The perception information may represent what an ordinary driver would perceive surrounding a vehicle in which the driver is driving. The perception can include the lane configuration (e.g., straight or curve lanes), traffic light signals, a relative position of another vehicle, a pedestrian, a building, crosswalk, or other traffic related signs (e.g., stop signs, yield signs), etc., for example, in a form of an object .” ; ) aggregating the k-1 binary predictions to obtain an ordinal prediction of multiple objects, each having their own locations and trajectories ; and (Zhuang [par(s) 60] “In one embodiment, a motion plane would consider the vehicle chassis information (e.g., speed, acceleration/deceleration, heading, location, etc.), perception information (e.g., obstacles' location, speed, heading and its motion trajectory ), detection information (e.g., dependency module's status) and its history information. The history information can be a history list of planned trajectory and its corresponding states. The output of a motion plan would be a vector of points, which include not only the planned positions, but also its speed, heading and station.” [par(s) 32] “The perception information may represent what an ordinary driver would perceive surrounding a vehicle in which the driver is driving. The perception can include the lane configuration (e.g., straight or curve lanes), traffic light signals, a relative position of another vehicle, a pedestrian, a building, crosswalk, or other traffic related signs (e.g., stop signs, yield signs), etc., for example, in a form of an object .” ; ) navigate around the objects to avoid potential collisions there between. (Zhuang [par(s) 38] “Decision module 303/planning module 304 may further include a collision avoidance system or functionalities of a collision avoidance system to identify, evaluate, and avoid or otherwise negotiate potential obstacles in the environment of the autonomous vehicle . For example, the collision avoidance system may effect changes in the navigation of the autonomous vehicle by operating one or more subsystems in control system 111 to undertake swerving maneuvers, turning maneuvers, braking maneuvers, etc. The collision avoidance system may automatically determine feasible obstacle avoidance maneuvers on the basis of surrounding traffic patterns, road conditions, etc. The collision avoidance system may be configured such that a swerving maneuver is not undertaken when other sensor systems detect vehicles, construction barriers, etc. in the region adjacent the autonomous vehicle that would be swerved into. The collision avoidance system may automatically select the maneuver that is both available and maximizes safety of occupants of the autonomous vehicle. The collision avoidance system may select an avoidance maneuver predicted to cause the least amount of acceleration in a passenger cabin of the autonomous vehicle.” [par(s) 27] “Based on the real-time traffic information, MPOI information, and location information, as well as real-time local environment data detected or sensed by sensor system 115 (e.g., obstacles, objects, nearby vehicles ), perception and planning system 110 can plan an optimal route and drive vehicle 101, for example, via control system 111, according to the planned route to reach the specified destination safely and efficiently .” [par(s) 19] “Autonomous vehicle 101 may further include certain common components included in ordinary vehicles, such as, an engine, wheels, steering wheel, transmission, etc., which may be controlled by vehicle control system 111 and/or perception and planning system 110 using a variety of communication signals and/or commands, such as, for example, acceleration signals or commands, deceleration signals or commands, steering signals or commands, braking signals or commands, etc.” ; ) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of the copending application, Cao with the multiple object prediction of Zhuang . One of ordinary skill in the art would have been motived to combine in order to be accurate and smooth under different circumstances by applying different motion planning and control operations to different types of vehicles. (Zhuang [par(s) 2-3] “Motion planning and control are critical operations in autonomous driving. However, conventional motion planning operations estimate the difficulty of completing a given path mainly from its curvature and speed, without considering the differences in features for different types of vehicles. Same motion planning and control is applied to all types of vehicles, which may not be accurate and smooth under some circumstances.”) Similarly, Claims 10, 12-18 are rejected on the ground of provisional nonstatutory double patenting, mutatis mutandis, as being unpatentable over claim(s) 10, 12-18 of copending Application No. 18/152,238 (reference application) in view of Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Zhuang et al. (US 20180188727 A1). Similarly, Claims 19 are rejected on the ground of provisional nonstatutory double patenting, mutatis mutandis, as being unpatentable over claim(s) 19 of copending Application No. 18/152,238 (reference application) in view of Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Zhuang et al. (US 20180188727 A1). Claim(s) 2 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 2 of copending Application No. 18/152,238 (reference application) in view of Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Zhuang et al. (US 20180188727 A1) further in view of Liu et al. (Deep Learning in Latent Space for Video Prediction and Compression). This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Instant application Reference 2. The computer-implemented method of claim 1, wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data [from a high dimensional space above x dimensions into a low dimensional space below y dimensions, where x and y are integers, and x > y] . 2. (Currently amended) The computer-implemented method of claim 1, wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data from a first dimensional space into a second dimensional space. * The copending application does not teach the bracketed claim languages of the instant application. Regarding claim 2 wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data [from a high dimensional space above x dimensions into a low dimensional space below y dimensions, where x and y are integers, and x > y] . Liu teaches wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data from a high dimensional space above x dimensions into a low dimensional space below y dimensions, where x and y are integers, and x > y . (Liu [sec(s) Abs] “We propose a novel DNN based framework that predicts and compresses video sequences in the latent vector space. The proposed method first learns the efficient lower-dimensional latent space representation of each video frame and then performs inter-frame prediction in that latent domain” [sec(s) 1] “• Learning based video prediction in latent domain: We use an convolutional long short-term memory (ConvLSTM) network to predict a compact latent representation of the next frame substituting for motion compensation in conventional codecs . This approach only stores the differences between the predicted and actual representation in low dimensional latent space, resulting in entropy reduction of the residuals. The predictor is adversarially trained against a discriminator which significantly enhances the quality of prediction to bring down the entropy (i.e., density and magnitude of nonzero elements) of residuals.” [sec(s) 3] “To further reduce the video code size, we encode the residual with quantization and entropy coding. A desired compression rate is controlled by the size of latent dimension in the image compression stage as well as the number of quantization levels used in residual encoding.” ; ) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of the copending application, Cao, Zhuang with the latent space dimension of Liu . One of ordinary skill in the art would have been motived to combine in order to achieve significant quality improvements and provide reliable and accurate prediction for compression for videos. (Liu [sec(s) 4] “In the video compression task, we alleviate this issue and achieve significant quality improvements by saving and transmitting key-frames and residuals. For the majority of normal video frame sequences that have strong temporal correlations, the proposed prediction method provides reliable and accurate prediction for compression.”) Similarly, Claims 11 are rejected on the ground of provisional nonstatutory double patenting, mutatis mutandis, as being unpatentable over claim(s) 11 of copending Application No. 18/152,238 (reference application) in view of Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Zhuang et al. (US 20180188727 A1) in view of Liu et al. (Deep Learning in Latent Space for Video Prediction and Compression). Similarly, Claims 20 are rejected on the ground of provisional nonstatutory double patenting, mutatis mutandis, as being unpatentable over claim(s) 20 of copending Application No. 18/152,238 (reference application) in view of Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Zhuang et al. (US 20180188727 A1) in view of Liu et al. (Deep Learning in Latent Space for Video Prediction and Compression). Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 1, 3-8, 10, 12-17, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Kim et al. (Instance-Level Future Motion Estimation in a Single Image Based on Ordinal Regression and Semi-Supervised Domain Adaptation) in view of Zhuang et al. (US 20180188727 A1) Regarding claim 1 ( Note : Hereinafter, if a limitation has bold brackets (i.e. [·] ) around claim languages, the bracketed claim languages indicate that they have not been taught yet by the current prior art reference but they will be taught by another prior art reference afterwards.) Cao teaches A computer-implemented method for ordinal prediction of [multiple objects, each having their own locations and trajectories] , comprising: encoding time series data with a temporal encoder to obtain latent space representations; (Cao [fig(s) 2] [sec(s) 1] “Aging can be regarded as a non-stationary process since age progression effects appear differently depending on the person’s age. During childhood, facial aging is primarily associated with changes in the shape of the face, whereas aging during adulthood is defined mainly by changes in skin texture [16,20]. Based on this assumption, age prediction can be modeled using ordinal regression-based approaches [2,3,13,29].” [sec(s) 4.1] “The MORPH-2 dataset [24], containing 55,608 face images, was downloaded from https://www.faceaginggroup.com/morph/ and preprocessed by locating the average eye-position in the respective dataset using facial landmark detection [26] and then aligning each image in the dataset to the average eye position using EyepadAlign function in MLxtend v0.14 [22]. The faces were then re-aligned such that the tip of the nose was located in the center of each image. The age labels used in this study were in the range of 16–70 years ” [sec(s) 4.2] “To evaluate the performance of CORAL for age estimation from face images, we chose the ResNet-34 architecture [9], which is a modern CNN architecture that achieves good performance on a variety of image classification tasks [8]. For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN. To implement a ResNet-34 CNN for ordinal regression using the proposed CORAL method, we replaced the last output layer with the corresponding binary tasks (Fig. 2) and refer to this implementation as CORAL-CNN. Similar to CORAL-CNN, we modified the output layer of ResNet-34 to implement the ordinal regression reference approach described in Niu et al. [16]; we refer to this architecture as OR-CNN” ; e.g., feature representations based on ResNet-34 read(s) on “latent space representations”. ) optimizing the temporal encoder using [semi] -supervised learning to distinguish different classes in the labeled space using labeled data, and augment the latent space representations using [un] labeled training data, to obtain [semi] -supervised representations; (Cao [fig(s) 2] “ResNet-34” [sec(s) 3] “Given a training dataset D = {x i , y i } N i=1 , a rank y i is first extended into K − 1 binary labels y (1) i , . . ., y (K−1) i such that y (k) i ∈ {0, 1} indicates whether y i exceeds rank r k , for instance, y (k) i = 1{y i > r k }. The indicator function 1{·} is 1 if the inner condition is true and 0 otherwise. Using the extended binary labels during model training , we train a single CNN with K − 1 binary classifiers in the output layer , which is illustrated in Fig. 2. Based on the binary task responses, the predicted rank label for an input x i is obtained via h(x i ) = r q . The rank index 1 q is given by PNG media_image1.png 137 340 media_image1.png Greyscale , (1) where f k (x i ) ∈ {0, 1} is the prediction of the kth binary classifier in the output layer. We require that {f k } K−1 k=1 reflect the ordinal information and are rank-monotonic, f 1 (x i ) ≥ f 2 (x i ) ≥ . . . ≥ f K−1 (x i ), which guarantees consistent predictions. To achieve rank-monotonicity and guarantee binary classifier consistency (Theorem 1), the K − 1 binary tasks share the same weight parameters 2 but have independent bias units (Fig. 2). … which is the weighted cross-entropy of K − 1 binary classifiers. For rank prediction (Eq. (1)), the binary labels are obtained via PNG media_image2.png 86 750 media_image2.png Greyscale (5)” [sec(s) 4.2] “To evaluate the performance of CORAL for age estimation from face images, we chose the ResNet-34 architecture [9], which is a modern CNN architecture that achieves good performance on a variety of image classification tasks [8]. For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN.” ; ) training k-1 binary classifiers on top of the [semi] -supervised representations to obtain k-1 binary predictions; (Cao [sec(s) 3] “Given a training dataset D = {x i , y i } N i=1 , a rank y i is first extended into K − 1 binary labels y (1) i , . . ., y (K−1) i such that y (k) i ∈ {0, 1} indicates whether y i exceeds rank r k , for instance, y (k) i = 1{y i > r k }. The indicator function 1{·} is 1 if the inner condition is true and 0 otherwise. Using the extended binary labels during model training, we train a single CNN with K − 1 binary classifiers in the output layer , which is illustrated in Fig. 2.” ; ) identifying and correcting inconsistent ones of the k-1 binary predictions by matching the inconsistent ones to consistent ones of the k-1 binary predictions; and (Cao [fig(s) 1] “a rank-inconsistent model (left) versus a rank-consistent model where the probabilities decrease consistently (right).” [fig(s) 2] [fig(s) 3] “OR-CNN”, “CORAL-CNN” [sec(s) 1] “ This inconsistency problem among the predictions of individual binary classifiers is illustrated in Fig. 1 . We propose a new method and theorem for guaranteed classifier consistency that can easily be implemented in various neural network architectures.” [sec(s) 2] “ Niu et al. [16] acknowledged the classifier inconsistency as not being ideal and also noted that ensuring the K − 1 binary classifiers are consistent would increase the training complexity substantially [16]. The CORAL method proposed in this paper addresses both these issues with a theoretical guarantee for classifier consistency and without increasing the training complexity.” [sec(s) 3.2] “Given a training dataset D = {x i , y i } N i=1 , a rank y i is first extended into K − 1 binary labels y (1) i , . . ., y (K−1) i such that y (k) i ∈ {0, 1} indicates whether y i exceeds rank r k , for instance, y (k) i = 1{y i > r k }. The indicator function 1{·} is 1 if the inner condition is true and 0 otherwise . Using the extended binary labels during model training, we train a single CNN with K − 1 binary classifiers in the output layer, which is illustrated in Fig. 2. Based on the binary task responses, the predicted rank label for an input x i is obtained via h(x i ) = r q . The rank index 1 q is given by PNG media_image3.png 177 449 media_image3.png Greyscale (1) where f k (x i ) ∈ {0, 1} is the prediction of the kth binary classifier in the output layer. We require that {f k } K−1 k=1 reflect the ordinal information and are rank-monotonic, f 1 (x i ) ≥ f 2 (x i ) ≥ . . . ≥ f K−1 (x i ), which guarantees consistent predictions . To achieve rank-monotonicity and guarantee binary classifier consistency (Theorem 1), the K − 1 binary tasks share the same weight parameters 2 but have independent bias units (Fig. 2).” ; ) aggregating the k-1 binary predictions to obtain an ordinal prediction of [multiple objects, each having their own locations and trajectories] ; and (Cao [sec(s) 3] “This section describes our proposed CORAL framework that addresses the problem of classifier inconsistency in the OR-CNN by Niu et al. [16], which is based on multiple binary classification tasks for ranking. … Based on the binary task responses, the predicted rank label for an input x i is obtained via h(x i ) = r q . The rank index 1 q is given by PNG media_image1.png 137 340 media_image1.png Greyscale , (1) where f k (x i ) ∈ {0, 1} is the prediction of the kth binary classifier in the output layer. We require that {f k } K−1 k=1 reflect the ordinal information and are rank-monotonic, f 1 (x i ) ≥ f 2 (x i ) ≥ . . . ≥ f K−1 (x i ), which guarantees consistent predictions. To achieve rank-monotonicity and guarantee binary classifier consistency (Theorem 1), the K − 1 binary tasks share the same weight parameters 2 but have independent bias units (Fig. 2). … which is the weighted cross-entropy of K − 1 binary classifiers. For rank prediction (Eq. (1)), the binary labels are obtained via PNG media_image2.png 86 750 media_image2.png Greyscale (5)” [sec(s) 1] “Note that this work focuses on age estimation to study the proposed method’s efficacy for ordinal regression. However, the proposed technique can be used for other ordinal regression problems, such as crowd-counting, depth estimation, biological cell counting, customer satisfaction , and others.” [sec(s) 2.1] “Moreover, the machine learning field developed ordinal regression models based on extensions of well-studied classification algorithms, by reformulating the problem to utilize multiple binary classification tasks ” ; ) However, Cao does not appear to explicitly teach: A computer-implemented method for ordinal prediction of [multiple objects, each having their own locations and trajectories] , comprising: optimizing the temporal encoder using [semi] -supervised learning to distinguish different classes in the labeled space using labeled data, and augment the latent space representations using [un] labeled training data, to obtain [semi] -supervised representations; training k-1 binary classifiers on top of the [semi] -supervised representations to obtain k-1 binary predictions; aggregating the k-1 binary predictions to obtain an ordinal prediction of [multiple objects, each having their own locations and trajectories] ; and navigate around the objects to avoid potential collisions there between. ( Note : Hereinafter, if a limitation has one or more bold underlines, the one or more underlined claim languages indicate that they are taught by the current prior art reference, while the one or more non-underlined claim languages indicate that they have been taught already by one or more previous art references.) Kim teaches optimizing the temporal encoder using semi -supervised learning to distinguish different classes in the labeled space using labeled data, and augment the latent space representations using un labeled training data, to obtain semi -supervised representations; (Kim [fig(s) 6] “ Labeled input”, “ Unlabeled input” and “The architecture of the proposed FM-Net in the semi-supervised domain adaptation setting.” [sec(s) IV] “Let x be an instance and y x ∈ C be its class. For COR, binary classifiers, f 0 , f 1 , . . . , f K/2−1 , are used. Each binary classifier f n is defined as PNG media_image4.png 204 1216 media_image4.png Greyscale (2) where (n)K denotes the modulo operator returning the remainder after the division of n by K. … Note that, in the linear ordinal regression [42], the classes in a line segment is divided into two parts. Therefore, for K-way classification, K −1 binary classifiers are required. In contrast, in the proposed COR, a circle is halved into two semicircles, as done in [45]. Consequently, only K/2 binary classifiers are needed.” [sec(s) V] “When a source domain for training and a target domain for the FM inference are different, FM-Net may fail to estimate the FMs of test instances in the target domain accurately. In this work, we attempt to improve the generalization performance of FM-Net by adapting it to a new target domain in a semi-supervised manner. More specifically, FM-Net is trained in the semi-supervised domain adaptation setting, in which a sufficient number of labeled data are available in the source domain, a limited number of labeled data are in the target domain, and a large number of unlabeled data are in the target domain ” ; ) training k-1 binary classifiers on top of the semi -supervised representations to obtain k-1 binary predictions; (Kim [fig(s) 6] “ Labeled input”, “ Unlabeled input” and “The architecture of the proposed FM-Net in the semi-supervised domain adaptation setting.” [sec(s) IV] “Let x be an instance and y x ∈ C be its class. For COR, binary classifiers, f 0 , f 1 , . . . , f K/2−1 , are used. Each binary classifier f n is defined as PNG media_image4.png 204 1216 media_image4.png Greyscale (2) where (n)K denotes the modulo operator returning the remainder after the division of n by K. … Note that, in the linear ordinal regression [42], the classes in a line segment is divided into two parts. Therefore, for K-way classification, K −1 binary classifiers are required. In contrast, in the proposed COR, a circle is halved into two semicircles, as done in [45]. Consequently, only K/2 binary classifiers are needed .” [sec(s) V] “When a source domain for training and a target domain for the FM inference are different, FM-Net may fail to estimate the FMs of test instances in the target domain accurately. In this work, we attempt to improve the generalization performance of FM-Net by adapting it to a new target domain in a semi-supervised manner. More specifically, FM-Net is trained in the semi-supervised domain adaptation setting, in which a sufficient number of labeled data are available in the source domain, a limited number of labeled data are in the target domain, and a large number of unlabeled data are in the target domain ” ; ) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Cao with the semi-supervised representations of Kim . One of ordinary skill in the art would have been motived to combine in order to yield remarkable future motion estimation results despite variations in camera viewpoints and capturing environments, and improve future motion estimation accuracies. (Kim [sec(s) I] “Experimental results demonstrate that the proposed FM-Net yields remarkable FM estimation results for pedestrian, car, and animal instances despite variations in camera viewpoints and capturing environments, when a sufficient number of labeled training data are provided. Moreover, it is demonstrated that the proposed semi-supervised domain adaptation learning improves FM estimation accuracies, when only a limited number of labeled data for a new domain are available.”) However, the combination of Cao, Kim does not appear to explicitly teach: A computer-implemented method for ordinal prediction of [multiple objects, each having their own locations and trajectories] , comprising: aggregating the k-1 binary predictions to obtain an ordinal prediction of [multiple objects, each having their own locations and trajectories] ; and navigate around the objects to avoid potential collisions there between. Zhuang teaches A computer-implemented method for ordinal prediction of multiple objects, each having their own locations and trajectories , comprising: (Zhuang [par(s) 60] “In one embodiment, a motion plane would consider the vehicle chassis information (e.g., speed, acceleration/deceleration, heading, location, etc.), perception information (e.g., obstacles' location, speed, heading and its motion trajectory ), detection information (e.g., dependency module's status) and its history information. The history information can be a history list of planned trajectory and its corresponding states. The output of a motion plan would be a vector of points, which include not only the planned positions, but also its speed, heading and station.” [par(s) 32] “The perception information may represent what an ordinary driver would perceive surrounding a vehicle in which the driver is driving. The perception can include the lane configuration (e.g., straight or curve lanes), traffic light signals, a relative position of another vehicle, a pedestrian, a building, crosswalk, or other traffic related signs (e.g., stop signs, yield signs), etc., for example, in a form of an object .” ; ) aggregating the k-1 binary predictions to obtain an ordinal prediction of multiple objects, each having their own locations and trajectories ; and (Zhuang [par(s) 60] “In one embodiment, a motion plane would consider the vehicle chassis information (e.g., speed, acceleration/deceleration, heading, location, etc.), perception information (e.g., obstacles' location, speed, heading and its motion trajectory ), detection information (e.g., dependency module's status) and its history information. The history information can be a history list of planned trajectory and its corresponding states. The output of a motion plan would be a vector of points, which include not only the planned positions, but also its speed, heading and station.” [par(s) 32] “The perception information may represent what an ordinary driver would perceive surrounding a vehicle in which the driver is driving. The perception can include the lane configuration (e.g., straight or curve lanes), traffic light signals, a relative position of another vehicle, a pedestrian, a building, crosswalk, or other traffic related signs (e.g., stop signs, yield signs), etc., for example, in a form of an object .” ; ) navigate around the objects to avoid potential collisions there between. (Zhuang [par(s) 38] “Decision module 303/planning module 304 may further include a collision avoidance system or functionalities of a collision avoidance system to identify, evaluate, and avoid or otherwise negotiate potential obstacles in the environment of the autonomous vehicle . For example, the collision avoidance system may effect changes in the navigation of the autonomous vehicle by operating one or more subsystems in control system 111 to undertake swerving maneuvers, turning maneuvers, braking maneuvers, etc. The collision avoidance system may automatically determine feasible obstacle avoidance maneuvers on the basis of surrounding traffic patterns, road conditions, etc. The collision avoidance system may be configured such that a swerving maneuver is not undertaken when other sensor systems detect vehicles, construction barriers, etc. in the region adjacent the autonomous vehicle that would be swerved into. The collision avoidance system may automatically select the maneuver that is both available and maximizes safety of occupants of the autonomous vehicle. The collision avoidance system may select an avoidance maneuver predicted to cause the least amount of acceleration in a passenger cabin of the autonomous vehicle.” [par(s) 27] “Based on the real-time traffic information, MPOI information, and location information, as well as real-time local environment data detected or sensed by sensor system 115 (e.g., obstacles, objects, nearby vehicles ), perception and planning system 110 can plan an optimal route and drive vehicle 101, for example, via control system 111, according to the planned route to reach the specified destination safely and efficiently .” [par(s) 19] “Autonomous vehicle 101 may further include certain common components included in ordinary vehicles, such as, an engine, wheels, steering wheel, transmission, etc., which may be controlled by vehicle control system 111 and/or perception and planning system 110 using a variety of communication signals and/or commands, such as, for example, acceleration signals or commands, deceleration signals or commands, steering signals or commands, braking signals or commands, etc.” ; ) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Cao, Kim with the multiple object prediction of Zhuang . One of ordinary skill in the art would have been motived to combine in order to be accurate and smooth under different circumstances by applying different motion planning and control operations to different types of vehicles. (Zhuang [par(s) 2-3] “Motion planning and control are critical operations in autonomous driving. However, conventional motion planning operations estimate the difficulty of completing a given path mainly from its curvature and speed, without considering the differences in features for different types of vehicles. Same motion planning and control is applied to all types of vehicles, which may not be accurate and smooth under some circumstances.”) Regarding claim 3 The combination of Cao, Kim, Zhuang teaches claim 1. Cao further teaches discarding a linear layer after the temporal encoder and fixing the temporal encoder, (Cao [fig(s) 2] [sec(s) 4.2] “To evaluate the performance of CORAL for age estimation from face images, we chose the ResNet-34 architecture [9], which is a modern CNN architecture that achieves good performance on a variety of image classification tasks [8]. For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN. To implement a ResNet-34 CNN for ordinal regression using the proposed CORAL method, we replaced the last output layer with the corresponding binary tasks (Fig. 2) and refer to this implementation as CORAL-CNN. Similar to CORAL-CNN, we modified the output layer of ResNet-34 to implement the ordinal regression reference approach described in Niu et al. [16]; we refer to this architecture as OR-CNN” ; Note that the last output layer is a linear layer. ) wherein the linear layer is a classifier on the latent space representations. (Cao [fig(s) 2] [sec(s) 4.1] “The MORPH-2 dataset [24], containing 55,608 face images, was downloaded from https://www.faceaginggroup.com/morph/ and preprocessed by locating the average eye-position in the respective dataset using facial landmark detection [26] and then aligning each image in the dataset to the average eye position using EyepadAlign function in MLxtend v0.14 [22]. The faces were then re-aligned such that the tip of the nose was located in the center of each image. The age labels used in this study were in the range of 16–70 years ” [sec(s) 4.2] “To evaluate the performance of CORAL for age estimation from face images, we chose the ResNet-34 architecture [9], which is a modern CNN architecture that achieves good performance on a variety of image classification tasks [8]. For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN. To implement a ResNet-34 CNN for ordinal regression using the proposed CORAL method, we replaced the last output layer with the corresponding binary tasks (Fig. 2) and refer to this implementation as CORAL-CNN. Similar to CORAL-CNN, we modified the output layer of ResNet-34 to implement the ordinal regression reference approach described in Niu et al. [16]; we refer to this architecture as OR-CNN” ; e.g., feature representations based on ResNet-34 read(s) on “latent space representations”. Note that the last output layer of ResNet-34 is a linear layer. ) Regarding claim 4 The combination of Cao, Kim, Zhuang teaches claim 1. Cao further teaches wherein the temporal encoder and the linear layer are both trainable. (Cao [sec(s) 3] “Given a training dataset D = {x i , y i } N i=1 , a rank y i is first extended into K − 1 binary labels y (1) i , . . ., y (K−1) i such that y (k) i ∈ {0, 1} indicates whether y i exceeds rank r k , for instance, y (k) i = 1{y i > r k }. The indicator function 1{·} is 1 if the inner condition is true and 0 otherwise. Using the extended binary labels during model training , we train a single CNN with K − 1 binary classifiers in the output layer , which is illustrated in Fig. 2. Based on the binary task responses, the predicted rank label for an input x i is obtained via h(x i ) = r q . The rank index 1 q is given by PNG media_image1.png 137 340 media_image1.png Greyscale , (1) where f k (x i ) ∈ {0, 1} is the prediction of the kth binary classifier in the output layer. We require that {f k } K−1 k=1 reflect the ordinal information and are rank-monotonic, f 1 (x i ) ≥ f 2 (x i ) ≥ . . . ≥ f K−1 (x i ), which guarantees consistent predictions. To achieve rank-monotonicity and guarantee binary classifier consistency (Theorem 1), the K − 1 binary tasks share the same weight parameters 2 but have independent bias units (Fig. 2).” [sec(s) 4.2] “To evaluate the performance of CORAL for age estimation from face images, we chose the ResNet-34 architecture [9], which is a modern CNN architecture that achieves good performance on a variety of image classification tasks [8]. For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN. To implement a ResNet-34 CNN for ordinal regression using the proposed CORAL method, we replaced the last output layer with the corresponding binary tasks (Fig. 2) and refer to this implementation as CORAL-CNN. Similar to CORAL-CNN, we modified the output layer of ResNet-34 to implement the ordinal regression reference approach described in Niu et al. [16]; we refer to this architecture as OR-CNN” ; Note that the last output layer of ResNet-34 is a linear layer. ) Regarding claim 5 The combination of Cao, Kim, Zhuang teaches claim 1. Cao further teaches further comprising training the k-1 classifiers using a nominal loss. (Cao [sec(s) 3] “For model training , we minimize the loss function PNG media_image5.png 288 1280 media_image5.png Greyscale (4) which is the weighted cross-entropy of K − 1 binary classifiers . For rank prediction (Eq. (1)), the binary labels are obtained via PNG media_image6.png 84 748 media_image6.png Greyscale . (5) In Eq. (4), λ (k) denotes the weight of the loss associated with the kth classifier (assuming λ (k) > 0). In the remainder of the paper, we refer to λ (k) as the importance parameter for task k. Some tasks may be less robust or harder to optimize, which can be considered by choosing a non-uniform task weighting scheme.” [sec(s) 4] “For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN.” [sec(s) 5] “We conducted a series of experiments on three independent face image datasets for age estimation (Section 4.1) to compare the proposed CORAL method (CORAL-CNN) with the ordinal regression approach proposed by Niu et al. [16] (OR-CNN). All implementations were based on the ResNet-34 architecture, as described in Section 4.2. We include the standard ResNet-34 classification network with cross-entropy loss (CE-CNN) as a performance baseline.” ; ) Regarding claim 6 The combination of Cao, Kim, Zhuang teaches claim 1. Cao further teaches wherein a nominal loss is selected from the group consisting of a softmax cross-entropy loss and a binary cross-entropy loss. (Cao [sec(s) 3] “For model training , we minimize the loss function PNG media_image5.png 288 1280 media_image5.png Greyscale (4) which is the weighted cross-entropy of K − 1 binary classifiers . For rank prediction (Eq. (1)), the binary labels are obtained via PNG media_image6.png 84 748 media_image6.png Greyscale . (5) In Eq. (4), λ (k) denotes the weight of the loss associated with the kth classifier (assuming λ (k) > 0). In the remainder of the paper, we refer to λ (k) as the importance parameter for task k. Some tasks may be less robust or harder to optimize, which can be considered by choosing a non-uniform task weighting scheme.” [sec(s) 4] “For the remainder of this paper, we refer to the original ResNet-34 CNN with standard cross-entropy loss as CE-CNN.” [sec(s) 5] “We conducted a series of experiments on three independent face image datasets for age estimation (Section 4.1) to compare the proposed CORAL method (CORAL-CNN) with the ordinal regression approach proposed by Niu et al. [16] (OR-CNN). All implementations were based on the ResNet-34 architecture, as described in Section 4.2. We include the standard ResNet-34 classification network with cross-entropy loss (CE-CNN) as a performance baseline.” ; For more details on a binary cross-entropy loss, please refer to Ho et al. (The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling) - [sec(s) III] ) Regarding claim 7 The combination of Cao, Kim, Zhuang teaches claim 1. Cao further teaches wherein said identifying step searches for sequences of ones having an unexpected zero therein and sequences of zeros having an unexpected one therein. (Cao [fig(s) 1] “a rank-inconsistent model (left) versus a rank-consistent model where the probabilities decrease consistently (right).” [fig(s) 2] [sec(s) 1] “ This inconsistency problem among the predictions of individual binary classifiers is illustrated in Fig. 1 . We propose a new method and theorem for guaranteed classifier consistency that can easily be implemented in various neural network architectures.” [sec(s) 2] “ Niu et al. [16] acknowledged the classifier inconsistency as not being ideal and also noted that ensuring the K − 1 binary classifiers are consistent would increase the training complexity substantially [16]. The CORAL method proposed in this paper addresses both these issues with a theoretical guarantee for classifier consistency and without increasing the training complexity.” [sec(s) 3.2] “Given a training dataset D = {x i , y i } N i=1 , a rank y i is first extended into K − 1 binary labels y (1) i , . . ., y (K−1) i such that y (k) i ∈ {0, 1} indicates whether y i exceeds rank r k , for instance, y (k) i = 1{y i > r k }. The indicator function 1{·} is 1 if the inner condition is true and 0 otherwise . Using the extended binary labels during model training, we train a single CNN with K − 1 binary classifiers in the output layer, which is illustrated in Fig. 2. Based on the binary task responses, the predicted rank label for an input x i is obtained via h(x i ) = r q . The rank index 1 q is given by PNG media_image3.png 177 449 media_image3.png Greyscale (1) where f k (x i ) ∈ {0, 1} is the prediction of the kth binary classifier in the output layer. We require that {f k } K−1 k=1 reflect the ordinal information and are rank-monotonic, f 1 (x i ) ≥ f 2 (x i ) ≥ . . . ≥ f K−1 (x i ), which guarantees consistent predictions . To achieve rank-monotonicity and guarantee binary classifier consistency (Theorem 1), the K − 1 binary tasks share the same weight parameters 2 but have independent bias units (Fig. 2).” ; ) Regarding claim 8 The combination of Cao, Kim, Zhuang teaches claim 7. Cao further teaches wherein said correcting step removes unexpected zeros and unexpected ones from the sequence of ones and the sequences of zeros, respectively. (Cao [fig(s) 1] “a rank-inconsistent model (left) versus a rank-consistent model where the probabilities decrease consistently (right).” [fig(s) 2] [fig(s) 3] “OR-CNN”, “CORAL-CNN” [sec(s) 1] “ This inconsistency problem among the predictions of individual binary classifiers is illustrated in Fig. 1 . We propose a new method and theorem for guaranteed classifier consistency that can easily be implemented in various neural network architectures.” [sec(s) 2] “ Niu et al. [16] acknowledged the classifier inconsistency as not being ideal and also noted that ensuring the K − 1 binary classifiers are consistent would increase the training complexity substantially [16]. The CORAL method proposed in this paper addresses both these issues with a theoretical guarantee for classifier consistency and without increasing the training complexity.” [sec(s) 3.2] “Given a training dataset D = {x i , y i } N i=1 , a rank y i is first extended into K − 1 binary labels y (1) i , . . ., y (K−1) i such that y (k) i ∈ {0, 1} indicates whether y i exceeds rank r k , for instance, y (k) i = 1{y i > r k }. The indicator function 1{·} is 1 if the inner condition is true and 0 otherwise . Using the extended binary labels during model training, we train a single CNN with K − 1 binary classifiers in the output layer, which is illustrated in Fig. 2. Based on the binary task responses, the predicted rank label for an input x i is obtained via h(x i ) = r q . The rank index 1 q is given by PNG media_image3.png 177 449 media_image3.png Greyscale (1) where f k (x i ) ∈ {0, 1} is the prediction of the kth binary classifier in the output layer. We require that {f k } K−1 k=1 reflect the ordinal information and are rank-monotonic, f 1 (x i ) ≥ f 2 (x i ) ≥ . . . ≥ f K−1 (x i ), which guarantees consistent predictions . To achieve rank-monotonicity and guarantee binary classifier consistency (Theorem 1), the K − 1 binary tasks share the same weight parameters 2 but have independent bias units (Fig. 2).” ; ) Regarding claim 10 The claim is a computer program product claim corresponding to the method claim 1, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 12 The claim is a computer program product claim corresponding to the method claim 3, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 13 The claim is a computer program product claim corresponding to the method claim 4, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 14 The claim is a computer program product claim corresponding to the method claim 5, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 15 The claim is a computer program product claim corresponding to the method claim 6, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 16 The claim is a computer program product claim corresponding to the method claim 7, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 17 The claim is a computer program product claim corresponding to the method claim 8, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 19 The claim is a system claim corresponding to the method claim 1, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim . 07-21-aia AIA Claim (s) 2, 11, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Kim et al. (Instance-Level Future Motion Estimation in a Single Image Based on Ordinal Regression and Semi-Supervised Domain Adaptation) in view of Zhuang et al. (US 20180188727 A1) further in view of Liu et al. (Deep Learning in Latent Space for Video Prediction and Compression) Regarding claim 2 The combination of Cao, Kim, Zhuang teaches claim 1. However, the combination of Cao, Kim, Zhuang does not appear to explicitly teach: wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data from a high dimensional space above x dimensions into a low dimensional space below y dimensions, where x and y are integers, and x > y. Liu teaches wherein the temporal encoder comprises a Long Short-Term Memory (LSTM) encoding the time series data from a high dimensional space above x dimensions into a low dimensional space below y dimensions, where x and y are integers, and x > y. (Liu [sec(s) Abs] “We propose a novel DNN based framework that predicts and compresses video sequences in the latent vector space. The proposed method first learns the efficient lower-dimensional latent space representation of each video frame and then performs inter-frame prediction in that latent domain” [sec(s) 1] “• Learning based video prediction in latent domain: We use an convolutional long short-term memory (ConvLSTM) network to predict a compact latent representation of the next frame substituting for motion compensation in conventional codecs . This approach only stores the differences between the predicted and actual representation in low dimensional latent space, resulting in entropy reduction of the residuals. The predictor is adversarially trained against a discriminator which significantly enhances the quality of prediction to bring down the entropy (i.e., density and magnitude of nonzero elements) of residuals.” [sec(s) 3] “To further reduce the video code size, we encode the residual with quantization and entropy coding. A desired compression rate is controlled by the size of latent dimension in the image compression stage as well as the number of quantization levels used in residual encoding.” ; ) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Cao, Kim, Zhuang with the latent space dimension of Liu . One of ordinary skill in the art would have been motived to combine in order to achieve significant quality improvements and provide reliable and accurate prediction for compression for videos. (Liu [sec(s) 4] “In the video compression task, we alleviate this issue and achieve significant quality improvements by saving and transmitting key-frames and residuals. For the majority of normal video frame sequences that have strong temporal correlations, the proposed prediction method provides reliable and accurate prediction for compression.”) Regarding claim 11 The claim is a computer program product claim corresponding to the method claim 2, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Regarding claim 20 The claim is a system claim corresponding to the method claim 2, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim . 07-21-aia AIA Claim (s) 9, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al. (Rank consistent ordinal regression for neural networks with application to age estimation) in view of Kim et al. (Instance-Level Future Motion Estimation in a Single Image Based on Ordinal Regression and Semi-Supervised Domain Adaptation) in view of Zhuang et al. (US 20180188727 A1) in view of Zhang et al. (Non-Uniform Discretization-Based Ordinal Regression for Monocular Depth Estimation of an Indoor Drone) Regarding claim 9 The combination of Cao, Kim, Zhuang teaches claim 1. However, the combination of Cao, Kim, Zhuang does not appear to explicitly teach: further comprising automatically controlling a vehicle system for collision avoidance responsive to the ordinal prediction predicting an impending collision. Zhang teaches further comprising automatically controlling a vehicle system for collision avoidance responsive to the ordinal prediction predicting an impending collision. (Zhang [table(s) 1] “fps 75” [sec(s) 1] “In this paper, NSID is proposed, and is shown to yield better results than those of non-uniform discretization (NUD). In important decision areas, the performance of the ordinal regression algorithm proposed in this paper reaches the performance of the state-of-the-art two-stream regression algorithm [14], and the inference time of NSIDORA is 3.4 times faster than that of the two-stream regression algorithm, which is of great significance for autonomous drone navigation and obstacle avoidance with high security requirements .” [sec(s) 2] “This algorithm is currently the most advanced indoor navigation algorithm. The latter two perception methods obtain better navigation performance in actual indoor environments. However, the prediction results of typical classification algorithms are rough, which is not conducive to fine control, whereas the state-of-art regression model treats the distance equally, resulting in a decrease in the performance of close-range prediction. In order to avoid these problems, based on the strong orderly correlation of the distance value, we turned the autonomous navigation and obstacle avoidance of the UAV into an ordinal regression problem .” [sec(s) 4] “The comparison of the overall performance of the four algorithms is shown in Table 1. It can be seen from Table 1 that the classification performance of NSIDORA in the non-decision area is similar to the NUD-based algorithm. The RMSE (the lower the better) of NSIDORA in the decision area is 33.5% lower than that of the NUD-based ordinal regression, and is 6% higher than that of the state-of-the-art two-stream regression algorithm; however, it is better than that of two-stream regression in the first ten categories, shown in Figure 7 and Table 2. Furthermore, on a hardware device with an Intel Xeon E5-2640v4 32GB RAM, the inference fps of NSIDORA, another important indicator of drone indoor depth estimation [35], can reach 75 , which is 3.4 times faster than that of the two-stream regression algorithm.” ; ) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Cao, Kim, Zhuang with the automatic vehicle system of Zhang . One of ordinary skill in the art would have been motived to combine in order to achieve an inference speed of 3.4 times faster than the conventional method so that the vehicle may move avoid obstacles. (Zhang [sec(s) 5] “The experimental results show that the RMSE of NSIDORA in the decision area is 33.5% lower than that of the NUD-based ordinal regression method. Although the RMSE is higher than that of the state-of-the-art two-stream regression algorithm, the inference speed of NSIDORA is 3.4 times faster than that of two-stream ordinal regression method. Furthermore, the RMSE of our distance decoder in the decision area is 20.7% lower than that of the distance decoder presented in [30].”) Regarding claim 18 The claim is a computer program product claim corresponding to the method claim 9, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEHWAN KIM whose telephone number is (571)270-7409. The examiner can normally be reached Mon - Fri 9:00 AM - 5:00 PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /SEHWAN KIM/Examiner, Art Unit 2129 Application/Control Number: 18/545,055 Page 2 Art Unit: 2129 Application/Control Number: 18/545,055 Page 3 Art Unit: 2129 Application/Control Number: 18/545,055 Page 4 Art Unit: 2129 Application/Control Number: 18/545,055 Page 5 Art Unit: 2129 Application/Control Number: 18/545,055 Page 6 Art Unit: 2129 Application/Control Number: 18/545,055 Page 7 Art Unit: 2129 Application/Control Number: 18/545,055 Page 8 Art Unit: 2129 Application/Control Number: 18/545,055 Page 9 Art Unit: 2129 Application/Control Number: 18/545,055 Page 10 Art Unit: 2129 Application/Control Number: 18/545,055 Page 11 Art Unit: 2129 Application/Control Number: 18/545,055 Page 12 Art Unit: 2129 Application/Control Number: 18/545,055 Page 13 Art Unit: 2129 Application/Control Number: 18/545,055 Page 14 Art Unit: 2129 Application/Control Number: 18/545,055 Page 15 Art Unit: 2129 Application/Control Number: 18/545,055 Page 16 Art Unit: 2129 Application/Control Number: 18/545,055 Page 17 Art Unit: 2129 Application/Control Number: 18/545,055 Page 18 Art Unit: 2129 Application/Control Number: 18/545,055 Page 19 Art Unit: 2129 Application/Control Number: 18/545,055 Page 20 Art Unit: 2129 Application/Control Number: 18/545,055 Page 21 Art Unit: 2129 Application/Control Number: 18/545,055 Page 22 Art Unit: 2129 Application/Control Number: 18/545,055 Page 24 Art Unit: 2129 Application/Control Number: 18/545,055 Page 25 Art Unit: 2129 Application/Control Number: 18/545,055 Page 26 Art Unit: 2129 Application/Control Number: 18/545,055 Page 27 Art Unit: 2129 Application/Control Number: 18/545,055 Page 28 Art Unit: 2129 Application/Control Number: 18/545,055 Page 29 Art Unit: 2129 Application/Control Number: 18/545,055 Page 30 Art Unit: 2129 Application/Control Number: 18/545,055 Page 31 Art Unit: 2129 Application/Control Number: 18/545,055 Page 32 Art Unit: 2129 Application/Control Number: 18/545,055 Page 33 Art Unit: 2129 Application/Control Number: 18/545,055 Page 34 Art Unit: 2129 Application/Control Number: 18/545,055 Page 35 Art Unit: 2129 Application/Control Number: 18/545,055 Page 36 Art Unit: 2129 Application/Control Number: 18/545,055 Page 37 Art Unit: 2129 Application/Control Number: 18/545,055 Page 38 Art Unit: 2129 Application/Control Number: 18/545,055 Page 39 Art Unit: 2129 Application/Control Number: 18/545,055 Page 40 Art Unit: 2129 Application/Control Number: 18/545,055 Page 41 Art Unit: 2129 Application/Control Number: 18/545,055 Page 42 Art Unit: 2129 Application/Control Number: 18/545,055 Page 43 Art Unit: 2129 Application/Control Number: 18/545,055 Page 44 Art Unit: 2129 Application/Control Number: 18/545,055 Page 45 Art Unit: 2129 Application/Control Number: 18/545,055 Page 46 Art Unit: 2129 Application/Control Number: 18/545,055 Page 47 Art Unit: 2129
Prosecution Insights