DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.f
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/16/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
Claims 2 & 9 recite “an input unit configured to receive a plurality of position coordinates….”
Claim 2 additionally recites “a control unit configured to generate a past movement trajectory…”
Claim 9 further recites “a control unit configured to train a trajectory estimation model.…”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
“an input unit” (element 110 of Fig. 2, component of the estimation model learning system 100; pg. 21, ln. 15-24 | pg. 23, ln. 4-24 | pg. 31, ln. 5-7)
The associated structure of the input unit 110 is the CPU of an electronic device that implements the estimation model learning system 100.
The associated algorithm performed by the input unit 110 is receiving the image or video via a server or device where an image is stored via a wireless or wired network. It then extracts the position coordinate 11 for a pedestrian in the image or video.
“a control unit” (element 130 of Fig. 2, which performs the processes outlined in Figs. 5-10; pg. 22, ln. 8-19 | pg. 23, ln. 4-24 | pgs. 24-28 | pg. 31, ln. 5-7)
The associated structure of the control unit 130 is the CPU of an electronic device that implements the estimation model learning system 100.
The associated algorithm is as follows: the control unit 130 may control the overall operation of the integrated trajectory estimation model learning system 100. It may receive the plurality of position coordinates 11 to generate the movement trajectory set for a pedestrian and define a singular space based on the movement trajectory to calculate the movement pattern in the singular space. Additionally, it may train the estimation model 121 to estimate future movement trajectories 12 based on the past movement trajectories of a pedestrian based on a predetermined noise distribution using the previously calculated movement pattern. The processes performed by the control unit are outlined in greater detail across Figs. 5-10.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 3 & 10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because signals per se do not fall into one of the four statutory categories.
Claims 3 & 10 recites, inter alia, “a program stored on a computer-readable recording medium,…” After close inspection, the examiner respectfully notes that the disclosure, as a whole, does not definitively describe what can and cannot be considered the “computer readable storage media”. Applicant’s specification discusses the “computer-readable recording medium in pg. 30, ln. 13-24. Applicant describes the “computer-readable recording medium” as possibly being a non-transitory storage medium, such as a HDD, SSD, etc... However, applicant uses only exemplary language to describe the fact that it might be a non-transitory storage medium. Thus, the “computer-readable recording medium” could be other forms – such as a signal.
An Examiner is obliged to give claims their broadest reasonable interpretation consistent with the specification during examination. The broadest reasonable interpretation of a claim drawn to a computer program product (also called a computer readable medium, machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal, per se, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter.
Therefore, given the non-definitive disclosure and the broadest reasonable interpretation, the machine-readable storage medium of the claim may include transitory propagating signals. As a result, the claim pertains to non-statutory subject matter.
However, the Examiner respectfully submits a claim drawn to such a computer program product or computer readable storage medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation “non-transitory” to the claim. Such an amendment would typically not raise the issue of new matter, even when the specification is silent because the broadest reasonable interpretation relies on the ordinary and customary meaning that includes signals per se. For additional information, please see the Patents’ Official Gazette notice published February 23, 2010 (1351 OG 212).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-7 & 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al (“Pedestrian Trajectory Prediction Based on LSTM”, 2023, 7th International Conference on Vehicular Control and Intelligence), hereinafter referred to as “Li”, in view of Holzwarth et al (US 2023/0297113 A1), hereinafter referred to a “Holzwarth”.
Regarding claim 1, Li teach An integrated trajectory estimation method (the pedestrian detection and trajectory prediction framework outlined in Fig. 3 [Sec II. Method Descriptions - Subsec A-B]) comprising:
receiving a plurality of position coordinates according to a movement of a pedestrian (YOLOv5 is used to detect pedestrians to output both a position and ID of pedestrians from a video [Sec II-A. Pedestrian Detection and Data Association - 01; Fig. 2]);
generating a past movement trajectory of the pedestrian based on the plurality of position coordinates (the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4]); and
estimating a future movement trajectory corresponding to the past movement trajectory using a pre-trained trajectory estimation model (future movements trajectories are predicted using historical trajectories for each associated pedestrian ID using a LSTM for human trajectory prediction model [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 01-02; Figs. 3 & 4]),
wherein the pre-trained trajectory estimation model is trained according to an integrated trajectory estimation model learning method (LSTM for human trajectory prediction model is trained to perform trajectory estimation of pedestrians based on historical trajectories using the JAAD dataset [Sec III. Experiments - 01-03; Figs. 3 & 4; Table I]), and
the integrated trajectory estimation model learning method includes
receiving a plurality of learning position coordinates according to the movement of the pedestrian (using the JAAD dataset as inputs, YOLOv5 is used to detect pedestrians to output both a position and ID of pedestrians from a video [Sec II-A. Pedestrian Detection and Data Association - 01 & Sec III. Experiments - 01-02; Fig. 2]),
generating a movement trajectory set related to a learning movement trajectory of the pedestrian based on the plurality of learning position coordinates (using the JAAD dataset as inputs, the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network, which the LSTM trajectory prediction model then uses to generate future trajectories for each pedestrian [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4]),
and
training the trajectory estimation model by using the movement pattern (using the JAAD dataset as inputs, the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network, to train the LSTM trajectory prediction model to predict future trajectories corresponding to each pedestrian ID, with an overall accuracy of 0.79 [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4; Table 1.]).
Li, however, does not conduct singular value decomposition for generating the movement trajectory predictions. Holzwarth, on the other hand, is analogous art pertinent to the field of endeavor of the present application and disclose a method for describing vehicular trajectories by carrying out singular decomposition. More specifically, Holzwarth teach defining a singular space having a singular space coordinate system based on the movement trajectory set (Holzwarth: the preprocessed reference data 11 (trajectories) are stacked column-wise into matrix Y (a movement trajectory set), where Y is decomposed as
Y
=
U
S
V
T
, with singular values
σ
i
arranged diagonally in S (a singular value matrix) and singular vectors stacked column-wise in U and V (left and right singular vector matrices) [0070, 82-83], the dominant singular values
σ
d
,
i
are selected and a matrix
U
d
of the corresponding dominant left singular vector is determined [0072-73], the resulting subspace as
s
p
a
n
(
U
d
)
thus defines the space for estimated trajectories 9 – which the examiner interprets as the "singular space" [0050-52, 65, 78]), and calculating a movement pattern of the pedestrian corresponding to the movement trajectory set on the singular space (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]), and
...movement pattern calculated based on the singular space to estimate the future movement trajectory from the past movement trajectory... (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]).
Holzwarth additionally explain that the use of singular value decomposition informs the optimal basis functions for describing trajectories, with singular value decomposition enabling low-rank approximation that are easy to implement with good approximation results for either offline or online strategies [0015-16]. One of ordinary skill before the effective filing date of the present application would recognize the advantages of Holzwarth's singular value decomposition-based estimation of vehicular trajectories and implement it to inform the pedestrian trajectory prediction-based LSTM by Li to enable fast and effective predictive pedestrian pathing.
Considering claim 3, Li teach the program including instructions to execute:
receiving a plurality of position coordinates according to a movement of a pedestrian (Li: YOLOv5 is used to detect pedestrians to output both a position and ID of pedestrians from a video [Sec II-A. Pedestrian Detection and Data Association - 01; Fig. 2]);
generating a past movement trajectory of the pedestrian based on the plurality of position coordinates (Li: the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4]); and
estimating a future movement trajectory corresponding to the past movement trajectory using a pre-trained trajectory estimation model (Li: future movements trajectories are predicted using historical trajectories for each associated pedestrian ID using a LSTM for human trajectory prediction model [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 01-02; Figs. 3 & 4]),
wherein the pre-trained trajectory estimation model is trained according to an integrated trajectory estimation model learning method (Li: LSTM for human trajectory prediction model is trained to perform trajectory estimation of pedestrians based on historical trajectories using the JAAD dataset [Sec III. Experiments - 01-03; Figs. 3 & 4; Table I]), and
the integrated trajectory estimation model learning method includes
receiving a plurality of learning position coordinates according to the movement of the pedestrian (Li: using the JAAD dataset as inputs, YOLOv5 is used to detect pedestrians to output both a position and ID of pedestrians from a video [Sec II-A. Pedestrian Detection and Data Association - 01 & Sec III. Experiments - 01-02; Fig. 2]),
generating a movement trajectory set related to a learning movement trajectory of the pedestrian based on the plurality of learning position coordinates (Li: using the JAAD dataset as inputs, the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network, which the LSTM trajectory prediction model then uses to generate future trajectories for each pedestrian [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4]),
and
training the trajectory estimation model by using the movement pattern (Li: using the JAAD dataset as inputs, the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network, to train the LSTM trajectory prediction model to predict future trajectories corresponding to each pedestrian ID, with an overall accuracy of 0.79 [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4; Table 1.]).
Li, however, does not conduct singular value decomposition for generating the movement trajectory predictions. Holzwarth, per contra, teach A program stored on a computer-readable recording medium (Holzwarth: external computer 10 such as a PC – the examiner notes a PC would inherently comprise some form of storage medium [¶0067; Fig. 1]), and executed by one or more processors in an electronic device (processor 7 of the external computer 10 [¶0067; Fig. 1]),
defining a singular space having a singular space coordinate system based on the movement trajectory set (Holzwarth: the preprocessed reference data 11 (trajectories) are stacked column-wise into matrix Y (a movement trajectory set), where Y is decomposed as
Y
=
U
S
V
T
, with singular values
σ
i
arranged diagonally in S (a singular value matrix) and singular vectors stacked column-wise in U and V (left and right singular vector matrices) [0070, 82-83], the dominant singular values
σ
d
,
i
are selected and a matrix
U
d
of the corresponding dominant left singular vector is determined [0072-73], the resulting subspace as
s
p
a
n
(
U
d
)
thus defines the space for estimated trajectories 9 – which the examiner interprets as the "singular space" [0050-52, 65, 78]), and calculating a movement pattern of the pedestrian corresponding to the movement trajectory set on the singular space (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]), and
...movement pattern calculated based on the singular space to estimate the future movement trajectory from the past movement trajectory... (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]).
Holzwarth additionally explain that the use of singular value decomposition informs the optimal basis functions for describing trajectories, with singular value decomposition enabling low-rank approximation that are easy to implement with good approximation results for either offline or online strategies [0015-16]. One of ordinary skill before the effective filing date of the present application would recognize the advantages of Holzwarth's singular value decomposition-based estimation of vehicular trajectories and implement it to inform the pedestrian trajectory prediction-based LSTM by Li to enable fast and effective predictive pedestrian pathing.
With respect to claim 4, Li teach An integrated trajectory estimation model learning method (the pedestrian detection and trajectory prediction framework outlined in Fig. 3 [Sec II. Method Descriptions - Subsec A-B]) comprising:
receiving a plurality of position coordinates according to a movement of a pedestrian (using the JAAD dataset as inputs, YOLOv5 is used to detect pedestrians to output both a position and ID of pedestrians from a video [Sec II-A. Pedestrian Detection and Data Association - 01 & Sec III. Experiments - 01-02; Fig. 2]);
generating a movement trajectory set related to a movement trajectory of the pedestrian based on the plurality of position coordinates (using the JAAD dataset as inputs, the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network, which the LSTM trajectory prediction model then uses to generate future trajectories for each pedestrian [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4]);
and
training the trajectory estimation model by using the movement pattern (using the JAAD dataset as inputs, the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network, to train the LSTM trajectory prediction model to predict future trajectories corresponding to each pedestrian ID, with an overall accuracy of 0.79 [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4; Table 1.]).
Li however does not conduct singular value decomposition for generating the movement trajectory predictions. Holzworth, per contra, teach defining a singular space having a singular space coordinate system based on the movement trajectory set (Holzwarth: the preprocessed reference data 11 (trajectories) are stacked column-wise into matrix Y (a movement trajectory set), where Y is decomposed as
Y
=
U
S
V
T
, with singular values
σ
i
arranged diagonally in S (a singular value matrix) and singular vectors stacked column-wise in U and V (left and right singular vector matrices) [0070, 82-83], the dominant singular values
σ
d
,
i
are selected and a matrix
U
d
of the corresponding dominant left singular vector is determined [0072-73], the resulting subspace as
s
p
a
n
(
U
d
)
thus defines the space for estimated trajectories 9 – which the examiner interprets as the "singular space" [0050-52, 65, 78]), and calculating a movement pattern of the pedestrian corresponding to the movement trajectory set on the singular space (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]), and
...movement pattern calculated based on the singular space to estimate the future movement trajectory from the past movement trajectory... (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]).
Holzwarth additionally explain that the use of singular value decomposition informs the optimal basis functions for describing trajectories, with singular value decomposition enabling low-rank approximation that are easy to implement with good approximation results for either offline or online strategies [0015-16]. One of ordinary skill before the effective filing date of the present application would recognize the advantages of Holzwarth's singular value decomposition-based estimation of vehicular trajectories and implement it to inform the pedestrian trajectory prediction-based LSTM by Li to enable fast and effective predictive pedestrian pathing.
Turning to claim 5, Li in view of Holzwarth teach The integrated trajectory estimation model learning method of the claim 4 (as described above), wherein the calculating of the movement pattern of the pedestrian (Li: the pedestrian detection and trajectory prediction framework outlined in Fig. 3, which predicts the movement patterns of pedestrians [Sec II. Method Descriptions - Subsec A-B]).
Li, again, fails to disclose performing singular value decomposition for their pedestrian movement pattern predictions. Holzwarth, per contra, teach includes
performing singular value decomposition on the movement trajectory set (Holzwarth: a singular value decomposition is performed on matrix Y (the movement trajectory set) [0070]),
defining the singular space based on a singular vector generated according to the singular value decomposition (Holzwarth: matrix U of left singular vectors is used to generate
U
d
, wherein the estimated trajectories 9 are determined or optimized in the subspace
s
p
a
n
(
U
d
)
[0070, 73, 78]), and
projecting the movement trajectory set onto the singular space and calculating the movement pattern (Holzwarth: the movement trajectory set (matrix Y) is inherently projected onto the singular space (subspace
s
p
a
n
(
U
d
)
given that
U
d
is derived from decomposition of Y, with estimated trajectories 9 (calculated movement patterns) being based in the subspace
s
p
a
n
(
U
d
)
[0070-78]).
Holzwarth again explain that the use of singular value decomposition informs the optimal basis functions for describing trajectories, with singular value decomposition enabling low-rank approximation that are easy to implement with good approximation results for either offline or online strategies [0015-16]. One of ordinary skill before the effective filing date of the present application would recognize the advantages of Holzwarth's singular value decomposition-based estimation of vehicular trajectories and implement it to inform the pedestrian trajectory prediction-based LSTM by Li to enable fast and effective predictive pedestrian pathing.
As for claim 6, Li in view of Holzwarth The integrated trajectory estimation model learning method of the claim 5 (described above), wherein the performing of the singular value decomposition includes
performing singular value decomposition on the movement trajectory set, and calculating a diagonal matrix including a plurality of singular values, and the singular vector (Holzwarth: singular value decomposition is performed on matrix Y (the movement trajectory set), wherein a diagonal singular value matrix S comprising singular values
σ
i
is calculated as well as a left and right matrix of singular vectors U and V [0070]),
extracting a predetermined number of singular values among the plurality of singular values included in the diagonal matrix (Holzwarth: at least one of the singular values are identified as dominant singular values
σ
d
,
i
from the diagonal matrix S, with a defined number of sorted singular values to be selected as dominant [0071-72]), and
correcting the singular vector based on the extracted singular values (Holzwarth: a matrix
U
d
is formed using only the dominant left singular vectors, excluding non-dominant singular values
σ
n
d
,
i
– the examiner notes that the inclusion of only some (e.g., dominant) singular values in the matrix
U
d
functions as a form of correcting the matrix U [0072-73]).
Holzwarth additionally explain that the use of singular value approximation informs the optimal basis functions for describing trajectories, with singular value decomposition enabling low-rank approximation that are easy to implement with good approximation results for either offline or online strategies [0015-16]. One of ordinary skill before the effective filing date of the present application would recognize the advantages of Holzwarth's singular value decomposition-based estimation of vehicular trajectories and implement it to inform the pedestrian trajectory prediction-based LSTM by Li to enable fast and effective predictive pedestrian pathing.
Concerning claim 7, Li in view of Holzwarth teach The integrated trajectory estimation model learning method of the claim 4 (as previously described), wherein the calculating of the movement pattern of the pedestrian includes calculating (Li: the pedestrian detection and trajectory prediction framework outlined in Fig. 3, which predicts the movement patterns of pedestrians [Sec II. Method Descriptions - Subsec A-B]), but Li is silent on calculating a first and second movement pattern based on the singular space utilizing different trajectory sets.
Holzwarth, on the other hand, teach by using a first movement pattern calculated in advance based on the singular space, a second movement pattern corresponding to a second movement trajectory set different from a first movement trajectory set used to calculate the first movement pattern (Holzwarth: a first and second estimated trajectory (9a and 9b) are determined using basis functions 13 derived from singular values
σ
d
[0097; Fig. 6], wherein the number of basis functions illustrate potential trajectories for a given vehicle [0093-95; Fig. 5]).
Holzwarth again explain that the use of singular value approximation informs the optimal basis functions for describing trajectories, with singular value decomposition enabling low-rank approximation that are easy to implement with good approximation results for either offline or online strategies [0015-16]. One of ordinary skill before the effective filing date of the present application would recognize the advantages of Holzwarth's singular value decomposition-based estimation of vehicular trajectories and implement it to inform the pedestrian trajectory prediction-based LSTM by Li to enable fast and effective predictive pedestrian pathing.
With respect to claim 10, Li teach the program including instructions to execute:
receiving a plurality of position coordinates according to a movement of a pedestrian (Li: YOLOv5 is used to detect pedestrians to output both a position and ID of pedestrians from a video [Sec II-A. Pedestrian Detection and Data Association - 01; Fig. 2]);
generating a past movement trajectory of the pedestrian based on the plurality of position coordinates (Li: the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4]); and
and
training the trajectory estimation model by using the movement pattern (Li: using the JAAD dataset as inputs, the historical trajectory of pedestrians is obtained via data association of each pedestrian's ID for each frame based off their detected coordinate position from YOLOv5 network, to train the LSTM trajectory prediction model to predict future trajectories corresponding to each pedestrian ID, with an overall accuracy of 0.79 [Sec II. Methods Descriptions - 01; Sec II-B. Pedestrian Trajectory Prediction Model - 02; Figs. 3 & 4; Table 1.]).
Li, however, does not conduct singular value decomposition for generating the movement trajectory predictions. Holzwarth, per contra, teach A program stored on a computer-readable recording medium (Holzwarth: external computer 10 such as a PC – the examiner notes a PC would inherently comprise some form of storage medium [¶0067; Fig. 1]), and executed by one or more processors in an electronic device (processor 7 of the external computer 10 [¶0067; Fig. 1]),
defining a singular space having a singular space coordinate system based on the movement trajectory set (Holzwarth: the preprocessed reference data 11 (trajectories) are stacked column-wise into matrix Y (a movement trajectory set), where Y is decomposed as
Y
=
U
S
V
T
, with singular values
σ
i
arranged diagonally in S (a singular value matrix) and singular vectors stacked column-wise in U and V (left and right singular vector matrices) [0070, 82-83], the dominant singular values
σ
d
,
i
are selected and a matrix
U
d
of the corresponding dominant left singular vector is determined [0072-73], the resulting subspace as
s
p
a
n
(
U
d
)
thus defines the space for estimated trajectories 9 – which the examiner interprets as the "singular space" [0050-52, 65, 78]), and calculating a movement pattern of the pedestrian corresponding to the movement trajectory set on the singular space (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]), and
...movement pattern calculated based on the singular space to estimate the future movement trajectory from the past movement trajectory... (Holzwarth: estimated trajectories 9 are determined and/or optimized by an approximation in the subspace
s
p
a
n
(
U
d
)
via singular value decomposition of matrix Y [0070-78; Fig. 6]).
Holzwarth additionally explain that the use of singular value decomposition informs the optimal basis functions for describing trajectories, with singular value decomposition enabling low-rank approximation that are easy to implement with good approximation results for either offline or online strategies [0015-16]. One of ordinary skill before the effective filing date of the present application would recognize the advantages of Holzwarth's singular value decomposition-based estimation of vehicular trajectories and implement it to inform the pedestrian trajectory prediction-based LSTM by Li to enable fast and effective predictive pedestrian pathing.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al (“Pedestrian Trajectory Prediction Based on LSTM”, 2023, 7th International Conference on Vehicular Control and Intelligence), hereinafter referred to as “Li”, in view of Holzwarth et al (US 2023/0297113 A1), hereinafter referred to a “Holzwarth”, further in view of Liu et al (“Faster R-CNN for Robust Pedestrian Detection Using Semantic Segmentation Network”, 2018, Frontiers in Neurorobotics), hereinafter referred to as “Liu”.
Regarding claim 8, Li in view of Holzwarth teach The integrated trajectory estimation model learning method of the claim 4 (as described previously), wherein the trajectory estimation model is implemented to estimate a walking area from an image corresponding to the plurality of received position coordinates (Li: the pedestrian detection and trajectory prediction framework predicts walking trajectories from an existing first view of a defined walking area based on historical movement trajectories - the examiner notes that the walking area is defined by the first view of a video [Sec II-A. Pedestrian Detection and Data Association - 01; Figs. 1 & 3]).
Li fails to disclose using a semantic segmentation model, instead using YOLOv5, an instance segmentation model for pedestrian detection. Liu, per contra, is analogous art pertinent to the field of endeavor of the present application and discloses a semantic segmentation network for pedestrian detection. Liu teach ...using a pre-equipped semantic segmentation model… (Liu: semantic features of pedestrians are generated via implementation of SegNet to classify and detect pedestrians in a provided image or video, which was trained using Caffe-SegNet on the Caltech benchmark pedestrian dataset [Sec 3.2. Semantic Network for Semantic Feature Extraction - 01-03 & Sec 4.1. Datasets and Implementations - 01; Fig. 3]).
Liu clarifies that their region-based CNN uses semantic clues to classify and detect pedestrians with a high degree of accuracy while accounting for pedestrians at different scales (as a consequence of the view) [Sec. Abstract]. One of ordinary skill before the effective filing date would recognize the advantage of substituting YOLOv5 of Li with the semantic segmentation model taught by Liu to ensure proper detection and classification of pedestrians in the image view, avoiding trajectory calculation for any "non-pedestrian"-type objects.
Allowable Subject Matter
Claims 2 & 9 are allowed. The following is a statement of reasons for the indication of allowable subject matter:
Claims 2 & 9 similarly recite “an input unit” and “a control unit” which was previously indicated to be interpreted under 35 U.S.C. § 112(f), which necessitate mapping to the structure and associated algorithm for performing the claimed function as indicated in the claim interpretation section of this office action. None of the prior art cited nor made of record teach the claimed “control unit” in accordance with its associated algorithm described in the specification, more particularly the processes outlined in Figs. 5-10; pg. 22, ln. 8-19 | pg. 23, ln. 4-24 | pgs. 24-28 | pg. 31, ln. 5-7. The closest prior art cited, Li and Holzwarth, teach towards some, but not all of the embodiments outlined throughout Figs. 5-10.
Therefore, claims 2 & 9 are allowed.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Gutierrez et al (US 2020/0023842 A1) describe an apparatus for observing and predicting pedestrian trajectories for collision avoidance.
Ye et al (US 12,559,099 B1) teach a machine-learning model for predicting a future position and orientation of object based on its prior state history.
Bae et al (“EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecasting”, 2023, IEEE/CVF) disclose a pedestrian trajectory prediction system that leverages singular value decomposition for extracting key information from raw data.
Wang et al (“Singular Value Decomposition Based Pedestrian Trajectory Prediction”, 2023, 26th International Conference on Computer Supported Cooperative Work in Design) outline a pedestrian trajectory prediction algorithm utilizing singular value decomposition to model pedestrian-environment interactions.
Chib et al (“LG-Traj: LLM Guided Pedestrian Trajectory Prediction, 2024, arXiv) utilize LLMs to generate motion cues from previous pedestrian trajectories to inform future predictions.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael M. Sofroniou whose telephone number is (571)272-0287. The examiner can normally be reached M-F: 8:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M. Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL M SOFRONIOU/Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights