DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a Non-Final Action on the Merits. Claims 1, 3-10, and 12-19 are currently pending and are addressed below.
Response to Amendments
The amendment filed on November 21st, 2025 has been considered and entered. Accordingly, claims 1, 5, 10, 14, and 19 have been amended.
Response to Arguments
The applicant’s arguments with respect to claims 1, 3-10, and 12-19 have been considered and found to be persuasive. Accordingly, the instant application is Non-Final.
The applicant states (Amend. 8-10) that Sakr (US 20200096359 A1) (“Sakr”) fails to disclose the limitation “wherein the geospatial offset is different for each of the one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory” in amended claim 1. The examiner respectfully disagrees. Sakr teaches applying a geospatial offset to a trajectory, where in certain cases the offset can be the same throughout all of the points in the trajectory (See at least Sakr Paragraph 14 “The method where the V2X wireless message further includes path history data describing multiple path history points of the remote vehicle, the remote road parameter data includes a lateral offset of the remote vehicle from a center of a reference lane, and generating the HD map data describing the real-time HD map further includes: generating one or more interpolated points based on the multiple path history points, where a lateral offset for each of the one or more interpolated points is estimated based on the lateral offset of the remote vehicle; and generating a path of the remote vehicle on the HD map based on the multiple path history points and the one or more interpolated points. The method where the remote road parameter data further includes one or more of a relative heading of the remote vehicle, curvature of the reference lane, and a curvature change rate of the reference lane. The method where the real-time HD map provides an estimate of a position of the remote vehicle that is accurate within plus or minus half a width of a lane on the road. The method where the remote GPS data describes a geographical location of the remote vehicle, and the ego GPS data describes a geographical location of the ego vehicle.” | Paragraph 178 “The map generation module 210 generates five interpolated points 607 between each two adjacent path history points using one or more interpolation techniques such as a linear interpolation technique. A lateral offset for each of the five interpolated points 607 is determined based on the lateral offsets of the two adjacent path history points. For example, if the two adjacent path history points 605B and 605C have a same lateral offset (the lateral offset 601B=the lateral offset 601C), then the five interpolated points 607 between the two adjacent path history points 605B and 605C may have the same lateral offset as the two adjacent path history points 605B and 605C.”) such that the teachings of Sakr can be combined with Kroepfl (US 20210063200 A1) (“Kroepfl”) such that for each of the one or more of the discrete trajectorie1s that geospatial offset is different and the geospatial offset is the same across all geospatial observations of a respective discrete trajectory.
The applicant states (Amend. 10-12) that Cheng (“Exploring Dynamic Context for Multi-Path Trajectory Prediction”) (“Cheng”) (Attached) does not teach the limitations of claim 3. The examiner respectfully disagrees. Cheng teaches the application of a multi-head self-attentional layer with a distance kernel defined as a Euclidean distance to the discrete trajectories (See at least Chen III C “The spatial-temporal context from both the observation time and prediction time are encoded by Encoder X and Y, respectively. Both encoders have the same two-stream structure: both streams consist of stacked self-attention layers; as illustrated in Fig. 2 one stream is followed by a global average pooling (GApool), while the other one is followed by an LSTM module. The upper stream is trained to learn motion information from the observed trajectory, whose input is the locations vector of the observed trajectory of the target agent Xi = {x 1 i ,··· ,x T i } ∈ R T×2 . The lower stream is trained to explore dynamic interactions among agents from the dynamic maps noted as DM = {O,S,P} ∈ R T×H×W×3 (discussed in Sec. III-B). For simplicity, we take the upper stream for illustration. T … To attend to different information from different representation subspaces jointly, the multi-head attention [24] strategy is applied as a conventional operation, where a head is an independent scaled dot-product attention module:” | III D “As shown in Fig. 2, during the training, both the observed trajectory Xi and its future trajectory Yi are encoded by Encoder X and Y (see Sec. III-C), respectively. Then, their encodings are concatenated and passed through two FC layers (each is followed by a ReLU activation) for fusion. Then, two side-by-side FC layers are used to estimate the mean µzi and the standard deviation σzi of the latent variables zi . A trajectory Yˆ i is reconstructed by an LSTM decoder step by step by taking zi and the encodings of observation as input. Because the random sampling process of zi can not be back propagated during training, the standard reparameterization trick [43] is adopted to make it differentiable. To minimize the error between the predicted trajectory Yˆ i and the ground truth Yi , the reconstruction loss is defined as the L2 loss (Euclidean distance). Thus, the whole network is trained by minimizing the loss function using the stochastic gradient descent method” | IV B “We adopt the most popular evaluation metrics: the mean average displacement error (ADE) and the final displacement error (FDE) to measure the trajectory prediction performance. ADE measures the aligned Euclidean distance from the prediction to its corresponding ground truth trajectory averaged over all steps. The mean value across all the trajectories is reported. FDE measures the Euclidean distance between the last position from the prediction to the corresponding ground truth position. In addition, the most-likely prediction is decided by the ranking method as described in Sec III-E. Compared with the ground truth (only if it is available), @top10 is the one out of ten predicted trajectories that has the smallest ADE and FDE”) such that the teachings of Cheng can be combined with the teachings of Kroepfl (US 20210063200 A1) (“Kroepfl”) in view of Sakr (US 20200096539 A1) (“Sakr”) in view of Cresswell (US 2023026027 A1) (“Cresswell”) such that the application of a multi-head self-attentional layer with a distance kernel defined as a Euclidean distance to the discrete trajectories is used for determining geospatial offsets for the one or more of the discrete trajectories in order to align the discrete trajectories.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 6-7, 10-11, 15-16, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kroepfl (US 20210063200 A1) (“Kroepfl”) in view of Sakr (US 20200096359 A1) (“Sakr”) in view of Cresswell (US 2023026027 A1) (“Cresswell”)
With respect to claim 1, Kroepfl teaches an apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:
receive a plurality of sequences of geospatial observations from discrete trajectories (See at least Kroepfl Paragraph 48 “To generate mapstreams, any number of vehicles 1500—e.g., consumer vehicles, data collection vehicles, a combination thereof—may be execute any number of drives … Each individual vehicle 1500 may generate the sensor data 102 may use the sensor data 102 for mapstream generation 104 corresponding to the particular drive of the vehicle 1500. The mapstreams generated to correspond to different drives from a single vehicle 1500 and the drives from any number of other vehicles 1500 may be used in map creation 106, as described in more detail herein” | Paragraph 85 “The map creation process 106 may include receiving the mapstreams 210 from one or more vehicles 1500 corresponding to any number of drives …” | Paragraph 122 “The method 700, at block B702, includes receiving data representative of a plurality of mapstreams corresponding to a plurality of drives from a plurality of vehicles. For example, the mapstreams 210 may be received for the map creation 106 process, where the mapstreams 210 may correspond to any number of drives from any number of vehicles 1500.”)
align the discrete trajectories to generate aligned geospatial observations for one or more of the discrete trajectories and concatenate the aligned geospatial observations to form aligned geospatial observations (See at least Kroepfl Paragraph 97 “Referring again to FIGS. 4 and 5A, the pairs of sections may then be registered to one another to generate pose links between them, which may be used for pose optimization 404. Registration 402 may be executed to determine the geometric relationships between multipole drives or mapstreams 210 corresponding thereto in many different locations where the drives or mapstreams 210 overlap—e.g., the portions 552A-552D of FIG. 5C. The output of the registration process may include relative pose links (e.g., rotation and translation between poses of a first mapstream frame or sections and poses of a second mapstream frame or sections) between pairs of frames or sections in different mapstreams. In addition to the relative poses, the pose links may further represent covariances representing the confidence in the respective pose link. The relative pose links are then used to align the maps 504 corresponding to each of the mapstreams 210 such that landmarks and other features—e.g., points clouds, LiDAR image maps, RADAR image maps, etc.—are aligned in a final, aggregate, HD map. As such, the registration process is executed by localizing one map 504—or portion thereof—to the other map 504—or portion thereof—of the pair. Registration 402 may be executed for each sensor modality and/or for each map layer corresponding to various sensor modalities.” | Paragraph 106 “The frame graph 600A is an example of a portion of a larger frame graph, where the larger frame graph may correspond to any portion—or all portions—of a road structure or layout. Four drives 602A-602D may have been registered together to generate pose links 608 (e.g., pose links 608A-608C) between poses 604 or positions of different drives 602. Each pose 604 or position of a single drive 602 may be represented by links 606. For example, the links 606 may represent a translation, rotation, and/or covariance from one pose 604 to a next pose 604. As such, link 606A(6) may represent a rotation, translation, and/or covariance between the poses 604 of the drive 602A that the link 606A(5) connects. Pose links 608A may correspond to the outputs of the registration process 402, and may encode a translation, rotation, and/or covariance between poses 604 of different drives 602. For example, pose link 608A may encode six degrees of freedom transformations between poses—e.g., the translation (e.g., difference in (x, y, z) location), rotation (e.g., difference in x, y, and z axis angles), and/or covariance (e.g., corresponding to the confidence in the values of the pose link) between pose 604A(1) of drive 602A and pose 604B(2) of the drive 602B”)
and provide for at least one of navigational assistance or at least semi-autonomous vehicle control based on the map geometries (See at least Kroepfl Paragraph 128 “The method 700, at block B714, includes transmitting data representative of the fused map to one or more vehicles for use in executing one or more operations.”)
Kroepfl fails to explicitly disclose the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is different for each of the one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, and process the concatenated, aligned geospatial observations using one or more Set Transformers; generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations.
Sakr teaches the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory (See at least Sakr Paragraph 14 “The method where the V2X wireless message further includes path history data describing multiple path history points of the remote vehicle, the remote road parameter data includes a lateral offset of the remote vehicle from a center of a reference lane, and generating the HD map data describing the real-time HD map further includes: generating one or more interpolated points based on the multiple path history points, where a lateral offset for each of the one or more interpolated points is estimated based on the lateral offset of the remote vehicle; and generating a path of the remote vehicle on the HD map based on the multiple path history points and the one or more interpolated points. The method where the remote road parameter data further includes one or more of a relative heading of the remote vehicle, curvature of the reference lane, and a curvature change rate of the reference lane. The method where the real-time HD map provides an estimate of a position of the remote vehicle that is accurate within plus or minus half a width of a lane on the road. The method where the remote GPS data describes a geographical location of the remote vehicle, and the ego GPS data describes a geographical location of the ego vehicle.” | Paragraph 178 “The map generation module 210 generates five interpolated points 607 between each two adjacent path history points using one or more interpolation techniques such as a linear interpolation technique. A lateral offset for each of the five interpolated points 607 is determined based on the lateral offsets of the two adjacent path history points. For example, if the two adjacent path history points 605B and 605C have a same lateral offset (the lateral offset 601B=the lateral offset 601C), then the five interpolated points 607 between the two adjacent path history points 605B and 605C may have the same lateral offset as the two adjacent path history points 605B and 605C.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the apparatus of Kroepfl to include the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, as taught by Sakr as disclosed above, such that the geospatial offset is different for each of the one or more of the discrete trajectories and that the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, in order to ensure accurate map geometries (Sakr Abstract “The disclosure includes embodiments for generating a real-time high-definition (HD) map for an ego vehicle using wireless vehicle data of a remote vehicle”).
Kroepfl in view of Sak fail to explicitly disclose to process the concatenated, aligned geospatial observations using one or more Set Transformers; generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations.
Cresswell, however, teaches to process the concatenated, aligned geospatial observations using one or more Set Transformer, and generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations (See at least Cresswell FIGS. 3-7 and Paragraph 14 “When the attention mechanism is applied to map the alignment input to the alignment output the historical feature representations may be stored in slots of a slot-based memory and the alignment output may define an assignment of each current entity to a different one of the slots. In a slot-based memory the memory may be organized as a set of slots in which each slot is configured to store a feature representation. The feature representation stored in each slot may be updated using the current feature representation of the current entity assigned to the slot by the assignment output, e.g. using a slot-specific neural network.” | Paragraphs 94-117 “In general an attention mechanism maps a query and a set of key-value pairs to an output computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility or similarity function of the query with the corresponding key. The query, keys, and values are all vectors and each may be derived by transforming an object e.g. a feature representation or set of feature representations into a respective vector e.g. using a learned linear transform. The attention mechanism may be a self-attention mechanism in which case the query and the set of key-value pairs are all derived from the set of input objects i.e. input feature representations … In some cases, when the alignment outputs generated by the alignment neural network 120 are used to control an agent, the alignment input can also include data identifying the action performed by the agent at the preceding time point … FIG. 4 is a flow diagram of an example process 400 for generating an alignment output at a given time point. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, an alignment system, e.g., the alignment system 100 of FIG. 1 , appropriately programmed in accordance with this specification, can perform the process 400 … The system processes an alignment input that includes (i) the respective historical feature representations for the set of historical entities and (ii) the current feature representations for the set of current entities using an alignment neural network to generate an alignment output that defines, for each of one or more of the current entities, a corresponding historical entity that is the same as the current entity (step 406).”).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have modified the apparatus of Kroepfl in view of Sakr to include processing the concatenated, aligned geospatial observations using one or more Set Transformer, and generating, from the one or more Set Transformer, map geometries including objects from the geospatial observations, as taught by Cresswell as disclose above, in order to ensure an accurate map geometry for the navigational assistance (Cresswell Paragraph 4 “This specification describes a system implemented as computer programs on one or more computers in one or more locations that aligns entities, e.g., objects, across multiple image observations of an environment using a neural network”).
With respect to claim 6, and similarly claim 15, Kroepfl in view of Sakr in view of Cresswell teach that the objects from the geospatial observations included in the map geometries comprise point objects and linear objects, wherein point objects comprise signs and poles, and wherein linear objects comprise road markings and road boundaries (See at least Kroepfl Paragraph 58 “As an example, the DNNs 202 may process the sensor data 102 to generate detections of lane markings, road boundaries, signs, poles, trees, static objects, vehicles and/or other dynamic objects, wait conditions, intersections, distances, depths, dimensions of objects, etc. For example, the detections may correspond to locations (e.g., in 2D image space, in 3D space, etc.), geometry, pose, semantic information, and/or other information about the detection. As such, for lane lines, locations of the lane lines and/or types of the lane lines (e.g., dashed, solid, yellow, white, crosswalk, bike lane, etc.) may be detected by a DNN(s) 202 processing the sensor data 102. With respect to signs, locations of signs or other wait condition information and/or types thereof (e.g., yield, stop, pedestrian crossing, traffic light, yield light, construction, speed limit, exits, etc.) may be detected using the DNN(s) 202. For detected vehicles, motorcyclists, and/or other dynamic actors or road users, the locations and/or types of the dynamic actors may be identified and/or tracked, and/or may be used to determine wait conditions in a scene (e.g., where a vehicle behaves a certain way with respect to an intersection, such as by coming to a stop, the intersection or wait conditions corresponding thereto may be detected as an intersection with a stop sign or a traffic light).”).
With respect to claims 7, and similarly claims 16, Kroepfl in view of Sakr in view of Cresswell teach that the objects from the geospatial observations are used to facilitate autonomous vehicle control (See at least Kroepfl Paragraph 128 “The method 700, at block B714, includes transmitting data representative of the fused map to one or more vehicles for use in executing one or more operations.”).
With respect to claim 10, Kroepfl teaches A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:
receive a plurality of sequences of geospatial observations from discrete trajectories (See at least Kroepfl Paragraph 48 “To generate mapstreams, any number of vehicles 1500—e.g., consumer vehicles, data collection vehicles, a combination thereof—may be execute any number of drives … Each individual vehicle 1500 may generate the sensor data 102 may use the sensor data 102 for mapstream generation 104 corresponding to the particular drive of the vehicle 1500. The mapstreams generated to correspond to different drives from a single vehicle 1500 and the drives from any number of other vehicles 1500 may be used in map creation 106, as described in more detail herein” | Paragraph 85 “The map creation process 106 may include receiving the mapstreams 210 from one or more vehicles 1500 corresponding to any number of drives …” | Paragraph 122 “The method 700, at block B702, includes receiving data representative of a plurality of mapstreams corresponding to a plurality of drives from a plurality of vehicles. For example, the mapstreams 210 may be received for the map creation 106 process, where the mapstreams 210 may correspond to any number of drives from any number of vehicles 1500.”)
align the discrete trajectories to generate aligned geospatial observations through application of a geospatial offset for one or more of the discrete trajectories and concatenate the aligned geospatial observations to form aligned geospatial observations (See at least Kroepfl Paragraph 97 “Referring again to FIGS. 4 and 5A, the pairs of sections may then be registered to one another to generate pose links between them, which may be used for pose optimization 404. Registration 402 may be executed to determine the geometric relationships between multipole drives or mapstreams 210 corresponding thereto in many different locations where the drives or mapstreams 210 overlap—e.g., the portions 552A-552D of FIG. 5C. The output of the registration process may include relative pose links (e.g., rotation and translation between poses of a first mapstream frame or sections and poses of a second mapstream frame or sections) between pairs of frames or sections in different mapstreams. In addition to the relative poses, the pose links may further represent covariances representing the confidence in the respective pose link. The relative pose links are then used to align the maps 504 corresponding to each of the mapstreams 210 such that landmarks and other features—e.g., points clouds, LiDAR image maps, RADAR image maps, etc.—are aligned in a final, aggregate, HD map. As such, the registration process is executed by localizing one map 504—or portion thereof—to the other map 504—or portion thereof—of the pair. Registration 402 may be executed for each sensor modality and/or for each map layer corresponding to various sensor modalities.” | Paragraph 106 “The frame graph 600A is an example of a portion of a larger frame graph, where the larger frame graph may correspond to any portion—or all portions—of a road structure or layout. Four drives 602A-602D may have been registered together to generate pose links 608 (e.g., pose links 608A-608C) between poses 604 or positions of different drives 602. Each pose 604 or position of a single drive 602 may be represented by links 606. For example, the links 606 may represent a translation, rotation, and/or covariance from one pose 604 to a next pose 604. As such, link 606A(6) may represent a rotation, translation, and/or covariance between the poses 604 of the drive 602A that the link 606A(5) connects. Pose links 608A may correspond to the outputs of the registration process 402, and may encode a translation, rotation, and/or covariance between poses 604 of different drives 602. For example, pose link 608A may encode six degrees of freedom transformations between poses—e.g., the translation (e.g., difference in (x, y, z) location), rotation (e.g., difference in x, y, and z axis angles), and/or covariance (e.g., corresponding to the confidence in the values of the pose link) between pose 604A(1) of drive 602A and pose 604B(2) of the drive 602B”)
and provide for at least one of navigational assistance or at least semi-autonomous vehicle control based on the map geometries (See at least Kroepfl Paragraph 128 “The method 700, at block B714, includes transmitting data representative of the fused map to one or more vehicles for use in executing one or more operations.”)
Kroepfl fails to explicitly disclose the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is different for each of the one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, and process the concatenated, aligned geospatial observations using one or more Set Transformers; generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations.
Sakr teaches the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory (See at least Sakr Paragraph 14 “The method where the V2X wireless message further includes path history data describing multiple path history points of the remote vehicle, the remote road parameter data includes a lateral offset of the remote vehicle from a center of a reference lane, and generating the HD map data describing the real-time HD map further includes: generating one or more interpolated points based on the multiple path history points, where a lateral offset for each of the one or more interpolated points is estimated based on the lateral offset of the remote vehicle; and generating a path of the remote vehicle on the HD map based on the multiple path history points and the one or more interpolated points. The method where the remote road parameter data further includes one or more of a relative heading of the remote vehicle, curvature of the reference lane, and a curvature change rate of the reference lane. The method where the real-time HD map provides an estimate of a position of the remote vehicle that is accurate within plus or minus half a width of a lane on the road. The method where the remote GPS data describes a geographical location of the remote vehicle, and the ego GPS data describes a geographical location of the ego vehicle.” | Paragraph 178 “The map generation module 210 generates five interpolated points 607 between each two adjacent path history points using one or more interpolation techniques such as a linear interpolation technique. A lateral offset for each of the five interpolated points 607 is determined based on the lateral offsets of the two adjacent path history points. For example, if the two adjacent path history points 605B and 605C have a same lateral offset (the lateral offset 601B=the lateral offset 601C), then the five interpolated points 607 between the two adjacent path history points 605B and 605C may have the same lateral offset as the two adjacent path history points 605B and 605C.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the apparatus of Kroepfl to include the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, as taught by Sakr as disclosed above, such that the geospatial offset is different for each of the one or more of the discrete trajectories and that geospatial offset is the same across all geospatial observations of a respective discrete trajectory, in order to ensure accurate map geometries (Sakr Abstract “The disclosure includes embodiments for generating a real-time high-definition (HD) map for an ego vehicle using wireless vehicle data of a remote vehicle”).
Kroepfl in view of Sak fail to explicitly disclose to process the concatenated, aligned geospatial observations using one or more Set Transformers; generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations.
Cresswell, however, teaches to process the concatenated, aligned geospatial observations using one or more Set Transformer, and generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations (See at least Cresswell FIGS. 3-7 and Paragraph 14 “When the attention mechanism is applied to map the alignment input to the alignment output the historical feature representations may be stored in slots of a slot-based memory and the alignment output may define an assignment of each current entity to a different one of the slots. In a slot-based memory the memory may be organized as a set of slots in which each slot is configured to store a feature representation. The feature representation stored in each slot may be updated using the current feature representation of the current entity assigned to the slot by the assignment output, e.g. using a slot-specific neural network.” | Paragraphs 94-117 “In general an attention mechanism maps a query and a set of key-value pairs to an output computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility or similarity function of the query with the corresponding key. The query, keys, and values are all vectors and each may be derived by transforming an object e.g. a feature representation or set of feature representations into a respective vector e.g. using a learned linear transform. The attention mechanism may be a self-attention mechanism in which case the query and the set of key-value pairs are all derived from the set of input objects i.e. input feature representations … In some cases, when the alignment outputs generated by the alignment neural network 120 are used to control an agent, the alignment input can also include data identifying the action performed by the agent at the preceding time point … FIG. 4 is a flow diagram of an example process 400 for generating an alignment output at a given time point. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, an alignment system, e.g., the alignment system 100 of FIG. 1 , appropriately programmed in accordance with this specification, can perform the process 400 … The system processes an alignment input that includes (i) the respective historical feature representations for the set of historical entities and (ii) the current feature representations for the set of current entities using an alignment neural network to generate an alignment output that defines, for each of one or more of the current entities, a corresponding historical entity that is the same as the current entity (step 406).”).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have modified the apparatus of Kroepfl in view of Sakr to include processing the concatenated, aligned geospatial observations using one or more Set Transformer, and generating, from the one or more Set Transformer, map geometries including objects from the geospatial observations, as taught by Cresswell as disclose above, in order to ensure an accurate map geometry for the navigational assistance (Cresswell Paragraph 4 “This specification describes a system implemented as computer programs on one or more computers in one or more locations that aligns entities, e.g., objects, across multiple image observations of an environment using a neural network”).
With respect to claim 19, Kroepfl teaches a method comprising:
receive a plurality of sequences of geospatial observations from discrete trajectories (See at least Kroepfl Paragraph 48 “To generate mapstreams, any number of vehicles 1500—e.g., consumer vehicles, data collection vehicles, a combination thereof—may be execute any number of drives … Each individual vehicle 1500 may generate the sensor data 102 may use the sensor data 102 for mapstream generation 104 corresponding to the particular drive of the vehicle 1500. The mapstreams generated to correspond to different drives from a single vehicle 1500 and the drives from any number of other vehicles 1500 may be used in map creation 106, as described in more detail herein” | Paragraph 85 “The map creation process 106 may include receiving the mapstreams 210 from one or more vehicles 1500 corresponding to any number of drives …” | Paragraph 122 “The method 700, at block B702, includes receiving data representative of a plurality of mapstreams corresponding to a plurality of drives from a plurality of vehicles. For example, the mapstreams 210 may be received for the map creation 106 process, where the mapstreams 210 may correspond to any number of drives from any number of vehicles 1500.”)
align the discrete trajectories to generate aligned geospatial observations through application of a geospatial offset for one or more of the discrete trajectories and concatenate the aligned geospatial observations to form aligned geospatial observations (See at least Kroepfl Paragraph 97 “Referring again to FIGS. 4 and 5A, the pairs of sections may then be registered to one another to generate pose links between them, which may be used for pose optimization 404. Registration 402 may be executed to determine the geometric relationships between multipole drives or mapstreams 210 corresponding thereto in many different locations where the drives or mapstreams 210 overlap—e.g., the portions 552A-552D of FIG. 5C. The output of the registration process may include relative pose links (e.g., rotation and translation between poses of a first mapstream frame or sections and poses of a second mapstream frame or sections) between pairs of frames or sections in different mapstreams. In addition to the relative poses, the pose links may further represent covariances representing the confidence in the respective pose link. The relative pose links are then used to align the maps 504 corresponding to each of the mapstreams 210 such that landmarks and other features—e.g., points clouds, LiDAR image maps, RADAR image maps, etc.—are aligned in a final, aggregate, HD map. As such, the registration process is executed by localizing one map 504—or portion thereof—to the other map 504—or portion thereof—of the pair. Registration 402 may be executed for each sensor modality and/or for each map layer corresponding to various sensor modalities.” | Paragraph 106 “The frame graph 600A is an example of a portion of a larger frame graph, where the larger frame graph may correspond to any portion—or all portions—of a road structure or layout. Four drives 602A-602D may have been registered together to generate pose links 608 (e.g., pose links 608A-608C) between poses 604 or positions of different drives 602. Each pose 604 or position of a single drive 602 may be represented by links 606. For example, the links 606 may represent a translation, rotation, and/or covariance from one pose 604 to a next pose 604. As such, link 606A(6) may represent a rotation, translation, and/or covariance between the poses 604 of the drive 602A that the link 606A(5) connects. Pose links 608A may correspond to the outputs of the registration process 402, and may encode a translation, rotation, and/or covariance between poses 604 of different drives 602. For example, pose link 608A may encode six degrees of freedom transformations between poses—e.g., the translation (e.g., difference in (x, y, z) location), rotation (e.g., difference in x, y, and z axis angles), and/or covariance (e.g., corresponding to the confidence in the values of the pose link) between pose 604A(1) of drive 602A and pose 604B(2) of the drive 602B”)
and provide for at least one of navigational assistance or at least semi-autonomous vehicle control based on the map geometries (See at least Kroepfl Paragraph 128 “The method 700, at block B714, includes transmitting data representative of the fused map to one or more vehicles for use in executing one or more operations.”)
Kroepfl fails to explicitly disclose the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is different for each of the one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, and process the concatenated, aligned geospatial observations using one or more Set Transformers; generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations.
Sakr teaches the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory (See at least Sakr Paragraph 14 “The method where the V2X wireless message further includes path history data describing multiple path history points of the remote vehicle, the remote road parameter data includes a lateral offset of the remote vehicle from a center of a reference lane, and generating the HD map data describing the real-time HD map further includes: generating one or more interpolated points based on the multiple path history points, where a lateral offset for each of the one or more interpolated points is estimated based on the lateral offset of the remote vehicle; and generating a path of the remote vehicle on the HD map based on the multiple path history points and the one or more interpolated points. The method where the remote road parameter data further includes one or more of a relative heading of the remote vehicle, curvature of the reference lane, and a curvature change rate of the reference lane. The method where the real-time HD map provides an estimate of a position of the remote vehicle that is accurate within plus or minus half a width of a lane on the road. The method where the remote GPS data describes a geographical location of the remote vehicle, and the ego GPS data describes a geographical location of the ego vehicle.” | Paragraph 178 “The map generation module 210 generates five interpolated points 607 between each two adjacent path history points using one or more interpolation techniques such as a linear interpolation technique. A lateral offset for each of the five interpolated points 607 is determined based on the lateral offsets of the two adjacent path history points. For example, if the two adjacent path history points 605B and 605C have a same lateral offset (the lateral offset 601B=the lateral offset 601C), then the five interpolated points 607 between the two adjacent path history points 605B and 605C may have the same lateral offset as the two adjacent path history points 605B and 605C.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the apparatus of Kroepfl to include the application of a geospatial offset for one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, as taught by Sakr as disclosed above, such that the geospatial offset is different for each of the one or more of the discrete trajectories, wherein the geospatial offset is the same across all geospatial observations of a respective discrete trajectory, in order to ensure accurate map geometries (Sakr Abstract “The disclosure includes embodiments for generating a real-time high-definition (HD) map for an ego vehicle using wireless vehicle data of a remote vehicle”).
Kroepfl in view of Sak fail to explicitly disclose to process the concatenated, aligned geospatial observations using one or more Set Transformers; generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations.
Cresswell, however, teaches to process the concatenated, aligned geospatial observations using one or more Set Transformer, and generate, from the one or more Set Transformers, map geometries including objects from the geospatial observations (See at least Cresswell FIGS. 3-7 and Paragraph 14 “When the attention mechanism is applied to map the alignment input to the alignment output the historical feature representations may be stored in slots of a slot-based memory and the alignment output may define an assignment of each current entity to a different one of the slots. In a slot-based memory the memory may be organized as a set of slots in which each slot is configured to store a feature representation. The feature representation stored in each slot may be updated using the current feature representation of the current entity assigned to the slot by the assignment output, e.g. using a slot-specific neural network.” | Paragraphs 94-117 “In general an attention mechanism maps a query and a set of key-value pairs to an output computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility or similarity function of the query with the corresponding key. The query, keys, and values are all vectors and each may be derived by transforming an object e.g. a feature representation or set of feature representations into a respective vector e.g. using a learned linear transform. The attention mechanism may be a self-attention mechanism in which case the query and the set of key-value pairs are all derived from the set of input objects i.e. input feature representations … In some cases, when the alignment outputs generated by the alignment neural network 120 are used to control an agent, the alignment input can also include data identifying the action performed by the agent at the preceding time point … FIG. 4 is a flow diagram of an example process 400 for generating an alignment output at a given time point. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, an alignment system, e.g., the alignment system 100 of FIG. 1 , appropriately programmed in accordance with this specification, can perform the process 400 … The system processes an alignment input that includes (i) the respective historical feature representations for the set of historical entities and (ii) the current feature representations for the set of current entities using an alignment neural network to generate an alignment output that defines, for each of one or more of the current entities, a corresponding historical entity that is the same as the current entity (step 406).”).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have modified the apparatus of Kroepfl in view of Sakr to include processing the concatenated, aligned geospatial observations using one or more Set Transformer, and generating, from the one or more Set Transformer, map geometries including objects from the geospatial observations, as taught by Cresswell as disclose above, in order to ensure an accurate map geometry for the navigational assistance (Cresswell Paragraph 4 “This specification describes a system implemented as computer programs on one or more computers in one or more locations that aligns entities, e.g., objects, across multiple image observations of an environment using a neural network”).
Claims 3 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Kroepfl (US 20210063200 A1) (“Kroepfl”) in view of Sakr (US 20200096539 A1) (“Sakr”) in view of Cresswell (US 2023026027 A1) (“Cresswell”) further in view of Cheng (“Exploring Dynamic Context for Multi-Path Trajectory Prediction”) (“Cheng”) (Attached).
With respect to claim 3, and similarly claim 12, Kroepfl in view of Sakr in view of Cresswell teach determining offsets for the trajectories in order to align the discrete trajectories (See at least Kroepfl Paragraph 106) (See at least Sakr Paragraph 14 | Paragraph 178)
Kroepfl in view of Sakr in view of Cresswell fail to explicitly disclose causing the apparatus to apply a multi-head self-attentional layer with a distance kernel defined as a Euclidean distance to the discrete trajectories.
Chen, however, teaches applying a multi-head self-attentional layer with a distance kernel defined as a Euclidean distance to the discrete trajectories (See at least Chen III C “The spatial-temporal context from both the observation time and prediction time are encoded by Encoder X and Y, respectively. Both encoders have the same two-stream structure: both streams consist of stacked self-attention layers; as illustrated in Fig. 2 one stream is followed by a global average pooling (GApool), while the other one is followed by an LSTM module. The upper stream is trained to learn motion information from the observed trajectory, whose input is the locations vector of the observed trajectory of the target agent Xi = {x 1 i ,··· ,x T i } ∈ R T×2 . The lower stream is trained to explore dynamic interactions among agents from the dynamic maps noted as DM = {O,S,P} ∈ R T×H×W×3 (discussed in Sec. III-B). For simplicity, we take the upper stream for illustration. T … To attend to different information from different representation subspaces jointly, the multi-head attention [24] strategy is applied as a conventional operation, where a head is an independent scaled dot-product attention module:” | III D “As shown in Fig. 2, during the training, both the observed trajectory Xi and its future trajectory Yi are encoded by Encoder X and Y (see Sec. III-C), respectively. Then, their encodings are concatenated and passed through two FC layers (each is followed by a ReLU activation) for fusion. Then, two side-by-side FC layers are used to estimate the mean µzi and the standard deviation σzi of the latent variables zi . A trajectory Yˆ i is reconstructed by an LSTM decoder step by step by taking zi and the encodings of observation as input. Because the random sampling process of zi can not be back propagated during training, the standard reparameterization trick [43] is adopted to make it differentiable. To minimize the error between the predicted trajectory Yˆ i and the ground truth Yi , the reconstruction loss is defined as the L2 loss (Euclidean distance). Thus, the whole network is trained by minimizing the loss function using the stochastic gradient descent method” | IV B “We adopt the most popular evaluation metrics: the mean average displacement error (ADE) and the final displacement error (FDE) to measure the trajectory prediction performance. ADE measures the aligned Euclidean distance from the prediction to its corresponding ground truth trajectory averaged over all steps. The mean value across all the trajectories is reported. FDE measures the Euclidean distance between the last position from the prediction to the corresponding ground truth position. In addition, the most-likely prediction is decided by the ranking method as described in Sec III-E. Compared with the ground truth (only if it is available), @top10 is the one out of ten predicted trajectories that has the smallest ADE and FDE”).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have modified the apparatus of Kroepfl in view of Sakr in view of Cresswell to include applying a multi-head self-attentional layer with a distance kernel defined as a Euclidean distance to the discrete trajectories, as taught by Cheng as disclosed above, for determining geospatial offsets for the one or more of the discrete trajectories in order to align the discrete trajectories, in order to ensure an accurate alignment (Cheng Introduction “It provides a novel framework to predict trajectories of heterogeneous agents (pedestrians, bicycles, vehicles, etc.) in various traffic situations, i.e., 20 different shared spaces and four intersections with mixed traffic.”).
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Kroepfl (US 20210063200 A1) (“Kroepfl”) in view of Sakr (US 20200096539 A1) (“Sakr”) in view of Cresswell (US 2023026027 A1) (“Cresswell”) further in view of Hunter (“The path inference filter: model-based low-latency map matching of probe vehicle data”) (“Hunter”) (Attached).
With respect to claim 4, and similarly claim 13, Kroepfl in view of Sakr in view of Cresswell teach concatenating the aligned geospatial trajectories (See at least Kroepfl Paragraph 97).
Kroepfl in view of Cresswell fail to explicitly disclose the plurality of sequences of geospatial observations from discrete trajectories include unique trajectory identifiers for each trajectory, wherein causing the apparatus to concatenate the aligned geospatial observations comprises causing the apparatus to remove the unique trajectory identifiers associated with the geospatial observations.
Hunter, however, teaches that the plurality of sequences of geospatial observations from discrete trajectories include unique trajectory identifiers for each trajectory, wherein causing the apparatus to concatenate the aligned geospatial observations comprises causing the apparatus to remove the unique trajectory identifiers associated with the geospatial observations (See at least Hunter IV E “The sequence of variables z represents the choices associated with a single trajectory, i.e., the concatenation of the x’s and p’s. In general, we will observe and would like to learn from multiple trajectories at the same time. This is why we need to consider a collection of variables (z(u) )u, which follows the given form and each can define a potential ψ(z(u) ; θ) and a partition function Z(u) (θ). There, the variable u indexes the set of sequences of observations, i.e., the set of consecutive GPS measurements of a vehicle. Since each of these trajectories will take place on a different portion of the road network, each of the sequences z(u) will have a different state space. For each of these sequences of variables z(u) , we observe the respective realizations z(u) (which correspond to the observation of a trajectory), and we wish to infer the parameter vector θ∗ that maximizes the likelihood of all the realizations of the trajectories where again the indexing u is for sets of measurements of a given trajectory. Similarly, the length of a trajectory is indexed by u : L(u) . From (18), it is clear that the log-likelihood function simply sums together the respective likelihood functions of each trajectory. For clarity, we consider a single sequence z(u) only, and we remove the indexing with respect to u. With this simplification, we have, for a single trajectory,”).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have modified the apparatus of Kroepfl in view of Sakr in view of Cresswell to include that the plurality of sequences of geospatial observations from discrete trajectories include unique trajectory identifiers for each trajectory, wherein causing the apparatus to concatenate the aligned geospatial observations comprises causing the apparatus to remove the unique trajectory identifiers associated with the geospatial observations, as taught by Hunter as disclosed above, in order to ensure a clear and efficient concatenated aligned geospatial observation (Hunter Abstract “We consider the problem of reconstructing vehicle trajectories from sparse sequences of GPS points, for which the sampling interval is between 1 s and 2 min.”).
Claims 5, 8-9, 14, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Kroepfl (US 20210063200 A1) (“Kroepfl”) in view of Sakr (US 20200096539 A1) (“Sakr”) in view of Cresswell (US 2023026027 A1) (“Cresswell”) further in view of Li (US 20210146963 A1) (“Li”).
With respect to claim 5, and similarly claim 14, Kroepfl in view of Sakr in view of Cresswell teach aligning the discrete trajectories to generate aligned geospatial observations (See at least Kroepfl Paragraph 97).
Kroepfl in view of Sakr in view of Cresswell fail to explicitly disclose apply a machine learning model to the plurality of geospatial observations of the sequences of geospatial observations for the discrete trajectories; determine, from the machine learning model, a displacement for each of the geospatial observations; average, for each discrete trajectory, the displacement for each of the geospatial observations in a respective discrete trajectory to generate an average displacement for each discrete trajectory; and apply the average displacement for each discrete trajectory to the respective discrete trajectory to align the discrete trajectories.
Li, however, teaches to apply a machine learning model to the geospatial observations of the sequences of the plurality of geospatial observations for the discrete trajectories; determine, from the machine learning model, a displacement for each of the geospatial observations; average, for each discrete trajectory, the displacement for each of the geospatial observations in a respective discrete trajectory to generate an average displacement for each discrete trajectory; and apply the average displacement for each discrete trajectory to the respective discrete trajectory to align the discrete trajectories (See at least Li Paragraphs 127-129 “The method 500 can include steps 505, 510, 515, 520, 525, 530, and 535 corresponding with 405, 410, 415, 420, 425, 430, and 435 described above with reference to FIG. 4. The method 500 can further include, at (540), updating one or more parameters of the interaction transformer model, the prediction model, and/or any other machine-learned models utilized (e.g., object detection model(s), attention model(s), recurrent model(s), interaction model(s), transformer model(s), etc. … In some implementations, hard negative mining can be employed. For object detection, the distance between BEV voxels and their closest ground-truth box centers can be used to determine positive and negative samples. Samples having distances smaller than a threshold can be considered as positive. Samples having distances larger than the threshold can be considered as negative. As a large proportion of the samples are negative in dense object detection, online hard negative mining can be employed. In some implementations, only the most difficult negative samples (with largest loss) can be kept and easy negative samples can be ignored. Classification loss can be averaged over both positive and negative samples while regression loss can be averaged over positive samples only. In some implementations, online association can be performed between detection results and ground truth labels to compute prediction loss. For each detection, the ground-truth box can be assigned with the maximum (oriented) intersection of union IoU. If a ground truth box is assigned to multiple detections, only the detection with maximum IoU can be kept while other detections are ignored. Regression on future motion can then be averaged over those detections with the associated ground-truth”).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have modified the apparatus of Kroepfl in view of Sakr in view of Cresswell so that the apparatus apply a machine learning model to the plurality of geospatial observations of the sequences of geospatial observations for the discrete trajectories; determine, from the machine learning model, a displacement for each of the geospatial observations; average, for each discrete trajectory, the displacement for each of the geospatial observations in a respective discrete trajectory to generate an average displacement for each discrete trajectory; and apply the average displacement for each discrete trajectory to the respective discrete trajectory to align the discrete trajectories, as taught by Li as disclosed above, in order to ensure accurate model outputs of the geospatial observations (Li Paragraph 2 “The present disclosure relates generally to controlling vehicles. In particular, the present disclosure is directed to systems and methods for generating motion forecast data for a plurality of actors with respect to an autonomous vehicle”).
With respect to claim 8, and similarly claim 17, Kroepfl in view of Sakr in view of Cresswell fail to explicitly disclose that the map geometries including objects from the geospatial observations are model outputs, wherein the apparatus is further caused to compute a loss for the model outputs for backpropagation-based model parameter training using a Chamfer loss function.
Li, however, teaches that the map geometries including objects from the geospatial observations are model outputs, wherein the apparatus is further caused to compute a loss for the model outputs for backpropagation-based model parameter training using a Chamfer loss function (See at least Li Paragraph 56 “Aspects of the present disclosure are directed to training one or more machine-learned models for generating motion forecast data for a plurality of actors with respect to an autonomous vehicle. The method can include evaluating a loss function that evaluates a difference between a ground truth associated with training data and the second respective trajectories for the plurality of actors and/or the respective projected trajectories for the second time step. In some implementations, the method can include adjusting one or more parameters of the interaction transformer model and/or the prediction model based on the evaluation of the loss function. As such, in some implementations, the two models can be trained in an “end-to-end configuration.” For example, errors can be sequentially back-propagated through each of the prediction model and the interaction transformer model to evaluate a joint loss function. A gradient of the joint loss function can be calculated to adjust the parameter(s) to reduce the joint loss function to jointly train the models” | Paragraph 128 “For example, the interaction transformer and prediction model(s) can be fully differentiable and thus can be trainable end-to-end through back-propagation. More particularly, a loss function can evaluate a difference between respective ground truth trajectories for the plurality of actors and the second respective trajectories for the plurality of actors at the second time step. As an example, detection loss can represent the weighted sum of object classification, detection box regression, and orientation classification losses based on differences between the associated ground truths and the second respective trajectories. Prediction loss can represent the box center and orientation regression, as well as orientation classification losses for all future prediction steps based on differences between the associated ground truths and the second respective trajectories. Prediction loss for proposal and refinement can also summed together. For example, binary cross entropy can be utilized as the classification loss and smooth L1-loss can be used as the regression loss”)
It would have been obvious to one of ordinary skill in the art before the effective filing date to have modified the apparatus of Kroepfl in view of Sakr in view of Cresswell so that the map geometries including objects from the geospatial observations are model outputs, wherein the apparatus is further caused to compute a loss for the model outputs for backpropagation-based model parameter training using a Chamfer loss function, as taught by Li as disclosed above, in order to ensure accurate model outputs of the geospatial observations (Li Paragraph 2 “The present disclosure relates generally to controlling vehicles. In particular, the present disclosure is directed to systems and methods for generating motion forecast data for a plurality of actors with respect to an autonomous vehicle”).
With respect to claim 9, and similarly claim 18, Kroepfl in view of Sakr in view of Cresswell in view of Li teach that the apparatus is further caused to remove duplicate output objects using clustering or domain-specific heuristics (See at least Kroepfl Paragraph 66 “In some examples, the system may minimize how often a trajectory point or frame is generated and/or included in the mapstream 210. For example, instead of including every frame in the mapstream 210, a distance threshold, a time threshold, or a combination thereof may be used to determine which frames to include in the mapstream 210. As such, if a certain distance (e.g., half a meter, one meter, two meters, five meters, ten meters, etc.) has been travelled by the vehicle 1500 and/or a certain amount of time has elapsed (e.g., half a second, a second, two seconds, etc.), a trajectory point or frame may be included in the mapstream. This distance or time thresholds may be used based on which is met first, or which is met last. For example, a first frame may be included in the mapstream 210, then a distance threshold may be met and a second frame at the distance threshold may be included in the mapstream 210. Once the second frame is included, the distance and time thresholds may be reset, and then a time threshold may be met and a third frame may be included in the mapstream 210, and so on. As a result, less duplicative data may be included in the mapstream 210”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM ABDOALATIF ALSOMAIRY whose telephone number is (571)272-5653. The examiner can normally be reached M-F 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Faris Almatrahi can be reached at 313-446-4821. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IBRAHIM ABDOALATIF ALSOMAIRY/ Examiner, Art Unit 3667 /KENNETH J MALKOWSKI/Primary Examiner, Art Unit 3667