Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings were received on December 21st 2023. These drawings are accepted.
Specification
The specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware of, in the specification.
Status of the Claims
This non-final action is in response to the applicant’s filing on December 21st 2023.
Claims 1-20 are pending and examined below.
Information Disclosure Statement
The information disclosure statement filed on October 19th 2022 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 5-13 and 15-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Analysis for claim 1:
In January, 2019 (updated October 2019), the USPTO released new examination guidelines setting forth a two-step inquiry for determining whether a claim is directed to non-statutory subject matter. According to the guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claim 1 is directed toward non-statutory subject matter, as shown below:
STEP 1: Does claim 1 fall within one of the statutory categories? Yes. The claim is directed mental process which falls within one of the statutory categories.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? Yes, the claim is directed to an abstract idea.
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
Claim 1
A method comprises:
obtaining, at a computing system coupled to a vehicle, sensor data representing an environment of the vehicle, wherein the sensor data is obtained from a sensor coupled to the vehicle while the vehicle is autonomously navigating a path in the environment; detecting, based on the sensor data, an unexpected issue impeding the vehicle from autonomously navigating the path;
generating a question based on the unexpected issue;
providing, by the computing system, the question and a portion of the sensor data used to detect the unexpected issue to a remote computing device, wherein the remote computing device inputs the question and the portion of the sensor data into a visual language model (VLM) trained to answer the question using the portion of the sensor data;
receiving, at the computing system, a response from the remote computing system; and generating, based on the response, a control strategy for controlling the vehicle.
The method in claim 1 is a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. Specifically, the limitations of claim 1 highlighted above merely consist of detecting/recognizing that there is an obstacle on the vehicle path and choosing a path to avoid the obstacle can be done with pen and paper. These steps are merely to determine if the change in direction due to construction or obstacle on the vehicle path. More specifically, a person can mentally decide on the available path to avoid obstacle. Thus, the claims recite a mental process.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? No, the claim does not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim 1
A method comprises:
obtaining, at a computing system coupled to a vehicle, sensor data representing an environment of the vehicle, wherein the sensor data is obtained from a sensor coupled to the vehicle while the vehicle is autonomously navigating a path in the environment; detecting, based on the sensor data, an unexpected issue impeding the vehicle from autonomously navigating the path;
generating a question based on the unexpected issue;
providing, by the computing system, the question and a portion of the sensor data used to detect the unexpected issue to a remote computing device, wherein the remote computing device inputs the question and the portion of the sensor data into a visual language model (VLM) trained to answer the question using the portion of the sensor data;
receiving, at the computing system, a response from the remote computing system; and generating, based on the response, a control strategy for controlling the vehicle.
Claim 1 does not recite any of the exemplary considerations that are indicative of an
abstract idea having been integrated into a practical application. The sensor data is recited at a high level of generality, i.e., as a generic generic computing device limitation is no more than mere a generic computer. Accordingly, this additional element does not integrate the
abstract idea into a practical application because it does not impose any meaningful limits on
practicing the abstract idea. The claim is directed to the abstract idea.
Still further, outputting the…data recited at a high level of generality (as merely the determined/observed information) and amounts to mere insignificant extra solution activity.
As such, the additional elements do not integrate the abstract idea into practical application.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No, the claim does not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Claim 1 does not recite any specific limitation or combination of limitations that are not well-understood, routine, conventional (WURC) activity in the field. Further, applicant’s specification does not provide any indication that the acquiring steps are performing using anything other than a conventional computer. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354-55, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016); (Fed. Cir. 2015) or indicate that mere performance of an action is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here).
CONCLUSION
Thus, since claim 1 is: (a) directed toward an abstract idea, (b) does not recite additional
elements that integrate the judicial exception into a practical application, and (c) does not
recite additional elements that amount to significantly more than the judicial exception, it is
clear that claim 1 is directed towards non-statutory subject matter.
Dependent claims 2-3 and 5-13 further limit the abstract idea without integrating the abstract idea into practical application or adding significantly more. For example, the limitations of claims 2 and 3 are further limitations that, under their broadest reasonable interpretation, covers
performance of the limitation in the mind using a similar analysis to claim 1 above.
With respect to independent claim 15, please see the rejection above with respect to claim 1 which is commensurate in scope to claim 15, with claim 1 being drawn to a method and claim 15 being drawn to a method as well.
Dependent claims 16-19 further limit the abstract idea without integrating the abstract idea into practical application or adding significantly more. For example, the limitations of
claims 14, 15 and 16 are further limitations that, under their broadest reasonable interpretation, covers
performance of the limitation in the mind using a similar analysis to claim 1 above.
With respect to independent claim 20, please see the rejection above with respect to claim 1 which is commensurate in scope to claim 20, with claim 1 being drawn to a method and claim 20 being drawn to a non-transitory computer-readable media.
As such, claims 1-3, 5-13 and 15-20 are rejected under 35 USC 101 as being drawn to an abstract idea without significantly more, and thus are ineligible.
Office Note: In order to overcome this rejection, the Office suggests further defining the limitations of the independent claims, for example linking the claimed subject matter to a non-generic device and controlling a vehicle as captured in dependent Claim 14. Limitations such as these suggested above would allow to overcome the rejection under 35 USC 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-11 and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar (Patent No. US20220281475A1) in view of Palai (Patent No. US20220270415A1).
Regarding claim 1 Kumar teaches a method comprises: obtaining, at a computing system coupled to a vehicle; (Kumar paragraph 0080; “…The operations of the exemplary method may be executed by any computing system, for example, by the vehicle guiding system 102…”); sensor data representing an environment of the vehicle; (See Kumar paragraph 0020; “…FIG. 1, a block diagram that illustrates an environment 100 for a vehicle guiding system 102 to maneuver vehicles using adjustable ultrasound sensors is depicted, in accordance with an embodiment. The environment 100 includes the vehicle guiding system 102, a sensor unit 104 that includes high range sensors 106 and ultrasound sensors 108…”); wherein the sensor data is obtained from a sensor coupled to the vehicle while the vehicle is autonomously navigating a path in the environment; (See Kumar paragraph 0009, 0019 and Figure 1; “ FIG. 1 is a block diagram that illustrates an environment for a vehicle guiding system to maneuver vehicles using adjustable ultrasound sensors, in accordance with an embodiment…the assigned ultrasound sensor(s) may be deployed on a rotatable mount to increase a Field of View (FOV) of the assigned ultrasound sensor(s). Exemplary aspects of the disclosure may be used for closer object detection, when the vehicle moves in a constrained space…”); detecting, based on the sensor data, an unexpected issue impeding the vehicle from autonomously navigating the path; (See Kumar paragraph 0019; “…the assigned ultrasound sensor(s) may be deployed on a rotatable mount to increase a Field of View (FOV) of the assigned ultrasound sensor(s). Exemplary aspects of the disclosure may be used for closer object detection, when the vehicle moves in a constrained space. In an embodiment, the vehicle guiding system may be configured to assist in providing navigational suggestions to a user of the vehicle or the vehicle itself to avoid mishaps…”); generating a question based on the unexpected issue; (See Kumar paragraph 0036-0037; “The I/O device 206 may include suitable logic, circuitry, and/or interfaces that may be configured to act as an I/O interface between a user (such as, a user associated with the vehicle 114 or the vehicle 114 itself) and the vehicle guiding system 102. The I/O device 206 may include various input and output devices, which may be configured to communicate with different operational components of the vehicle guiding system 102. The I/O device 206 may be configured to communicate data between the vehicle guiding system 102 and the application 110A of the UE 110 associated with the vehicle 114. In accordance with an embodiment, the I/O device 206 may be configured to output the vehicular trajectory plan data to a user device such as the UE 110 of FIG. 1.
In some embodiments, the vehicle guiding system 102 may be configured to provide an environment for development of parking strategy recommendation solutions for navigation systems in accordance with the embodiments disclosed herein. The environment may be accessed using the I/O interface 206 of the vehicle guiding system 102 disclosed herein…”); providing, by the computing system, the question and a portion of the sensor data used to detect the unexpected issue to a remote computing device; (See Kumar paragraph 0080-0082; “FIG. 7 is a flowchart that illustrates an exemplary method for maneuvering vehicles using adjustable ultrasound sensors, in accordance with an embodiment of the present disclosure. FIG. 7 is explained in conjunction with FIG. 1 to FIG. 6C. With reference to FIG. 7, there is shown a flowchart 700. The operations of the exemplary method may be executed by any computing system, for example, by the vehicle guiding system 102 of FIG. 1. The operations of the flowchart 700 may start at 702 and proceed to 704.
At 702, at least one first position of an object 118 may be determined relative to a vehicle 114 using at least one high range sensor 106. In accordance with an embodiment, the SODCTM 312 of the vehicle guiding system 102 may be configured to determine at least one first position of an object 118 relative to a vehicle 114 using at least one high range sensor 106.
At 704, determining for each of the at least one first position of the object 118 non-conformance with one or more object position criteria. In accordance with an embodiment, the SODCTM 312 may be configured to determine for each of the at least one first position of the object non-conformance with one or more object position criteria.”);
receiving, at the computing system, a response from the remote computing system; and generating, based on the response, a control strategy for controlling the vehicle; (See Kumar paragraph 0053-0054; “The arrangement of shifting of the ultrasound sensors 108 may happen continuously as the vehicle 114 moves on, the position of the object 118 changes with respect to a vehicle center and the position of the object 118 changes with respect to each ultrasound sensor from the ultrasound sensors 108. With this more precise and robust proximity information is received about the object 118, which the vehicle 114 needs to consider for its maneuvering operation.
The TPVDM 308 may be configured to generate different possible maneuvering trajectory routes for current position of the vehicle 114 up to a certain distance along a global path. The VCM 310 may be configured to provide controller command to vehicle motion and breaking actuator system of the vehicle 114.”).
Kumar does not explicitly teach but Palai teaches, wherein the remote computing device inputs the question and the portion of the sensor data into a visual language model (VLM) trained to answer the question using the portion of the sensor data; (See Palai paragraph 0018; “The control system may comprise a vehicle diagnostics manager (VDM) and a vehicle lifecycle manager (VLM). The VDM may be configured to initiate and control diagnostic conversations. The VLM may be configured to monitor and control the power state of the vehicle. The VDM may be configured to send a request to operate signal to the vehicle lifecycle manager upon receiving the request to initiate a diagnostic conversation. The request to operate signal may identify the one or more target participants that are required for participation in the requested diagnostic conversation. The VLM may be configured to grant or deny the request to operate in dependence on the state of the vehicle's energy storage system, and optionally further in dependence on the identities of the target participants.”).
Both Kumar and Palai are in the same field of vehicle remote assistance. It would have been obvious for one ordinary skilled in the art before the effective filing date of present invention to modify Kumar a computing system coupled to a vehicle with Palai visual language model (VLM). No new functionality would arise from the combination and the combination would improve usability of Kumar by adding visual language model (VLM), which will allow better image to text communication regarding vehicle environment. Further, finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 2, Kumar in view of Palai teaches the method of claim 1, Kumar does not teach but Palai teaches, wherein generating the question based on the unexpected issue comprises: generating a multiple choice question and at least two answer options based on the unexpected issue; and wherein providing the question and the portion of the sensor data used to detect the unexpected issue to the remote computing device comprises: providing the multiple choice question and the at least two answer options to the remote computing device; (See Palai paragraph 0057 and 0067; “…a request to initiate a non-intrusive diagnostic conversation that is received while the vehicle 1 is in an awake state, once the request has been accepted by the VLM 35, the VDM 34 is then free to initiate the requested diagnostic conversation with the target participants using the on-board tester 36 (or alternatively the off-board dongle tester 80, if present). The target participants are then able to communicate with the off-board backend server 100 as required in order for the non-intrusive diagnostic conversation to be completed.
…the VDM 34 will send a request to operate signal to the VLM 35 indicating that all legislative OBD ECUs 20 of the vehicle 1 are required for participation in a diagnostic conversation. The VLM 35 will then compare the list of target participants to the charge state and health state of the vehicle's battery system 60 in order to determine whether or not the on-board energy status of the vehicle 1 is sufficient to sustain a diagnostic conversation including all legislative OBD ECUs 20 of the vehicle 1. If it is determined that the on-board energy status is sufficient then the VLM 35 will energise all legislative OBD ECUs 20 of the vehicle 1 and latch the vehicle 1 into a diagnostics state before handing control over to the VDM 34 so that the requested diagnostic conversation can proceed as originally requested.”).
Both Kumar and Palai are in the same field of vehicle remote assistance. It would have been obvious for one ordinary skilled in the art before the effective filing date of present invention to modify Kumar a computing system coupled to a vehicle with Palai visual language model (VLM). No new functionality would arise from the combination and the combination would improve usability of Kumar by adding visual language model (VLM), which will allow better image to text communication regarding vehicle environment. Further, finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 3, Kumar in view of Palai teaches the method of claim 2, Kumar further teaches, wherein detecting the unexpected issue that impedes the vehicle from autonomously navigating the path comprises: determining that a confidence corresponding to an identification of an object located along the path is below a threshold confidence; (See Kumar paragraph 0083; “The one or more object position criteria may correspond to position of the object 118 relative to the vehicle 114. In this case, the one or more object position criteria may comprise a distance between the vehicle 114 and the object 118 being below a predefined distance. In accordance with an embodiment, assignment of the at least one ultrasound sensor may be based on the distance between the vehicle 114 and the object 118. The one or more object position criteria may correspond to visibility of the object 118 to the at least one high range sensor 106. In this case, the one or more object position criteria may include visibility of the object 118 to the at least one high range sensor 106 being above a predefined visibility threshold.”); and wherein generating the multiple choice question and the at least two answer options comprises: generating the multiple choice question that requests for the identification of the object; and generating a first answer option based on a first identification determined for the object by the computing system and a second answer option based on a second identification determined for the object by the computing system; (See Kumar paragraph 0022; ”The vehicle guiding system 102 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a first position of the object 118 relative to the vehicle 114 using one or more of high range sensors 106. The vehicle guiding system 102 may be configured to determine a second position of the object 118 relative to the vehicle 114 using one or more of ultrasound sensors 108. In accordance with an embodiment, the object 118 may correspond to a static object, such as, but not limited to, a pole, a guard rail, a lamp post, a signal pole, and a fixed wall. The vehicle guiding system 102 may be configured to assign at least one ultrasound sensor from the ultrasound sensors 108 to dynamically focus on the object 118 to determine at least one object attribute associated with the object 118. The vehicle guiding system 102 may be configured to maneuver the vehicle 114 on a trajectory plan based on the at least object attribute.”).
Regarding claim 4, Kumar in view of Palai teaches the method of claim 3, Kumar further teaches; wherein receiving the response from the remote computing system comprises: receiving a selection of the first answer option from the remote computing device; and wherein generating the control strategy for controlling the vehicle comprises: generating the control strategy based on the first identification determined for the object by the computing system; (See Kumar paragraph 0022; “The vehicle guiding system 102 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a first position of the object 118 relative to the vehicle 114 using one or more of high range sensors 106. The vehicle guiding system 102 may be configured to determine a second position of the object 118 relative to the vehicle 114 using one or more of ultrasound sensors 108. In accordance with an embodiment, the object 118 may correspond to a static object, such as, but not limited to, a pole, a guard rail, a lamp post, a signal pole, and a fixed wall. The vehicle guiding system 102 may be configured to assign at least one ultrasound sensor from the ultrasound sensors 108 to dynamically focus on the object 118 to determine at least one object attribute associated with the object 118. The vehicle guiding system 102 may be configured to maneuver the vehicle 114 on a trajectory plan based on the at least object attribute.”).
Regarding claim 5, Kumar in view of Palai teaches the method of claim 1, Kumar further teaches, wherein generating the question based on the unexpected issue comprises: generating a trajectory question that requests for one or more modifications to the path; and wherein providing the question and sensor data representing the unexpected issue to the remote computing device comprises: providing the trajectory question to the remote computing device; (See Kumar paragraph 0036; “The I/O device 206 may include suitable logic, circuitry, and/or interfaces that may be configured to act as an I/O interface between a user (such as, a user associated with the vehicle 114 or the vehicle 114 itself) and the vehicle guiding system 102. The I/O device 206 may include various input and output devices, which may be configured to communicate with different operational components of the vehicle guiding system 102. The I/O device 206 may be configured to communicate data between the vehicle guiding system 102 and the application 110A of the UE 110 associated with the vehicle 114. In accordance with an embodiment, the I/O device 206 may be configured to output the vehicular trajectory plan data to a user device such as the UE 110 of FIG. 1.”).
Regarding claim 6, Kumar in view of Palai teaches the method of claim 5, Kumar further teaches, wherein detecting the unexpected issue that impedes the vehicle from autonomously navigating the path comprises: detecting an obstacle blocking the path; (See Kumar paragraph 0026; “The sensor unit 104 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a position of an object (such as, the object 118) relative to a vehicle (such as, the vehicle 114) using high range sensors 106 or ultrasound sensors 108. The sensor unit 104 may include various types of sensors. For example, the sensor unit 104 may include an image capture device (such as, a camera) to capture the object 118. In accordance with an embodiment, the sensor unit 104 may be positioned on or within the vehicle 114. Examples of the high range sensors 106 in the sensor unit 104 may include, but are not limited to, stereoscopic depth cameras and a Light Detection and Ranging (LIDAR) sensor.”).
Regarding claim 7, Kumar in view of Palai teaches the method of claim 6, Kumar further teaches, further comprising: based on detecting the obstacle blocking the path, determining a first modified path and a second modified path, wherein navigating according to the first modified path or the second modified path enables the vehicle to circumvent the obstacle; and wherein providing the trajectory question to the remote computing device comprises: providing a first answer option representing the first modified path and a second answer option representing the second modified path along with the trajectory question to the remote computing device; (See Kumar paragraph 0022; “The vehicle guiding system 102 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a first position of the object 118 relative to the vehicle 114 using one or more of high range sensors 106. The vehicle guiding system 102 may be configured to determine a second position of the object 118 relative to the vehicle 114 using one or more of ultrasound sensors 108. In accordance with an embodiment, the object 118 may correspond to a static object, such as, but not limited to, a pole, a guard rail, a lamp post, a signal pole, and a fixed wall. The vehicle guiding system 102 may be configured to assign at least one ultrasound sensor from the ultrasound sensors 108 to dynamically focus on the object 118 to determine at least one object attribute associated with the object 118. The vehicle guiding system 102 may be configured to maneuver the vehicle 114 on a trajectory plan based on the at least object attribute.”).
Regarding claim 8, Kumar in view of Palai teaches the method of claim 7, Kumar also teaches, further comprising: determining that a confidence associated with navigating according the first modified path or the second modified path is below a threshold confidence; (See Kumar paragraph 0083; “The one or more object position criteria may correspond to position of the object 118 relative to the vehicle 114. In this case, the one or more object position criteria may comprise a distance between the vehicle 114 and the object 118 being below a predefined distance. In accordance with an embodiment, assignment of the at least one ultrasound sensor may be based on the distance between the vehicle 114 and the object 118. The one or more object position criteria may correspond to visibility of the object 118 to the at least one high range sensor 106. In this case, the one or more object position criteria may include visibility of the object 118 to the at least one high range sensor 106 being above a predefined visibility threshold.”); and wherein providing data representing the first modified path and the second modified path along with the trajectory question to the remote computing device comprises: providing the data representing the first modified path and the second modified path along with the trajectory question to the remote computing device in response to determining that the confidence associated with navigating according to the first modified path or the second modified path is below the threshold confidence; (See Kumar paragraph 0022; ”The vehicle guiding system 102 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a first position of the object 118 relative to the vehicle 114 using one or more of high range sensors 106. The vehicle guiding system 102 may be configured to determine a second position of the object 118 relative to the vehicle 114 using one or more of ultrasound sensors 108. In accordance with an embodiment, the object 118 may correspond to a static object, such as, but not limited to, a pole, a guard rail, a lamp post, a signal pole, and a fixed wall. The vehicle guiding system 102 may be configured to assign at least one ultrasound sensor from the ultrasound sensors 108 to dynamically focus on the object 118 to determine at least one object attribute associated with the object 118. The vehicle guiding system 102 may be configured to maneuver the vehicle 114 on a trajectory plan based on the at least object attribute.”).
Regarding claim 9, Kumar in view of Palai teaches the method of claim 7, Kumar also teaches, wherein receiving the response from the remote computing system comprises: receiving a selection of the first answer option from the remote computing system; and wherein generating the control strategy for controlling the vehicle comprises: generating the control strategy based on the first modified path; (See Kumar paragraph 0022 and 0047-0048; “The vehicle guiding system 102 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a first position of the object 118 relative to the vehicle 114 using one or more of high range sensors 106. The vehicle guiding system 102 may be configured to determine a second position of the object 118 relative to the vehicle 114 using one or more of ultrasound sensors 108. In accordance with an embodiment, the object 118 may correspond to a static object, such as, but not limited to, a pole, a guard rail, a lamp post, a signal pole, and a fixed wall. The vehicle guiding system 102 may be configured to assign at least one ultrasound sensor from the ultrasound sensors 108 to dynamically focus on the object 118 to determine at least one object attribute associated with the object 118. The vehicle guiding system 102 may be configured to maneuver the vehicle 114 on a trajectory plan based on the at least object attribute…
The NIM 302 may be configured to initiate navigation process (such as, path planning, velocity generation, and autonomous drive) from a source location to a destination location for the vehicle 114. The NIM 302 may correspond to a UI layer to the vehicle guiding system 102. In accordance with an alternate embodiment, the I/O device 206 may be used to initiate the navigation process from the source location to the destination location for the vehicle 114. In accordance with an embodiment, a user may enter the destination location using the NIM 302 of the vehicle guiding system 102. In accordance with an embodiment, a user may enter the destination location, via the UI 110B of the application 110A installed in the UE 110 and the vehicle guiding system 102 may receive the destination location using the network interface 208, via the communication network 112.
The BRPM 304 may be configured to generate a base path for navigation related function for the vehicle 114 from a current vehicle position to the destination location. In accordance with an embodiment, the base path may be generated using algorithms, such as, but not limited to, Dijkstra A* or any other path planning algorithm on a 2D occupancy grid map.”).
Regarding claim 10, Kumar in view of Palai teaches the method of claim 1, Kumar further teaches, wherein obtaining sensor data representing the environment of the vehicle from the sensor coupled to the vehicle comprises: receiving images from a camera coupled to the vehicle; and wherein detecting, based on the sensor data, the unexpected issue impeding the vehicle from autonomously navigating the path comprises: detecting, based on the images, an obstacle impeding the vehicle from autonomously navigating the path; (See Kumar paragraph 0026 and 0055; “The sensor unit 104 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a position of an object (such as, the object 118) relative to a vehicle (such as, the vehicle 114) using high range sensors 106 or ultrasound sensors 108. The sensor unit 104 may include various types of sensors. For example, the sensor unit 104 may include an image capture device (such as, a camera) to capture the object 118. In accordance with an embodiment, the sensor unit 104 may be positioned on or within the vehicle 114. Examples of the high range sensors 106 in the sensor unit 104 may include, but are not limited to, stereoscopic depth cameras and a Light Detection and Ranging (LIDAR) sensor.
The VLM 318 may be configured to provide current position of the vehicle 114 on a map. In accordance with an embodiment, the VLM 318 may use a camera-based location identification by observing unique landmarks and fetching previous observation data record from a known position.”).
Regarding claim 11, Kumar in view of Palai teaches the method of claim 1, Kumar further teaches, wherein obtaining sensor data representing the environment of the vehicle from the sensor coupled to the vehicle comprises: obtaining images representing an interior environment of the vehicle from a camera coupled to the vehicle; (See Kumar paragraph 0049; “The PM 306 may be configured to analyze sensor data from the sensor unit 104. The PM 306 may be configured to prepare a comprehensive data about an environment for motion planning and localization of the vehicle 114.”).
Regarding claim 14, Kumar in view of Palai teaches the method of claim 1, Kumar also teaches, further comprising: based on detecting the unexpected issue, causing the vehicle to pull over or remain stationary; monitoring the unexpected issue using subsequent sensor data; and causing the vehicle to proceed along the path based on receiving the response from the remote computing system or detecting a change to the unexpected issue; (See Kumar paragraph 0088;” At 710, the vehicle 114 may be maneuvered on a trajectory plan based on the at least one object attribute. In accordance with an embodiment, the TPVDM 308 may be configured to maneuver the vehicle 114 on a trajectory plan based on the at least object attribute. In accordance with an embodiment, maneuvering the vehicle 114 may comprise generating one or more maneuvering trajectory routes.”).
Regarding claim 15 Kumar teaches, a method comprising: receiving, at a computing system; (Kumar paragraph 0080; “…The operations of the exemplary method may be executed by any computing system, for example, by the vehicle guiding system 102…”); a question and sensor data from a vehicle, wherein the question corresponds to an unexpected issue impeding the vehicle from autonomously navigating a path and the sensor data represents the unexpected issue;(See Kumar paragraph 0019-0020; “…the assigned ultrasound sensor(s) may be deployed on a rotatable mount to increase a Field of View (FOV) of the assigned ultrasound sensor(s). Exemplary aspects of the disclosure may be used for closer object detection, when the vehicle moves in a constrained space. In an embodiment, the vehicle guiding system may be configured to assist in providing navigational suggestions to a user of the vehicle or the vehicle itself to avoid mishaps……FIG. 1, a block diagram that illustrates an environment 100 for a vehicle guiding system 102 to maneuver vehicles using adjustable ultrasound sensors is depicted, in accordance with an embodiment. The environment 100 includes the vehicle guiding system 102, a sensor unit 104 that includes high range sensors 106 and ultrasound sensors 108…”).
Kumar does not explicitly teach but Palai teaches, providing the question and the sensor data as inputs into a visual language model (VLM) to generate an output that addresses the question; and transmitting, by the computing system and to the vehicle, a response based on the output that addresses the question; (See Palai paragraph 0018; “The control system may comprise a vehicle diagnostics manager (VDM) and a vehicle lifecycle manager (VLM). The VDM may be configured to initiate and control diagnostic conversations. The VLM may be configured to monitor and control the power state of the vehicle. The VDM may be configured to send a request to operate signal to the vehicle lifecycle manager upon receiving the request to initiate a diagnostic conversation. The request to operate signal may identify the one or more target participants that are required for participation in the requested diagnostic conversation. The VLM may be configured to grant or deny the request to operate in dependence on the state of the vehicle's energy storage system, and optionally further in dependence on the identities of the target participants.”).
Both Kumar and Palai are in the same field of vehicle remote assistance. It would have been obvious for one ordinary skilled in the art before the effective filing date of present invention to modify Kumar a computing system coupled to a vehicle with Palai visual language model (VLM). No new functionality would arise from the combination and the combination would improve usability of Kumar by adding visual language model (VLM), which will allow better image to text communication regarding vehicle environment. Further, finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 16 Kumar in view of Palai teaches a method of claim 15, Kumar further teaches, wherein receiving the question and sensor data from the vehicle comprises: receiving a multiple choice question and at least two answer options from the vehicle; (See Kumar paragraph 0022; ”The vehicle guiding system 102 may include suitable logic, circuitry, interfaces, and/or code that may be configured to determine a first position of the object 118 relative to the vehicle 114 using one or more of high range sensors 106. The vehicle guiding system 102 may be configured to determine a second position of the object 118 relative to the vehicle 114 using one or more of ultrasound sensors 108. In accordance with an embodiment, the object 118 may correspond to a static object, such as, but not limited to, a pole, a guard rail, a lamp post, a signal pole, and a fixed wall. The vehicle guiding system 102 may be configured to assign at least one ultrasound sensor from the ultrasound sensors 108 to dynamically focus on the object 118 to determine at least one object attribute associated with the object 118. The vehicle guiding system 102 may be configured to maneuver the vehicle 114 on a trajectory plan based on the at least object attribute.”).
Regarding claim 17 Kumar in view of Palai teaches a method of claim 16, Kumar does not teach but Palai teaches, wherein providing the question and the sensor data as inputs into the VLM to generate the output that addresses the question comprises: providing the multiple choice question, the at least two answer options, and the sensor data as inputs into the VLM; and generating the output that selects a first answer option from the at least two answer options; (See Palai paragraph 0057 and 0067; “…a request to initiate a non-intrusive diagnostic conversation that is received while the vehicle 1 is in an awake state, once the request has been accepted by the VLM 35, the VDM 34 is then free to initiate the requested diagnostic conversation with the target participants using the on-board tester 36 (or alternatively the off-board dongle tester 80, if present). The target participants are then able to communicate with the off-board backend server 100 as required in order for the non-intrusive diagnostic conversation to be completed.
…the VDM 34 will send a request to operate signal to the VLM 35 indicating that all legislative OBD ECUs 20 of the vehicle 1 are required for participation in a diagnostic conversation. The VLM 35 will then compare the list of target participants to the charge state and health state of the vehicle's battery system 60 in order to determine whether or not the on-board energy status of the vehicle 1 is sufficient to sustain a diagnostic conversation including all legislative OBD ECUs 20 of the vehicle 1. If it is determined that the on-board energy status is sufficient then the VLM 35 will energise all legislative OBD ECUs 20 of the vehicle 1 and latch the vehicle 1 into a diagnostics state before handing control over to the VDM 34 so that the requested diagnostic conversation can proceed as originally requested.”).
Both Kumar and Palai are in the same field of vehicle remote assistance. It would have been obvious for one ordinary skilled in the art before the effective filing date of present invention to modify Kumar a computing system coupled to a vehicle with Palai visual language model (VLM). No new functionality would arise from the combination and the combination would improve usability of Kumar by adding visual language model (VLM), which will allow better image to text communication regarding vehicle environment. Further, finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 18 Kumar in view of Palai teaches a method of claim 15, Kumar further teaches wherein receiving the question and sensor data from the vehicle comprises: receiving, from the vehicle, a trajectory question and sensor data representing a position of an obstacle, wherein the obstacle corresponds to the unexpected issue; (See Kumar paragraph 0036; “The I/O device 206 may include suitable logic, circuitry, and/or interfaces that may be configured to act as an I/O interface between a user (such as, a user associated with the vehicle 114 or the vehicle 114 itself) and the vehicle guiding system 102. The I/O device 206 may include various input and output devices, which may be configured to communicate with different operational components of the vehicle guiding system 102. The I/O device 206 may be configured to communicate data between the vehicle guiding system 102 and the application 110A of the UE 110 associated with the vehicle 114. In accordance with an embodiment, the I/O device 206 may be configured to output the vehicular trajectory plan data to a user device such as the UE 110 of FIG. 1.”).
Regarding claim 19 Kumar in view of Palai teaches a method of claim 18, Kumar further teaches wherein providing the question and the sensor data as inputs into the VLM to generate the output that addresses the question comprises: providing the trajectory question and the sensor data representing the position of the obstacle as inputs into the VLM; and generating a given output that indicates a modified path for the vehicle to navigate to circumvent the obstacle; (See Kumar paragraph 0054-0055; “The TPVDM 308 may be configured to generate different possible maneuvering trajectory routes for current position of the vehicle 114 up to a certain distance along a global path. The VCM 310 may be configured to provide controller command to vehicle motion and breaking actuator system of the vehicle 114.
The VLM 318 may be configured to provide current position of the vehicle 114 on a map. In accordance with an embodiment, the VLM 318 may use a camera-based location identification by observing unique landmarks and fetching previous observation data record from a known position.”).
With respect to independent claim 20, please see the rejection above with respect to claim 1 which is commensurate in scope to claim 20, with claim 1 being drown to a method, and claim 20 being drawn to corresponding a non-transitory computer-readable media.
Claims 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar (Patent No. US20220281475A1) in view of Palai (Patent No. US20220270415A1) and Golston (Patent No. US20180251122A1).
Regarding claim 12, Kumar in view of Palai teaches the method of claim 11, Kumar does not teach but Golston teaches, wherein detecting the unexpected issue impeding the vehicle from autonomously navigating the path comprises: detecting an item located inside the vehicle after a passenger exited the vehicle; and wherein generating the question based on the unexpected issue comprises: generating a particular question that requests whether the vehicle includes any items left behind by the passenger; (See Golston paragraph 0066;”… Some configurations may detect that one or more occupants are complaining about temperature and adjust temperature control (e.g., heating, air conditioning, HVAC, etc.). Some configurations may detect bad driving conditions and adjust lane or speed. Some configurations may detect that an occupant needs to read and adjust in-cabin lighting. Some configurations may detect that an occupant is forgetting their luggage and alert the occupant to ensure that the occupant gets his bag before leaving. Some configurations may detect that an occupant is saying they need to change their destination based on an emergency or otherwise change plans (e.g., no longer will reach the location in time, a high-priority interrupt has caused a change in plans, etc.). Some configurations may detect that one occupant has an urgent medical situation and may reroute the plan and call an emergency service (e.g., police, ambulance, 911, etc.). Some configurations may recognize emotions (e.g., fear for a passenger, neutral emotion for a driver, etc.) to determine whether to take any action (e.g., change lane). Some configurations may detect and/or determine that a child has been left behind in a vehicle.”).
Both Kumar and Golston are in the same field of vehicle remote assistance. It would have been obvious for one ordinary skilled in the art before the effective filing date of present invention to modify Kumar a computing system coupled to a vehicle with Golston whether the vehicle includes any items left behind by the passenger. No new functionality would arise from the combination and the combination would improve usability of Kumar by adding whether the vehicle includes any items left behind by the passenger, which helps with passenger not leaving their items in the vehicle. Further, finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 13, Kumar in view of Palai teaches the method of claim 12, Kumar does not teach but Golston teaches, wherein receiving the response from the remote computing system comprises: receiving a given response that confirms the vehicle includes the item left behind by the passenger; and wherein generating the control strategy for controlling the vehicle comprises: generating the control strategy based on the given response; (See Golston paragraph 0066;”… Some configurations may detect that one or more occupants are complaining about temperature and adjust temperature control (e.g., heating, air conditioning, HVAC, etc.). Some configurations may detect bad driving conditions and adjust lane or speed. Some configurations may detect that an occupant needs to read and adjust in-cabin lighting. Some configurations may detect that an occupant is forgetting their luggage and alert the occupant to ensure that the occupant gets his bag before leaving. Some configurations may detect that an occupant is saying they need to change their destination based on an emergency or otherwise change plans (e.g., no longer will reach the location in time, a high-priority interrupt has caused a change in plans, etc.). Some configurations may detect that one occupant has an urgent medical situation and may reroute the plan and call an emergency service (e.g., police, ambulance, 911, etc.). Some configurations may recognize emotions (e.g., fear for a passenger, neutral emotion for a driver, etc.) to determine whether to take any action (e.g., change lane). Some configurations may detect and/or determine that a child has been left behind in a vehicle.”).
Both Kumar and Golston are in the same field of vehicle remote assistance. It would have been obvious for one ordinary skilled in the art before the effective filing date of present invention to modify Kumar a computing system coupled to a vehicle with Golston whether the vehicle includes any items left behind by the passenger. No new functionality would arise from the combination and the combination would improve usability of Kumar by adding whether the vehicle includes any items left behind by the passenger, which helps with passenger not leaving their items in the vehicle. Further, finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LIDIA KWIATKOWSKA whose telephone number is (571)272-5161. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott A. Browne can be reached at (571) 270-0151. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/L.K./Examiner, Art Unit 3666
/SCOTT A BROWNE/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights