DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 3 June 2026 have been fully considered but they are persuasive only in part.
First, while applicant indicates that the ADS has been revised, no revised ADS is seen in the USPTO electronic application record. Accordingly, the examiner has initialed the BIB DATA SHEET. (The ADS may still be corrected or updated, if prudently desired, until payment of the issue fee by a new application data sheet providing corrected or updated information. See e.g., 37 CFR 1.76.)
Second, in view of applicant’s arguments, the drawing objection is withdrawn.
Third, the claim objections have been overcome by applicant’s claim amendments.
Fourth, applicant’s claim amendments largely overcome the previous rejections under 35 U.S.C. 112(b), with new issues under 35 U.S.C. 112 in the amended claim language and new claims being dealt with below. (The only previous issue which has not been overcome is the labeling of the invalid instruction in claims 7 and 13, which applicant has apparently not particularly addressed in the Remarks.)
In this respect, a new issue arises under 35 U.S.C. 112(b) regarding the newly claimed “scene graph(s)” which applicant asserts is “more contextually appropriate” than “scene map”. However, 37 CFR 1.75(d)(1) requires that, “The claim or claims must conform to the invention as set forth in the remainder of the specification and the terms and phrases used in the claims must find clear support or antecedent basis in the description so that the meaning of the terms in the claims may be ascertainable by reference to the description. (See § 1.58(a).)” and MPEP 608.01(o) indicates that, “amendments to the claims already in the application should be scrutinized not only for new matter but also for new terminology” The examiner is unaware of any reason for changing “scene map” to “scene graph”, and believes a “scene graph”[1] must be different from a “scene map”, with neither the specification nor an EPO machine translation of the priority application (attached) mentioning any “scene graph” (e.g., the priority document translation refers to both scene maps and scene diagrams, with the examiner having no difficulty interpreting either of those priority document translation phrases.)
Fifth, applicant’s arguments regarding the patentability of the claims under 35 U.C.C. 103 are not convincing.
In this respect, applicant argues:
As described above, in McNew, the candidate driving maneuver includes slowing down, speeding up, lane changes or takeovers. Therefore, the candidate driving maneuver is not an input command but a command for the vehicle to perform actions. In other words, in McNew, the candidate driving maneuver can enable the vehicle to perform actions directly, However, in amended claim 1, the pending voice command refers to user voice commands recognizable by the vehicle, but the pending voice command can't enable the vehicle to perform actions directly, therefore, the candidate driving maneuver in McNew is not equal to the pending voice command in amended claim 1. Therefore, McNew fails to disclose the above highlighted features (A).
Applicant’s arguments are apparently not commensurate with the claim scope that does not require that “the pending voice command can't enable the vehicle to perform actions directly”. Moreover, McNew (‘610) explicitly teaches performing the driving maneuver (at 224 in FIG. 2) after confirmation (confirmation instruction) by the user (at 222). Accordingly, applicant’s arguments are not persuasive.
Next, applicant argues:
In contrast [to McNew], the amended Claim 1 recites that "displaying the first scene graph to prompt a user to confirm or modify a pending voice command SO as to correspondingly generate a confirmation instruction or an invalid instruction". It can be seen that the visual simulation defined in the amended Claim 1 is applicable to all pending voice commands, rather than being limited to risky commands only. Besides, the purpose of the visual simulation in the amended Claim 1 is to allow the user to verify the rationality and feasibility of pending commands and judge whether there are errors therein.
Again, applicant’s arguments are not commensurate with the claim scope that does not require that claim 1 is “applicable to all pending voice commands”, and for anticipation/obviousness, the examiner apparently needs to show only a single “pending voice command” that satisfied the claim limitations, by the wording of the claim. McNew (‘610) therefore shows the claim limitations e.g., for the risky maneuver, at 222, Yes or No in FIG. 2, and at 212 and/or 224.
Accordingly, applicant’s arguments are not persuasive.
Priority, ADS, and BIB DATA SHEET
Receipt is again acknowledged of the certified copy of Chinese application number 202311215928.7, with a decimal point before the “7”, filed on 20 September 2023, e.g., obtained through the PDX program.
In this respect, the examiner notes that the (previously submitted) ADS claims priority to application number “2023112159287”, without the decimal point. It is unclear to the examiner what the legal effect(s) of the (decimal point) discrepancy between the actual application number and the application number to which priority is claimed in the ADS may be, if there is/are any. Applicant is encouraged (though not required) to prudently correct the ADS in this respect, to eliminate the (decimal point) discrepancy and any concomitant legal effects/ramifications/consequences that the examiner cannot foresee.
The BIB DATA SHEET has now be initialed by the examiner.
Drawings
The drawings were received on 20 September 2024. These drawings are accepted by the examiner.
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o)2. Correction of the following is required: antecedent basis for the new claim terminology, i) “current scene graph”, “first scene graph”, and “second scene graph” of the amended independent and dependent claims, and ii) “allowing the user to [confirm and verify] whether the voice command is accurately recognized by the vehicle based on the first scene graph” of new claims 15 and 16, should be provided in the specification, without adding new matter.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 15 and 16 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding new claim 15 and 16, applicant has apparently not described, in sufficient detail, by what algorithm(s)3, or by what steps or procedure, he “allow[ed] the user” to confirm or verify “whether the voice command is accurately recognized by the vehicle”. No such accuracy of the recognition, or confirmation/verification of such by the user is apparently described, or even allowing the user to make such a confirmation/verification of any recognition accuracy is apparently described. Accordingly, the examiner believes applicant has not demonstrated, to those skilled in the art, possession of the (now) claimed invention, but has instead only (now) described a desired result.
In this respect, the disclosed/described invention apparently allow the user to confirm (or modify/correct) a pending voice command based on a scene map [e.g., paragraphs [0005], [0007], [0043], [0044], [0085], etc.), or even to confirm the feasibility of the input voice command (or that the voice command is not feasible) through the first scene map (published paragraph [0051], [0059], [0074], etc.) However, it has not been previously described that (or by what algorithm(s)) the user was somehow “allow[ed]” to confirm or verify whether the “voice command [was] accurately recognized by the vehicle”. Accordingly, the examiner believes applicant has not demonstrated, to those skilled in the art, possession of the (now) claimed invention, but has instead only (now) described a desired result.
Claims 1 to 16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Throughout the claim set, and e.g., as introduced at claim 1, line 5, claim 3, line 5, claim 8, line 10, claim 10, line 4, etc., and in light of the lack of antecedent basis for the new claim terminology in the specification, all references to “scene graph[s]” (e.g., current scene graph, first scene graph, second scene graph, etc.) are indefinite and not reasonably certain4 because i) no scene graph is apparently described in the description, and ii) the examiner cannot determine what the intended difference between the currently claimed “scene graph”[5] and a “scene map’ (as previously claimed) might possibly be with reasonable certainty, while understanding there MUST be a difference, or applicant would not have (logically) made the amendment to change the claim terminology. Accordingly, the metes and bounds of the “scene graph[s]” as are now claimed cannot be determined with reasonable certainty, when the claims are read in light of the specification.
In claim 1, line 13, “to confirm whether the pending command is erroneous” is indefinite when read in light of the specification that does not define or use “erroneous”, nor define its metes and bounds (e.g., is “erroneous” in the claim broader or narrower than “infeasible”?)
In claim 2, lines 1ff, and in claim 9, lines 1ff, “wherein constructing the current scene graph comprising:” is grammatically incorrect and unclear[6] (e.g., does applicant intend, “the current scene graph comprises:”?)
In claim 7, lines 3ff, and in claim 14, lines 4ff, “labeling the invalid instruction” is fully indefinite (e.g., “labeling” the “invalid instruction” in what way or with/on what instrumentality particularly?)
Claim 13 is unclear in its entirety, with “wherein further comprises” in lines 1ff of the preamble being grammatically incorrect and confusing, and with the body of the claim being indefinite for apparently incorporating method steps (planning, detecting, labeling) into an apparatus claim. See MPEP 2173.05(p), II. (“A single claim which claims both an apparatus and the method steps of using the apparatus is indefinite under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. See In re Katz Interactive Call Processing Patent Litigation, 639 F.3d 1303, 1318, 97 USPQ2d 1737, 1748-49 (Fed. Cir. 2011).”)
Claims 15 and 16 are indefinite in their entireties for apparently being undescribed in the specification, with the meaning of the new claim terminology not being ascertainable by reference to the description (37 CFR 1.75(d)(1)). For example, what does it mean that the “voice command is accurately recognized by the vehicle” in applicant’s invention when the specification apparently references nor defines any such accuracy related to recognition outside of the Background of the Invention, and the claimed “accuracy” would apparently be facially subjective (MPEP 2173.05(b), IV.)? And how can the user “confirm” or “verify” such a subjective “accura[cy]” in/of the recognition by the vehicle when the claimed “accuracy” would apparently be facially subjective (MPE 2173.05(b), IV.), with no objective standard provided in the specification for either the reader (of the claim) or the user (of the vehicle, in order to make the claimed confirmation/verification) to measure the scope of the term in order to be able to confirm/verify whether the voice command is “accurately” recognized by the vehicle or not? This is unclear.
Claim(s) depending from claims expressly noted above are also rejected under 35 U.S.C. 112 by/for reason of their dependency from a noted claim that is rejected under 35 U.S.C. 112, for the reasons given.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 8, 9, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over McNew (2018/0029610) in view of Jafari et al. (2025/0091603).
McNew (‘610) reveals:
per claim 1, a voice-based control method for a vehicle, comprising:
when receiving a voice command [e.g., the verbal input at paragraph [0057] for identifying the candidate driving maneuver; e.g., obtained from the microphones 50 and used at step 204 in FIG. 2 to identify (obviously recognize) requests from the user], recognizing the voice command to obtain a pending voice command [e.g., for the identified driving maneuver that will be used in FIG. 2, to be rejected or performed];
planning vehicle actions of the vehicle based on the pending voice command [e.g., the driving plan generated at step 206 in FIG. 2];
constructing a current scene graph of the vehicle to obtain a first scene graph [e.g., FIG. 5],
displaying the first scene graph to prompt user to confirm or modify the pending voice command to correspondingly generate a confirmation instruction or an invalid instruction [e.g., paragraphs [0091]ff, etc., “For example, the queries may include, before or contemporaneously with the visual simulations of the driving plans, introductory descriptions of the predicate determinations that the overall risk of performing their driving maneuvers renders performing the driving maneuvers feasible, but with some risk, as well as descriptions that, because of the risk of their performance, the driving maneuvers are, although acceptable, only provisionally accepted. In cases of identification of the driving maneuvers by the vehicle 10, the queries may further include descriptions that offers are being made by the vehicle 10, to the user, to perform the driving maneuvers. In cases of identification of the driving maneuvers from user requests, the queries may further include descriptions that the user requests for the vehicle 10 to perform the driving maneuvers are only provisionally granted. The queries may then follow up on these descriptions with the questions of whether the user confirms the driving maneuvers notwithstanding the risk of their performance. . . . The queries are output to the user at the various interfaces implemented by the components of the audio/video system 46. . . . Similarly, the planning/decision making module 94 may, for instance, identify the user responses to queries for confirmation of the driving maneuvers from input signals transformed from corresponding mechanical inputs detected by touch screens in the displays 54. . . . If the user is not comfortable with a driving maneuver because of the risk of its performance, or otherwise, the user response to a query for confirmation of the driving maneuver could be that the user does not confirm the driving maneuver. . . . However, if a user is comfortable with a driving maneuver notwithstanding the risk of its performance, their user response to a query for confirmation of the driving maneuver could be that the user confirms the driving maneuver.”], wherein the first scene graph is configured to present, in graphical form [e.g., paragraph [0105], “An example visual simulation of a driving plan for performing a takeover is represented in FIG. 5 as a conceptual rendering of visual outputs at the surfaces of the displays 54 of the audio/video system 46 at a sequence of times (a)-(g)”], a scenario in which the vehicle is about to perform the vehicle actions [e.g., paragraph [0105], “the driving plan for performing the takeover may be visually simulated. An example visual simulation of a driving plan for performing a takeover is represented in FIG. 5 as a conceptual rendering of visual outputs at the surfaces of the displays 54 of the audio/video system 46 at a sequence of times (a)-(g). Since performing the takeover includes the performance of other driving maneuvers, such as slowing down, speeding up and lane changes, and combinations of these, it will be understood that the represented example visual simulation for performing the takeover is applicable in principle to visual simulations of driving plans for performing the included or any other driving maneuvers.”] for viewing by the user [e.g., paragraph [0024], “The displays 54 are operable to receive, among other signals, signals representing media from the remainder of the audio/video system 46 and the vehicle 10, and employ any of various display technologies to transform those signals into visual outputs at their surfaces that may be seen by users of the vehicle 10.”];
when the confirmation instruction is received, controlling the vehicle to execute the voice command [e.g., 222, Yes and 224 in FIG. 2], and when the invalid instruction is received, controlling the vehicle to ignore the voice command [e.g., 222, No and 212 in FIG. 2], enabling the user to confirm whether the pending command is erroneous or feasible [e.g., at Yes (e.g., for feasible) or No (for erroneous) at 222 in FIG. 2];
It may be alleged that McNew (‘610) does not expressly indicate that the voice command is recognized, although he teaches at paragraph [0057] that user requests for the vehicle 10 to perform driving maneuvers are identified from input signals transformed from corresponding verbal inputs detected by the microphones 50, and the examiner believes this renders obvious, to one having ordinary skill in the art, the recognition of the voice command, even without further teaching.
However, in the context/field of an improved voice controlled autonomous driving system, Jafari et al. (‘603) teaches at paragraphs [0018], [0042], [0048], [0073], etc. that a voice command generated by an occupant is converted to a text-based command (in a voice recognition module 50) based on one or more speech recognition algorithms, classified into one of a plurality of pre-compiled driving maneuvers. When an alternative driving maneuver is identified, the feedback message generation module 60 instructs the HMI 26 (a touchscreen; paragraph [0030]) to generate a message asking the occupant 40 to confirm the alternative driving maneuver. The occupant is thus afforded an opportunity to confirm or reject driving maneuvers first before the driving maneuvers are executed by the vehicle.
It would have been obvious before the effective filing date of the claimed invention to implement or modify the McNew (‘610) method and vehicle for visually simulating driving plans so that that the user requests for the vehicle 10 to perform driving maneuvers would have been identified from input signals transformed from corresponding verbal inputs detected by the microphones 50 by use of a voice recognition module (50) using one or more speech recognition algorithms, as taught by Jafari et al. (‘603), and so that the opportunity to confirm or reject the driving maneuver would have been provided via a generated message on the touch screen HMI of the display 54 in McNew (‘610), e.g., which displayed the simulation, as taught by Jafari et al. (‘603), in order to identify the verbal input using speech recognition algorithms and provide feedback to the driver for confirmation of the maneuver, as taught by Jafari et al. (‘603), with a reasonable expectation of success, and e.g., as a use of a known technique to improve similar devices (methods, or products) in the same way.
As such, the implemented or modified McNew (‘610) method and vehicle for visually simulating driving plans would have rendered obvious:
per claim 1, . . . when receiving a voice command [e.g., in McNew (‘610), the verbal input at paragraph [0057] for identifying the candidate driving maneuver; e.g., obtained from the microphones 50 and used at step 204 in FIG. 2 to identify (obviously recognize) requests from the user], recognizing the voice command to obtain a pending voice command [e.g., by using the voice recognition module (50) using one or more speech recognition algorithms as taught by Jafari et al. (‘603); and in McNew (‘610), for the identified driving maneuver that will be used in FIG. 2, to be rejected or performed];
planning vehicle actions of the vehicle based on the pending voice command [e.g., in McNew (‘610), the driving plan generated at step 206 in FIG. 2];
constructing a current scene graph of the vehicle to obtain a first scene graph [e.g., in McNew (‘610), FIG. 5], and
displaying the first scene graph to prompt user to confirm or modify the pending voice command to correspondingly generate a confirmation instruction or an invalid instruction [e.g., the opportunity to confirm or reject the driving maneuver would have obviously been provided via a generated message on the touch screen HMI of the display 54 in McNew (‘610), e.g., which displayed the simulation, as taught by Jafari et al. (‘603); and in McNew (‘610), paragraphs [0091]ff, etc., “For example, the queries may include, before or contemporaneously with the visual simulations of the driving plans, introductory descriptions of the predicate determinations that the overall risk of performing their driving maneuvers renders performing the driving maneuvers feasible, but with some risk, as well as descriptions that, because of the risk of their performance, the driving maneuvers are, although acceptable, only provisionally accepted. In cases of identification of the driving maneuvers by the vehicle 10, the queries may further include descriptions that offers are being made by the vehicle 10, to the user, to perform the driving maneuvers. In cases of identification of the driving maneuvers from user requests, the queries may further include descriptions that the user requests for the vehicle 10 to perform the driving maneuvers are only provisionally granted. The queries may then follow up on these descriptions with the questions of whether the user confirms the driving maneuvers notwithstanding the risk of their performance. . . . The queries are output to the user at the various interfaces implemented by the components of the audio/video system 46. . . . Similarly, the planning/decision making module 94 may, for instance, identify the user responses to queries for confirmation of the driving maneuvers from input signals transformed from corresponding mechanical inputs detected by touch screens in the displays 54. . . . If the user is not comfortable with a driving maneuver because of the risk of its performance, or otherwise, the user response to a query for confirmation of the driving maneuver could be that the user does not confirm the driving maneuver. . . . However, if a user is comfortable with a driving maneuver notwithstanding the risk of its performance, their user response to a query for confirmation of the driving maneuver could be that the user confirms the driving maneuver.”], wherein the first scene graph is configured to present, in graphical form [e.g., in McNew (‘610), paragraph [0105], “An example visual simulation of a driving plan for performing a takeover is represented in FIG. 5 as a conceptual rendering of visual outputs at the surfaces of the displays 54 of the audio/video system 46 at a sequence of times (a)-(g)”], a scenario in which the vehicle is about to perform the vehicle actions [e.g., in McNew (‘610), paragraph [0105], “the driving plan for performing the takeover may be visually simulated. An example visual simulation of a driving plan for performing a takeover is represented in FIG. 5 as a conceptual rendering of visual outputs at the surfaces of the displays 54 of the audio/video system 46 at a sequence of times (a)-(g). Since performing the takeover includes the performance of other driving maneuvers, such as slowing down, speeding up and lane changes, and combinations of these, it will be understood that the represented example visual simulation for performing the takeover is applicable in principle to visual simulations of driving plans for performing the included or any other driving maneuvers.”] for viewing by the user [e.g., in McNew (‘610), paragraph [0024], “The displays 54 are operable to receive, among other signals, signals representing media from the remainder of the audio/video system 46 and the vehicle 10, and employ any of various display technologies to transform those signals into visual outputs at their surfaces that may be seen by users of the vehicle 10.”];
. . .
per claim 2, depending from claim 1, wherein constructing a current scene graph comprising:
obtaining a current background image based on current environment surrounding the vehicle [e.g., the information about the environment surrounding the vehicle as gathered in step 102 of FIG. 2 in McNew (‘610)];
planning a vehicle pose based on the vehicle actions [e.g., respective poses as shown in the simulation of FIG. 5 in McNew (‘610)]; and
placing a representation of the vehicle in the vehicle pose in the current background image to obtain the first scene graph [e.g., as shown (e.g., by the representation of the vehicle 10 at times (a) through (g) along the roadway 500 having lanes and other vehicles) in FIG. 5 of McNew (‘610); see e.g., paragraphs [0105] to [0114], etc.];
per claim 8, a vehicle, comprising:
a memory [e.g., 82 in McNew (‘610)], storing program instructions; and
a processor [e.g., 80 in McNew (‘610)], executing the program instructions to enable the vehicle to perform a voice-based control method for the vehicle, the voice-based control method for the vehicle comprising:
when receiving a voice command [e.g., in McNew (‘610), the verbal input at paragraph [0057] for identifying the candidate driving maneuver; e.g., obtained from the microphones 50 and used at step 204 in FIG. 2 to identify (obviously recognize) requests from the user], recognizing the voice command to obtain a pending voice command [e.g., by using the voice recognition module (50) using one or more speech recognition algorithms as taught by Jafari et al. (‘603); and in McNew (‘610), for the identified driving maneuver that will be used in FIG. 2, to be rejected or performed];
planning vehicle actions of the vehicle based on the pending voice command [e.g., in McNew (‘610), the driving plan generated at step 206 in FIG. 2];
constructing a current scene graph of the vehicle to obtain a first scene graph [e.g., in McNew (‘610), FIG. 5], and
displaying the first scene graph to prompt a user to confirm or modify the pending voice command to correspondingly generate a confirmation instruction or an invalid instruction [e.g., the opportunity to confirm or reject the driving maneuver would have obviously been provided via a generated message on the touch screen HMI of the display 54 in McNew (‘610), e.g., which displayed the simulation, as taught by Jafari et al. (‘603); and in McNew (‘610), paragraphs [0091]ff, etc., “For example, the queries may include, before or contemporaneously with the visual simulations of the driving plans, introductory descriptions of the predicate determinations that the overall risk of performing their driving maneuvers renders performing the driving maneuvers feasible, but with some risk, as well as descriptions that, because of the risk of their performance, the driving maneuvers are, although acceptable, only provisionally accepted. In cases of identification of the driving maneuvers by the vehicle 10, the queries may further include descriptions that offers are being made by the vehicle 10, to the user, to perform the driving maneuvers. In cases of identification of the driving maneuvers from user requests, the queries may further include descriptions that the user requests for the vehicle 10 to perform the driving maneuvers are only provisionally granted. The queries may then follow up on these descriptions with the questions of whether the user confirms the driving maneuvers notwithstanding the risk of their performance. . . . The queries are output to the user at the various interfaces implemented by the components of the audio/video system 46. . . . Similarly, the planning/decision making module 94 may, for instance, identify the user responses to queries for confirmation of the driving maneuvers from input signals transformed from corresponding mechanical inputs detected by touch screens in the displays 54. . . . If the user is not comfortable with a driving maneuver because of the risk of its performance, or otherwise, the user response to a query for confirmation of the driving maneuver could be that the user does not confirm the driving maneuver. . . . However, if a user is comfortable with a driving maneuver notwithstanding the risk of its performance, their user response to a query for confirmation of the driving maneuver could be that the user confirms the driving maneuver.”], wherein the first scene graph is configured to present a scenario in which the vehicle is about to perform the vehicle actions [e.g., in McNew (‘610), paragraph [0105], “the driving plan for performing the takeover may be visually simulated. An example visual simulation of a driving plan for performing a takeover is represented in FIG. 5 as a conceptual rendering of visual outputs at the surfaces of the displays 54 of the audio/video system 46 at a sequence of times (a)-(g). Since performing the takeover includes the performance of other driving maneuvers, such as slowing down, speeding up and lane changes, and combinations of these, it will be understood that the represented example visual simulation for performing the takeover is applicable in principle to visual simulations of driving plans for performing the included or any other driving maneuvers.”] for viewing by the user [e.g., in McNew (‘610), paragraph [0024], “The displays 54 are operable to receive, among other signals, signals representing media from the remainder of the audio/video system 46 and the vehicle 10, and employ any of various display technologies to transform those signals into visual outputs at their surfaces that may be seen by users of the vehicle 10.”];
when the confirmation instruction is received, controlling the vehicle to execute the voice command [e.g., 222, Yes and 224 in FIG. 2 of McNew (‘610)], and when the invalid is received, controlling the vehicle to ignore the voice command [e.g., 222, No and 212 in FIG. 2 of McNew (‘610)], enabling the user to confirm whether the pending command is valid [e.g., at Yes (e.g., for valid) at 222 in FIG. 2];
per claim 9, depending from claim 8, wherein constructing the current scene graph comprising:
obtaining a current background image based on current environment surrounding the vehicle [e.g., the information about the environment surrounding the vehicle as gathered in step 102 of FIG. 2 in McNew (‘610)];
planning a vehicle pose based on vehicle actions [e.g., respective poses as shown in the simulation of FIG. 5 in McNew (‘610)]; and
placing a representation of the vehicle in the vehicle pose in the current background image to obtain the first scene graph [e.g., as shown (e.g., by the representation of the vehicle 10 at times (a) through (g) along the roadway 500 having lanes and other vehicles) in FIG. 5 of McNew (‘610); see e.g., paragraphs [0105] to [0114], etc.];
per claim 15, depending from claim 1, wherein enabling the user to confirm whether the pending command is erroneous comprises allowing the user to confirm whether the voice command is accurately recognized by the vehicle based on the first scene graph [e.g., when the user decides, based on the visual simulation such as shown in FIG. 5 of McNew (‘610), whether or not to confirm the driving maneuver (at 222 in FIG. 2), obviously indicating (among other things) whether the driving maneuver was properly recognized or not];
per claim 16, depending from claim 8, wherein enabling the user to confirm whether the pending command is valid comprises allowing the user to verify whether the voice command is accurately recognized by the vehicle based on the first scene graph [e.g., when the user decides, based on the visual simulation such as shown in FIG. 5 of McNew (‘610), whether or not to confirm the driving maneuver (at 222 in FIG. 2), obviously indicating (among other things) whether the driving maneuver was properly recognized or not];
Claims 3 to 7 and 10 to 14 are rejected under 35 U.S.C. 103 as being unpatentable over McNew (2018/0029610) in view of Jafari et al. (2025/0091603) as applied to claims 1 and 8 above, and further in view of Hayakawa et al. (2021/0309242).
McNew (‘610) as implemented or modified in view of Jafari et al. (‘603) has been described above.
The implemented or modified McNew (‘610) method and vehicle for visually simulating driving plans may not expressly reveal that multiple scene graphs are created/obtained, or that the confirmation/invalid instruction input controls are displayed in the scene graph, although the examiner understands these limitations would have been obvious to one of ordinary skill in the art from the teachings of McNew (‘610) alone, e.g., when multiple candidate driving maneuvers were to be sequentially identified on the route using the flow chart of FIG. 2, to obviously make sequential lane changes, as was conventional, and to use the single (as obviously depicted in FIG. 1) display of the one or more displays 54 to obviously display both the simulation and the questions of whether the user confirms the driving maneuvers (e.g., paragraphs [0091], [0093], etc.), in order to obviously use the suggested “one” display (54) for both its indicated/described purposes (e.g., paragraph [0022], FIG. 1, etc.)
However, in the context/field of an improved vehicle travel control method and device, Hayakawa et al. (‘242) teaches using the travel control information presentation function (e.g., FIG. 3) in order to encourage the driver to confirm safety by himself/herself each time lane change is performed, such as when a lane is first changed (to the lane L3, in FIG. 3) and then subsequently changed back (to the lane L2, in FIG. 3), wherein it is detected whether the travel scene is suitable for changing lanes at predetermined time intervals (paragraph [0086], [0146], etc.) while the vehicle is being operated and the questions confirming each of the first (e.g., previous) and second (e.g., present) lane changes are presented, together with the input units for the driver’s acceptance (or non-acceptance), and the lane change destination using a visual pattern such as an arrow (FIGS. 4A to 4E, paragraphs [0079], etc.).
It would have been obvious before the effective filing date of the claimed invention to implement or modify the McNew (‘610) method and vehicle for visually simulating driving plans so that multiple driving scenes with obstacles/cars would have been simulated as in FIG. 2 (at 212) and in FIG. 5 of McNew (‘610) and presented on a presentation device (15) as taught by Hayakawa et al. (‘242), for multiple sequential lane changes, and so that, with each lane change simulation presentation, questions and input units for acceptance (151, 152, voice recognition using a microphone, a steering button, a blinker lever, etc.) would have been presented, as taught by Hayakawa et al. (‘242), and so that when a (previous or present) lane change was not accepted the invalid instruction (e.g., NG in FIG. 4B of Hayakawa et al. (‘242)) as the input unit for not accepting the lane change would have been provided/utilized by the driver, as taught by Hayakawa et al. (‘242) e.g., at S12, No, in order to not accept the lane change, with a reasonable expectation of success, and e.g., as a use of a known technique to improve similar devices (methods, or products) in the same way.
As such, the implemented or modified McNew (‘610) method and vehicle for visually simulating driving plans would have rendered obvious:
per claim 3, depending from claim 1, wherein when the confirmation instruction is received, controlling the vehicle to execute the voice command, comprises:
when the confirmation instruction is received, creating a current driving scene graph of the vehicle to obtain a second scene graph [e.g., 222, Yes and FIG. 5 in McNew (‘610); and the multiple travel scenes (see e.g., the right hand sides of FIGS. 4A to 4E) determined and presented for lane change acceptance by the driver in FIGS. 4A to 5E of Hayakawa et al. (‘242), which would have obviously been simulated in McNew (‘610)]; and
displaying a voice command input control in the second scene graph to prompt the user to input a new voice command [e.g., paragraph [0079] and FIG. 4E in Hayakawa et al. (‘242), to prompt the use for a new lane change acceptance, where the character data of the question, “Do you accept the lane change?”, is displayed on the display (as a prompt), and voice recognition from the microphone is used to receive the driver’s input/response to the prompt, e.g., as an effective new command];
per claim 4, depending from claim 3, e further comprising:
planning a driving trajectory of the vehicle in the first scene graph [e.g., on the roadway 500 in FIG. 5 of McNew (‘610)] based on the voice command executed by the vehicle [e.g., the verbal inputs in McNew (‘610) and the voice recognition in Jafari et al. (‘603) and Hayakawa et al. (‘242)];
detecting whether there are obstacles within the driving trajectory based on the background image and the driving trajectory [e.g., the other vehicles (as obstacles) in the simulated driving plans (FIG. 5) of McNew (‘610)]; and
when there are the obstacles within the driving trajectory, labeling corresponding images of the obstacles detected within the driving trajectory [e.g., as shown in FIG. 5 of McNew (‘610) and in FIG. 3 of Hayakawa et al. (‘242)];
per claim 5, depending from claim 1, wherein when the invalid instruction is received [e.g., at 222, No in FIG. 2 of McNew (‘610); and/or at “NG”7 in FIG. 4B of Hayakawa et al. (‘242) for a previous lane change; and/or in FIG. 4E of Hayakawa et al. (‘242), by not making the input via the input device or voice recognition indicating acceptance of the lane change], controlling the vehicle to ignore the voice command, including:
when the invalid instruction is received, invalidating the voice command [e.g., when the acceptance is not received, in Hayakawa et al. (‘242); and similarly, at 222, No in FIG. 2 of McNew (‘610)] and displaying a command input control in the first scene graph to prompt the user to input a new voice command [e.g., paragraph [0079] and FIG. 4E in Hayakawa et al. (‘242), to prompt the use for a new lane change acceptance, where the character data of the question, “Do you accept the lane change?”, is displayed on the display (as a prompt), and voice recognition from the microphone is used to receive the driver’s input/response to the prompt];
per claim 6, depending from claim 4, further comprising:
when constructing the first scene graph, displaying confirmation instruction input controls and invalid instruction input controls in the first scene graph to remind the user to input the confirmation instruction or invalid instruction [e.g., as shown in FIG. 4B of Hayakawa et al. (‘242)];
per claim 7, depending from claim 6,
wherein when the invalid instruction is received, voice-based control method for the vehicle further comprises:
when there are obstacles within the driving trajectory, labeling the invalid instruction [e.g., the “NG” labeled instruction in FIG. 4B of Hayakawa et al. (‘242), obviously labeled when the driving plan simulation or the travel scene includes other vehicles, as taught by both McNew (‘610) and Hayakawa et al. (‘242)];
per claim 10, depending from claim 8, wherein when the confirmation instruction is received, controlling the vehicle to execute the voice command, comprises:
when the confirmation instruction is received, creating a current driving scene graph of the vehicle to obtain a second scene graph [e.g., 222, Yes and FIG. 5 in McNew (‘610); and the multiple travel scenes (see e.g., the right hand sides of FIGS. 4A to 4E) determined and presented for lane change acceptance by the driver in FIGS. 4A to 5E of Hayakawa et al. (‘242), which would have obviously been simulated in McNew (‘610)]; and
displaying a voice command input control in the second scene graph to prompt the user to input a new voice command [e.g., paragraph [0079] and FIG. 4E in Hayakawa et al. (‘242), to prompt the use for a new lane change acceptance, where the character data of the question, “Do you accept the lane change?”, is displayed on the display (as a prompt), and voice recognition from the microphone is used to receive the driver’s input/response to the prompt, e.g., as an effective new command];
per claim 11, depending from claim 10, wherein the voice-based control method for the vehicle further comprising:
when constructing the first scene graph, displaying confirmation instruction input controls and invalid instruction input controls in the first scene graph to remind the user to input the confirmation instruction or invalid instruction [e.g., as shown in FIG. 4B of Hayakawa et al. (‘242)];
per claim 12, depending from claim 8, wherein when the invalid instruction is received, controlling the vehicle to ignore the voice command, including:
when the invalid instruction is received, invalidating the voice command [e.g., when the acceptance is not received, in Hayakawa et al. (‘242); and similarly, at 222, No in FIG. 2 of McNew (‘610)] and displaying a command input control in the first scene graph to prompt the user to input a new voice command [e.g., paragraph [0079] and FIG. 4E in Hayakawa et al. (‘242), to prompt the use for a new lane change acceptance, where the character data of the question, “Do you accept the lane change?”, is displayed on the display (as a prompt), and voice recognition from the microphone is used to receive the driver’s input/response to the prompt];
per claim 13, depending from claim 12, wherein further comprises:
planning a driving trajectory of the vehicle in the first scene graph [e.g., on the roadway 500 in FIG. 5 of McNew (‘610)] based on the voice command executed by the vehicle [e.g., the verbal inputs in McNew (‘610) and the voice recognition in Jafari et al. (‘603) and Hayakawa et al. (‘242)];
detecting whether there are obstacles within the driving trajectory based on the background image and the driving trajectory [e.g., the other vehicles (as obstacles) in the simulated driving plans (FIG. 5) of McNew (‘610)]; and
when there are obstacles within the driving trajectory, labeling the corresponding images of the obstacles detected within the driving trajectory [e.g., as shown in FIG. 5 of McNew (‘610) and in FIG. 3 of Hayakawa et al. (‘242)];
per claim 14, depending from claim 13, wherein when the invalid instruction is received, the voice-based control method for the vehicle further comprises:
when there are obstacles within the driving trajectory, labeling the invalid instruction [e.g., the “NG” labeled instruction in FIG. 4B of Hayakawa et al. (‘242), obviously labeled when the driving plan simulation or the travel scene includes other vehicles, as taught by both McNew (‘610) and Hayakawa et al. (‘242)];
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For example only, CN 119659662 A apparently corresponds to the publication of the present priority document (CN202311215928 20230920), and the EPO translation of this document apparently uses the phrases “scene map” and “scene diagram”, but not “scene graph”.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David A Testardi whose telephone number is (571)270-3528. The examiner can normally be reached Monday, Tuesday, Thursday, 8:30am - 5:30pm E.T., and Friday, 8:30 am - 12:30 pm E.T.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rachid Bendidi can be reached at (571) 272-4896. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID A TESTARDI/Primary Examiner, Art Unit 3664
1 See the Wikipedia article entitled “Scene graph” cited herewith.
2 Quoting the MPEP, “New claims, including claims first presented after the application filing date where no claims were submitted on filing, and amendments to the claims already in the application should be scrutinized not only for new matter but also for new terminology. While an applicant is not limited to the nomenclature used in the application as filed, he or she should make appropriate amendment of the specification whenever this nomenclature is departed from by amendment of the claims so as to have clear support or antecedent basis in the specification for the new terms appearing in the claims. This is necessary in order to insure certainty in construing the claims in the light of the specification. See 37 CFR 1.75, MPEP § 608.01(i) and § 1302.01 and § 2103. Note that examiners should ensure that the terms and phrases used in claims presented late in prosecution of the application (including claims amended via an examiner’s amendment) find clear support or antecedent basis in the description so that the meaning of the terms in the claims may be ascertainable by reference to the description, see 37 CFR 1.75(d)(1). If the examiner determines that the claims presented late in prosecution do not comply with 37 CFR 1.75(d)(1), applicant will be required to make appropriate amendment to the description to provide clear support or antecedent basis for the terms appearing in the claims provided no new matter is introduced.”
3 See the 2019 35 U.S.C. 112 Compliance Federal Register Notice (Federal Register, Vol. 84, No. 4, Monday, January 7, 2019, pages 57 to 63). See also http://ptoweb.uspto.gov/patents/exTrain/documents/2019-112-guidance-initiative.pptx . Quoting the FR Notice at pages 61 and 62, "The Federal Circuit emphasized that ‘‘[t]he written description requirement is not met if the specification merely describes a ‘desired result.’ ’’ Vasudevan, 782 F.3d at 682 (quoting Ariad, 598 F.3d at 1349). . . . When examining computer-implemented, software-related claims, examiners should determine whether the specification discloses the computer and the algorithm(s) that achieve the claimed function in sufficient detail that one of ordinary skill in the art can reasonably conclude that the inventor possessed the claimed subject matter at the time of filing. An algorithm is defined, for example, as 'a finite sequence of steps for solving a logical or mathematical problem or performing a task.' Microsoft Computer Dictionary (5th ed., 2002). Applicant may 'express that algorithm in any understandable terms including as a mathematical formula, in prose, or as a flow chart, or in any other manner that provides sufficient structure.' Finisar, 523 F.3d at 1340 (internal citation omitted). It is not enough that one skilled in the art could theoretically write a program to achieve the claimed function, rather the specification itself must explain how the claimed function is achieved to demonstrate that the applicant had possession of it. See, e.g., Vasudevan, 782 F.3d at 682–83. If the specification does not provide a disclosure of the computer and algorithm(s) in sufficient detail to demonstrate to one of ordinary skill in the art that the inventor possessed the invention that achieves the claimed result, a rejection under 35 U.S.C. 112(a) for lack of written description must be made. See MPEP § 2161.01, subsection I."
4 See Nautilus, Inc. v. Biosig Instruments, Inc. (U.S. Supreme Court, 2014) which held, "A patent is invalid for indefiniteness if its claims, read in light of the patent’s specification and prosecution history, fail to inform, with reasonable certainty, those skilled in the art about the scope of the invention." See also In re Packard, 751 F.3d 1307 (Fed.Cir.2014)(“[A] claim is indefinite when it contains words or phrases whose meaning is unclear,” i.e., “ambiguous, vague, incoherent, opaque, or otherwise unclear in describing and defining the claimed invention.”) and Ex Parte McAward, Appeal No. 2015-006416 (PTAB, Aug. 25, 2017, Precedential) (“Applying the broadest reasonable interpretation of a claim, then, the Office establishes a prima facie case of indefiniteness with a rejection explaining how the metes and bounds of a pending claim are not clear because the claim contains words or phrases whose meaning is unclear.”)
5 See the Wikipedia article entitled “Scene graph” cited herewith.
6 E.g., “comprising” would (e.g., indefinitely) apparently indicate that contents of the current scene graph follow in the claim, while “comprises” would (e.g., definitely) indicate that aspects of the constructing follow in the claim.
7 Meaning, “no good” (as the opposite of “O.K.”) in standard Japanese patent parlance.
