DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 23, 2026, has been entered.
Status of Claims
This Office action is in response to the amendments filed on March 23, 2026. Claims 1-4, 6-20, and 22 are currently pending with Claims 1-2, 7, 11-12, 17-18, and 22 being amended.
Response to Amendments
In response to Applicant’s amendments, filed March 23, 2026, the Examiner withdraws the previous claim objections, withdraws the previous 35 U.S.C. 112 rejections, and maintains the previous 35 U.S.C. 103 rejections.
Response to Arguments
Applicant's arguments filed November 19, 2025, have been fully considered but they are not persuasive.
Regarding Applicant’s arguments pertaining to the providing POI information without user input (see page 13 of instant arguments), the Examiner is unpersuaded. Schulz does not explicitly state that the POI information is not displayed prior to input. However, Urmson, teaches that the vehicle may provide information of relevant POIs without user input, and may select a different POI based on determined conditions (see at least e.g. Col. 9 lines 57-62 of Urmson). Schulz, in view of Urmson, teaches that POI data may be provided in response to user input, and also without user input, such that the user can receive information relating to points-of-interest near the vehicle. The Examiner is unpersuaded, and maintains the corresponding rejections.
The remaining arguments are essentially the same as those addressed above and/or below and are unpersuasive for essentially the same reasons. Therefore, the corresponding rejections are maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 6-9, 12, 14, 17-20, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 2017/0350718 A1, to Schulz (hereinafter referred to as Schulz; previously of record), in view of U.S. Patent No. 10,372,129 B1, to Urmson, et al (hereinafter referred to as Urmson; previously of record).
As per Claim 1, Schulz discloses the features of a system for identifying a point of interest in a vicinity of an autonomous vehicle (e.g. Paragraphs [0016] [0022], [0052]; where the vehicle (100) may be an autonomous vehicle, and where the occupant of the vehicle may request and receive information from the vehicle about a point of interest (POI) that is external to the vehicle; and where the digital maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle)), comprising:
one or more sensors within the autonomous vehicle configured to sense data (e.g. Paragraphs [0054]-[0056]; where the occupant monitoring system (245) may include various tracking devices and other similar equipment for monitoring and measuring certain characteristics of an occupant, including any combination of the eye tracker (250), the body tracker (255), the audio tracker (360), the pressure tracker (265), the respiratory tracker (270), or the cameras (297), which may comprise optical sensors or magnetic-field sensors) related to
one or more of a group consisting of directional orientation, eye gaze, a pointing gesture or speech by an occupant in the autonomous vehicle (e.g. Paragraphs [0030], [0054]; where the occupant monitoring system (245) may include various tracking devices for monitoring and measuring characteristics of an occupant, which may include any combination of an eye tracker 250), body tracker (255), audio tracker (260), pressure tracker (265), respiratory tracker (270), or cameras (297), or directional characteristics of the occupant for determining a potential occupant vector with respect to a POI);
an output device within the autonomous vehicle (e.g. Paragraphs [0066]-[0067], [0070], [0075]; where the central processor (340) can be configured to receive inputs from the systems of the vehicle, and can provide output to the information-attainment system (300), such as through an interface, speakers, or a display unit); and
a computer and memory a memory within the autonomous vehicle (e.g. Paragraph [0071]; where the central processor (340) may receive input from any number of system of the vehicle (100), and can receive various inputs from the occupant monitoring system (245), and can contain memory units (350)), the computer executing instructions that,
in response to detecting, by the one or more sensors, selection by the occupant of an automatic push mode (e.g. Paragraphs [0016], [0025], [0083]; where the occupant can initiate an automated process that identifies the POI, the vehicle may automatically identify the POI and retrieves the relevant information about the POI on behalf of the occupant (i.e. in response to selection by the user), and presents the information to the occupant), where
points of interest are automatically selected by the system (e.g. Paragraphs [0016], [0108]; where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant), cause the computer to automatically:
obtain data relating to locations of points of interest in the vicinity of the autonomous vehicle (e.g. Paragraphs [0023], [0025], [0052], [0091]; where the process can be automated, where the system identifies the POI, retrieves relevant information about the POI, and presents the information to the occupant; and where the vehicle may be traveling along a surface and may see any number of points-of-interest (POS) along or some distance away from and external to the vehicle; and where the digital maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle));
select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle (e.g. Paragraphs [0016], [0052], [0072], [0091], [0094]-[0095]; where the position of the vehicle may be within a reasonable distance of the POI (110) of interest; and where the system can identify one or more POI on which the occupant is focused; and where the occupant may select a user interface (UI) to confirm the accuracy of the selection of the POI (110); and where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant) ‘…’;
determine, based on the data sensed from the one or more sensors (e.g. Paragraph [0071]; where the central processor (340) may receive input from any number of system of the vehicle (100), and can receive various inputs from the occupant monitoring system (245)),
at least the first directional orientation of a body of the occupant in the autonomous vehicle (e.g. Paragraphs [0057]-[0058], [0080]; where the body tracker (255) may be configured to monitor the positioning of one or more body parts of an occupant; and where the cameras focus on the occupant’s gesture or direction the occupant is leaning their body);
calculate a directional vector between the autonomous vehicle and the point of interest (e.g. Paragraphs [0087]-[0089]; where the head tracker may generate a first potential occupant vector, which may be determined based on the speed, orientation, and elevation of the vehicle, and the vector is input to the location determination system and compared against the current position of the vehicle as the origin of the vector to plot the vector against the map to extrapolate a reference marker that corresponds to the intended POI);
push output information to the occupant about the selected point of interest including the directional vector of the point of interest relative to autonomous vehicle or the data sensed by the one or more sensors (e.g. Paragraphs [0067], [0092]-[0093]; Figure 4; where the data relevant to the POI may be presented to the occupant, such as on the display device or by broadcasting the information through the speakers); and
guide the autonomous vehicle ‘…’ (e.g. Paragraphs [0022], [0092]; where the vehicle (100) may be autonomous, using one or more computing systems to navigate and/or maneuver the vehicle (100) along a travel route with minimal or no input from a human driver; and where the central processor can determine directions to the POI, or time and distance to the POI).
Schulz fails to disclose every feature of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest; and guide the autonomous vehicle to the selected point of interest.
However, Urmson, in a similar field of endeavor, more explicitly teaches the features of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest
Urmson teaches a method for providing recommendations to users in a vehicle, where the autonomous vehicle may make unsolicited recommendations in addition to responding to requests from the user, for example, if the data from the vehicle indicates that road conditions near an intended destination POI are unsafe, the vehicle may, without prompting, display a message to the user recommending a different POI (e.g. Col. 9 lines 57-62).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of selecting a POI without user input in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
Urmson further teaches the features of guide the autonomous vehicle to the selected point of interest.
Urmson teaches a method for providing recommendations to users in a vehicle, where the central control computer may maneuver the car in response to information from the sensors, and may automatically maneuver the vehicle to the particular point of interest (e.g. Claim 3).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of guiding the vehicle to the point of interest in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
As per Claim 2, Schulz, in view of Urmson, teaches the features of Claim 1, and Schulz further discloses the features of wherein the data sensed from the set of one or more sensors relates to a position of a head and eyes of the occupant (e.g. Paragraph [0031]; where the system (245) can include one or more eye trackers (250), one or more body trackers (255) to track movements or the gaze of the eyes of the occupant or monitor the positioning of one or more body parts of the occupant, such as the head or arms).
As per Claim 3, Schulz, in view of Urmson, teaches the features of Claim 1, and Schulz further discloses the features of wherein the data sensed from the one or more sensors relates to the pointing gesture performed by the occupant (e.g. Paragraph [0028]; where the system may include a gesture recognition device (225), which can include any suitable combination of circuitry and software for identifying and processing gestures from the occupants, to include detecting hand or facial gestures exhibited by the occupant).
As per Claim 4, Schulz, in view of Urmson, teaches the features of Claim 1, and Schulz further discloses the features of wherein the data sensed from the set of one or more sensors relates to speech (e.g. Paragraphs [0027], [0083]; where the system can include a voice recognition device (220) to detect voice or other audio from the occupant, relating to an inquiry directed to obtaining information about a particular POI; and where the occupant may select an option from the user interface to provide a voice command, which may activate the voice recognition device (220)).
As per Claim 6, Schulz, in view of Urmson, teaches the features of Claim 1, and Schulz further discloses the features of wherein the instructions when executed further cause the computer to receive data relating to locations in the vicinity of the autonomous vehicle including at least one of global positioning system (GPS) data, data sensed by other sensors on the autonomous vehicle and data received from a cloud service (e.g. Paragraphs [0024], [0029], [0048]-[0049], [0051]; where the system may be based on a satellite positioning system, which can include coordinates derived from GPS signals; and where the location determination system (230) can obtain positional information, particularly positional information of a POI using a GPS unit; and where the digital maps and other information may be retrieved from an external server).
As per Claim 7, Schulz discloses the features of a system for identifying a point of interest in a vicinity of an autonomous vehicle (e.g. Paragraphs [0016] [0022], [0052]; where the vehicle (100) may be an autonomous vehicle, and where the occupant of the vehicle may request and receive information from the vehicle about a point of interest (POI) that is external to the vehicle; and where the digital maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle)), comprising:
one or more sensors within the autonomous vehicle (e(e.g. Paragraphs [0054]-[0056]; where the occupant monitoring system (245) may include various tracking devices and other similar equipment for monitoring and measuring certain characteristics of an occupant, including any combination of the eye tracker (250), the body tracker (255), the audio tracker (360), the pressure tracker (265), the respiratory tracker (270), or the cameras (297), which may comprise optical sensors or magnetic-field sensors)) configured to
sense data related to at least one of directional orientation, eye gaze, a pointing gesture and speech by an occupant in the autonomous vehicle (e.g. Paragraphs [0030], [0054]; where the occupant monitoring system (245) may include various tracking devices for monitoring and measuring characteristics of an occupant, which may include any combination of an eye tracker 250), body tracker (255), audio tracker (260), pressure tracker (265), respiratory tracker (270), or cameras (297), or directional characteristics of the occupant for determining a potential occupant vector with respect to a POI);
an output device within the autonomous vehicle (e.g. Paragraphs [0066]-[0067], [0070], [0075]; where the central processor (340) can be configured to receive inputs from the systems of the vehicle, and can provide output to the information-attainment system (300), such as through an interface, speakers, or a display unit); and
a computer within the autonomous vehicle (e.g. Paragraphs [0071]; where the central processor (340) may receive input from any number of system of the vehicle (100), and can receive various inputs from the occupant monitoring system (245), and can contain memory units (350)), the computer executing instructions to:
detect selection of an automatic push mode (e.g. Paragraphs [0016], [0025], [0083]; where the occupant can initiate an automated process that identifies the POI, the vehicle may automatically identify the POI and retrieves the relevant information about the POI on behalf of the occupant (i.e. in response to selection by the user), and presents the information to the occupant)
where points of interest are automatically selected by the system (e.g. Paragraphs [0016], [0108]; where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant),
in response to detecting selection of the automatic push mode (e.g. Paragraph [0025]; where the occupant can initiate an automated process that identifies the POI, retrieves the relevant information about the POI, and presents the information to the occupant), automatically:
obtain data relating to locations of points of interest in the vicinity of the autonomous vehicle (e.g. Paragraphs [0023], [0025], [0052], [0091]; where the process can be automated, where the system identifies the POI, retrieves relevant information about the POI, and presents the information to the occupant; and where the vehicle may be traveling along a surface and may see any number of points-of-interest (POS) along or some distance away from and external to the vehicle; and where the digital maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle)));
select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle (e.g. Paragraphs [0016], [0052], [0072], [0091], [0094]-[0095]; where the position of the vehicle may be within a reasonable distance of the POI (110) of interest; and where the system can identify one or more POI on which the occupant is focused; and where the occupant may select a user interface (UI) to confirm the accuracy of the selection of the POI (110); and where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant) ‘…’;
determine, based on the data sensed from the one or more sensors (e.g. Paragraph [0071]; where the central processor (340) may receive input from any number of system of the vehicle (100), and can receive various inputs from the occupant monitoring system (245)),
at least the directional orientation of a body of the occupant in the autonomous vehicle (e.g. Paragraphs [0057]-[0058], [0080]; where the body tracker (255) may be configured to monitor the positioning of one or more body parts of an occupant; and where the cameras focus on the occupant’s gesture or direction the occupant is leaning their body);
calculate a directional vector between the autonomous vehicle and the point of interest (e.g. Paragraphs [0087]-[0089]; where the head tracker may generate a first potential occupant vector, which may be determined based on the speed, orientation, and elevation of the vehicle, and the vector is input to the location determination system and compared against the current position of the vehicle as the origin of the vector to plot the vector against the map to extrapolate a reference marker that corresponds to the intended POI);
push output information to the occupant about the selected point of interest including the directional vector of the point of interest relative to autonomous vehicle or the data sensed by the one or more sensors (e.g. Paragraphs [0067], [0092]-[0093]; Figure 4; where the data relevant to the POI may be presented to the occupant, such as on the display device or by broadcasting the information through the speakers), and when not operating in the automatic push mode:
receive data relating to locations of points of interest in the vicinity of the autonomous vehicle (e.g. Paragraphs [0023], [0025], [0052], [0091]; where the system identifies the POI, retrieves relevant information about the POI, and presents the information to the occupant; and where the vehicle may be traveling along a surface and may see any number of points-of-interest (POS) along or some distance away from and external to the vehicle and request information about the POI; and maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle));
determine a directional result vector from the data sensed by the or more sensors (e.g. Paragraphs [0006], [0030], [0032]; where the system can determine one or more directional characteristics of the occupant),
determine a first directional orientation of a body of the occupant in the autonomous vehicle (e.g. Paragraphs [0057]-[0058], [0080]; where the body tracker (255) may be configured to monitor the positioning of one or more body parts of an occupant; and where the cameras focus on the occupant’s gesture or direction the occupant is leaning their body),
identify the point of interest in the vicinity of the autonomous vehicle which lies in the vicinity of the directional vector (e.g. Figure 4; where the point of interest (POI) is determined and identified based on the input and feedback from the occupant using the directional characteristics), wherein
the computer identifies the point of interest from the directional result vector and from the received data relating to locations of points of interest in the vicinity of the autonomous vehicle (e.g. Figure 4; where the point of interest (POI) is determined and identified based on the input and feedback from the occupant using the directional characteristics);
send an instruction to the output device to output the point of interest identified, the output device outputting the point of interest as a result of receiving the instruction sent from the computer (e.g. Paragraphs [0039]-[0040]; where the presentation of the information about the identified POI may be provided to any number of occupants in the vehicle), wherein
the instructions include a first information related to the point of interest output to the occupant based on the first directional orientation of a body of the occupant (e.g. Paragraphs [0039]-[0040], [0072]; Figure 5; where the presentation of the information about the identified POI may be provided to any number of occupants in the vehicle; and where the system may allow the occupants to initiate an inquire about one or more POIs, enable their relevant characteristics to be monitored, and to be presented with information related to the POI; and where information is retrieved for a multiple number of POIs); and
guide the autonomous vehicle ‘…’ (e.g. Paragraph [0022]; where the vehicle (100) may be autonomous, using one or more computing systems to navigate and/or maneuver the vehicle (100) along a travel route with minimal or no input from a human driver).
Schulz fails to disclose every feature of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest; and guide the autonomous vehicle to the selected point of interest.
However, Urmson, in a similar field of endeavor, more explicitly teaches the features of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest
Urmson teaches a method for providing recommendations to users in a vehicle, where the autonomous vehicle may make unsolicited recommendations in addition to responding to requests from the user, for example, if the data from the vehicle indicates that road conditions near an intended destination POI are unsafe, the vehicle may, without prompting, display a message to the user recommending a different POI (e.g. Col. 9 lines 57-62).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of selecting a POI without user input in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
Urmson further teaches the features of guide the autonomous vehicle to the selected point of interest.
Urmson teaches a method for providing recommendations to users in a vehicle, where the central control computer may maneuver the car in response to information from the sensors, and may automatically maneuver the vehicle to the particular point of interest (e.g. Claim 3).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of guiding the vehicle to the point of interest in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
As per Claim 8, Schulz, in view of Urmson, teaches the features of Claim 7, and Schulz further discloses the features of wherein, when not in the automatic push mode, the computer further receives and processes the speech from the occupant to assist in identifying the point of interest (e.g. Paragraphs [0016], [0025], [0027], [0083]; Figure 4; where the system can include a voice recognition device (220) to detect voice or other audio from the occupant, relating to an inquiry directed to obtaining information about a particular POI; and where the occupant may select an option from the user interface to provide a voice command, which may activate the voice recognition device (220); and where the vehicle may switch to a manual mode, or a mode in which a human driver controls the navigation of the vehicle (i.e. not an automated push mode) and the vehicle may search a database to identify the POI in response to a user request (i.e. manually enacted by the user)).
As per Claim 9, Schulz, in view of Urmson, teaches the features of Claim 7, and Schulz further discloses the features of wherein, when not in the automatic push mode, the computer further receives external information in order to identify the point of interest which lies along the directional result vector (e.g. Paragraphs [0030], [0037], [0052], [0073]; where the system may be equipped with an occupant monitoring system to track directional characteristics of the occupant for determining a potential occupant vector with respect to a POI; and where once a potential occupant vector is calculated, an extrapolation of the reference marker may be performed based on obtaining the current position of the vehicle; and where the system can capture images that are external to the vehicle).
As per Claim 12, Schulz, in view of Urmson, teaches the features of Claim 7, and Schulz further discloses the features of further comprising an eye gaze vector module implemented by the computer for determining an eye gaze vector indicating a direction the occupant’s eyes are looking (e.g. Paragraph [0031]; where the system (245) can include one or more eye trackers (250), one or more body trackers (255) to track movements or the gaze of the eyes of the occupant or monitor the positioning of one or more body parts of the occupant, such as the head or arms).
As per Claim 14, Schulz, in view of Urmson, teaches the features of Claim 7, and Schulz further discloses the features of further comprising a speech recognition module implemented by the computer for recognizing speech related to an identity of the point of interest (e.g. Paragraphs [0027], [0083]; where the system can include a voice recognition device (220) to detect voice or other audio from the occupant, relating to an inquiry directed to obtaining information about a particular POI; and where the occupant may select an option from the user interface to provide a voice command, which may activate the voice recognition device (220)).
As per Claim 17, Schulz discloses the features of a method of identifying a point of interest in a vicinity of an autonomous vehicle (e.g. Paragraphs [0016] [0022], [0052]; where the vehicle (100) may be an autonomous vehicle, and where the occupant of the vehicle may request and receive information from the vehicle about a point of interest (POI) that is external to the vehicle; and where the digital maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle)), comprising:
receiving an indication of a direction from data obtained from an occupant of the autonomous vehicle (e.g. Paragraphs [0057]-[0058], [0080]; where the body tracker (255) may be configured to monitor the positioning of one or more body parts of an occupant; and where the cameras focus on the occupant’s gesture or direction the occupant is leaning their body),
wherein the data relates to directional orientation of the occupant (e.g. Paragraphs [0030], [0054]; where the occupant monitoring system (245) may include various tracking devices for monitoring and measuring characteristics of an occupant, which may include any combination of an eye tracker 250), body tracker (255), audio tracker (260), pressure tracker (265), respiratory tracker (270), or cameras (297), or directional characteristics of the occupant for determining a potential occupant vector with respect to a POI);
receive data relating to locations of points of interest in the vicinity of the autonomous vehicle (e.g. Paragraphs [0016], [0022], [0052]; where the occupant of the vehicle may request and receive information from the vehicle about a point of interest (POI) that is external to the vehicle; and where the digital maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle));
detect selection of an automatic push mode (e.g. Paragraphs [0016], [0025], [0083]; where the occupant can initiate an automated process that identifies the POI, the vehicle may automatically identify the POI and retrieves the relevant information about the POI on behalf of the occupant (i.e. in response to selection by the user), and presents the information to the occupant)
where points of interest are automatically selected by the system (e.g. Paragraphs [0016], [0108]; where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant), automatically:
select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle (e.g. Paragraphs [0016], [0052], [0072], [0091], [0094]-[0095]; where the position of the vehicle may be within a reasonable distance of the POI (110) of interest; and where the system can identify one or more POI on which the occupant is focused; and where the occupant may select a user interface (UI) to confirm the accuracy of the selection of the POI (110); and where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant) ‘…’;
calculate a directional vector between the autonomous vehicle and the point of interest (e.g. Paragraphs [0087]-[0089]; where the head tracker may generate a first potential occupant vector, which may be determined based on the speed, orientation, and elevation of the vehicle, and the vector is input to the location determination system and compared against the current position of the vehicle as the origin of the vector to plot the vector against the map to extrapolate a reference marker that corresponds to the intended POI);
push output information to the occupant about the selected point of interest including the directional vector of the point of interest relative to autonomous vehicle or the directional characteristic of the occupant (e.g. Paragraphs [0067], [0092]-[0093]; Figure 4; where the data relevant to the POI may be presented to the occupant, such as on the display device or by broadcasting the information through the speakers); and
guiding the autonomous vehicle ‘…’ (e.g. Paragraph [0022]; where the vehicle (100) may be autonomous, using one or more computing systems to navigate and/or maneuver the vehicle (100) along a travel route with minimal or no input from a human driver).
Schulz fails to disclose every feature of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest; and guiding the autonomous vehicle to the selected point of interest.
However, Urmson, in a similar field of endeavor, more explicitly teaches the features of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest
Urmson teaches a method for providing recommendations to users in a vehicle, where the autonomous vehicle may make unsolicited recommendations in addition to responding to requests from the user, for example, if the data from the vehicle indicates that road conditions near an intended destination POI are unsafe, the vehicle may, without prompting, display a message to the user recommending a different POI (e.g. Col. 9 lines 57-62).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of selecting a POI without user input in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
Urmson further teaches the features of guiding the autonomous vehicle to the selected point of interest.
Urmson teaches a method for providing recommendations to users in a vehicle, where the central control computer may maneuver the car in response to information from the sensors, and may automatically maneuver the vehicle to the particular point of interest (e.g. Claim 3).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of guiding the vehicle to the point of interest in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
As per Claim 18, Schulz, in view of Urmson, teaches the features of Claim 17, and Schulz further discloses the features of said step of receiving an indication of a direction at which the point of interest is located comprises the step of receiving data related to a position of the head and eyes of the occupant (e.g. Paragraph [0031]; where the system (245) can include one or more eye trackers (250), one or more body trackers (255) to track movements or the gaze of the eyes of the occupant or monitor the positioning of one or more body parts of the occupant, such as the head or arms).
As per Claim 19, Schulz, in view of Urmson, teaches the features of Claim 17, and Schulz further discloses the features of receiving an indication of a direction at which the point of interest is located comprises the step of receiving and recognizing a pointing gesture performed by the occupant (e.g. Paragraph [0028]; where the system may include a gesture recognition device (225), which can include any suitable combination of circuitry and software for identifying and processing gestures from the occupants, to include detecting hand or facial gestures exhibited by the occupant).
As per Claim 20, Schulz, in view of Urmson, teaches the features of Claim 17, and Schulz further discloses the features of receiving an indication of a direction at which the point of interest is located comprises the step of receiving and recognizing speech from the occupant describing a direction at which the point of interest is located (e.g. Paragraphs [0027], [0083]; where the system can include a voice recognition device (220) to detect voice or other audio from the occupant, relating to an inquiry directed to obtaining information about a particular POI; and where the occupant may select an option from the user interface to provide a voice command, which may activate the voice recognition device (220)).
As per Claim 22, Schulz discloses the features of a non-transitory computer-readable medium storing computer instructions (e.g. Paragraph [0111]; where the system may comprise a computer program product embodied in one or more computer-readable media, having program code stored thereon) that when executed by one or more processors cause the one or more processors to
operate, in response to detecting by the one or more sensors selection by the occupant of an automatic push mode (e.g. Paragraphs [0016], [0025], [0083]; where the occupant can initiate an automated process that identifies the POI, the vehicle may automatically identify the POI and retrieves the relevant information about the POI on behalf of the occupant (i.e. in response to selection by the user), and presents the information to the occupant)
where points of interest are automatically selected by the system (e.g. Paragraphs [0016], [0108]; where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant), causes the one or more processors to automatically:
determine a directional orientation of a body of the occupant in an autonomous vehicle (e.g. Paragraphs [0057]-[0058], [0080]; where the body tracker (255) may be configured to monitor the positioning of one or more body parts of an occupant; and where the cameras focus on the occupant’s gesture or direction the occupant is leaning their body);
receive data relating to locations of points of interest in a vicinity of the autonomous vehicle (e.g. Paragraphs [0016] [0022], [0052]; where the occupant of the vehicle may request and receive information from the vehicle about a point of interest (POI) that is external to the vehicle; and where the digital maps are selected based on the current position of the vehicle, and if it is within a reasonable distance of the POI (i.e. in vicinity of the vehicle));
select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle (e.g. Paragraphs [0016], [0052], [0072], [0091], [0094]-[0095]; where the position of the vehicle may be within a reasonable distance of the POI (110) of interest; and where the system can identify one or more POI on which the occupant is focused; and where the occupant may select a user interface (UI) to confirm the accuracy of the selection of the POI (110); and where the vehicle may automatically identify the POI and fetch relevant information on behalf of the occupant and in response to a request from the occupant)) ‘…’;
calculate a directional vector between the autonomous vehicle and the point of interest (e.g. Paragraphs [0087]-[0089]; where the head tracker may generate a first potential occupant vector, which may be determined based on the speed, orientation, and elevation of the vehicle, and the vector is input to the location determination system and compared against the current position of the vehicle as the origin of the vector to plot the vector against the map to extrapolate a reference marker that corresponds to the intended POI);
push output information to the occupant about the selected point of interest including the directional vector of the point of interest relative to autonomous vehicle or the directional orientation of a body of the occupant (e.g. Paragraphs [0067], [0092]-[0093]; Figure 4; where the data relevant to the POI may be presented to the occupant, such as on the display device or by broadcasting the information through the speakers); and not operate in the automatic push mode that causes the one or more processors to automatically:
receive a direction at which a second point of interest is located (e.g. Paragraphs [0023], [0025], [0091]; where the system identifies the POI, retrieves relevant information about the POI, and presents the information to the occupant; and where the vehicle may be traveling along a surface and may see any number of points-of-interest (POS) along or some distance away from and external to the vehicle) from
data received from the occupant of the autonomous vehicle related to at least one of directional orientation of the body and speech recognition of the occupant (e.g. Paragraphs [0030], [0054]; where the occupant monitoring system (245) may include various tracking devices for monitoring and measuring characteristics of an occupant, which may include any combination of an eye tracker 250), body tracker (255), audio tracker (260), pressure tracker (265), respiratory tracker (270), or cameras (297), or directional characteristics of the occupant for determining a potential occupant vector with respect to a POI);
determine the second point of interest lying in the direction of the indication of direction received, wherein the second point of interest is determined from the indication of direction received, from the received data relating to the locations of points of interest in the vicinity of the autonomous vehicle (e.g. Paragraphs [0039]-[0040], [0072]; Figure 4-5; where the point of interest (POI) is determined and identified based on the input and feedback from the occupant using the directional characteristics; and where the presentation of the information about the identified POI may be provided to any number of occupants in the vehicle; and where the system may allow the occupants to initiate an inquire about one or more POIs, enable their relevant characteristics to be monitored, and to be presented with information related to the POI; and where information is retrieved for a multiple number of POIs);
determine a second directional vector from the occupant and the point of interest (e.g. Paragraphs [0019], [0030], [0040]; where the tracker may monitor and detect variations in one or more occupants of the vehicle; and where a combination of occupants may work in tandem to acquire information about a POI, where identification of the POI may be based on measurable characteristics of a second occupant; where the measurable characteristics include direction characteristics of one or more occupants);
output information related to the second point of interest and second directional vectors to an output device within the autonomous vehicle (e.g. Paragraphs [0039]-[0040]; where the presentation of the information about the identified POI may be provided to any number of occupants in the vehicle),
wherein the output information including information related to the second point of interest output to the occupant (e.g. Paragraphs [0039]-[0040], [0072]; where the presentation of the information about the identified POI may be provided to any number of occupants in the vehicle; and where the system may allow the occupants to initiate an inquire about one or more POIs, enable their relevant characteristics to be monitored, and to be presented with information related to the POI; and where information is retrieved for a multiple number of POIs); and
guide the autonomous vehicle ‘…’ (e.g. Paragraph [0022]; where the vehicle (100) may be autonomous, using one or more computing systems to navigate and/or maneuver the vehicle (100) along a travel route with minimal or no input from a human driver).
Schulz fails to disclose every feature of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest; and guide the autonomous vehicle to the selected point of interest.
However, Urmson, in a similar field of endeavor, more explicitly teaches the features of select, automatically by the system, one of the points of interest that is within a predetermined radius of the autonomous vehicle without receiving an inquiry from the occupant identifying the point of interest
Urmson teaches a method for providing recommendations to users in a vehicle, where the autonomous vehicle may make unsolicited recommendations in addition to responding to requests from the user, for example, if the data from the vehicle indicates that road conditions near an intended destination POI are unsafe, the vehicle may, without prompting, display a message to the user recommending a different POI (e.g. Col. 9 lines 57-62).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of selecting a POI without user input in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
Urmson further teaches the features of guide the autonomous vehicle to the selected point of interest.
Urmson teaches a method for providing recommendations to users in a vehicle, where the central control computer may maneuver the car in response to information from the sensors, and may automatically maneuver the vehicle to the particular point of interest (e.g. Claim 3).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the information attainment system based on occupant behaviors in the system of Schulz, with the feature of guiding the vehicle to the point of interest in the system of Urmson, in order to allow the vehicle to respond to its environment in order to navigate safely (see at least Col. 3 lines 63-66 of Urmson).
Claims 10-11, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Schulz, in view of Urmson, as applied to Claim 7 above, and further in view of U.S. Patent Publication No. 2014/0361973 A1, to Raux, et al (hereinafter referred to as Raux; previously of record).
As per Claim 10, Schulz, in view of Urmson, teaches the features of Claim 7, but the combination of Schulz, in view of Urmson, fails to teach every feature of further comprising a body and gesture detection module implemented by the computer, the body and gesture detection module detecting a skeletal model of the occupant at least one instant in time.
However, Raux, in a similar field of endeavor, teaches a method for multimodal human-vehicle interaction, where the motion/gesture recommendation module (202) can be configured to recognize motion and gesture events based on the interaction data, and can include skeletal tracking to determine the body position or movement of the vehicle occupant or an appendage of the vehicle occupant (e.g., hands, arms), and gaze tracking can be used to determine the vehicle occupant’s eye gaze, and head pose tracking can be used to determine a position, orientation, or movement of the vehicle occupant’s head) (e.g. Paragraphs [0033], [0043]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the information attainment system based on occupant behaviors in the system of Schulz, in view of Urmson, with the feature of using a skeletal model in the system of Raux, in order to accurately track the position, orientation, or movement of the occupant (see at least Paragraph [0033] of Raux).
As per Claim 11, Schulz, in view of Urmson and Raux, teaches the features of Claim 10, and Raux further teaches the features of further comprising a head vector module implemented by the computer for determining a head vector from the skeletal model, the head vector indicating a direction the occupant’s head is facing.
Raux teaches a method for multimodal human-vehicle interaction, where the motion/gesture recommendation module (202) can be configured to recognize motion and gesture events based on the interaction data, and can include skeletal tracking to determine the body position or movement of the vehicle occupant or an appendage of the vehicle occupant (e.g., hands, arms), and gaze tracking can be used to determine the vehicle occupant’s eye gaze, and head pose tracking can be used to determine a position, orientation, or movement of the vehicle occupant’s head (e.g. Paragraphs [0033], [0043]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the information attainment system based on occupant behaviors in the system of Schulz, in view of Urmson, with the feature of using a skeletal model in the system of Raux, in order to accurately track the position, orientation, or movement of the occupant (see at least Paragraph [0033] of Raux).
As per Claim 13, Schulz, in view of Urmson and Raux, teaches the features of Claim 10, and Raux further teaches the features of further comprising a finger pointing vector module implemented by the computer for determining a finger pointing vector from the skeletal model indicating a direction along which the occupant is pointing.
Raux teaches a method for multimodal human-vehicle interaction, where the motion/gesture recommendation module (202) can be configured to recognize motion and gesture events based on the interaction data, and can include skeletal tracking to determine the body position or movement of the vehicle occupant or an appendage of the vehicle occupant (e.g., hands, arms), and where the recognition module can include a motion/gesture recognition module based on gesture input, which includes skeletal tracking to determine the body position of movement of the vehicle occupant or an appendage of the vehicle occupant, which can be used to extract at least one visual point of interest from the gesture (e.g. Paragraphs [0033], [0036], [0043]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the information attainment system based on occupant behaviors in the system of Schulz, in view of Urmson, with the feature of using a skeletal model in the system of Raux, in order to accurately track the position, orientation, or movement of the occupant (see at least Paragraph [0033] of Raux).
As per Claim 15, Schulz, in view of Urmson and Raux, teaches the features of Claim 11, and Schulz further discloses the features of further comprising a multimodal response interpretation module for receiving at least one of the head vector, eye gaze vector, finger pointing vector and recognized speech, and inferring the directional result vector from the received at least one of the head vector, the eye gaze vector, the finger pointing vector and the recognized speech (e.g. Paragraphs [0031], [0096]; Figure 4; where the system (245) can include one or more eye trackers (250), one or more body trackers (255) to track movements or the gaze of the eyes of the occupant or monitor the positioning of one or more body parts of the occupant, such as the head or arms; and where the system may determine directional characteristics of the occupant, determine potential POIs based on the potential occupant vector, present the information to the user, and determine if the POI is properly identified).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Schulz, in view of Urmson and Raux, as applied to Claim 15 above, and further in view of U.S. Patent Publication No. 2019/0079659 A1, to Adenwala, et al (hereinafter referred to as Adenwala; previously of record).
As per Claim 16, Schulz, in view of Urmson and Raux, teaches the features of Claim 15, and Adenwala further teaches the features of wherein the multimodal response interpretation module is implemented using a neural network.
Adenwala teaches an autonomous driving vehicles social network system, where a neural network is used to determine how to generate a profile and assess the condition of the immediate surrounding area of the host vehicle and determine a vehicle outcome based on the current state of the vehicle, sensor data received from nearby vehicles (e.g. Paragraphs [0071], [0076]-[0077]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the information attainment system based on occupant behaviors in the system of Schulz, in view of Urmson and Raux, with the feature of using a neural network in the system of Adenwala, in order to generate a profile and determine when and how to share the profile data (see at least Paragraphs [0074]-[0075] of Adenwala).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Beyeler, et al (U.S. 8,810,437 B2), which teaches a method for displaying points-of-
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MERRITT LEVY whose telephone number is (571)270-5595. The examiner can normally be reached Mon-Fri 0630-1600.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn can be reached at (571) 272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MERRITT LEVY/Examiner, Art Unit 3663
/ABBY J FLYNN/Supervisory Patent Examiner, Art Unit 3663
Prosecution Insights