Detailed Action
Applicant amended claims 1, 11, 15, 16, and 18-20, canceled claims 17, and 21-22, added claim 23 and presented claims 1-16, 18-20, and 23 for reconsideration on 07/08/2026.
Examiner's Notes
The Examiner cites particular sections in the references as applied to the claims below for the convenience of the applicant(s). Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant(s) fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 102 that forms the basis for all the rejections under this section made in this Office Action:
A person shall be entitled to a patent unless—
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-11, 16, 18-20 and 23 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Li et al., Pub. No.: US 2025/0139837 A1 (hereinafter Li).
Claim 1. Li discloses:
A system, comprising: a processor; and a memory storing:
an image reception module including instructions that, when executed by the processor, cause the processor to receive an image that includes a representation of an object, but lacks a representation of a human-provided indication associated with the object; (¶¶ 71-73, 80, 88, 91, 160, objects of interests are captured “by one or more cameras located in, on, or otherwise associated the vehicle” the object lacks the gaze that detected by a gaze detector)
a recording reception module including instructions that, when executed by the processor, cause the processor to receive a recording that includes the representation of the human-provided indication, but lacks the representation of the object; (¶¶ 71-73, 80, 88, 91, 160, objects of interests are captured “by one or more cameras located in, on, or otherwise associated the vehicle” and a gaze detector determines whether a user is focusing on the captured object)
a relationship determination module including instructions that, when executed by the processor, cause the processor to determine an existence of a relationship between the object and the human-provided indication; and ¶¶ 71-73, 80, 88, 90-91, a relationship between a gaze of user and the identified object is determined, e.g., by long duration of the user’s gaze on an object)
an image annotation module including instructions that, when executed by the processor, cause the processor to cause the image that includes the representation of the object to be annotated with supplemental information that includes information about an identification of the object; and (¶¶ 71-73, 80, 88, 91, 115, 174, the image includes an object of interest; the object of interest is associated with the user gaze and based on the association the image is annotated with additional information such as location of the object: ¶ 115, “once the object and/or scene is identified, additional information about the object and/or scene is provided to the user. For example, a location of the object and/or scene is determined, and this information is displayed on the display”)
a database query module including instructions that, when executed by the processor, cause the processor to cause, based on the existence of the relationship, the database to produce, in response to a query about a subject of the human-provided indication, information about the object including the supplemental information. (¶¶ 71-73, 80, 88, 91, 115, 174, the image including the object of interest is annotated with additional information about the object and displayed to the user: ¶ 115, “once the object and/or scene is identified, additional information about the object and/or scene is provided to the user. For example, a location of the object and/or scene is determined, and this information is displayed on the display…when the identified object and/or scene is displayed on the display and the user performs a voice query (e.g., "What's that building?"), an answer to the question is determined and output in any suitable manner. As such, entertainment and educational opportunities are provided to users”)
Claims 16 and 20 are rejected under the same rationale as above.
Claim 2. The system of claim 1, wherein at least one of:
the recording is a recording of sound, or the image is a first image and the recording is a second image. (¶ 78, interior and exterior cameras are used for identifying objects and gaze direction)
Claim 3. The system of claim 2, wherein:
the first camera is a first camera, and the second image was produced by a second camera. (¶ 78, interior and exterior cameras are used for identifying objects and gaze direction)
Claim 4. The system of claim 3, wherein:
the first camera is at least one of: a forward-facing camera disposed on a vehicle, or a rearward-facing camera disposed on the vehicle, and the second camera is a cabin view camera disposed on the vehicle. (¶ 78, interior and exterior cameras are used for identifying objects and gaze direction)
Claim 5. The system of claim 1, wherein the human-provided indication comprises at least one of:
a hand gesture, a gaze, or an audible comment. (¶¶ 71-73, 80, 88, 91, 160, objects of interests are captured “by one or more cameras located in, on, or otherwise associated the vehicle” and a gaze detector determines whether a user is focusing on the captured object)
Claim 6. The system of claim 5, wherein at least one of:
the hand gesture is a gesture to point in a specific direction, the gaze is in the specific direction, or the audible comment includes information that signifies the specific direction. (¶ 81, the gaze detector detects gaze direction)
Claim 7. The system of claim 5, wherein at least one of:
the hand gesture signifies an opinion of a human that produced the hand gesture, the gaze signifies an opinion of a human that produced the gaze, or the audible comment signifies an opinion of a human that produced the audible comment. (¶¶ 80, 87, 91-92, a user’s gaze signifies an interest of the user)
Claim 8. The system of claim 1, wherein:
the human-provided indication signifies a specific direction, a location of the object at the second time is in the specific direction from a human that produced the human-provided indication, and (¶¶ 91, 112, a gaze trajectory comprises direction and location of gaze during a first time and a second time)
the instructions to determine the existence of the relationship include instructions to determine that the location of the object at the first time corresponds to the representation of the object included in the image. (¶¶ 71-73, 80, 88, 91, 160 the object of interest in the image is in the direction of the user’s gaze)
Claim 9. The system of claim 8, wherein the memory further stores at least one of:
a hand gesture module including instructions that, when executed by the processor, cause the processor to cause, in response to the human-provided indication including a hand gesture, an operation of a hand gesture technique, the hand gesture technique including: operating gesture recognition technology to determine that the hand gesture is a gesture to point in the specific direction, and producing a hand gesture vector in the specific direction, an origin of the hand gesture vector being a hand arranged to produce the hand gesture, or
a gaze module including instructions that, when executed by the processor, cause the processor to cause, in response to the human-provided indication including a gaze, an operation of a gaze technique, the gaze technique including: operating eye point-of-gaze tracking technology to determine that a point of gaze of an eye is in the specific direction, and producing a gaze vector in the specific direction, an origin of the gaze vector being the eye. (¶ 11, “Eye gaze is tracked in some embodiments. At least one of the user's gaze, a position of one or more of the user's eyes, a direction of the user's gaze, a duration of time that the user's gaze is incident on an object, a speed of movement of the user's gaze, combinations of the same, or the like is determined”, ¶¶ 84-85)
Claim 10. The system of claim 1, wherein the relationship information includes:
the recording that includes the representation of the human-provided indication, and (¶¶ 71-73, 78, 80, 88, 91, 160, a gaze is the human provided indication captured by interior camera and an object in the direction of the gaze is captured by the exterior camera)
at least one of the image that includes the representation of the object or another image that includes the representation of the object. (¶¶ 71-73, 78, 80, 88, 91, 160, a gaze is the human provided indication captured by interior camera and an object in the direction of the gaze is captured by the exterior camera)
Claim 11. The system of claim 10, wherein at least one of:
the image annotation module further includes instructions that cause the processor to cause the other image that includes the representation of the object to be annotated with the supplemental information, or the supplemental information is further based on at least one of information signified by the human provided indication or information produced concurrently with a production of the human provided indication. (¶¶ 71-73, 80, 87-88, 91-92, 115, 174, the object of the interest is identified by the user’s gaze and the object is displayed with further annotation/additional information: ¶ 115, “once the object and/or scene is identified, additional information about the object and/or scene is provided to the user. For example, a location of the object and/or scene is determined, and this information is displayed on the display…when the identified object and/or scene is displayed on the display and the user performs a voice query (e.g., "What's that building?"), an answer to the question is determined and output in any suitable manner. As such, entertainment and educational opportunities are provided to users”)
Claim 23. The method of claim 16, wherein:
the image was produced at a first time, and the recording was produced at a second time. (¶¶ 13, 74, 90, 101, 109, 116, wherein capturing outside scene performed at a first time and the user indication for determining an object of interest is determined at a second time)
Claim 18. The method of claim 23, wherein the second time is after the first time. (¶¶ 13, 74, 90, 101, 109, 116, wherein capturing outside scene and the user indication for determining an object of interest discloses capturing an image of the object at a first time and determining the user indication at the second time and wherein the user indication can be after the first time or before the first time)
Claim 19. The method of claim 23, wherein the second time is before the first time. (¶¶ 13, 74, 90, 101, 109, 116, wherein capturing outside scene and the user indication for determining an object of interest discloses capturing an image of the object at a first time and determining the user indication at the second time and wherein the user indication can be after the first time or before the first time)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 12-15 are rejected under 35 U.S.C. 103(a) as being unpatentable over Li as applied to claims 1 and 10 above in view of Dharia et al., Pub. No.: 2022/0188560 A1 (Dharia).
Claim 12. Li taught the system of claim 10; Li did not explicitly disclose but Dharia discloses: wherein the memory further stores an object recognition and classification module including instructions that, when executed by the processor, cause the processor to cause the object to be recognized and classified. (Dharia, ¶¶ 24-25, 51, 129, wherein identified objects are classified: “the detecting of the at least one object of interest may include determining… a classification of the at least one object of interest”)
Li implicitly discloses the feature by categorizing object of interest such as mountain, horse, etc. as in ¶ 101. It would have been obvious before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine the applied references for disclosing wherein the memory further stores an object recognition and classification module including instructions that, when executed by the processor, cause the processor to cause the object to be recognized and classified because doing so would further provide for recognizing and classifying object of interest explicitly for achieving the same predictable result.
Claim 13. The system of claim 10, wherein the memory further stores an image transformation module including instructions that, when executed by the processor, cause the processor to produce, based on the image that includes the representation of the object, the other image that includes the representation of the object, the other image being a transformation of the image. (Dharia. ¶ 21, “These 5 frames can be used to perform image processing tasks such as, for example, reducing noise, image enhancement, image sharpening, object detection, object tracking, and so forth”)
Claim 14. The system of claim 10, wherein:
the image that includes the representation of the object is a member of a set of images that include the representation of the object, and the memory further stores:
an image quality measurement module including instructions that, when executed by the processor, cause the processor to determine an image, of the set of images, in which a measurement of an image quality of the object is a greatest value; and an image designation module including instructions that, when executed by the processor, cause the processor to designate the image, of the set of images, in which the measurement of the image quality of the object is the greatest value, as the other image that includes the representation of the object. (Li, ¶ 153, Dharia. ¶¶ 100-104, wherein high-quality image data is produced: “The selected mode of operation may be based on the initial image data captured, and may be selected to improve the quality of future images captured in the environment represented by the initial image data… the first image sensor(s) 602 may capture additional image data, which may have higher quality (e.g., has a better exposure, magnification, white balance, etc.) than the initial image data captured”)
Claim 15. The system of claim 10, wherein the memory further stores a map augmentation module including instructions that, when executed by the processor, cause the processor to cause useful map information to be included in a map of a vicinity of a location of the object, the useful map information including at least one of: the at least one of the image that includes the representation of the object or the other image that includes the representation of the object, or supplemental information, the supplemental information being based on at least one of information signified by the human-provided indication or information produced concurrently with a production of the human-provided indication. (Li, ¶ 14, wherein “the user selection of the image and/or the icon triggers turn-by-tum directions via a mapping application” and ¶ 143, “in response to user selection of the supplemental content, turn-by-turn navigation directions are provided”, indicates that a map augmentation module is used for displaying the object of interest as in ¶ 115)
Response to Amendment and Arguments
In view of the amended claims, claim objections are withdrawn.
Applicant’s arguments with respect to amended claims have been fully considered but are moot in view of the new ground of rejections as provided above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. It is suggested that the Applicant review these documents before submitting any amendments.
Applicant’s amendment necessitated the new ground(s) of rejection presented in this office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mohsen Almani whose telephone number is (571)270-7722. The examiner can normally be reached on M-F, 9 AM-5 PM, ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J. Lo can be reached on 571-272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHSEN ALMANI/Primary Examiner, Art Unit 2159
Prosecution Insights