Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
1. This communication is responsive to the Amendment filed 5/26/2026.
2. Claims 21-22, 27-32 and 35-38 and 40-47 are pending in this application. Claims 21, 37 and 40 are independent claims. In the instant Amendment, claims 21-22, 27, 29, 37-38 and 40-41 were amended, claims 26 and 39 were canceled and claims 46-47 were added. This is a Non-Final action on the RCE filed 5/26/2026.
Claim Rejections - 35 USC § 103
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claim(s) 21-22, 27-32 and 35-38 and 40-47 are rejected under 35 U.S.C. 103 as being unpatentable over Hwang et al (“Hwang” US 2018/0217975) in view of Lee et al (“Lee” US 2022/0191157) and further in view of Morris et al (“Morris” US 2018/0017144).
Regarding claim 21, Hwang discloses a method comprising:
at a device including an input device, an environmental sensor, a display, one or more processors, and non-transitory memory (see paragraph [0037]; e.g., GPS sensor; fig. 10: processors, memory);
obtaining, using the environmental sensor, environmental information of a physical environment (see paragraph see paragraph [0037]; e.g., GPS information);
obtaining one or more semantic labels associated with the physical environment based on the environmental information of the physical environment (see paragraph see paragraph [0037]; e.g., words associated with land marks; GPS);
receiving, via the input device, text (see paragraph [0067] or [0073]);
determining a first set of one or more text suggestions based on the text and one or more semantic labels (see paragraph [0067] or [0073]); and
displaying, on the display, the first set of one or more text suggestions (see paragraph [0067]; e.g., as indicated, the word suggestion can be touched i.e. it is display; par. 73: the suggestion provision is in line with fig. 5-6 as indicated in par. 73, and as can be in corresponding par. 53, the suggestions are displayed on the GUN).
Hwang does not expressly disclose wherein the environmental sensor includes at least one of a thermometer, barometer, hygrometer or light sensor; nor determining a second set of one or more text suggestions based on the text independent of the one or more semantic labels; and displaying the set of one or more semantic labels.
However, Lee discloses that it is well-known in the art to configure a messaging device with temperature sensors (see paragraph [0077]; e.g., “Each of the messaging devices 510 is configured with one or more sensors, such as, but not limited to cameras, microphones, temperature sensors, pressure sensors, vehicle or device telemetric sensors, accelerometers, gyroscopes, inertial movement sensors, biometric sensors and/or any other sensors that are capable of detecting, identifying and or analyzing expressive effect data associated with the messaging participant(s) associated with the messaging device(s) 510)”). It would have been obvious to an artisan before the effective filing date of the present invention to include Lee’s teachings in Hwang’s user interface as a well-known alternative with predictable results.
Moreover, Morris discloses determining a second set of one or more text suggestions based on the text independent of the one or more semantic labels; and displaying the set of one or more semantic labels (see fig 4, 406(1) and 406(N); also see paragraphs [0090]-[0092]; e.g., “FIG. 4 also includes an example of suggestions ranked by relatedness to surroundings of a user from most related to the surroundings (i.e., suggestion 406(1) when an environment identifier and/or a salient object label includes “grocery store”) to not related to the surroundings (i.e., suggestion 406(N)).”).
It would have been obvious to an artisan before the effective filing date of the present invention to include Morris’ teachings in Hwang’s user interface in an effort to provide a more user-friendly interface that simplifies user inputs.
Regarding claim 22, Hwang discloses wherein the device further comprises an image sensor configured to capture one or more images of the physical environment (see paragraphs [0021], [0063], [0069]; e.g., "image capture device, such as a head up display or glasses that contain a camera’; par. 63, 69: image capture device 710 e.g. HUD or wearable device such as google glass; fig. 7; par. 93, 95-96), wherein obtaining the one or more semantic labels associated with the physical environment includes detecting an object in the one or more images of the physical environment and obtaining one or more semantic labels associated with the object (see paragraph [0071] and [0073]; e.g., identifying objects).
Regarding claim 27, Hwang discloses wherein the device further comprises a microphone and the environmental information includes audio of the physical environment (see paragraph [0037], [0066] and [0067]).
Regarding claim 28, Hwang discloses wherein obtaining the one or more semantic labels associated with the physical environment includes detecting an event in the audio of the physical environment and obtaining one or more semantic labels associated with the event (see paragraph [0037], [0066] and [0067]).
Regarding claim 29, Hwang discloses wherein the device further comprises a physical locator (see paragraph [0037]; e.g., GPS).
Regarding claim 30, Hwang discloses further comprising determining a respective one or more weights for the one or more semantic labels associated with the physical environment, wherein determining the first set of one or more text suggestion is further based on the respective one or more weights (see paragraphs [0040]-[0042] and [0053]; e.g., determine highest ranking candidate word).
Regarding claim 31, Hwang discloses wherein the text is entered into a text field, further comprising receiving, via the input device, selection of a particular text suggestion of the first set of one or more text suggestions and entering the particular text suggestion in the text field (see paragraph [0073]; e.g., select suggested word).
Regarding claim 32, Morris discloses receiving, via the input device, selection of a particular text suggestion of the first set of one or more text suggestions and performing a search with the particular text suggestion as a search query (see paragraph [0043]; e.g., input search in search query).
Regarding claim 35, Morris discloses wherein the first set of one or more text suggestions are displayed proximate to the text (see fig 4, 406).
Regarding claim 36, Morris discloses wherein the first set of one or more text suggestions are displayed proximate to a keyboard used to receive the text (see fig 4, 406).
Claim 37 is similar in scope to claim 21 and is therefore rejected under similar rationale.
Regarding claim 38, Hwang discloses wherein the one or more processors are to obtain the one or more semantic labels associated with the physical environment by detecting an object in the one or more images of the physical environment and obtaining one or more semantic labels associated with the object (see paragraph [0071] and [0073]; e.g., identifying objects).
Claim 40 is similar in scope to claim 21 and is therefore rejected under similar rationale.
Regarding claim 41, Morris discloses wherein the first set of one or more text suggestions is displayed differently than the second set of one or more text suggestions (see paragraph [0097]; e.g., visual indicator- color, size, etc).
Regarding claim 42, Morris discloses wherein text of the first set of one or more text suggestions is displayed differently than the text of the second set of one or more text suggestions (see paragraph [0097]; e.g., visual indicator- color, size, etc).
Regarding claim 43, Morris discloses wherein text of the first set of one or more text suggestions is a different color than the text of the second set of one or more text suggestions (see paragraph [0097]; e.g., visual indicator- color, size, etc).
Regarding claim 44, Morris discloses wherein the first set of one or more text suggestions is displayed on a first portion of the display separate from a second portion of the display in which the second set of one or more text suggestions is displayed (see fig 4).
Regarding claim 45, Morris discloses wherein the first portion of the display is next to the second portion of the display (see fig 4).
Regarding claim 46, Morris discloses wherein determining the first set of one or more text suggestions comprises increasing a ranking of a candidate text suggestion associated with a semantic label obtained based on the environmental information relative to another candidate text suggestion that is not associated with the semantic label (see fig 4, 406(1) and 406(N); also see paragraphs [0090]-[0092]; e.g., “FIG. 4 also includes an example of suggestions ranked by relatedness to surroundings of a user from most related to the surroundings (i.e., suggestion 406(1) when an environment identifier and/or a salient object label includes “grocery store”) to not related to the surroundings (i.e., suggestion 406(N)).”).
Regarding claim 47, Morris discloses wherein the first set of one or more text suggestions includes a text suggestion associated with a current environmental condition indicated by the environmental information, and wherein the second set of one or more text suggestions is determined without using the current environmental condition (see fig 4, 406(1) and 406(N); also see paragraphs [0090]-[0092]; e.g., “FIG. 4 also includes an example of suggestions ranked by relatedness to surroundings of a user from most related to the surroundings (i.e., suggestion 406(1) when an environment identifier and/or a salient object label includes “grocery store”) to not related to the surroundings (i.e., suggestion 406(N)).”).
Response to Arguments
5. Applicant’s arguments with respect to the claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
6. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ouyang et al (US 2017/0308522).
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RASHAWN N TILLERY whose telephone number is (571)272-6480. The examiner can normally be reached M-F 9:00a - 5:30p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William L Bashore can be reached at (571) 272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RASHAWN N TILLERY/Primary Examiner, Art Unit 2174
Prosecution Insights