DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Allowable Subject Matter
Claim 6 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 17-19 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 17 is amended to recite “indicating that there is a possibility of an end of utterance by the nearby person before an actual utterance by the nearby person is started.” However, the written description describes an end possibility suggestion notification taking place mid-utterance (para. 0098), not pre-utterance.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-5, 7, 9-12, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yasuda (US 2018/0260187 A1) in view of Mese et al. (US 11,184,560 B1, “Mese”).
As to claims 1, 20, Yasuda discloses an information processing system comprising a processor (Fig. 2),
wherein the processor acquires status information that is information related to a status of a nearby person who is a person located around a subject (information processing device 100 acquires line-of-sight or face direction information from communication unit 14 to determine whether or not there is any person facing in a direction of the user A near the user A, para. 0050-0054), and
when the status specified by the status information acquired is a particular status, the processor generates control information that is used for controlling a device owned by the subject and causes the device to issue a notification indicating that there is a possibility of utterance by the nearby person (on the basis of the received line-of-sight information, the surrounding image information, etc., a notification is indicated to user A that the possibility is high that the user A is spoken to so that the user A can respond in the direction in which the person X is located; para. 0062, 072-0073, 0102) before an actual utterance by the nearby person is started.
Yasuda differs from claim 1 in that it does not disclose the above underlined limitation.
Mese teaches determining that a video conference participant is about to speak, e.g. when the participant is opening his/her mouth, and cuing the system that the participant is about to speak (col. 10, lines 38-64). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yasuda with the above teaching of Mese in order to provide earlier notification to the user.
As to claim 2, Yasuda in view of Mese teaches: wherein the processor acquires the status information of the nearby person based on a video in which the nearby person appears (Yasuda: imaging units 10a, 10b image the area around the user A in a format of a moving image, para. 0052-0053; face direction of the person X, Y or Z near the user A is determined based on the captured surrounding image information, para. 0054).
As to claim 4, Yasuda in view of Mese teaches: wherein the processor generates, as the control information that causes the device to issue the notification indicating that there is the possibility of utterance, control information that causes the device to issue a notification appealing to a vision of the subject (Yasuda: notification control unit 126 changes the display in order to urge the user A to respond to person X, para. 0101-0102, 0108-0109).
As to claim 5, Yasuda in view of Mese teaches: wherein the processor generates, as the control information that causes the device to issue the notification indicating that there is the possibility of utterance, control information that causes a display unit of the device to display a display image that is an image displayed on the device and indicating that there is the possibility of utterance by the nearby person before the actual utterance by the nearby person is started (Yasuda: balloon in Fig. 7 indicates the position and direction from which a person X is determined to be speaking, para. 0102; Mese: participant is about to speak, col. 10, lines 38-64).
As to claim 7, Yasuda in view of Mese teaches: wherein the processor generates, as the control information that causes the display image to be displayed on the display unit of the device, control information that causes the display image to be displayed in association with the nearby person (Yasuda: balloon in Fig. 7 indicates the position and direction from which a nearby person X is determined to be speaking, para. 0102; position of the nearby person X is displayed with respect to the user A in Fig. 8).
As to claim 9, Yasuda in view of Mese teaches: wherein, when the nearby person who has the possibility of utterance actually makes an utterance, utterance content that is content of the utterance is displayed on the display unit of the device, and the processor generates, as the control information that causes the display image to be displayed on the display unit of the device, control information that causes the display image to be displayed in association with a display section in which the utterance content is displayed in the display unit of the device (Yasuda: text of the utterance is displayed, Fig. 10, para. 0108).
As to claim 10, Yasuda in view of Mese teaches: wherein the processor generates, as the control information that causes the display image to be displayed in association with the display section, control information that causes the display image having a shape surrounding the display section to be displayed on the display unit (Yasuda: Fig. 7, display image “!” is surrounded by a speech balloon shape, para. 0102).
As to claim 11, Yasuda in view of Mese teaches: wherein, when the nearby person who is out of a range of a field of view of the subject who visually recognizes a front of the subject by the device is in the particular status, the processor generates control information that causes the display image that is an image indicating that there is a possibility of utterance by the nearby person who is out of the range of the field of view to be displayed on the display unit of the device (Yasuda: Fig. 8, display indicates person X speaks to the user A from the right rear, para. 0103).
As to claim 12, Yasuda in view of Mese teaches: wherein the processor generates, as the control information that causes the display image of the nearby person out of the range of the field of view to be displayed on the display unit of the device, control information that causes the display image to be displayed on a line connecting a position of the nearby person and a center of the display unit when the nearby person is projected onto a virtual plane along the display unit (Yasuda: Fig. 8 shows arrows in the form of a line which connect the position of the nearby person X to the user A in a radar map LM centering on the position of the user A, para. 0103).
Claim(s) 3, 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yasuda in view of Mese, as applied to claim 1 above, and further in view of Pruthi et al. (US 2015/0036856 A1, “Pruthi”).
Yasuda in view of Mese differs from claim 3 in that although it analyzes persons in the surrounding image, which includes other nearby persons Y and Z, for example, who may not be speaking (Yasuda: Fig. 1, para. 0051-0054, 0107), it does not disclose: when there is no utterance by the nearby person, the processor generates control information that causes the device to issue a notification indicating that there is no utterance.
Pruthi teaches displaying an indication that a nearby person is currently not speaking (Fig. 5, element 530; para. 0028). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yasuda in view of Mese with the above teaching of Pruthi in order to provide more complete information regarding the surrounding image.
As to claim 8, Yasuda in view of Mese and Pruthi teaches: wherein, when a plurality of the nearby persons is in the particular status, the processor generates, as the control information that causes the display image to be displayed in association with the nearby persons, control information that causes the display image to be displayed in association with each of the plurality of nearby persons (Pruthi: see Fig. 5).
Claim(s) 13-15, 17, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yasuda in view of Mese, as applied to claim 1 above, and further in view of Kimber et al. (US 2006/0132607 A1, “Kimber”).
As to claim 17, Yasuda discloses an information processing system comprising a processor, wherein the processor acquires status information that is information related to a status of a nearby person who is located around a subject and is uttering, and when the status specified by the status information acquired is a particular status (Yasuda: information processing device 100 detects the motion of a surrounding person X speaking to the user A, para. 0077; particular status acquired may be after moving images and sound is interrupted, and determining the person X is no longer speaking to person X, para. 0078-0079).
Yasuda in view of Mese differs from claim 13 in that it does not disclose: wherein, in addition to pre-utterance status information that is the status information acquired before the nearby person actually utters, the processor further acquires mid-utterance status information that is information related to a status of the nearby person who actually has started an utterance, and when the status specified by the mid-utterance status information acquired is a particular status, and from claims 13 and 17 in that it does not disclose: the processor generates control information that is used for controlling the device and that causes the device to issue an end suggestion notification that is a notification indicating that there is a possibility of an end of utterance by the nearby person or generates control information that causes the device to issue an end notification that is a notification indicating that the utterance of the nearby person has ended.
Kimber teaches determining and indicating when a person is about to speak or is about to give up the floor (para. 0046) and the use of a control button by a requesting conference member which may indicate a communicating person is about to give up the floor (para. 0046). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yasuda in view of Mese with the above teaching of Kimber in order to prepare the user for speaker transitions, thus, improving the flow of the conversation.
As to claims 14, 19, Yasuda in view of Mese and Kimber teaches: wherein, when the status specified by the pre-utterance status information is the particular status, the processor generates control information that causes a display image determined in advance to be displayed on the display unit provided in the device, and generates, as the control information that causes the device to issue the end suggestion notification, control information that causes the display mode of the display image displayed on the display unit to be changed (Kimber: display object becomes lighter or darker to indicate that a user is about to speak or about to stop speaking, para. 0046).
As to claim 15, Yasuda in view of Mese and Kimber teaches: wherein the processor generates, as the control information that causes the device to issue the end notification, control information that causes the display mode of the display image whose display mode has been changed by the control information that causes the display mode of the display image to be changed to be further changed (Yasuda: image information is stopped after the motion of the person X speaking to the user A has ended, para. 0078).
Claim(s) 16, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yasuda in view of Mese and Kimber, as applied to claim 13 above, and further in view of Taki et al. (US 2023/0064042 A1, “Taki”).
As to claim 16, Yasuda in view of Mese and Kimber teaches: wherein the processor acquires the pre-utterance status information based on a video in which the nearby person appears (Yasuda: imaging units 10a, 10b image the area around the user A in a format of a moving image, para. 0052-0053; face direction of the person X, Y or Z near the user A is determined based on the captured surrounding image information, para. 0054; Kimber: determines user is about to speak based on a member’s mouth motion, body motion/gesture, etc., para. 0046), but differs from claims 16 and 18 in that it does not teach: acquires the mid-utterance status information based on voice information that is information related to voice of the nearby person.
Taki teaches predicting during an utterance that an utterance end is approaching based on a degree of decrease in inspiration information (para. 0078-0079). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yasuda in view of Mese and Kimber with the above teaching of Taki in order to detect an approaching end of utterance based on captured audio as an alternative or as a supplement to captured video.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Desai et al. (US 11,606,508 B1) teach identifying gestures which indicate that one or more individuals are about to speak (Abstract). Tosh et al. (US 2022/0308825 A1) teach recognizing facial gestures indicating a user is about to speak (para. 0030).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stella L Woo whose telephone number is (571)272-7512. The examiner can normally be reached Monday - Friday, 8 a.m. to 5 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached at 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Stella L. Woo/ Primary Examiner, Art Unit 2693
Prosecution Insights