Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to continuation request
The request for a continuation is acknowledged. For a continuation to be applied as per the MPEP, the application discloses and claims only subject matter disclosed in prior Applications, and names the inventor or at least one joint inventor named in the prior application. Accordingly, this application may constitute a continuation or divisional. Should applicant desire to claim the benefit of the filing date of the prior application, attention is directed to 35 U.S.C. 120, 37 CFR 1.78, and MPEP § 211 et seq.
Claims 1-20 are currently pending.
Please refer to the action below.
Examiner Notes
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. However, the claimed subject matter, not the specification, is the measure of the invention.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1, and 11 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-3 of US Patent 12117838, “7838”.
Instant Application 18889248 (“9248”)
Allowed Patent US 12117838 (“7838”)
1. A computer-implemented method, comprising: receiving, from at least a first image capture component of a device, first image data of an environment; performing object detection using the first image data to determine an object; based on determining the object, determining first position data corresponding to a first position of the object; receiving first input data representing a natural language input; performing natural language processing on the first input data to generate natural language processing data; determining that the natural language processing data indicates the object; and determining output data corresponding to a natural language description of the first position.
2. The computer-implemented method of claim 1, further comprising: determining second position data corresponding to the device, wherein the first position data is determined based at least in part on the second position data.
3. The computer-implemented method of claim 1, further comprising: determining time data corresponding to a first time at which the object was at the first position; and including in the output data a representation of a natural language description of the time data.
4. The computer-implemented method of claim 1, wherein the natural language input is captured by the device.
5. The computer-implemented method of claim 1, wherein: the first input data comprises first audio data representing speech; performing natural language processing on the first input data to generate natural language processing data comprises performing speech processing on the first audio data to generate speech processing data; and determining that the natural language processing data indicates the object comprises determining that the speech processing data indicates the object.
6. The computer-implemented method of claim 1, further comprising: performing text-to-speech processing using the output data to determine output audio data representing synthesized speech indicating the first position.
7. The computer-implemented method of claim 1, wherein the first input data is received after determination of the first position data.
8. The computer-implemented method of claim 1, further comprising, prior to receiving the first input data: receiving second image data; performing object detection using the second image data to determine the object; based on determining the object using the second image data, determining second position data corresponding to a second position of the object; and after determining the first position data, determining the second position data does not correspond to a current position of the object.
9. The computer-implemented method of claim 1, wherein the natural language input corresponds to a request for a location of the object.
10. The computer-implemented method of claim 1, wherein the natural language input corresponds to a request for the device to move to a location of the object.
11. A system comprising: at least one processor; and at least one memory comprising instructions that, when executed by the at least one processor, cause the system to: receive, from at least a first image capture component of a device, first image data of an environment; perform object detection using the first image data to determine an object; based on determination of the object, determine first position data corresponding to a first position of the object; receive first input data representing a natural language input; perform natural language processing on the first input data to generate natural language processing data; determine that the natural language processing data indicates the object; and determine output data corresponding to a natural language description of the first position.
12. The system of claim 11, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine second position data corresponding to the device, wherein the first position data is determined based at least in part on the second position data.
13. The system of claim 11, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine time data corresponding to a first time at which the object was at the first position; and include in the output data a representation of a natural language description of the time data.
14. The system of claim 11, wherein the natural language input is captured by the device.
15. The system of claim 11, wherein: the first input data comprises first audio data representing speech; the instructions that cause the system to perform natural language processing on the first input data to generate natural language processing data comprise comprises instructions that, when executed by the at least one processor, cause the system to perform speech processing on the first audio data to generate speech processing data; and the instructions that cause the system to determine that the natural language processing data indicates the object comprise comprises instructions that, when executed by the at least one processor, cause the system to determine that the speech processing data indicates the object.
16. The system of claim 11, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: perform text-to-speech processing using the output data to determine output audio data representing synthesized speech indicating the first position.
17. The system of claim 11, wherein the first input data is received after determination of the first position data.
18. The system of claim 11, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to, prior to receipt of the first input data: receive second image data; perform object detection using the second image data to determine the object; based on determination of the object using the second image data, determine second position data corresponding to a second position of the object; and after determination of the first position data, determine the second position data does not correspond to a current position of the object.
19. The system of claim 11, wherein the natural language input corresponds to a request for a location of the object.
20. The system of claim 11, wherein the natural language input corresponds to a request for the device to move to a location of the object.
1. A computer-implemented method, the method comprising: receiving environment data representing a three-dimensional map of an environment; moving, by a device, to a first location in the environment; determining a configuration of a mechanical component of the device, the mechanical component comprising a camera; determining first position data representing the first location and the configuration; receiving, from the camera, first image data representing the environment; performing object detection using the first image data to determine an object; based on determining the object, determining first stored data corresponding to a previous location of the object; determining, using the environment data and the first position data, a first direction in which the camera is directed while the device is at the first location; determining a first bounding box corresponding to a portion of the first image data representing the object; based at least in part on the first position data and the first direction, determining the first bounding box corresponds to a second location; determining second position data corresponding to the second location; and determining second stored data corresponding to the object being located at the second location.
2. The computer-implemented method of claim 1, further comprising: receiving, by the device, first audio data representing speech of a user; performing speech processing on the first audio data to generate speech processing data; determining that the speech processing data indicates the object; and causing an action to be performed based at least in part on the second stored data.
3. The computer-implemented method of claim 2, further comprising: receiving, by the device, second image data including a second representation of the environment; processing the second image data to determine at least one of a first direction in which a face of the user is oriented or a second direction in which the user is pointing; and determining that at least one of the first direction or the second direction is associated with the second stored data, wherein determining that the speech processing data indicates the object is based at least in part on determining that at least one of the first direction or the second direction is associated with the second stored data.
Regarding Instant independent claims 1, and 11 corresponding to respectively at least claims 1-3 of the allowed patent “7838”:
Although the claims at issue are not identical, they are not patentably distinct from each other. As at least claims 1-3 of the patent “7838” collectively encompasses all the teachings of the instant claims 1, and 11.
It has been held that the generic invention is “anticipated” by the “species”. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993). Since instant application claims 1, and 11 are at least anticipated and obvious by the combined claims 1-3 of the reference patent.
As one skill in the art would further appreciate that the methods/systems of the patent “7838” means to receive, via the capturing device, image data of an environment while said capturing device is directed at the first location, to secondly perform object detection using the first image data to determine an object; based on determining the object, determining first position data corresponding to a first position of the object; the system further in claims 2-3 detect and identify a user second input instruction indicative of second image data by determining at least in claim 3 at least one of a first direction in which a face of the user is oriented or a second direction in which the user is pointing at as the second input instruction from which the system further correlates that said second input instruction represent a user query position search/question of the previously detected object; and the system further in at least claim subsequently configured to cause an action to be performed based at least in part on the second data corresponding to the query position instruction of the user. Thereby would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of the instant claims to include wherein said receiving, performing and determining for at least determining output data corresponding to a natural language description of the first position, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Dependent claims 2-10, and 12-20 are also rejected as they failed to solve the above problem.
Accordingly, the claimed subject matter of this application as currently claimed is unpatentable under the provisions of the nonstatutory obviousness-type double patenting rejection. Therefore, those claims are rejected as best understood by examiner as indicated in this office action above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 is/are further rejected under 35 U.S.C. 103 as obvious over Rosenstein et al. (US 2012/0185094, A1) in view of Nambiar et al. (WO 2018033715, A1).
Regarding claim 1, Rosenstein teaches a computer-implemented method (Figs. 1-3 and 17, 31 and para. 0133, 0190, and 0254-0263 teaches one or more first image capture component 400/450 of a robotic device 100 further configured for capturing at least one more image data of an environment and to assist at least in para. 0133 a user object in locating in the environment, based on a detected gesture, an object of interest and/or directing the device 100, based on a detected gesture, to the object of interest) comprising:
receiving, from at least a first image capture component of a device, first image data of an environment (a controller 500 of at least Figs. 17 and para. 0190, 0254-0257 further configured for receiving, from at least a first image capture component 400/450, first image data of an environment depicted in at least Figs. 17 and para. 0190, 0254-0257);
performing object detection using the first image data to determine an object (further object detection performing of at least Figs. 17, and 31 and para. 0190, and 0254-0263 to determine one or more detected objects);
based on determining the object, determining first position data corresponding to a first position of the object (at least para. 0146, 0190, 0226, and 0262-0263 further teaches tracked and determined movement positions of the detected objects in the environment scene, based on determining the object, said at least one or more first tracked position data corresponding to a first position of the object may comprises in the art previously detected positions of the detected objects, current positions, and/or updated positions of the objects in the environment);
receiving first input data representing a natural language input (the device of at least para. 0133, 0255-0256 and 0263 is configured further to recognize an inputted user gesture of a person such as “determined gesture(s) (e.g., hand pointing, waving, and or hand signals). For example, the controller 500 may issue a drive command in response to a recognized hand point in a particular direction ………..gesture event can be raised for moved objects 12, and in response to that event, the operations may include sending the robot 100 to the corresponding room 3020 for verifying the object's location” as the at least received first input data);
performing natural language processing on the first input data to generate natural language processing data (para. 0254-0256 further teaches the performing natural language processing such as “The identified body gesture may be classified and an event raised based on the classification. Additional details and features on gesture recognition, which may combinable with those described herein, can be found in U.S. Pat. No. 7,340,077, Entitled "Gesture Recognition System Using Depth Perceptive Sensors", the contents of which are hereby incorporated by reference in its entirety” to generate understoodly said natural language processing data for in a case provide assistance or to go locate a person of the environment scene needing assistance);
Rosenstein is silent regarding wherein the above lined-out items such as determining that the natural language processing data indicates the object; and determining output data corresponding to a natural language description of the first position.
Nambiar teaches at least at least in Figs. 18, and 20 receiving, from at least a first image capture component of a device, first image data of an environment and performing object detection and based on determining the object, determining first position data corresponding to a first position of the object; Nambiar further teaches in the disclosure and Figs. 18, and 20 “locate an object in response to a user prompt made at a node” as the receiving first input data representing a natural language input to locate one or more misplaced objects detected in one or more environment maps or the image data of an environment; the system further teaches in the disclosure “may retrieve the most recent time at which a change in the location of a specific object located within a room was identified in combination with the user having previously been located near to the location of the object, and output this information to the user……. system will then identify the last known position of the mobile phone when it was visible on the table. In this way, the system may be able to infer the current location of the mobile phone from previously stored mapping information.” further implying in the art determining that the natural language prompt processing data indicates the misplaced object and determining output data to the user corresponding to obviously the natural language prompt description of the first detected position of the misplaced object. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Rosenstein in view of Nambiar to include wherein determining that the natural language processing data indicates the object; and determining output data corresponding to a natural language description of the first position, as discussed above, as Rosenstein in view of Nambiar are in the same of endeavor of generating at least an environmental map of a detected scene to at least detect and locate objects location, positions and types, furthermore, Nambiar’s combination of objects detection and recognition in addition to telling and finding based on a user query location of a misplaced/missing objects in the detected environment further complements the objects recognition of Rosenstein, in a sense that when combined with the determining and locating, based on the user query, current and previous location of the misplaced/missing objects architecture of Nambiar, it enables the methods and systems of Rosenstein to further assist at least a recognized human object of Rosenstein, via text and/or audible output response, the locating of possible misplaced objects and/or to tell the human object of the current and/or previous detected positions of the target objects based in a case of detecting the user facing and staring at a specific location of a possible missing or misplaced object, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 2 (according to claim 1), Rosenstein further teaches wherein further comprising: determining second position data corresponding to the device (para. 0254-0263 further teaches the detecting and tracking of moved objects of the detected environment, where the tracked moves comprises at least in the art determining second position data corresponding to the device in the environment);
wherein the first position data is determined based at least in part on the second position data (a case further in at least para. 0254-0263 of locating a possible injured object as one of the object detected in the environment where said object first position data is determined based obviously at least in part on the second position data).
Regarding claim 3 (according to claim 1), Rosenstein further teaches wherein further comprising: determining time data corresponding to a first time at which the object was at the first position (the system further in at least para. 0167 is setup for at least timestamp captured object images corresponding to a first time at which the object was obviously at the first position);
and including in the output data a representation of a natural language description of the time data (an object output generated further in at least para. 0193 further comprises updated location and position data further comprises in a case an output data a representation of a natural language description disclosed in para. 0133 of the time data).
Regarding claim 4 (according to claim 1), Rosenstein further teaches wherein the natural language input is captured by the device (the captured gestures of further of 0255-0263 further comprises in the art received natural language input captured by the device).
Regarding claim 5 (according to claim 1), Rosenstein further teaches wherein: the first input data comprises first audio data representing speech (the device of at least para. 0266 may receive other than a gesture command from the user object audible query as said first input data);
performing natural language processing on the first input data to generate natural language processing data comprises performing speech processing on the first audio data to generate speech processing data (the device of further para. 0266 is configured to receive audible response and performed an action based on the provided audible response thereby obviously adapted for performing said natural language processing on the first input data to generate natural language processing data comprises performing speech processing on the first audio data to generate speech processing data); and
determining that the natural language processing data indicates the object comprises determining that the speech processing data indicates the object (the device of further para. 0266 based on query to find and locate the fallen object, detect the fallen object based obviously on the said processing data thereby understoodly determining that said natural language processing data indicates the fallen object comprises said determining that the speech processing data indicates the object).
Regarding claim 6 (according to claim 1), Rosenstein further implies wherein further comprising: performing text-to-speech processing audible response to a user query thereby obviously capable of providing or performing obviously text-to-speech processing).
However, Rosenstein is silent regarding wherein said text-to-speech processing using the output data to determine output audio data representing synthesized speech indicating the first position.
Nambiar teaches in at least step 2002 and S2010 inputting of at least the first input command comprising at least a natural language request of implied text via the user interface 308, voice, audio, gestures or the like and further the object output module 1706 configured to output via speech, text or the like by performing at least as understood in the art text-to-speech processing or the like using the output data to determine output audio data representing synthesized speech indicating detected previous/current first position of the misplaced or lost objects.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Rosenstein in view of Nambiar to include wherein performing said text-to-speech processing using the output data to determine output audio data representing synthesized speech indicating the first position, as discussed above, as Rosenstein in view of Nambiar are in the same of endeavor of generating at least an environmental map of a detected scene to at least detect and locate objects location, positions and types, furthermore, Nambiar’s combination of objects detection and recognition in addition to telling and finding based on a user query location of a misplaced/missing objects in the detected environment further complements the objects recognition of Rosenstein, in a sense that when combined with the determining and locating, based on the user query, current and previous location of the misplaced/missing objects architecture of Nambiar, it enables the methods and systems of Rosenstein to further assist at least a recognized human object of Rosenstein, via text and/or audible output response, the locating of possible misplaced objects and/or to tell the human object of the current and/or previous detected positions of the target objects based in a case of detecting the user facing and staring at a specific location of a possible missing or misplaced object, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 7 (according to claim 1), Rosenstein further teaches wherein the first input data is received after determination of the first position data (the input of at least para. 0133 and 0263 citing gestures such as “determined gesture(s) (e.g., hand pointing, waving, and or hand signals). For example, the controller 500 may issue a drive command in response to a recognized hand point in a particular direction ………..gesture event can be raised for moved objects 12, and in response to that event, the operations may include sending the robot 100 to the corresponding room 3020 for verifying the object's location” as the at least received first input data) further comprises in the art one or more first input natural language data received after determination of obviously a previous first position data).
Regarding claim 8 (according to claim 1), Rosenstein further teaches wherein further comprising, prior to receiving the first input data:
receiving second image data (the system may in at least para. 0257-0263 may receive image data of moved objects as the said second image data before the receiving of the first input assistance query data);
performing object detection using the second image data to determine the object (the system further in at least para. 0262-0263 further comprises performing object detection using the second falling image data to determine the target object needing assistance);
based on determining the object using the second image data, determining second position data corresponding to a second position of the object (para. 0262-0263); and after determining the first position data, determining the second position data does not correspond to a current position of the object (sending the robot 100 of further para. 0262-0263 further entails comparing the corresponding room 3020 previous object position data to current objects position data for verifying the object's location which may in a case result in a case determining that said second position data obviously in a case does not correspond to a current position of the object).
Regarding claim 9 (according to claim 1), Rosenstein further teaches wherein the natural language input corresponds to a request for a location of the object (para. 0133 and 0262-0263 further discloses received natural language inputs corresponding to a request for a location verification of the object).
Regarding claim 10 (according to claim 1), Rosenstein further teaches wherein the natural language input corresponds to a request for the device to move to a location of the object (para. 0133 and 0262-0263 further discloses received natural language inputs corresponding to a request for the device 100 to move as noted further in para. 0133 and 0262-0263 to a location of the user object).
Regarding claim 11, Rosenstein teaches a system (Figs. 1-3 and 17, 31 and para. 0133, 0190, and 0254-0263 teaches said system comprising at least one or more first image capture component 400/450 of a robotic device 100 further configured for capturing at least one more image data of an environment and to assist at least in para. 0133 a user object in locating in the environment, based on a detected gesture, an object of interest and/or directing the device 100, based on a detected gesture, to the object of interest) comprising:
at least one processor (para. 0269); and
at least one memory (para. 0269) comprising instructions that, when executed by the at least one processor, cause the system to:
receive, from at least a first image capture component of a device, first image data of an environment (a controller 500 of at least Figs. 17 and para. 0190, 0254-0257 further configured for receiving, from at least a first image capture component 400/450, first image data of an environment depicted in at least Figs. 17 and para. 0190, 0254-0257);
perform object detection using the first image data to determine an object (further object detection performing of at least Figs. 17, and 31 and para. 0190, and 0254-0263 to determine one or more detected objects);
based on determination of the object, determine first position data corresponding to a first position of the object (at least para. 0146, 0190, 0226, and 0262-0263 further teaches tracked and determined movement positions of the detected objects in the environment scene, based on determining the object, said at least one or more first tracked position data corresponding to a first position of the object may comprises in the art previously detected positions of the detected objects, current positions, and/or updated positions of the objects in the environment);
receive first input data representing a natural language input (the device of at least para. 0133, 0255-0256 and 0263 is configured further to recognize an inputted user gesture of a person such as “determined gesture(s) (e.g., hand pointing, waving, and or hand signals). For example, the controller 500 may issue a drive command in response to a recognized hand point in a particular direction ………..gesture event can be raised for moved objects 12, and in response to that event, the operations may include sending the robot 100 to the corresponding room 3020 for verifying the object's location” as the at least received first input data);
perform natural language processing on the first input data to generate natural language processing data (para. 0254-0256 further teaches the performing natural language processing such as “The identified body gesture may be classified and an event raised based on the classification. Additional details and features on gesture recognition, which may combinable with those described herein, can be found in U.S. Pat. No. 7,340,077, Entitled "Gesture Recognition System Using Depth Perceptive Sensors", the contents of which are hereby incorporated by reference in its entirety” to generate understoodly said natural language processing data for in a case provide assistance or to go locate a person of the environment scene needing assistance);
Rosenstein is silent regarding wherein the above lined-out items such as specifically determine that the natural language processing data indicates the object; and determine output data corresponding to a natural language description of the first position.
Nambiar teaches at least at least in Figs. 18, and 20 receiving, from at least a first image capture component of a device, first image data of an environment and performing object detection and based on determining the object, determining first position data corresponding to a first position of the object; Nambiar further teaches in the disclosure and Figs. 18, and 20 “locate an object in response to a user prompt made at a node” as the receiving first input data representing a natural language input to locate one or more misplaced objects detected in one or more environment maps or the image data of an environment; the system further teaches in the disclosure “may retrieve the most recent time at which a change in the location of a specific object located within a room was identified in combination with the user having previously been located near to the location of the object, and output this information to the user……. system will then identify the last known position of the mobile phone when it was visible on the table. In this way, the system may be able to infer the current location of the mobile phone from previously stored mapping information.” further implying in the art determining that the natural language prompt processing data indicates the misplaced object and determining output data to the user corresponding to obviously the natural language prompt description of the first detected position of the misplaced object. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Rosenstein in view of Nambiar to include wherein determine that the natural language processing data indicates the object; and determine output data corresponding to a natural language description of the first position, as discussed above, as Rosenstein in view of Nambiar are in the same of endeavor of generating at least an environmental map of a detected scene to at least detect and locate objects location, positions and types, furthermore, Nambiar’s combination of objects detection and recognition in addition to telling and finding based on a user query location of a misplaced/missing objects in the detected environment further complements the objects recognition of Rosenstein, in a sense that when combined with the determining and locating, based on the user query, current and previous location of the misplaced/missing objects architecture of Nambiar, it enables the methods and systems of Rosenstein to further assist at least a recognized human object of Rosenstein, via text and/or audible output response, the locating of possible misplaced objects and/or to tell the human object of the current and/or previous detected positions of the target objects based in a case of detecting the user facing and staring at a specific location of a possible missing or misplaced object, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 12 (according to claim 11), Rosenstein further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine second position data corresponding to the device (para. 0254-0263 further teaches the detecting and tracking of moved objects of the detected environment, where the tracked moves comprises at least in the art determining second position data corresponding to the device in the environment);
wherein the first position data is determined based at least in part on the second position data (a case further in at least para. 0254-0263 of locating a possible injured object as one of the object detected in the environment where said object first position data is determined based obviously at least in part on the second position data).
Regarding claim 13 (according to claim 11), Rosenstein further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine time data corresponding to a first time at which the object was at the first position (the system further in at least para. 0167 is setup for at least timestamp captured object images corresponding to a first time at which the object was obviously at the first position);
and including in the output data a representation of a natural language description of the time data (an object output generated further in at least para. 0193 further comprises updated location and position data further comprises in a case an output data a representation of a natural language description disclosed in para. 0133 of the time data).
Regarding claim 14 (according to claim 11), Rosenstein further teaches wherein the natural language input is captured by the device (the captured gestures of further of 0255-0263 further comprises in the art received natural language input captured by the device).
Regarding claim 15 (according to claim 11), Rosenstein further teaches wherein: the first input data comprises first audio data representing speech (the device of at least para. 0266 may receive other than a gesture command from the user object audible query as said first input data);
the instructions that cause the system to perform natural language processing on the first input data to generate natural language processing data comprise comprises instructions that, when executed by the at least one processor, cause the system to perform speech processing on the first audio data to generate speech processing data (the device of further para. 0266 is configured to receive audible response and performed an action based on the provided audible response thereby obviously adapted for performing said natural language processing on the first input data to generate natural language processing data comprises performing speech processing on the first audio data to generate speech processing data); and
and the instructions that cause the system to determine that the natural language processing data indicates the object comprise comprises instructions that, when executed by the at least one processor, cause the system to determine that the speech processing data indicates the object (the device of further para. 0266 based on query to find and locate the fallen object, detect the fallen object based obviously on the said processing data thereby understoodly determining that said natural language processing data indicates the fallen object comprises said determining that the speech processing data indicates the object).
Regarding claim 16 (according to claim 11), Rosenstein further implies wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: perform text-to-speech processing
However, Rosenstein is silent regarding wherein said text-to-speech processing using the output data to determine output audio data representing synthesized speech indicating the first position.
Nambiar teaches in at least step 2002 and S2010 inputting of at least the first input command comprising at least a natural language request of implied text via the user interface 308, voice, audio, gestures or the like and further the object output module 1706 configured to output via speech, text or the like by performing at least as understood in the art text-to-speech processing or the like using the output data to determine output audio data representing synthesized speech indicating detected previous/current first position of the misplaced or lost objects. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Rosenstein in view of Nambiar to include wherein performing said text-to-speech processing using the output data to determine output audio data representing synthesized speech indicating the first position, as discussed above, as Rosenstein in view of Nambiar are in the same of endeavor of generating at least an environmental map of a detected scene to at least detect and locate objects location, positions and types, furthermore, Nambiar’s combination of objects detection and recognition in addition to telling and finding based on a user query location of a misplaced/missing objects in the detected environment further complements the objects recognition of Rosenstein, in a sense that when combined with the determining and locating, based on the user query, current and previous location of the misplaced/missing objects architecture of Nambiar, it enables the methods and systems of Rosenstein to further assist at least a recognized human object of Rosenstein, via text and/or audible output response, the locating of possible misplaced objects and/or to tell the human object of the current and/or previous detected positions of the target objects based in a case of detecting the user facing and staring at a specific location of a possible missing or misplaced object, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 17 (according to claim 11), Rosenstein further teaches wherein the first input data is received after determination of the first position data (the input of at least para. 0133 and 0263 citing gestures such as “determined gesture(s) (e.g., hand pointing, waving, and or hand signals). For example, the controller 500 may issue a drive command in response to a recognized hand point in a particular direction ………..gesture event can be raised for moved objects 12, and in response to that event, the operations may include sending the robot 100 to the corresponding room 3020 for verifying the object's location” as the at least received first input data) further comprises in the art one or more first input natural language data received after determination of obviously a previous first position data).
Regarding claim 18 (according to claim 11), Rosenstein further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to, prior to receipt of the first input data: receive second image data (the system may in at least para. 0257-0263 may receive image data of moved objects as the said second image data before the receiving of the first input assistance query data);
perform object detection using the second image data to determine the object (the system further in at least para. 0262-0263 further comprises performing object detection using the second falling image data to determine the target object needing assistance);
based on determination of the object using the second image data, determine second position data corresponding to a second position of the object (para. 0262-0263); and after determination of the first position data, determine the second position data does not correspond to a current position of the object (sending the robot 100 of further para. 0262-0263 further entails comparing the corresponding room 3020 previous object position data to current objects position data for verifying the object's location which may in a case result in a case determining that said second position data obviously in a case does not correspond to a current position of the object).
Regarding claim 19 (according to claim 11), Rosenstein further teaches wherein the natural language input corresponds to a request for a location of the object (para. 0133 and 0262-0263 further discloses received natural language inputs corresponding to a request for a location verification of the object).
Regarding claim 20 (according to claim 11), Rosenstein further teaches wherein the natural language input corresponds to a request for the device to move to a location of the object (para. 0133 and 0262-0263 further discloses received natural language inputs corresponding to a request for the device 100 to move as noted further in para. 0133 and 0262-0263 to a location of the user object).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARCELLUS AUGUSTIN whose telephone number is (571)270-3384. The examiner can normally be reached 9 AM- 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BENNY TIEU can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARCELLUS J AUGUSTIN/Primary Examiner, Art Unit 2682 06/24/2026
Prosecution Insights