Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendments/Remarks
Applicant’s Amendments/Remarks filed on 12/09/2025 have been received and made of record.
Claims 1, and 21 have been amended.
Claim 2 has been cancelled.
Claims 1, and 3-24 remained pending.
Please refer to the action below.
Examiner Notes
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. However, the claimed subject matter, not the specification, is the measure of the invention.
Response to Remarks/Arguments
Applicants’ arguments of 12/09/2025, corresponding to pages 8-10 pertaining to currently amended independent claims 1 and 21 citing “obtaining data representing a user input identifying a first portion of the scene in real space, wherein the data representing the user input includes data representing a physical gesture of the user” further associated with the remarks of “respectfully submitted that the cited art respectively fail to disclose each and every feature of the claims. Independent claim 1, as amended, recites "[a] computer implemented method for providing feedback to a user input, the method comprising: obtaining data representing a scene model derived from scene data representing a scene in real space; obtaining data representing a user input identifying a first portion of the scene in real space, wherein the data representing the user input includes data representing a physical gesture of the user; estimating, based on the user input data, a first position within the scene model corresponding to the first portion of the scene; determining control information for controlling a light source to direct light at or near a second position in real space corresponding to the first position within the scene model; and outputting the control information to control a light source, thereby to provide for feedback to the user input." Claim 1 has been amended to incorporate dependent claim 2. Accordingly, as claim 2 is not rejected in view of D3 in the Office action, D3 clearly fails to disclose each and every feature of claim 1. Regarding D1 and D2, it is submitted that the cited art (individually or in combination) does not disclose a physical gesture user input that "identifies a first portion of the scene in real space". D1, for example, discloses gestures for action commands, such as selection or movement of a tool (e.g. "controls the magnet" as noted in the Office Action). D2 describes gestures to control a searchlight such that the gestures control brightness, type of filter, etc. (see D2; paragraph [0023]). Accordingly, the gesture input in D1 and D2 relate to action commands or other control functions. Neither D1 nor D2 describes gesture input used for "identifying a first portion of the scene in real space" as recited in claim 1. In view of the above, it is submitted that independent claim 1 is patentable over D1-D3, individually or combination. Independent claim 21 recites similar features as claim 1. Accordingly, claim 21 is also allowable over the cited art. It is respectfully submitted that the cited art, alone or in combination, does not disclose or suggest all features of claims 1 and 21 (or the claims depending therefrom). Therefore, it is respectfully requested that the 102 rejections of the claims are withdrawn”; have been considered.
However, the arguments of the newly added and argued claimed limitations of “wherein the data representing the user input includes data representing a physical gesture of the user” pertaining to the obtaining data, are moot in light of the current new ground of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-8, 12, 17, and 21-24 is/are rejected under 35 U.S.C. 103 as obvious over Ambrus et al. (CN 107111979, previously), in view of Barsoum et al. (US 2013/0310652, A1).
Regarding claim 1, Ambrus teaches a computer implemented method for providing feedback to a user input (the system is configured with a moving cursor and projected light source of at least Figs. 3 further comprises said computer implemented method for providing feedback to a user input such as (“an eye tracking sensor and a circuit for continuously determining user gaze vector (in other embodiments, the headset can use different techniques to determine user gaze vector, such as determining in 3D space at the point and user two eyes between the world facing "forward" direction of the current head tracking technology). When the user of staring points to the holographic object (such as object 36A) (such as by, for example, the light projection from the view of the user determined), the object becomes transparent boundary body (frame such as shown in FIG. 3B 42) to highlight. (It should be noted that, in order to simplify the explanation, an object other than the object 36A and the object 36B is not shown in FIG. 3B to 3J.) In addition, cursor (in this example is round) 43 appears in frame 42 of any one of the current is from the view of the user of the light source projection hits the surface. frame 42 and cursor 43 together help the user visualization if and when magnet tool for picking up the object 36A, object 36A is oriented. shown in FIG. 3C, if the user gaze vector changes such that projection light hits the rim 42 of a different surface, the position and appearance of cursor 43 is correspondingly changed to appear on the surface. For example, FIG. 3C shows cursor 43 appearance of the user gaze vector hits the upper surface of the frame 42”),
the method comprising:
obtaining data representing a scene model derived from scene data representing a scene in real space (displayed 3D scene of Figs. 3 comprising said obtaining data representing obviously a scene model derived from scene data representing a scene in real space);
obtaining data representing a user input identifying a first portion of the scene in real space (obtaining further in Figs. 3 and the disclosure obtained user input data in the form of voice commands and gaze input data identifying a first portion of the scene in real space);
estimating, based on the user input data, a first position within the scene model corresponding to the first portion of the scene (determining staring points of further Figs. 3 and the disclosure including said at least first position within the scene model corresponding to the first portion of the scene);
determining control information for controlling a light source to direct light at or near a second position in real space corresponding to the first position within the scene model (control projection of the light of at least Fig. 3B so as “the light is continuously projected in the scene from the cursor to display to a user based on the user's current gaze vector which object is currently being selected as the target object, for example, by high bright light hits any one of object (possibly abides by some selection criterion according to the following discussion). the light of the projected aiming at the actual shape of the target object...” thereby controlling said light source to direct light obviously at the first position and/or at or near a second position in real space corresponding to the first position within the scene model); and outputting the control information to control a light source, thereby to provide for feedback to the user input (executing projected light beam at the highlighted position of the detected position further in Figs. 3 and the disclosure by outputting as implied control information to control a light source, thereby to provide for feedback to the user input).
However, Ambrus is silent regarding the above lined-out items such as citing specifically said obtaining data representing the user input includes data representing specifically a physical gesture of the user.
Barsoum teaches in at least para. 0037-0041 and Fig. 5 a control module 50 configured for receiving user input image data representing a user physical hand gesture input identifying one or more targeting spotlight illumination surgical operating site command further illustrated in at least para. 0038 in real time and in real space of an operating site scene, the inputted physical hand gesture further represents user input for controlling a light source or spotlights to direct and/or reposition light beam at or near a first position and/or a second position in real space corresponding in a case to a first position of or within a surgical operating site scene region. It would have been obvious to one of ordinary skill in the art at the time the invention was made to combine the teachings of Ambrus in view of Barsoum to include wherein said obtaining data representing the user input includes data representing specifically a physical gesture of the user, as discussed above, as Ambrus in view of Barsoum are in the same field of endeavor of determining control information from obtained scene input data for controlling a light source to direct light and/or reposition the light beam at or near a second position in real space corresponding to the first position within the scene model, Barsoum’s light beam control directing and/or repositioning architecture corresponding to received physical user gestures complements the derived scene model system of Ambrus in the sense that when combined with the architecture of Barsoum enables scene and/or terrain mapping by a scene model or the like for properly identifying and recognizing scene data and scene input data to utilize and control directing of spotlights and/or floodlights light beam projection at a specific position of a detected scene according to the user physical input gestures to readily assess in real time a state of a detected scene in real space further providing guidance, and/or accurate depiction of a current scene to the user, said physical gesture input image command as would be appreciated in the art may further obviously realize in a case improved command detection, disambiguation and the like, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 2 (according to claim 1), Ambrus further teaches wherein the user input data comprises data representing a physical gesture of the user (the user of at least Figs. 3A-C further controls the magnet by user gestures comprising said user input data comprising data representing a physical gesture of the user).
Regarding claim 3 (according to claim 2), Ambrus further teaches wherein the user input data comprises a vector or a sequence of vectors derived from a model of the user (the user gestures of further Figs. 3A-C further comprises said user input data including at least user gaze vectors comprises said vector).
Regarding claim 6 (according to claim 1), Ambrus further teaches wherein the user input data comprises data representing an utterance of the user (the system of at least Figs. 3 further configured where “device may be configured to track user gaze direction, voice identifying the gesture and/or identification of the user, and for the tool to any one of these types of operation input” further including said user input data comprises data representing an utterance of the user).
Regarding claim 7 (according to claim 6), Ambrus further teaches wherein the user input data comprises an identifier of an object or region derived from sound data representing the utterance of the user (the system of at least Figs. 3 further configured where “device may be configured to track user gaze direction, voice identifying the gesture and/or identification of the user, and for the tool to any one of these types of operation input” further including said user input data comprising an identifier of an object or region derived from sound data representing the utterance of the user).
Regarding claim 8 (according to claim 7), Ambrus further teaches wherein estimating the first position comprises: mapping the identifier onto an object or region of the scene model; and determining a position of the object or region onto which the identifier is mapped as the first position (a magnet of at least Figs. 3A-3G is activated based on the user inputs data to pick up an identified object based on the user input gestures by obviously mapping the identifier onto an object or region of the scene model and determining a position of the object or region onto which the identifier is mapped as the first position).
Regarding claim 12 (according to claim 1), Ambrus further teaches wherein the control information is for controlling the light source to project a symbol onto a first surface at or near the second position (project the light source of at least Figs. 3B-3G to project at least a projected cursor symbol onto a first surface at or near the second position).
Regarding claim 17 (according to claim 1), Ambrus further teaches wherein the control information is for controlling the light source to direct light to outline a region in real space located at the second position (Fig. 3B-3G).
Regarding claim 21, Ambrus teaches an apparatus for providing feedback to a user input (the system of at least Fig. 3 comprises said apparatus configured with a projected and moving cursor symbol and projected light source of at least Figs. 3 for providing feedback to a user input such as (“an eye tracking sensor and a circuit for continuously determining user gaze vector (in other embodiments, the headset can use different techniques to determine user gaze vector, such as determining in 3D space at the point and user two eyes between the world facing "forward" direction of the current head tracking technology). When the user of staring points to the holographic object (such as object 36A) (such as by, for example, the light projection from the view of the user determined), the object becomes transparent boundary body (frame such as shown in FIG. 3B 42) to highlight. (It should be noted that, in order to simplify the explanation, an object other than the object 36A and the object 36B is not shown in FIG. 3B to 3J.) In addition, cursor (in this example is round) 43 appears in frame 42 of any one of the current is from the view of the user of the light source projection hits the surface. frame 42 and cursor 43 together help the user visualization if and when magnet tool for picking up the object 36A, object 36A is oriented. shown in FIG. 3C, if the user gaze vector changes such that projection light hits the rim 42 of a different surface, the position and appearance of cursor 43 is correspondingly changed to appear on the surface. For example, FIG. 3C shows cursor 43 appearance of the user gaze vector hits the upper surface of the frame 42”),
the apparatus comprising a processing unit (the apparatus of further Figs. 3 further comprising said at least one processing unit 21) configured to:
obtain data representing a scene model derived from scene data representing a scene in real space (displayed 3D scene of Figs. 3 comprising said obtaining data representing obviously a scene model derived from scene data representing a scene in real space);
obtain data representing a user input identifying a first portion of the scene in real space (obtaining further in Figs. 3 and the disclosure obtained user input data in the form of voice commands and gaze input data identifying a first portion of the scene in real space);
estimate, based on the user input data, a first position within the scene model corresponding to the first portion of the scene (determining staring points of further Figs. 3 and the disclosure including said at least first position within the scene model corresponding to the first portion of the scene);
determine control information for controlling a light source to direct light at or near a second position in real space corresponding to the first position within the scene model (control projection of the light of at least Fig. 3B so as “the light is continuously projected in the scene from the cursor to display to a user based on the user's current gaze vector which object is currently being selected as the target object, for example, by high bright light hits any one of object (possibly abides by some selection criterion according to the following discussion). the light of the projected aiming at the actual shape of the target object...” thereby controlling said light source to direct light obviously at the first position and/or at or near a second position in real space corresponding to the first position within the scene model); and output the control information to control a light source, thereby to provide for feedback to the user input (executing projected light beam at the highlighted position of the detected position further in Figs. 3 and the disclosure by outputting as implied control information to control a light source, thereby to provide for feedback to the user input).
However, Ambrus is silent regarding the above lined-out items such as citing specifically said obtain data representing the user input includes data representing specifically a physical gesture of the user.
Barsoum teaches in at least para. 0037-0041 and Fig. 5 a control module 50 configured for receiving user input image data representing a user physical hand gesture input identifying one or more targeting spotlight illumination surgical operating site command further illustrated in at least para. 0038 in real time and in real space of an operating site scene, the inputted physical hand gesture further represents user input for controlling a light source or spotlights to direct and/or reposition light beam at or near a first position and/or a second position in real space corresponding in a case to a first position of or within a surgical operating site scene region. It would have been obvious to one of ordinary skill in the art at the time the invention was made to combine the teachings of Ambrus in view of Barsoum to include wherein said obtaining data representing the user input includes data representing specifically a physical gesture of the user, as discussed above, as Ambrus in view of Barsoum are in the same field of endeavor of determining control information from obtained scene input data for controlling a light source to direct light and/or reposition the light beam at or near a second position in real space corresponding to the first position within the scene model, Barsoum’s light beam control directing and/or repositioning architecture corresponding to received physical user gestures complements the derived scene model system of Ambrus in the sense that when combined with the architecture of Barsoum enables scene and/or terrain mapping by a scene model or the like for properly identifying and recognizing scene data and scene input data to utilize and control directing of spotlights and/or floodlights light beam projection at a specific position of a detected scene according to the user physical input gestures to readily assess in real time a state of a detected scene in real space further providing guidance, and/or accurate depiction of a current scene to the user, said physical gesture input image command as would be appreciated in the art may further obviously realize in a case improved command detection, disambiguation and the like, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 22 (according to claim 21), Ambrus further teaches wherein the apparatus comprises the light source, wherein the light source is configured to receive the output control information and direct light according to the control information (projected light of further Figs. 3B-3G from the light source, wherein the light source is configured to receive the output control information and direct light according to the control information).
Regarding claim 23 (according to claim 21), Ambrus further teaches wherein the apparatus comprises a sensor configured to capture the user input and provide the data representing the user input to the processing unit (said apparatus further adapted wherein user gestures and user gazes of further Figs. 3A-3G further captured by at least said sensor configured to capture the user input and provide the data representing the user input to the processing unit).
Regarding claim 24 (according to claim 21), Ambrus further teaches wherein the apparatus is a robot (the disclosure further cites the embodiments may be realized by a visualization device which may obviously comprise said robot).
Claim(s) 1-4, 6-10, 12-13, and 17-24 is/are further rejected under 35 U.S.C. 103 as obvious over Baladhandapani et al. (US 2021/0255625, previously cited), in view of Barsoum et al.
Regarding claim 1, Baladhandapani teaches a computer implemented method for providing feedback to a user input (the system of at least Figs. 1-2 and para. 0018-0020 is configured with said computer implemented method for providing feedback to a user input such as touch and gesture controls via at least device 24 allowing touch control of a live display scene to select areas of interest and similar area of interest selection by gesture control to generate searchlight control information to control a light source, thereby to provide for feedback to the user input),
the method comprising:
obtaining data representing a scene model derived from scene data representing a scene in real space (said obtaining data of Figs. 1-2 and para. 0018-0020 representing live scene data further representing obviously said scene model derived from scene data representing a scene in real space);
obtaining data representing a user input identifying a first portion of the scene in real space (obtaining tracking data 250 and voice data/gesture data of further Fig. 2 and para. 0018-0020 as said obtained user input data further identifying at least a first gestured portion of the scene in real space);
estimating, based on the user input data, a first position within the scene model corresponding to the first portion of the scene (obtained further in Fig. 2 and para. 0018-0020 data 72…. to be determined…. which represents where a search coordinator is looking in the augmented scene as said estimating, based on the user input data, a first position within the scene model corresponding to the first portion of the scene);
determining control information for controlling a light source to direct light at or near a second position in real space corresponding to the first position within the scene model (based on user inputs of further para. 0018-0020 of voice or gesture command, searchlight controller further provides said control information for controlling a light source to as cited “re-direct the searchlight 12 and includes a set of soft switches to change the orientation of the searchlight 12” such as to direct light at or near a second position in real space corresponding to the first position within the scene model); and
outputting the control information to control a light source, thereby to provide for feedback to the user input (controlling the light source of further Fig.2 and para. 0018-0020 to control a light source, thereby to provide for feedback to the user input).
However, Baladhandapani is silent regarding the above lined-out items such as citing specifically said obtaining data representing the user input includes data representing specifically a physical gesture of the user.
Barsoum teaches in at least para. 0037-0041 and Fig. 5 a control module 50 configured for receiving user input image data representing a user physical hand gesture input identifying one or more targeting spotlight illumination surgical operating site command further illustrated in at least para. 0038 in real time and in real space of an operating site scene, the inputted physical hand gesture further represents user input for controlling a light source or spotlights to direct and/or reposition light beam at or near a first position and/or a second position in real space corresponding in a case to a first position of or within a surgical operating site scene region. It would have been obvious to one of ordinary skill in the art at the time the invention was made to combine the teachings of Baladhandapani in view of Barsoum to include wherein said obtaining data representing the user input includes data representing specifically a physical gesture of the user, as discussed above, as Baladhandapani in view of Barsoum are in the same field of endeavor of determining control information from obtained scene input data for controlling a light source to direct light and/or reposition the light beam at or near a second position in real space corresponding to the first position within the scene model, Barsoum’s light beam control directing and/or repositioning architecture corresponding to received physical user gestures complements the derived scene model system of Baladhandapani in the sense that when combined with the architecture of Barsoum enables scene and/or terrain mapping by a scene model or the like for properly identifying and recognizing scene data and scene input data to utilize and control directing of spotlights and/or floodlights light beam projection at a specific position of a detected scene according to the user physical input gestures to readily assess in real time a state of a detected scene in real space further providing guidance, and/or accurate depiction of a current scene to the user, said physical gesture input image command as would be appreciated in the art may further obviously realize in a case improved command detection, disambiguation and the like, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 2 (according to claim 1), Baladhandapani further teaches wherein the user input data comprises data representing a physical gesture of the user (user inputs of further para. 0018-0020 and 0049 further comprises data representing a physical gesture of the user).
Regarding claim 3 (according to claim 2), Baladhandapani further teaches wherein the user input data comprises a vector or a sequence of vectors derived from a model of the user (user inputs of further para. 0018-0020 and 0026-0027 further comprises gaze or gestured vectors input data further comprises said vector or a sequence of vectors derived from a model of the user).
Regarding claim 4 (according to claim 3), Baladhandapani further teaches wherein the or each vector is a shoulder-to-wrist vector or a head-to-wrist vector derived from the model of the user (The system further cites in at least para. 0044 that “the present disclosure thus provides multimodal control of a searchlight 12, a camera 46 (or at least field of view in headset display 16) ……. through any combination of gesture, voice, gaze, head motion and touchscreen inputs” further implying said user input data may obviously comprises a vector embodying a shoulder-to-wrist vector or a head-to-wrist vector derived from the model of said user).
Regarding claim 6 (according to claim l), Baladhandapani further teaches wherein the user input data comprises data representing an utterance of the user (voice input command of further para. 0018-0020).
Regarding claim 7 (according to claim 6), Baladhandapani further teaches wherein the user input data comprises an identifier of an object or region derived from sound data representing the utterance of the user (para. 0018-0020 further cites “ gaze slaving control for direction of searchlight 12 is combined with voice activated control of other searchlight features like searchlight property commands, filter selection and/or tracking commands. Voice recognition module 30 generates voice control commands 90 for the given speech command and searchlight controller 42 is configured to react to voice control commands 90 to control searchlight 12” which further at least implies the searchlight direction voice command comprises user input data comprises an identifier of a region derived from sound data representing the utterance of the user).
Regarding claim 8 (according to claim 7), Baladhandapani further teaches wherein estimating the first position comprises: mapping the identifier onto an object or region of the scene model; and determining a position of the object or region onto which the identifier is mapped as the first position (identification of the gestured objects in real space of further para. 0018-0021 and the redirecting of the light beam further entails at least understoodly mapping the identifier onto an object or region of the scene model and determining a position of the object or region onto which the identifier is mapped as the first position).
Regarding claim 9 (according to claim 1), Baladhandapani further teaches wherein the control information is determined based on data representing a third position within the scene model corresponding to a position of the user in real space (light beam direction of further para. 0018 and Fig. 1 is further redirected based on implied control information determined based on data representing a third position or the like within the scene model corresponding to a position of a user 60 in real space).
Regarding claim 10 (according to claim 9), Baladhandapani further teaches wherein the control information is for controlling the light source to direct light at a fourth position in real space, at or near the second position, corresponding to a fifth position within the model, at or near the first position, and between which fifth position and the third position there is a clear line-of sight (the device of further para. 0019 further adapted for controlling said light source to direct light obviously at a fourth position or the like in real space, at or near the second position, corresponding to a fifth position within the model, at or near the first position, and between which fifth position and the third position there is a clear line-of sight).
Regarding claim 12 (according to claim 1), Baladhandapani further teaches wherein the control information is for controlling the light source to project a symbol onto a first surface at or near the second position (para. 0030 further notes display 18 configured to display a live scene where at least eye/head movements are used for controlling the light source to project a light beam or symbol onto a first surface at or near a redirected second position).
Regarding claim 13 (according to claim 12), Baladhandapani further teaches wherein the symbol is an animated symbol (displayed moving pointer of at least para. 0051).
Regarding claim 17 (according to claim 1), Baladhandapani further teaches wherein the control information is for controlling the light source to direct light to outline a region in real space located at the second position (projecting a light beam of further para. 0018-0030 to a region of a terrain by controlling the light source to direct light to outline a region in real space located at the second position).
Regarding claim 18 (according to claim 1), Baladhandapani further teaches wherein the control information comprises information for controlling actuation of one or more actuatable mirrors of a light source to direct a light beam at or near the second position (redirecting the light beam of further para. 0018-0030 further entails obviously controlling actuation of one or more actuatable mirrors of a light source to direct a light beam at or near the second position).
Regarding claim 19 (according to claim 1), Baladhandapani further teaches wherein the method further comprises: obtaining data representing further user input relating to an adjustment of the identified first portion of the scene (receiving further in at least para. 0018-0030 light source redirection and adjustment commands further representing user input relating to an adjustment of the identified first portion of the scene);
estimating, based on the further user input and based on the first position, an adjusted first position within the model corresponding to the adjusted first portion of the scene (upon receiving said light source redirection and adjustment commands of further para. 0018-0030, the system as understood in the art obviously estimating, or detect based on said user input and based on the first position, an adjusted first position within the model corresponding to the adjusted first portion of the scene);
determining adjusted control information for controlling a light source to direct light at or near an adjusted second position in real space corresponding to the adjusted first position (redirecting the searchlight and/or light source of further para. 0018-0030, further entails determining said adjusted control information for controlling a light source to direct light at or near an adjusted second position in real space corresponding to the adjusted first position);
and outputting the adjusted control information (redirecting the searchlight and/or light source of further para. 0018-0030, further entails said outputting the adjusted control information).
Regarding claim 20 (according to claim 19), Baladhandapani further teaches wherein the further user input identifies a direction relative to the first portion in real space (user inputs of further para. 0018-0030 identifies a direction and position to direct the searchlight); and wherein estimating the adjusted first position comprises: determining a direction within the model corresponding to the direction in real space (the device of further para. 0018-0030 adjusts and redirect the light beam in real space based on at determining direction within the model corresponding to the direction in real space); and estimating the adjusted first position based on the determined direction within the model (the adjustment and redirecting of the light beam of further para. 0018-0030 further understoodly realized by obviously estimating the adjusted first position based on the determined direction within the model).
Regarding claim 21, Baladhandapani teaches an apparatus for providing feedback to a user input (the system of at least Figs. 1-2 and para. 0018-0020 is configured with at least one apparatus for providing feedback to a user input such as touch and gesture controls via at least device 24 allowing touch control of a live display scene to select areas of interest and similar area of interest selection by gesture control to generate searchlight control information to control a light source, thereby to provide for feedback to the user input),
the apparatus comprising a processing unit (the search light control system of at least Fig. 1 further comprises at least one processing means) configured to:
obtain data representing a scene model derived from scene data representing a scene in real space (said obtaining data of Figs. 1-2 and para. 0018-0020 representing live scene data further representing obviously said scene model derived from scene data representing a scene in real space);
obtain data representing a user input identifying a first portion of the scene in real space (obtaining tracking data 250 and voice data/gesture data of further Fig. 2 and para. 0018-0020 as said obtained user input data further identifying at least a first gestured portion of the scene in real space);
estimate, based on the user input data, a first position within the scene model corresponding to the first portion of the scene (obtained further in Fig. 2 and para. 0018-0020 data 72…. to be determined…. which represents where a search coordinator is looking in the augmented scene as said estimating, based on the user input data, a first position within the scene model corresponding to the first portion of the scene);
determine control information for controlling a light source to direct light at or near a second position in real space corresponding to the first position within the scene model (based on user inputs of further para. 0018-0020 of voice or gesture command, searchlight controller further provides said control information for controlling a light source to as cited “re-direct the searchlight 12 and includes a set of soft switches to change the orientation of the searchlight 12” such as to direct light at or near a second position in real space corresponding to the first position within the scene model); and
output the control information to control a light source, thereby to provide for feedback to the user input (controlling the light source of further Fig.2 and para. 0018-0020 to control a light source, thereby to provide for feedback to the user input).
However, Baladhandapani is silent regarding the above lined-out items such as citing specifically said obtain data representing the user input includes data representing specifically a physical gesture of the user.
Barsoum teaches in at least para. 0037-0041 and Fig. 5 a control module 50 configured for receiving user input image data representing a user physical hand gesture input identifying one or more targeting spotlight illumination surgical operating site command further illustrated in at least para. 0038 in real time and in real space of an operating site scene, the inputted physical hand gesture further represents user input for controlling a light source or spotlights to direct and/or reposition light beam at or near a first position and/or a second position in real space corresponding in a case to a first position of or within a surgical operating site scene region. It would have been obvious to one of ordinary skill in the art at the time the invention was made to combine the teachings of Baladhandapani in view of Barsoum to include wherein said obtaining data representing the user input includes data representing specifically a physical gesture of the user, as discussed above, as Baladhandapani in view of Barsoum are in the same field of endeavor of determining control information from obtained scene input data for controlling a light source to direct light and/or reposition the light beam at or near a second position in real space corresponding to the first position within the scene model, Barsoum’s light beam control directing and/or repositioning architecture corresponding to received physical user gestures complements the derived scene model system of Baladhandapani in the sense that when combined with the architecture of Barsoum enables scene and/or terrain mapping by a scene model or the like for properly identifying and recognizing scene data and scene input data to utilize and control directing of spotlights and/or floodlights light beam projection at a specific position of a detected scene according to the user physical input gestures to readily assess in real time a state of a detected scene in real space further providing guidance, and/or accurate depiction of a current scene to the user, said physical gesture input image command as would be appreciated in the art may further obviously realize in a case improved command detection, disambiguation and the like, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 22 (according to claim 21), Baladhandapani further teaches wherein the apparatus comprises the light source, wherein the light source is configured to receive the output control information and direct light according to the control information (aerial vehicle 40 of Figs. 1-2 further comprises said apparatus further comprises said light source, wherein the light source is configured to receive the output control information and direct light according to the control information).
Regarding claim 23 (according to claim 21), Baladhandapani further teaches wherein the apparatus comprises a sensor configured to capture the user input and provide the data representing the user input to the processing unit (the aerial vehicle 40 of Figs. 1-2 comprising at least a camera further comprising said sensor).
Regarding claim 24 (according to claim 21), Baladhandapani further teaches wherein the apparatus is a robot (the aerial vehicle 40 of Figs. 1-2 further indicative of said robot).
Claim(s) 1 and 21 is/are further rejected under 35 U.S.C. 103 as obvious over Summa et al. (US 2020/0227034, previously cited), and in view of Barsoum et al.
Regarding claim 1, Summa teaches a computer implemented method for providing feedback to a user input (Fig. 1 teaches a computer implemented method configured with gestured user inputs associated with projected light source of at least in Figs. 3-5 for providing further feedback to a user input),
the method comprising:
obtaining data representing a scene model derived from scene data representing a scene in real space (obtaining further in Figs. 3-5 data representing a scene model derived from scene data representing a scene in real space);
obtaining data representing a user input identifying a first portion of the scene in real space (obtaining gazed landmark data 375 of further Figs. 3-5 further representing a user input identifying a first portion of the scene in real space);
estimating, based on the user input data, a first position within the scene model corresponding to the first portion of the scene (estimating further in at least Figs. 3-5 a position of the gazed landmarks data depicted on the scene based obviously further on said user input data, said first position within the scene model corresponding to the first portion of the scene);
determining control information for controlling a light source to direct light at or near a second position in real space corresponding to the first position within the scene model (determinized further in at least Figs. 3-5 and para. 0028, and 0031-0032 said control information for controlling a light source to direct light at or near a second position 115 in real space corresponding to the first position within the scene model); and
outputting the control information to control a light source, thereby to provide for feedback to the user input (executing projected light beam at the highlighted position of the detected position further in Figs. 3-5 and para. 0028, and 0031-0032 by outputting as implied control information to control a light source to opaque region 115 thereby to provide for feedback to the user input).
However, Summa is silent regarding the above lined-out items such as citing specifically said obtaining data representing the user input includes data representing specifically a physical gesture of the user.
Barsoum teaches in at least para. 0037-0041 and Fig. 5 a control module 50 configured for receiving user input image data representing a user physical hand gesture input identifying one or more targeting spotlight illumination surgical operating site command further illustrated in at least para. 0038 in real time and in real space of an operating site scene, the inputted physical hand gesture further represents user input for controlling a light source or spotlights to direct and/or reposition light beam at or near a first position and/or a second position in real space corresponding in a case to a first position of or within a surgical operating site scene region. It would have been obvious to one of ordinary skill in the art at the time the invention was made to combine the teachings of Summa in view of Barsoum to include wherein said obtaining data representing the user input includes data representing specifically a physical gesture of the user, as discussed above, as Summa in view of Barsoum are in the same field of endeavor of determining control information from obtained scene input data for controlling a light source to direct light and/or reposition the light beam at or near a second position in real space corresponding to the first position within the scene model, Barsoum’s light beam control directing and/or repositioning architecture corresponding to received physical user gestures complements the derived scene model system of Summa in the sense that when combined with the architecture of Barsoum enables scene and/or terrain mapping by a scene model or the like for properly identifying and recognizing scene data and scene input data to utilize and control directing of spotlights and/or floodlights light beam projection at a specific position of a detected scene according to the user physical input gestures to readily assess in real time a state of a detected scene in real space further providing guidance, and/or accurate depiction of a current scene to the user, said physical gesture input image command as would be appreciated in the art may further obviously realize in a case improved command detection, disambiguation and the like, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Regarding claim 21, Summa teaches an apparatus for providing feedback to a user input (Fig. 1 teaches said apparatus configured with gestured user inputs associated with projected light source of at least in Figs. 3-5 for providing further feedback to a user input),
the apparatus comprising a processing unit (apparatus of at least Fig. 1 further comprises at least one processing means) configured to:
obtain data representing a scene model derived from scene data representing a scene in real space (obtaining further in Figs. 3-5 data representing a scene model derived from scene data representing a scene in real space);
obtain data representing a user input identifying a first portion of the scene in real space (obtaining gazed landmark data 375 of further Figs. 3-5 further representing a user input identifying a first portion of the scene in real space);
estimate, based on the user input data, a first position within the scene model corresponding to the first portion of the scene (estimating further in at least Figs. 3-5 a position of the gazed landmarks data depicted on the scene based obviously further on said user input data, said first position within the scene model corresponding to the first portion of the scene);
determine control information for controlling a light source to direct light at or near a second position in real space corresponding to the first position within the scene model (determinized further in at least Figs. 3-5 and para. 0028, and 0031-0032 said control information for controlling a light source to direct light at or near a second position 115 in real space corresponding to the first position within the scene model); and
output the control information to control a light source, thereby to provide for feedback to the user input (executing projected light beam at the highlighted position of the detected position further in Figs. 3-5 and para. 0028, and 0031-0032 by outputting as implied control information to control a light source to opaque region 115 thereby to provide for feedback to the user input).
However, Summa is silent regarding the above lined-out items such as citing specifically said obtain data representing the user input includes data representing specifically a physical gesture of the user.
Barsoum teaches in at least para. 0037-0041 and Fig. 5 a control module 50 configured for receiving user input image data representing a user physical hand gesture input identifying one or more targeting spotlight illumination surgical operating site command further illustrated in at least para. 0038 in real time and in real space of an operating site scene, the inputted physical hand gesture further represents user input for controlling a light source or spotlights to direct and/or reposition light beam at or near a first position and/or a second position in real space corresponding in a case to a first position of or within a surgical operating site scene region. It would have been obvious to one of ordinary skill in the art at the time the invention was made to combine the teachings of Summa in view of Barsoum to include wherein said obtain data representing the user input includes data representing specifically a physical gesture of the user, as discussed above, as Summa in view of Barsoum are in the same field of endeavor of determining control information from obtained scene input data for controlling a light source to direct light and/or reposition the light beam at or near a second position in real space corresponding to the first position within the scene model, Barsoum’s light beam control directing and/or repositioning architecture corresponding to received physical user gestures complements the derived scene model system of Summa in the sense that when combined with the architecture of Barsoum enables scene and/or terrain mapping by a scene model or the like for properly identifying and recognizing scene data and scene input data to utilize and control directing of spotlights and/or floodlights light beam projection at a specific position of a detected scene according to the user physical input gestures to readily assess in real time a state of a detected scene in real space further providing guidance, and/or accurate depiction of a current scene to the user, said physical gesture input image command as would be appreciated in the art may further obviously realize in a case improved command detection, disambiguation and the like, according to further known methods to yield predictable results since known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art as said combination is thus the adaptation of an old idea or invention using newer technology that is either commonly available and understood in the art thereby a variation on already known art (See MPEP 2143, KSR Exemplary Rationale F).
Claims Objections
Claims 5, 11, and 14-16 remained objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARCELLUS AUGUSTIN whose telephone number is (571)270-3384. The examiner can normally be reached 9 AM- 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BENNY TIEU can be reached on 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARCELLUS J AUGUSTIN/Primary Examiner, Art Unit 2682 002/03/2026