DETAILED ACTION
Claims 1-32 filed May 20th 2024 are pending in the current action.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 2, 15, 16, 27, 30 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1, 12, 15 of U.S. Patent No. 11, 989,351. Although the claims at issue are not identical, they are not patentably distinct from each other because the current claims are broader than the claims in the ‘351 patent.
Current claims
US 11, 989,351
1. A system comprising: a sensor array comprising a plurality of sensors each arranged to monitor a current position of one or more parts of the user's body; a plurality of aerial vehicles, each one of the plurality of aerial vehicles having disposed thereon a respective one of the plurality of sensors; a sensor array controller configured to reposition the plurality of aerial vehicles so as to provide a line of sight between each of the plurality of sensors disposed on one of the plurality of aerial vehicles and a respective part of the user's body monitored by said one of the plurality of sensors; and a controller configured to determine a current position of the user's body based on information obtained using the plurality of sensors.
1. A system comprising: a sensor array comprising a plurality of sensors each arranged to monitor a current position of one or more parts of the user's body; a plurality of aerial vehicles, at least some of the plurality of sensors being disposed on respective ones of the plurality of aerial vehicles; a sensor array controller configured to reposition the plurality of aerial vehicles so as to provide a line of sight between each of the plurality of sensors disposed on one of the plurality of aerial vehicles and a respective part of the user's body monitored by said one of the plurality of sensors; a physical feedback mechanism for providing physical feedback to the user, the physical feedback mechanism being configured to be worn on the user's body; a plurality of location sensors distributed over at least part of the user's body, each of the location sensors being configured to provide information on a current location of a respective surface of the user's body; and a controller configured to determine a current position of the user's body based on information obtained using the plurality of sensors and the information provided by the plurality of location sensors, and to control the physical feedback mechanism to provide physical feedback to the user in dependence on the determined position of the user's body, wherein the user's body may comprise visible surfaces and occluded surfaces depending on the current position of the user's body and a current spatial arrangement of the plurality of sensors disposed on the plurality of aerial vehicles, the visible surfaces comprising parts of the user's body visible to one or more of said plurality of sensors and the occluded surfaces comprising parts of the user's body hidden from view of said plurality of sensors, wherein the plurality of location sensors are disposed on at least two opposing surfaces of the one or more parts of the user's body, such that when one or more of the opposing surfaces is an occluded surface the controller is configured to determine the current position of said part of the user's body based on the information provided by said location sensors, and to determine the current position of any visible surfaces based on the information obtained using the plurality of sensors disposed on the plurality of aerial vehicles.
2. The system of claim 1, comprising: a physical feedback mechanism for providing physical feedback to the user, the physical feedback mechanism being configured to be worn on the user's body, wherein the controller is configured to control the physical feedback mechanism to provide physical feedback to the user in dependence on the determined position of the user's body.
See claim 1
15. The system of claim 14, wherein the user's body may comprise visible surfaces and occluded surfaces depending on the current position of the user's body and a current spatial arrangement of the plurality of sensors, the visible surfaces comprising parts of the user's body visible to one or more of the plurality of sensors and the occluded surfaces comprising parts of the user's body hidden from view of the plurality of sensors, and wherein when at least one of the location sensors is disposed on an occluded surface of the user's body, the controller is configured to determine the current position of the user's body by determining the current position of said occluded surface based on information obtained from said at least one of the location sensors, and by determining the current position of any visible surfaces based on the information obtained using the plurality of sensors.
See Claim 1
16. The system of claim 15, wherein the plurality of location sensors include location sensors disposed on at least two opposing surfaces of the one or more parts of the user's body, such that when one or more of the opposing surfaces is an occluded surface the controller may determine the current position of said part of the user's body based on the information provided by said location sensors, and to determine the current position of any visible surfaces based on the information obtained using the plurality of sensors disposed on the plurality of aerial vehicles and/or based on information obtained from one or more head-mounted sensors configured to be worn on the user's head.
See claim 1
27. The system of claim 1, wherein the plurality of sensors comprises one or more head-mounted sensors configured to be worn on the user's head, wherein the controller is configured to reconfigure the plurality of sensors to enable a current position of one or more parts of the user's body and/or one or more objects to be determined with a desired degree of accuracy, and wherein the controller is configured to reconfigure the plurality of sensors by reorienting at least one of said one or more head-mounted sensors.
12. The system of claim 1, wherein the plurality of sensors are moveably mounted such that each sensor can be independently reoriented, wherein the sensor array controller is configured to reorient one or more of the plurality of sensors in addition to repositioning the plurality of aerial vehicles, so as to provide a line of sight between each one of the plurality of sensors and a respective part of the user's body monitored by said one of the plurality of sensors.
30. The system of claim 1, wherein the sensor array controller is configured to determine a number of the aerial vehicles required to determine the current position of the user's body and/or of one or more objects with a desired degree of accuracy, wherein in response to the determined number of aerial vehicles being less than a number of aerial vehicles currently comprised in the sensor array, the sensor array controller is configured to switch one or more redundant aerial vehicles into a standby mode in which said one or more redundant aerial vehicles do not form part of the sensor array, and wherein in response to the determined number of aerial vehicles being greater than the number of aerial vehicles currently comprised in the sensor array, the sensor array controller is configured to switch one or more aerial vehicles from the standby mode into an active mode in which said one or more aerial vehicles form part of the sensor array.
15. The system of claim 1, wherein the sensor array controller is configured to determine a number of the aerial vehicles required to determine the current position of the user's body with a desired degree of accuracy, wherein in response to the determined number of aerial vehicles being less than a number of aerial vehicles currently comprised in the sensor array, the sensor array controller is configured to switch one or more redundant aerial vehicles into a standby mode in which said one or more redundant aerial vehicles do not form part of the sensor array, and wherein in response to the determined number of aerial vehicles being greater than the number of aerial vehicles currently comprised in the sensor array, the sensor array controller is configured to switch one or more aerial vehicles from the standby mode into an active mode in which said one or more aerial vehicles form part of the sensor array, optionally wherein the sensor array controller is configured to take into account a current body position of the user when determining the number of aerial vehicles required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7, 8, 10-12, 14, 17, 25, 32 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shapira (US2016/00349835) in view of McDermott et al. (US9,785,147)
Consider claim 1, where Shapira teaches a system comprising: a sensor array comprising a plurality of sensors each arranged to monitor a current position of one or more parts of the user's body; (See Shapira Fig. 1 and ¶29-31 where a tactile autonomous drones (TAD) 100, 120, 140 are capable of tracking moving real world objects in order to position the TAD to provide tactile feedback to the user, thus tracking the position of the user’s hand to position the objects 110 for tactile feedback) a plurality of aerial vehicles, each one of the plurality of aerial vehicles having disposed thereon a respective one of the plurality of sensors; (See Shapira ¶26-27, 105-108 where the TADs are enabled by a variety of Natural User Interfaces (NUI) which comprise information may be captured using various types of 2D or depth imaging devices such as stereoscopic or time-of-flight camera systems, infrared camera systems, RGB (red, green and blue) camera systems, and the like, or any combination of such devices.) a sensor array controller configured to reposition the plurality of aerial vehicles so as to provide a line of sight between each of the plurality of sensors disposed on one of the plurality of aerial vehicles and a respective part of the user's body monitored by said one of the plurality of sensors; (See Shapira ¶26-29, 105-108 the sensor suite 200 enables the TAD 200 to track its own position and the position and motions of other TADs, track one or more users, track moving real objects, people, etc. in the real world environment around the user, navigate and position itself (and objects or surfaces carried by the TAD) relative to the user, etc..) and a controller configured to determine a current position of the user's body based on information obtained using the plurality of sensors. (See Shapira ¶26-27 where one or more of the TADs (100, 120, 140) automatically position themselves in the real-world environment in a way that maps to a corresponding position in the immersive virtual environment in order to present real surfaces and objects to the user 160)
Shapira provides suggestion for providing a line of sight to a respective part of the user’s body. (See Shapira ¶27 where TAD 120 can position itself, and the real (or toy) object in such a way that it appears to the user that a virtual character in the immersive virtual environment is offering the object to the user 160. The user 160 can then reach out and physically grasp that object as if the virtual character had actually handed it to the user. Thus, providing suggestion that the user’s hand position is tracked) However, this is not explicitly stated. In an analogous field of endeavor McDermott teaches providing a line of sight to a respective part of the user’s body. (See McDermott Figs 4A, 4B, 5A, 5B, 7 and col 5 line 58- col 6 line 32 where the reference component pixel sets based on reference images of the clothing of the subject 102. In the next frame 134b, the pixels at the left side of the grid 140 are consistent with the background of the frame 134b, while groups of pixels near the upper right of the grid 140 (and to the right of the central axis 142 of the frame 134b) may now be consistent with the subject pixel set 126 (and therefore with the subject 102). The visual recognition system 114 may conclude that the subject 102 is moving to the right with respect to the frame 134b, and that it may be desirable for the UAV 100 to rotate to its right (with respect to the yaw axis of the UAV 100) to keep the subject 102 centrally framed. Thereby maintaining a line of sight with respect to the torso of the user) Therefore, it would have been obvious for one of ordinary skill in the art that the drones of Shapira tracks the user’s hand by moving the drone such that the hand is maintained within the frame of the camera. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of maintaining a consistent orientation between the drone and the subject to maintain perspective. (See McDermott col 3 line 9-23)
Consider claim 2, where Shapira in view of McDermott teaches the system of claim 1, comprising: a physical feedback mechanism for providing physical feedback to the user, the physical feedback mechanism being configured to be worn on the user's body, wherein the controller is configured to control the physical feedback mechanism to provide physical feedback to the user in dependence on the determined position of the user's body. (See Shapira ¶2 where wearable gloves that provide sensation or stimulation to the user's hand, movable arms that contact the user's finger or hand, directed air blasts, directed sound fields, etc. are known in the prior art)
Consider claim 3, where Shapira in view of McDermott teaches the system of claim 1, comprising: a rendering unit configured to render an image from a viewpoint of one of the plurality of sensors. (See McDermott col 3 line 9-59 where the camera 104 may be coupled to an image processing system 108. The image processing system 108 may compress the incoming stream of images for broadcast or retransmission. The image processing system 106 may store a processed image stream based on captured images in onboard memory 110, or transmit the image stream to a wireless device or third-party viewer via a wireless transceiver 112.) It would be obvious to one of ordinary skill in the art that the cameras on the TADs in Shapira are also capable of capturing an image stream as taught by McDermott.
Consider claim 7, where Shapira in view of McDermott teaches the system of claim 1, wherein the sensor array controller is configured to predict that contact is expected to occur between the user and an object, in dependence on a determination that current trajectories of the user and/or said object are due to intersect at a future point in time. (See Shapira ¶27 where TAD 120 can position itself, and the real (or toy) object in such a way that it appears to the user that a virtual character in the immersive virtual environment is offering the object to the user 160. The user 160 can then reach out and physically grasp that object as if the virtual character had actually handed it to the user. Thus, providing suggestion that the user’s hand position is tracked)
Consider claim 8, where Shapira in view of McDermott teaches the system of claim 7, wherein the sensor array controller is configured to predict a location and time at which said contact is expected to occur, and is configured to control the plurality of aerial vehicles to adopt suitable positions relative to the predicted location of contact with the object before the predicted time. (See Shapira ¶27 where TAD 120 can position itself, and the real (or toy) object in such a way that it appears to the user that a virtual character in the immersive virtual environment is offering the object to the user 160. The user 160 can then reach out and physically grasp that object as if the virtual character had actually handed it to the user. Thus, providing suggestion that the user’s hand position is tracked)
Consider claim 10, where Shapira in view of McDermott teaches the system of claim 1, comprising: a first group of aerial vehicles arranged to monitor the current position of said one or more parts of the user's body, the first group of aerial vehicles comprising said plurality of aerial vehicles; (See Shapira ¶21 where one or more TADs, or sensors accessible to one or more TADs, track TAD positions relative to the positions and motions of users as they move through a real-world space while immersed in the VR environment) and a second group of aerial vehicles each having disposed thereon a respective one of a second plurality of sensors, wherein the second group of aerial vehicles is arranged to monitor a physical environment surrounding the user, and/or to monitor one or more objects within said physical environment. (See Shapira ¶21 one or more TADs apply the resulting tracking information to automatically position themselves, or one or more physical surfaces or objects carried by the TADs, in a way that enables physical contact between those surfaces or objects and one or more portions of the user's body. In further implementations, a computer-based or cloud-based control system directs and positions one or more TADs based on user and TAD position and tracking information)
Consider claim 11, where Shapira in view of McDermott teaches the system of claim 10, wherein the sensor array controller is configured to subsequently control one or more of the second group of aerial vehicles to monitor the current position of said one or more parts of the user's body, instead of or in addition to monitoring one or more objects. (See Shapira ¶21 one or more TADs apply the resulting tracking information to automatically position themselves, or one or more physical surfaces or objects carried by the TADs, in a way that enables physical contact between those surfaces or objects and one or more portions of the user's body. In further implementations, a computer-based or cloud-based control system directs and positions one or more TADs based on user and TAD position and tracking information)
Consider claim 12, where Shapira in view of McDermott teaches the system of claim 10, wherein the sensor array controller is configured to predict that contact is expected to occur between the user and an object, in dependence on a determination that current trajectories of the user and/or said object are due to intersect at a future point in time, and wherein the sensor array controller is configured to predict a location and time at which said contact is expected to occur, and is configured to control the second group of aerial vehicles to adopt suitable positions relative to the predicted location of contact with the object before the predicted time. (See Shapira ¶21, 62 one or more TADs apply the resulting tracking information to automatically position themselves, or one or more physical surfaces or objects carried by the TADs, in a way that enables physical contact between those surfaces or objects and one or more portions of the user's body. In further implementations, a computer-based or cloud-based control system directs and positions one or more TADs based on user and TAD position and tracking information. When the distance between the user's hands (or other body part) and a tactile virtual object (or surface) decreases, a corresponding TAD and its corresponding tactile virtual object is maneuvered into position based on a prediction of the physical point, p, in the real-world environment where the user will contact the corresponding virtual object, and the time t.sub.p before that contact is predicted to occur.)
Consider claim 14, where Shapira in view of McDermott teaches the system of claim 1, comprising a plurality of location sensors distributed over at least part of the user's body, each of the location sensors being configured to provide information on a current location of a respective surface of the user's body, wherein the controller is configured to determine the current position of the user's body based on the information obtained using the plurality of sensors and based on the information provided by the plurality of location sensors. (See Shapira ¶47-49 where the sensors used for dynamically tracking the user include any combination of head mounted sensors (e.g., coupled to the VR display device), body worn sensors, sensors embedded in or coupled to each TAD, sensors dispersed throughout the real-world space, etc.)
Consider claim 17, where Shapira in view of McDermott teaches the system of claim 1, wherein the plurality of sensors comprises one or more head-mounted sensors configured to be worn on the user's head, and the sensor array controller is configured to set a respective position for one or more of the aerial vehicles relative to a position and/or orientation of the head-mounted sensor. (See Shapira ¶47-49 where the sensors used for dynamically tracking the user include any combination of head mounted sensors (e.g., coupled to the VR display device), body worn sensors, sensors embedded in or coupled to each TAD, sensors dispersed throughout the real-world space, etc.)
Consider claim 25, where Shapira in view of McDermott teaches the system of claim 1, wherein the plurality of sensors are moveably mounted such that each sensor can be independently reoriented, and wherein the sensor array controller is configured to reorient one or more of the plurality of sensors and/or reposition the plurality of aerial vehicles, so as to provide a line of sight between each one of the plurality of sensors and a respective part of the user's body monitored by said one of the plurality of sensors. (See McDermott Figs 4A, 4B, 5A, 5B, 7 and col 5 line 58- col 6 line 32 where the reference component pixel sets based on reference images of the clothing of the subject 102. In the next frame 134b, the pixels at the left side of the grid 140 are consistent with the background of the frame 134b, while groups of pixels near the upper right of the grid 140 (and to the right of the central axis 142 of the frame 134b) may now be consistent with the subject pixel set 126 (and therefore with the subject 102). The visual recognition system 114 may conclude that the subject 102 is moving to the right with respect to the frame 134b, and that it may be desirable for the UAV 100 to rotate to its right (with respect to the yaw axis of the UAV 100) to keep the subject 102 centrally framed. Thereby maintaining a line of sight with respect to the torso of the user) Therefore, it would have been obvious for one of ordinary skill in the art that the drones of Shapira tracks the user’s hand by moving the drone such that the hand is maintained within the frame of the camera. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of maintaining a consistent orientation between the drone and the subject to maintain perspective. (See McDermott col 3 line 9-23)
Consider claim 32, where Shapira in view of McDermott teaches the system of claim 1, comprising: a communication system configured to remotely control a robotic device in dependence on the user's body movements. (See Shapira Figs, 3, 4 and ¶105 where mobile devices are employed to provide tactile feedback in accordance with user movement in the virtual reality environment)
Claim(s) 4-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shapira in view of McDermott as applied to claim 1 above, in further view of Sheftel et al. (US2019/0104250)
Consider claim 4, where Shapira in view of McDermott teaches the system of claim 3, however, they do not explicitly teach comprising: a user interface configured to receive user input indicative of a desired viewpoint of said image that is rendered by the rendering unit, wherein the sensor array controller is configured to control one of the plurality of aerial vehicles to adopt a position corresponding to the desired viewpoint indicated by the user input. However, in an analogous field of endeavor Sheftel teaches the limitation. (See Sheftel Fig. 2 and ¶54-55, 87 where users of the system may be provided a programmable software interface for the drone's trajectory, allowing the users to create even more complex trajectories than those provided by the system by default. A cameraman may control the flight of the drones) Therefore, it would have been obvious for one of ordinary skill in the art to modify the TADs of Shapira to have additional interfaces to control the drone flights as taught by Sheftel. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of providing the desired viewpoints in a VR scene. (See Sheftel ¶13)
Consider claim 5, where Shapira in view of McDermott teaches the system of claim 1, wherein the sensor array controller is configured to reconfigure the sensor array by repositioning and/or reorienting one or more of the plurality of aerial vehicles according to a pre-programmed routine, in dependence on one or more predefined cues or events being detected. However, in an analogous field of endeavor Sheftel teaches the limitation. (See Sheftel Fig. 2 and ¶49-55, 87 where users of the system may be provided a programmable software interface for the drone's trajectory, allowing the users to create even more complex trajectories than those provided by the system by default. A cameraman may control the flight of the drones or the drones may react to a gesture performed by the actress) Therefore, it would have been obvious for one of ordinary skill in the art to modify the TADs of Shapira to have additional interfaces to control the drone flights as taught by Sheftel. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of providing the desired viewpoints in a VR scene. (See Sheftel ¶13)
Consider claim 6, where Shapira in view of McDermott teaches the system of claim 1, however they do not explicitly teach wherein the sensor array controller is configured to reorient one or more of the plurality of sensors and/or reposition the plurality of aerial vehicles, so that one or more parts of the user's body and/or one or more objects currently being monitored by said one or more sensors has at least a minimum angular size within a field of view of said one or more sensors. However, in an analogous field of endeavor Sheftel teaches the limitation. (See Sheftel Fig. 2 and ¶49-55, 87 and claim 13 where users of the system may be provided a programmable software interface for the drone's trajectory, allowing the users to create even more complex trajectories than those provided by the system by default. The camera may be maintained at a fixed angle from the subject) Therefore, it would have been obvious for one of ordinary skill in the art to modify the TADs of Shapira to have additional interfaces to control the drone flights as taught by Sheftel. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of providing the desired viewpoints in a VR scene. (See Sheftel ¶13)
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shapira in view of McDermott as applied to claim 7 above, in further view of Bond et al. (US10,535,199)
Consider claim 9, where Shapira in view of McDermott teaches the system of claim 7, comprising: a rendering unit configured to render a first image from a first viewpoint to be displayed to the user, (See Shapira ¶83 where a user is immersed in a real-time rendering of a virtual environment) however, Shapira does not explicitly teach wherein in dependence on a prediction that said contact is expected to occur between the user and the object, the rendering unit is configured to render a second image from a second viewpoint, the second viewpoint being a viewpoint that affords a clearer view of said object compared to the first viewpoint. However, in an analogous field of endeavor Bond teaches wherein in dependence on a prediction that said contact is expected to occur between the user and the object, the rendering unit is configured to render a second image from a second viewpoint, the second viewpoint being a viewpoint that affords a clearer view of said object compared to the first viewpoint. (See Bond Figs. 7A, 7B and col 15 line 64- col 16 line 63 where when the user is immersed in a virtual environment and about to make contact with real-world objects, such as shelf 406D, a window 704 opens in the virtual environment to allow the user to clearly see the object that the user is about to bump into.) Therefore, it would have been obvious for one of ordinary skill in the art to modify the virtual environment of Shapira with the safety boundary as taught by Bond. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of providing the user with a measure of safety regarding real-world environment without interrupting or otherwise adversely affecting the presentation of virtual environment.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shapira in view of McDermott as applied to claim 1 above, in further view of Raffa et al. (US2016/0378109)
Consider claim 13, where Shapira in view of McDermott teaches the system of claim 12, however they do not explicitly teach wherein one or more aerial vehicles of the second group of aerial vehicles are in a standby mode prior to said contact being predicted, and the sensor array controller is configured to cause said one or more aerial vehicles to switch from the standby mode into an active mode in dependence on said contact being predicted. However, in an analogous field of endeavor Raffa teaches the limitation. (See Raffa paragraph 33-38 where when the parameters of a task are rechecked, they can involve the use of more or fewer drones and in the case of more drones’ additional drones may be deployed.) Therefore, it would have been obvious for one of ordinary skill in the to modify the drone swarm of Shapira with the operational characteristics of Raffa. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of scaling the resources as needed.
Claim(s) 18 and 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shapira in view of McDermott as applied to claim 1 above, in further view of Lee et al. (US2018/0181119)
Consider claim 18, where Shapira in view of McDermott teaches the system of claim 1, however they do not explicitly teach wherein the controller is configured to determine a current gaze point of the user, and the sensor array controller is configured to position one or more of the aerial vehicles such that one or more sensors disposed on said one or more of the aerial vehicles is directed towards the determined gaze point. However, in an analogous field of endeavor Lee teaches wherein the controller is configured to determine a current gaze point of the user, and the sensor array controller is configured to position one or more of the aerial vehicles such that one or more sensors disposed on said one or more of the aerial vehicles is directed towards the determined gaze point. (See Lee Figs. 9, 11A-C and ¶127-138, ¶170-176 where the UAV will move as the user moves their gaze point) Therefore, it would have been obvious for one of ordinary skill in the art that the drones in Shapira operating in a live video feed mode (See Shapira ¶73) could be controlled via the control method for controlling a live video feed as taught by Lee. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of using known methods of control to yield the desired result.
Consider claim 26, where Shapira in view of McDermott teaches the system of claim 1, wherein the plurality of sensors comprises one or more head-mounted sensors configured to be worn on the user's head, (See Shapira ¶47-49 where the sensors used for dynamically tracking the user include any combination of head mounted sensors (e.g., coupled to the VR display device), body worn sensors, sensors embedded in or coupled to each TAD, sensors dispersed throughout the real-world space, etc.) however Shapira does not explicitly teach wherein the controller is configured to determine a current gaze point of the user, and the sensor array controller is configured to reorient at least one of said one or more head-mounted sensors so as to be directed towards the determined gaze point. However, in an analogous field of endeavor Lee teaches wherein the controller is configured to determine a current gaze point of the user, and the sensor array controller is configured to position one or more of the aerial vehicles such that one or more sensors disposed on said one or more of the aerial vehicles is directed towards the determined gaze point. (See Lee Figs. 9, 11A-C and ¶127-138, ¶170-176 where the UAV will move as the user moves their gaze point) Therefore, it would have been obvious for one of ordinary skill in the art that the drones in Shapira operating in a live video feed mode (See Shapira ¶73) could be controlled via the control method for controlling a live video feed as taught by Lee. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of using known methods of control to yield the desired result.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shapira in view of McDermott as applied to claim 1 above, in further view of Kamalakantha et al. (US2018/0314251)
Consider claim 19, where Shapira in view of McDermott teaches the system of claim 1, however Shapira does not explicitly teach wherein the controller is configured to determine whether one or more sensors of the plurality of aerial vehicles currently have a clear view of a part of the user's body and/or one or more objects, and to position another one or more of the aerial vehicles to monitor said part of the user's body and/or said one or more objects in dependence on a determination that said one or more sensors do not have a clear view of said part of the user's body and/or said one or more objects. However, in an analogous field of endeavor Kamalakantha teaches determine whether one or more sensors of the plurality of aerial vehicles currently have a clear view of a part of the user's body and/or one or more objects, and to position another one or more of the aerial vehicles to monitor said part of the user's body and/or said one or more objects in dependence on a determination that said one or more sensors do not have a clear view of said part of the user's body and/or said one or more objects. (See Kamalakantha Fig. 3 and ¶40-43 where processor 410 may fetch, decode, and execute instructions 422, 424, 426 to implement tracking of users with wearable devices and changing aerial movement based on a line-of-sight with one or more of the users/wearable devices and controlling aerial movement of the drone to stay within a line-of-sight of at least one of the humans.) Therefore, it would have been obvious for one of ordinary skill in the art to modify the tracking of Shapira (See Shapira ¶23) to ensure line of sight tracking as taught by Kamalakantha. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of enhancing reliability of the tracking by ensuring that the tracked object is within a line-of-sight.
Consider claim 20, where Shapira in view of McDermott in view of Kamalakantha teaches the system of claim 19, wherein the controller is configured to determine that said one or more sensors do not have a clear view if said part of the user's body and/or said one or more objects are obscured from a view of said one or more sensors, and/or if said part of the user's body and/or said one or more objects have less than a minimum angular size in a field of view of said one or more sensors. (See Kamalakantha ¶24-25 where rules 124 can include an action to take by the drone 110 based on what criteria has been fulfilled. In one example, the drone 110 can be caused to return to a line-of-sight of at least one of the devices 150 based on a determination of a potential lack of line-of-sight or a confirmed lack of line-of-sight. A confirmed lack of sight would imply that the drone does not have a clear view of the user.)
Claim(s) 21-24, 29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shapira in view of McDermott as applied to claim 1 above, in further view of Thörn et al. (US11,434,004)
Consider claim 21, where Shapira in view of McDermott teaches the system of claim 1, wherein the plurality of aerial vehicles comprise a first aerial vehicle and one or more second aerial vehicles, however they do not explicitly teach wherein the sensor array controller is configured to set a respective position for each of the one or more second aerial vehicles relative to the position of the first aerial vehicle. However, in an analogous field of endeavor Thörn teaches wherein the sensor array controller is configured to set a respective position for each of the one or more second aerial vehicles relative to the position of the first aerial vehicle. (See Thörn Fig. 3 and col 4 line 56- col8 line 57 where the drones are arranged in a predefined hierarchy by the priority values, and step 307 gives preference to lower-hierarchy drones over higher-hierarchy drones so that the former are primarily selected for adjustment. In another embodiment, step 307 may determine the priority value of each drone as a function of its current position and/or its current camera setting. For example, step 307 may seek to refrain from adjusting drones located within a predefined range of distances and/or angles with respect to the object or to a reference in the terrain. The drone 12a is effectively a master drone that controls the group of drones based on DDb, DDc from the other drones 12b, 12c.) Therefore, it would have been obvious for one of ordinary skill in the art to modify the multi-drone camera system of Shapira to prioritize the movement of lower-hierarchy drones as taught by Thörn for the advantage of/ benefit of using known methods of organization to yield the desired result.
Consider claim 22, where Shapira in view of McDermott in view of Thörn teaches the system of claim 21, wherein different ones of the plurality of aerial vehicles may be designated as the first aerial vehicle, and wherein the sensor array controller is configured to designate one of the one or more second aerial vehicles as the first aerial vehicle in response to a change in the orientation and/or position of the user's body, or in an orientation and/or position of one or more objects being monitored by the first aerial vehicle. (See Thörn Fig. 3 and col 4 line 56- col8 line 57 where the drones are arranged in a predefined hierarchy by the priority values, and step 307 gives preference to lower-hierarchy drones over higher-hierarchy drones so that the former are primarily selected for adjustment. In another embodiment, step 307 may determine the priority value of each drone as a function of its current position and/or its current camera setting. For example, step 307 may seek to refrain from adjusting drones located within a predefined range of distances and/or angles with respect to the object or to a reference in the terrain. The drone 12a is effectively a master drone that controls the group of drones based on DDb, DDc from the other drones 12b, 12c.) Therefore, it would have been obvious for one of ordinary skill in the art to modify the multi-drone camera system of Shapira to prioritize the movement of lower-hierarchy drones as taught by Thörn for the advantage of/ benefit of using known methods of organization to yield the desired result.
Consider claim 23, where Shapira in view of McDermott in view of Thörn teaches the system of claim 22, wherein the controller is configured to determine whether a predicted time for the first aerial vehicle to move from its current location to the set position for the first aerial vehicle exceeds a time limit, and to control the first aerial vehicle to move from its current location to the set position for the first aerial vehicle in response to a determination that the predicted time does not exceed the time limit, or to designate said one of the second aerial vehicles as the first aerial vehicle in dependence on a determination that the predicted time exceeds the time limit. (See Thörn Fig. 3 and col 4 line 56- col8 line 57 where step 304 associates a no-fly zone (NFZ) with the respective drone. Specifically, step 304 sets the NFZ based on the projected viewing frustum. Depending on implementation, step 304 may set the NFZ equal to the viewing frustum or larger or smaller than the viewing frustum with a certain margin. Ultimately, step 304 defines a projected time-space NFZ trajectory for the respective drone, by combining the projected flight path of the drone with the projected viewing frustum of the camera on the drone. The projected time-space NFZ trajectory thereby defines the spatial extent and location of the NFZ at one or more future time points. Reverting to FIG. 2A, the projected time-space NFZ trajectories of drones 12a, 12b correspond to the location and extent of the viewing frustum 18a, 18b, respectively, from the current time point to one or more of time points t1, t2 and t3..) Therefore, it would have been obvious for one of ordinary skill in the art to modify the multi-drone camera system of Shapira to prioritize the movement of lower-hierarchy drones as taught by Thörn for the advantage of/ benefit of using known methods of organization to yield the desired result.
Consider claim 24, where Shapira in view of McDermott in view of Thörn teaches the system of claim 23, wherein the time limit is set in dependence on the current speed of movement of the user and/or of said one or more objects, such that a shorter time limit is set when the user and/or said one or more objects is moving more quickly. (See Thörn Fig. 3 and col 4 line 56- col8 line 57 where step 304 associates a no-fly zone (NFZ) with the respective drone. Specifically, step 304 sets the NFZ based on the projected viewing frustum. Depending on implementation, step 304 may set the NFZ equal to the viewing frustum or larger or smaller than the viewing frustum with a certain margin. Ultimately, step 304 defines a projected time-space NFZ trajectory for the respective drone, by combining the projected flight path of the drone with the projected viewing frustum of the camera on the drone. The projected time-space NFZ trajectory thereby defines the spatial extent and location of the NFZ at one or more future time points. Reverting to FIG. 2A, the projected time-space NFZ trajectories of drones 12a, 12b correspond to the location and extent of the viewing frustum 18a, 18b, respectively, from the current time point to one or more of time points t1, t2 and t3. Thus, an object moving quickly would have a shorter time limit) Therefore, it would have been obvious for one of ordinary skill in the art to modify the multi-drone camera system of Shapira to prioritize the movement of lower-hierarchy drones as taught by Thörn for the advantage of/ benefit of using known methods of organization to yield the desired result.
Consider claim 29, where Shapira in view of McDermott teaches the system of claim 1, wherein the sensor array controller is configured to provide a line of sight between each one of the plurality of sensors and a respective part of the user's body and/or of one or more objects monitored by said one of the plurality of sensors. (See McDermott Figs 4A, 4B, 5A, 5B, 7 and col 5 line 58- col 6 line 32 where the reference component pixel sets based on reference images of the clothing of the subject 102. In the next frame 134b, the pixels at the left side of the grid 140 are consistent with the background of the frame 134b, while groups of pixels near the upper right of the grid 140 (and to the right of the central axis 142 of the frame 134b) may now be consistent with the subject pixel set 126 (and therefore with the subject 102). The visual recognition system 114 may conclude that the subject 102 is moving to the right with respect to the frame 134b, and that it may be desirable for the UAV 100 to rotate to its right (with respect to the yaw axis of the UAV 100) to keep the subject 102 centrally framed. Thereby maintaining a line of sight with respect to the torso of the user) Therefore, it would have been obvious for one of ordinary skill in the art that the drones of Shapira tracks the user’s hand by moving the drone such that the hand is maintained within the frame of the camera. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of maintaining a consistent orientation between the drone and the subject to maintain perspective. (See McDermott col 3 line 9-23)
McDermott teaches repositioning the drone; however McDermott does not explicitly teach focal length. However, in an analogous field of endeavor Thörn teaches increase or decrease a focal length of one or more of the plurality of sensors in addition to repositioning the plurality of aerial vehicles and/or reorienting one or more head-mounted sensors configured to be worn on the user's head, so as to. (See Thörn col 5 line 4-21 where step 302 obtains a projected camera setting of the camera on the respective drone. The camera setting may include values of camera parameters that affect the location and extent of the viewing frustum in relation to the drone, for example zoom level, aperture setting (“f-number”), focal length, and camera orientation in relation to the drone (“camera angle”)) Therefore, it would have been obvious to one of ordinary skill in the art that positioning the camera according to McDermott would factor in the focal length of the mounted camera as taught by Thörn. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of factoring in known variables in the art.
Allowable Subject Matter
Claims 15, 16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Claim 15 recites the allowable subject matter from the parent application now US Pat #11,989,351. Thus, claim 15 is allowed for similar reasons to claim 1 of the ‘351 patent. Claim 16 is objected to as allowable based upon its dependency from claim 16.
Claims 27, 28 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The claim recites the limitation: “wherein the controller is configured to reconfigure the plurality of sensors by reorienting at least one of said one or more head-mounted sensors.” The closest the Examiner could find is McClain (US5,372,132) which teaches the limitation; however, the Examiner finds it non-obvious to combine these teachings with Shapira and McDermott.
The Examiner notes that the limitation: “wherein the controller is configured to reconfigure the plurality of sensors to enable a current position of one or more parts of the user's body and/or one or more objects to be determined with a desired degree of accuracy” is not explicitly mentioned in Shapira and McDermott, however, Thörn et al. (US11,424,004) teaches “the refraining from adjustment of drones located within a predefined range of distances and/ or angles with respect to the object or to a reference in the terrain” Thus, a degree of accuracy measured by the relative object size in the image.
Claim 28 is objected based upon its dependence from claim 27.
Claims 30, 31 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The Examiner notes that the limitation: “wherein the controller is configured to reconfigure the plurality of sensors to enable a current position of one or more parts of the user's body and/or one or more objects to be determined with a desired degree of accuracy” is not explicitly mentioned in Shapira and McDermott, however, Thörn et al. (US11,424,004) teaches “the refraining from adjustment of drones located within a predefined range of distances and/ or angles with respect to the object or to a reference in the terrain” Thus, a degree of accuracy measured by the relative object size in the image.
However, the prior art does not explicitly teach “wherein in response to the determined number of aerial vehicles being less than a number of aerial vehicles currently comprised in the sensor array, the sensor array controller is configured to switch one or more redundant aerial vehicles into a standby mode in which said one or more redundant aerial vehicles do not form part of the sensor array, and wherein in response to the determined number of aerial vehicles being greater than the number of aerial vehicles currently comprised in the sensor array, the sensor array controller is configured to switch one or more aerial vehicles from the standby mode into an active mode in which said one or more aerial vehicles form part of the sensor array.” Nishio et al. (US11,417, 088) teaches UAV 100 receives a recovery instruction from the server 200, the UAV 100 takes off from the station and moves again to the round start point (in other words, the state of the UAV 100 transitions from the “charging/standby” state to the “round” state through the “movement” state). The Examiner finds the technical challenge of incorporating the teachings of Shapira, McDermott, Thörn, and Nishio to be non-obvious. Claim 31 is allowed based upon its dependence from claim 30.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM LU whose telephone number is (571)270-1809. The examiner can normally be reached 10am-6:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Eason can be reached at 571-270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
WILLIAM LU
Primary Examiner
Art Unit 2624
/WILLIAM LU/Primary Examiner, Art Unit 2624