DETAILED ACTION
This office action is responsive to communications filed on October 31, 2023. Claims 1-10 are pending in the application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The Information Disclosure Statement filed on 10/31/2023 has been considered.
Claim Objections
Claim 9 is objected to because of the following informalities: Claim 9 recites “according to any one of the local multi-device fast spatial anchor point synchronization method for mixed reality of claim 1”. In order to improve grammar and clarity, the Examiner recommends amending claim 9 to recite “according to . Appropriate correction is required.
Claim 10 is objected to because of the following informalities: Claim 10 recites “according to any one of the local multi-device fast spatial anchor point synchronization method for mixed reality of claim 1”. In order to improve grammar and clarity, the Examiner recommends amending claim 10 to recite “according to . Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the following limitations which have insufficient antecedent basis:
“in the local space” in line 3. However, there is no previous recitation of any local space;
“synchronizing time between the host and the guest” in line 6. Although there is a previous recitation of plural “guests”, there is no antecedent basis for the specific singular “guest” recited in line 6. Furthermore, the term “the guest” is repeated throughout the claim and lacks antecedent basis for the same reasons;
“the time offset” in line 8. However, there is no previous recitation of any time offset;
“the time synchronization between the guest and the host” in lines 9-10. Although there is a recitation of “synchronizing time between the host and the guest”, there is no antecedent basis for “the time synchronization” in line 9;
“performing the spatial anchor point synchronization” in line 11. However, there is no previous recitation of any spatial anchor point synchronization;
“the display system of the host” in line 12. However, there is no previous recitation of any display system;
“as the anchor point” in lines 12-13. However, there is no previous recitation of any anchor point;
“the augmented reality picture tracking technology” in lines 13-14. However, there is no previous recitation of any augmented reality picture tracking technology;
“the second position and rotation of the image anchor point in the space coordinate system” in lines 15-16. However, there is no previous recitation of any “second position and rotation of the image anchor point” or “space coordinate system”;
“the first position and rotation of the anchor point in the host space coordinate system under the same time stamp” in lines 16-17. However, there is no previous recitation of any “first position and rotation of the anchor point” or “host space coordinate system”;
“the least square method” in lines 18-19. However, there is no previous recitation of a “least square method”; and
“the third position and rotation of the origin of the space coordinate of the host” in line 19. However, there is no previous recitation of any “third position and rotation” or “origin of the space coordinate of the host”.
Furthermore, the term “several times” in Claim 1 is a relative term which renders the claim indefinite. The term “several” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Additionally, the phrase “other guests may synchronize” renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Claim 3 recites the following limitations which have insufficient antecedent basis:
“the time point when ” in line 4. However, there is no previous recitation of any time point;
“the time point of the local time stamp two as the local time stamp three” in lines 6-7. However, there is no previous recitation of any “time point of local time stamp two” or “local time stamp three”;
“the timestamp offset” in line 7. However, there is no previous recitation of any timestamp offset;
“the number of timestamp offsets reaches the set value” in line 9. However, there is no previous recitation of any “number of timestamp offsets” or “set value”;
“the standard variance” in line 11. However, there is no previous recitation of any “standard variance”;
“the preset value” in line 12. However, there is no previous recitation of any preset value;
“the earliest calculated timestamp offset” in line 12. However, there is no previous recitation of any earliest calculated timestamp offset;
“the average value of the multiple timestamp offsets as the final timestamp offset result” in line 14. However, there is no previous recitation of any “average value of the multiple timestamp offsets” or “final timestamp offset result”; and
“the final timestamp offset result between each other guest and the host” in line 15. However, there is no previous recitation of any “final timestamp offset result between each other guest and the host”.
Claim 4 recites the following limitations which have insufficient antecedent basis:
“the formula for calculating” in line 2. However, there is no previous recitation of any “formula”.
Claim 6 recites the following limitations which have insufficient antecedent basis:
“the position and rotation” in line 5. However, it is not clear whether this limitation refers to the first, second, or third position and rotation from claim 1;
“the invariant offset vector” in line 6. However, there is no previous recitation of any invariant offset vector;
“the imaging system of the host” in lines 6-7. However, there is no previous recitation of any “imaging system of the host”;
“in the scene” in line 10. However, there is no previous recitation of any scene;
“the best displacement vector and the best rotation” in lines 15-16. However, there is no previous recitation of any “best displacement vector” or “best rotation”;
“the origin of the local space coordinate system of the guest” in line 17-18. However, there is no previous recitation of any “origin of the local space coordinate system of the guest”;
“the origin of the local space coordinate system of the host” in lines 18-19. However, there is no previous recitation of any “origin of the local space coordinate system of the host”;
“the space coordinate system of the equipment in the local network is synchronized by resetting the origin of its own local space coordinate system to the origin of the local space coordinate system of the host or the origin of the local space coordinate system of any synchronized guest” in lines 20-23. However, there is no previous recitation of any “space coordinate system of the equipment”, “origin of its own local space coordinate system”, or “origin of the local space coordinate system of any synchronized guest”;
“the same size as the host in the mixed reality virtual space of the guest” in lines 24-25. However, there is no previous recitation of any “same size” or “mixed reality virtual space of the guest”; and
“the position and rotation of the virtual object” in line 25. However, there is no previous recitation of any “position and rotation of the virtual object”.
Furthermore, the term “several pairs” in Claim 6 is a relative term which renders the claim indefinite. The term “several” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Claim 7 recites the following limitations which have insufficient antecedent basis:
“the optimal displacement vector” in line 2. However, there is no previous recitation of any “optimal displacement vector”.
Furthermore, the terms “P⊥−_a”, “P⊥−_b”, and “R_y(ϕ)” are formatted incorrectly and do not match precisely with the respective terms in the formula of claim 7. Appropriate correction is required.
Claim 8 recites the following limitations which have insufficient antecedent basis:
“the calculation of the optimal rotational ϕ” in line 2. However, there is no previous recitation of any “calculation” or “optimal rotational ϕ”.
Furthermore, the terms “P_(a,i,x)^'”, “P_(a,i,z)^'”, “P_(b,i,x)^'”, “P_(b,i,z)^'”, “R_(i,1,x)^'”, “R_(i,1,z)^'”, “R_(i,3,x)^'”, “R_(i,3,z)^'” are formatted incorrectly and do not match precisely with the respective terms in the formula of claim 8. Appropriate correction is required.
Additionally, the phrase “by adjusting the value of c can make the formula more biased to a smaller rotation error” renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Claims 2, 9, and 10 depend from claim 1 and are rejected based on the same reasoning.
Claim 5 depends from claim 3 and is rejected based on the same reasoning.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter.
Claim 10 is drawn to “A machine-readable storage medium.” According to the broadest reasonable interpretation, a machine-readable storage medium may include signals. Signals are not statutory because they are not limited to a process, machine, article of manufacture, or composition of matter. In order to overcome this rejection, the Examiner recommends amending claim 10 to recite “A non-transitory machine-readable storage medium.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 9, and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2022/0379197) in view of Xu (US 2022/0292796), Werner et al. (US 2023/0394111), and Przybyla et al. (US 2020/0064439).
Regarding Claim 1, Zhang teaches a local multi-device fast spatial anchor point synchronization method for mixed reality comprising following steps:
S1: establishing a local network connection for multiple devices located in the local space to achieve data transmission between devices, using any one of the multiple devices as a host to create a local network, other devices as guests to join the local network created by the host (“the head-mounted display as a master device and the gamepad as a slave device” – See [0007]; “the head-mounted display may further include a first wireless transmission module, the gamepad may further include a second wireless transmission module” – See [0008]; “After the head-mounted display 1 is started, the first wireless transmission module and the second wireless transmission module establish a wireless connection” – See [0074]; A local network connection is established between multiple devices in a local space, wherein a head-mounted display is used as a master (host) and the gamepad (other device) joins as a slave (guest));
S2: synchronizing time between the host and the guest in the local network, wherein the guest and the host first exchange their respective local time stamps to each other, and then calculate the time offset between the local time stamp of the guest and the local time stamp of the host, according to the time offset to adjust and realize the time synchronization between the guest and the host (“the first wireless transmission module is configured to send a synchronization signal at time T1, the second wireless transmission module is configured to receive the synchronization signal at time T2, the first wireless transmission module is configured to send a data packet of the master device containing the time T1 to the slave device, after a latency of set time, the second wireless transmission module is configured to send a data packet of the slave device containing time T3 to the master device at the time T3, the master device is configured to receive the data packet of the slave device at time T4, the first wireless transmission module is configured to send a data packet of the master device containing the time T4 to the slave device, and the first time synchronization module is configured to obtain a time deviation T0 between the head-mounted display and the gamepad through the following equation: T0=(T2−T1−T4+T3)/2” – See [0009]; The master (host) and slave (guest) synchronize their timing to each other, wherein the master and slave exchange their time stamps and calculate a deviation/offset between time stamps of the master and slave to adjust timing so that synchronization is achieved).
Zhang does not explicitly teach S3: performing the spatial anchor point synchronization between the host in the local network and the guest, and using a specific picture displayed on the display system of the host as the anchor point, wherein the guest scans the specific picture through the augmented reality picture tracking technology to obtain the second position and rotation of the image anchor point in the space coordinate system of the guest; the first position and rotation of the anchor point in the host space coordinate system under the same time stamp is requested to the host, and the guest obtains two position and rotation data as a pose pair, the guest scans the anchor point several times to obtain multiple pose pairs, and calculates the multiple pose pairs, the third position and rotation of the origin of the space coordinate of the host in the space coordinate system of the guest are obtained, by resetting the origin of the space coordinate system of the guest to the third position and rotation, to make the position and rotation of the origin of the space coordinate system of the host and the guest the same, and the synchronization of the space coordinate system between the devices is completed.
However, Xu teaches performing the spatial anchor point synchronization between the host in the local network and the guest, and using a specific picture displayed on the display system of the host as the anchor point, wherein the guest scans the specific picture through the augmented reality picture tracking technology to obtain the second position and rotation of the image anchor point in the space coordinate system of the guest (“Once the initial poses of the first device 104 and the second device 116 are detected, the first device 104 may present a fiducial marker 112 (or some other type of a reference) on display 108 of the first device 104. Fiducial marker 112 may be predetermined such that the size, shape, color, pattern, shading, etc. may be known to the devices of multi-device augmented-reality system 100. For instance, fiducial marker 112 may be a checkerboard pattern of black and white squares of known size. Upon displaying fiducial marker 112, the first device 104 may transmit its pose defined in the first coordinate system 102, instructions for capturing an image of fiducial marker 112, and, optionally, information about the geometry of the first device 104 (e.g., size of the screen), and a fiducial marker identifier that indicates the particular fiducial marker presented by the first device 104 to the second device 116” – See [0028]; “a fiducial marker may include one or more shapes within the fiducial marker that appear differently when rotated or transformed so as to indicate a degree of rotation and/or transformation upon detection. The degree of rotation/transformation may be used to determine the orientation of the device that captured the image” – See [0035]; See also Figs. 1 and 3A; The first and second devices (host and guest) perform a spatial anchor point synchronization where a fiducial marker (specific picture) is displayed on the display of the first device/host, and the second device/guest captures the fiducial marker to obtain transformation (position) and rotation values for the image anchor point in the spatial coordinate system);
the first position and rotation of the anchor point in the host space coordinate system under the same time stamp is requested to the host, and the guest obtains two position and rotation data as a pose pair, the guest scans the anchor point several times to obtain multiple pose pairs, and calculates the multiple pose pairs (“At block 404, a first pose (T11) of a first mobile device may be received. The first mobile device may execute a position tracking process, such as a SLAM process, that tracks the pose (T11) of the first mobile device associated with the coordinate system of the first mobile device. The first pose, (T11), received at block 404 may be a particular pose at a particular instant in time. The first pose may be associated with a first timestamp corresponding to an instant in which the first pose was determined. In some instances, a sequence of poses may be received with each pose of the sequence of poses being associated with a corresponding first timestamp” – See [0057]; “At block 406, a second pose (T22) may be determined by the second device. The second pose (T22) may be defined relative to the coordinate system associated with the second mobile device. The second pose may be determined using a position tracking process, such as a SLAM process” – See [0058]; See also Fig. 3A; The second device requests first pose information (i.e., first rotation and first transformation/position) from the first device under the same time stamp, and the second device obtains a pose pair including pose T11 and T22, where each of pose T11 and T22 includes respective rotation and transformation/position data. Thus, when the guest scans the first device/anchor point several times, multiple pose pairs are calculated),
the third position and rotation of the origin of the space coordinate of the host in the space coordinate system of the guest are obtained, by resetting the origin of the space coordinate system of the guest to the third position and rotation, to make the position and rotation of the origin of the space coordinate system of the host and the guest the same, and the synchronization of the space coordinate system between the devices is completed (“At block 428, a third pose (T21) of the second mobile device relative to the first coordinate system may be defined using the correspondence between the three-dimensional coordinates and the set of feature points. For instance, a perspective-n-point process may use the correspondence as input (along with camera calibration data such as scaled focal lengths, skew parameter, principal point, scale factor, and/or the like) to generate an estimated third pose of the second mobile device. The third pose of the second mobile device may be associated with the coordinate system of the first mobile device” – See [0065]; “For instance, during the calibration, it can be determined that a current position of the first device 104 is at the origin of its own coordinate system 102 … The first device 104 may not know the environment within which it is positioned, but it may track its position relative to where the first device 104 was initially calibrated (e.g., its origin). For instance, if the internal sensor indicate that the first device 104 has moved a meter after the calibration, the position of the first device 104 may be determined as being a meter (in a particular direction) from the origin” – See [0025]; “Coordinate-system transform may be used to map points (e.g., coordinates) of the second coordinate system to corresponding coordinates in the first coordinate system and vice versa” – See [0055]; The guest resets its origin to the position it was in when the first/second/third position and rotation data are obtained and coordinate synchronization was completed. Furthermore, after the coordinate synchronization is completed, a transform is used to map the coordinate system of the guest to corresponding coordinates in the host, making the position and rotation of the origin of the space coordinate system of the host and the guest the same).
Zhang does not explicitly teach calculating the multiple pose pairs by the least square method.
However, Werner teaches calculating the multiple pose pairs by the least square method (“estimator module 245 may use a least-squares estimator to estimate the location of electronic device 140 from the range value and pose pairs corresponding to the stored anchor points” – See [0029]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang to calculate the multiple pose pairs by the least square method. Motivation for doing so would be to enable a confidence metric to be associated with the estimated location of the devices, which ensures that the estimated location meets a predetermined minimum likelihood (See Werner, [0029]).
Zhang does not explicitly teach that other guests may synchronize spatial coordinate systems with hosts in the local network, or with guests that have completed spatial coordinate systems synchronization in the local network.
However, Przybyla teaches that other guests may synchronize spatial coordinate systems with hosts in the local network, or with guests that have completed spatial coordinate systems synchronization in the local network (“FIG. 11 is a process flow diagram of a method for using two frequencies to locate a head-mounted device (HMD) and hand-held controllers (HHCs) in accordance with an embodiment of the present invention” – See [0017]; “Position and orientation measurements may be expressed with regard to a “room coordinate system” that is defined with respect to an environment in which the system 10 is operated, e.g. walls, floor, ceiling, or other fixed structures. As described below, position and orientation of the controllers 14a may be determined with respect to a coordinate system defined with respect to the HMD 22. This facilitates displaying representations of the controllers 14 in a display of the HMD 22 inasmuch as they are rendered according to the HMD coordinate system” – See [0151]; See also Fig. 1; Other guests (e.g., a second controller) may synchronize their spatial coordinate system with the host (e.g., HMD)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang such that other guests may synchronize spatial coordinate systems with hosts in the local network, or with guests that have completed spatial coordinate systems synchronization in the local network. Motivation for doing so would be to enable multiple controllers to be tracked by a host (See Przybyla, [0017]).
Regarding Claim 9, Zhang in view of Xu, Werner, and Przybyla teaches the method of Claim 1. Zhang further teaches a local multi-device fast spatial anchor point synchronization system for mixed reality, wherein the system comprises a memory and a processor, instructions are stored on the processor and the instructions enable the processor to execute according to any one of the local multi-device fast spatial anchor point synchronization method for mixed reality of claim 1 (“The embodiments of the present disclosure further provide a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program, when being executed by a processor, causes the processor to implement the positioning and tracking method according to any one of the aforementioned embodiments and exemplary embodiments” – See [0043]).
Regarding Claim 10, Zhang in view of Xu, Werner, and Przybyla teaches the method of Claim 1. Zhang further teaches a machine-readable storage medium, wherein instructions are stored on the machine-readable storage medium, and the instructions are used to cause the machine to execute: according to any one of the local multi-device fast spatial anchor point synchronization method for mixed reality of claim 1 (“The embodiments of the present disclosure further provide a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program, when being executed by a processor, causes the processor to implement the positioning and tracking method according to any one of the aforementioned embodiments and exemplary embodiments” – See [0043]).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2022/0379197) in view of Xu (US 2022/0292796), Werner et al. (US 2023/0394111), and Przybyla et al. (US 2020/0064439) and further in view of Ikeda et al. (US 2018/0205870).
Regarding Claim 2, Zhang in view of Xu, Werner, and Przybyla teaches the method of Claim 1. Zhang, Xu, Werner, and Przybyla do not explicitly teach that the step S1 is specifically: S1-1: the host creates the local network and broadcasts on the local network through wireless communication; S1-2: the guest searches the broadcast of the host through the wireless communication mode; S1-3: the guest discovers the broadcast and sends a connection request to the host using the wireless communication mode; and S1-4: the host receives the connection request, agrees to the connection request, and establishes a network connection between the guest and the host.
However, Ikeda teaches S1-1: the host creates the local network and broadcasts on the local network through wireless communication; S1-2: the guest searches the broadcast of the host through the wireless communication mode; S1-3: the guest discovers the broadcast and sends a connection request to the host using the wireless communication mode; and S1-4: the host receives the connection request, agrees to the connection request, and establishes a network connection between the guest and the host (“Pairing is then carried out in order to establish communication between the digital camera 100 and the smartphone 200. The digital camera 100 broadcasts, on the wireless LAN network, a discovery signal for discovering the presence of an apparatus to serve as a connection partner. This discovery signal includes device information of the digital camera 100, such as a model name of the digital camera 100, names of functions the digital camera 100 can handle, and so on. If the smartphone 200 has received the discovery signal transmitted from the digital camera 100, the smartphone 200 transmits a connection request to the digital camera 100. Upon the connection request being permitted by the digital camera 100, the digital camera 100 and the smartphone 200 enter a state of being capable of exchanging data through wireless communication (that is, communication is established). FIG. 3C illustrates a state in which communication is established between the digital camera 100 in simple AP mode and the smartphone 200” – See [0051]; Device 100 (host) wirelessly broadcasts a discovery signal, device 200 (guest) searches for and receives the discovery signal, device 200 sends a connection request to device 100 after receiving the discovery signal, and device 100 receives the connection request and permits/agrees to the connection, thereby establishing the connection between devices 100 and 200).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang such that the step S1 is specifically: S1-1: the host creates the local network and broadcasts on the local network through wireless communication; S1-2: the guest searches the broadcast of the host through the wireless communication mode; S1-3: the guest discovers the broadcast and sends a connection request to the host using the wireless communication mode; and S1-4: the host receives the connection request, agrees to the connection request, and establishes a network connection between the guest and the host. Motivation for doing so would be to enable to host to determine whether the connection with the guest should be permitted. For example, the host may determine whether to permit the connection based on whether an encryption key received from the guest is correct (See Ikeda, [0109]).
Allowable Subject Matter
Claims 3-8 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Scott M Sciacca whose telephone number is (571)270-1919. The examiner can normally be reached Monday thru Friday, 7:30 A.M. - 5:00 P.M. EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Avellino can be reached at (571) 272-3905. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT M SCIACCA/ Primary Examiner, Art Unit 2478