CTNF 19/097,273 CTNF 91490 DETAILED ACTION Claims 1-20 are considered in this office action. Claims 1-20 are pending examination. Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention. 07-16-aia AIA Claim s 1-2, 8-9, and 15-16 are rejected under 35 U.S.C. 102(a)(1) based upon a public use or sale or other public availability of the invention Yu et al. "Multi-Modal Recurrent Fusion for Indoor Localization," ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, Singapore, 2022, pp. 5083-5087, doi: 10.1109/ICASSP43922.2022.9746071) . Regarding Claim 1, Yu teaches a method for indoor localization (See at least Abstract: “This paper considers indoor localization using multi-modal wireless signals including Wi-Fi, inertial measurement unit (IMU), and ultra-wideband (UWB). By formulating the localization as a multi modal sequence regression problem, a multi-stream recurrent fusion method is proposed to combine the current hidden state of each modality in the context of recurrent neural networks while accounting for the modality uncertainty which is directly learned from its own immediate past states.”), comprising: locating a target object in an indoor space by employing sensors of different modalities; converting data from the sensors of different modalities into a single modality by employing a sensor-agnostic modality converter (Fig.2 and Section 3.3 : “Compared with the standard multi-stream LSTM [30] for computer vision applications, the data quality of multi-modal RF sensors may vary drastically over time due to the surrounding environment, e.g., non-line-of-sight (NLOS) scenarios, and sensor failures. For instance, the UWB works well when the user is in sight while degrading quickly at NLOS locations. Motivated by the observation, we propose to project a concatenated “immediately preceding” hidden states hu……, M refers to the sensor type such as the RSSI, CSI, IMU and UWB, and m αm =1.” AND also See Section 4.2: “We standardize each input entry by subtracting the mean and normalizing it with the standard deviation. For the LSTM, we use a network configuration of 2 layers, a hidden dimension of Nh = 256, an optional bi-directional implementation, an uncertainty estimation block of a varying number of hidden states F = {1,2}, and an output block of Q = 2 FC layers with a hidden layer dimension of 128. To regularize the training from overfitting, we adopt the dropout that randomly sets the hidden state outputs of both the first and second LSTM layers to zeros with a probability of 0.2 [32].”); PNG media_image1.png 390 1242 media_image1.png Greyscale and determining from the data in the single modality a range of the target object from a fixed point to locate a position of the target object within the indoor space (Section 3.4: “3.4. Recurrent Fusion With the multi-stream LSTM and the uncertainty estimates, we propose to fuse the last hidden states from multiple LSTM streams, weighted by the learned uncertainty: M hfusion = m=1 αmhm t . (15) Thefusedstate is then fed into the coordinate estimation block which consists of several FC layers along with the ReLU activation σR(·): [ˆxt; ˆ yt] = WQ ovQ o +bQ o,vq o = σR(Wq−1 o vq−1 o +bq−1 o ), (16) where q = 1,··· ,Q, v0 o = hfusion, and Q is the number of FC layers. We train the multi-modal fusion network in an end-to-end fashion with the loss function of mean squared error (MSE) between the estimated coordinate [ˆxt, ˆyt] and the ground truth [xt,yt]. Remark: Kalmanfilter-like approaches also use relative importance or uncertainty for multi-sensor fusion via the propagation of (cross and self-) covariance matrices of multiple sensor modalities with known measurement and (and likely Markovian) dynamics models. In contrast, the multi-stream LSTM is a data-driven approach that utilizes a standard LSTM to learn both nonlinear measurement and dynamics models over a long-term horizon and introduces a nonlinear mapping of immediate past hidden states to estimate the relative importance”). Similarly Claim 8 and 15 are rejected on the similar rational. Regarding Claim 2, Yu teaches method of claim 1. Yu also teaches determining from the data in the single modality an angle of the target object from a fixed point to locate a position of the target object within the indoor space (Section 3.1 and Equation 4: “3.1. Data Curation Commodity CSI measurements are known to be impacted by hardware impairments such as the sample frequency offset (SFO) and carrier frequency offset (CFO). To mitigate these impacts, we take a standard procedure to compress the raw CSI data. As shown in the top figures of Fig. 2, null subcarriers are first removed and the ‘remaining CSI are flipped to smooth the whole frequency spectrum. Later, the CSI is locally calibrated by normalizing each CSI by its total sum, i.e., ˜xc i(t) = |xc i(t)|/ j |xc i,k(t)| where xc i,k(t) is the k-th subcarrier CSI of the i-th anchor at time t. The local calibration is able to fix the gain fluctuation due to the use of automatic gain control (AGC) at commodity Wi-Fi devices. To reduce the data overhead, we apply the polynomial fitting to compress the calibrated CSI into a weight vector ai(t) of dimension P [28], ˜ xc i(t) = Kai(t) (4) where K = [1,k,··· ,kP] is the polynomial basis matrix with k grouping the remaining subcarrier indices and P denoting the polynomial order, and ai(t) = [a0,i(t),··· ,aP,i(t)]T is the coefficient vector for the i-th calibrated CSI amplitude.”). Similarly Claim 9 and 16 are rejected on the similar rational . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries set forth in Graham v. John Deere Co. , 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim s 3-6, 10-13 and 17-19 rejected under 35 U.S.C. 103 as being unpatentable over Yu in view of Kusens et al. (US2016/0049028) and herein after will be referred as Kusens . Regarding Claim 3, Yu teaches method of claim 1. Kusens teaches registering the target object to a network once a connection between the sensors and the network is established and assigning a registration status to the target object (Para [0029-0032]: “At step F 2 c , the access control and location computer system can directly send the guest an electronic key to their smartphone or other electronic device via electronic communication methods including but not limited to direct data connection, SMS, Email, MMS and voice. A confirmation electronic message can be sent to the member to inform them that their guest's key was approved and sent to the guest. Alternatively, the system can be programmed that the guest key is first sent to the member, and the member forwards it to the guest. The key is imported to a software application, which is stored locally on the guest's device. This application acts as an electronic keychain of access keys. In one non-limiting embodiment, the digital key can be an electronic file, which is preferably encrypted. The key can be auto-imported where it is sent to person's electronic device through an app directly that is downloaded on the electronic device or manually added if the key is sent through SMS or email. For the manual method, the guest can click on the file and than have an app import the key to the local device database. Once the guest receives the key, the guest has all access rights, which have been granted to them by a member, as seen/discussed in step F 2 a and FIG. 7. The guest can have a key provided by multiple members within the same Access Control & Location Tracking System location or keys for multiple locations (with separate instances of the Access Control & Location Tracking System). As a non-limiting example, if the guest is a service provider (i.e. plumber, electrician, personal trainer, delivery person, etc.) the guest may need to have keys from multiple members at any given time. Also in some instances a person can be a member at one location and a guest at other locations and may have member key(s) and guest(s) keys on his or her electronic keychain database stored on his or her electronic device.[0030] At step F 2 d , the electronic key is electronically stored in the access control & location tracking keychain database on the guest's device. [0031] FIG. 3 illustrates how the system grants or denies access to a member or guest based on the electronic key on their device. [0032] At step F 3 a , the member or guest attempting to enter a controlled access location will have an electronic key on their device, such as the electronic key the guest receives from the steps described in FIG. 2. Through a wireless radio, sound and/or light enabled application, their device will retrieve all electronic keys stored in the device's keychain database and transmit them to any wireless radio, sound and/or light-based beacons in an immediate proximity to the controlled access area entrance. FIG. 9 shows one non-limiting embodiment where the electronic device can be configured for its owner to manually select the digital key to transmit (i.e. virtual clicker), while FIG. 10 shows another non-limiting embodiment where the electronic device can be configured to auto-sense that it is at a beacon and then have the user manually select the digital key to transmit to the beacon and FIG. 11 shows a further non-limiting embodiment where the electronic device can be configured to automatically sense that it is at a beacon and then automatically send the digital key(s) to the beacon. The member or guest can also choose which specific key to transmit if so configured and desired. The key can be manually chosen via a user interface provided by the software installed on the member's or guest's electronic device or it can also just send all keys available on the users keychain to the system and it will continue to check each key on the keychain to see if one grants them access for the location, date and time. The system can be programmed such that access denial is only given after all available keys are checked. Preferably, the built in capabilities of conventional smartphones/electronic devices can be used, as they currently come with Wifi, Bluetooth and sometimes NFC radios or InfraRed sensors, and some also have ultrasonic capable microphones or lifi built in. If not provided, these technologies can be provided or later acquired by the electronic device. The Access Control and Location Tracking system, through instructions provided by the programmed software, accesses the radios and other communication hardware available on the electronic device and uses them as needed.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yu to incorporate the teachings of Kusens to include registering the target object to a network once a connection between the sensors and the network is established and assigning a registration status to the target object. Doing so would optimize the indoor tracking/ localization process as disclosed in Kusens. Similarly Claim 10 is rejected on the similar rational. Regarding Claim 4, Yu in view of Kusens teaches the method of claim 3. Kusens teaches identifying the target object from the range and the registration status (Para [0038-0040]: “At step F4 a, physical wireless radio, sound and/or light-based beacons are placed throughout a controlled access area. These are arranged so that when a member or guest with a wireless radio, sound and/or light enabled device and the permissions application running enters the area, they are preferably constantly within range of a beacon. The member or guests access key can be automatically electronically retrieved from the keychain database stored in their electronic device and transmitted by the wireless radio, sound and/or light-based beacons to the access control & location tracking system preferably in continuous intervals.[0039]At step F4 b, the access control & location tracking system receives the access key(s) and compares the key(s) to the access control & location tracking database to determine the permissions afforded to each specific key that is received.[0040]At step F4 c, if the member or guest is in an authorized location based on the permissions retrieved in F4 b, then the system will update the database to reflect the current location of the member or guest.”). Similarly Claim 11 and 17 are rejected on the similar rational. Regarding Claim 5, Yu in view of Kusens teaches the method of claim 4. Kusens teaches selectively allowing access to one or more of a plurality of regions of the indoor space to the target object to corresponding to the target object identification and a corresponding access level within the network (Para [0038-0041]: “At step F4 a, physical wireless radio, sound and/or light-based beacons are placed throughout a controlled access area. These are arranged so that when a member or guest with a wireless radio, sound and/or light enabled device and the permissions application running enters the area, they are preferably constantly within range of a beacon. The member or guests access key can be automatically electronically retrieved from the keychain database stored in their electronic device and transmitted by the wireless radio, sound and/or light-based beacons to the access control & location tracking system preferably in continuous intervals.[0039] At step F4 b, the access control & location tracking system receives the access key(s) and compares the key(s) to the access control & location tracking database to determine the permissions afforded to each specific key that is received.[0040] At step F4 c, if the member or guest is in an authorized location based on the permissions retrieved in F4 b, then the system will update the database to reflect the current location of the member or guest. [0041] At step F4 d. If the member or guest is in an unauthorized location, then the system administrator and/or security staff is notified. In the case of a guest, the member who granted the guest access can be notified as well that the guest has gone beyond the parameters of their authorization. The alert is generated by the access control & location tracking system and can be sent through computer, voice, email, IM, SMS, MMS, pager or other communication method. The access control & location tracking database can also be updated with the member or guests current location. Additionally, the termination or suspension of all or some of the guest/member's access permissions as described above can also be performed by the Access Control & Location Tracking system.”). Similarly Claims 12 and 18 are rejected on the similar rational. Regarding Claim 6, Yu teaches the method of claim 1. Yu teaches computing a trajectory of the target object according to the range (Section 4.1); and Kusens teaches providing navigation services to the target object based on the computed trajectory of the target object (Para [0047]). Similarly Claims 13 and 19 are rejected on the similar rational . 07-21-aia AIA Claim s 7, 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yu in view of Kusens and in further view of Bao et al. ("A Sensor Fusion Strategy for Indoor Target Three-dimensional Localization based on Ultra-Wideband and Barometric Altimeter Measurements," 2022 19th International Conference on Ubiquitous Robots (UR), Jeju, Korea, Republic of, 2022, pp. 181-187, doi: 10.1109/UR55393.2022.9826288) and herein after will be referred as Bao . Regarding Claim 7, Yu teaches the method of claim 1. Bao teaches wherein the sensors further include at least one barometric sensor (See atleast abstract: “Obtaining relative spatial localization information of target objects is crucial in scenarios such as robotics operations and augmented reality. In this paper, a strategy based on data fusion of ultra-wideband (UWB) sensors and barometric pressure (BMP) sensors are proposed to identify the three-dimensional (3D) localization information of indoor targets”). Similarly Claims 14 and 20 are rejected on the similar rational . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Davis et al. (US8660581) discloses a method for indoor navigation in a venue derives positioning of a mobile device based on sounds captured by the microphone of the mobile device from the ambient environment. It is particularly suited to operate on smartphones, where the sounds are captured using microphone that captures sounds in a frequency range of human hearing. The method determines a position of the mobile device in the venue based on identification of the audio signal, monitors the position of the mobile device, and generates a position-based alert on an output device of the mobile device when the position of the mobile device is within a pre-determined position associated with the position based alert. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDHESH K JHA whose telephone number is (571)272-6218. The examiner can normally be reached M-F:0800-1700. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James J Lee can be reached at 571-270-5965. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /ABDHESH K JHA/Primary Examiner, Art Unit 3668 Application/Control Number: 19/097,273 Page 2 Art Unit: 3668 Application/Control Number: 19/097,273 Page 3 Art Unit: 3668 Application/Control Number: 19/097,273 Page 4 Art Unit: 3668 Application/Control Number: 19/097,273 Page 5 Art Unit: 3668 Application/Control Number: 19/097,273 Page 6 Art Unit: 3668 Application/Control Number: 19/097,273 Page 7 Art Unit: 3668 Application/Control Number: 19/097,273 Page 8 Art Unit: 3668 Application/Control Number: 19/097,273 Page 9 Art Unit: 3668 Application/Control Number: 19/097,273 Page 10 Art Unit: 3668 Application/Control Number: 19/097,273 Page 11 Art Unit: 3668 Application/Control Number: 19/097,273 Page 12 Art Unit: 3668
Prosecution Insights