CTNF 19/273,540 CTNF 89230 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim (s) 1, 2, 4-10, 12-18 and 20-24 , is/are rejected under 35 U.S.C. 103 as being unpatentable over Bendale et al. U.S. Patent Publication No. 2018/0341811 (hereinafter Bendale) . Consider claim 1, Bendale teaches an electronic device comprising (Figure 2, 200) : a first sensor ([0035], one or more cameras, one or more sensors) ; a second sensor ([0035], one or more cameras, one or more sensors) ; one or more processors [0035] ; and memory storing instructions configured to be executed by the one or more processors, the instructions for ([0035], storage…supporting various functionalities. [0101-0102] and figure 11, storage 1106, memory 1104, processor 1102) : while the electronic device is at a first position, obtaining, using at least the first sensor, first sensor data for a physical environment ([0030-0031], the camera system 202 may transmit the captured media content to the AR engine 204 which processes the media content. The AR Engine 204 may combine the images or videos captured by the cameras with some annotations in order to generate augmented media content (e.g., images with annotations) ; storing, in the memory, a first data set for the physical environment based at least on the first sensor data ([0048], previous frame) ; receiving a query regarding the physical environment [0033], an annotation may refer to one or more specific objects within content a user is viewing, user is viewing) ; and in accordance with receiving the query regarding the physical environment and while the electronic device is within a threshold distance of the first position ([0053], motion (change of distance) is less than the threshold) : obtaining second sensor data from the sensor ([0053], previous frame and current frame) ; and using the second sensor data to reference the first data set ([0053] When the motion is less than the threshold, the system 400 may automatically identify the object by accessing from the previous frames an identification of the object, metadata associated with the object, annotations associated with the objects, and masks associated with the objects. The system may automatically determine an annotation for the object based on the accessed identification, metadata, annotations, and mask from previous frames). Bendale’s [0053] do not appear to specifically disclose data from the second sensor. However, Bendale teaches in [0030], camera system may also include one or more sensors including, for example, but not limited to, a microphone, a motion sensor, an accelerometer, a gyroscope, a light sensor, a direction sensor, an inertial measurement unit, etc. In particular embodiments, the camera system 202 may capture one or more frames of a media content including, for example, images (including still images and/or video) and/or audio. Each frame of the media content may correspond to at least part of an image captured by one or more cameras. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to provide data based on second sensors as taught by Bendale with the benefit that the AR Engine 204 may combine the images or videos captured by the cameras with some annotations in order to generate augmented media content (e.g., images with annotations) as suggested in [0031]. Consider claim 2, Bendale teaches all the limitations of claim 1. In addition, Bendale teaches wherein the instructions further comprise instructions for: receiving an additional query regarding the physical environment ([0033], queries for different annotations (see also figures 3a-c)) ; and in accordance with receiving the additional query regarding the physical environment and while the electronic device is outside the threshold distance from the first position ([0053], when the motion is greater than the threshold) : obtaining, using at least the first sensor, third sensor data for the physical environment ([0053], the system may automatically identify the object using one or more object detectors (i.e., object classifiers) and identify the frames containing the motion as a key frame) ; and storing, in the memory, a second data set for the physical environment based at least on the third sensor data ([0052-0053], the system 400 may compare a current frame (i.e., the frame being analyzed) with a preceding frame or a previous frame). Consider claim 4, Bendale teaches all the limitations of claim 1. In addition, Bendale teaches wherein the first sensor comprises a depth sensor, a camera, or a motion sensor [0030] . Consider claim 5, Bendale teaches all the limitations of claim 1. In addition, Bendale teaches wherein using the second sensor data to reference the first data set comprises using the second sensor data to reference the first data set without using the first sensor ([0051], the system 400 may only perform object detection and object refinement on key frames and may not perform object detection and object refinement for non-key frames, thereby resulting in a significant computation reduction) . Consider claim 6, Bendale teaches all the limitations of claim 1. In addition, Bendale teaches wherein the instructions further comprise instructions for, in accordance with receiving the query regarding the physical environment ([0033], queries for different annotations (see also figures 3a-c)) and while the electronic device is within the threshold distance of the first position ([0053], motion (change of distance) is less than the threshold) : determining, using a third sensor, a direction of gaze ([0033], if the user is viewing (gaze)) , wherein using the second sensor data to reference the first data set comprises identifying a physical object aligned with the direction of gaze using the second sensor data and the first data set without using the first sensor ([0051], the system 400 may only perform object detection and object refinement on key frames and may not perform object detection and object refinement for non-key frames, thereby resulting in a significant computation reduction. [0051], non-key frames may contain one or more changes relative to a preceding frame, but those changes are less than a threshold, and thus information from the previous frame can be used in the current frame.[0033], if the user is viewing) . Consider claim 7, Bendale teaches all the limitations of claim 1. In addition, Bendale teaches wherein the first data set is a scene understanding data set ([0053], current frame (see also figure 3a)) , wherein the first data set comprises a spatial mesh that represents the physical environment ([0053], object (see objects in figures 3a-c)) , and wherein the first data set comprises identities of physical objects at corresponding positions or directions relative to the first position (Figures 3a-c, objects in front of user. [0053], motion/movement of an object) . Consider claim 8, Bendale teaches all the limitations of claim 1. In addition, Bendale teaches wherein the instructions further comprise instructions for: based on a result from using the second sensor data to reference the first data set [0053] , presenting content associated with the query ([0033], annotation) , wherein presenting content associated with the query comprises presenting visual content associated with the query using a display (See annotations in figures 3a-c (e.g. 312)) and presenting audio content associated with the query using a speaker [0087], audio) . Consider claim 9, it includes the limitations of claim 1 and thus these limitations are rejected by the same reasoning. In addition, Bendale teaches a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device ([0101-0102] and figure 11, storage 1106, memory 1104, processor 1102) . Consider claim 10, it includes the limitations of claim 2, and thus rejected by the same reasoning. Consider claim 12, it includes the limitations of claim 4, and thus rejected by the same reasoning. Consider claim 13, it includes the limitations of claim 5, and thus rejected by the same reasoning. Consider claim 14, it includes the limitations of claim 6, and thus rejected by the same reasoning. Consider claim 15, it includes the limitations of claim 7, and thus rejected by the same reasoning. Consider claim 16, it includes the limitations of claim 8, and thus rejected by the same reasoning. Consider claim 17, it includes the limitations of claim 1, and thus rejected by the same reasoning. Consider claim 18, it includes the limitations of claim 2, and thus rejected by the same reasoning. Consider claim 20, it includes the limitations of claim 4, and thus rejected by the same reasoning. Consider claim 21, it includes the limitations of claim 5, and thus rejected by the same reasoning. Consider claim 22, it includes the limitations of claim 6, and thus rejected by the same reasoning. Consider claim 23, it includes the limitations of claim 7, and thus rejected by the same reasoning. Consider claim 24, it includes the limitations of claim 8, and thus rejected by the same reasoning . 07-22-aia AIA Claim (s) 3, 11 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bendale as applied to claim s 1, 9 and 17 above, and further in view of Xiong U.S. Patent Publication No. 2024/0078765 (hereinafter Xiong) . Consider claim 3, Bendale teaches all the limitations of claim 1. In addition, Bendale teaches wherein the instructions further comprise instructions for: receiving an additional query regarding the physical environment ([0033], queries for different annotations (see also figures 3a-c)) ; and in accordance with receiving the additional query regarding the physical environment, while the electronic device is within the threshold distance of the first position [0053] . Bendale does not appear to specifically disclose and in accordance with determining that the first data set is older than a threshold duration of time; obtaining, using at least the first sensor, fourth sensor data for the physical environment; and storing, in the memory, a third data set for the physical environment based at least on the fourth sensor data. However, in a related field of endeavor, Xiong teaches an augmented reality device (abstract) and further teaches and in accordance with determining that the first data set is older than a threshold duration of time ([0055], oldest) ; obtaining, using at least the first sensor, fourth sensor data for the physical environment ([0055], depth maps and camera) ; and storing, in the memory, a third data set for the physical environment based at least on the fourth sensor data ([0055], latest depth map, store and images queue) . Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to store a third data set with the benefit that the image queue can be updated by replacing oldest image with the latest captured image as suggested in [0055]. Consider claim 11, it includes the limitations of claim 3, and thus rejected by the same reasoning. Consider claim 19, it includes the limitations of claim 3, and thus rejected by the same reasoning. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERTO W FLORES whose telephone number is (571)272-5512. The examiner can normally be reached Monday-Friday, 7am-4pm, EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, AMR A AWAD can be reached at (571)272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /ROBERTO W FLORES/Primary Examiner, Art Unit 2621 Application/Control Number: 19/273,540 Page 2 Art Unit: 2621 Application/Control Number: 19/273,540 Page 3 Art Unit: 2621 Application/Control Number: 19/273,540 Page 4 Art Unit: 2621 Application/Control Number: 19/273,540 Page 5 Art Unit: 2621 Application/Control Number: 19/273,540 Page 6 Art Unit: 2621 Application/Control Number: 19/273,540 Page 7 Art Unit: 2621 Application/Control Number: 19/273,540 Page 8 Art Unit: 2621 Application/Control Number: 19/273,540 Page 9 Art Unit: 2621
Prosecution Insights