DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over US 20220116105 A1 (Robinson et al.) (hereinafter Robinson) in view of US 20200218690 A1 (Huston et al.) (hereinafter Huston).
In re claims 1, 8 and 16, Robinson discloses a computation system and a method to determine availability of continuous communication with a satellite constellation ([0006], “a device for detecting a zone of communication between a phased array antenna and a satellite constellation including a plurality of satellites in non-geosynchronous orbit (non-GEO) is provided”. [0051], “Embodiments disclosed herein relate to systems and methods for positioning endpoint terminals 102 based on obstructions that may prevent signals from being transmitted between the endpoint terminals 102 and satellites within satellite constellations”), the system comprising: a mobile device (Fig. 12, [0115], “Referring to FIG. 12, a block diagram of an example device 1200 that facilitates configuring the endpoint terminal 102 for communication with a satellite constellation is provided. In a non-limiting example, the device 1200 may correspond to a mobile device (e.g., a smart phone, a tablet, a laptop, etc.)”), comprising a non-transitory processor readable medium, a display, one or more processors ([0006], “The device includes one or more processors and memory. The memory stores thereon instructions that, as a result of being executed by the one or more processors, cause the device to: determine a location of the device, wherein the location corresponds to a field of regard for detecting the zone of communication between the phased array antenna and the satellite constellation”), and a camera (Fig. 12:1200, [0115], “As shown, device 1200 may also include a display 1214 that displays graphics and/or images (e.g., for viewing by a user of the device 1200). The device 1200 may include one or more additional components not illustrated in FIG. 12, such as an accelerometer, a gyroscope, a magnetometer, an inertial measurement unit (IMU), a camera device, an image sensor, a radar sensor, a light detection and ranging (LIDAR) sensor, a Global Positioning System (GPS), a graphics processing unit (GPU) 114, a digital signal processor (DSP), an image signal processor (ISP), among other components”), the mobile device having installed an augmented reality (AR) application ([0115], “For instance, a user of the device 1200 may download and install the application on the device 1200 in order to facilitate configuring the endpoint terminal 102 for communication with a satellite constellation”. [0166], “In some instances, rather than overlaying the obstructed and unobstructed regions of the field of regard onto the image data in real-time, the display engine 1208 may generate a virtual reality (VR) or panoramic visualization of the captured image data. The display engine 1208 may overlay the obstructed and unobstructed regions of the field of regard onto the VR or panoramic visualization of the captured image data such that the user may discern these regions from the processed image data. This may reduce the processing requirements of the device 1200 in performing real time update of the display to provide the portions of the estimated obstruction map corresponding to the portion of the field of regard corresponding to the present attitude of the device 1200”) configured to:
output, via the display of the mobile device (Fig. 12:1214), a user interface depicting a field of view (Fig. 12:1210) being captured by the camera of the mobile device ([0127], “The display engine 1208 may indicate and/or emphasize the portion of the field of regard included within the image frame 1216 in any suitable manner. In some cases, the field of regard outline 1214 may be implemented as a 3-dimensional (3D) scene rendered on top of a live view of a camera of the device 1200”);
capture, via the camera of the mobile device, a plurality of frames of an overhead area through which the continuous communication with the satellite constellation is to be provided, the plurality of frames collectively mapping a 360-degree representation of the overhead area ([0132], “As noted above, the scene engine 1204 may receive one or more image frames (e.g., an image frame 1216) captured by the device 1200, whereby the image frame 1216 may include image data corresponding to a scene surrounding the device 1200”);
determine that the continuous communication with the satellite constellation is available using the plurality of frames ([0130], “In some examples, the display engine 1208 may visually indicate any obstructions that are visible within the field of regard. For instance, the display engine 1208 may highlight, outline, or otherwise indicate the trees and/or the telephone pole shown in FIG. 14A. Further, the display engine 1208 may display one or more satellites and/or orbital paths within the display 1214. Displaying satellites and/or orbital paths within the display 1214 may indicate and/or emphasize the importance of selecting a location that provides an unobstructed field of regard for the endpoint terminal”); and
output, via the display, an obstruction map based on the plurality of frames that indicates open sky available for communication with the satellite constellation (Fig. 16A, Fig. 21, [0128], “In some cases, the display engine 1208 may output an instruction 1212 to the user within the display 1214. For example, the display engine 1208 may generate an instruction 1212 that facilitates finding a suitable location for the endpoint terminal 102”. [0137], “Additionally, the shape of the field of regard may be dynamically updated in real-time based on any obstructions detected by the obstruction engine 1206 based on one or more image frames (e.g., an image frame 1216) captured by the device 1200” (obstruction map indicating open sky available for a suitable location)).
Robinson does not explicitly disclose the mobile device having installed an augmented reality (AR) application and capture, via the camera of the mobile device, a plurality of frames of an overhead area, the plurality of frames collectively mapping a 360-degree representation of the overhead area.
Huston discloses the mobile device having installed an augmented reality (AR) application (Fig. 2, Fig. 8, [0043], “A method for extracting a plane uses Simultaneous Localization and Map-building (SLAM)/Parallel Tracking And Mapping (PTAM) algorithm for tracking three-dimensional positional information of a camera and three-dimensional positional information of feature points in real time and providing AR using the plane has been suggested. However, since the SLAM/PTAM algorithm acquires the image to search for the feature points, computes the three-dimensional position of the camera and the three-dimensional positions of the feature points”. [0134], “Additionally, the user can also view augmented reality enhancements relevant to the particular location they are viewing using a mobile device, such as mobile device 10, 12 or 220”. [0047], “Because the use of a real environment as the background is common, “augmented reality” often refers to a technology of inserting a virtual reality graphic (object) into an actual digital image and generating an image in which a real object and a virtual object are mixed”) and capture, via the camera of the mobile device, a plurality of frames of an overhead area, the plurality of frames collectively mapping a 360-degree representation of the overhead area ([0009], “a system for creating an environment for use with a location based experience includes a plurality of mobile devices accompanying a number of random contributors, each having a camera to capture random images and associated metadata near a point of interest, wherein the metadata for each image includes location of the mobile device and the orientation of the camera”. [0010], “The images and metadata are processed to build a 3D model of the region near a point of interest”. [0112], “Such scanning systems are available from Matterport of Mountain View, California, which include both conventional images and structured light data acquired in a 360′ area around the scanner. A 3D model of an area can be created by scanning an area from a number of points creating a series of panoramas. Each panorama is a 3D model consisting of images stitched to form a mosaic, along with the 3D depth information (from the depth camera) and associated metadata. In other words, traversing an area near a point of interest and scanning and collecting images over multiple points create a high-fidelity 3D model of the area near the point of interest”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Robinson with Huston to provide a system and technique to detect a zone of communication for earth-based satellite communication systems using 360-degree imaging of points of interest and virtual reality tools. The advantage of doing so is to enable the end user to maintain a continuous communication with the satellites and minimize obstructions such as trees, buildings etc.
In re claims 2, 9 and 17, the combination discloses the system of claim 1, the method of claim 8 and the computational system of claim 16, wherein Robinson discloses the method further comprising:
determining a recommendation with respect to installing an antenna for a user terminal based on a predefined threshold with respect to the open sky available for communication with the satellite constellation ([0006], “and output, to a user of the device, an indication of the level of communication between the phased array antenna and the satellite constellation associated with the field of regard”. [0142], “Based on detecting the tree T and/or the building B within the field of regard 162a, the device 1200 may determine that location 1 is unsuitable for installation of the endpoint terminal 102. For example, the device 1200 may determine that the obstructions will prevent a sufficient level of communication between the endpoint terminal 102 and the constellation of satellites 166. Thus, the device 1200 may provide a visual indication and/or output an instruction to the user directing the user to move the device 1200 to a new (e.g., different) location” (providing recommendation)); and
outputting, via the user interface (Fig. 12:1214), the obstruction map, wherein the obstruction map comprises a visual indicator associated with the open sky available for communication with the satellite constellation and the recommendation ([0162], “Referring to FIG. 21, the display engine 1208 may use the obstruction data to generate the obstruction map visualization, which may be presented to the user via the display of the device 1200”).
In re claims 3, 10 and 18, the combination discloses the system of claim 2, the method of claim 9 and the computational system of claim 17, wherein Robinson discloses wherein the predefined threshold with respect to the open sky available for communication with the satellite constellation is independent of an orientation of the antenna (Fig. 16A, [0168], “In some embodiments, the endpoint terminal 102 is a device that is installed at an end-user premises in order to provide access to the communication network to the end-user premises. As shown, the endpoint terminal 102 includes an endpoint communication interface 2202. The endpoint communication interface 2202 allows the endpoint terminal 102 to communicate with a satellite, such as the first satellite 104 (see FIG. 1). In some embodiments, the endpoint communication interface 2202 may include a phased array antenna configured to communicate with the first satellite 104, for example, via the Ku band”. [0169], “The location determination engine 2200 may receive information regarding the latitude location for the endpoint terminal, a longitude location of the endpoint terminal, obstructions, geological features, population density, an altitude of the end point terminal, a load balancing analysis of the satellite constellation, one or more angles of inclination of the satellite constellation, a geographical cell to which the end point terminal belongs, and combinations thereof” (suitable location depends on many other factors and not the orientation of the antenna)).
In re claims 4, 11 and 19, the combination discloses the system of claim 1, the method of claim 8 and the computational system of claim 16, wherein Robinson discloses wherein the determining that the continuous communication with the satellite constellation is available further comprises executing a machine-learning model for:
identifying a block region of the plurality of frames, wherein the block region comprises a plurality of pixels ([0157], “In an embodiment, the scene engine 1204 uses a convolutional neural network (CNN) to segment the image data into obstructed and unobstructed regions of the field of regard. The CNN may perform binary segmentation of the stored images, whereby for each pixel in a stored image, the CNN may predict whether that pixel represents an obstructed or unobstructed region of the stored image and/or field of regard represented in the stored image. The CNN may be trained using sample images and segmentation masks from open-source and/or proprietary datasets”);
determining that at least one pixel of the plurality of pixels includes an obstruction by performing image segmentation ([0157], “The CNN may perform binary segmentation of the stored images, whereby for each pixel in a stored image, the CNN may predict whether that pixel represents an obstructed or unobstructed region of the stored image and/or field of regard represented in the stored image”); and
assigning an obstruction identifier to the block region associated with the at least one pixel, wherein the obstruction identifier indicates that the overhead area corresponding to the block region is obstructed ([0159], “the scene engine 1204, in addition to using a CNN to segment the image data into obstructed and unobstructed regions of the field of regard, can use a separate machine learning algorithm or artificial intelligence for semantic classification of the image data”. [0163], “Within the graphical representation of the field of regard, the display engine 1208 may divide the field of regard into obstructed and unobstructed regions based on the obtained obstruction data...For example, as illustrated in FIG. 21, the display engine 1208 may use a blue color to graphically denote unobstructed regions of the field of regard and a red color to graphically denote obstructed regions of the field of regard. It should be noted that while different colors are used to distinguish between obstructed and unobstructed regions of the field of regard, other techniques may be used to distinguish between these regions”).
In re claims 5, 13 and 20, the combination discloses the system of claim 1, the method of claim 8 and the computational system of claim 16, wherein Robinson discloses wherein prior to capturing the plurality of frames of the overhead area: output an elevation indicator overlaying the user interface, wherein the elevation indicator is configured to indicate a predefined angle at which to tilt the mobile device prior to capturing the plurality of frames of the overhead area; and subsequent to the mobile device being tilted at the predefined angle, provide, via the user interface, an interface element configured to be selected to initiate the capturing step (Fig. 14B).
In re claims 6 and 14, the combination discloses the system of claim 1 and the method of claim 8, wherein Robinson discloses wherein the method comprises outputting, via the user interface, one or more graphical elements overlaying the field of view, wherein an amount of the one or more graphical elements is configured to indicate progress of the capturing step; capturing a particular frame of the plurality of frames, wherein the particular frame corresponds to a subset of the one or more graphical elements; in response to capturing the particular frame, removing a subset of the one or more graphical elements from the user interface; and outputting a progress indicator corresponding to the subset of the one or more graphical elements removed from the user interface (Fig. 19B, Fig. 20).
In re claims 7 and 15, the combination discloses the system of claim 1 and the method of claim 8, wherein Robinson discloses wherein the method comprises wherein the obstruction map further comprises a two-dimensional representation of the overhead area, azimuthal identifiers with respect to the two-dimensional representation, and cardinal direction indicators with respect to the two- dimensional representation, and wherein the two-dimensional representation includes a visual indicator configured to indicate a respective location of obstructions detected in the overhead area (Fig. 13, [0163], “The obstruction map may provide a visual representation of obstructed and unobstructed regions of the field of regard”).
In re claim 12, the combination discloses the method of claim 11, wherein Robinson discloses wherein the machine-learning model is configured for retraining using an updated training dataset, and wherein the updated training dataset includes at least one frame of the plurality of frames captured during the capturing step ([0157], “Further, in some instances, as new image data is obtained from different users for corresponding devices, the scene engine 1204 may store this new image data, along with the classification of regions of the fields of regard corresponding to the new image data, to supplement the datasets used to train the CNN”. [0158], “In some instances, as new image data and corresponding classifications are obtained, the datasets used to train the CNN may be changed”).
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SWATI JAIN whose telephone number is (571)270-0699. The examiner can normally be reached Mon - Fri (830 am - 530 pm).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pan Yuwen can be reached on 5712727855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SWATI JAIN/Examiner, Art Unit 2649 /YUWEN PAN/Supervisory Patent Examiner, Art Unit 2649
Prosecution Insights