DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Ribah et al. (LAPAN-A2 Microsatellite Performance on Target Pointing Maneuver for Imaging Mission, IEEE International Conference on Aerospace Electronics and Remote Sensing Technology (ICARES), DOI:10.1109/ICARES53960.2021.9665184, hereinafter “Ribah”) in view of Peng et al. (US 20200007914 A1, hereinafter “Peng”).
Regarding claim 1, Ribah discloses An observation assistance apparatus comprising: (Abstract, “AGI System Tool Kit (STK) is used for simulating satellite orbit and determining the angular rate profile, elevation, and azimuth angle by computing access from the satellite to the target.”). Note that: the computing device running STK can be regarded as an observation assistance apparatus.
generate target region quasi-image that shows a target region and is estimated to be obtained when an image of the target region is captured from a viewpoint of a mobile body that is equipped with an observation apparatus for observing the Earth's surface and moves along a path in a space above the Earth's surface; and (page 2, col. right, Fig. 3: a satellite moves along a “trajectory” as a path and captures the images of “Region to be imaged” as a target region while the camera points to the target region indicated as arrows can be regarded as the satellite / camera viewpoints (“1”, “2”, and “3”), respectively; page 3, col. left, Fig. 4: “LAPAN-TUBSAT/A1 target pointing result” as a target region quasi-image can be acquired in the past for reference; page 2, col. right, para. 2, “Inside LAPAN-A2, there has been installed the matrix camera which provides imagery with 4 m spatial resolution and is able to view a swath 7 km wide”). Note that: (1) LAPAN-A2 Microsatellite is a mobile body that is equipped with an observation apparatus for observing the Earth's surface and moves along a path in a space above the Earth's surface; and (2) the target region quasi-image includes a target region.
generate, if the target region quasi-image is selected by an operator, control instruction information that is used for controlling at least a timing when the mobile body observes the target region, a position at which the mobile body performs observation, and a direction in which the mobile body performs observation, which are information for causing the mobile body to observe an actual observation image of the target region equivalent to the target region quasi-image, based on mobile body position information indicating a position of the mobile body and mobile body direction information indicating a direction of the mobile body that were used when the selected target region quasi-image was generated. (page 3, col. right, para. 1, “the target pointing maneuver is simplified by selecting the target that located in line with the satellite ground track. Hereby the satellite will focus on the pitch rotation without any disturbance in the roll and yaw axis. The target and the time execution will be determined by simulating LAPAN-A2 in AGI STK software. Afterward, the access data between the target and the satellite has been generated. These data consist of Azimuth, Elevation, and Range (AER), and angulare rate during the time execution.”; page 4, col. left, para. 1, “AGI System Tool Kit (STK) were used to simulate the satellite orbit and placed the target. The calculation of the satellite access and angular rate are obtained during simulation”; page 4, col. right, para 1, “Based on the simulation, the target pointing maneuver would be executed on August 30th, 2021 starting from 02:23:38 UTC to 02:23:58 UTC, the target is located in 3,009o of latitude and 101.690o of longitude. Furthermore, elevation angle and angular rate also have been generated. By using (3), the pitch angle could be determined based on generated satellite-to-target elevation data at each second. Then, the angular rate from the simulation will be used as the profile of the angular rate on the target pointing maneuver”; page 4, col. right, para. 2, “The next step is arranging the satellite command schedule for the maneuver. The schedule consists of several commands contains specific order values for the camera system and the ADCS that the satellite system could execute the commands automatically. This process will produce two data, both are the captured image from the camera and the attitude data telemetry from the ADCS. The schematic diagram for the whole process of the research is shown in Fig. 10. Both data will be analyzed in the next section. The schedule that has already been arranged will be uploaded to the satellite by the operator when the satellite passes the ground stations before the time execution.”; page 1, Abstract, “AGI System Tool Kit (STK) is used for simulating satellite orbit and determining the angular rate profile, elevation, and azimuth angle by computing access from the satellite to the target.”). Note that: (1) It obvious to one having ordinary skills in the art that a set of images captured in the past for the target region can be candidates for an operator’s selection for the target region quasi-image. Once the target region quasi-image is selected, STK can be used to simulate the satellite orbit and place the target to the region corresponding to the target region quasi-image; (2) based on the simulation, the timing (“starting from 02:23:38 UTC to 02:23:58 UTC”) that the satellite pointing to the target (3,009o of latitude and 101.690o of longitude), the satellite / camera position that can be computed for simulating satellite orbit by STK (see Fig. 3 of Ribah above, “1”, “2”, or “3”), and attitude or direction of the satellite with the attached camera (elevation angle, angular rate, and the pitch angle) have been generated; and (3) the above information can be regarded as control instruction information as a schedule will be uploaded to the satellite by the operator.
However, Ribah fails to disclose, but in the same art of computer graphics, Peng discloses
at least one memory storing instructions; and
at least one processor configured to execute the instructions to: (Peng, (Peng, para. [0006], “The electronic device includes at least one processor and a computer readable storage. The computer readable storage is coupled to the at least one processor and stores at least one computer executable instruction thereon which, when executed by the at least one processor, causes the at least one processor to:”).
Ribah and Peng are in the same field of endeavor, namely computer graphics and image processing. Before the effective filing date of the claimed invention, it would have been obvious to apply an apparatus with a memory and a processor, as taught by Peng into Ribah. The motivation would have been “The electronic device includes at least one processor and a computer readable storage. The computer readable storage is coupled to the at least one processor and stores at least one computer executable instruction thereon which, when executed by the at least one processor” (Peng, para. [0006]). The suggestion for doing so would allow to use an apparatus with a memory and a processor to implement observation assistance. Therefore, it would have been obvious to combine Ribah and Peng.
Claim 5 reciting “An observation assistance method to be performed by a computer, comprising:” is corresponding to the apparatus of claim 1. Therefore, claim 5 is rejected for the same rationale for claim 1.
In addition, Ribah in view of Peng discloses An observation assistance method (Ribah, Abstract, “AGI System Tool Kit (STK) is used for simulating satellite orbit and determining the angular rate profile, elevation, and azimuth angle by computing access from the satellite to the target.”). Note that: (1) simulating satellite orbit and determining the angular rate profile, elevation, and azimuth angle can be regarded as an observation assistance method; and (2) the method can be implemented to be performed by a computer, comprising: (Peng, para. [0006], “The electronic device includes at least one processor and a computer readable storage. The computer readable storage is coupled to the at least one processor and stores at least one computer executable instruction thereon which, when executed by the at least one processor, causes the at least one processor to:”). Note that: the electronic device can be regarded as a computer.
The motivation to combine Ribah and Peng given in claim 1 is incorporated here.
Claim 9 reciting “A non-transitory computer-readable recording medium that includes a program recording thereon, the program including instructions that cause a computer to carry out:” is corresponding to the apparatus of claim 1. Therefore, claim 9 is rejected for the same rationale for claim 1.
In addition, Ribah in view of Peng discloses A non-transitory computer-readable recording medium that includes a program recording thereon, the program including instructions that cause a computer to carry out: (Peng, para. [0007], “a non-transitory computer readable storage medium is provided. The non-transitory computer readable storage medium is configured to store a computer program which, when executed by a processor, causes the processor to carry out following actions”).
Ribah in view of and Peng, are in the same field of endeavor, namely computer graphics. Before the effective filing date of the claimed invention, it would have been obvious to apply non-transitory computer readable media comprising computer readable instructions which, when executed, configure a data processing system to perform operations, as taught by Peng into Ribah. The motivation would have been “a non-transitory computer readable storage medium is provided. The non-transitory computer readable storage medium is configured to store a computer program which, when executed by a processor, causes the processor to carry out following actions” (Peng, para. [0007]). The suggestion for doing so would allow to use one or more non-transitory computer readable media to perform computer graphics operations. Therefore, it would have been obvious to combine Ribah and Peng.
Claims 2, 6 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Ribah in view of Peng and Archive_1 (3D Sun-Path, Archive.org, https://web.archive.org/web/20211216212217 /http://andrewmarsh.com/software/sunpath3d-web/, hereinafter “Archive_1”).
Regarding claim 2, Ribah in view of Peng discloses The observation assistance apparatus according to claim 1,
wherein the one or more processors further:
However, Ribah in view of Peng fails to discloses, but in the same art of computer graphics, Archive_1 discloses
adds a shadow image to the target region quasi-image using a timing when it is possible to observe the generated target region quasi-image, insolation information indicating an altitude and an amplitude of the sun in the target region, and three- dimensional map information of the target region. (Archive_1, page 1, para. 1, “This app demonstrates the relationship between geographic location and solar position throughout the year. You can use the map to drag the location around and interactively see how the Sun-path diagram and shadow projections change. You can also directly relate location and the 30 Sun-path to day-length and a range of different 20 Sun-path projections.”; page 1, Figure 1: “A screenshot of the 3D Sun-Path Diagram app running inside a web browser”, “
PNG
media_image1.png
866
1328
media_image1.png
Greyscale
”, adding shadow image to the target region quasi-image showing the building blocks of the target region indicated by “GEOGRAPHIC LOCATION” and corresponding shadows according to the solar information, when seeing the generated target region quasi-image, “solar information”, “azi / alt” (an altitude and an amplitude of the sun in the target region at time point “OATEAND TIME”, “Rise / Set” of the sun, and 3D map of the building blocks).
Ribah in view of and Peng, and Archive_1, are in the same field of endeavor, namely computer graphics. Before the effective filing date of the claimed invention, it would have been obvious to apply adding shadows to the target region quasi-image, as taught by Archive_1 into Ribah in view of Peng. The motivation would have been “You can use the map to drag the location around and interactively see how the Sun-path diagram and shadow projections change” (Archive_1, page 1, para. 1). The suggestion for doing so would allow to add shadow image to the target region quasi-image dynamically according to the target region quasi-image, sun-path, and 3D map information. Therefore, it would have been obvious to combine Ribah, Peng, and Archive_1.
Claims 6 and 10 are corresponding to the apparatus of claim 2. Therefore, claims 6 and 10 are rejected for the same rationale for claim 2.
Claims 3-4, 7-8 and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Ribah, Peng, Archive_1, and Mattila et al. (WO 2015107263 A1, hereinafter “Mattila”).
Regarding claim 3, the combination of Ribah, Peng, and Archive_1 discloses The observation assistance apparatus according to claim 1,
wherein the one or more processors further:
adds a shadow image to the target region quasi-image using a timing when it is possible to observe the generated target region quasi-image, (Archive_1, page 1, para. 1, “This app demonstrates the relationship between geographic location and solar position throughout the year. You can use the map to drag the location around and interactively see how the Sun-path diagram and shadow projections change. You can also directly relate location and the 30 Sun-path to day-length and a range of different 20 Sun-path projections.”; page 1, Figure 1: “A screenshot of the 3D Sun-Path Diagram app running inside a web browser”, “
PNG
media_image1.png
866
1328
media_image1.png
Greyscale
”, adding shadow image to the target region quasi-image showing the building blocks of the target region indicated by “GEOGRAPHIC LOCATION” and corresponding shadows according to the solar information, when seeing the generated target region quasi-image, “solar information”, “azi / alt” (an altitude and an amplitude of the sun in the target region at time point “OATEAND TIME”, “Rise / Set” of the sun, and 3D map of the building blocks).
However, the combination of Ribah, Peng, and Archive_1 fails to disclose, but in the same art of computer graphics, Mattila discloses
… weather information indicating weather in the target region (Mattila, para. [0066], “the shadows on the building facades with or without reflective materials can produce useful features, such as, high scores and/or better rankings, if they are consistent with the weather and/or the time of the day and/or the position of the camera and/or the position of the sun etc … a high-contrast pattern of the shadows on a building facade with no features (uniform color and/or a flat surface with no texture) may generate useful features for detection and tracking … the display platform 109 may cause an accurate environmental lighting based, at least in part, on the time of the day, the position of the sun, the weather, and 3D geometry of the objects in the environment (i.e. buildings, trees, statutes etc.)”). Note that: (1) the shadows on the building can produce useful features for detection and tracking; (2) the display platform 109 may cause an accurate environmental lighting based, weather and other factors on the time of the day, the position of the sun, the weather, and 3D geometry of the objects in the environment to deal with shadows; and (3) weather information in the target region can substitute the position of sun to be observed or used to add shadow to the target region quasi-image.
Ribah in view of and Peng, and Archive_1, and Mattila, are in the same field of endeavor, namely computer graphics. Before the effective filing date of the claimed invention, it would have been obvious to apply adding shadows to the target region quasi-image observing or using weather information, as taught by Mattila into the combination of Ribah, Peng, and Archive_1. The motivation would have been “the shadows on the building facades with or without reflective materials can produce useful features, such as, high scores and/or better rankings, if they are consistent with the weather and/or the time of the day and/or the position of the camera and/or the position of the sun etc” (Mattila, para. [0066]). The suggestion for doing so would allow to add shadow image to the target region quasi-image observing or using weather information. Therefore, it would have been obvious to combine Ribah, Peng, Archive_1, and Mattila.
Regarding claim 4, Ribah in view of Peng discloses The observation assistance apparatus according to claim 1,
wherein the target region quasi-image and the control instruction information (Ribah, page 4, col. left, Fig. 7: “LAPAN-A2 simulation in AGI STK 3D view”, “
PNG
media_image2.png
690
1078
media_image2.png
Greyscale
”). Note that: Fig. 7 as an image shows both the target region quasi-image and the control instruction information (timing / position / directions or attitude).
are displayed on a display device (Mattila, para. [0075], “Other external devices coupled to bus 1010, used primarily for interacting with humans, include a display device 1014, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED) display, a plasma screen, or a printer for presenting text or images”). Note that: the image combining both the target region quasi-image and the control instruction information can be displayed on a display device (e.g., a liquid crystal display (LCD), a light emitting diode (LED) display).
Ribah in view of and Peng, and Archive_1, and Mattila, are in the same field of endeavor, namely computer graphics. Before the effective filing date of the claimed invention, it would have been obvious to apply displaying images on a display device, as taught by Mattila into the combination of Ribah, Peng, and Archive_1. The motivation would have been “Other external devices coupled to bus 1010, used primarily for interacting with humans, include a display device 1014, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED) display, a plasma screen, or a printer for presenting text or images” (Mattila, para. [0075]). The suggestion for doing so would allow to display images on a display device. Therefore, it would have been obvious to combine Ribah, Peng, Archive_1, and Mattila.
Claims 7-8 and 11-12 are corresponding to the apparatus of claims 3-4, respectively. Therefore, claims 7-8 and 11-12 are rejected for the same rationale for claims 3-4, respectively.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BIAO CHEN whose telephone number is (703)756-1199. The examiner can normally be reached M-F 8am-5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee M Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Biao Chen/
Patent Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611