DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is in response to Applicant’s Amendments and Remarks filed on 8/18/2026. Claims 2 and 4 have been canceled. Claims 1, 3, 5, 19, and 20 have been amended. Claims 21 and 22 have been added. Claims 1, 3, and 5-22 are present for examination.
The 35 USC 112(f) interpretation of claim 19 has been withdrawn in view of the amendments.
The 35 USC 101 rejection of claim 20 as program per se has been withdrawn in view of the amendments.
The 35 USC 112(b) rejections of claims 3 and 5 have been withdrawn in view of the amendments.
Response to Arguments
Applicant's arguments filed 8/18/2026 have been fully considered but they are not persuasive.
Regarding the 35 USC 101 rejections:
Applicant submits: “Just as the Dejardins claims reflected ‘an improvement to how the machine learning model itself operates,’ the present claims reflect an improvement to how a rephotographing instructions is produced, in that the photographing mode is determined from a factor derived from the point cloud data rather than from the position at which a failure was observed.” (See Remarks filed on 8/18/2026, p. 11, 2nd para.)
The examiner respectfully disagrees. The generating rephotographing instructions are mental process because a person can determine how to rephotograph in the form of instructions in the mind. It is different than machine learning which has been determined as not a mental process. MPEP 2106.05(a) states, “the judicial exception alone cannot provide the improvement… it is important to keep in mind that an improvement in the abstract idea itself is not an improvement in technology.” An improvement on generating rephotographing instructions is an improvement in the mental process (as an abstract idea) itself. Therefore, it cannot provide an improvement in the corresponding technology.
Applicant submits: “Amended Claim 1 recites creating three-dimensional point cloud data of a photographing target from a plurality of photographed images shot at a plurality of photographing points by a mobile body, and detecting the defective region as a region "for which point data is not created in the three-dimensional point cloud data." The specification describes that the point cloud data is calculated using, for example, SfM (see publication, [0100]). Creating three-dimensional point cloud data from a plurality of photographed images, and locating within that data a region for which point data is not created, cannot practically be performed in the human mind or with pen and paper. Under the August 2025 Memo, limitations of that kind do not fall within the mental process grouping.” (See Remarks filed on 8/18/2026, p. 11, last para.)
The examiner respectfully disagrees. As stated in the non-final rejection dated 5/19/2026, the limitation of creating three-dimensional point cloud data has been considered as an additional element of mere data gathering because the method of claim 1 is to use the three-dimensional point cloud data to detect defective regions and generate rephotographing instructions. A person could look at the point cloud data and mentally determine defective regions based on the point cloud data and generate rephotographing instructions. Therefore, the creating point cloud data is merely data gathering for the mental process of detecting defective regions and generating rephotographing instructions.
Applicant submits:
PNG
media_image1.png
174
790
media_image1.png
Greyscale
PNG
media_image2.png
360
792
media_image2.png
Greyscale
(See Remarks filed on 8/18/2026, p. 12, last two paras.)
The examiner respectfully disagrees. Claims 7-9 have been indicated as eligible because they recite changing the movement route of the mobile body. Such limitations can be considered as improvement to the photographing technology because they are additional elements that allow the mobile body to photograph the subject along a different movement route to remedy the photographed defective region. However, such limitations are not recited in claim 1 because claim 1 merely recites generating instruction information without any further limitations to change the movement route of the mobile body according to the instruction information.
Applicant submits:
PNG
media_image3.png
448
794
media_image3.png
Greyscale
(See Remarks filed on 8/18/2026, p. 13, first para.)
The examiner respectfully disagrees. The created point cloud data is not gathered and analyzed to determine defective regions. This is similar to the example of pre-solution activity of a step of gathering data for use in a claimed process, e.g., a step of obtaining information about credit card transactions, which is recited as part of a claimed process of analyzing and manipulating the gathered information by a series of steps in order to detect whether the transactions were fraudulent provided in MPEP 2106.05(9) “Gathering and analyzing information. The amended claim 1 creating three-dimensional point cloud data of a photographing target based on photographed images and detecting defective region based on the images and point cloud data, which is similar to obtaining information of a photographing target as part of a process of analyzing the gathered information (images and point cloud data) to detect whether there is a defective region in the images. Therefore, the creating step in amended claim 1 is considered insignificant extra-solution activity of data gathering.
Applicant submits:
PNG
media_image4.png
266
772
media_image4.png
Greyscale
(See Remarks filed on 8/18/2026, p. 13, 2nd para.)
The examiner respectfully disagrees. MPEP 2106.05(a) (II) states that merely adding generic computer components to perform the method is not sufficient to show the involvement of a computer assist in improving the technology. Claim 19 has been amended to recite processing circuitry configured to perform the steps recited in claim 1. Such implementation is considered merely using the processing circuitry as a tool to perform an existing process. (See MPEP 2106.05(f).) Therefore, the processing circuitry cannot integrate the abstract ideas into practical application.
Applicant also submits that the amended claims recite significantly more than an abstract idea itself under Steo 2B. Applicant argues that the Office Action merely provided conclusory statements and resting on the assertion that no additional elements are recited. Applicant also argues that a non-conventional arrangement of known elements supplies an inventive concept and the claims (even if determined to be an abstract idea as suggested by the Office Action) provide improvements over existing technological processes and thus recite “significantly more” than an abstract idea itself. (See Remarks filed on 8/18/2026, p. 13, last para. – p. 14, 4th para.)
The examiner respectfully disagrees. The amended claim 1 recites additional elements such as creating point cloud data but merely data gathering as stated in the non-final rejection in rejecting claim 2 under 35 USC 101. Such additional elements are considered as significant extra solution therefore do not amount significantly more to the abstract idea. Also as discussed above, an improvement to the abstract idea only provides an improved abstract idea, which is still an abstract idea.
Therefore, Applicant’s arguments are not persuasive. Amended claim 1 can still be rejected under 35 USC 101 as being ineligible.
Regarding the 35 USC 103 rejections:
Applicant submits: Gu’s loss area is an area in which point group data has been generated but falls below a threshold quantity or ratio. Gu does not describe or reasonably suggest “detecting a defective region … for which point data is not created in the three-dimensional point cloud data,” as recited in Claim 1. Gu also does not describe or reasonably suggest “determining, as a state of the defective region a non-creation factor of the photographing target, that the point data is not created,” as recited in Claim 1. No step in that sequence disclosed in Gu determines, as a state of the detected area, a factor of the photographing target that point data is not created. Gu, [0183]-[0186] address the acquisition of position information on the loss area and the generation of and flight along the flight routes FPB1 and FPC1. The position of a loss area is not a non-creation factor of a photographing target, and a photographing mode determined from the position of the loss area is not determined according a non-creation factor. Thus, Gu does not describe or reasonably suggest creating three-dimensional point cloud data of a photographing target based on a plurality of photographed images shot at a plurality of photographing points by a mobile body that moves on a movement route; detecting a defective region, of the photographing target based on a photographed image of the plurality of photographed images, for which point data is not created in the three-dimensional point cloud data; and determining, as a state of the defective region a non-creation factor of the photographing target, that the point data is not created, as recited in Claim 1. (See Remarks filed on 8/18/2026, pp. 16-18.)
The examiner respectfully disagrees.
As stated in the 5/19/2026 non-final rejection, Gu, para. [0118], discloses detecting whether there is a loss area in the three-dimensional shape data, the loss area is an area in the three-dimensional space where point group data configuring the generated three-dimensional shape data is lower than a predetermined number or ratio, indicating the loss area can correspond to the defective region, which is detected corresponding to a region for which the point data is created at a number lower than a predetermined number in a region of point group data corresponding to the three-dimensional point cloud data point data. Although Gu does not expressly disclose the point data is not created in the three-dimensional point cloud data in the detected region, the predetermined number disclosed in Gu can be one (1), and the number lower than one (1) will be zero (0). Accordingly, the point data at zero (0) means the point data is not created. According to MPEP 2144.05, where the claimed ranges overlap or lie inside ranges disclosed by the prior art (here is Gu), a prima facie case of obviousness exists. Therefore, before the invention was effectively filed, it is obvious that Gu teaches detecting as the defective region a region for which the point data is not created in the three-dimensional point cloud data. The suggestion/motivation would have been to supplement the loss area where the point group data is insufficient to improve the restoration accuracy of the three-dimensional shape data.
Amended claim 1 recites “determining, as a state of the defective region a non-creation factor of the photographing target, that the point data is not created”. According to amended claim 1, “the point data is not created” is determined as a state of the defective region is a non-creation factor of the photographing. Gu, para. [0118], discloses detecting whether there is a loss area in the three-dimensional shape data, the loss area is an area in the three-dimensional space where point group data configuring the generated three-dimensional shape data is lower than a predetermined number or ratio. Gu, para. [0163], discloses detecting one loss area or a plurality of loss areas, the loss area may occur because information on pixel values at positions in the three-dimensional space is insufficient, and the loss area may occur because the image information is insufficient in one portion covered by the other portion. As discussed above, although Gu does not expressly disclose the point data is not created in the three-dimensional point cloud data in the detected region, the predetermined number disclosed in Gu can be one (1), and the number lower than one (1) will be zero (0). Accordingly, the point data at zero (0) means the point data is not created. Therefore, the reason the loss area may occur or the number of loss areas can correspond to a non-creation factor as a state of the defective region and when the loss area corresponds to an area where the point data is not created (the predetermined number of points is zero (0), it can be determined as a state of the defective region a non-creation factor of the photographing target. Therefore, Gu discloses determining, as a state of the defective region a non-creation factor of the photographing target, that the point data is not created.
Therefore, Applicant’s arguments regarding the 35 USC 103 rejections are not persuasive.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3, 5, 6, 10 -12, and 17-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. The claim(s) recite(s) abstract idea.
MPEP 2106 III provides a flowchart for the subject matter eligibility test for product and processes. The claim analysis following the flowchart is as follows:
Regarding claim 1, it recites
A photographing instruction method comprising:
creating three-dimensional point cloud data of a photographing target based on a plurality of photographed images shot at a plurality of photographing points by a mobile body that moves on a movement route;
detecting a defective region, of the photographing target based on a photographed image of the plurality of photographed images, for which point data is not created in the three-dimensional point cloud data;
determining, as a state of the defective region a non-creation factor of the photographing target, that the point data is not created;
determining a photographing mode of rephotographing of the defective region according the non-creation factor when detecting the defective region; and
generating instruction information for instructing the rephotographing according to the determined photographing mode.
Step 1: Is the claim to a process, machine, manufacture or composition of matter?
Yes. It recites a method, which is a process.
Step 2A, Prong One: Does the claim recite an abstract idea, law of nature, or nature phenomenon?
Yes.
The step of detecting a defective region, of the photographing target based on a photographed image of the plurality of photographed images, for which point data is not created in the three-dimensional point cloud data can be a mental process because a person can look at the point cloud data and the photographed images and detect a region where point data is not created in the point cloud data in the mind, and mentally determine such region as a defective region.
The step of determining, as a state of the defective region a non-creation factor of the photographing target, that the point data is not created can be a mental process because a person can mentally determine that the point data is not created as a state of the defective region a non-creation factor of the photographing target.
The step of determining a photographing mode of rephotographing of the defective region according the non-creation factor when detecting the defective region can be a mental process because a person can mentally determine a photographing mode according to the state of the defective region the photographing target.
The step of generating instruction information for instructing the rephotographing according to the determined photographing mode can be a mental process because a person can mentally generate instruction information such as steps for the rephotographing according to the determined photographing mode, or with the simple tool such as pen and paper to mentally generate instruction information and write them on a piece of paper using a pen.
Step 2A, Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application?
No.
Claim recites “creating three-dimensional point cloud data of a photographing target based on a plurality of photographed images shot at a plurality of photographing points by a mobile body that moves on a movement route”. This limitation can be considered as an additional element of data gathering because it provides data for the detecting step. Data gathering is usually considered as insignificant extra solution, which cannot integrate the abstract idea into practical application.
Therefore, this judicial exception is not integrated into a practical application because there are the additional element recited in claim 1 is mere data gathering and thus the abstract idea cannot be integrated into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
No.
There are no additional elements recited in the claim.
Therefore, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no additional elements recited in the claim and thus no additional elements to amount to significantly more than the abstract idea.
Therefore, claim 1 is not eligible subject matter under 35 USC 101.
Regarding claim 3, it depends from claim 1 and further recites detecting a portion at which pixels do not match between the plurality of photographed images as the region for which the point data is not created.
The step of detecting a portion at which pixels do not match between the plurality of photographed images as the region for which the point data is not created can be considered as mental process because a person can look at the images and the point cloud to mentally determine a portion at which pixels do not match between photographed images.
Claim 3 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 3 is not eligible subject matter under 35 USC 101.
Regarding claim 5, it depends from claim 3 and further recites determining the non-creation factor based on image processing of the photographed image in which the defective region has been detected.
The step of determining the non-creation factor based on image processing can be considered as mental process because a person can mentally determine the non-creation factor based on how the images are processed and what the image processing results are.
Claim 5 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 5 is not eligible subject matter under 35 USC 101.
Regarding claim 6, it depends from claim 1 and further recites further comprising determining to change the movement route as the photographing mode.
The step of determining to change the movement route as the photographing mode can be considered as mental process because a person can mentally determine what will be a photographing mode including changing the movement route.
Claim 6 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 6 is not eligible subject matter under 35 USC 101.
Regarding claim 10, it depends from claim 1 and further recites further comprising determining to perform photographing a plurality of times as the photographing mode.
The step of determining to perform photographing a plurality of times as the photographing mode can be considered as mental process because a person can mentally determine what will be a photographing mode including how many times to perform photographing.
Claim 10 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 10 is not eligible subject matter under 35 USC 101.
Regarding claim 11, it depends from claim 1 and further recites further comprising determining to change a shutter speed as the photographing mode.
The step of determining to change a shutter speed as the photographing mode can be considered as mental process because a person can mentally determine what will be a photographing mode including changing the shutter speed.
Claim 11 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 11 is not eligible subject matter under 35 USC 101.
Regarding claim 12, it depends from claim 1 and further recites further comprising determining to change a posture of a camera included in the mobile body as the photographing mode.
The step of determining to change a shutter speed as the photographing mode can be considered as mental process because a person can mentally determine what will be a photographing mode including changing the posture of a camera.
Claim 12 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 12 is not eligible subject matter under 35 USC 101.
Regarding claim 17, it depends from claim 1 and further recites further comprising determining the photographing mode of the rephotographing based on a remaining battery life of the mobile body when detecting the defective region.
The step of determining the photographing mode of the rephotographing based on a remaining battery life of the mobile body when detecting the defective region can be considered as mental process because a person can check the battery life of the mobile body when detecting the defective region and mentally determine what will be a photographing mode accordingly.
Claim 17 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 17 is not eligible subject matter under 35 USC 101.
Regarding claim 18, it depends from claim 17 and further recites further comprising determining to change the movement route or change a posture of a camera included in the mobile body based on the remaining battery life.
The step of determining to change the movement route or change a posture of a camera included in the mobile body based on the remaining battery life can be considered as mental process because a person can check the remaining battery life and mentally determine to change the movement route or change a posture of a camera included in the mobile body.
Claim 18 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 18 is not eligible subject matter under 35 USC 101.
Regarding claim 19, it recites a mobile body comprising processing circuitry with similar functions discussed above with respect to claim 1. The mobile body and processing circuitry can be interpreted as additional elements but they are generic computer components.
Claim 19 does not recite any additional elements that can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 19 is not eligible subject matter under 35 USC 101.
Regarding claim 20, it has been rejected under 35 USC 101 as program per se and recites similar limitations discussed above with respect to claim 1. Claim 20 does not recite any additional elements that can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 20 is not eligible subject matter under 35 USC 101.
Regarding claim 21, it depends from claim 1 and further recites the detecting includes scanning a surface of the three-dimensional point cloud data and detecting, as the defective region, a region whose surroundings are surrounded by point data and for which the point data is not created.
The step of scanning a surface of the three-dimensional point cloud data and detecting, as the defective region, a region whose surroundings are surrounded by point data and for which the point data is not created can be considered as mental process because a person can look at the surface of the three-dimensional point cloud data and mentally detect a region as the defective region by identifying which region’s surroundings are surrounded by point data and for which the point data is not created.
Claim 21 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 21 is not eligible subject matter under 35 USC 101.
Regarding claim 22, it depends from claim 1 and further recites the detecting includes detecting the defective region according to whether or not there is a texture, at a portion corresponding to a region for which the point data is not created, in the photographed image that is a creation source of the three-dimensional point cloud data.
The step of detecting the defective region according to whether or not there is a texture, at a portion corresponding to a region for which the point data is not created, in the photographed image that is a creation source of the three-dimensional point cloud data can be considered as mental process because a person can look at the three-dimensional point cloud data and the photographed image, then mentally detect whether or not there is a texture, at a portion corresponding to a region for which the point data is not created, in the photographed image, and therefore detecting the defective region.
Claim 22 does not recite any additional elements and thus no additional elements can integrate the abstract idea into practical application nor can it amount to significantly more to the abstract idea.
Therefore, claim 22 is not eligible subject matter under 35 USC 101.
Claims 7-9 depend from claim 6 and respectively further recites changing the movement route so that the mobile body can move accordingly regarding the defective region. These steps can be considered as additional elements that can improve the photographing results and therefore can integrate the abstract ideas into practical application. Accordingly, claims 7-9 are eligible under 35 USC 101.
Claim 13 depend from claim 12 and further recites adding on the movement route a photographing point for the rephotographing; and changing the posture of the camera at the added photographing point. The steps can be considered as additional elements that changing the camera posture that can perform photographing differently and improve the photographing results, and therefore can integrate the abstract ideas into practical application. Accordingly, claim 3 is eligible under 35 USC 101.
Claims 14-16 depends from claim 13 and respectively further recites other photographing mode. Because claim 13 recites additional elements that can integrate the abstract ideas into practical application, claims 14-16 also can integrate the abstract ideas into practical application. Accordingly, claims 14-16 are eligible under 35 USC 101.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 21 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 21 recites, inter alia, “the detecting includes scanning a surface of the three-dimensional point cloud data and detecting, as the defective region, a region whose surroundings are surrounded by point data and for which the point data is not created.”
However, a person skilled in the art will understand that a three-dimensional point cloud data comprises points, while mesh data comprises surfaces. It is not clear how a surface is defined in the three-dimensional point cloud data.
Therefore, claim 21 is indefinite.
For examination purposes, the “surface” recited in claim 21 has been interpreted as any region in the point cloud data.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 5-10, 12, 13, 16, and 19-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Publication No. 20190253635 A1 to Gu et al.
Regarding claim 1, Gu discloses A photographing instruction method comprising:
creating three-dimensional point cloud data of a photographing target based on a plurality of the photographed images shot at a plurality of photographing points by a mobile body that moves on a movement route (Gu, para. [0085], disclosing an UAV including imaging devices, para. [0105], disclosing UAV control unit that performs processing related to generation of a flight route,, para. [0114], disclosing controlling flight of the UAV according to a generated flight route, para. [0115], disclosing capturing images at each imaging position on the flight route, para. [0116], disclosing generating three-dimensional shape of an object (subject) based on images captured at different imaging positions by the imaging devices, para. [0117], disclosing generating three-dimensional shape including two overlapping images, para. [0118], disclosing detecting whether there is a loss area in the three-dimensional shape data, the loss area is an area in the three-dimensional space where point group data configuring the generated three-dimensional shape data is lower than a predetermined number or ratio, indicating the UAV can be a mobile body, and images captured by the imaging device on the UAV as photographed image shot at a plurality of photographing points by a mobile body (UAV) that moves on a generated flight route as a movement route, the point group data can correspond to three-dimensional point cloud data of the photographing target created based on the plurality of the photographed images shot at a plurality of photographing points on the flight route as the movement route);
detecting a defective region, of the photographing target based on a photographed image of the plurality of photographed images, for which point data is not created in the three-dimensional point clout data (Gu, para. [0115], disclosing imaging at each imaging position on the flight route to capture images, para. [0116], disclosing generating three-dimensional information indicating three-dimensional shape of an object from a plurality of images captured at different imaging positions, para. [0117], disclosing generating three-dimensional shape including two overlapping images, para. [0118], disclosing detecting whether there is a loss area in the three-dimensional shape data, the loss area is an area in the three-dimensional space where point group data configuring the generated three-dimensional shape data is lower than a predetermined number or ratio, indicating the loss area can correspond to a defective region, which is detected based on an image captured by the imaging device on the UAV and corresponding to a region for which the point data is created at a number lower than a predetermined number in a region of point group data corresponding to the three-dimensional point cloud data point data. Although Gu does not expressly disclose the point data is not created in the three-dimensional point cloud data in the detected region, the predetermined number can be one (1), and the number lower than one (1) will be zero (0), indicating the point data at zero (0) means the point data is not created. According to MPEP 2144.05, where the claimed ranges overlap or lie inside ranges disclosed by the prior art (here is Gu), a prima facie case of obviousness exists. Accordingly, before the invention was effectively filed, it is obvious that Gu teaches detecting a defective region, of the photographing target based on a photographed image of the plurality of photographed images, for which the point data is not created in the three-dimensional point cloud data. The suggestion/motivation would have been to supplement the loss area where the point group data is insufficient to improve the restoration accuracy of the three-dimensional shape data);
determining, as a state of the defective region a non-creation factor of the photographing target, that the point data is not created (Gu, para. [0163], disclosing detecting one loss area or a plurality of loss areas, the loss area may occur because information on pixel values at positions in the three-dimensional space is insufficient, and the loss area may occur because the image information is insufficient in one portion covered by the other portion, the reason the loss area may occur or the number of loss areas can correspond to a non-creation factor as a state of the defective region); and
determining a photographing mode of rephotographing of the defective region according to the non-creation factor when detecting the defective region (Gu, para. [0166], disclosing generating flight route with imaging position intervals for imaging the loss area, para. [0167], disclosing generating a plurality of flight routes for a plurality of loss areas, Gu, para. [0183], disclosing detecting the loss area LA in the generated three-dimensional shape data SD1, acquiring position information on the loss area, para. [0184], disclosing generating the flight route FPB for imaging the loss area in step S16, para. [0185], disclosing generating flight route FPC1 comprising the flight section fpa in which the imaging position interval is interval d12 and dense and flight section fpb generated in S16, the flight route FPC1 is a flight route for detailed imaging, indicating the detailed imaging can correspond to a photographing mode of rephotographing of the loss areas as the defective region, and the generating of flight routes can correspond to determining the photographing mode of the rephotographing according to the non-creation factor (one flight route or plurality of flight routes based on how many loss areas corresponding to the non-creation factor) when detecting the defective region);
and generating instruction information for instructing the rephotographing according to the determined photographing mode (Gu, para. [0185], disclosing the flight route FPC1 includes the flight section fpa corresponding to the original flight route FPA1, when approaching the loss area LA1 according to the route of the flight section fpa1 at the time of flying again after the schematic image pickup at S13, the unmanned aerial vehicle transits to the flight section fpb1 corresponding to the flight route FPB1 and descends at the imaging point of the loss area LA1, para. [0186], disclosing controlling the flight of the unmanned aerial vehicle according to the flight route FPC1, the imaging device are caused to capture images at each imaging position CP of the flight route FPC1, indicating the controlling the flight of the unmanned aerial vehicle according to the flight route FPC 1 and causing the imaging device to capture images at each imaging position of the flight route FPC1 can correspond to the instruction information generated for instructing the rephotographing according to the determined photographing mode).
Although Gu does not expressly disclose the point data is not created in the three-dimensional point cloud data in the detected region, the predetermined number can be one (1), and the number lower than one (1) will be zero (0), indicating the point data at zero (0) means the point data is not created. According to MPEP 2144.05, where the claimed ranges overlap or lie inside ranges disclosed by the prior art (here is Gu), a prima facie case of obviousness exists. Accordingly, before the invention was effectively filed, it is obvious that Gu teaches detecting a defective region, of the photographing target based on a photographed image of the plurality of photographed images, for which the point data is not created in the three-dimensional point cloud data. The suggestion/motivation would have been to supplement the loss area where the point group data is insufficient to improve the restoration accuracy of the three-dimensional shape data.
Regarding claim 3, Gu discloses the photographing instruction method according to claim 1, further comprising detecting a portion at which pixels do not match between the plurality of photographed images as the region for which the point data is not created (Gu, para. [0118], disclosing when performing imaging, there is one part of the object becomes the shadow of another part of the object and is not able to be captured, information on pixel values as data (an example of the point group data) for each position in the three-dimensional space within the imaging range is insufficient, the peripheral area around this position is the loss area, indicating when a portion of the object cannot be captured, the pixels in the plurality of images corresponding to that portion will not match between the images because of insufficient information, and the portion will be corresponding to a region for which the point data is not created).
Regarding claim 5, Gu discloses the photographing instruction method according to claim 3, further comprising determining the non-creation factor based on image processing of the photographed image in which the defective region has been detected (Gu, para. [0161],, disclosing shape data processing unit generates three-dimensional shape data based on a plurality of images, para. [0163], disclosing detecting a loss area in the generated three-dimensional shape data, indicating the generating three-dimensional shape data based on a plurality of images can correspond to image processing of the photographed image in which the loss area corresponding to the defective region has been detected, and detecting loss areas can correspond to determining the non-creation factor (how many loss areas) based on the three-dimensional shape data obtained by image processing on the plurality of images).
Regarding claim 6, Gu discloses the photographing instruction method according to claim 1, further comprising determining to change the movement route as the photographing mode (Gu, para. [0180], disclosing generating the flight route FPA (FPA1), para. [0181], disclosing capturing images at each imaging position of the flight route FPA1, para. [0183], disclosing detecting the loss area LA in the generated three-dimensional shape data SD1, acquiring position information on the loss area, para. [0184], disclosing generating the flight route FPB for imaging the loss area in step S16, para. [0185], disclosing generating flight route FPC1 comprising the flight section fpa in which the imaging position interval is interval d12 and dense and flight section fpb generated in S16, the flight route FPC1 is a flight route for detailed imaging, indicating generation of the flight route FPB and FPC1 can correspond to determining to change the flight route FPA1 as the photographing mode).
Regarding claim 7, Gu discloses the photographing instruction method according to claim 6, further comprising changing the movement route such that the mobile body goes around the defective region (Gu, para. [0168], disclosing the flight route FPB may be a flight route surrounding sideways of the loss area, indicating the flight route surrounding the sideways of the loss area can correspond to the movement route changed to allow the mobile body goes around the loss area as the defective region).
Regarding claim 8, Gu discloses the photographing instruction method according to claim 6, further comprising changing the movement route such that the mobile body approaches the defective region (Gu, Fig. 12, showing the flight route inserted between CPA and CPB where the mobile body approaches the loss area, para. [0172], disclosing inserting a flight section fpb between the specified imaging positions CPA and CPB in the flight section fpa, the inserted flight section FPB is approaching the loss area as the defective region; also, para. [0215], disclosing determining whether there is a loss area LA (LA3), if there is a loss area LA3,, the unmanned aerial vehicle leaves the flight route FPA3 and fly toward the loss area LA3, para. [0216], disclosing generating a flight route FPB (FPB3) for imaging the loss area LA3, indicating the unmanned aerial vehicle flies toward the loss area can correspond to changing the movement route such that the unmanned aerial vehicle as the mobile body approaches the loss area as the defective region).
Regarding claim 9, Gu discloses the photographing instruction method according to claim 6, further comprising changing the movement route such that the mobile body moves away from the defective region (Gu, para. [0215], disclosing determining whether there is a loss area LA (LA3), if there is a loss area LA3,, the unmanned aerial vehicle leaves the flight route FPA3 and fly toward the loss area LA3, para. [0216], disclosing generating a flight route FPB (FPB3) for imaging the loss area LA3, para. [0218], disclosing when the imaging at the imaging positions CP of the flight route FPB3 is completed and the flight of the flight route FPB3 is completed, the unmanned aerial vehicle returns to the flight according to the flight route FPA3, indicating returning the unmanned aerial vehicle to the flight route FPA3 can correspond to the movement route changes such that the unmanned aerial vehicle moves away from the loss area as the defective region because when imaging the loss area, the unmanned aerial vehicle moves toward the loss area and accordingly, when the imaging of the loss area is finished, the unmanned aerial vehicle moves away from the loss area).
Regarding claim 10, Gu discloses the photographing instruction method according to claim 1, further comprising determining to perform photographing a plurality of times as the photographing mode (Gu, para. [0185], disclosing generating the flight route FPC (FPC1) comprising the flight section fpa (fpa1) and flight section fpb (fpb1), the flight route FPC1 includes the flight section fpa (fpa1) corresponding to the original flight route FPA1, and when approaches the loss area according to the route of the flight section fpa1 at the time of flying again after the schematic image pickup, the unmanned aerial vehicle transits to the flight section fpb1 corresponding to the flight route FPB1 and descends at the imaging point of the loss area, para. [0186], disclosing controlling the flight of the unmanned aerial vehicle and the imaging device is caused to capture images at each imaging position of the flight route FPC1, indicating capturing images at each imaging position of the flight route FPC1 at the time of flying again can correspond to perform photographing a plurality of times as the photographing mode).
Regarding claim 12, Gu discloses the photographing instruction method according to claim 1, further comprising determining to change a posture of a camera included in the mobile body as the photographing mode (Gu, para. [0085], disclosing the unmanned aerial vehicle includes an imaging device, para. [0185], disclosing generating the flight route FPC (FPC1) comprising the flight section fpa (fpa1) and flight section fpb (fpb1), the flight route FPC1 includes the flight section fpa (fpa1) corresponding to the original flight route FPA1, and when approaches the loss area according to the route of the flight section fpa1 at the time of flying again after the schematic image pickup, the unmanned aerial vehicle transits to the flight section fpb1 corresponding to the flight route FPB1 and descends at the imaging point of the loss area, para. [0186], disclosing the imaging device are caused to capture images at each imaging position, indicating descending at the imaging point of the loss area can correspond to changing a posture of the imaging device as a camera included in the unmanned aerial vehicle as the mobile body as the photographing mode).
Regarding claim 13, Gu discloses the photographing instruction method according to claim 12, further comprising: adding on the movement route a photographing point for the rephotographing (Gu, para. [0172], disclosing inserting the flight section fpb corresponding to the flight route FPB between specified imaging positions CPA and CPB in the flight route FPC, para. [0184], disclosing generating the flight route FPB (FPB1) for imaging the loss area (S16), para. [0185], disclosing the flight route FPC (FPC1) comparing the flight section fpa(fpa1) and flight section fpb (fpb1) generated in S16, when approaches the loss area according to the route of the flight fpa1 at the time of flying again after the schematic image pickup at S13, the unmanned aerial vehicle transits to the flight section fpb1 and descends at the imaging point of the loss area, indicating the inserted imaging position along flight section fpb can correspond to a photographing point for rephotographing added on the movement route FPC); and changing the posture of the camera at the added photographing point (Gu, para. [0085], disclosing the unmanned aerial vehicle includes an imaging device, para. [0172], disclosing inserting the flight section fpb corresponding to the flight route FPB between specified imaging positions CPA and CPB in the flight route FPC, para. [0184], disclosing generating the flight route FPB (FPB1) for imaging the loss area (S16), para. [0185], disclosing the flight route FPC (FPC1) comparing the flight section fpa(fpa1) and flight section fpb (fpb1) generated in S16, when approaches the loss area according to the route of the flight fpa1 at the time of flying again after the schematic image pickup at S13, the unmanned aerial vehicle transits to the flight section fpb1 and descends at the imaging point of the loss area, indicating descending at the imaging point of the loss area can correspond to changing the posture of the imaging device as the camera at the added photographing point correspond to the flight section fpb).
Regarding claim 16, Gu discloses the photographing instruction method according to claim 13, further comprising determining to change a focal distance at the added photographing point as the photographing mode (Gu, para. [0165], disclosing acquiring an imaging distance and an imaging position interval related to the flight route FPB, and as the imaging distance becomes shorter, a larger image with the loss area LA as the subject can be acquired, para. [0166], disclosing deciding the imaging positions based on the imaging distance for imaging the loss area LA. Although Gu does not expressly disclose changing a focal distance at the added photographing point corresponding to flight route FPB as the photographing mode, Gu allows the imaging distance to become shorter to take larger image with the loss area while on flight route FPB, which can cause the focal distance become shorter. Before the invention was effectively filed, it would have been obvious for a person skilled in the art to change the focal distance corresponding to imaging distance, so that the shape data processing unit can improve the estimation accuracy of the three-dimensional shape, as suggested by Gu (see Gu, para. [0165])).
Regarding claim 19, it recite similar limitations of claim 1 but in a mobile body form. The rationale of claim 1 rejection is applied to reject claim 19. In addition, Gu discloses a mobile body (Gu, para. [0085]), and processing circuitry (Gu, FIG. 4, showing UAV CONTROL UNIT 110).
Regarding claim 20, it recites similar limitations of claim 1 but in a non-transitory computer-readable storage medium form. The rationale of claim 1 rejection is applied to reject claim 20. In addition, Gu discloses a non-transitory computer-readable storage medium (Gu, FIG. 4, showing MEMORY 160).
Regarding claim 21, Gu discloses the photographing instruction method according to claim 1, wherein the detecting includes scanning a surface of the three-dimensional point cloud data and detecting, as the defective region, a region whose surroundings are surrounded by point data and for which the point data is not created (Gu, para. [0115], disclosing imaging at each imaging position on the flight route to capture images, para. [0116], disclosing generating three-dimensional information indicating three-dimensional shape of an object from a plurality of images captured at different imaging positions, para. [0117], disclosing generating three-dimensional shape including two overlapping images, para. [0118], disclosing detecting whether there is a loss area in the three-dimensional shape data, the loss area is an area in the three-dimensional space where point group data configuring the generated three-dimensional shape data is lower than a predetermined number or ratio, indicating the loss area can correspond to a defective region, which is detected by scanning regions of the point group data as the three-dimensional point cloud data, and detect the loss area corresponding to a region for which the point data is created at a number lower than a predetermined number in a region of point group data and surrounding regions having the point data above the predetermined number. Although Gu does not expressly disclose the point data is not created in the three-dimensional point cloud data in the detected region, the predetermined number can be one (1), and the number lower than one (1) will be zero (0), indicating the point data at zero (0) means the point data is not created. According to MPEP 2144.05, where the claimed ranges overlap or lie inside ranges disclosed by the prior art (here is Gu), a prima facie case of obviousness exists. Accordingly, before the invention was effectively filed, it is obvious that Gu teaches detecting, as the defective region, a region whose surroundings are surrounded by point data and for which the point data is not created. The suggestion/motivation would have been to supplement the loss area where the point group data is insufficient to improve the restoration accuracy of the three-dimensional shape data).
Claim(s) 11, 15, 17, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu as applied to claim 1 above, and further in view of US Patent Publication No. 20210266464 A1 to Chida et al.
Regarding claim 11, Gu discloses the photographing instruction method according to claim 1. However, Gu does not expressly discloses further comprising determining to change a shutter speed as the photographing mode.
On the other hand, Chida discloses determining to change a shutter speed as the photographing mode (Chida, para. [0089], disclosing analyzing captured image to correct image-capturing conditions, para. [0090], disclosing corrected image-capturing conditions such as image-capturing speed, image-capturing route based on the analysis result, para. [0093], disclosing controlling driving of the mobile body on the basis of the corrected image-capturing conditions, para. [0223], disclosing analyzing captured images from the mobile body and calculating correction value of an image-capturing speed on the basis of the analysis result of the captured images, indicating the image-capturing speed can correspond to a shutter speed and is determined to be changed as the photographing mode).
Before the invention was effectively filed, it would have been obvious for a person skilled in the art to combine Gu and Chida. The suggestion/motivation would have been for enabling image capturing under optimum image-capturing conditions, as suggested by Chida (see Chida, para. [0001]).
Regarding claim 15, Gu discloses the photographing instruction method according to claim 13. However, Gu does not expressly discloses further comprising determining to change a shutter speed at the added photographing point as the photographing mode.
On the other hand, Chida discloses determining to change a shutter speed as the photographing mode (Chida, para. [0089], disclosing analyzing captured image to correct image-capturing conditions, para. [0090], disclosing corrected image-capturing conditions such as image-capturing speed, image-capturing route based on the analysis result, para. [0093], disclosing controlling driving of the mobile body on the basis of the corrected image-capturing conditions, para. [0223], disclosing analyzing captured images from the mobile body and calculating correction value of an image-capturing speed on the basis of the analysis result of the captured images, indicating the image-capturing speed can correspond to a shutter speed and is determined to be changed as the photographing mode). Because Gu discloses photographing at the added photographing point, combining Gu and Chida would allow determining to change a shutter speed at the added photographing point as the photographing mode.
Before the invention was effectively filed, it would have been obvious for a person skilled in the art to combine Gu and Chida. The suggestion/motivation would have been for enabling image capturing under optimum image-capturing conditions, as suggested by Chida (see Chida, para. [0001]).
Regarding claim 17, Gu discloses the photographing instruction method according to claim 11. However, Gu does not expressly discloses further comprising determining the photographing mode of the rephotographing based on a remaining battery life of the mobile body when detecting the defective region.
On the other hand, Chida discloses determining the photographing mode of the rephotographing based on a remaining battery life of the mobile body when detecting the defective region (Chida, para. [0090], disclosing calculating an image-capturing condition-correction value for correcting image-capturing conditions, para. [0091], disclosing calculating, on the basis of the remaining power information supplied from the power supplying section, a predicted remaining power amount as of a time when image capturing is fully finished, and correcting the image-capturing conditions on the basis of the calculated predicted remaining power amount, para. [0100], disclosing when an image having a low resolution, it cannot express fineness,, para. [0103], disclosing correcting the band of a captured image according to the fineness of a subject and controls is performed at the corrected image-capturing altitude to obtain image capturing with a suitable quality, indicating the image before correction can correspond to detected having the defective region and the correcting the image-capturing conditions based on the power information can correspond to the photographing mode of the rephotographing determined based on a remaining battery life of the mobile body).
Before the invention was effectively filed, it would have been obvious for a person skilled in the art to combine Gu and Chida. The suggestion/motivation would have been for enabling image capturing under optimum image-capturing conditions and allow image capturing to be finished while the remaining power is efficiently used, as suggested by Chida (see Chida, paras. [0001] and [0193]).
Regarding claim 18, Gu in view of Chida discloses the photographing instruction method according to claim 17, further comprising determining to change the movement route or change a posture of a camera included in the mobile body based on the remaining battery life (Chida, para. [0199], disclosing modifying the correction value of the image-capturing altitude on the basis of the predicted remaining power amount correction value of an image-capturing altitude is modified by, indicating the image-capturing altitude can correspond to a posture of a camera included in the mobile body being changed based on the remaining power amount as the remaining battery life).
Before the invention was effectively filed, it would have been obvious for a person skilled in the art to combine Gu and Chida. The suggestion/motivation would have been for enabling image capturing under optimum image-capturing conditions and allow image capturing to be finished while the remaining power is efficiently used, as suggested by Chida (see Chida, paras. [0001] and [0193]).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu as applied to claim 13 above, and further in view of US Patent Publication No. 20220132040 A1to McCrackin et al.
Regarding claim 14, Gu discloses the photographing instruction method according to claim 13. However, Gu does not expressly disclose further comprising determining to perform photographing a plurality of times at the added photographing point as the photographing mode.
On the other hand, McCrackin discloses determining to perform photographing a plurality of times at the added photographing point as the photographing mode (McCrackin, para. [0041], disclosing performing new inspection by changing at least one of a position and an orientation of a camera to a new position/a new orientation, activate a high-resolution camera to acquire a high-resolution image, activate one or more cameras to capture multiple images to generate 3D point cloud image, indicating activate the high-resolution camera and one or more cameras at the new position to capture multiple images can correspond determining to performing photographing a plurality of times at the added photographing point (the new position) as the photographing mode).
Before the invention was effectively filed, it would have been obvious for a person skilled in the art to combine Gu and McCrackin. The suggestion/motivation would have been to allow new inspection of the inspection region based on control signals generated according to an analytical model, as suggested by McCrackin (see McCrackin, para. [0003]).
Claim(s) 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu as applied to claim 1 above, and further in view of US Patent Publication No. 20130087059 A1to Baird et al.
Regarding claim 22, Gu discloses the photographing instruction method according to claim 1. However, Gu does not expressly disclose wherein the detecting includes detecting the defective region according to whether or not there is a texture, at a portion corresponding to a region for which the point data is not created, in the photographed image that is a creation source of the three-dimensional point cloud data.
On the other hand, Baird discloses wherein the detecting includes detecting the defective region according to whether or not there is a texture, at a portion corresponding to a defective region, in the image (Baird, para. [0028], disclosing detecting a defective at a location with a missing color in the image, a color can correspond to a type of texture). Because Gu discloses detecting the defective region corresponding to a region for which the point data is not created in the photographed image that is a creation source of the three-dimensional point cloud data, combining Gu and Baird could allow detecting the defective region according to whether or not there is a texture at the portion corresponding to a region for which the point data is not created in the photographed image that is a creation source of the three-dimensional point cloud data.
Before the invention was effectively filed, it would have been obvious for a person skilled in the art to combine Gu and Baird. The suggestion/motivation would have been to identify or categorize the type of defect, as suggested by Baird (see Baird, para. [0028]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAIXIA DU whose telephone number is (571)270-5646. The examiner can normally be reached Monday - Friday 8:00 am-4:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAIXIA DU/Primary Examiner, Art Unit 2611
Prosecution Insights