DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
The amendments to claims 1-4 and 6-9 are accepted and entered.
Claims 1-9 are pending regarding this application.
Response to Arguments
Applicant’s arguments, see Remarks, filed 10/31/2025, with respect to the claim objection applied to claim 4 have been fully considered and are persuasive. The Objection of claim 4 has been withdrawn.
Applicant's arguments, see Remarks, filed 10/31/2025, with respect to the 102 and 103 rejections of claims 1-9 have been fully considered but they are not persuasive. The originally applied prior art, Oami (JP WO 2014/010174 A1), still reads on the amended claim language. Applicant argues that, “in Oami, the update of the fixed point information means that a fixed point detected newly is added to the fixed point information stored in the fixed point information storing means 102. However, the update of the fixed point information does not mean that a first position of a fixed point is set to a second position which has been moved from the first position. … Accordingly, Applicant respectfully submits that Oami fails to disclose the above noted elements of claim 1: "update the reference image information after being generated, so that the position of the reference image information is set to the position of the corresponding part, when the position of the reference image is different from the position of the corresponding part." However, the examiner believes that Oami continues to teach newly amended claim 1. While Oami may not specifically teach that the reference part of the reference image is updated to reflect a changed angle of view of the same object with regards to an updated position of a fixed point, nowhere in the claim language does it recite that the updated reference part must correspond to the same object of the initial reference part. Therefore, the step of “updating the reference image information after being generated, so that the position of the reference image information is set to the position of the corresponding part , when the position of the reference image is different from the position of the corresponding part" can be broadly interpreted to mean converting a fixed point and feature points of an image to match a new fixed point its corresponding feature points. Please see the updated citations and rejections below regarding Oami’s teaching of claim 1.
Claim Objections
Claim 7 is objected to because of the following informalities:
Line 9 of claim 7 should be amended to read “update [[in]] the reference image”.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3, and 6-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Oami (JP WO 2014/010174 A1, see English translation for citations).
Regarding claim 1, Oami teaches the imaging status monitoring system (Oami para. [0051]) comprising:
at least one memory configured to store instructions (Oami “the program is stored in a storage unit (not shown) of the view angle variation determination unit 13” para. [0051]); and
at least one processor configured to execute the instructions (Oami “The angle-of-view variation detection unit 101, the feature amount extraction unit 103, and the feature amount tracking unit 104 are realized by a CPU of a computer that operates according to a program (view angle variation detection program)” para. [0051]) to:
generate reference image information about a reference image including a reference part, based on a first captured image (Oami, see para. [0011]-[0012] regarding captured images) of an imaging target imaged in a predetermined period (Oami “the angle-of-view variation detection device is a fixed point detection unit (for example, fixed point detection) that detects, as a fixed point, a feature point that remains in the same position for a predetermined time among the feature points extracted by the feature point extraction unit” para. [0095]; the predetermined time is interpreted as equivalent to the claimed predetermined period. Here, the fixed point information is interpreted as the reference part which is extracted from the video imaged by the video acquisition unit; additionally, Oami teaches a reference image including the fixed point and the feature point information in para. [0098]);
acquire a second captured image of the imaging target imaged after the predetermined period (Oami “the state determination unit 82 may determine that the angle of view of the imaging device has changed when the change in the angle of view variation parameter between frames of the video imaged by the imaging device has stabilized for a certain period” para. [0093]; the multiple frames being analyzed here implies the existence of at least a second image (frame); see para. [0073] regarding analyzing frames outside of a predetermined period); and
determine an imaging status of whether or not the imaging target has been imaged, based on a relationship between a position of the reference part in the reference image and a position of a corresponding part corresponding to the reference part in the second captured image (Oami “when the feature point information extracted in the processing for the next frame is input from the feature amount extraction unit 103, the fixed point detection unit 204 compares the input feature point information with the stored fixed point candidate, It is determined whether or not the extracted feature points are continuously present at the same position” para. [0069]),
and update the reference image information after being generated, so that the position of the reference image information is set to the position of the corresponding part, when the position of the reference image is different from the position of the corresponding part (Oami “the angle-of-view variation detection device converts a feature point extraction unit (for example, feature amount extraction unit 103) that extracts a feature point from a video captured by the imaging device, and converts the position of the feature point into a corresponding fixed point position” para. [0092]; Oami then teaches setting the position only when the fixed point if the angle of view of the video acquisition unit has changed in para. [0082-0083]; here, once the fixed point is changed to match the fixed point in the second image (updated), the feature points (of the reference part as cited above) are changed based on the fixed point (corresponding part) established in the “second captured image”).
Regarding claim 3, Oami teaches the imaging status monitoring system according to claim 1,
the at least one processor is further configured to execute the instructions to perform processing relating to an output so that an output with respect to the imaging status is performed to a user based on a difference between the position of the reference part and the position of the corresponding part (Oami “when the number of matching fixed points does not exist more than a certain number, the angle-of-view variation detecting unit 101 determines that the angle of view has changed, and displays information indicating that the angle of view has changed and information indicating the content of the change” para. [0038]; here, a display of information is shown based on a result of that the angle of view has changed; information indicating the content of the change and the mismatching number of fixed points is interpreted as equivalent to the difference between the position of the reference part and the position of the corresponding part).
Regarding claim 6, Oami teaches the imaging status monitoring system according to claim 1,
wherein the at least one processor is further configured to execute the instructions to generate, based on a plurality of captured images of the imaging target imaged in the predetermined period (Oami teaches that “the video imaged by the video acquisition means 10 and means that the position and the feature amount are stable for a certain time or more” para. [0033]; here, the first captured image is interpreted as equivalent to the video imaged by the video acquisition means), the reference image information about the reference image with setting a part with no change of the plurality of captured images as the reference part (Oami teaches that “the fixed point information accumulating unit 102 stores fixed point information including the position of the fixed point and the feature amount indicating the feature of the fixed point. Here, the fixed point is a point specified from the video imaged by the video acquisition means 10 and means that the position and the feature amount are stable for a certain time or more” para. [0033]).
Regarding claim 7, Oami teaches the imaging status monitoring system according to claim 1, wherein,
the reference part includes at least one feature point of the imaging target as a fixed point (Oami “the fixed point detection unit 204 detects, as fixed points, feature points that are continuously located at the same position for a predetermined time among the feature points extracted by the feature amount extraction unit 103” para. [0068]; as shown in claim 1, the reference part is interpreted as equivalent to the fixed point. As a result, it is inherent that the reference part includes a fixed point),
the reference image information includes fixed point information about the fixed point included in the reference part (Oami “the fixed point information accumulating unit 302 stores, as fixed point information, information indicating a situation in which the fixed point is used in addition to the feature amount indicating the position of the fixed point and the feature of the fixed point” para. [0080]), and
the at least one processor is further configured to execute the instructions to:
determine the imaging status based on a relationship between a position of the fixed point in the reference image and the position of the corresponding part (Oami “it is determined whether or not the extracted feature points are continuously present at the same position. Note that whether or not feature points are continuously present at the same position can be determined based on a change in the feature amount of a point corresponding to the feature point” para. [0069]), and
update the reference image information after being generated, so that the position of the reference image information is set to the position of the corresponding part in the second captured image, when the position of the fixed point in the reference image is different from the position of the corresponding part (Oami “the angle-of-view variation detection device converts a feature point extraction unit (for example, feature amount extraction unit 103) that extracts a feature point from a video captured by the imaging device, and converts the position of the feature point into a corresponding fixed point position” para. [0092]; here, the conversion of the position of the feature point into a corresponding fixed point position is equivalent to the setting of the position of the reference part to the position of the corresponding part in the second captured image, and the adjustment of the reference image information is interpreted as equivalent to the conversion of the feature points as shown in para. [0083]; “the state determination unit 82 may determine that the angle of view of the imaging device has changed when the change in the angle of view variation parameter between frames of the video imaged by the imaging device has stabilized for a certain period” para. [0092-0093]).
Regarding claim 8, Oami teaches an imaging status monitoring method comprising:
generating reference image information about a reference image including a reference part, on the basis of a first captured image of an imaging target imaged in a predetermined period (Oami “the angle-of-view variation detection device is a fixed point detection unit (for example, fixed point detection) that detects, as a fixed point, a feature point that remains in the same position for a predetermined time among the feature points extracted by the feature point extraction unit” para. [0095]; the fixed point here is interpreted as the reference part which is extracted from the video imaged by the video acquisition unit);
acquiring a second captured image of the imaging target imaged after the predetermined period (Oami “the state determination unit 82 may determine that the angle of view of the imaging device has changed when the change in the angle of view variation parameter between frames of the video imaged by the imaging device has stabilized for a certain period” para. [0093]; the multiple frames being analyzed here implies the existence of at least a second image (frame)); and
determining an imaging status of whether or not the imaging target has been imaged, on the basis of a relationship between a position of the reference part in the reference image and a position of a corresponding part corresponding to the reference part in the second captured image (Oami “when the feature point information extracted in the processing for the next frame is input from the feature amount extraction unit 103, the fixed point detection unit 204 compares the input feature point information with the stored fixed point candidate, It is determined whether or not the extracted feature points are continuously present at the same position” para. [0069]), and
setting the position of the reference part included in the reference image information to the position of the corresponding part, when the position of the reference image is different from the position of the corresponding part (Oami “the angle-of-view variation detection device converts a feature point extraction unit (for example, feature amount extraction unit 103) that extracts a feature point from a video captured by the imaging device, and converts the position of the feature point into a corresponding fixed point position” para. [0092]; “the angle-of-view fluctuation detecting unit 101 selects fixed point information to be used based on the situation identification information, and whether or not the angle of view of the video acquisition unit 10 has changed from the input video based on the selected fixed point information. Is detected. The feature amount tracking unit 104 selects fixed point information to be used based on the situation identification information, and calculates a view angle variation parameter for converting the position of each feature point to the position of the selected fixed point” para. [0082-0083]).
Regarding claim 9, Oami teaches a non-transitory recording medium (storage unit, see paragraph 0051) on which a computer program that allows a computer execute an imaging status monitoring method is recorded (Oami “hardware”, “a CPU of a computer that operates according to a program” para. [0051]), the imaging status monitoring method comprising:
generating reference image information about a reference image including a reference part, based on a first captured image (Oami, see para. [0011]-[0012] regarding captured images) of an imaging target imaged in a predetermined period (Oami “the angle-of-view variation detection device is a fixed point detection unit (for example, fixed point detection) that detects, as a fixed point, a feature point that remains in the same position for a predetermined time among the feature points extracted by the feature point extraction unit” para. [0095]; the predetermined time is interpreted as equivalent to the claimed predetermined period. Here, the fixed point information is interpreted as the reference part which is extracted from the video imaged by the video acquisition unit; additionally, Oami teaches a reference image including the fixed point and the feature point information in para. [0098]);
acquiring a second captured image of the imaging target imaged after the predetermined period (Oami “the state determination unit 82 may determine that the angle of view of the imaging device has changed when the change in the angle of view variation parameter between frames of the video imaged by the imaging device has stabilized for a certain period” para. [0093]; the multiple frames being analyzed here implies the existence of at least a second image (frame); see para. [0073] regarding analyzing frames outside of a predetermined period); and
determining an imaging status of whether or not the imaging target has been imaged, based on a relationship between a position of the reference part in the reference image and a position of a corresponding part corresponding to the reference part in the second captured image (Oami “when the feature point information extracted in the processing for the next frame is input from the feature amount extraction unit 103, the fixed point detection unit 204 compares the input feature point information with the stored fixed point candidate, It is determined whether or not the extracted feature points are continuously present at the same position” para. [0069]),
and updating the reference image information after being generated, so that the position of the reference image information is set to the position of the corresponding part, when the position of the reference image is different from the position of the corresponding part (Oami “the angle-of-view variation detection device converts a feature point extraction unit (for example, feature amount extraction unit 103) that extracts a feature point from a video captured by the imaging device, and converts the position of the feature point into a corresponding fixed point position” para. [0092]; Oami then teaches setting the position only when the fixed point if the angle of view of the video acquisition unit has changed in para. [0082-0083]; here, once the fixed point is changed to match the fixed point in the second image (updated), the feature points (of the reference part as cited above) are changed based on the fixed point (corresponding part) established in the “second captured image”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Oami (JP WO 2014/010174 A1, see English translation for citations) in view of Yonezawa (JP 61166940 B2, see English translation for citations).
Regarding claim 2, Oami teaches the imaging status monitoring system according to claim 1, wherein:
the reference image information includes information about an imaging environment (Oami “the fixed point information storage unit 102 stores fixed point information including the position of the fixed point and a feature amount indicating the feature of the fixed point. Here, the fixed point is a point specified from the video imaged by the video acquisition means 10” para. [0033]);
the at least one processor is further configured to execute the instructions (Oami “The angle-of-view variation detection unit 101, the feature amount extraction unit 103, and the feature amount tracking unit 104 are realized by a CPU of a computer that operates according to a program (view angle variation detection program” para. [0051]) to determine the imaging status based on a predetermined threshold and a difference between the position of the reference part and the position of the corresponding part (Oami “the feature point extraction unit calculates a feature amount of the feature point extracted from the video imaged by the imaging apparatus, and the parameter calculation unit determines that the distance between the feature point feature amount and the fixed point feature amount is a predetermined threshold value” para. [0094]).
Oami fails to teach the predetermined threshold being set based on the imaging environment.
However, Yonezawa teaches the predetermined threshold being set based on the imaging environment (Yonezawa “the processing unit S acquires the position indicated by the imaging position data acquired by the process of step S10 and the imaging position data acquired by the process of step S10 in the immediately preceding imaging position data process (hereinafter, the imaging position data). Is referred to as “immediate imaging position data”), and a distance (on the imaging position data) between them is calculated (step S11)” para. [0041]; “the processing unit S determines whether the distance calculated by the processing of step S11 is within a threshold distance set in advance corresponding to the accuracy of the current position data output from the sensor unit 6” para. [0041]; here, the threshold is determined based on the accuracy of the current position data output; see also para. [0008]).
Oami and Yonezawa are both considered to be analogous to the claimed invention because they are in the same field of tracking fixed points. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Oami to incorporate the teachings of Yonezawa and “the predetermined threshold being set based on the imaging environment”. The motivation for doing so would have been to determine the accuracy of current position data output from the sensor unit, as suggested by Yonezawa in para. [0041]. Therefore, it would have been obvious to combine Oami with Yonezawa to obtain the invention specified in claim 2.
Claims 4 is rejected under 35 U.S.C. 103 as being unpatentable over Oami (JP WO 2014/010174 A1, see English translation for citations) in view of Furihata (U.S. Publication No. 2017/0032530 A1).
Regarding claim 4, Oami teaches the imaging status monitoring system according to claim 1.
While Oami teaches setting a reference part which corresponds to information from the imaging target (See claim 1), Oami fails to teach including a shape information about a shape of the imaging target, wherein the at least one processor is further configured to execute the instructions to set a part corresponding to the shape information as the reference part, in the first captured image.
However, Furihata teaches including a shape information about a shape of the imaging target (Furihata “the reference features on an image of the observed measurement target object are calculated to measure shape information on the measurement target object onto which the reference features are projected” para. [0024]),
wherein the at least one processor is further configured to execute the instructions to set a part corresponding to the shape information as the reference part, in the first captured image (Furihata “for the candidates for the shape information set on the image parts on the image 700, the candidate evaluation unit 250 evaluates the degrees of correspondence between image areas including the image parts and areas including the corresponding parts on the illumination pattern 100 equivalent to the candidates for the shape information, based on the correlation between the areas” para. [0036]; “when the shape information is estimated with only the position information as a variable, the normal may be assumed to be opposite to the line-of-sight direction of the imaging apparatus 400 and be given as a fixed value” para. [0075]).
Oami and Furihata are both considered to be analogous to the claimed invention because they are in the same field of identifying corresponding information across multiple images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Oami to incorporate the teachings of Furihata and “includ[e] a shape information about a shape of the imaging target, wherein the at least one processor is further configured to execute the instructions to set a part corresponding to the shape information as the reference part, in the first captured image”. The motivation for doing so would have been that “the normal information is used for calculation to allow the shape of the measurement target object to be measured more accurately”, as suggested by Furihata in para. [0073]. Therefore, it would have been obvious to combine Oami with Furihata to obtain the invention specified in claim 4.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Oami (JP WO 2014/010174 A1, see English translation for citations) in view of Ryuzaki (JP 2018-19410 A, see English translation for citations).
Regarding claim 5, Oami teaches imaging status monitoring system according to claim 1,
Oami fails to teach wherein the at least one processor is further configured to execute the instructions to allow an imaging apparatus to reimage the imaging target and reacquires the second captured image, when the corresponding part is not extracted from the second captured image.
However, Ryuzaki teaches wherein the at least one processor is further configured to execute the instructions to allow an imaging apparatus to reimage the imaging target and reacquires the second captured image, when the corresponding part is not extracted from the second captured image (Ryuzaki “the manual search mode is a mode in which the photographer re- captures the subject by operating the FA zoom operation switch when the subject is out of the frame. The automatic tracking mode is a mode in which the camera automatically detects the subject and supports the angle adjustment. The automatic search mode is a mode for recapturing the subject automatically by detecting the movement of the camera” para. [0036] “when a desired subject is found in the zoom-out state (subject search state) as shown in FIG. 3B, the subject is photographed so as to fit inside the FA zoom frame 300” para. [0041]).
Oami and Ryuzaki are both considered to be analogous to the claimed invention because they are in the same field of tracking fixed points. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Oami to incorporate the teachings of Ryuzaki and include “wherein the at least one processor is further configured to execute the instructions to allow an imaging apparatus to reimage the imaging target and reacquires the second captured image, when the corresponding part is not extracted from the second captured image”. The motivation for doing so would have been to obtain the optimal framing state, as suggested by Ryuzaki in para. [0041]. Therefore, it would have been obvious to combine Oami with Ryuzaki to obtain the invention specified in claim 5.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can
normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/
Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights