DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claim 12 is objected to because of the following informalities: the limitation “wherein the first polyline data is included in polyline data included in the first polyline data group” appears to contain a clerical error. For the purpose of further examination, the limitation has been interpreted as “wherein the first polyline data is included in the first polyline data group.” Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 16 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “similar” in claim 16 is a relative and/or subjective term which renders the claim indefinite. The term is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Imaging conditions that are similar in a specific situation may be considered to be dissimilar in other applications/scenarios (e.g., brightness level within 0.01% vs. 99%, identical vs. different camera/lens models, etc.), and the degree to which the similarity is determined is not disclosed.
A claim that requires the exercise of subjective judgment without restriction renders the claim indefinite. In re Musgrave, 431 F.2d 882, 893, 167 USPQ 280, 289 (CCPA 1970). Claim scope cannot depend solely on the unrestrained, subjective opinion of a particular individual purported to be practicing the invention. Datamize LLC v. Plumtree Software, Inc., 417 F.3d 1342, 1350, 75 USPQ2d 1801, 1807 (Fed. Cir. 2005)); see also Interval Licensing LLC v. AOL, Inc., 766 F.3d 1364, 1373, 112 USPQ2d 1188 (Fed. Cir. 2014).
For the purpose of further examination, the limitation has been interpreted as:
“whether or not imaging conditions … match” and
“… determined to have matching imaging conditions to the first image …”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-7, 9-10, and 17-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koichi Magai (US 20210035327 A1), in view of Taraben et al (“Automated linking of 3D inspection data for damage analysis,” Bridge Maintenance, Safety, Management, Life-Cycle Sustainability and Innovations Yokota & Frangopol (eds) © 2021 Taylor & Francis Group, London, ISBN978-0-367-23278-8), hereinafter referred to as Magai and Taraben, respectively.
Regarding claim 1, Magai teaches an information processing apparatus, comprising:
at least one processor (Magai ¶¶0135: “a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiments and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiments. The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU))”); and
a memory configured to combine with the at least one processor, and when executed by the processor, the memory includes a command for executing operations as (Magai ¶¶0135 discussed above):
an obtaining unit configured to obtain first position information indicating a position of a target detected in a first image captured of a target object and a second position information indicating a position of the target detected in a second image captured of the target object at a different time from the first image (Magai ¶¶0003: “a first estimating unit configured to perform a first estimation process to estimate the position of the target object in at least one of the plurality of images; a second estimating unit configured to perform a second estimating process to estimate the position of the target object in at least one of the plurality of images … wherein the second image is an image captured at a different time from the first image”; Magai Fig. 2: S203, S206; Magai ¶¶0033: “in Step S201, the image acquiring unit 101 acquires images to be processed in time series order”);
an estimating unit configured to estimate a position shift amount of the target on a basis of the first position information and the second position information (Magai ¶¶0027: “An image processing apparatus according to an embodiment detects the concerned object across the images of multiple frames to track the concerned object … the tracking position of the object to be tracked may be slightly shifted generally due to the influence of a peripheral similar pattern including noise and background or an adjacent object … Here, the passing line is set in the horizontal direction in a central portion of the screen for description … since the shift is corrected on the subsequent frame … it is possible to suppress the shift of the tracking position of an object in an image to suppress an occurrence of the error count in the counting of the number of human bodies passing through the certain line”); and
a correcting unit configured to correct at least one of the first position information and the second position information on a basis of the position shift amount (Magai ¶¶0003: “a correcting unit configured to, based on the position of the target object estimated by the first estimation process on a first image of the plurality of images, correct the position of the target object estimated by the second estimation process on a second image of the plurality of the images”; Magai Fig. 2: S300; Magai ¶¶0029: “a function to correct the shift of the tracking position of the concerned object using past locus information that is recorded”; Magai ¶¶0041: “After Step S206, the process goes to Step S300 to perform correction of the shift of the tracking position. The correction of the shift of the tracking position in Step S300 is performed by the locus analyzing unit 103, the locus smoothing unit 104, and the tracking position correcting unit 105”),
wherein the first position information includes information of each locus point of the target in the first image, and the second position information includes information of each locus point of the target in the second image (Magai ¶¶0029 & ¶¶0041 discussed above; Magai Fig. 4: 401-409; Magai ¶¶0039: “tracking position on the concerned frame image (the current frame image) using an average velocity vector calculated from the locus to the preceding frame in time series order”).
However, Magai does not appear to explicitly teach that the position information includes information of each vertex of the target in the first image, and the second position information includes information of each vertex of the target in the second image.
Pertaining to the same field of endeavor, Taraben teaches obtaining information of each vertex of the target in the first image, and the second position information includes information of each vertex of the target in the second image (Taraben Fig. 1: “a nested polyline (red) and its vertices (black dots)”; Taraben pg. 3716: “a vertex-based distance check for fine registration and an optional comparison for linked cracks … the distance between each vertex of the individual crack geometry and the filtered building element geometries (in the example E4 and E5) are checked against the defined threshold dmax”; Taraben Figs. 6-7).
Magai and Taraben are considered to be analogous art because they are directed to image processing for detecting and tracking objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the image processing apparatus and method for tracking objects by estimating the positions (as taught by Magai) to use a vertex-based distance check (as taught by Taraben) because the combination makes the matching between the compared objects more precise (Taraben pg. 3716 right column).
Regarding claim 2, Magai, in view of Taraben, teaches the information processing apparatus according to claim 1, wherein
the obtaining unit obtains, as the first position information, a first polyline data group including one or more pieces of polyline data in which a shape of the target in the first image is indicated by information of each vertex and, as the second position information, a second polyline data group including one or more pieces of polyline data in which a shape of the target in the second image is indicated by the information of each vertex (Magai Fig. 4 & Taraben Figs. 1, 6-7 discussed above; further see Magai Figs. 5-10; Magai ¶¶0038: “geometric pattern matching and color pattern matching are used in the pattern matching process”).
Regarding claim 3, Magai, in view of Taraben, teaches the information processing apparatus according to claim 2, further comprising:
a first generating unit configured to, on a basis of the position shift amount, generate a function for transforming coordinates in a coordinate system of first polyline data included in the first polyline data group to coordinates in a coordinate system of second polyline data included in the second polyline data group (Magai Figs. 4-10, ¶¶0029 discussed above; Magai ¶¶0099: “the tracking unit 1302 determines whether the tracking position shift correcting function is set to ‘enabled’ by the user. If the tracking unit 1302 determines that the tracking position shift correcting function is set to ‘enabled’ by the user (YES in Step S1402), the process goes to Step S300”),
wherein the correcting unit corrects coordinates indicated by the first polyline data on a basis of the function (Magai Figs. 4-10, ¶¶0029 & ¶¶0099 discussed above).
Regarding claim 4, Magai, in view of Taraben, teaches the information processing apparatus according to claim 3, wherein
the estimating unit estimates a similarity between polyline data included in the first polyline data group and polyline data included in the second polyline data group as the position shift amount (Taraben Figs. 1, 6-7 discussed above; Magai Fig. 12 & ¶¶0087: “the tracking position correcting unit 105 calculates the degree of similarity between the tracking area i and its peripheral area”), and
the first generating unit generates the function on a basis of the similarity (Magai Fig. 12: S1201, S307, S309).
Regarding claim 5, Magai, in view of Taraben, teaches the information processing apparatus according to claim 4, further comprising:
a second generating unit configured to generate a target object image corresponding to the first polyline data on a basis of the first polyline data included in the first polyline data group and generating a template image corresponding to the second polyline data on a basis of the second polyline data included in the second polyline data group, wherein the estimating unit, via template matching using the template image and the target object image, estimates a similarity of polyline data for each position of the template image (Magai Fig. 12 & ¶¶0087 discussed above; Magai ¶¶0002: “perform tracking using template matching considering how templates are overlapped with each other to estimate a search position on the subsequent frame image from a motion vector”; Taraben pg. 3716 left column: “(1) the linking of damages to building model and (2) the linking and comparison of damages to previous recordings”; Taraben Fig. 4: building model; Taraben pg. 3716 right column: “the correspondence of identified cracks from a data set to a previous inspection must be determined”; Taraben Fig. 6: “Steps for linking of crack geometries from different states t1 and t2” – Green/t2 serves as a template).
Regarding claim 6, Magai, in view of Taraben, teaches the information processing apparatus according to claim 5, wherein the second generating unit generates a bounding box region of the first polyline data as the target object image (Taraben Fig. 4: “Bounding box filter”; Taraben pg. 3716 left column: “axis aligned bounding boxes around all single elements of both models are crated and checked on intersections”).
Regarding claim 7, Magai, in view of Taraben, teaches the information processing apparatus according to claim 5, wherein the second generating unit generates a bounding box region of the second polyline data as the template image (Taraben Fig. 4 & pg. 3716 left column discussed above).
Regarding claim 9, Magai, in view of Taraben, teaches the information processing apparatus according to claim 5, wherein the second generating unit generates, as the template image, a partial region of a divided region including the first polyline data (Magai Figs. 5-10: see first and second areas; Taraben Fig. 5: E4, E5).
Regarding claim 10, Magai, in view of Taraben, teaches the information processing apparatus according to claim 5, wherein the second generating unit generates, as the template image, a bounding box region of third polyline data including the third polyline data obtained by dividing the first polyline data (Taraben Fig. 5: C1-C4).
Regarding claim 17, Magai, in view of Taraben, teaches the information processing apparatus according to claim 5, wherein a number of pieces of polyline data included in the first polyline data group is less than a number of pieces of polyline data included in the second polyline data group (Magai Fig. 11: S1101; Taraben pg. 3718 right column: “general information about the damage data set, such as the timestamp of the first occurrence of a crack or the development of the number of cracks over the time, can be derived from the resulting link set”).
Regarding claim 18, Magai teaches the information processing apparatus according to claim 3, further comprising:
a second obtaining unit configured to obtain input of an operator; a setting unit configured to set a parameter to be used in correction processing using the function on a basis of an input obtained by the second obtaining unit; and a presenting unit configured to present to the operator a coordinates correction result from the correcting unit (Magai ¶¶0076: “the output from the output unit 106 is changed so that the user is capable of knowing the area where the shift of the tracking position occurs to present the change to the user”; Magai ¶¶0097: “the setting unit 1301 sets a tracking position shift correcting function to ‘enabled’ or ‘disabled’ depending on a user's operation … Step S1401 is performed by the setting unit 1301 only before the first image acquisition and when the user performs a specific operation with a user interface (not illustrated). The operation with the user interface is, for example, depression of a setting button on the screen or a certain key operation”).
Regarding claim 19, Magai teaches the information processing apparatus according to claim 1, further comprising:
a displaying unit configured to display polyline data in which a shape of the target is indicated by information of each vertex on a display apparatus on the basis of at least one of the first position information and the second position information corrected by the correcting unit (Magai ¶¶0076: “when the tracking frame of the human body is output on the screen of a display device (not illustrated), the output unit 106 may change the color of the tracking frame to a certain color or may adjust the width of the frame line for the area where the shift of the tracking position occurs to present the occurrence of the shift of the tracking position to the user. Alternatively, if the shift of the tracking position occurs, the output unit 106 may continue displaying of the change of the output for a predetermined time. In this case, the output unit 106 may stop the presentation after a predetermined time elapsed”).
Regarding claim 20, Magai, in view of Taraben, teaches the information processing apparatus according to claim 1, wherein the target object is a construction, and the target is deformation of the construction (Taraben Abstract: “The inspection of existing bridge structures”; Taraben Figs. 1-2, 8-10; Taraben pg. 3715 right column: “structural deformations and inaccuracies”).
Regarding claim 21, Magai teaches that the information processing apparatus performs a method comprising the processes described in claim 1 (Magai ¶¶0001: “The present disclosure relates to an image processing apparatus, a method of tracking a target object, and a storage medium”). Therefore, claim 21 is rejected using the same rationale as applied to claim 1 discussed above.
Regarding claim 22, Magai teaches a non-transitory computer readable storage medium storing a program that, when executed by a computer, causes the computer to perform an information processing method, the information processing method comprising the processes described in claim 1 (Magai ¶¶0001discussed above; Magai ¶¶0135: “realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiments”). Therefore, claim 22 is rejected using the same rationale as applied to claim 1 discussed above.
Allowable Subject Matter
Claims 8 and 11-15 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 16 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 8, the prior art of record teaches that it was known at the time the application was filed to use the information processing apparatus according to claim 7. However, the prior art, alone or in combination, does not appear to teach or suggest that the template image is an image including a gradient of pixel values in a line width direction centered on the first polyline data in the bounding box region (shown in Applicant’s Fig. 8C).
Regarding claim 11, the prior art of record teaches that it was known at the time the application was filed to use the information processing apparatus according to claim 5, but does not appear to explicitly teach or suggest using a calculating unit configured to calculate a vector between a position of the template image before the template matching and a position where the similarity is maximum, wherein the first generating unit generates the function on a basis of the vector calculated by the calculating unit from a plurality of pieces of polyline data included in the first polyline data group.
Claims 12-15 depend from claim 11 and therefore are objected to for the same reason as claim 11 discussed above.
Regarding claim 16, the prior art of record teaches that it was known at the time the application was filed to use the information processing apparatus according to claim 3, but does not appear to explicitly teach or suggest using a determining unit configured to determine whether or not imaging conditions of the first image and a third image are similar, wherein the first generating unit, for the third image determined to have similar imaging conditions to the first image, generates a function for transforming coordinates in a coordinate system of fourth polyline data included in a third polyline data group including one or more pieces of polyline data in which a shape of a target in the third image is indicated by information of each vertex to coordinates in a coordinate system of the second polyline data as a function identical to a function for transforming coordinates in a coordinate system of the first polyline data to coordinates in a coordinate system of the second polyline data.
Note that the examiner’s statement applies to the claims only as interpreted by the examiner due to indefinite limitations. Please refer to the 35 U.S.C. 112(b) rejections above. The statement may no longer apply if amended to change the scope of the claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOO J SHIN whose telephone number is (571)272-9753. The examiner can normally be reached M-F; 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Soo Shin/Primary Examiner, Art Unit 2667 571-272-9753
soo.shin@uspto.gov
Prosecution Insights