DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/15/2024 and 06/04/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11-19 are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (US Pub No. 20190158799 A1) in view of Lai et al. ("Fast and robust template matching with majority neighbour similarity and annulus projection transformation" Pattern recognition 98 (2020): 107029, as provided).
Regarding Claim 11,
Gao discloses A method for ascertaining whether a specified image detail of a first image occurs in a second image, the method comprising the following steps: ascertaining at least one image feature for a plurality of pixels within the specified image detail; (Gao, [0033], discloses aligning two images may involve matching specific features (or feature points) between the two images. First, within a first image, an image feature (e.g., a feature descriptor, a feature point or pixel, etc.) is identified. This feature may be in the contextual format of an edge, a corner, a blob (region of interest points), a ridge, etc. Second, within a second image, a group of candidate features are identified. Each candidate feature must be of the same format (edge, corner, blob, ridge, etc.) and value as the first image feature. Significantly, this group of second image candidate features may be distributed throughout the second image. Lastly, a brute force matching (e.g., using k nearest neighbor, i.e., kNN algorithm) is performed to match the group of second image candidate features to the first image feature. Distances between the first image feature and each of the second image candidate features are computed and sorted. A best-matched pair is readily identified. This process is repeated several times to generate a multitude of best-matched pairs between the first and second images. Information from these best-matched pairs is then used to further align the two images; group of candidate features are extracted pixels from captured image)
obtaining the second image; (Gao, [0033], discloses aligning two images may involve matching specific features (or feature points) between the two images. First, within a first image, an image feature (e.g., a feature descriptor, a feature point or pixel, etc.) is identified. This feature may be in the contextual format of an edge, a corner, a blob (region of interest points), a ridge, etc. Second, within a second image, a group of candidate features are identified. Each candidate feature must be of the same format (edge, corner, blob, ridge, etc.) and value as the first image feature. Significantly, this group of second image candidate features may be distributed throughout the second image. Lastly, a brute force matching (e.g., using k nearest neighbor, i.e., kNN algorithm) is performed to match the group of second image candidate features to the first image feature. Distances between the first image feature and each of the second image candidate features are computed and sorted. A best-matched pair is readily identified. This process is repeated several times to generate a multitude of best-matched pairs between the first and second images. Information from these best-matched pairs is then used to further align the two images; first and second image are obtained)
ascertaining at least one image feature for a plurality of pixels of the second image; (Gao, [0033], discloses aligning two images may involve matching specific features (or feature points) between the two images. First, within a first image, an image feature (e.g., a feature descriptor, a feature point or pixel, etc.) is identified. This feature may be in the contextual format of an edge, a corner, a blob (region of interest points), a ridge, etc. Second, within a second image, a group of candidate features are identified. Each candidate feature must be of the same format (edge, corner, blob, ridge, etc.) and value as the first image feature. Significantly, this group of second image candidate features may be distributed throughout the second image. Lastly, a brute force matching (e.g., using k nearest neighbor, i.e., kNN algorithm) is performed to match the group of second image candidate features to the first image feature. Distances between the first image feature and each of the second image candidate features are computed and sorted. A best-matched pair is readily identified. This process is repeated several times to generate a multitude of best-matched pairs between the first and second images. Information from these best-matched pairs is then used to further align the two images; features from second image including edge, corner, and blob of pixels are extracted)
comparing the image features of the second image with the image features of the first image to determine whether the two have an identical value, and outputting that the specified image detail is present in the second image based on at least one pair of identical image features is present, otherwise outputting that the specified image detail is not present in the second image; (Gao, [0033], discloses aligning two images may involve matching specific features (or feature points) between the two images. First, within a first image, an image feature (e.g., a feature descriptor, a feature point or pixel, etc.) is identified. This feature may be in the contextual format of an edge, a corner, a blob (region of interest points), a ridge, etc. Second, within a second image, a group of candidate features are identified. Each candidate feature must be of the same format (edge, corner, blob, ridge, etc.) and value as the first image feature. Significantly, this group of second image candidate features may be distributed throughout the second image. Lastly, a brute force matching (e.g., using k nearest neighbor, i.e., kNN algorithm) is performed to match the group of second image candidate features to the first image feature. Distances between the first image feature and each of the second image candidate features are computed and sorted. A best-matched pair is readily identified. This process is repeated several times to generate a multitude of best-matched pairs between the first and second images. Information from these best-matched pairs is then used to further align the two images; similar features pixels from first image is tracked in second image and compared for similarity)
Gao does not explicitly disclose otherwise outputting that the specified image detail is not present in the second image; wherein the image features are ascertained depending on pixel values of adjacent pixels, and in the second image, a rotationally symmetrical area is determined and the area is transformed into a quadrangular area, and the image features of the pixels of the second image are calculated from pixels within the transformed area.
Lai discloses otherwise outputting that the specified image detail is not present in the second image; wherein the image features are ascertained depending on pixel values of adjacent pixels, and in the second image, a rotationally symmetrical area is determined and the area is transformed into a quadrangular area, and the image features of the pixels of the second image are calculated from pixels within the transformed area. (Lai, 3.1, Definition of MNS, 3.2, MNS for template matching, Fig. 1-5, apply MNS to template matching, it needs to convert all template image patches to a 2D numbered-point set P = { p i } N i =1 , which we call this first procedure the numbered point generating. Then performing local matching using these template patches to a given scene image to find the corresponding candidate patches and also converting them to a corresponding 2D numbered-point set Q = _ q j i _ j=1 , ···,n i =1 , ···,N , which we call this second procedure the Candidates Finding. The Candidates Finding procedure utilizes the lo- cal features extracting from the template image patches to perform matching to the scene image to find the corresponding matched Candidates; intuition for MNS is shown in Fig. 2 . The numbers 0 to 9 in Fig. 2 (a) represent the template, and the corresponding tar- get in Fig. 2 (b) undergoes rotation transformation, occlusion and background clutter. For humans, obviously, the rectangle area in Fig. 2 (b) is the best matched region corresponding to the template Fig. 2 (a), and q 1 0 in Fig. 2 (b) is the best matched point selected among candidates q 0 0 and q 1 0 because the relative position of q 1 0 equals to p 0 . For computer, we utilize MNS to measure the similarity between p 0 and q 1 0 to make the decision; discloses matching candidate pixel features from first image to template images that are rotated or transformed to match the first image by rotation).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Gao in view of Lai having a method of tracking similar regions in first and second image captured by comparing the image region pixel values, with the teachings of Lai having method of adjusting shape of region similar to in the first image within the second image determining if the similar details that exist in first image exist in second image by such transformation in order to accurately recognize images in applications including pattern recognition in automotive industry.
Regarding Claim 12,
The combination of Gao and Lai further discloses wherein after the step of ascertaining the image features of the first image, a step of assigning takes place in which each image feature of the image features of the first image is assigned a relative position of the pixel for which the first image feature was ascertained with respect to a reference position of the specified image detail, the assigned relative position of the image feature of the first image being used and a pixel position of the pixel of the second image for which the identical image feature is present being stored so as to be displaced in relation to the relative position in the step of comparing when values of the image features of the first and second images are identical, and a pixel-by-pixel aggregation of the displaced, stored positions being performed, it being output that the specified image detail is present in the second image if a plurality of the aggregated positions is present at a substantially identical pixel position, otherwise it being output that the specified image detail is not present in the second image. (Lai, 3.1, Definition of MNS, 3.2, MNS for template matching, Fig. 1-5, apply MNS to template matching, it needs to convert all template image patches to a 2D numbered-point set P = { p i } N i =1 , which we call this first procedure the numbered point generating. Then performing local matching using these template patches to a given scene image to find the corresponding candidate patches and also converting them to a corresponding 2D numbered-point set Q = _ q j i _ j=1 , ···,n i =1 , ···,N , which we call this second procedure the Candidates Finding. The Candidates Finding procedure utilizes the lo- cal features extracting from the template image patches to perform matching to the scene image to find the corresponding matched Candidates; intuition for MNS is shown in Fig. 2 . The numbers 0 to 9 in Fig. 2 (a) represent the template, and the corresponding tar- get in Fig. 2 (b) undergoes rotation transformation, occlusion and background clutter. For humans, obviously, the rectangle area in Fig. 2 (b) is the best matched region corresponding to the template Fig. 2 (a), and q 1 0 in Fig. 2 (b) is the best matched point selected among candidates q 0 0 and q 1 0 because the relative position of q 1 0 equals to p 0 . For computer, we utilize MNS to measure the similarity between p 0 and q 1 0 to make the decision; discloses matching candidate pixel features from first image to template images that are rotated or transformed to match the first image by rotation). Additionally, the rational and motivation to combine the references Gao and Lai as applied in rejection of claim 1 apply to this claim.
Regarding Claim 13,
The combination of Gao and Lai further discloses wherein a plurality of different image details of the first image are specified, the step of comparing being carried out for each of the different image details, an overall confidence measure being ascertained by accumulation of confidences. (Lai, 5.2, performance evaluation, discloses photometric changes, motion blur, in/out-of-plane rota- tion and more. We generate four datasets by sampling {870, 830, 780, 750} image pairs with a corresponding constant frame dif- ference dF rame = { 10 , 25 , 50 , 100 } in each video. Each such pair consists of frames f and f + dF rame , where f was randomly se- lected. The annotated ground-truth bounding box in frame f is used to define the template, while frame f + dF rame is the target image. We measure the detection accuracy by using the intersec- tion over union (IoU) measure between the detection result and the ground-truth bounding boxes: Accuracy = | Rest ∩ Rt ruth | | Rest ∪ Rt ruth | ∗100% , where, | ·| calculates the pixel area of the region, Rtruth and Rest are the ground-truth and estimated bounding boxes respectively; accuracy of matching is determined by feature matching in first and second images and their confidence is determined according to the accuracy of percentage of matching accuracy). Additionally, the rational and motivation to combine the references Gao and Lai as applied in rejection of claim 1 apply to this claim.
Regarding Claim 14,
The combination of Gao and Lai further discloses wherein the plurality of the image details have a specified arrangement in relation to one another within the first image, it being ascertained in the step of comparing, in addition to the aggregated positions, what arrangement the aggregated positions have in relation to one another, and the arrangements being compared with one another, the output as to whether the image details occur in the second image being additionally dependent on whether the arrangements are substantially the same. (Lai, 3.1, Definition of MNS, 3.2, MNS for template matching, Fig. 1-5, apply MNS to template matching, it needs to convert all template image patches to a 2D numbered-point set P = { p i } N i =1 , which we call this first procedure the numbered point generating. Then performing local matching using these template patches to a given scene image to find the corresponding candidate patches and also converting them to a corresponding 2D numbered-point set Q = _ q j i _ j=1 , ···,n i =1 , ···,N , which we call this second procedure the Candidates Finding. The Candidates Finding procedure utilizes the lo- cal features extracting from the template image patches to perform matching to the scene image to find the corresponding matched Candidates; intuition for MNS is shown in Fig. 2 . The numbers 0 to 9 in Fig. 2 (a) represent the template, and the corresponding tar- get in Fig. 2 (b) undergoes rotation transformation, occlusion and background clutter. For humans, obviously, the rectangle area in Fig. 2 (b) is the best matched region corresponding to the template Fig. 2 (a), and q 1 0 in Fig. 2 (b) is the best matched point selected among candidates q 0 0 and q 1 0 because the relative position of q 1 0 equals to p 0 . For computer, we utilize MNS to measure the similarity between p 0 and q 1 0 to make the decision; discloses matching candidate pixel features from first image to template images that are rotated or transformed to match the first image by rotation). Additionally, the rational and motivation to combine the references Gao and Lai as applied in rejection of claim 1 apply to this claim.
Regarding Claim 15,
The combination of Gao and Lai further discloses wherein a number and/or positions and/or dimensions of the image details are ascertained, initially random or specified positions being selected, the method being carried out for all image details for a plurality of first images and with at least one second image, a confidence being ascertained for each pair of first and second images, the number and/or the positions and/or the dimensions being changed using an optimization procedure such that the confidence is as high as possible when the first and second images show the same object and, otherwise, the confidence being as small as possible. (Lai, 5.2, performance evaluation, discloses photometric changes, motion blur, in/out-of-plane rota- tion and more. We generate four datasets by sampling {870, 830, 780, 750} image pairs with a corresponding constant frame dif- ference dF rame = { 10 , 25 , 50 , 100 } in each video. Each such pair consists of frames f and f + dF rame , where f was randomly se- lected. The annotated ground-truth bounding box in frame f is used to define the template, while frame f + dF rame is the target image. We measure the detection accuracy by using the intersec- tion over union (IoU) measure between the detection result and the ground-truth bounding boxes: Accuracy = | Rest ∩ Rt ruth | | Rest ∪ Rt ruth | ∗100% , where, | ·| calculates the pixel area of the region, Rtruth and Rest are the ground-truth and estimated bounding boxes respectively; accuracy of matching is determined by feature matching in first and second images and their confidence is determined according to the accuracy of percentage of matching accuracy). Additionally, the rational and motivation to combine the references Gao and Lai as applied in rejection of claim 1 apply to this claim.
Regarding Claim 16,
The combination of Gao and Lai further discloses wherein the method is used or checking whether a surface image of an object belongs to a specified object, wherein the first and second images depict a surface of the same object, it being output that the surfaces of the first and second images are the same when at least one image detail occurs in the second image. (Lai, 3.1, Definition of MNS, 3.2, MNS for template matching, Fig. 1-5, apply MNS to template matching, it needs to convert all template image patches to a 2D numbered-point set P = { p i } N i =1 , which we call this first procedure the numbered point generating. Then performing local matching using these template patches to a given scene image to find the corresponding candidate patches and also converting them to a corresponding 2D numbered-point set Q = _ q j i _ j=1 , ···,n i =1 , ···,N , which we call this second procedure the Candidates Finding. The Candidates Finding procedure utilizes the lo- cal features extracting from the template image patches to perform matching to the scene image to find the corresponding matched Candidates; intuition for MNS is shown in Fig. 2 . The numbers 0 to 9 in Fig. 2 (a) represent the template, and the corresponding tar- get in Fig. 2 (b) undergoes rotation transformation, occlusion and background clutter. For humans, obviously, the rectangle area in Fig. 2 (b) is the best matched region corresponding to the template Fig. 2 (a), and q 1 0 in Fig. 2 (b) is the best matched point selected among candidates q 0 0 and q 1 0 because the relative position of q 1 0 equals to p 0 . For computer, we utilize MNS to measure the similarity between p 0 and q 1 0 to make the decision; discloses matching candidate pixel features from first image to template images that are rotated or transformed to match the first image by rotation). Additionally, the rational and motivation to combine the references Gao and Lai as applied in rejection of claim 1 apply to this claim.
Regarding Claim 17,
The combination of Gao and Lai further discloses wherein the method is used to verify an authenticity of a product, the first and second images depicting a surface of an identical product, it being output that the surfaces of the first and second images are the same when at least one image detail occurs in the second image. (Lai, 3.1, Definition of MNS, 3.2, MNS for template matching, Fig. 1-5, apply MNS to template matching, it needs to convert all template image patches to a 2D numbered-point set P = { p i } N i =1 , which we call this first procedure the numbered point generating. Then performing local matching using these template patches to a given scene image to find the corresponding candidate patches and also converting them to a corresponding 2D numbered-point set Q = _ q j i _ j=1 , ···,n i =1 , ···,N , which we call this second procedure the Candidates Finding. The Candidates Finding procedure utilizes the lo- cal features extracting from the template image patches to perform matching to the scene image to find the corresponding matched Candidates; intuition for MNS is shown in Fig. 2 . The numbers 0 to 9 in Fig. 2 (a) represent the template, and the corresponding tar- get in Fig. 2 (b) undergoes rotation transformation, occlusion and background clutter. For humans, obviously, the rectangle area in Fig. 2 (b) is the best matched region corresponding to the template Fig. 2 (a), and q 1 0 in Fig. 2 (b) is the best matched point selected among candidates q 0 0 and q 1 0 because the relative position of q 1 0 equals to p 0 . For computer, we utilize MNS to measure the similarity between p 0 and q 1 0 to make the decision; discloses matching candidate pixel features from first image to template images that are rotated or transformed to match the first image by rotation). Additionally, the rational and motivation to combine the references Gao and Lai as applied in rejection of claim 1 apply to this claim.
Claims 18 and 19 recite device with elements and storage medium with instructions corresponding to the method steps recited in Claim 11. Therefore, the recited elements of the device claim 18 and instructions of storage medium claim 19 are mapped to the proposed combination in the same manner as the corresponding steps of Claim 11. Additionally, the rationale and motivation to combine the Gao and Lai references presented in rejection of Claim 1, apply to these claims.
Furthermore, the combination of Gao and Lai further discloses A device configured to ascertain whether a specified image detail of a first image occurs in a second image, the device configured (Gao, [0010], Fig. 6, discloses a multiple camera imaging system that includes a first camera sensor to image a scene to produce a first image from a first perspective vantage point, a second camera sensor to image the same scene to produce a second image from a second perspective vantage point, and an image signal processor (ISP) to process and align the first and second images; device system is disclosed).
Furthermore, the combination of Gao and Lai further discloses non-transitory machine-readable storage medium on which is stored a computer program for ascertaining whether a specified image detail of a first image occurs in a second image, the computer program, when executed by a computer (Gao, [0010], Fig. 6, discloses a multiple camera imaging system that includes a first camera sensor to image a scene to produce a first image from a first perspective vantage point, a second camera sensor to image the same scene to produce a second image from a second perspective vantage point, and an image signal processor (ISP) to process and align the first and second images; device system is disclosed).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
DE 102012005325 A1 (method for the automatic recognition of image data of an overall image or a sequence of images, characterized by the following steps: acquisition of the image and its division and classification into sub-elements, that is, detailed image elements based on a picture element and / or image object feature analysis in particular as regards basic geometric shapes, textures, colors, material, perspective, wherein the pixel and / or image feature analysis is realized by analytical deterministic software techniques of image processing and image analysis, in particular Fourier analysis, edge sampling, color analysis and the like; Recognizing and identifying the classified picture element and / or picture object features using artificial intelligence, in particular a neural network, such that one or more descriptive text names are respectively assigned to the picture elements and / or picture objects; Feeding the text designations associated with the picture elements and / or picture objects into a textual knowledge base, in which a further analysis of the relationships of the picture elements and / or picture objects to each other and to one another and to the picture and / or parts of the picture by means of a text-based search engine, in particular based on a Neural network can be made such that the content and context of the image or the sequence of images is determined)
CN 119027762 A (a method for determining whether a predetermined image segment of a first image is present in a second image. The method begins by determining (S21) at least one image feature for a plurality of pixels within the predetermined image segment, respectively. At least one image feature is then respectively determined (S24) for a plurality of pixels of the second image. The image characteristic of the determined second image is then compared (S25) with the image characteristic of the determined first image to determine whether both have the same value. The method is characterized by determining a rotationally symmetric surface in the second image and converting the surface into a rectangular surface, and calculating image characteristics of pixels of the second image from pixels within the converted surface; use a camera to capture a two-dimensional image (so called texture capture) in the form of a grey value or a multi-dimensional value (colour value) of the (partial) surface of the object and the image can be reproduced as much as possible for different cameras. In other words, the imaging should appear to be as similar as possible. Here, the easily corrected differences do not cause problems, which may be corrected, for example, by rotation, scaling, shifting or brightness adaptation of the image content)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PINALBEN V PATEL whose telephone number is (571)270-5872. The examiner can normally be reached M-F: 10am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at 571-272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Pinalben Patel/Examiner, Art Unit 2673