Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prelim. Amdt./Amendment
Receipt is acknowledged of the Preliminary Amendment filed on September 09, 2024.
Claim Objections
Claims 17 and 26 are objected to because of the following informalities:
Re claim 17: Please substitute “they” with –the objects--.
Claim 26 recites the limitation "The system" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 2, 14-17, 24, 26 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Davis et al. (US 2013/0223673, cited by the applicant).
Re claim 1: Davis teaches a method for detection and identification of objects comprising capturing a plurality of views of one or more objects (12) from multiple cameras (figs. 3A-3C); detecting the one or more objects in the images using an object detector (paragraph 0084); extracting features from the one or more detected objects (paragraph 0088); matching the extracted features from the images with features of objects enrolled in a database (paragraphs 0069, 0285); and fusing results of the matching to identify an object in the images as an object enrolled in the database (i.e., a display device (375) displaying item information retrieved from a database system according to the item recognized portion read by cameras (372) (fig. 37; paragraph 0402)) (see figs. 3A-3C, 6, 7, 37; paragraphs 0067-0089, 0094-0095, 0402-0405).
Re claim 2: Wherein objects are detected in images from each of the plurality of cameras (figs. 3A-3C; paragraphs 0094-0101).
Re claim 14: Wherein the one or more objects are detected by one or more of semantic segmentation, background subtraction and color segmentation for each of the plurality of cameras (figs. 3A-3C, 23-24; paragraphs 0096, 0136, 0184-0187).
Re claim 15: Wherein extracted features from each of the semantic segmentation, background subtraction and color segmentation for each of the plurality of cameras camera are used to match with features in a database of enrolled objects (figs. 3C, 25; paragraphs 0096, 0188).
Re claim 16: Wherein matches based on each of the semantic segmentation, background subtraction and color segmentation are fused together to create a match (i.e., the process of image rectification) for each of the plurality of cameras (paragraph 0096).
Re claim 17: Wherein the matches from each of the plurality of cameras are fused together to create a final match to the object in the image (paragraphs 0094, 0098, 0224).
Re claim 24: Wherein the objects (12) are products in a retail setting and further wherein the objects are detected as the objects are placed on or after the objects are placed on a checkout counter (Abstract).
Re claim 26: Davis teaches a system (fig. 37) comprising a plurality of cameras positioned to collect still or video imagery from different angles of a scene (figs. 3A-3C); a processor coupled to the one or more cameras such as to be able to collect the still or video imagery from each of the cameras; and software (fig. 7) that, when executed by the processor, cause the system to: capture a plurality of views of one or more objects (12) from the plurality of cameras (Figs. 3A-3C); detect one or more objects in the images using an object detector(paragraph 0084); extract features from the one or more detected objects (paragraph 0088); match the extracted features from the images with features of objects enrolled in a database(paragraphs 0069, 0285); and fuse results of the matching to identify an object in the images as an object enrolled in the database(372) (fig. 37; paragraph 0402)) (see figs. 3A-3C, 6, 7, 37; paragraphs 0067-0089, 0094-0095, 0402-0405).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 5-11, 21-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Davis in view of Dal Mutto et al. (US 2019/0108396).
Re claims 3, 5, 6: The teachings of Davis have been discussed above.
Although, Davis teaches the method of detecting objects in the images captured by a plurality of cameras and features are extracted from the detected objects and the extracted featured are matched with features of objects in the database, he fairly suggests that the method comprises a trained network.
Dal Mutto teaches a method for identifying objects comprising means for analyzing object (1350) with a trained conventional neural network serving as a trained network for detecting particular shape in an image (paragraph 0203), wherein the analysis module (350) detecting location of the logo within the region (i.e., a bounding box containing the logo and/or shapes, paragraph 0214) (see figs. 9-11, 19; paragraphs 0131-0136, 0149, 0203, 0214).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Dal Mutto to the teachings of Davis in order to improve detecting of features retrieved from the image using well-known conventional neural network.
Re claim 7: Wherein the matches comprise an object identified in the database and a confidence score that the features associated with the object in identified in the database match the features extracted from the detected objects (figs. 31-36; paragraphs 0023, 0267-0281).
Re claim 8: Wherein the confidence scores of the matches derived from images from each camera are fused together to form a final probability that the object identified in the database matches an object in the captured image (i.e., an object can be identified by a barcode on the object; fig. 31, paragraph 0277).
Re claim 9: Wherein the confidence scores are weighted (paragraph 0277).
Re claim 10: Wherein the confidence scores are weighted based on a determination of an angle of an object in an image with respect to the camera to capture the image (e.g., object 6 is assigned a certainty score of 0 because the object is not located on an angle of the camera, fig. 36, paragraph 0277).
Re claim 11: Wherein confidence scores are weighted based on which side of the object is facing the camera (e.g., an object 4 was given confidence score of 7 from the captured image because the top side is facing the camera before the reducing the confidence score using the thermal image data (paragraph 0276)).
Re claim 21: Wherein the database contains metadata regarding the objects in the database (e.g., reference data for objects retrieved from a database wherein the reference data serves as a metadata, paragraph 0283-0285).
Re claim 22: Wherein the metadata includes the weight and size (e.g., dimension of the objects, paragraph 0277) of the one or more objects
Re claim 23: Wherein the metadata is used to improve the probability of a match between the one or more objects detected in the image and an object enrolled in the database (paragraphs 0277, 0283-0285).
Allowable Subject Matter
Claims 4, 12, 13, 18-20, 25 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
None of prior art teaches the method of for detection and identification of objects comprising the trained network placing bounding boxes around objects wherein the bounding boxes are complex concave polygons, wherein the confidence scores are weighted based on a temporal component wherein images of the object at certain times may be weighted more heavily than images of the object at other times, and the one or more objects detected in images from each camera are fused together to create one or more optimized views of the objects, the probability of a match is increased when matching products identified in the database match objects listed on a receipt from a retail checkout as set forth in the claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wong et al. (US 12406502) and Balasubramaniam et al. (US 11869032) teach methods for processing images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEUNG H LEE whose telephone number is (571)272-2401. The examiner can normally be reached 7-4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Steven Paik can be reached at 571-272-2404. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEUNG H LEE/ Primary Examiner, Art Unit 2876