DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims: claims 1-19 are pending below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on October 17th 2024 was filed and considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Petrou et al (US 2013/0188886).
Claim 1:
Petrou et al (US 2013/0188886) anticipated the following subject matter:
A system (figure 1 detail a system) for identifying an item in an image comprising:
an application running on a device capable of receiving an image from the device (0002-0003 detail application that display, screen, camera to user for image captured scene to process pictures for UFP, OCR and other additional information about the object from the image data); and
a product ID server in communication with an Optical Character Recognition (OCR) server and a text recognition server capable of receiving the image from the device (0003 detail use of OCR, where 0004 further detail connection to server for further analysis; 0005-0007 detail further processing and analysis of servers; 0066 detail use of OCR provide information such as bottle, company logo, shape, bar code…etc).
Claim 2:
The system of claim 1wherein the device is a mobile phone, a laptop computer, or any other device capable of capturing an image (0004 detail cell phone with camera, mobile device to server; figure 1 and 0030 detail device such as cell phone (smartphone) and general-purpose computers).
Claim 3:
The system of claim 1 wherein the product ID server identifies an object in the image from the device and relays information about an identified object back to a user (0002-0003 use of OCR of object to extract information to be display on screen for user; 0008 detail display the images and additional information at a speed that, from a human perception point of view, corresponds with the scene being captured by the camera at the time of display).
Claim 4:
The system of claim 1wherein the application and the product ID server function remotely in a cloud server or locally on the smart device (0004-0007 detail captured images may then be sent wirelessly by the mobile device to a server for further analysis; 0034-0035 detail device 100 may be at one node of a network 195 and capable of directly and indirectly communicating with other nodes of the network such as a server 180).
Claim 5:
The system of claim 1 wherein the product identification server may identify at least one image region of interest (IROI) from the image and the IROI may be stored in binary form (0033 detail image data stored in bitmaps or grids of pixel, compares/uncompress, BMP, JPEG…which are all in binary data format).
Claim 6:
A method (0008-0011 detail method) of identifying an item in a captured image, the method comprising:
capturing an image containing one or more objects with a device (0002-0003 detail application that display, screen, camera to user for image captured scene to process pictures for UFP, OCR and other additional information about the object from the image data; 0004-0006);
transmitting a captured image to a product ID server, the product ID server in communication with an Optical Character Resolution (OCR) server and a text recognition server (0003 detail use of OCR; 0066 detail use of OCR provide information such as bottle, company logo, shape, bar code…etc; 0004-0007 detail captured images may then be sent wirelessly by the mobile device to a server for further analysis; 0034-0035 detail device 100 may be at one node of a network 195 and capable of directly and indirectly communicating with other nodes of the network such as a server 180);
identifying and processing text and/or regions of interest contained on the one or more objects in the captured image (0003 detail use of OCR; 0066 detail use of OCR provide information such as bottle, company logo, shape, bar code; 0007 detail bounding box around the objects for analysis, where above detail OCR for text extraction); and
providing information to a user about the one or more objects seen in the captured image (0002-0003 use of OCR of object to extract information to be display on screen for user; 0008 detail display the images and additional information at a speed that, from a human perception point of view, corresponds with the scene being captured by the camera at the time of display).
Claim 7:
The method of claim 6 wherein a barcode is recognized in the captured image and the user is prompted to further photograph the barcode (0050 detail detect and recognize such as barcodes; figure 9 and 0065; 0069 detail capture of bar code and user to zoom in on bar code across many frames).
Claim 8:
The method of claim 6 wherein the captured image is deemed unacceptable and the user is prompted to photograph a clearer image (0096 detail request further annotation with second image if server receives request from mobile service (user terminal)).
Claim 9:
The method of claim 6 wherein no barcode is found in the captured image and the image is subject to OCR (0052 detail different extraction of information such as shape, color, size, ingredient…etc of object capture in image data, but if image capture UPC or QR bar code (which indicates there is none in the image)).
Claim 10:
The method of claim 9 wherein characters from the captured image were not recognized by the OCR and are then subject to an autocorrect function to cure potential deficiencies (0044 detail object in image data that is unrecognizable due out of focus but second image will be in focus; 0054-0055 detail not recognized due to glare autocorrect function to detect object to other previous or sequencing images; 0081).
Claim 11:
The method of claim 9 wherein the OCR identifies zero objects in the captured image and the user is prompted to manually enter the object's identifying information (0036 detail user inputting such as keyboard, touch screen and microphone; 0067 detail absence of signal (zero), the search engine rank result to structure search object in image, where 0068 user appears to show interest in objects or specific objects).
Claim 12:
The method of claim 9 wherein one object is identified in the captured image and the user is presented with an object profile for the identified object (0002-0003 use of OCR of object to extract information to be display on screen for user; 0008 detail display the images and additional information at a speed that, from a human perception point of view, corresponds with the scene being captured by the camera at the time of display).
Claim 13:
The method of claim 9 wherein more than one object is identified in the captured image and the more than one identified objects from the captured image are subject to text recognition (above detail OCR from images, where 0009 detail different objects in image data for information extraction).
Claim 14:
The method of claim 13 wherein an image similarity value (ISV) is assigned to each of the more than one identified objects of the captured image based on the more than one identified object's similarity to the object in the captured image (0009 detail processor may also determine that objects in different images relate to the same item if the objects are visually similar and their locations would overlap if one image was superimposed over the other; 0045, 0054 detail matching to similar with number of matched descriptors (ISV); 0052 search database for similarities).
Claim 15:
The method of claim 13 wherein the user is presented with a list of potentially matching objects and is prompted to choose the object that most closely resembles the object in the captured image (0009 detail processor may also determine that objects in different images relate to the same item if the objects are visually similar and their locations would overlap if one image was superimposed over the other; 0045, 0054 detail matching to similar with number of matched descriptors; 0052 search database for similarities).
Claim 16:
The method of claim 14 wherein a list of more than one identified objects is returned to the user and ranked based on ISV (above 0054 detail matching to similar with number of matched descriptors (ISV), where 0061, 0063-0064 detail ranking results from the features (descriptors); 0082).
Claim 17:
The method of claim 13 wherein the text recognition does not recognize the object in the captured image and the object in the captured image is further subjected to image matching (0054 detail matching to similar with number of matched descriptors; 0052 search database for similarities).
Claim 18:
The method of claim 13 wherein the text recognition comprises a multi-level search (0089 detail pyramid level searching for annotated, position, feature, bounding box, area of interest; 0072 detail subset (multi-level) selected by user from annotation; 0087, 0094 subset regarding text being present and server capable to detect, recognizing or tracking (searching)).
Claim 19:
The method of claim 9 wherein a user may rate, take notes on, or add objects of the captured image to a list (0060 detail user wants all information and asks to store all the information in a single document (list); 0072 detail user ranking result in optimum annotation).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bridges et al (US 2018/0373934) teaches Image Sensor For Outputting The Image Pixel Data, Comprises An Image Buffer Configured To Temporarily Store The Image Pixel Data, An Environment Sensor, An Image Processing Apparatus And A Controller – figure 1
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSUNG-YIN TSAI whose telephone number is (571)270-1671. The examiner can normally be reached 7am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at (571) 272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSUNG YIN TSAI/Primary Examiner, Art Unit 2656
Prosecution Insights