DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Venable et al. (US 20180108120 A1, hereinafter “Venable”)
Zhu et al. (KR 20210059712 A, hereinafter “Zhu”)
Bogolea et al. (US 20170337508 A1, hereinafter “Bogolea2”)
Zhang et al. (CN 114493552 A. hereinafter “Zhang”)
Interview Summary
Several calls were placed to applicant’s Attorney of record (Landon E. Wiebusch at 469-396-7738) in order to try to advance prosecution but they were never return as of the time of issuance of the office action.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/24/2025 has been entered.
Response to Arguments
Applicant's arguments/Remarks filed 02/17/2026 have been fully considered but they are moot in view of the new ground of rejections set forth below persuasive.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 8 and 11-16, 18, 21-26 are rejected under 35 U.S.C. 103 as being unpatentable over Venable et al. (US 20180108120 A1, hereinafter “Venable”) in view of Zhu et al. (KR 20210059712 A, hereinafter “Zhu”) in view Bogolea et al. (US 20170337508 A1, hereinafter “Bogolea”) and in view of Zhang et al. (CN 114493552 A. hereinafter “Zhang”).
Regarding claims 1, 11 and 21:
Venable teaches a system comprising:
an image capture device configured to capture at least one image of a product storage structure at a product storage facility (Venable [0059]-[0061], where Venable teaches the image capture assembly 22 includes an imaging component 38 which includes one or more image capture devices, such as digital cameras 40, 42, 44, that are carried by a support frame 46); and
a computing device including a control circuit, the computing device being communicatively coupled to the image capture device, the control circuit being configured to:
obtain the at least one image captured by the image capture device (Venable [0053], where Venable teaches applications of the store shelf imaging systems described herein include automated, shelf-level retail prescriptive analytic services which utilizes a mobile image capture system to capture and store images of the display areas tagged with location information, analyze the images with location data and return the detailed store spatial layout and classification of (1) regular shelf signage (e.g., regular price labels with barcodes), (2) promotional shelf signage, and (3) products at the shelf display facings. This output can be used on its own for generating detailed views of current store product and signage layout, identifying misplaced or out of stock products or printing signage updates in store-walk order. This output can also be used in comparison with retailer standards or plan reference information to generate views of display conformance to aid in tracking and improving retailer operations);
analyze the at least one image of the product storage structure captured by the image capture device to detect at least one of individual ones of product labels and products located on the product storage structure (Venable [0074], where Venable teaches The product data recognition component 84, which may be a part of the image data processing component 82, analyses the processed images for detecting price tag locations, extracting product data 26, such as price tag data, and performs image coordinate conversion (from pixel position to real-world coordinates));
detect individual products and each one of the detected individual product labels from the at least one image to generate a plurality of images, each of the images depicting an individual one of the detected products or an individual one of the detected product labels (Venable [0143], [0145], where Venable teaches For the application of store profiling discussed above, obtaining individual spatial profiles for each image and determining whether the overlap FOV of adjacent images is great than a threshold value (e.g., zero) or not is generally sufficient for characterizing the image capture assembly 22. However, additional information may be extracted for configuring/reconfiguring the image capture assembly 22 if the configuration has not been determined or optimized or has been adjusted for a different retail application of interest);
stitch together the images of the individual product and the cropped image of the individual product label to obtain a stitched image (Venable [0159], [0198], where Venable teaches According to the exemplary embodiment described, the requirement is to cover displayed products 0-8 feet high and planogram typical widths of 2, 4, 6 & 8 feet. These images could be generated by stitching multiple high resolution images together vertically and horizontally from the high resolution system, or by stitching multiple low resolution images together vertically and horizontally or only vertically if the fields of view were acceptable in width but not vertically to cover the 0-8 feet product facing service requirement); and
associate, based on known positional coordinates of each of the products and each of the product labels in the at least one stitched image, the received one or more characters extracted from each one of the individual products and product labels detected in the at least one stitched image with corresponding ones of the plurality of images of the products and product labels (Venable [0054], [0143], The store profile 12 may be in the form of a 2-dimensional or 3-dimensional plan of the store which indicates the locations of products, for example, by providing product data for each product, such as an SKU or barcode, and an associated location, such as x,y coordinates (where x is generally a direction parallel to an aisle and y is orthogonal to it), a position on an aisle, or a position on a predefined path, such as a walking path through the store. In some embodiments, the store profile may include a photographic panorama of a part of the store generated from a set of captured images, or a graphical representation generated therefrom).
crop the at least one image of the product storage structure to obtain multiple cropped images of different individual product labels affixed to the product storage structure, the different individual product labels associated with different individual products housed by the product storage structure;
stitch together the multiple cropped images of the different individual product labels to obtain a stitched image that includes all of the different individual product labels (Venable [0068], [0195], where Venable teaches Display facing profiling involves providing full planogram field of view images to either an external service or an internal image analysis processing module that will identify actual products and their locations on a display or otherwise classify a facing location as “empty”. Product identification is typically done via image processing techniques comparing the segmented product images from the store scan with a product reference image library);
receive, from an OCR service, the extracted characters and positional coordinates of the extracted characters within the stitched image, wherein the positional coordinates of the extracted characters within the stitched image identify x-y pixel ranges for the extracted characters within the stitched image (Venable [0136], [0143], where Venable teaches the images are processed using optical character recognition (OCR) software to identify the marks 158 within the detected boxes just above each dot and a formula is applied to compute the actual location, in the x,z plane, of each dot); and
Venable fails to teach explicitly each label is from a cropped image instead of each label from each image;
However, Zhu teaches According to another aspect, a system for dividing an input image into multiple image portions. The system can then supply the parts as separate inputs to the machine learning system. The system can be configured to stitch the individual enhanced output portions together to produce the final enhanced image (Zhu [0140]).
Therefore, taking the teachings of Venable and Zhu as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to crop the labels and other information out of the relevant image, in order to allow the used of less image captured device which would reduce cost and facilitate stitching (Venable [0061]).
Venable in view of Zhu fails to teach associate, based on positional coordinates of the individual product within the stitched image and positional coordinates of the individual product label within the stitched image, one or more characters extracted from the cropped image of the individual product with the one or more characters extracted from the cropped image of the individual product label, wherein the positional coordinates of the individual product and the positional coordinates of the individual product label identify x-y pixel ranges for the individual product and the individual product label, respectively, within the stitched image.
However, Bogolea teaches cropping an image to obtain region or area of interest (Bogolea [0044]).
the system also extracts relative positions of these shelf tags from this section of the image of the first shelving segment and compare these relative positions to relative slot positions assigned to the first shelf by the planogram in order to confirm that real positions of these shelf tags align to the current stocking plan of the store. In this example, the system can define slot positions along the first shelf based on shelf tag locations, as described above, if the number and relative positions of these shelf tags match the planogram; otherwise, the system can project slot locations defined in the planogram onto the image of the first shelving segment or onto the first region—in the image—of the first shelf, as described below (the object (tag and item) location is identify in the actual image, in other words pixel range).
In this example, the system can also: read a barcode, extract a SKU or other product identifier, and/or read a price from a first shelf tag detected in the first image; confirm that these data read from the first shelf tag align to data assigned to the adjacent slot (e.g., above and to the right of the first shelf tag) by the planogram. If data read from the first shelf tag align to data specified in the planogram, the system can define a first slot position relative to the first shelf tag in the image, as described above, and treat these shelf tag data as ground truth for the slot. Otherwise, the system can flag the first shelf tag for correction in Block S160 and treat the planogram as ground truth for the first slot, such as by projecting slot locations defined in the planogram onto the image to define the first slot, as described below (Bogolea [0054]-[0057], [0059] and [0062], [0074]).
Therefore, taking the teachings of Venable, Zhu and Bogolea as whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to associate the tag and its determine location with the corresponding products such that the stitched image is configure as a list with titles (tag, price bar code, etc) and corresponding items under, over, to the left or to the right of the title or tag that described the items or a characteristics they have in common, in order to reduce the need for multiple titles or used of multiple tags. Such as one tag can correspond to a plurality of item on a shelve of a particular section of a shelve whether viewed in parson or displayed on a screen.
Venable in view of Zhu and in view of Bogolea fails to teach send the stitched image to an internet-based optical character recognition (OCR) service, the internet-based OCR service extracting extract one or more characters from the stitched image;
receive, from the internet-based OCR service, the extracted characters and positional coordinates of the extracted characters within the stitched image, wherein the positional coordinates of the extracted characters within the stitched image identify x-y pixel ranges for the extracted characters within the stitched image
However, Zhang teaches after the contract scanning piece downloaded, transmitting the contract piece to the OCR server. the information page comprises the amount of the supplier payment the supplier account, and the key information horizontally dividing the pattern by horizontal projection, obtaining the image of each line; vertically dividing each row of image by vertical projection, finally determining the coordinate position of each character, dividing each character; respectively a certain pixel of the binarization pre-processed image in the horizontal and vertical directions, for the binarization image non-black and white, counting the white point or black point, judging the upper and lower boundary of each row and the left and right boundary of each column according to the statistical result, so as to realize the purpose of cutting (Zheng [0048], [0075]).
`Therefore, taking the teachings of Venable, Zhu, Bogolea and Zhang as whole, it would have been obvious to one of ordinary skill in the art before he effective filing date of the application to use a remote OCR service and to determine position or coordinate of each character using X-Y pixel coordinate, since it is a well-known technique allowing the use or more capable processing method as oppose to a handheld that has less processing capability and using pixel as a unit to locate each character in the image since each character is made and can be represented in the image using a group of pixels.
Regarding claims 2, 12 and 22:
Venable in view of Zhu, in view of Bogolea and in view of Zhang teaches wherein the image capture device comprises a motorized robotic unit that includes wheels that permit the motorized robotic unit to move about the product storage facility, and a camera to permit the motorized robotic unit to capture the one or more images of the product storage structure (Venable [0057], where Venable teaches in a fully-autonomous mode, the motorized mobile base 20 may include a navigation component 30 and an associated power source 32, such as a battery, motor, drive train, etc., to drive wheels 34 of the of the mobile base in order to move the system 10 to a desired location with desired facing according to a request from the control unit 24. The navigation component 30 may be similarly configured to the control unit 24 and may include memory and a processor for implementing the instructions provided by the control unit and reporting location and orientation information back to the control unit).
Regarding claims 3, 13 and 23:
Venable in view of Zhu, in view of Bogolea and in view of Zhang teaches wherein the control circuit is programmed to generate a first virtual boundary line surrounding the individual product and a second virtual boundary line surrounding the individual product label (Venable [0055], [0136]; Zhu [0129], [0140]-[0141]; Bogolea [0051], where Venable teaches The store profile 12 is generated by capturing images of product display units 14, such as store shelf units, at appropriate locations with appropriate imaging resolutions. As illustrated in FIG. 1, each shelf unit 14 may include two or more vertically-spaced shelves 16, to which product labels 18, such as product price tags, displaying product-related information, are mounted, adjacent related products 19; and Bogolea teaches In one example, the system implements computer vision techniques to: detect features in a first region of a first image cropped around an area or volume above a first shelf; and to detect a set of discrete objects arranged across the first shelf in the first region of the first image, such as by identifying edge features delineating bounds of each of these objects. ).
Regarding claims 4, 14 and 24:
Venable in view of Zhu, in view of Bogolea and in view of Zhang teaches wherein the stitched image has a predetermined pixel size, and wherein the control circuit is programmed to employ a first fit decreasing height algorithm to populate the stitched image with pixel within at least one of the first virtual boundary line and the second virtual boundary line (Venable [0095], [0113], [0162], [0164]; Zhu [0129], [0140]-[0141]; Bogolea [0082]-[0083], [0096], where the combination teaches the system can implement template matching techniques to compare regions (or subregions) of an image to a set of template images representing a set of products specifically assigned to a slot, shelf, shelving segment, or shelving structure shown in the image. For example, the system can: segment an image by shelf; extract a first region of the image corresponding to a first shelf shown in the image; crop the first region around objects shown on the first shelf (i.e., remove a background area from the first region); and implement edge detection, object detection, or other computer vision techniques to identify discrete subregions in the first region of the image, wherein each subregion represents a single object (e.g., a single unit of a product) arranged on the first shelf. (In this example, the system can also project a slot dimension defined in the planogram for the first shelf onto the first region of the image to inform or guide detection of discrete objects on the first shelf.) The system can then implement template matching techniques to compare each subregion of the first region of the image to template images in the set of template images selected for the first shelf until a match is found or until the set of template images is exhausted for each subregion).
Regarding claims 5, 15 and 25:
Venable in view of Zhu, in view of Bogolea and in view of Zhang teaches wherein the control circuit is programmed implement a synchronous architecture in combination with the first fit decreasing height algorithm to obtained a stitched image (Venable [0095], [0113], [0162], [0164]; Zhu [0129], [0140]-[0141]; Bogolea [0051], [0082]-[0083], [0096], where Zhu teaches In some embodiments, the system may determine the size of the portion of the image, including additional pixels, to take into account the subsequent cropping operation to be performed by the system (e.g., the system may The enhanced portion of the image can be cropped before stitching together. For example, the system can perform a filtering operation on the truncated 100x100 portion of the image portion, so that a 102x102 size image By removing additional pixels during the filtering operation, the truncated portion may not have the edge effect discussed above.).
Regarding claims 6, 16 and 26:
Venable in view of Zhu, in view of Bogolea and in view of Zhang teaches wherein the control circuit is programmed to implement an asynchronous architecture in combination with the first fit decreasing height algorithm to obtain the stitched image (Venable [0095], [0113], [0162], [0164], This map file is then used as input to the mission planner function in the Master Controller Module 324 which allows a user and algorithms to define the scan areas and desired mission path, and calculates interface instructions for the Robotic Mobile Base 20; Zhu [0140], In some embodiments, the system may determine the size of the portion of the image, including additional pixels, to take into account the subsequent cropping operation to be performed by the system (e.g., the system may The enhanced portion of the image can be cropped before stitching together. For example, the system can perform a filtering operation on the truncated 100x100 portion of the image portion, so that a 102x102 size image By removing additional pixels during the filtering operation, the truncated portion may not have the edge effect discussed above.).
Regarding claims 8 and 18:
Venable in view of Zhu, in view of Bogolea and in view of Zhang teaches wherein the positional coordinate are defined by x-y pixel ranges (Venable [0107], [0051]; Bogolea [0054], In one example, the system: implements methods and techniques described above to detect the upper leading edge and lower leading edge of a first shelf in a first image cropped around a first shelving segment; detects features in a section of the first image between the upper leading edge and the lower leading edge of the first shelf; identifies groups of features along this section of the first image as shelf tags affixed to the first shelf; and detects a left edge of each identified shelf tag. For a first region of the first image representing the first shelf in the first shelving segment, the system can then associate a subregion of the image 1) extending laterally (X coordinate) from the left edge of a first shelf tag to the left edge of a second shelf tag to the right edge of the first shelf tag (or to the right edge of the image) and 2) extending vertically (Y coordinate) from the upper leading edge of the first shelf to the lower leading edge to a second shelf above (or to the top edge of the image) within a single slot. The system can repeat this process for other subregions in the first region of the image to detect multiple discrete slots on the shelf (Since Bogolea is talking about digital image thus pixels) ).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 April 27, 2026
Prosecution Insights