DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2-7, 10-14 are rejected under 35 U.S.C. 103 as being unpatentable over Higgins et al (US2021374467) in view of Havir et al (US11636602).
Referring to claim 2. Higgins et al (herein “Higgins”) discloses a “Correlated Slice and View Image Annotation For Machine Learning”. See Figs. 1-6 and respective portions of the specification. Higgins further discloses systems and methods for human-assisted graphical user interface (GUI) based image annotation that allow a user to select features depicted in images and assign category labels to those features, which labels are then used to train and retrain supervised machine learning algorithms (See Sects. 0012-0016, 0025-0029). It should be noted that as used herein, a “feature” corresponds to an “item” as recited in the claims, as both represent user-selectable entities depicted within an imaged that are identified and associated with classification information. Higgins explicitly teaches that such features are individually selectable, labeled, and stored with associated characteristic information. Moreover, Higgins discloses a method comprising: displaying, at an electronic device, an image of a plurality of items and a visual representation of one or more item categories. More specifically, Higgins discloses an editing GUI configured to display images of a correlated image set, wherein each displayed image depicts a plurality of features (items) together with associated characterization information (labels) (See at least Sect. 0029-0030, 0049-0051, 0077-0079). Higgins further discloses receiving an input, at the electronic device, the input selecting a portion of the image of the plurality of items and selecting one of the one or more item categories. Specifically, Higgins discloses that a user may interact with GUI to select a feature present in an image and assign a label (category) to that item (See Sects. 0045, 0065, 0067, 0109). Higgins further discloses in response to the input, associating an item of the selected portion of the image with the selection of the one or more item categories (See at least Sect. 0044, 0064, 0112). If applicant disagrees that Higgins explicitly discloses displaying a visual representation of one or more item categories. Havir et al (herein “Havir”) discloses a “Prelabeling For Semantic Segmentation Tasks”. See Figs. 1-6 and respective portions of the specification. Havir further discloses a graphical user interface including a display portion that presents an image and user-interface elements comprising a list of classes, each having a name and corresponding color, which are displayed concurrently with the image (Sect. 0066) and discloses that labeled regions corresponding to these classes are displayed as color-coded overlays on the image (See Sects. 0065-0066, 0070). Havir further discloses user-interface tools that allow a user to select pixels or regions of pixels using drawing tools, thereby allowing selection of a portion of the image independent of predefined object boundaries (See Sects. 0067, 0072). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Havir’s structured category display and region-based selection tools into Higgins annotation system as it would provide a more efficient labeling system by enabling direct region-based selection of image portions and improve labeling speed and accuracy through visual guidance.
Referring to claim 3. Higgins discloses training one or more machine learning models based on the associating of the selected portion of the image with the selection of the one or more categories (See Sects. 0053-0054, 0081).
Referring to claim 4. Higgins discloses displaying a suggested one of the one or more item categories for a portion of the image of the plurality of items, the suggested one of the one or more categories based on the one or more machine learning models (Sects. 0046-0047). It should further be noted Havir discloses machine-learning prelabels displayed as overlays representing suggested categories (See Sects. 0042, 0069-0070).
Referring to claim 5. Higgins discloses identifying the items as each belonging to one of the item categories, so as to determine an identified item category for each item; and transmitting each identified item category for display on the electronic device in association with the corresponding item (See at least Sects. 0040-0046, 0049).
Referring to claims 6-7. Higgins discloses forming labeled images of ones of the plurality of items based on the associating and appending the labeled images of ones of the plurality of items to a training data set for training one or more machine learning models, the one or more machine learning models having images of the items as inputs, and having outputs corresponding ones of the item categories for each input item and further after the appending, training the one or more machine learning models using the labeled images of the training data set (See Sects. 0035-0037, 0053, 0054).
Referring to claim 10. Higgins discloses designating portions of the image corresponding to the items as being selectable portions, wherein the displaying includes displaying the selectable portions of the image (See at least Sects. 0067, 0081).
Referring to claims 11-12. Higgins doesn’t disclose displaying an overlay over the image of the plurality of items to highlight selectable items or wherein the overlay includes an indication of a suggested item category. Havir discloses displaying overlays highlighting labeled regions (See at least Sects. 0042, 0062, 0066, 0070). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Havir into the system of Higgins as it would provide a more efficient categorizing and labeling system by improving speed and accuracy through visual guidance and reducing cognitive load through structured category presentation.
Referring to claims 13-14. Higgins in view of Havir disclose the combination as set forth above and applied to claim 2. It should be noted claim 13 and claim 14 recite a system and a non-transitory computer-readable storage medium, respectively comprising instructions for performing the same three operative steps as claim 2 (displaying, receiving, associating). Higgins further discloses an electronic device comprising one or more processors (202) and memory (204), with one or more programs (feature determination module 206, tagging module 210, editing module 212, etc.) stored in memory (204) that include instructions for displaying, receiving, and associating as presented in claim 13 (See at least Sects. 0035, 0045, 0049, 0056-0057). Higgins further discloses a non-transitory computer readable-media containing instructions that, when executed by the one or more processors, cause the processors to perform the disclosed method (See at least Sect. 0057, 0162).
Claim(s) 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Higgins et al (US2021374467) in view of Havir et al (US11636602) and in further view of Hamid (US20120188366).
Referring to claims 8-9. Higgins in view of Havir disclose the combination as set forth above and applied to claims 1-7. Higgins doe not disclose wherein the items comprise one or more of recyclable items or non-recyclable items or wherein the one or more item categories comprise glass, plastic, paper, or metal. Hamid discloses an “Inspection Apparatus and Method Using A Graphical User Interface”. See Figs. 1-13 and respective portions of the specification. Hamid further discloses optical sorting systems that classify items based on material characteristics and sorting criteria (See Sects. 0003-0004, 0033-0036). It should be noted that the selection of categories such as glass, plastic, paper, and metal represent a well-known set of material classifications in sorting systems. It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Hamid and incorporate these known classification categories to the labeling system of Higgins, as this would allow for the most well-known material classification categories to be selected and items labeled for these categories.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TERRELL HOWARD MATTHEWS whose telephone number is (571)272-5929. The examiner can normally be reached Monday thru Friday; 8:00 AM - 4:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael McCullough can be reached at (571)272-7805. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TERRELL H MATTHEWS/Primary Examiner, Art Unit 3653
Prosecution Insights