DETAILED ACTION
This final office action is responsive to the amendment received June 8, 2026. Claims 1-20 are pending. Claims 1, 8, and 15 are independent.
The objection to the specification is withdrawn in light of applicant’s amendment.
A new grounds of rejection under 35 USC §103 is made in light of applicant’s amendment. See sections Claim Rejections – 35 USC §103 and Response to Amendments below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-8, and 10-20 rejected under 35 U.S.C. 103 as being unpatentable over Buhler et al. (US10769503), hereinafter Buhler, in view of Bloch et al. (Directional relative position between objects in image processing: a comparison between fuzzy approaches), hereinafter Bloch, in view of Mandal et al. (Cognitive system to achieve human-level accuracy in automated assignment of helpdesk email tickets), hereinafter Mandal.
Regarding claim 1, Buhler teaches the method:
a processor device operatively coupled to a memory, the processor device being configured for:
intercepting an incoming message that comprises graphical content; (Buhler, column 20, lines 25-28: “The method 700 includes receiving (708) an input image having content . For example , the client device 104-1 receives an input image 108-1 via an image or document search application 106, according to some implementations.”)
identifying characteristics of one or more ontological structures from the graphical content; (Buhler, column 20, lines 32-34: “Referring next to FIG. 7B, the method 700 also includes generating (712) a feature vector corresponding to the input image using a trained classifier model (e.g., a trained CNN, such as the classifier described above in reference to FIG. 5.” – The feature vector being generated using the trained classifier model is analogous to identifying characteristics of one or more ontological structures as the feature vector has a plurality of components.)
associating one or more numeric values with each of the identified characteristics of the one or more ontological structures; (Buhler, column 2, lines 39-43: “In some implementations, the content comprises one or more of: semantic content, color histogram content, facial features, words, audio content, and graphical content. In some implementations, the feature vector corresponds to one or more features of the content of the image.” And column 20, lines 36-44: “The feature vector has a plurality of components. In some implementations, each component is (714) represented using a floating-point number. In some implementations, a majority of the components range (716) between 0.0 and 1.0. In some implementations, a first integer value corresponding to a first component of the plurality of components has (718) a length that is distinct from a second integer value corresponding to a second component of the plurality of components” – The characteristics is analogous to the plurality of components; thus each component being represented using a floating-point number is analogous to the one or more numeric values associated with the characteristics while the components (characteristics) comprising semantic content, color histogram content, facial features, words, audio, and graphic content is analogous to the ontological structures.)
computing a hash code of the incoming message by performing a hash function upon a subset of the numeric values, (Buhler, column 20, lines 51-54: “Referring next to FIG. 7C, the method 700 also includes encoding (724) the feature vector as a similarity hash (e.g., the hash patterns 112 generated by the hashing engine 328) by quantizing each component. Examples of hash generation are described above in the section on Semantic Similarity Hashes. Some implementations map (726) each component to a corresponding integer value of a predetermined length using a linear scale. In some implementations, the predetermined length is (728) 4 bits. In some implementations, the method 700 includes, prior to quantizing, clamping (730) each component to a floating-point value between 0.0 and 1.0. For example, a value below 0.0 is normalized to a floating-point value of 0.0, and a value above 1.0 is normalized to a floating-point value of 1.0. In some implementations, the range for clamping is dynamically determined, or determined based on profiling one or more applications. Some implementations concatenate (734) the integer values to form a string. Some implementations map (732) each component to a corresponding integer value of a predetermined length using a corresponding scale (instead of a linear scale or map).” – Encoding the feature vector as a similarity has is analogous to computing a hash code of the incoming message. The different methods of quantizing, clamping, or concatenating the feature vector to generate the has is analogous to a subset of the numeric values.)
retrieving, as a function of the hash code, one or more matching templates of a set of previously stored templates; (Buhler, column 21, lines 5-14: “Referring next to FIG. 7D, the method 700 also includes performing (734) a sequence of steps for each reference image in a plurality of reference images (e.g., the images in the image library 120). The sequence of steps includes obtaining (736) a reference hash (e.g., from the hash patterns 112) corresponding to the respective reference image. The sequence of steps also includes computing (738) similarity between the input image and the respective reference image by computing a distance between the reference hash and the similarity hash.” – The reference hash corresponding to the reference image is analogous to the stored templates. Performing a similarity between the reference has is analogous to retrieving one or more matching templates.)
organizing the matching templates into a hierarchical structure as a function of the ontological structures; and (Buhler, column 21, lines 23-29: “In some implementations, the method 700 further includes grouping (746) the input image with one or more images of the plurality of reference images that are similar to the input image to form a cluster of images, as illustrated in FIGS. 6A-6D. Some implementations also assign a label to the cluster. Examples of clustering in hash space are described above in the section on Clustering in Hash Space.” Clustering the images with the one or more reference images that are similar is analogous to organizing the matching templates into a hierarchical structure.)
Buhler does not explicitly teach:
wherein the subset is identified based on locations of the one or more ontological structures within the graphical content and relative positions among the one or more ontological structures within the graphical content;
assigning the incoming message to an appropriate responder of a plurality of responders as a function of a semantic meaning associated with the matching templates.
However, Bloch teaches:
wherein the subset is identified based on locations of the one or more ontological structures within the graphical content and relative positions among the one or more ontological structures within the graphical content; (Bloch, page 1564, column 1, paragraph 2: “The use of fuzzy approaches for representing directional relative position allows to integrate both quantitative and qualitative knowledge, using the semiquantitative interpretation of fuzzy sets.”, page 1571, column 1, paragraph 1: “A fuzzy landscape is first defined around the reference object R as a fuzzy set such that the membership value of each point corresponds to the degree of satisfaction of the spatial relation under examination…Then the object A is compared to the fuzzy landscape attached to R, in order to evaluate how well the object matches with the areas having high membership values (i.e. areas that are in the desired direction).” – The fuzzy approaches for representing directional relative positions and th4e comparisons to the landscape of the reference R to comprise the sets is analogous to the subset being identified based on locations of one or more ontological structures within the graphical content and the relative positions among those ontological structures.)
Bloch is considered analogous to the claimed invention as it is in the same field of endeavor, machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified Buhler, which already teaches computing has codes of the incoming message by performing a hash function upon a subset of numeric values but does not explicitly teach that the subset is identified based off locations of ontological structures and relative positions among those ontological structures, to include the teachings of Bloch which does teach that the subset is identified based off locations of ontological structures and relative positions among those ontological structures in order to "recognize objects based on comparison between the characteristics of objects in the scene and objects in the model, and on comparison between relationships of groups of two or more objects in the scene and in the model." (Bloch, page 1563, column 1, paragraph 1)
Buhler and Bloch do not explicitly teach:
assigning the incoming message to an appropriate responder of a plurality of responders as a function of a semantic meaning associated with the matching templates.
However, Mandal teaches:
assigning the incoming message to an appropriate responder of a plurality of responders as a function of a semantic meaning associated with the matching templates. (Mandal, abstract: “In this paper, we present an end-to-end automated helpdesk email ticket assignment system, which is also offered as a service. The objective of the system is to determine the nature of the problem mentioned in an incoming email ticket and then automatically dispatch it to an appropriate resolver group (or team) for resolution.” – Buhler performs the semantic similarity (see Buhler, column 19, lines 6-16) which matches the semantic meaning of the incoming message with the matching template. This is analogous to Mandal’s determining the nature of the problem which then gets dispatched to an appropriate resolver group which is analogous to assigning the incoming message to an appropriate responder.)
Mandal is considered analogous to the claimed invention as it is in the same field of endeavor, machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified Buhler and Bloch, which already teaches an image classification system that classifies incoming documents and messages but does not explicitly teach that the classified documents and messages are being routed to an appropriate responder, to include the teachings of Mandal which does teach that the classified documents and messages are being routed to an appropriate responder in order to "achieve human level accuracy with more than 90% coverage on all the datasets with the proposed system using minimal computational resources." (Mandal, page 2, section 1.1, paragraph 1)
Regarding claim 4, Buhler, Bloch, and Mandal teach the system of claim 1, as cited above.
Buhler and Bloch do not explicitly teach:
the semantic meaning comprises information from which the reporting of the error condition can be inferred; and
the responder is a technical-support resource configured to respond to the error condition.
However, Mandal further teaches:
the semantic meaning comprises information from which the reporting of the error condition can be inferred; and the responder is a technical-support resource configured to respond to the error condition. (Mandal, page 2, paragraph 1: “The dispatch of a ticket to the correct group of practitioners is a critical step in the speedy resolution of a ticket. Incorrect dispatch decisions can significantly increase the total turnaround time for ticket resolution, as observed in a study of an actual production system [2]. Several factors make the dispatcher’s job challenging such as requirement of knowledge of the IT portfolio being managed, roles and responsibilities of the individual groups, ability to quickly parse the ticket text describing the problem and map it to the right group, which is often not straightforward given the heterogeneous and informal nature of the problem description.” – The IT portfolio being managed would include IT services. Thus, parsing the ticket text describing the problem and dispatching to the correct group of practitioners would be indicative that the IT services needed would include the semantic meaning comprising information reporting of error conditions. Thus, the ticket being dispatched to the correct group of practitioners is analogous to the responder being a technical-support resource.)
Regarding claim 5, Buhler, Bloch, and Mandal teach the system of claim 1, as cited above.
Buhler further teaches:
wherein the identified characteristics comprise an absolute location, within the graphical content, of a first structure of the one or more ontological structures. (Buhler, column 27, lines 20-29: “Some implementations use intra-document locational data generated from the OCR process to reconnect labels separated from their respective data. Some implementations use data returned from the OCR process (e.g., polygon vertex coordinates) specifying the location of the discovered text. Some implementations iteratively extend polygons (e.g., from output returned by the OCR process). For example, a polygon is extended in the direction of text until the polygon intersects with other OCR polygons and “join” the text from intersecting OCR polygons.” – The intra-document locational data specifying the location of the discovered text is analogous to the identified characteristics comprising an absolute location of a first structure, e.g., the location of the discovered text is the absolute location of a first structure.)
Regarding claim 6, Buhler, Bloch, and Mandal teach the system of claim 1, as cited above.
Buhler does not explicitly teach:
wherein the identified characteristics comprise a set of relative locations, within the graphical content, of a plurality of structures of the one or more ontological structures.
However, Bloch further teaches:
wherein the identified characteristics comprise a set of relative locations, within the graphical content, of a plurality of structures of the one or more ontological structures. (Bloch page 1, column 2, paragraph 1: “In this paper, we consider only directional relative position, which provides an important information about the spatial arrangement of objects in the scene.” – The relative position providing important information about the spatial arrangement of objects in the scene is analogous to the set of relative locations where the objects in the scene are the one or more ontological structures.)
Regarding claim 7, Buhler, Bloch, and Mandal teach the system of claim 1, as cited above.
Buhler further teaches:
wherein the identified characteristics characterize a graphical element comprised by a first structure of the one or more ontological structures. (Buhler, column 27, lines 20-29: “Some implementations use intra-document locational data generated from the OCR process to reconnect labels separated from their respective data. Some implementations use data returned from the OCR process (e.g., polygon vertex coordinates) specifying the location of the discovered text. Some implementations iteratively extend polygons (e.g., from output returned by the OCR process). For example, a polygon is extended in the direction of text until the polygon intersects with other OCR polygons and “join” the text from intersecting OCR polygons.” – The location of the discovered text is the graphical element comprised by a first structure of one or more ontological structures.)
Regarding claim 8, Claim 8 has all the same limitations of claim 1 which are taught by Buhler, Bloch, and Mandal – see claim 1 above.
Regarding claim 10, Buhler, Bloch, and Mandal teach the method of claim 8, as cited above.
Claim 10 additionally has the same limitations of claim 4 which are taught by Buhler, Bloch, and Mandal – see claim 4 above.
Regarding claim 11, Buhler, Bloch, and Mandal teach the method of claim 8, as cited above.
Claim 11 additionally has the same limitations of claim 5 which are taught by Buhler, Bloch, and Mandal – see claim 5 above.
Regarding claim 12, Buhler, Bloch, and Mandal teach the method of claim 8, as cited above.
Claim 12 additionally has the same limitations of claim 6 which are taught by Buhler, Bloch, and Mandal – see claim 6 above.
Regarding claim 13, Buhler, Bloch, and Mandal teach the method of claim 8, as cited above.
Claim 13 additionally has the same limitations of claim 7 which are taught by Buhler, Bloch, and Mandal – see claim 7 above.
Regarding claim 14, Buhler, Bloch, and Mandal teach the method of claim 13, as cited above.
Buhler further teaches:
providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable program code in the computer system, wherein the computer-readable program code in combination with the computer system is configured to implement the intercepting, the identifying, the associating, the computing, the retrieving, and the assigning. (Buhler, column 8, lines 26-32: “In some implementations, the memory 214, or the computer readable storage medium of the memory 214, stores the following programs, modules, and data structures, or a subset thereof: an operating system 216, which includes procedures for handling various basic system services and for per forming hardware dependent tasks;” and column 8, lines 46-61: “an image or document search or organizer application 106, which enables a user to search and retrieve, or organize images or documents from one or more remote image or document libraries 120 and/or a local image or document library 234. The search or organizer application 106 provides a user interface 224. The image or document search application 106 also includes a retrieval module 226, which retrieves images or documents corresponding to a match identified by the server 110. The image or document search application 106 accesses one or more sample images or documents 108, which can be selected and/or identified by a user to be the basis for the search (e.g., to match the sample image or document 108 with one or more images or documents in remote image or document libraries 120 or a local image or document library 234).” – The basic system services stored on a computer readable storage medium is analogous to the computer readable program code hosted in the computer system. The services include searching, organizing, and retrieving documents from one or more remote image or document libraries implementing the steps recited.)
Regarding claim 15, Claim 15 has all the same limitations of claim 1 which are taught by Buhler, Bloch, and Mandal – see claim 1 above.
Buhler further teaches:
A computer program product including one or more computer readable storage mediums collectively storing program instructions for a self-learning ontology-based cognitive assignment engine that are executable by a processor or programmable circuitry to cause the processor or programmable circuitry to perform operations comprising: (Buhler, column 8, lines 26-32: “In some implementations, the memory 214, or the computer readable storage medium of the memory 214, stores the following programs, modules, and data structures, or a subset thereof: an operating system 216, which includes procedures for handling various basic system services and for per forming hardware dependent tasks;”)
Regarding claim 16, Buhler, Bloch, and Mandal teach the computer program product of claim 15, as cited above.
Claim 16 additionally has the same limitations of claim 4 which are taught by Buhler, Bloch, and Mandal – see claim 4 above.
Regarding claim 17, Buhler, Bloch, and Mandal teach the computer program product of claim 15, as cited above.
Claim 17 additionally has the same limitations of claim 5 which are taught by Buhler, Bloch, and Mandal – see claim 5 above.
Regarding claim 18, Buhler, Bloch, and Mandal teach the computer program product of claim 15, as cited above.
Claim 18 additionally has the same limitations of claim 6 which are taught by Buhler, Bloch, and Mandal – see claim 6 above.
Regarding claim 19, Buhler, Bloch, and Mandal teach the computer program product of claim 15, as cited above.
Claim 19 additionally has the same limitations of claim 7 which are taught by Buhler, Bloch, and Mandal – see claim 7 above.
Regarding claim 20, Buhler, Bloch, and Mandal teach the computer program product of claim 19, as cited above.
Claim 20 additionally has the same limitations of claim 14 which are taught by Buhler, Bloch, and Mandal – see claim 14 above.
Claims 2, 3, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Buhler in view of Bloch in view of Mandal in view of Kulis et al. (Kernelized Locality-Sensitive Hashing for Scalable Image Search), hereinafter Kulis.
Regarding claim 2, Buhler, Bloch, and Mandal teach the system of claim 1, as cited above.
Buhler, Bloch, and Mandal do not explicitly teach:
wherein the hash code is retrieved from a distributed hash table.
However, Kulis teaches:
wherein the hash code is retrieved from a distributed hash table. (Kulis et al., page 2134, section 6, paragraph 2: “Throughout, we present results showing the percentage of database items searched with hashing as opposed to timing results, which are dependent on the particular optimizations of the code. In terms of additional overhead, finding the approximate nearest neighbors given the query hash key is very fast, particularly if the computation can be distributed across several machines (since the random permutations of the hash bits are independent of one another).”)
Kulis is considered analogous to the claimed invention as it is relatively pertinent to the problem faced by the invention of image processing and classification. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified Buhler, Bloch, and Mandal, which already teaches computing the hash codes by performing a hash function but does not explicitly teach that the has codes are stored in a distributed hash table, to include the teachings of Kulis which does teach that the hash codes are stored in a distributed hash table in order to achieve “accurate and fast performance for example-based object classification, feature matching, and content-based retrieval.” (Kulis, abstract)
Regarding claim 3, Buhler, Bloch, Mandal, and Kulis teach the system of claim 2, as cited above.
Buhler, Bloch, and Mandal do not explicitly teach:
retrieving from the distributed hash table a matching value that most closely matches a value of the hash code; and
determining that the matching value is an index value capable of enabling the system to retrieve the matching template.
However, Kulis further teaches:
retrieving from the distributed hash table a matching value that most closely matches a value of the hash code; and (Kulis, page 2132, column 1, paragraph 3: “A query hash key indexes into each sorted order with a binary search, and the 2M nearest examples found are the approximate nearest neighbors.” And paragraph 4: “In this work, the similarity function of interest is an arbitrary kernel function κ: sim(xi,xj)=κ(xi,xj)=φ(xi)Tφ(xj), for some (possibly unknown) embedding function φ(·).” – The binary search finding the examples that are the nearest neighbors is analogous to retrieving the matching value that most closely matches a value of the hash code.)
determining that the matching value is an index value capable of enabling the system to retrieve the matching template. (Kulis, page 2136, column 1, paragraph 1: “However, we can use the data to qualitatively show the kinds of images that are retrieved to quantitatively show how well KLSH approximates a linear scan, and to confirm that our algorithm is amenable to rapidly searching very large image collections.” – The query has indexes discussed above show that the matching value is an index value. Thus, the algorithm searching the collections of images and retrieving them is analogous to using the matching value to retrieve the matching template.)
Regarding claim 9, Buhler, Bloch, and Mandal teach the method of claim 8, as cited above.
Buhler, Bloch, and Mandal do not explicitly teach:
retrieving from a distributed hash table a matching value that most closely matches a value of the hash code; and
determining that the matching value is an index value capable of enabling the system to retrieve the matching template.
However, Kulis teaches:
retrieving from a distributed hash table a matching value that most closely matches a value of the hash code; and (Kulis et al., page 2134, section 6, paragraph 2: “Throughout, we present results showing the percentage of database items searched with hashing as opposed to timing results, which are dependent on the particular optimizations of the code. In terms of additional overhead, finding the approximate nearest neighbors given the query hash key is very fast, particularly if the computation can be distributed across several machines (since the random permutations of the hash bits are independent of one another).” And page 2132, column 1, paragraph 3: “A query hash key indexes into each sorted order with a binary search, and the 2M nearest examples found are the approximate nearest neighbors.” And paragraph 4: “In this work, the similarity function of interest is an arbitrary kernel function κ: sim(xi,xj)=κ(xi,xj)=φ(xi)Tφ(xj), for some (possibly unknown) embedding function φ(·).” – The binary search finding the examples that are the nearest neighbors is analogous to retrieving the matching value that most closely matches a value of the hash code.)
determining that the matching value is an index value capable of enabling the system to retrieve the matching template. (Kulis, page 2136, column 1, paragraph 1: “However, we can use the data to qualitatively show the kinds of images that are retrieved to quantitatively show how well KLSH approximates a linear scan, and to confirm that our algorithm is amenable to rapidly searching very large image collections.” – The query has indexes discussed above show that the matching value is an index value. Thus, the algorithm searching the collections of images and retrieving them is analogous to using the matching value to retrieve the matching template.)
Kulis is considered analogous to the claimed invention as it is relatively pertinent to the problem faced by the invention of image processing and classification. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified Buhler, Bloch, and Mandal, which already teaches the method of using hash codes to determine a matching template for image classification and routing the messages to the appropriate responder but does not explicitly teach retrieving the matching values from a distributed hash table to enable the system to retrieve the matching template, to include the teachings of Kulis which does teach that the hash codes are stored in a distributed hash table in order to achieve “accurate and fast performance for example-based object classification, feature matching, and content-based retrieval.” (Kulis, abstract)
Response to Arguments
Applicant’s arguments, see 11, filed June 8, 2026, with respect to the rejection(s) of claim(s) 1-20 under 35 USC §103 have been fully considered and are persuasive. In particular, examiner agrees that Buhler does not teach “wherein the subset is identified based on locations of the one or more ontological structures within the graphical content and relative positions among the one or more ontological structures within the graphical content;” Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Buhler, Bloch, and Mandal as Bloch, which was previously relied upon to teach the limitations of claims 6, 12, and 18, does teach the above cited limitation. See section Claim Rejections – 35 USC §103 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lin et al. (US 11048966)
Dubey et al. (US 10679008)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACQUELINE MEYER whose telephone number is (703)756-5676. The examiner can normally be reached M-F 8:00 am - 4:30 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571-272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.C.M./Examiner, Art Unit 2144
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144