DETAILED ACTION
Status of the Application
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This communication is a non-final action in response to the communications filed on 2/26/2024. Claims 1-9 are currently pending and have been considered below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “an image acquisition module to acquire”, “a target identification module to subject”, “a positional relationship determining module to determine”, “a knowledge graph generating module to generate” in claims 5-8.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
These limitations have structural support in page 19 of the specification and are interpreted as generic software/hardware.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Claims 1-9 are determined to be directed to an abstract idea.
The claims 1-9 are directed to a judicial exception (i.e., law of nature, natural phenomenon, or abstract idea), without providing a practical application integration and without providing significantly more.
As per Step 1 of the subject matter eligibility analysis, Claims 1-9 are directed to a method (i.e., process) and apparatus which are statutory categories of invention.
As per Step 2A-Prong 1 of the subject matter eligibility analysis, Claims 1, 5 and 9 are directed specifically to the abstract idea of generating a knowledge graph by acquiring an image of a target system; subjecting the image to target identification including obtaining a category of each object in the target system and position information of each object in the image; determining a relative positional relationship between objects in the target system according to the position information of each object in the image; and generating a knowledge graph of the target system according to the relative positional relationship and the category of each object identified; which include mental processes (evaluating object data in an image for a judgement or opinion of positional relationship between the objects), and certain methods of organizing human activity based on fundamental economic practice (managing objects (people and equipment etc.) in a work environment), and based on personal behavior and interactions between people (following rules and instructions for generating a knowledge graph of the target system). Claims 2-4 and 6-8 are directed to the abstract idea of claim 1 or 5 with further details on the parameters/attributes of the abstract idea which includes mental processes and certain methods of organizing human activity for similar reasons as provided above for claim 1 or 5. After considering all claim elements, both individually and in combination and in ordered combination, it has been determined that the claims do not amount to significantly more than the abstract idea itself.
As per Step 2A-Prong 2 of the subject matter eligibility analysis, while the claims 1-9 recite additional limitations which are hardware or software elements, such as an apparatus, modules, a memory storing computer readable code; at least one processor to call the computer readable code and upon execution of the code…, these limitations are not enough to qualify as a practical application being recited in the claims along with the abstract idea since these elements are merely invoked as a tool to apply instructions of an abstract idea in a particular technological environment, and mere application of an abstract idea in a particular technological environment and merely limiting the use of an abstract idea to a particular technological field do not integrate an abstract idea into a practical application (MPEP 2106.05(f)&(h)). The claims do not amount to "practical application" for the abstract idea because they neither (1) recite any improvements to another technology or technical field; (2) recite any improvements to the functioning of the computer itself; (3) apply the judicial exception with, or by use of, a particular machine; (4) effect a transformation or reduction of a particular article to a different state or thing; (5) provide other meaningful limitations beyond generally linking the use of the judicial exception to a particular technological environment.
As per Step 2B of the subject matter eligibility analysis, while the claims 1-9 recite additional limitations which are hardware or software elements, such as an apparatus, modules, a memory storing computer readable code; at least one processor to call the computer readable code and upon execution of the code…, these limitations are not enough to qualify as “significantly more” being recited in the claims along with the abstract idea since these elements are merely invoked as a tool to apply instructions of an abstract idea in a particular technological environment, and mere application of an abstract idea in a particular technological environment and merely limiting the use of an abstract idea to a particular technological field do provide significantly more to an abstract idea (MPEP 2106.05 (f) & (h)). The claims do not amount to "significantly more" than the abstract idea because they neither (1) recite any improvements to another technology or technical field; (2) recite any improvements to the functioning of the computer itself; (3) apply the judicial exception with, or by use of, a particular machine; (4) effect a transformation or reduction of a particular article to a different state or thing; (5) add a specific limitation other than what is well-understood, routine and conventional in the field; (6) add unconventional steps that confine the claim to a particular useful application; nor (7) provide other meaningful limitations beyond generally linking the use of the judicial exception to a particular technological environment.
Therefore, since there are no limitations in the claims 1-9 that transform the exception into a patent eligible application such that the claims amount to significantly more than the exception itself, and looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually, the claims are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shrestha (US 20190347518 A1)
As per Claim 1, Shrestha teaches a method for generating a knowledge graph (Abstract and para. 0045), the method comprising:
acquiring an image of a target system (para. 0022, “In some embodiments, the semantic module 121 includes a multi-task model core that instructed to perform native video deep learning on video data received by a plurality of video cameras (e.g., thousands of video cameras), and generate semantic information by processing video from multiple streams locally, and in real-time.”; para. 0056, “the comprehension system 120 automatically generating the contextual metadata. In some embodiments, contextual metadata includes at least one description (e.g., event description, scene description) generated by the semantic module 121 for data received from a sensor (e.g., image data received form an image data source110), as described herein. In some embodiments, contextual metadata includes at least one description (e.g., event description, scene description) generated by the semantic module 121 for data received from each sensor of the system 100 (e.g., image data received form an image data source110), as described herein.”);
subjecting the image to target identification including obtaining a category of each object in the target system and position information of each object in the image (para. 0029, “The comprehension system 120 functions to analyze and/or process image data input preferably originating from the one or more image data sources no. The semantic module 121 preferably includes a high-level deep learning model (e.g., a convolutional neural network, etc.) 131 that functions to perform edge and/or border detection. Accordingly, the high-level deep learning model may function to extract coarse semantic information from the image data input from the one or more image data sources 110. For instance, the high-level deep learning model implementing an artificial neural network may function to first extract broad scene level data and may generate descriptive metadata tags, such as outdoor, street, traffic, raining, and the like.”; para. 0063, “The spatial intelligence (e.g., the contextual metadata associated with locations within the coordinate space of the rendering) data generated by the method 200 preferably includes insights and data relating to any or all perceivable objects, activities (e.g., situations, circumstances, etc.), persons, and the like. In one or more embodiments, the method 200 produces the spatial intelligence data (contextual metadata, semantic information etc., associated with locations within a rendering) by extracting semantic information from the spatially intelligent sensors. The method 200 may function to intelligently map within the spatial rendering (e.g., by associating with a location within a coordinate space of the rendering) any object, person, or activity identified within the semantic information. The augmentation of the real-time semantic information (e.g., contextual metadata, semantic information) to the spatial rendering of the predefined space (e.g., by associating the data with rendering coordinates, for example, by using a knowledge graph managed by a graph database) enables a live presentation (e.g., included in a user interface, e.g., 125, 126) via the spatial rendering that may function to provide real-time insights surrounding any circumstance or object within the predefined space.”; also see para. 0049-0062; also see figs. 5A-5D),
determining a relative positional relationship between objects in the target system according to the position information of each object in the image (para. 0029, “The comprehension system 120 functions to analyze and/or process image data input preferably originating from the one or more image data sources no. The semantic module 121 preferably includes a high-level deep learning model (e.g., a convolutional neural network, etc.) 131 that functions to perform edge and/or border detection. Accordingly, the high-level deep learning model may function to extract coarse semantic information from the image data input from the one or more image data sources 110. For instance, the high-level deep learning model implementing an artificial neural network may function to first extract broad scene level data and may generate descriptive metadata tags, such as outdoor, street, traffic, raining, and the like.”; para. 0063, “The spatial intelligence (e.g., the contextual metadata associated with locations within the coordinate space of the rendering) data generated by the method 200 preferably includes insights and data relating to any or all perceivable objects, activities (e.g., situations, circumstances, etc.), persons, and the like. In one or more embodiments, the method 200 produces the spatial intelligence data (contextual metadata, semantic information etc., associated with locations within a rendering) by extracting semantic information from the spatially intelligent sensors. The method 200 may function to intelligently map within the spatial rendering (e.g., by associating with a location within a coordinate space of the rendering) any object, person, or activity identified within the semantic information. The augmentation of the real-time semantic information (e.g., contextual metadata, semantic information) to the spatial rendering of the predefined space (e.g., by associating the data with rendering coordinates, for example, by using a knowledge graph managed by a graph database) enables a live presentation (e.g., included in a user interface, e.g., 125, 126) via the spatial rendering that may function to provide real-time insights surrounding any circumstance or object within the predefined space.”; also see para. 0049-0062; also see figs. 5A-5D) and
generating a knowledge graph of the target system according to the relative positional relationship and the category of each object identified (para. 0045, “the comprehension system 120 generating a knowledge graph for the predefined space, and adding the contextual metadata to the knowledge graph in association with the rendering data. In some embodiments, the comprehension system 120 generates the knowledge graph by using the contextual metadata module 127.”; para. 0046, “associating contextual metadata with the rendering data includes: associating a data item of contextual metadata with a location within a coordinate space of a rendering represented by the rendering data 122.”; para. 0029, “The comprehension system 120 functions to analyze and/or process image data input preferably originating from the one or more image data sources no. The semantic module 121 preferably includes a high-level deep learning model (e.g., a convolutional neural network, etc.) 131 that functions to perform edge and/or border detection. Accordingly, the high-level deep learning model may function to extract coarse semantic information from the image data input from the one or more image data sources 110. For instance, the high-level deep learning model implementing an artificial neural network may function to first extract broad scene level data and may generate descriptive metadata tags, such as outdoor, street, traffic, raining, and the like.”; para. 0063, “The spatial intelligence (e.g., the contextual metadata associated with locations within the coordinate space of the rendering) data generated by the method 200 preferably includes insights and data relating to any or all perceivable objects, activities (e.g., situations, circumstances, etc.), persons, and the like. In one or more embodiments, the method 200 produces the spatial intelligence data (contextual metadata, semantic information etc., associated with locations within a rendering) by extracting semantic information from the spatially intelligent sensors. The method 200 may function to intelligently map within the spatial rendering (e.g., by associating with a location within a coordinate space of the rendering) any object, person, or activity identified within the semantic information. The augmentation of the real-time semantic information (e.g., contextual metadata, semantic information) to the spatial rendering of the predefined space (e.g., by associating the data with rendering coordinates, for example, by using a knowledge graph managed by a graph database) enables a live presentation (e.g., included in a user interface, e.g., 125, 126) via the spatial rendering that may function to provide real-time insights surrounding any circumstance or object within the predefined space.”; also see para. 0049-0062).
As per Claim 2, Shrestha teaches a method as provided in claim 1 above. Shrestha further teaches wherein generating a knowledge graph of the target system according to the relative positional relationship and the category of each object identified comprises:
determining the categories of the objects identified as being entities in a knowledge graph of the target system; determining a relationship between entities in the knowledge graph corresponding to objects in the target system according to the relative positional relationship between objects; and generating the knowledge graph of the target system according to the entities in the knowledge graph of the target system and the relationship between the entities (para. 0045, “the comprehension system 120 generating a knowledge graph for the predefined space, and adding the contextual metadata to the knowledge graph in association with the rendering data. In some embodiments, the comprehension system 120 generates the knowledge graph by using the contextual metadata module 127.”; para. 0046, “associating contextual metadata with the rendering data includes: associating a data item of contextual metadata with a location within a coordinate space of a rendering represented by the rendering data 122.”; para. 0029, “The comprehension system 120 functions to analyze and/or process image data input preferably originating from the one or more image data sources no. The semantic module 121 preferably includes a high-level deep learning model (e.g., a convolutional neural network, etc.) 131 that functions to perform edge and/or border detection. Accordingly, the high-level deep learning model may function to extract coarse semantic information from the image data input from the one or more image data sources 110. For instance, the high-level deep learning model implementing an artificial neural network may function to first extract broad scene level data and may generate descriptive metadata tags, such as outdoor, street, traffic, raining, and the like.”; para. 0063, “The spatial intelligence (e.g., the contextual metadata associated with locations within the coordinate space of the rendering) data generated by the method 200 preferably includes insights and data relating to any or all perceivable objects, activities (e.g., situations, circumstances, etc.), persons, and the like. In one or more embodiments, the method 200 produces the spatial intelligence data (contextual metadata, semantic information etc., associated with locations within a rendering) by extracting semantic information from the spatially intelligent sensors. The method 200 may function to intelligently map within the spatial rendering (e.g., by associating with a location within a coordinate space of the rendering) any object, person, or activity identified within the semantic information. The augmentation of the real-time semantic information (e.g., contextual metadata, semantic information) to the spatial rendering of the predefined space (e.g., by associating the data with rendering coordinates, for example, by using a knowledge graph managed by a graph database) enables a live presentation (e.g., included in a user interface, e.g., 125, 126) via the spatial rendering that may function to provide real-time insights surrounding any circumstance or object within the predefined space.”; also see para. 0049-0062; also see figs. 5A-5D).
As per Claim 3, Shrestha teaches a method as provided in claim 1 above. Shrestha further teaches acquiring common knowledge of a relationship between the identified objects; wherein determining a relationship between entities in a knowledge graph corresponding to objects in the target system according to the relative positional relationship between objects, comprises: determining a relationship between entities in the knowledge graph according to the relative positional relationship between identified objects and the acquired common knowledge (para. 0045, “the comprehension system 120 generating a knowledge graph for the predefined space, and adding the contextual metadata to the knowledge graph in association with the rendering data. In some embodiments, the comprehension system 120 generates the knowledge graph by using the contextual metadata module 127.”; para. 0046, “associating contextual metadata with the rendering data includes: associating a data item of contextual metadata with a location within a coordinate space of a rendering represented by the rendering data 122.”; para. 0029, “The comprehension system 120 functions to analyze and/or process image data input preferably originating from the one or more image data sources no. The semantic module 121 preferably includes a high-level deep learning model (e.g., a convolutional neural network, etc.) 131 that functions to perform edge and/or border detection. Accordingly, the high-level deep learning model may function to extract coarse semantic information from the image data input from the one or more image data sources 110. For instance, the high-level deep learning model implementing an artificial neural network may function to first extract broad scene level data and may generate descriptive metadata tags, such as outdoor, street, traffic, raining, and the like.”; para. 0063, “The spatial intelligence (e.g., the contextual metadata associated with locations within the coordinate space of the rendering) data generated by the method 200 preferably includes insights and data relating to any or all perceivable objects, activities (e.g., situations, circumstances, etc.), persons, and the like. In one or more embodiments, the method 200 produces the spatial intelligence data (contextual metadata, semantic information etc., associated with locations within a rendering) by extracting semantic information from the spatially intelligent sensors. The method 200 may function to intelligently map within the spatial rendering (e.g., by associating with a location within a coordinate space of the rendering) any object, person, or activity identified within the semantic information. The augmentation of the real-time semantic information (e.g., contextual metadata, semantic information) to the spatial rendering of the predefined space (e.g., by associating the data with rendering coordinates, for example, by using a knowledge graph managed by a graph database) enables a live presentation (e.g., included in a user interface, e.g., 125, 126) via the spatial rendering that may function to provide real-time insights surrounding any circumstance or object within the predefined space.”; also see para. 0049-0062; also see figs. 5A-5D and fig. 7).
As per Claim 4, Shrestha teaches a method as provided in claim 1 above. Shrestha further teaches the target system comprising a factory, a production line, or a processing step (Figs. 5A-5D and fig. 7).
As per Claims 5-8, Claims 5-8 recite substantially similar limitations as claims 1-4, respectively; therefore, claims 5-8 are rejected with the same reasoning and rationale provided above for claims 1-4, respectively.
As per Claim 9, Claim 9 recites substantially similar limitations as claim 1; therefore, claim 9 is rejected with the same reasoning and rationale provided above for claim 1.
As per claim 9, Shrestha teaches a memory storing computer readable code;
at least one processor to call the computer readable code and upon execution of the code (para. 0118, 0121, 0124, 0127)
Conclusion
Additional relevant art not relied upon, specifically related to blockchain and PBFT combination, includes:
Silverman (US 20200090053 A1), regarding “are disclosed. In one embodiment, in an information processing apparatus comprising at least one computer processor a method for generating a knowledge graph may include: (1) receiving data from at least one data source; (2) identifying facts in the data; (3) generating a relationship triple for each fact, wherein the relationship triples identify a subject, a predicate, and an object; and (4) populating the knowledge graph comprising nodes and edges with the relationship triples, wherein the nodes represent the subjects and the objects, and the edges represent the predicates, wherein each edge is associated with a weighting indicating a strength of the predicate relationship between the subject and the object.”;
Tokarev Sela (US 20200372373 A1), regarding “A system and method for generating a semantic graph. The method includes: parsing each of a plurality of events into a plurality of objects, wherein the plurality of events includes a plurality of queries, wherein each event of the plurality of events is related to an interaction with at least one data source; determining, for each of the plurality of events, a relationship between two objects of the plurality of objects; and generating a semantic knowledge graph based on the determined relationships, the semantic knowledge graph including a plurality of query nodes and a plurality of edges, wherein each query node corresponds to a respective object of the plurality of objects, wherein each query node is connected to another query node of the plurality of query nodes by one of the plurality of edges, wherein each edge represents a relationship between the objects connected by the edge.”;
Lee (US 20140211044 A1), regarding “A system for generating crowdsourcing-based image knowledge content, includes an image provision unit configured to provide image data; and an image crowdsourcing unit configured to generate, store, or manage image knowledge in order to provide an image-base knowledge service. Further, the system includes an image acquisition and processing unit configured to connect the image provision unit and the image crowdsourcing unit and actively or automatically provide a crowdsourcing technique.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEHMET YESILDAG whose telephone number is (571)272-3257. The examiner can normally be reached M-F 8:30 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O'Connor can be reached on (571) 272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Sincerely,
/MEHMET YESILDAG/Primary Examiner, Art Unit 3624
Prosecution Insights