Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
Claims 1-6 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Regarding to claims 1-4
Claim 1
A method of searching vectors for multimodal data in a server, comprising:
generating a first vector index structure for first modality data;
generating a second vector index structure for second modality data different from the first modality data;
connecting the first vector index structure and the second vector index structure; and
searching for a node similar to a query vector using the connected first vector index structure and second vector index structure.
Step 1, This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP 2106.03. The claim recites at least one step or act, including steps a) -dh). Thus, the claim is to a process/method, which is one of the statutory categories of invention. (Step 1: YES).
Step 2A – Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Step a) generating a first vector index structure for first modality data and b) generating a second vector index structure for second modality data different from the first modality data. Step a) and b) simply mapping first and second modality data to first and second vector index structure. Both of steps can be done by human based on observations, evaluations, judgments that can be performed in human mind (i.e., a mental process [Wingdings font/0xF3] abstract idea)
Step c) connecting the first vector index structure and the second vector index structure. Connecting is simply associating one thing to another thing. This step can be done by human based on observations, evaluations, judgments that can be performed in human mind (i.e., a mental process [Wingdings font/0xF3] abstract idea)
Step d) searching for a node similar to a query vector using the connected first vector index structure and second vector index structure. Identify an object similar to another object is observations, evaluations, judgments that can be performed in human mind (i.e., a mental process [Wingdings font/0xF3] abstract idea)
“Unless it is clear that a claim recites distinct exceptions, such as a law of nature and an abstract idea, care should be taken not to parse the claim into multiple exceptions, particularly in claims involving abstract ideas.” MPEP 2106.04, subsection II.B. However, if possible, the examiner should consider the limitations together as a single abstract idea rather than as a plurality of separate abstract ideas to be analyzed individually. “For example, in a claim that includes a series of steps that recite mental steps as well as a mathematical calculation, an examiner should identify the claim as reciting both a mental process and a mathematical concept for Step 2A, Prong One to make the analysis clear on the record.” MPEP 2106.04, subsection II.B. Under such circumstances, however, the Supreme Court has treated such claims in the same manner as claims reciting a single judicial exception. Id. (discussing Bilski v. Kappos, 561 U.S. 593 (2010)). Here, steps a-d fall within the mental process grouping of abstract ideas. Limitations (b) - (f) are considered together as a single abstract idea for further analysis. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d).
The claim recites the additional elements/limitations “a server”
a) MPEP § 2106.05(a) "Improvements to the Functioning of a Computer or to Any Other Technology or Technical Field."
There is no improvement to Functioning of a Computer or to Any Other Technology or Technical Field. The limitation “server” does not make any improvements to the functionalities of a computer, database technology, or any other technologies.
b) MPEP § 2106.05(b) Particular Machine. The judicial exception does not apply to any particular machine.
The claim is silent regarding specific limitations directed to an improved computer system, processor, memory, network, database, or Internet, nor do applicant direct examiner’s attention to such specific limitations. "[T]he mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention." Alice, 573 U.S. at 223; see also Bascom Glob. Internet Servs., Inc. v. AT&T Mobility LLC, 827 F.3d 1341, 1348 (Fed. Cir. 2016) ("An abstract idea on 'an Internet computer network' or on a generic computer is still an abstract idea."). Applying this reasoning here, the claim is not directed to a particular machine, but rather merely implement an abstract idea using generic computer components such as “server.” Thus, the claims fail to satisfy the "tied to a particular machine" prong of the Bilski machine-or-transformation test.
c) MPEP § 2106.05(c) Particular Transformation.
The claim operates to mapping data to vector and comparing/matching data based on similarities. The steps are not a "transformation or reduction of an article into a different state or thing constituting patent-eligible subject matter[.]" See In re Bilski, 545 F.3d 943, 962 (Fed. Cir. 2008) (en bane), aff'd sub nom, Bilski v. Kappas, 561 U.S. 593 (2010); see also CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1375 (Fed. Cir. 2011) ("The mere manipulation or reorganization of data ... does not satisfy the transformation prong."). Applying this guidance here, the claims fail to satisfy the transformation prong of the Bilski machine-or-transformation test.
d) MPEP § 2106.05(e) Other Meaningful Limitations.
This section of the MPEP guides: Diamond v. Diehr provides an example of a claim that recited meaningful limitations beyond generally linking the use of the judicial exception to a particular technological environment. 450 U.S. 175, ... (1981). In Diehr, the claim was directed to the use of the Arrhenius equation ( an abstract idea or law of nature) in an automated process for operating a rubber-molding press. 450 U.S. at 177-78 .... The Court evaluated additional elements such as the steps of installing rubber in a press, closing the mold, constantly measuring the temperature in the mold, and automatically opening the press at the proper time, and found them to be meaningful because they sufficiently limited the use of the mathematical equation to the practical application of molding rubber products. 450 U.S. at 184... In contrast, the claims in Alice Corp. v. CLS Bank International did not meaningfully limit the abstract idea of mitigating settlement risk. 573 U.S._ .... In particular, the Court concluded that the additional elements such as the data processing system and communications controllers recited in the system claims did not meaningfully limit the abstract idea because they merely linked the use of the abstract idea to a particular technological environment (i.e., "implementation via computers") or were well-understood, routine, conventional activity. MPEP § 2106.05(e).
The limitation “server” is not meaningful limitations.
e) MPEP § 2106.05(g) Insignificant Extra-Solution Activity.
The limitations “server” is not meaningful limitation.
6) MPEP § 2106.05(h) Field of Use and Technological Environment.
[T]he Supreme Court has stated that, even if a claim does not wholly pre-empt an abstract idea, it still will not be limited meaningfully if it contains only insignificant or token pre- or post-solution activity-such as identifying a relevant audience, a category of use, field of use, or technological environment. Ultramercial, Inc. v. Hulu, LLC, 722 F.3d 1335, 1346 (Fed. Cir. 2013). “server” limitation is simply a field of use that attempts to limit the abstract idea to a particular technological environment.
Accordingly, the additional limitation “server” does not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not recite any non-convention or non-generic arrangement because collecting data, ranking collected data, and displaying the results are all conventional activities. Taking these limitations as an ordered combination adds nothing that is not already present when the elements are taken individually. Therefore, the claim does not amount to significantly more than the recited abstract idea. The claim is not patent eligible.
Claim 2 depends on claim 1 and includes all the limitations of claim 1. Claim 2 recites “wherein the generating of the second vector index structure includes extracting a vector representation for the second modality data from the same embedding space as the first vector index structure to generate the second vector index structure.” This limitation is pre-solution activities. The claim does not have any addition limitation that amount to significantly more than the abstract idea.
Claim 3 depends on claim 1 and includes all the limitations of claim 1. Claim 3 recites “wherein the connecting of the first vector index structure and the second vector index structure includes connecting all target nodes of the first vector index structure and the second vector index structure.” This limitation is pre-solution activities. The claim does not have any addition limitation that amount to significantly more than the abstract idea.
Claim 4 depends on claim 1 and includes all the limitations of claim 1. Claim 4 recites “wherein the connecting of the first vector index structure and the second vector index structure includes randomly extracting all target nodes of the first vector index structure and the second vector index structure and connecting the extracted target nodes.” This limitation is pre-solution activities. The claim does not have any addition limitation that amount to significantly more than the abstract idea.
Claim 5 and 6 are similar to claim 1. The claim 1 is rejected based on the same reason.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6 are rejected under 35 U.S.C. 103 as being unpatentable over Xu (U.S. Pub 2023/0103305 A1), in view of Maheshwari (U.S. Pub 2022/0391433 A1)
Claim 1
Xu discloses a method of searching vectors for multimodal data in a server, comprising:
generating a first vector index structure for first modality data ([0014], line 2-7, “… the graph matching system obtains ungrounded label graphs… corresponding to digital images… by utilizing a natural language parsing model to extract entity labels from a description of the digital image (e.g., a caption for the digital image)…” <examiner note: first modality data [Wingdings font/0xF3] text caption for the digital image> [0015], line 4-6, “… utilizes a label embedding model to generate label graph embeddings from entity labels in the ungrounded label graph…” <examiner note: first vector index structure [Wingdings font/0xF3] label graph embeddings>);
generating a second vector index structure for second modality data different from the first modality data ([0014], line 13-16, “… the graph matching system obtains a visual graph for the digital image by utilizing digital image processing (e.g., object detection) to detect entity bounding regions within the digital image…” <examiner note: second modality data [Wingdings font/0xF3] image data> [0015], line 9-11, “… utilizes a visual embedding model to generate visual graph embeddings from entity bounding regions in the visual graph…” <examiner note: second vector index structure [Wingdings font/0xF3] visual embedding model>);
connecting the first vector index structure and the second vector index structure ([0017], line 5-16, “… the graph matching system… determine correspondences between the label graph embeddings and the visual graph embeddings…. Such as Hungarian algorithm to align the nodes of the ungrounded label graph and nodes of the visual graph according to the similarity metrics. By aligning the nodes of the ungrounded label graph with the nodes of the visual graph, the graph matching system generates a semantic scene graph for the digital image…” <examiner note: The Hungarian algorithm is used to align/connect nodes between two data sets [Wingdings font/0xF3] a semantic scene graph)
Xu discloses the digital image processing system 112 processes digital images for one or more visual reasoning tasks… for visual question answering, digital image captioning, digital image editing or reconstruction, digital image generation, training machine-learning models (e.g., via dataset augmentation), or image classification… the digital image processing system 112 provides results of the tasks (e.g., responses for visual question answering, digital image captions, edited/generated digital images, or trained models) for use or display at the client device 106 via the client application 114.
However, Xe does not explicitly disclose searching for a node similar to a query vector using the connected first vector index structure and second vector index structure.
Maheswari discloses searching for a node similar to a query vector using the connected first vector index structure and second vector index structure ([0049, “… the image processing apparatus leverages structured representations of images (i.e., scene graphs 405). Scene graphs 405 concisely summarize the semantics of the scene in the image, with the nodes corresponding to objects in the image, and edges denoting the relationship between them…” [0040], line 1-2, “… the system generates a scene graph of the query image…” [0046], line 10-12, “… the query image 300 includes a first person jumping over a fire hydrant and a second person standing behind the first person…” fig. 3 similar images 320 also includes objects/nodes a person, a fire hydrant…” <examiner note: a person, fire hydrant are objects/nodes in the query image are similar to person and fire hydrant nodes in search results>)
Xu discloses a semantic scene graph is generated for one or more visual reasoning tasks… for visual question answering, digital image captioning, digital image editing or reconstruction, digital image generation, training machine-learning models (e.g., via dataset augmentation), or image classification… However, Xu does not explicitly disclose searching for a node similar to a query vector using the connected first vector index structure and second vector index structure. Maheswari also disclose a semantic scene graph is generated for each image and query for retrieval search objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Maheswari into Xu for visual question answering because Maheswari’s teaching can be used for semantic image retrieval by capturing high-order concepts based on a set of objects in the image and their relationship.
Claim 2
Claim 1 is included, Xu further discloses wherein the generating of the second vector index structure includes extracting a vector representation for the second modality data from the same embedding space as the first vector index structure to generate the second vector index structure ([0016], “… the graph matching system matches nodes from the ungrounded label graph to nodes of the visual graph. For example, the graph matching system matches nodes by determining similarities between pairs of label graph embeddings and visual graph embeddings. In particular, the graph matching system determines a similarity metric (e.g., a cosine similarity metric) between a label graph embedding and a visual graph embedding…” <examiner note: obviously, the label graph embeddings and visual graph embeddings are in the same embedding space in order to determine similarity metric/cosine similarity metric>)
Claim 3
Claim 1 is included, Maheswari discloses wherein the connecting of the first vector index structure and the second vector index structure includes connecting all target nodes of the first vector index structure and the second vector index structure (fig. 4)
PNG
media_image1.png
637
1021
media_image1.png
Greyscale
Claim 4
Claim 1 is included, Maheswari discloses wherein the connecting of the first vector index structure and the second vector index structure includes randomly extracting all target nodes of the first vector index structure and the second vector index structure and connecting the extracted target nodes (fig. 4, for instance, extracted target nodes a girl a racket in visual graph is connected to entity label girl and racket in the label graph.)
Claims 5 and 6 are similar to claim 1. The claim is rejected based on the same reason.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAU HAI HOANG whose telephone number is (571)270-5894. The examiner can normally be reached 1st biwk: Mon-Thurs 7:00 AM-5:00 PM; 2nd biwk: Mon-Thurs: 7:00 am-5:00pm, Fri: 7:00 am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at 571-270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
HAU HAI. HOANG
Primary Examiner
Art Unit 2154
/HAU H HOANG/Primary Examiner, Art Unit 2154