DETAILED ACTION
This action is written in response to the application filed 10/2/23. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Subject Matter Eligibility
In determining whether the claims are subject matter eligible, the examiner has considered and applied guidance from MPEP § 2106. The examiner finds that the independent claims are directed to the practical application of training an encoder and decoder to generate—for a particular input mathematical expression—a mathematically equivalent output expressions. Furthermore, the combination of steps performed in the recited method cannot be practically performed as a mental process.
Claim Rejections - 35 USC § 112
The following is a quotation of the second paragraph of 35 U.S.C. 112:
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 16-17 are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which applicant regards as the invention.
Dependent claims 16 and 17 each depend upon claim 21, which doesn’t exist. Therefore, these claims are indefinite. For the purpose of examination with respect to the prior art, the examiner will interpret these claims as if they instead depended upon independent claim 13.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 13, 16 and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang.
Regarding claim 1, Zhang discloses a computer-implemented method comprising:
obtaining a training dataset, wherein the training dataset contains a plurality of sets of mathematical expressions, wherein each set of mathematical expressions of the plurality of sets of mathematical expressions includes two or more mathematically equivalent but not identical mathematical expressions; and
P. 1, fig. 1 (b) and (c) (reproduced below). The examiner notes that each equation comprises two mathematical expressions which are equivalent but not identical.
PNG
media_image1.png
140
378
media_image1.png
Greyscale
See also p. 5, sec. 4.2 discussion training data used.
using the training dataset, training an encoder and a decoder to generate, as an output of the decoder, an output mathematical expression that is mathematically equivalent to but not identical to an input mathematical expression that is applied as an input to the encoder, wherein the encoder generates, as an output that is provided as an input to the decoder, a continuous vector that is representative of the input mathematical expression.
P. 2, fig. 2 (reproduced below).
PNG
media_image2.png
492
526
media_image2.png
Greyscale
‘output’ :: P. 3, “Finally, the output of the decoder is LaTeX strings.”
Regarding claim 2, Zhang discloses the further limitation wherein training the encoder and the decoder comprises:
parsing the input mathematical expression into an input ordered sequence of mathematical symbols; and
Id.
‘ordered sequence of mathematical symbols’ :: fig. 2, “The first stage: location and classification”.
applying each of the mathematical symbols of the input ordered sequence of mathematical symbols to a mapping function of the encoder to generate respective embedding vectors, thereby generating an ordered sequence of embedding vectors that represent the input ordered sequence of mathematical symbols in an embedding space.
Id. ‘encoder’.
Regarding claim 13, Zhang discloses a method comprising:
obtaining a target mathematical expression; and
P. 2, fig. 2 (reproduced below).
PNG
media_image2.png
492
526
media_image2.png
Greyscale
applying the target mathematical expression as an input to an encoder to generate a target continuous vector that is representative of the target mathematical expression, wherein the encoder has been trained by:
Id.
‘target continuous vector’ :: “x ^ 2 + y ^ 2 = 1 2 5 \div 5”
obtaining a training dataset, wherein the training dataset contains a plurality of sets of mathematical expressions, wherein each set of mathematical expressions of the plurality of sets of mathematical expressions includes two or more mathematically equivalent but not identical mathematical expressions; and
P. 1, fig. 1 (b) and (c) (reproduced below). The examiner notes that each equation comprises two mathematical expressions which are equivalent but not identical.
PNG
media_image1.png
140
378
media_image1.png
Greyscale
See also p. 5, sec. 4.2 discussion training data used.
using the training dataset, training the encoder and a decoder to generate, as an output of the decoder, an output mathematical expression that is mathematically equivalent to but not identical to an input mathematical expression that is applied as an input to the encoder, wherein the encoder generates, as an output that is provided as an input to the decoder, a continuous vector that is representative of the input mathematical expression.
P. 2, fig. 2 (reproduced above).
‘output’ :: P. 3, “Finally, the output of the decoder is LaTeX strings.”
Regarding claim 16, Zhang discloses the further limitation wherein obtaining the plurality of continuous vectors that represent the plurality of additional mathematical expressions comprises applying the plurality of additional mathematical expressions to the encoder to generate the plurality of continuous vectors.
The Examiner notes that the encoder and decoder models described throughout Zhang are trained iteratively over each instance in the training dataset. See generally p. 2477, sec. C (‘Dataset’) and V (‘Experiments’).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The following are the references relied upon in the rejections below:
Dhar (Dhar, Sourish, Sudipta Roy, and Arnab Paul. "Scientific document retrieval using structure encoded string with trie indexing." Information Services and Use 42.2 (2022): 241-259.)
Jianshu Zhang (Zhang, Jianshu, et al. "SRD: a tree structure based decoder for online handwritten mathematical expression recognition." IEEE transactions on multimedia 23 (2020): 2471-2480.)
Zhang (Zhang, Jin, Weipeng Ming, and Pengfei Liu. "A two-stage framework for mathematical expression recognition." OpenReview.net, ICLR 2020 conference blind submission, published online 25 Sept 2019.)
Claims 3 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Dhar.
Regarding claim 3, Dhar discloses the following further limitation which Zhang does not disclose wherein parsing the input mathematical expression into the input ordered sequence of mathematical symbols comprises generating the input ordered sequence of mathematical symbols to represent the input mathematical expression according to the reverse Polish notation.
P. 242, “In Egomath2 (designed by Jozef Misutka as an extended version of Egothor by Leo Galambos, MFF UK Prague) mathematical formulae are stored using reverse polish notation. In turn, it employs an augmentation algorithm to the input by applying both transformation and generalization rules along with an ordering algorithm [8,18,19]. For instance, x2 will be represented in simple text consisting of three terms 𝑥, ,̂ 2; but will be stored internally as postfix notation.”
At the time of filing, it would have been obvious to a person of ordinary skill to encode mathematical expressions in reverse Polish notation (as taught by Dhar) in combination with the Zhang system because it provides the advantage of not necessitating parentheses.
Claims 14 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Jianshu Zhang.
Regarding claim 14, Jianshu Zhang discloses the further limitation which Zhang does not disclose comprising:
obtaining a plurality of continuous vectors that represent a plurality of additional mathematical expressions; and
P. 2476, first col., “As for the training loss of primary symbol node, we can get an alignment distance vector dPrimti which denotes distance between primary symbol t and input point i (xy-coordinate of handwriting traces) by using the input of computing primary attention energy …”.
determining an output set of the additional mathematical expressions by determining a level of similarity between the target continuous vector and each of the plurality of continuous vectors and selecting those mathematical expressions of the plurality of additional mathematical expressions whose continuous vectors had a level of similarity to the target continuous vector that exceeded a threshold level of similarity.
Id.
“level of similarity” :: “alignment distance vector … which denotes distance between primary symbol t and input point”
At the time of filing, it would have been obvious to a person of ordinary skill to apply the equation similarity metrics (ie equation distance metric) and retrieval techniques (as disclosed by Jianshu Zhang) to the Zhang system because this would allow a user to search for identical or equivalent equations within a corpus.
Claims 15 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang, Jianshu Zhang and Dhar.
Regarding claim 15, Zhang discloses the further limitation comprising:
obtaining a plurality of continuous vectors that represent a plurality of additional mathematical expressions; and
P. 1, fig. 1 (b) and (c) (reproduced supra). The examiner notes that each equation comprises two mathematical expressions which are equivalent but not identical.
Dhar discloses the further limitation which Zhang/Jianshu Zhang do not disclose:
determining an output set of the additional mathematical expressions by determining a level of similarity between the target continuous vector and each of the plurality of continuous vectors and selecting the top N mathematical expressions of the plurality of additional mathematical expressions with respect to the level of similarity of their continuous vectors to the target continuous vector.
P. 251, sec. 4, “Thereafter, scores of all the MEs are calculated existent in the current posting list by using Jaro-Winkler similarity algorithm recursively. The scores so generated, are sorted in a descending order. … Top k results are then shown to the user.” (Emphasis added.)
At the time of filing, it would have been obvious to a person of ordinary skill to combine the search ranking and presentation technique disclosed by Dhar with the Zhang/Jianshu Zhang system because this would provide for the intuitive presentation of the most relevant (ie most useful) search results.
Regarding claim 17, Dhar discloses the further limitation which Zhang/Jianshu Zhang do not disclose comprising:
providing an indication of a set of citations to a set of references that contain mathematical expressions of the output set.
P. 251, sec. 4, “Thereafter, scores of all the MEs are calculated existent in the current posting list by using Jaro-Winkler similarity algorithm recursively. The scores so generated, are sorted in a descending order. … Top k results are then shown to the user.”
At the time of filing, it would have been obvious to a person of ordinary skill to combine the search ranking and presentation technique disclosed by Dhar with the Zhang/Jianshu Zhang system because this would provide for the intuitive presentation of the most relevant (ie most useful) search results.
Regarding claim 18, Zhang discloses a computer-implemented method comprising:
obtaining a representation of an input mathematical expression;
P. 2, fig. 2 (reproduced below).
PNG
media_image2.png
492
526
media_image2.png
Greyscale
J Zhang discloses the following further limitation which Zhang does not disclose:
generating an initial e-graph representation of the mathematical expression;
P. 2471, fig. 1(b) (reproduced below).
PNG
media_image3.png
212
282
media_image3.png
Greyscale
applying a set of mathematical rewrite rules to the initial e-graph a plurality of times to generate a saturated e-graph representation of the mathematical expression, wherein the saturated e-graph includes a root e-class that contains at least one e-node;
P. 2473, second col., “Unlike LaTeX string decoders, SRD aims to generate a complete mathematical expression tree. As shown in the top-left box in Fig. 2, the math tree can be decomposed into a sequence of math symbols and a sequence of sub-tree structures, called label graph style [33].”
generating a mathematical grammar based on the saturated e-graph by, for each e-class of the saturated e-graph, generating a respective set of one or more replacement expressions, wherein a replacement expression of a given e-class corresponds to a respective e-node of the given e-class; and
The examiner interprets “saturated e-graph” according to its broadest reasonable interpretation in view of the applicant’s written description:
[0103] "saturated" e-graph representation by applying mathematical rewrite rules to expand the representation until the rewrite rules can no longer be applied.”
P. 2471, fig. 1(b) (reproduced supra), illustrating a graph which is ‘saturated’ according the above-noted meaning.
At the time of filing, it would have been obvious to a person of ordinary skill to apply the equation similarity metrics (ie equation distance metric) and retrieval techniques (as disclosed by Jianshu Zhang) to the Zhang system because this would allow a user to search for identical or equivalent equations within a corpus.
Dhar discloses the further limitation which Zhang/Jianshu Zhang do not disclose:
generating a plurality of different output mathematical expressions that are equivalent to the input mathematical expression by, for strings representing each of the e-nodes in the root e-class, recursively applying the replacement expressions of the mathematical grammar to replace elements of the strings.
P. 251, sec. 4, “Thereafter, scores of all the MEs are calculated existent in the current posting list by using Jaro-Winkler similarity algorithm recursively. The scores so generated, are sorted in a descending order. … Top k results are then shown to the user.” (Emphasis added.)
At the time of filing, it would have been obvious to a person of ordinary skill to combine the search ranking and presentation technique disclosed by Dhar with the Zhang/Jianshu Zhang system because this would provide for the intuitive presentation of the most relevant (ie most useful) search results.
Regarding claim 19, Dhar discloses the following further limitation wherein recursively applying the replacement expressions of the mathematical grammar to replace elements of the strings includes terminating the recursion if the number of elements in the rewritten string exceeds a specified maximum number of elements.
P. 251, sec. 4, “Thereafter, scores of all the MEs are calculated existent in the current posting list by using Jaro-Winkler similarity algorithm recursively. The scores so generated, are sorted in a descending order. … Top k results are then shown to the user.” (Emphasis added.)
Allowable Claims and Additional Relevant Prior Art
Claims 4-12 and 20 are allowable over the prior art, but are objected to as depending upon a rejected parent claims. The following references were identified by the Examiner as being relevant to the disclosed invention, but are not relied upon in any particular prior art rejection:
Zanibbi surveys the state-of-the-art in recognition and retrieval of mathematical expressions. (Zanibbi, Richard, and Dorothea Blostein. "Recognition and retrieval of mathematical expressions." International Journal on Document Analysis and Recognition (IJDAR) 15.4 (2012): 331-357.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vincent Gonzales whose telephone number is (571) 270-3837. The examiner can normally be reached on Monday-Friday 7 a.m. to 4 p.m. MT. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang, can be reached at (571) 270-7092.
Information regarding the status of an application may be obtained from the USPTO Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
/Vincent Gonzales/Primary Examiner, Art Unit 2124
Prosecution Insights