Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Effective Filing Date
The effective filing date of April 6, 2022 is acknowledged.
Status of Claims
The present application is being examined under the claims filed on April 6, 2023.
Claim(s) 1-20 is/are rejected.
Claim(s) 1-20 is/are pending.
Prior Art References
Jlailaty, D., Grigori, D. and Belhajjame, K., 2018, May. Email business activities extraction and annotation. In International Workshop on Information Search, Integration, and Personalization (pp. 69-86). Cham: Springer International Publishing. (Hereafter, “Jlailaty”).
McClosky, D., Charniak, E. and Johnson, M., 2006, June. Effective self-training for parsing. In Proceedings of the human language technology conference of the NAACL, main conference (pp. 152-159). (Hereafter, “McClosky”).
Yao, C., Bai, X., Sang, N., Zhou, X., Zhou, S. and Cao, Z., 2016. Scene text detection via holistic, multi-channel prediction. arXiv preprint arXiv:1606.09002. (Hereafter, “Yao”).
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C. and Torr, P.H., 2015. Conditional random fields as recurrent neural networks. In Proceedings of the IEEE international conference on computer vision (pp. 1529-1537). (Hereafter, “Zheng”).
US 5812853 A - Method And Apparatus For Parsing Source Code Using Prefix Analysis (Hereafter, “Carroll”).
Claim Rejections - 35 U.S.C. § 112(b)
The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim(s) 1-20 is/are rejected as being indefinite under 35 U.S.C. § 112(b).
Independent claims 1, 11, 20 recite “identifying, by the one or more processors, metadata in a metadata file associated with the electronic communication documents to annotate the identified unstructured text; based upon the annotations, re-training, by the one or more processors, the parser; and”. There is insufficient antecedent basis for “the annotations”. The claim merely recites “identifying metadata […] to annotate” but does not recite the generation or acquisition of annotations. It remains unclear what “the annotations” refers to.
All remaining dependent claims are rejected by virtue of dependency.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. This judicial exception is not integrated into a practical application as outlined in the 2-step analyses for each claim that follows.
Combined Step 1 (Statutory Category)
Claims 1-10 are directed to a process.
Claims 11-20 are directed to machines.
In reference to independent claims 1, 11, 20, representative claim 1 recites:
Step 2A Prong 1 (Recited Judicial Exception)
“applying, by the one or more processors, a parser to the electronic communication documents included in the batch of electronic communication documents to identify unstructured text indicating one or more entities;” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“identifying, by the one or more processors, metadata in a metadata file associated with the electronic communication documents to annotate the identified unstructured text;” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“and applying, by the one or more processors, the re-trained parser to annotate additional electronic communication documents included in the corpus of documents.” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 (Integration into a Practical Application) & Step 2B (Significantly More or Amounting to an Inventive Concept)
“obtaining, by the one or more processors, a batch of electronic communication documents from a corpus of documents;” which amounts to insignificant extra-solution activity per MPEP2106.05(g). This is well-understood, routine, conventional computer functionality as recognized by MPEP2106.05(d)(II) i. Receiving or transmitting data over a network.
“based upon the annotations, re-training, by the one or more processors, the parser;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
In reference to claim 2:
Step 2A Prong 1 (Recited Judicial Exception)
“applying, by the one or more processors, a partially-trained email parser that was trained based on electronic communication documents not included in the corpus of documents.” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
In reference to claims 3, 12, representative claim 3 recites:
Step 2A Prong 1 (Recited Judicial Exception)
“a segmenter configured to segment portions of an electronic communication document that indicates document metadata from portions of the electronic communication document associated with document content;” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“a tagger configured to predict boundaries between fields indicated by the document metadata for the electronic communication document; and” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“an extractor configured to identify entities indicated by particular fields identified by the tagger.” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
In reference to claims 4, 13, representative claim 4 recites:
Step 2A Prong 1 (Recited Judicial Exception)
“executing, by the one or more processors, the segmenter to segment the electronic communication document into component communication segments and to identify the portions of the communication segments that indicate the document metadata;” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“identifying, by the one or more processors, an entry in the metadata file corresponding to a top-level segment of the electronic communication document;” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“annotating, by the one or more processors, the unstructured text of the electronic communication document based upon metadata included in the entry in the metadata file; and” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 (Integration into a Practical Application) & Step 2B (Significantly More or Amounting to an Inventive Concept)
“training, by the one or more processors, the tagger and the extractor based upon the annotated metadata.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
In reference to claims 5, 14, representative claim 5 recites:
Step 2A Prong 1 (Recited Judicial Exception)
“identifying, by the one or more processors, a plurality of entries in the metadata file respectively corresponding to electronic communication documents in which the communication segment is a top-level segment; and” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“annotating, by the one or more processors, the unstructured text of the communication segments using the respective entry in the metadata file.” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
In reference to claims 6, 15, representative claim 6 recites:
Step 2A Prong 1 (Recited Judicial Exception)
“comparing, by the one or more processors, the metadata of the communication segments to the metadata file to identify that a communication segment does not correspond to an entry in the metadata file; and” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
“excluding, by the one or more processors, the electronic communication document from a training set used to train the tagger and the extractor.” which, but for the inclusion of generic computing equipment (i.e., “processors”), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
In reference to claims 7, 16, representative claim 7 recites:
Step 2A Prong 1 (Recited Judicial Exception)
The claim inherits the judicial exception recited in the parent claim.
Step 2A Prong 2 (Integration into a Practical Application) & Step 2B (Significantly More or Amounting to an Inventive Concept)
“re-training, by the one or more processors, at least one of the segmenter, the tagger, or the extractor based upon human-applied annotations.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
In reference to claims 8, 17, representative claim 8 recites:
Step 2A Prong 1 (Recited Judicial Exception)
The claim inherits the judicial exception recited in the parent claim.
Step 2A Prong 2 (Integration into a Practical Application) & Step 2B (Significantly More or Amounting to an Inventive Concept)
“the segmenter includes a recurrent neural network (RNN) and conditional random fields (CRF) model; and re-training the segmenter comprises re-training, by the one or more processors, at least one of the RNN or the CRF model.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
In reference to claims 9, 18, representative claim 9 recites:
Step 2A Prong 1 (Recited Judicial Exception)
The claim inherits the judicial exception recited in the parent claim.
Step 2A Prong 2 (Integration into a Practical Application) & Step 2B (Significantly More or Amounting to an Inventive Concept)
“the tagger includes a fully convolutional network (FCN) and a prefix dictionary; and re-training the tagger comprises at least one of re-training, by the one or more processors, the FCN or updating, by the one or more processors, the prefix dictionary.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
In reference to claims 10, 19, representative claim 10 recites:
Step 2A Prong 1 (Recited Judicial Exception)
The claim inherits the judicial exception recited in the parent claim.
Step 2A Prong 2 (Integration into a Practical Application) & Step 2B (Significantly More or Amounting to an Inventive Concept)
“the extractor includes a fully convolutional network (FCN) and a recurrent neural network (RNN); and re-training the extractor comprises at least one of re-training, by the one or more processors, the FCN or the RNN.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 11, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable
over Jlailaty
in view of McClosky.
Claim(s) 3, 4, 5, 6, 7, 10, 12, 13, 14, 15, 16, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable
over Jlailaty
in view of McClosky
in further view of Yao.
Claim(s) 8, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable
over Jlailaty
in view of McClosky
in further view of Yao
in further view of Zheng.
Claim(s) 9, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable
over Jlailaty
in view of McClosky
in further view of Yao
in further view of Carroll.
In reference to claim 1.
“1. A computer-implemented method for self-training an electronic communication document parser, the method comprising:”
Jlailaty teaches:
“obtaining, by the one or more processors, a batch of electronic communication documents from a corpus of documents;”
(Jlailaty 73, “The input data of our approach is an email log. An email log is a set of emails exchanged between different entities (people, companies, etc.) for a specific purpose such as scheduling a meeting, organizing a conference, or purchasing an item etc. Each email is characterized by multiple attributes: subject, sender, receiver, body and timestamp. The most important information in an email is found in its subject and body texts, which are unstructured data-types. These texts should be translated into a format that is expected by the analysis tools.”)
“applying, by the one or more processors, a parser to the electronic communication documents included in the batch of electronic communication documents to identify unstructured text indicating one or more entities; identifying, by the one or more processors, metadata in a metadata file associated with the electronic communication documents to annotate the identified unstructured text;”
(Jlailaty 73, “Phase 4: Extracting Activity Metadata: In this phase, each activity type (represented by a cluster) will be associated with some metadata deduced from the set of information associated with the activity instances contained in the cluster. The metadata includes information like the role of the sender(s)/receiver(s) of the exchanged email, […] The input data of our approach is an email log. An email log is a set of emails exchanged between different entities (people, companies, etc.) for a specific purpose such as scheduling a meeting, organizing a conference, or purchasing an item etc. Each email is characterized by multiple attributes: subject, sender, receiver, body and timestamp. The most important information in an email is found in its subject and body texts, which are unstructured data-types. These texts should be translated into a format that is expected by the analysis tools.”)
The identified “unstructured text indicating one or more entities” is taught by the “activity type[s]” being associated with the “sender(s)/receiver(s)” where the senders and receivers teach the entities.
“metadata in a metadata file associated with the electronic communication documents” is taught by “Each email is characterized by multiple attributes: subject, sender, receiver, body and timestamp.” I.e., the “email” teaches the “metadata file”.
“Annotat[ing] the identified unstructured text” is taught by “each activity type will be associated with some metadata”
McClosky teaches:
“based upon the annotations, re-training, by the one or more processors, the parser; and”
(McClosky 154, “Finally, we mix a portion of parser-best or reranker best lists with the standard Wall Street Journal training data (sections 2-21) to retrain a new parser (but not reranker) model.”)
“applying, by the one or more processors, the re-trained parser to annotate additional electronic communication documents included in the corpus of documents.”
(McClosky 155, “Finally, we evaluate our new model on the test section of Wall Street Journal.”)
Motivation to combine Jlailaty, McClosky.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Jlailaty, McClosky.
Jlailaty discloses the extraction of business processes and entities from email logs.
McClosky discloses a self-training paradigm for training parsers.
One would be motivated to combine these references because Jlailaty operates on a corpus of text with a limited number of training samples, wherein it is costly to label new samples. Thus, the self-training paradigm of McClosky provides an apt methodology of expanding the training data set with additional unlabeled samples.
Further, MPEP § 2143(I) EXAMPLES OF RATIONALES sets forth the Supreme Court rationales for obviousness, including:
(C) Use of known technique to improve similar devices (methods, or products) in the same way;
(D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results;
In reference to claim 2.
“2. The computer-implemented method of claim 1, wherein applying the parser comprises:”
McClosky teaches:
“applying, by the one or more processors, a partially-trained email parser that was trained based on electronic communication documents not included in the corpus of documents.”
(McClosky 155, “Finally, we evaluate our new model on the test section of Wall Street Journal.”)
In reference to claim 3.
“3. The computer-implemented method of claim 1, wherein the parser comprises:”
Jlailaty teaches:
“a segmenter configured to segment portions of an electronic communication document that indicates document metadata from portions of the electronic communication document associated with document content;”
(Jlailaty 72, “Phase 1: Data Preprocessing: Since the important information of an email is contained in its body and subject unstructured texts, some preprocessing should be applied before any analysis. Texts are cleansed, segmented into separate sentences, and finally, verb-noun (verb-object) pairs are extracted for each sentence.”)
“and an extractor configured to identify entities indicated by particular fields identified by the tagger.”
(Jlailaty 74-75, “We define the following features that characterize the sentences as relevant and business oriented. […] Number of named entities: The existence of named entities such as names of people (actors), locations, names of conferences, etc. gives an indication about the possibility that this sentence contains an activity (which is performed by the mentioned actors or in the specified location).”)
Yao teaches:
“a tagger configured to predict boundaries between fields indicated by the document metadata for the electronic communication document;”
(Yao 3, “The original image is fed into the trained model and three prediction maps, corresponding to text regions, characters and linking orientations of adjacent characters, are produced. Detections are formed by performing segmentation, aggregation and partition on the three maps.”)
Motivation to combine Jlailaty, McClosky, Yao.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Jlailaty, McClosky, Yao.
Jlailaty, McClosky discloses the extraction of business processes and entities from email logs and a self-training paradigm for doing so.
Yao discloses scene text detection in images.
One would be motivated to combine these references because, a broadest reasonable interpretation of applicant’s use of “electronic communication documents” may include scans or images of physical documents that would require the types of optical text extraction delineated in Yao.
Further, MPEP § 2143(I) EXAMPLES OF RATIONALES sets forth the Supreme Court rationales for obviousness, including:
(A) Combining prior art elements according to known methods to yield predictable results;
(C) Use of known technique to improve similar devices (methods, or products) in the same way;
(D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results;
In reference to claim 4.
“4. The computer-implemented method of claim 3, wherein re-training the parser comprises:”
Jlailaty teaches:
“executing, by the one or more processors, the segmenter to segment the electronic communication document into component communication segments and to identify the portions of the communication segments that indicate the document metadata;”
(Jlailaty 72, “Phase 1: Data Preprocessing: Since the important information of an email is contained in its body and subject unstructured texts, some preprocessing should be applied before any analysis. Texts are cleansed, segmented into separate sentences, and finally, verb-noun (verb-object) pairs are extracted for each sentence.”)
“identifying, by the one or more processors, an entry in the metadata file corresponding to a top-level segment of the electronic communication document; annotating, by the one or more processors, the unstructured text of the electronic communication document based upon metadata included in the entry in the metadata file; and”
(Jlailaty 73, “Phase 4: Extracting Activity Metadata: In this phase, each activity type (represented by a cluster) will be associated with some metadata deduced from the set of information associated with the activity instances contained in the cluster. The metadata includes information like the role of the sender(s)/receiver(s) of the exchanged email, information about the resources that it uses (sent documents, information about the domain of web pages included in its description) and the actors performing the activity.”)
McClosky teaches:
“training, by the one or more processors, the tagger and the extractor based upon the annotated metadata.”
(McClosky 154, “Finally, we mix a portion of parser-best or reranker best lists with the standard Wall Street Journal training data (sections 2-21) to retrain a new parser (but not reranker) model.”)
In reference to claim 5.
“5. The computer-implemented method of claim 4, wherein annotating the metadata of the electronic communication document comprises:”
Jlailaty teaches:
“identifying, by the one or more processors, a plurality of entries in the metadata file respectively corresponding to electronic communication documents in which the communication segment is a top-level segment; and annotating, by the one or more processors, the unstructured text of the communication segments using the respective entry in the metadata file.”
(Jlailaty 73, “Phase 4: Extracting Activity Metadata: In this phase, each activity type (represented by a cluster) will be associated with some metadata deduced from the set of information associated with the activity instances contained in the cluster. The metadata includes information like the role of the sender(s)/receiver(s) of the exchanged email, information about the resources that it uses (sent documents, information about the domain of web pages included in its description) and the actors performing the activity.”)
“Identifying” and subsequent “annotating” is taught by “In this phase, each activity type (represented by a cluster) will be associated with some metadata deduced from the set of information associated with the activity instances contained in the cluster.”
In reference to claim 6.
“6. The computer-implemented method of claim 4, wherein training the tagger and the extractor comprises:”
Jlailaty teaches:
“comparing, by the one or more processors, the metadata of the communication segments to the metadata file to identify that a communication segment does not correspond to an entry in the metadata file; and excluding, by the one or more processors, the electronic communication document from a training set used to train the tagger and the extractor.”
(Jlailaty 74, “Not all the sentences that compose an email are about business activities. Some sentences may be talking about personal issues or greetings. We, therefore, need a means for identifying the sentences in the email that provide information about business activities such as the verb-noun pair representing the business activity or some other information characterizing the activity such as names of actors, locations, documents, links etc. We refer to such sentences by relevant sentences. To identify relevant sentences in an email, we use a classification technique that associates each email sentence with one of the following labels Relevant or Non-Relevant. This problem of sentence classification can be related to the problem known in the literature as extractive summarization [11], given that it reduces an input text by discarding non-informative sentences.”)
In reference to claim 7.
“7. The computer-implemented method of claim 3, further comprising:”
Jlailaty teaches:
“human-applied annotations”
(Jlailaty 79, “In order to build the training data set, we manually assign labels for each sentence. The labels refer to whether the sentence should be included or excluded from the analysis. The expert chooses 1 as a label if the sentence contains business process oriented activities information, 0 otherwise.”)
McClosky teaches:
“re-training, by the one or more processors, at least one of the segmenter, the tagger, or the extractor based upon [human-applied annotations].”
(McClosky 154, “Finally, we mix a portion of parser-best or reranker best lists with the standard Wall Street Journal training data (sections 2-21) to retrain a new parser (but not reranker) model.”)
In reference to claim 8.
“8. The computer-implemented process of claim 7, wherein:”
Zheng teaches:
“the segmenter includes a recurrent neural network (RNN) and conditional random fields (CRF) model;”
(Zheng Abstract, “Pixel-level labelling tasks, such as semantic segmentation, play a central role in image understanding. […] To solve this problem, we introduce a new form of convolutional neural network that combines the strengths of Convolutional Neural Networks (CNNs) and Conditional Random Fields (CRFs)-based probabilistic graphical modelling. To this end, we formulate Conditional Random Fields with Gaussian pairwise potentials and mean-field approximate inference as Recurrent Neural Networks. This network, called CRF-RNN”)
“and re-training the segmenter comprises re-training, by the one or more processors, at least one of the RNN or the CRF model.”
(Zheng 1534, “In the forward pass through the network, once the computation enters the CRF-RNN after passing through the CNN stage, it takes T iterations for the data to leave the loop created by the RNN. Neither the CNN that provides unary values nor the layers after the CRF-RNN (i.e., the loss layers) need to perform any computations during this time since the refinement happens only inside the RNN’s loop. Once the output Y leaves the loop, next stages of the deep network after the CRF-RNN can continue the forward pass. In our setup, a softmax loss layer directly follows the CRF-RNN and terminates the network. During the backward pass, once the error differentials reach the CRF-RNN’s output Y, they similarly spend T iterations within the loop before reaching the RNN input U in order to propagate to the CNN which provides the unary input.”)
Motivation to combine Jlailaty, McClosky, Yao, Zheng.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Jlailaty, McClosky, Yao, Zheng.
Jlailaty, McClosky, Yao discloses the extraction of business processes and entities from email logs, a self-training paradigm for doing so, and a system for extracting text from arbitrary scenes.
Zheng discloses pixel-level segmentation using conditional random fields and recurrent neural networks.
One would be motivated to combine these references because semantic segmentation is a key step disclosed specifically in Yao, and the methodology delineated in Zheng could serve as a substitute.
Further, MPEP § 2143(I) EXAMPLES OF RATIONALES sets forth the Supreme Court rationales for obviousness, including:
(A) Combining prior art elements according to known methods to yield predictable results;
(B) Simple substitution of one known element for another to obtain predictable results;
In reference to claim 9.
“9. The computer-implemented method of claim 7, wherein:”
Yao teaches:
“the tagger includes a fully convolutional network (FCN)”
(Yao 3, “Generally, the proposed algorithm follows the paradigm of the FCN [28] and HED framework [52], i.e., it infers properties of scene text in a holistic fashion by producing image-level, pixel-wise prediction maps. In this paper, we consider text regions (words or text lines), individual characters and their relationships (linking orientation between characters) as the three properties to be estimated at runtime, since these properties are effective for scene text detection […]”)
“re-training the tagger comprises at least one of re-training, by the one or more processors, the FCN or updating, by the one or more processors, the prefix dictionary.”
(Yao 2-3, “The proposed strategy is realized using the FCN framework, which was originally designed for semantic segmentation. […] “The prediction model is trained by feeding the training images and the corresponding ground truth maps into the network”)
Carroll teaches:
“and a prefix dictionary; and”
(Carroll (48), “The operation of the above exemplary implementation is as follows. First a given translation unit is macro-expanded, or preprocessed, and then a parser in the initial state is created. The function parse(t, T) then begins at the root of the prefix tree T and proceeds down the tree. At each node in the prefix tree, the function looks for a child whose text matches a prefix of the text e remaining in the translation unit. If there is such a child, the function moves to it, strips that prefix from the remaining text, and applies the parser delta to the parser. If there is no such child, the function parse(t, T) calls the function parseprefix(P, e) to parse a prefix of the remaining test. The function parse(t, T) then creates a prefix tree node with the information returned by the function parseprefix(P, e), makes it a child of the current node, and moves to that child. The function parse(t, T) is completed when there is no more text remaining in the translation unit.”)
The “prefix dictionary” is taught by the “prefix tree”.
Motivation to combine Jlailaty, McClosky, Yao, Carroll.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Jlailaty, McClosky, Yao, Carroll.
Jlailaty, McClosky, Yao discloses the extraction of business processes and entities from email logs, a self-training paradigm for doing so, and a system for extracting text from arbitrary scenes.
Carroll discloses a parsing procedure specifically directed to parsing source code.
One would be motivated to combine these references because The text extracted from the system that combines Jlailaty, McClosky, Yao could further be parsed for specific details using the rudimentary prefix parsing scheme delineated in Carroll. Further, the source code parsing of Carroll could be used to parse plaintext as in Jlailaty, McClosky, Carrol.
Further, MPEP § 2143(I) EXAMPLES OF RATIONALES sets forth the Supreme Court rationales for obviousness, including:
(A) Combining prior art elements according to known methods to yield predictable results;
(B) Simple substitution of one known element for another to obtain predictable results;
(E) "Obvious to try" – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success;
(F) Known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art;
In reference to claim 10.
“10. The computer-implemented method of claim 7, wherein:”
Yao teaches:
“the extractor includes a fully convolutional network (FCN)”
(Yao 3, “Generally, the proposed algorithm follows the paradigm of the FCN [28] and HED framework [52], i.e., it infers properties of scene text in a holistic fashion by producing image-level, pixel-wise prediction maps. In this paper, we consider text regions (words or text lines), individual characters and their relationships (linking orientation between characters) as the three properties to be estimated at runtime, since these properties are effective for scene text detection […]”)
Examiner is interpreting the “extractor” to include the “tagger” and “segmenter” as possible subcomponents (e.g., in order to extract, the source text must first be segmented and tagged). Thus, by the teachings of Yao and Zheng, the extractor would include an “FCN” and an “RNN”.
“and re-training the extractor comprises at least one of re-training, by the one or more processors, the FCN or the RNN.”
(Yao 2-3, “The proposed strategy is realized using the FCN framework, which was originally designed for semantic segmentation. […] “The prediction model is trained by feeding the training images and the corresponding ground truth maps into the network”)
Zheng teaches:
“and a recurrent neural network (RNN);”
(Zheng Abstract, “Pixel-level labelling tasks, such as semantic segmentation, play a central role in image understanding. […] To solve this problem, we introduce a new form of convolutional neural network that combines the strengths of Convolutional Neural Networks (CNNs) and Conditional Random Fields (CRFs)-based probabilistic graphical modelling. To this end, we formulate Conditional Random Fields with Gaussian pairwise potentials and mean-field approximate inference as Recurrent Neural Networks. This network, called CRF-RNN”)
In reference to claims 11-20.
Claims 11-20 are substantially similar to claims 1, 3-10 and are thus rejected using the same art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CODY RYAN GILLESPIE whose telephone number is (571)272-1331. The examiner can normally be reached M-F, 8 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker A Lamardo can be reached on 5172705871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CODY RYAN GILLESPIE/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147