DETAILED ACTION
Response to Amendment
The amendment filed on 19 February 2026 has been entered.
Claims 1-10 are pending.
Claims 2, 8 are cancelled.
Claims 1, 9-10 are amended.
Claims 1, 3-7, 9-10 will be pending.
Applicant’s amendments to the Claims have overcome each and every rejection under 35 USC 102 previously set forth in the Non-Final Office Action mailed 24 November 2025.
Response to Arguments
Applicant’s remarks, regarding the rejections of claims under 35 USC 101, have been fully considered.
Applicant respectfully submits that the subject matter recited in Claim 1 achieves a specific technical improvement and is applied in a practical application, and satisfies the requirements of §101. Applicant respectfully submits that Claim 1 is directed to a technical improvement as described, for example, in [0045] of the Specification, which corresponds to and is realized by the features that have been incorporated into Claim 1.
Examiner respectfully disagrees. As outlined in the Non-Final Office Action mailed 24 November 2025 and below, Claim 1 and similarly Claims 9, 10, recites a judicial exception (abstract idea) under Step 2A Prong One. Further, in consideration and analysis of Claim 1 and similarly Claims 9, 10, as a whole, to determine whether the claim integrates the recited judicial exception into a practical application under Step 2A Prong Two, Examiner submits the additional limitations are directed to mere data gathering, mere instructions to implement an abstract idea on a computer, and mere instructions indicating a field of use or technological environment in which to apply the judicial exception, as outlined in the Non-Final Office Action mailed 24 November 2025 and below.
The additional limitations directed to mere data gathering, mere instructions to implement an abstract idea on a computer, and mere instructions indicating a field of use or technological environment in which to apply the judicial exception, does not provide specific steps and details for improving the learning of the learning system, but merely directs to selecting information, for collection, analysis and display, MPEP 2106.05(g)(II)(iv.), recitation of limitations that attempt to cover solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, MPEP 2106.05(f)(1), and employing generic computer functions, even when limiting the use of the idea (such as listing of plurality of layouts) to one particular environment, does not add significantly more because the token addition to the claim does not alter or affect how the learning is performed, MPEP 2106.05(h), respectively. Further, the judicial exception alone cannot provide the improvement, MPEP 2106.05(a).
Finally, the claims do not include specific details or steps of how to apply the judicial exception in order to implement and improve the learning of the learning system as presented by Applicant, MPEP 2106.04(d)(III).
Examiner submits, that additional claim elements of the claimed invention are considered insufficient to transform a judicial exception to a patentable invention. The limitations of the claimed inventions do not appear to recite steps for a specific solution to a problem in an existing technology area, where the Applicant's Specification has set forth an improvement in technology in a non-conclusory manner. Under Step2B, Claim 1 and similarly Claims 9, 10, as a whole, does not amount to significantly more than the exception itself (there is no inventive concept in the claim), see MPEP § 2106.05(B)(II).
The rejection of Claim 1 under 35 USC 101 has been maintained. Similarly, the rejections of
Claims 9, 10 under 35 USC 101 have been maintained. Rejections of Claims 3-7, which depends directly or indirectly from Claim 1 under 35 USC 101 have been maintained.
Applicant’s remarks, regarding the rejections of claims under 35 USC 103, have been fully considered.
Applicant respectfully submits that the Office failed to establish a prima facie case of obviousness at least because the Office failed to properly determine the differences between the prior art and the claims based on whether the claimed subject matter, as a whole, would have been obvious. Applicant further submits that the rejected claims are not obvious for at least the additional reasons set forth below.
Claim 1 is amended herein to incorporate the subject matter of dependent Claims 2 and 8.
Applicant respectfully submits that the cited portions of Sha merely disclose cutting out a physical design layout pattern within a field of view (FOV) into K rows and M columns, and that the height of a two-dimensional array may correspond to the height H of the FOV. Applicant respectfully submits that these disclosures fail to teach or suggest extracting a constraint that a longitudinal length of a layout is quantized in accordance with a size of a column design, and fail to disclose or suggest generating constrained training data based on such quantization, as previously recited in Claim 2 and now recited in Claim 1. Thus, Sha is directed to physical design layout patterns and is unrelated to newspaper or magazine article layouts having predetermined column sizes, the subject matter of the present application. Souche does not disclose or suggest the quantization-based constraint extraction and constrained training data generation as previously recited in Claim 2 and now recited in Claim 1.
Applicant respectfully submits that neither Sha nor Souche discloses or suggests extracting a layout constraint based on a predetermined column size specific to newspaper or magazine layouts for the purpose of generating constrained training data. Further, Tagra and Hara fail to cure the deficiencies in Sha and Souche. Applicant further submits that a person of ordinary skill in the art would not have been motivated to modify Sha in view of Souche, Tagra, or Hara to arrive at the subject matter, as a whole, that is presently recited in Claim 1.
Examiner respectfully disagrees. In response to Applicant's argument that the references fail to
show certain features of the invention, it is noted that the features upon which Applicant relies, under broadest reasonable interpretation (BRI), are given their plain meaning, unless such meaning is
inconsistent with the Specification, see MPEP § 2111.01(I).
As outlined in the Non-Final Office Action mailed 24 November 2025, pg. 9-10, and henceforth further elaborated, Sha teaches conversion of a physical design layout pattern image to a format suitable for input to a generative adversarial neural network (cf. Sha, [0010]), in a similar manner to the claimed invention. Examiner notes, under broadest reasonable interpretation (BRI), “quantized” as used in Claim 1 and similarly Claims 9, 10, of the instant application is interpreted to mean the conversion and mapping of data, further described in Sha’s post processing, which teaches converting continuous values of the synthetic 2D arrays to binary values prior to conversion to the layout format (cf. Sha, [0046]), according to K rows and M columns of physical design layout patterns 301 clipped from the physical design layout 300 (cf. Sha, [0039]-[0040]), similar to extract such a constraint that a longitudinal length of a layout is quantized in accordance with a size of a column design, and generate the constrained training data of the instant application.
The rejection of Claim 1 under 35 USC 103 has been maintained. Similarly, the rejections of
Claims 9, 10 under 35 USC 103 have been maintained. Rejections of Claims 3-7, which depends directly or indirectly from Claim 1 under 35 USC 103 have been maintained.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-7, 9-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, abstract idea, without significantly more.
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory
category. MPEP 2106.03:
According to the first part of the Alice analysis, in the instant case, the claims were determined
to be directed to one of the four statutory categories: an article of manufacture, a method/process (Claim 9), a machine/system/product (Claims 1, 3-7, 10), and a composition of matter. Based on the claims being determined to be within of the four categories (i.e., process, machine, manufacture, or composition of matter), (Step 1), it must be determined if the claims are directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea).
Step 2A Prong One: This part of the eligibility analysis evaluates whether the claim(s) recites a
judicial exception.
Regarding independent claims 1, 9, 10, the claims recite a judicial exception (i.e., an abstract idea enumerated in the 2019 PEG) without significantly more (Step-2A: Prong One). The applicant's claim limitations under broadest reasonable interpretation covers activities classified under mental processes - concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection Ill) and the 2019 PEG. As evaluated below:
Claims 1, 9, 10:
“learn a layout generator that generates a generated layout by using a random number and a layout discriminator that discriminates the generated layout and the constrained training data” (mental process of judgement)
If the identified limitation(s) falls within at least one of the groupings of abstract ideas, it is
reasonable to conclude that the claim(s) recites an abstract idea in Step 2A Prong One.
Step 2A Prong Two: This part of the eligibility analysis evaluates whether the claim(s) as a whole integrates the recited judicial exception into a practical application of the exception. As evaluated below:
“to extract a constraint of an inputted layout and generate constrained training data”
“to extract such a constraint that a longitudinal length of a layout is quantized in accordance with a size of a column design”
“generate the constrained training data”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“by using Generative Adversarial Networks”
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“wherein the layout is a layout in a newspaper or a magazine”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B: This part of the eligibility analysis evaluates whether the claim, as a whole, amounts to
significantly more than the recited exception, i.e., whether any additional element, or combination of
additional elements, adds an inventive concept to the claim. MPEP 2106.05.
First, the additional elements considered as part of the preamble and the additional elements
directed to the use of computer technology are deemed insufficient to transform the judicial exception
to a patentable invention to a patentable invention because they generally link the judicial exception to
the technology environment, see MPEP 2106.05(h).
Second, the additional elements directed to mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Third, the claims are directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception. The courts have found these types of limitations insufficient to transform the judicial exception to a patentable invention, see MPEP 2106.05(g).
Lastly, the claims directed to data gathering activity as noted above, are deemed directed to an insignificant extra-solution activity. The courts have found these types of limitations insufficient to
qualify as "significantly more", see MPEP 2106.05(g).
Furthermore, when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018). Examiner notes Berkheimer: Option 2 - A citation to one or more of the court decisions discussed in MPEP § 2106.05(d}(II} as noting the well understood, routine, conventional nature of the additional element (s) (e.g., limitations directed to mere data gathering):
The courts have recognized the following computer functions as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity, see MPEP 2106.05(d).
The additional limitations, as analyzed, failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above. Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible. Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole, claims 1, 9, 10, do not recite what the courts have identified as "significantly more".
Furthermore, regarding dependent claims 3-7, which depend from claim 1, the claims are directed to a judicial exception (i.e., an abstract idea enumerated in the 2019 PEG, a law of nature, or a natural phenomenon) without significantly more as highlighted below in the claim limitations by evaluating the claim limitations under the Step2A and 2B:
Claim 3:
Incorporates the rejection of claim 1.
“extract such a constraint that a lateral length of a layout is quantized in accordance with a width of a line”
“generate the constrained training data”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
Limitations directed to instructions for mere data gathering or data output cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 4:
Incorporates the rejection of claim 1.
“perform learning by using Conditional Generative Adversarial Networks”
The recitation is directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 5:
Incorporates the rejection of claim 4.
“wherein a condition of the Conditional Generative Adversarial Networks is at least one of a length of an article included in the layout, a priority order of the article, a number of images associated with the article, and a number of headlines associated with the article”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 6:
Incorporates the rejection of claim 1.
“generate the constrained training data by reducing a layout image indicating the layout on the basis of the constraint”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
Limitations directed to instructions for mere data gathering or data output cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 7:
Incorporates the rejection of claim 6.
“determine a reduction size of the layout image on the basis of a difference between the layout image and an image that is obtained by reducing the layout image and then enlarging it to an original size” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
The dependent claims as analyzed above, do not recite limitations that integrated the judicial exception into a practical application. In addition, the claim limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception (Step-2B). Therefore, the claims do not recite any limitations, when considered individually or as a whole, that recite what have the courts have identified as "significantly more", see MPEP 2106.05; and therefore, as a whole the claims are not patent eligible. As shown above, the dependent claims do not provide any additional elements that when considered individually or as an ordered combination, amount to significantly more than the abstract idea identified. Therefore, as a whole, the dependent claims do not recite what have the courts have identified as "significantly more" than the recited judicial exception. Therefore, claims 3-7 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception and does not recite, when claim elements are examined individually and as a whole, elements that the courts have identified as "significantly more" than the recited judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3, 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Sha et al. (U.S. Pre-Grant Publication No. 20190377849, hereinafter ‘Sha’), in view of Souche et al. (U.S. Pre-Grant Publication No. 20200242195, hereinafter 'Souche').
Regarding claim 1 and analogous claims 9, 10, Sha teaches A learning system comprising:
at least one memory that is configured to store instructions; and at least one first processor that is configured to execute the instructions ([0063] Embodiments of the present invention include a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage one memory that is configured to store instructions medium (or media) having computer readable configured to execute the instructions program instructions thereon for causing a at least one first processor processor to carry out aspects of the present invention.)
to extract a constraint of an inputted layout and generate constrained training data ([0035]
The conversion module 120 provides appropriately formatted or converted physical design layout patterns to the GAN module 122. The GAN module 122 may be generate constrained training data trained on the input data (e.g., the extract a constraint of an inputted layout physical design layout pattern arrays produced from the input 101 by the conversion module 120).); and
learn, by using Generative Adversarial Networks, a layout generator that generates a generated layout by using a random number and a layout discriminator that discriminates the generated layout and the constrained training data ([0035] The conversion module 120 provides appropriately formatted or converted physical design layout patterns to the GAN module 122. The GAN module 122 may be trained on the input data (e.g., the physical design layout pattern arrays produced from the input 101 by the conversion module 120). After training, the GAN module 122 uses the generator network to generate synthetic physical design layout patterns 103 as output. In some embodiments, the synthetic physical design layout patterns 103 are provided in a desired format as described in further detail below, such as via conversion using the conversion module 120. Thus, the output of the GAN module 122 may be provided back to the conversion module 120 for conversion (e.g., from a 2D array to a desired layout pattern format).; [0036] FIG. 2 shows an example workflow of using a learn, by using Generative Adversarial Networks GAN 200, which includes a discriminator network 202 and a generator network 204. Real layout data 201 is provided as input, which may be converted into sample data 203 prior to being provided to the discriminator network 202. The conversion of input, such as layout files or images to 2D arrays described herein, is an example of conversion from real data 201 to sample data 203. Latent sample data 205, which may take the form of randomly generated information (e.g., by using a random number a noise vector), is provided to the generator network 204.; [0037] The a layout generator that generates a generated layout generator network 204, during training, provides generated or synthetic data to the discriminator network 202. The layout discriminator that discriminates the generated layout and the constrained training data discriminator network 202 is trained to output predicted labels 207, i.e. “real” (e.g., sample data 203) and “fake” (e.g., synthetic data from the generator network 202).).
wherein the at least one first processor that is configured to execute the instructions to extract such a constraint that a longitudinal length of a layout is quantized in accordance with a size of a column design, and generate the constrained training data ([0039] FIG. 3 illustrates capture of layout patterns from a physical design layout 300. FIG. 3 shows a top-down view of a physical design layout 300, as well as a number of physical design layout patterns 301 captured therefrom. The physical design layout patterns 301 are clipped out at different locations of the physical design layout 300 in a FOV 302. As shown, there are K rows and extract such a constraint that a longitudinal length of a layout is quantized in accordance with a size of a column design M columns of physical design layout patterns 301 clipped from the physical design layout 300.; [0040] FIG. 4 shows an example generate the constrained training data conversion of a single-layer physical design layout pattern image 401 to a format suitable for input to the GAN, such as a 2D array 403. The 2D array 403 may have a size of W×H, where W represents a width of the FOV and H represents a height of the FOV.; [0046] Post-processing 611 may include polygon post-processing and DRC. The polygon post-processing may include removing or correcting non-Manhattan shapes (e.g., shapes having edges not parallel to x- and y-axes) in an output layout pattern. As mentioned above, the post-processing may further include converting continuous values of the synthetic 2D arrays to binary values prior to conversion to the layout format. DRC during post-processing 611 ensures that the synthetic layout patterns meet specified design rules for layout patterns in a particular use case that may not be captured during training.), and
Sha fails to teach wherein the layout is a layout in a newspaper or a magazine.
Souche teaches wherein the layout is a layout in a newspaper or a magazine ([0059] In the example of FIG. 2, data 201 may be received or transmitted via the interface 202. Data 201 may be passed to the segmentation unit 204 and the design unit 206 of the AI system 200. The design unit 206 may include a design generation unit 208 and a design evaluation unit 210. The design unit 206 may receive information from the segmentation unit 204 and/or priority parameters 212 in order to wherein the layout create, generate, and validate various designs or layouts. Once a design or layout is generated and validated by the design unit 206, the design selection unit 214 may review, select, and output at least one result 215 via interface 202. These may include various designs or layouts in various formats.; [0060] Data 201 may include any type of graphic element, such as an image (e.g., of a flyer, is a or a magazine magazine, etc.), a screenshot (e.g., of a website), or layout in a newspaper other similar graphic element. The data 201 may also include a uniform resource locator (URL) of a website, a list of discrete elements, or other similar input. The data 201 received at the interface 202 may then be passed to other components of the AI system 200 for processing prior to design generation or evaluation or performing analytics.).
Sha and Souche are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Sha, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Souche to Sha before the effective filing date of the claimed invention in order to provide a more robust design generation and validation approach and solution, increasing design and prototyping efficiencies, reducing cost, and generating a better overall design or layout (cf. Souche, [0030] As described herein, the visual content optimization system using artificial intelligence (AI) and machine learning techniques may provide a more robust design generation and validation approach and solution. Among other things, the visual content optimization system may increase design and prototyping efficiencies, reduce cost, and generate a better overall design or layout product for a variety of targeted consumers to enhance business. These and other advantages may be apparent using the visual content optimization system described herein.).
Regarding claim 3, Sha, as modified by Souche, teaches The learning system of claim 1.
Sha teaches wherein the at least one first processor that is configured to execute the instructions to extract such a constraint that a lateral length of a layout is quantized in accordance with a width of a line, and generate the constrained training data ([0039] FIG. 3 illustrates capture of layout patterns from a physical design layout 300. FIG. 3 shows a top-down view of a physical design layout 300, as well as a number of physical design layout patterns 301 captured therefrom. The physical design layout patterns 301 are clipped out at different locations of the physical design layout 300 in a FOV 302. As shown, there are extract such a constraint that a lateral length of a layout is quantized in accordance with a width of a line K rows and M columns of physical design layout patterns 301 clipped from the physical design layout 300.; [0040] FIG. 4 shows an example generate the constrained training data conversion of a single-layer physical design layout pattern image 401 to a format suitable for input to the GAN, such as a 2D array 403. The 2D array 403 may have a size of W×H, where W represents a width of the FOV and H represents a height of the FOV.).
Sha and Souche are combinable for the same rationale as set forth above with respect to claim 1.
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Sha, Souche, and further in view of Tagra et al. (U.S. Pre-Grant Publication No. 20200175654, hereinafter 'Tagra').
Regarding claim 4, Sha, as modified by Souche, teaches The learning system of claim 1.
Sha, as modified by Souche, fails to teach wherein the at least one first processor that is configured to execute the instructions to perform learning by using Conditional Generative Adversarial Networks.
Tagra teaches wherein the at least one first processor that is configured to execute the instructions to perform learning by using Conditional Generative Adversarial Networks ([0069] FIG. 5 depicts an example of a process for perform learning by using Conditional Generative Adversarial Networks training a conditional adversarial network to detect boundaries of objects, according to an embodiment of the present disclosure. Machine learning model 105 can be a conditional generative adversarial network that includes generative model 106 and discriminative model 107. Training a generative adversarial network involves simultaneously optimizing generative model 106 that captures a data distribution, and discriminative model 107 that estimates the probability that a sample comes from the training data rather than the generative network.).
Sha, Souche, and Tagra are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Sha and Souche, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Tagra to Sha before the effective filing date of the claimed invention in order to combine the strength of deep generative models with segmentation information, yielding more realistic predictions, especially for boundaries between different objects (cf. Tagra, [0021] Additionally, disclosed solutions provide technical improvements over solutions that predict segmentation labels (e.g., type of objects present in the image) but fail to predict a boundary of each object in the image and determine each intersection between objects and the target object. In contrast, disclosed solutions predict a segmentation label of a missing area, thereby generating information about predicted object localization and shape details of the target object. Disclosed solutions can combine a segmentation mask with the input image to make a complete prediction. By guiding a segmentation process in this manner, disclosed solutions combine the strength of deep generative models with segmentation information, yielding more realistic predictions, especially for boundaries between different objects. Additionally, compared to previous solutions which can only make a single prediction given an input image, disclosed solutions provide interactive and multi-modal predictions.).
Regarding claim 5, Sha, as modified by Souche and Tagra, teaches The learning system of claim 4.
Tagra teaches wherein a condition of the Conditional Generative Adversarial Networks is at least one of a length of an article included in the layout, a priority order of the article, a number of images associated with the article, and a number of headlines associated with the article ([0023] As disclosed herein, “object” refers to a discrete component of an image. Example objects include shapes, letters, text boxes, background objects, etc. Each object has a boundary.; [0027] Input image 110 includes a first object 112 (a square) and a second object 113 (an oval). As can be seen, object 113 partially obscures object 112 in the lower right corner of object 112. Because input image 110 can be a digitized photo or a rendered image that contains pixels, image processing application 102 determines objects 112 or 113 and their boundaries. Image processing application 102 then receives a designation of object 113 as a target object. In turn, image processing application 102 determines the boundaries of objects 112 and 113, determines an intersection between objects 112 and 113, and applies content filling to the intersection. As can be seen, output image 140 shows object 142, which corresponds to object 112.; [0033] Object detection module 250 receives input image 211, which includes object 212, object 213, and object 214. Object 213 is designated as the target object, or the object to be removed from input image 211. As can be seen, objects 213 intersects object 212 and object 214. Object boundary module 252 detects the boundaries of objects 212 and 213. Two objects are shown for example purposes. In the case that input image 211 includes more than two objects, object boundary module 252 determines a boundary for each additional object.; [0069] FIG. 5 depicts an example of a process for condition of the Conditional Generative Adversarial Networks training a conditional adversarial network to detect boundaries of objects, according to an embodiment of the present disclosure. Machine learning model 105 can be a conditional generative adversarial network that includes generative model 106 and discriminative model 107. Training a generative adversarial network involves simultaneously optimizing generative model 106 that captures a data distribution, and discriminative model 107 that estimates the probability that a sample comes from the training data rather than the generative network.; Tagra teaches conditional generative adversarial network learning, which determines objects and their boundaries. Objects is defined to include shapes, letters, text boxes, background objects. In this manner, Tagra discloses at least one of a length of an article included in the layout, a priority order of the article, a number of images associated with the article, and a number of headlines associated with the article because the boundaries of Tagra detect lengths, widths, and placement area of the objects.).
Sha, Souche, and Tagra are combinable for the same rationale as set forth above with respect to claim 4.
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Sha, Souche, and further in view of Hara et al. (U.S. Pre-Grant Publication No. 20060147084, hereinafter 'Hara').
Regarding claim 6, Sha, as modified by Souche, teaches The learning system of claim 1.
Sha, as modified by Souche, fails to teach wherein the at least one first processor that is configured to execute the instructions to generate the constrained training data by reducing a layout image indicating the layout on the basis of the constraint.
Hara teaches wherein the at least one first processor that is configured to execute the instructions to generate the constrained training data by reducing a layout image indicating the layout on the basis of the constraint ([0034] The image layout analyzer 11 analyzes the a layout image indicating the layout layout of the document image. For example, the image layout analyzer 11 may extract at least one character line as a target character line from the document image, and at least one intercharacter space from the target character line as a target intercharacter space. The information embedder 12 embeds the additional information to the target intercharacter space by changing an original length of the target intercharacter space to an optimal length. The optimal length may be generate the constrained training data by reducing determined by any one of the quantizer 13, the adjuster 14, and the determinator 15.; [0035] In one example, the quantizer 13 obtains the basis of the constraint original length for each of the intercharacter spaces extracted by the image layout analyzer 11, and converts the original length to a quantized length using the additional information. The information embedder 12 changes the length of each of the intercharacter space from the original length to the quantized length.).
Sha, Souche, and Hara are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Sha and Souche, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Hara to Sha before the effective filing date of the claimed invention in order to use extracted additional information to determine integrity, validity, or ownership of the document image (cf. Hara, [0007] Another exemplary embodiment of the present invention includes an apparatus, method, system, computer program and product, each capable of extracting additional information from a document image by: analyzing a layout of a document image, the layout comprising a character line having a plurality of intercharacter spaces, each intercharacter space having an extracted length; and extracting additional information from the extracted length of each of the plurality of intercharacter spaces. The extracted additional information may be used to determine integrity, validity, or ownership of the document image, for example.).
Regarding claim 7, Sha, as modified by Souche and Hara, teaches The learning system of claim 6.
Hara teaches wherein the at least one first processor that is configured to execute the instructions to determine a reduction size of the layout image on the basis of a difference between the layout image and an image that is obtained by reducing the layout image and then enlarging it to an original size ([0035] In one example, the quantizer 13 obtains the original length for each of the intercharacter spaces extracted by the image layout analyzer 11, and determine a reduction size of the layout image converts the original length to a quantized length using the additional information. The information embedder 12 changes the length of each of the intercharacter space from the original length to the quantized length.; [0036] The quantized length may be further adjusted by the adjuster 14, when the determinator 15 determines to adjust any one of the quantized lengths of the intercharacter spaces. For example, the determinator 15 on the basis of a difference between the layout image obtains the difference between the original length and the an image that is obtained by reducing the layout image and then enlarging it to an original size quantized length for each of the intercharacter spaces in the target character line. The determinator 15 further adds the obtained difference values into an accumulated value, and obtains its absolute value (“absolute accumulated value”). If the absolute accumulated value is equal to or less than a threshold value, the determinator 15 determines that the adjustment is not necessary. If the absolute accumulated value is greater than the threshold value, the determinator 15 determines that the adjustment is necessary.; [0037] If the determinator 15 determines that adjustment is necessary, the adjuster 14 may select at least one of the intercharacter spaces for adjustment, using any kind of selection method. The adjuster 14 adjusts the quantized length of the selected intercharacter space by a predetermined amount in order to make the absolute accumulated value to be equal to or less than the threshold value. Once the quantized length has been adjusted, the adjuster 14 may cause the information embedder 12 to change the length of the selected intercharacter space from the original length to the adjusted quantized length, or from the quantized length to the adjusted quantized length.).
Sha, Souche, and Hara are combinable for the same rationale as set forth above with respect to claim 6.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAGGIE MAIDO whose telephone number is (703) 756-1953. The examiner can normally be reached M-Th: 6am - 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MM/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights